Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🔧 readme & makefile #156

Merged
merged 1 commit into from
Oct 17, 2022
Merged

🔧 readme & makefile #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 14 additions & 33 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,52 +1,33 @@
process: build
process: migrate
@node -r dotenv/config lib/processor.js

build:
@npm run build

serve:
@npx squid-graphql-server
build-processor-image:
@docker build . --target processor -t squid-processor

reset:
@npx sqd db drop
@npx sqd db create
@npx sqd db:migrate
build-query-node-image:
@docker build . --target query-node -t query-node

migrate:
@npx sqd db:migrate
build-images: build-processor-image build-query-node-image

serve:
@npx squid-graphql-server --subscriptions

migration:
migrate:
@npx squid-typeorm-migration apply


build:
@npm run build


codegen:
@npx sqd codegen

@npx squid-typeorm-codegen

typegen: kusamaVersions.json
typegen:
@npx squid-substrate-typegen typegen.json


kusamaVersions.json:
@make explore


explore:
@npx squid-substrate-metadata-explorer \
--chain wss://kusama-rpc.polkadot.io \
--archive https://kusama.indexer.gc.subsquid.io/v4/graphql \
--out kusamaVersions.json


up:
@docker-compose up -d


down:
@docker-compose down


.PHONY: process serve start codegen migration migrate up down reset
.PHONY: build serve process migrate codegen typegen up down
111 changes: 50 additions & 61 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,54 +2,42 @@

![](https://media.giphy.com/media/chOyZePGEHDoTSY2CA/giphy.gif)

This is a sample project generated by `hydra-cli scaffold`. Experiment by modifying `schema.graphql` and the mappings in the `mappings` folder.

## Fast Forward

```
just up
just codegen
just build
just reset
just explore
```

Start processor
```
just process
```

Run GraphQL queries
```
just serve
```

## Prerequisites

* Node v14x
* Docker
* node 16.x
* docker

## Quickly running the sample

## Bootstrap
Example commands below use [make(1)](https://www.gnu.org/software/make/).
Please, have a look at commands in [Makefile](Makefile) if your platform doesn't support it.
On Windows we recommend to use [WSL](https://docs.microsoft.com/en-us/windows/wsl/).

```bash
# The dependencies setup relies on de-duplication, use `ci` to get everything right
# 1. Install dependencies
npm ci

# Start a postgres instance
docker-compose up db # add optional -d flag to detach from terminal
# 2. Compile typescript files
make build

# 3. Start target Postgres database and detach
make up

# Apply migrations related to the processor's state keeping tables
npm run processor:migrate
# 4. Apply database migrations from db/migrations
make migrate

# Apply the project's migrations
npm run db:migrate
# 5. Start the processor
make process

# Now you can start processing chain data
npm run processor:start
# 6. The command above will block the terminal
# being busy with fetching the chain data,
# transforming and storing it in the target database.
#
# To start the graphql server open the separate terminal
# and run
make serve

# The above command will block
# Open a separate terminal and launch the graphql server to query the processed data
npm run query-node:start
# 7. Now you can see the resuls by visiting the localhost:4350/graphql
```

## Project structure
Expand All @@ -66,40 +54,41 @@ Hydra tools expect a certain directory layout:
If you do not plan to extend GraphQl server you can delete `server-extension` module and then remove
`type-graphql` and `class-validator` dependencies.

## Development flow
## Dev flow

If you modified `schema.graphql`:
### 1. Define database schema

```bash
# Run codegen to re-generate model/server files
npm run codegen
Start development by defining the schema of the target database via `schema.graphql`.
Schema definition consists of regular graphql type declarations annotated with custom directives.
Full description of `schema.graphql` dialect is available [here](https://docs.subsquid.io/docs/develop-a-squid/define-a-squid-schema).

# Analyze database state and create a new migration to match generated models
npm run db:create-migration # add -n "myName" to skip the migration name prompt
### 2. Generate TypeORM classes

# Apply the migrations
npm run db:migrate
```
Mapping developers use [TypeORM](https://typeorm.io) entities
to interact with the target database during data processing. All necessary entity classes are
generated by the squid framework from `schema.graphql`. This is done by running `npx squid-typeorm-codegen`
command.

You might want update the `Initial` migration instead of creating a new one (e.g. during the development phase when the production database is not yet set up). In that case it convenient to reset the database schema and start afresh:
### 3. Generate database migration

```bash
rm db/migrations/LastUnappliedMigration.ts
npm run db:reset
npm run db:create-migration
npm run db:migrate
```

To generate new type definitions for chain events and extrinsics:
All database changes are applied through migration files located at `db/migrations`.
`squid-typeorm-migration(1)` tool provides several commands to drive the process.
It is all [TypeORM](https://typeorm.io/#/migrations) under the hood.

```bash
# Review typegen section of manifest.yml (https://docs.subsquid.io/hydra-typegen)
# Connect to database, analyze its state and generate migration to match the target schema.
# The target schema is derived from entity classes generated earlier.
# Don't forget to compile your entity classes beforehand!
npx squid-typeorm-migration generate

# Create template file for custom database changes
npx squid-typeorm-migration create

# Delete old definitions
rm -rf src/types
# Apply database migrations from `db/migrations`
npx squid-typeorm-migration apply

# Run typegen tool
npm run typegen
# Revert the last performed migration
npx squid-typeorm-migration revert
```

## Setting up the project for resolvers
Expand Down