Hasura metadata export CLI command doesn't dump native_queries/logical_models to yaml in some cases

[NasaGeek]

Version Information

Server Version: v2.46.0 (docker image tag)
CLI Version (for CLI related issue): 2.48.1

Environment

Running OSS with Docker Compose locally

What is the current behaviour?

Running hasura metadata export does not dump all information to yaml file structure. Specifically, native_queries and logical_models are missing.

What is the expected behaviour?

All metadata should be dumped when running hasura metadata export.

How to reproduce the issue?

Environment configuration:

docker-compose.yml

services:
  postgres:
    image: postgres:17
    restart: always
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: postgrespassword
  graphql-engine:
    image: hasura/graphql-engine:v2.46.0
    init: true
    ports:
      - "0.0.0.0:8080:8080"
    restart: always
    volumes:
      - ./metadata:/hasura-metadata
    environment:
      HASURA_GRAPHQL_METADATA_DATABASE_URL: postgres://postgres:postgrespassword@postgres:5432/postgres
      PG_DATABASE_URL: postgresql://postgres:postgrespassword@postgres:5432/postgres
      HASURA_GRAPHQL_ENABLE_CONSOLE: "true"
      HASURA_GRAPHQL_DEV_MODE: "true"
      HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log
      HASURA_GRAPHQL_DISABLE_NATIVE_QUERY_VALIDATION: "true"
volumes:
  db_data:

metadata.json

{
  "resource_version": 32,
  "metadata": {
    "version": 3,
    "sources": [
      {
        "name": "metadata",
        "kind": "postgres",
        "tables": [],
        "native_queries": [
          {
            "arguments": {},
            "code": "select 1",
            "returns": "model",
            "root_field_name": "query"
          }
        ],
        "logical_models": [
          {
            "fields": [],
            "name": "model"
          }
        ],
        "configuration": {
          "connection_info": {
            "database_url": {
              "from_env": "PG_DATABASE_URL"
            },
            "isolation_level": "read-committed",
            "use_prepared_statements": false
          }
        }
      }
    ]
  }
}

1. Run docker compose up -d with the provided compose file
2. Open web console and import the provided metadata.json file
3. You should see one Native Query and one Logical Model in the metadata now
4. Run hasura metadata export -o yaml, you should get the following output (native_queries/logical_models present as expected)

version: 3
sources:
- name: metadata
  kind: postgres
  tables: []
  native_queries:
  - arguments: {}
    code: select 1
    returns: model
    root_field_name: query
  logical_models:
  - fields: []
    name: model
  configuration:
    connection_info:
      database_url:
        from_env: PG_DATABASE_URL
      isolation_level: read-committed
      use_prepared_statements: false

5. Run hasura metadata export
6. Search through the files that have now appeared at ./metadata. No native_queries or logical_models can be found.

Screenshots or Screencast

Please provide any traces or logs that could help here.

Any possible solutions/workarounds you're aware of?

Export the metadata as json or as a single yaml file rather than the full directory structure.

Keywords

metadata export

[robertjdominguez]

@NasaGeek — is it possible you're not using the CLI's hasura console command?

If you just open the console running at 8080 from your compose file, metadata changes won't be tracked.

I just want to confirm exactly what you're doing at this step:

Open web console and import the provided metadata.json file

[NasaGeek]

I'm intentionally not using hasura console, as I'm hoping not to have 2 different ways to run the console. In the repro instructions, I'm accessing the console by accessing <host>:8080.

I suppose running hasura console is another viable workaround here, though it seems to have its own quirks. It will not automatically dump native_queries/logical_models to the metadata folder. I must run hasura metadata export for them to be dumped into databases.yaml. Performing any other metadata-modifying operation (like adding a table, for instance) will actually cause the native_queries/logical_models sections to be removed from databases.yaml again.

Overall, my impression of hasura console was that it's the easiest way to get automatic metadata dumping, but the hasura metadata export command still seemed like it should work in other instances (and it does, mostly?).

[robertjdominguez]

When you open the console at <host>:8080, it’s the version bundled in the GraphQL Engine image — it doesn't have any awareness of the CLI project directory or the metadata files on disk. So while it lets you modify metadata in the running instance, those changes only exist in the server’s internal state. Running hasura metadata export afterward from the CLI will pull down that state — but only if you're connected to the same instance and project.

If you’re not using the CLI’s hasura console, you’re missing that local project context, which is why those objects aren’t showing up in the exported YAML structure.

Long story short: the safest way to round-trip this is:

1. Use hasura console from the CLI (so it’s tied to your metadata project).
2. After making changes, run hasura metadata export to write everything back to disk.

[NasaGeek]

Ah, your description of the project context and config.yaml file got me poking around a bit more in our project, and I found this interesting setup where we have a config.yaml sitting in our repo root

version: 3
metadata_directory: metadata
actions:
  kind: synchronous

which points to a directory named metadata, but then I was also running graphql-engine in Docker with that same metadata directory mounted in as /hasura-metadata (this is for running migrations at start; we use the special cli-migrations-v3 tagged image, but I left that out of my example in the interest of minimization).

Having 2 projects share the same metadata directory seems to be the source of my problems, as if I change my config.yaml to instead point to e.g. a directory named md and then run hasura metadata export, everything appears in md as expected, including native_queries and logical_models.

So then I think the solution to my immediate problem would be to mount the metadata directory into the Docker container as read-only, so graphql-engine doesn't try to overwrite what's ending up in there when I run hasura metadata export. Testing this approach out seems to work nicely.

You're probably shaking your head, wondering why I won't just use hasura console, the blessed approach. Call it masochism, I guess.

[NasaGeek]

I'm going to go ahead and close this, as I've obtained the desired behavior (and the title was inaccurate anyway, the fields were being dumped).