Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v23.1.x] Schema Registry: GET /schemas/ids/<id> should return references #11240

Merged
merged 6 commits into from
Jun 7, 2023
Merged

[v23.1.x] Schema Registry: GET /schemas/ids/<id> should return references #11240

merged 6 commits into from
Jun 7, 2023

Commits on Jun 6, 2023

  1. schema_registry/types: Refector: Shuffle some types around 

    This will make the next refactor slightly easier.

Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit f917875)
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    a23485f View commit details
    Browse the repository at this point in the history

  2. schema_registry/test: Refactor: Shuffle some definitions around 

    This will make the next refactor slightly easier to comprehend.

Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit 4972bcc)
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    d0dc8af View commit details
    Browse the repository at this point in the history

  3. schema_registry/test: Add more tests for references 

    Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit 5d5072b)
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    469a51e View commit details
    Browse the repository at this point in the history

  4. schema_registry: Refactor: Move references to schema 

    This should be a pure refactoring.

The store has two mappings:
1) `schema_map`: Schema ID to Schema (sharded across cores)
2) `subject_map`: Subject-Versions to Schema ID

Each of these mappings are then split across shards, coordinated
by sharded_store.

Prior to this change, references were attached to the Subject-Versions.

This is an incorrect modelling; Schema should own the references
to other schema. This commit does that refactoring by moving references
from `typed_schema` to `typed_schema_definition`.

There are a couple of things that are not straightforward:
1) `ValidSchema` (`protobuf_schema_definition`, `avro_schema_definition`)
   are the respective library instantiations of the representation,
   and are unable to reproduce their references to form a
   `canonical_schema_definition`, so references are stored inside the
   wrappers.
2) `referenced_by` and `is_reference` now require two phases to
   obtain the references, due to the sharded and diconnected nature of
   the two mappings (schema, and subject-versions).
   a) Obtain a list of schema ids that reference the subject-version
   b) Filter that list by subject-versions that are not soft-deleted

Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit 2fc2dfe)

Conflicts:
compatibility_protobuf.h due to 5a76733
store.h due to a376f21
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    ec04223 View commit details
    Browse the repository at this point in the history

  5. schema_registry/store: Soft-deleted sub-ver are not references 

    Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit c13d874)
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    4ddd787 View commit details
    Browse the repository at this point in the history

  6. schema_registry/store: get_schema now returns references 

    `GET /schemas/ids/<id>` now returns its references.

Fixes redpanda-data#11194

Signed-off-by: Ben Pope <ben@redpanda.com>
(cherry picked from commit 9298788)
    @BenPope
    BenPope committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    6c1efcf View commit details
    Browse the repository at this point in the history