You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a specific problem?
CTL Datasets could use some increased validation before they are saved to the database. This is coming up when we try to link an existing CTL Dataset to a DatasetConfig and it is missing expected elements.
Describe the solution you'd like
Before writing a CTL Dataset to the db, verify that supplied data categories match those in the database
Validate against the Fideslang Dataset schema, which requires that an organizational_fides_key and a data_qualifier exist.
Note this will be a little tricky because this is a crud endpoint which is written in a generic way to work with numerous ctl-resources, beyond just Datasets.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
I see in the crud routes that are programatically defined, it does look like there's a mapping for the fides lang models. The create route for example will have the resource validated against the appropriate Fideslang model, so Datasets are being validated through this endpoint.
However, the upsert endpoint just has resources as a list of Dictionaries, I assume this is where the issue is. Hopefully this is just a quick change.
pattisdr
changed the title
Add Increased CTL Dataset Validation on Create/Update
Add Increased CTL Dataset Validation on Create/Update/Upsert
Jan 5, 2023
- Fix crud upsert endpoints so they validate the request against the appropriate pydantic model.
- When creating, updating, or upserting Datasets, validate that the supplied data categories align with those currently in the db. (Note that this has to happen outside of typical Pydantic validation because we need access to a db session)
Is your feature request related to a specific problem?
CTL Datasets could use some increased validation before they are saved to the database. This is coming up when we try to link an existing CTL Dataset to a DatasetConfig and it is missing expected elements.
Describe the solution you'd like
Dataset
schema, which requires that anorganizational_fides_key
and adata_qualifier
exist.Context:
#2096 (comment)
#2113 (comment)
Describe alternatives you've considered, if any
Note this will be a little tricky because this is a crud endpoint which is written in a generic way to work with numerous ctl-resources, beyond just Datasets.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: