Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove legacy extension api #1637

Merged
merged 23 commits into from
Sep 29, 2023

Conversation

braingram
Copy link
Contributor

@braingram braingram commented Aug 29, 2023

Builds on changes from:

Some notes so far:

Closes: #752
Closes #1622

Deserialization types vs converters

The old type system caught TypeError raised on deserialization and turned it into a warning:

asdf/asdf/yamlutil.py

Lines 343 to 349 in 53b87b4

try:
return tag_type.from_tree_tagged(node, ctx)
except TypeError as err:
warnings.warn(
f"Failed to convert {tag} to custom type (detail: {err}). Using raw Python data structure instead",
AsdfConversionWarning,
)

The new converters do not catch TypeError:

asdf/asdf/yamlutil.py

Lines 311 to 315 in 53b87b4

if extension_manager.handles_tag(tag):
converter = extension_manager.get_converter_for_tag(tag)
obj = converter.from_yaml_tree(node.data, tag, _serialization_context)
_serialization_context._mark_extension_used(converter.extension)
return obj

There were a few tests that relied on this behavior. However it appears that the raw value is only passed on when a TypeError is raised (all other exceptions bubble up and result in a failure to load the file). I see no documentation of this (looking at the 2.14.4 docs which have all descriptions of the old type system) and it seems like an inconsistent way to allow partial conversion failures during open. As we already have an open issue for discussing options for partial/graceful loading on failed conversions (#1555) I vote for not replicating this undocumented behavior of the type system and address the issue in a separate PR.

@braingram
Copy link
Contributor Author

braingram commented Aug 31, 2023

dkist also contains at least 1 ref by tag:
https://github.com/DKISTDC/dkist/blob/c7e61155a2ac73d6c0bd0b389c5aea263b17076e/dkist/io/asdf/resources/schemas/varying_celestial_transform-1.0.0.yaml#L33C1-L33C57

Opened PR updating the schema to use a uri in the ref instead of a tag:
DKISTDC/dkist#298

@braingram braingram added this to the 3.0.0 milestone Sep 11, 2023
@braingram braingram force-pushed the remove_legacy_extension_api branch 2 times, most recently from e428962 to bc03ed4 Compare September 20, 2023 13:43
@braingram braingram changed the title TEST: Remove legacy extension api Remove legacy extension api Sep 20, 2023
this prevents the BuiltinExtension from being
loaded which means that the default tag_mapping
that maps 'tag:stsci.edu:asdf' tags to uris
is no longer included. This would be a breaking
change so ``asdf.schema._tag_to_uri`` is added
to map just these tags and issue an AsdfDeprecationWarning
so we can remove this entirely in a later version
splitting up this commit would make the changes
easier to understand. However the above change
had numerous knock-on and dependent changes that
required updating many tests and removing other
deprecated functions/features. However, as
everything removed was already deprecated this
commit will exist in it's current overly large
size.
@braingram braingram marked this pull request as ready for review September 29, 2023 17:31
@braingram braingram requested a review from a team as a code owner September 29, 2023 17:31
Copy link
Contributor

@eslavich eslavich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weeee!

@braingram braingram merged commit acdfd9b into asdf-format:main Sep 29, 2023
40 of 42 checks passed
@braingram braingram deleted the remove_legacy_extension_api branch September 29, 2023 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove internal use of legacy extension API Add warning when schema $ref is a tag
2 participants