Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce liftable constants to shaper to prepare for precompilation #32721

Closed
wants to merge 1 commit into from

Conversation

roji
Copy link
Member

@roji roji commented Jan 4, 2024

This PR is one piece of the precompiled query puzzle (#25009) - making sure that our generated shaper code is compatible with outputting to C#. This is something that materializer enthusiasts can work on independently, so we can start parallelizing the work. Of course, the actual infra introduced here also needs to be reviewed.

The problem is mainly removing Constant expression nodes which don't refer to literals (int, string), and therefore can't be outputted to C#. The fix is the "LiftableConstantExpression" mechanism; instead of integrating constants directly into the tree, materializer generation code can use a factory to generate these nodes, providing them with an expression that determines how to resolve the "constant", given starting points (e.g. the model). So instead of referencing an IEntityType as a constant, you create a liftable constant node that resolves that entity type given the model. In normal mode a visitor simply resolves these nodes, replacing them with the correct constant before compiling the expression; but in precompilation we'll instead generate code out of these resolvers.

This PR makes us fail if any non-liftable constant is in the shaper, so making all the tests green here should in principle mean that all our shaper generation is precompilation-compatible. Note also that the materializer changes you see in this PR were done in prototype/hacky mode a year ago, and need to be cleaned up/reviewed (but the concepts/infra should be OK).

In addition, this introduces a DEBUG-only ShaperPublicMethodVerifier, which makes sure that all method/constructor calls in the shaper reference fully public members (otherwise they can't be referenced in generated C# code).

Categories of test errors:

  1. "Materializer expression contains a non-literal constant of type X. Use a LiftableConstantExpression to reference any non-literal constants.". As the message says, we need to find the constant and replace it.
  2. "must be reducible node" - this is because materializer generation code contains a Compile() over a tree that contains a LiftableConstantExpression. We cannot use Compile() in any case - anywhere - since we lose the expression tree and can't generate C# from it. So all invocations of Compile() must be removed (except the top-level one that's called only for non-precompiled queries).
  3. "Method X isn't public and therefore cannot be invoked in shaper code" - this is the new ShaperPublicMethodVerifier. In principle it's sufficient to make sure that the method is public (with [EntityFrameworkInternal]!) - but also its declaring type, if it's on a nested type. But it might be an occasion to think a bit more widely about factoring etc., and maybe extract the method.

@roji roji requested review from ajcvickers and maumar January 4, 2024 18:41
@roji roji marked this pull request as draft January 4, 2024 18:53
@roji roji mentioned this pull request Jan 5, 2024
1 task
Comment on lines +25 to +26
var resolver = resolverExpression.Compile(preferInterpretation: true);
var value = resolver(_relationalMaterializerLiftableConstantContext);
Copy link
Member

@AndriySvyryd AndriySvyryd Feb 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to also store the constant value in LiftableConstant and only use Compile to assert it's the same value in Debug
That gives the C# generator a chance to determine whether it can just generate the constant instead of the resolver without having to compile

Copy link
Member Author

@roji roji Feb 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's something I debated with myself when writing this. I think you're right, and I like the DEBUG-only check (though I sometimes wish we ran tests in CI in DEBUG as well...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maumar if you're interested in doing this, it would be an opportunity to put hands inside the infra as well - let me know.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW there could be some issues with asserting "same value" here - I'm not sure whether in all cases reference equality works (i.e. if the resolver instantiates a new instance that's equivalent but not reference-equal, this starts to possibly get complicated). Anyway, we can think about this together once the other work in the PR is done.

Comment on lines +236 to +238
// IReadOnlyList<ValueComparer> parentIdentifierValueComparers,
// IReadOnlyList<ValueComparer> outerIdentifierValueComparers,
// IReadOnlyList<ValueComparer> selfIdentifierValueComparers,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndriySvyryd yeah, this code is very much what I hacked together to make basic things work during the prototype. This absolutely needs to be cleaned up (and support extended for all cases).

@roji
Copy link
Member Author

roji commented Apr 13, 2024

Replaced by #33351

@roji roji closed this Apr 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants