Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Json/Query: translate element access of a json array #28648

Closed
maumar opened this issue Aug 10, 2022 · 1 comment · Fixed by #29656
Closed

Json/Query: translate element access of a json array #28648

maumar opened this issue Aug 10, 2022 · 1 comment · Fixed by #29656
Assignees
@maumar
Labels
area-json closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-enhancement
Milestone
8.0.0

Comments

@maumar
Copy link
Contributor

maumar commented Aug 10, 2022 •
edited
Loading

Something like:

entity.Where(x => x.JsonCollection[0].NestedReference.Name != "Foo")

When done in projection, this breaks the ordinal key logic, so we need to force AsNoTracking for these queries.

We could maybe utilize json de-duplication logic when the owner is present. For now de-dup is only working on navigation names, would need to incorporate array indexes also.

i.e.

entity.Select(x => new { x, ArrayAccess = x.JsonCollection[0] })
@roji
Copy link
Member

roji commented Aug 10, 2022

Same as #25701 (for Cosmos). We should think if there's some way to evolve the two together.

@ajcvickers ajcvickers added this to the Backlog milestone Aug 10, 2022
@smitpatel smitpatel mentioned this issue Sep 7, 2022
Merged
@maumar maumar self-assigned this Oct 19, 2022
@maumar maumar mentioned this issue Nov 5, 2022
Closed
@maumar maumar modified the milestones: Backlog, 8.0.0 Nov 16, 2022
maumar added a commit that referenced this issue Nov 17, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
c0c0e08 
Converting indexer into AsQueryable().ElementAt(x) so that nav expansion can understand it and inject MaterializeCollectionNavigationExpression in the right places.
Then in translation we recognize the pattern and convert it back to element access on a JsonQueryExpression.
In order to shape the results correctly, we need to add all the array indexes (that are not constants) to the projection, so that we can populate the ordinal keys correctly (and also to do de-duplication).

Fixes #28648
@maumar maumar mentioned this issue Nov 18, 2022
Closed
maumar added a commit that referenced this issue Nov 23, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
6316634 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
@maumar maumar mentioned this issue Nov 23, 2022
Merged
maumar added a commit that referenced this issue Nov 23, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
8f489ed 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 23, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
e5d24bd 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 23, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
c7d144c 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
c4cf1b8 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
1067416 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
fc8de63 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
517e7c7 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
f0daace 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 24, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
e11438f 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
ab51c0f 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
6abc953 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
329ef8e 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
0ab8515 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
a67d570 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Nov 30, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
f852a75 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Dec 2, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
3f1b691 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase (specifically in ExpandSharedTypeEntities) we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Dec 5, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
3999868 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase (specifically in ExpandSharedTypeEntities) we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
maumar added a commit that referenced this issue Dec 5, 2022
@maumar
Fix to #28648 - Json/Query: translate element access of a json array
8f76e65 
Converting indexer over list/array into ElementAt, so that nav expansion understands it and can perform pushown and inject MaterializeCollectionNavigation expression where necessary.
In translation phase (specifically in ExpandSharedTypeEntities) we recognize the pattern that nav expansion creates and if the root is JsonQueryExpression, we apply collection index over it.
JsonQueryExpression path segment now consists of two components - string representing JSON property name and SqlExpression representing collection index (it can be constant, parameter or any arbitrary expression that resolves to int)

Deduplication is heavily restricted currently - we only de-duplicate projections whose additional path consists of JSON property accesses only (no collection indexes allowed).
All queries projecting entities that need JSON array access must be set to NoTracking (for now). This is because we don't flow those collection index values into shaper. Instead, the ordinal keys are filled with dummy values, which prohibits us from doing proper change tracking.

Fixes #28648
@maumar maumar closed this as completed in 94be38c Dec 5, 2022
@maumar maumar added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Dec 5, 2022
@ajcvickers ajcvickers mentioned this issue Jan 5, 2023
Closed
@ajcvickers ajcvickers modified the milestones: 8.0.0, 8.0.0-preview1 Jan 29, 2023
@roji roji mentioned this issue May 16, 2023
Closed
@ajcvickers ajcvickers modified the milestones: 8.0.0-preview1, 8.0.0 Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maumar maumar

Labels
area-json closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. type-enhancement
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

Fix to #28648 - Json/Query: translate element access of a json array dotnet/efcore
3 participants
@ajcvickers @maumar @roji