[feat] ppl #1770

wu-hui · 2024-07-30T16:24:21Z

No description provided.

setup nix for idx

…uandy/PipelineResult

pipeline result

generated-files-bot · 2024-07-30T16:24:32Z

Warning: This pull request is touching the following templated files:

google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreAdminStubSettings.java
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/FirestoreClient.java
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/FirestoreSettings.java
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/gapic_metadata.json
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreStub.java
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreStubSettings.java
google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/GrpcFirestoreStub.java
google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/FirestoreClientTest.java
google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/MockFirestoreImpl.java
grpc-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/FirestoreGrpc.java
proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentProto.java
proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/FirestoreProto.java
proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Value.java
proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ValueOrBuilder.java
proto-google-cloud-firestore-v1/src/main/proto/google/firestore/v1/document.proto
proto-google-cloud-firestore-v1/src/main/proto/google/firestore/v1/firestore.proto

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…vaPplPP

tom-andersen · 2024-08-12T16:59:38Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/FirestoreImpl.java

@@ -29,6 +30,8 @@
 import com.google.api.gax.rpc.StreamController;
 import com.google.api.gax.rpc.UnaryCallable;
 import com.google.cloud.Timestamp;
+import com.google.cloud.firestore.Transaction.AsyncFunction;


nit: I am inclined not to import nested classes. In this case, the qualifier Transaction helps understand that this is not a lambda, such as java.util.function.Function.

tom-andersen · 2024-08-13T01:24:30Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

+ * }</pre>
+ */
+@BetaApi
+public final class Pipeline {


I think this class can be simplified. Here is a suggestion: #1786

Thanks, that PR looks very helpful.

tom-andersen · 2024-08-13T01:57:03Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

+            }
+            if (response.getResultsCount() > 0) {
+              numDocuments += response.getResultsCount();
+              if (numDocuments % 100 == 0) {


This condition seems wrong.

Maybe you should output trace when numDocuments > 100, and then reset numDocuments?

The trace message could add return numDocuments value instead of 100.

Also, text still says Firestore.Query and should be changed.

Yeah, copy/paste from query does not work for pipeline here.

tom-andersen · 2024-08-13T14:31:26Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/stages/Stage.java

+
+public interface Stage {
+  String getName();
+}


Instead of StageUtils.toStageProto, I think Stage interface should be expanded:

Suggested change

}

com.google.firestore.v1.Pipeline.Stage toStageProto()

}

This is what polymorphism what invented to solve.

Yeah, this was on my TODO list. I will address this later though, not urgent.

tom-andersen · 2024-08-13T15:12:45Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Expr.java

+   * @return A new {@code Expr} representing the addition operation.
+   */
+  @BetaApi
+  default Add add(Object other) {


Ambiguous types in overloads cause problems. The danger is that an Expr is cast to Object causing wrong method to be called.

You could add:

Suggested change

default Add add(Object other) {

default Add add(Object other) {

if (other instanceof Expr) {

return add((Expr) other);

}

But can we instead come up with an API surface that doesn't require inspection of type?

I don't think this is a problem. Constant.of should take care of the type inspection?

tom-andersen · 2024-08-13T15:52:18Z

...-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Constant.java

+  @BetaApi
+  public static Constant ofVector(double[] value) {
+    // Convert double array to List<Double>
+    return new Constant(Arrays.stream(value).boxed().collect(Collectors.toList()));


Pull from main and use Marks work on VectorValues.

tom-andersen · 2024-08-13T16:01:41Z

...-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Constant.java

+
+  @BetaApi
+  public static <T> Constant of(T[] value) {
+    return new Constant(Arrays.asList(value)); // Convert array to list


We should make a copy of array, thereby preventing accidental mutation.

Suggested change

return new Constant(Arrays.asList(value)); // Convert array to list

return new Constant(Arrays.asList(value.clone())); // Convert array to list

tom-andersen · 2024-08-13T17:27:51Z

...-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Function.java

+  private final String name;
+  private final List<Expr> params;
+
+  protected Function(String name, List<? extends Expr> params) {


I think we should prefer using ImmutableList instead of wrapping with unmodifiableList. Using ImmutableList prevents both upstream changes as well as downstream changes. Whereas, simply wrapping a list, still allows the caller to modify the list. Alternatively, we can always makes copies, but that seems wasteful.

The customer facing API can still use a mutable List as parameter, but we should always make a copy internally into a ImmutableList internally. Once we have an ImmutableList, we no longer need to make copies, and can safely pass them around by reference.

Suggested change

protected Function(String name, List<? extends Expr> params) {

protected Function(String name, ImmutableList<? extends Expr> params) {

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

# Conflicts: # README.md # google-cloud-firestore/pom.xml # google-cloud-firestore/src/main/java/com/google/cloud/firestore/AggregateQuery.java # google-cloud-firestore/src/main/java/com/google/cloud/firestore/Query.java # proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentProto.java

…vaPplPP # Conflicts: # README.md

wu-hui

Thanks for the suggestions!

wu-hui · 2024-09-16T19:23:05Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/FirestoreImpl.java

@@ -29,6 +30,8 @@
 import com.google.api.gax.rpc.StreamController;
 import com.google.api.gax.rpc.UnaryCallable;
 import com.google.cloud.Timestamp;
+import com.google.cloud.firestore.Transaction.AsyncFunction;


wu-hui · 2024-09-16T19:25:13Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

+ * }</pre>
+ */
+@BetaApi
+public final class Pipeline {


Thanks, that PR looks very helpful.

wu-hui · 2024-09-16T19:29:39Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

+            }
+            if (response.getResultsCount() > 0) {
+              numDocuments += response.getResultsCount();
+              if (numDocuments % 100 == 0) {


Yeah, copy/paste from query does not work for pipeline here.

wu-hui · 2024-09-16T19:32:42Z

...-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Constant.java

+
+  @BetaApi
+  public static <T> Constant of(T[] value) {
+    return new Constant(Arrays.asList(value)); // Convert array to list


wu-hui · 2024-09-16T19:34:31Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Expr.java

+   * @return A new {@code Expr} representing the addition operation.
+   */
+  @BetaApi
+  default Add add(Object other) {


I don't think this is a problem. Constant.of should take care of the type inspection?

wu-hui · 2024-09-16T19:38:02Z

...-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Function.java

+  private final String name;
+  private final List<Expr> params;
+
+  protected Function(String name, List<? extends Expr> params) {


wu-hui · 2024-09-16T19:40:24Z

google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/stages/Stage.java

+
+public interface Stage {
+  String getName();
+}


Yeah, this was on my TODO list. I will address this later though, not urgent.

# Conflicts: # google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

wu-hui and others added 30 commits February 14, 2024 15:52

Pipeline operations AST

658c952

Getting ready for fluent api

4ab3382

Basic fluent

a3d7234

Joins too

a837f62

Some fixups, and group bys

c8e7854

joins but nicer

c5a01a6

address feedback

191eb44

pagination and others

27f2748

minor adds

55ace39

adding more sugar to improve brevity

56c3a41

address some feedback

a5a90ab

support fromData

424ac43

pipeline result basic

4d8a205

execute

9119b95

fixing compilation error and add some high level comments

3aa73de

setup nix for idx

b583791

Merge pull request #3 from wu-hui/wuandy/IdxSetup

670ae65

setup nix for idx

tweaks

7216745

pipeline result basic

c46601f

execute

99e9067

fixing compilation error and add some high level comments

f20e409

tweaks

742d3c4

Merge remote-tracking branch 'pipelines/wuandy/PipelineResult' into w…

637d494

…uandy/PipelineResult

fix errors

7c67e99

fix errors

ba521a0

Add function composition example.

6ae7b13

build into a single jar

c1825e0

better aliasing for joins

0fe5faa

Merge pull request #2 from wu-hui/wuandy/PipelineResult

0d52712

pipeline result

add stuff to support challenges

7ab3099

wu-hui requested a review from a team as a code owner July 30, 2024 16:24

product-auto-label bot added the size: xl Pull request size is extra large. label Jul 30, 2024

product-auto-label bot added the api: firestore Issues related to the googleapis/java-firestore API. label Jul 30, 2024

add public docs

ab2aaab

wu-hui force-pushed the wuandy/JavaPplPP branch from a2e5133 to ab2aaab Compare August 2, 2024 18:22

🦉 Updates from OwlBot post-processor

dc1b06b

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

wu-hui assigned tom-andersen Aug 2, 2024

wu-hui and others added 4 commits August 3, 2024 10:31

Add public ref doc

7c697a5

Merge remote-tracking branch 'origin/wuandy/JavaPplPP' into wuandy/Ja…

edc1cc9

…vaPplPP

Simplify Pipeline

77decb5

Mirror Query class pattern

f8b9386

tom-andersen reviewed Aug 13, 2024

View reviewed changes

tom-andersen assigned wu-hui and unassigned tom-andersen Aug 14, 2024

wu-hui and others added 9 commits August 21, 2024 12:45

Cleanup and sync with server function updates

7355e49

🦉 Updates from OwlBot post-processor

c51a9e5

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Add proper vector value support

e021fdf

Merge remote-tracking branch 'origin/wuandy/JavaPplPP' into wuandy/Ja…

8f4f27a

…vaPplPP # Conflicts: # README.md

Fix tests and license

eb5a815

dot product and arrayConcat fix

a62cdde

findnearest update and length rename

762b005

remove __path__ hack

5825cb6

wu-hui commented Sep 16, 2024

View reviewed changes

wu-hui added 4 commits September 17, 2024 14:41

Address feedback

3a965e9

Fix immutablelist parameters

8d63fb6

Merge branch 'tomandersen/ppSuggestion' into wuandy/JavaPplPP

2f0e505

# Conflicts: # google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java

Merge Tom's suggestions.

de7a147

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feat] ppl #1770

[feat] ppl #1770

wu-hui commented Jul 30, 2024

generated-files-bot bot commented Jul 30, 2024 •

edited

Loading

tom-andersen Aug 12, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

tom-andersen Aug 13, 2024

wu-hui Sep 16, 2024

wu-hui left a comment

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

wu-hui Sep 16, 2024

	return new Constant(Arrays.asList(value)); // Convert array to list
	return new Constant(Arrays.asList(value.clone())); // Convert array to list

	protected Function(String name, List<? extends Expr> params) {
	protected Function(String name, ImmutableList<? extends Expr> params) {

[feat] ppl #1770

Are you sure you want to change the base?

[feat] ppl #1770

Conversation

wu-hui commented Jul 30, 2024

generated-files-bot bot commented Jul 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wu-hui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

generated-files-bot bot commented Jul 30, 2024 •

edited

Loading