Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataConnect + PGLite prototype. #7615

Merged
merged 30 commits into from
Sep 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
feb3560
Hacking away at pglite+dataconnect emulator
joehan Aug 15, 2024
412ae45
Hacking away at it
joehan Aug 19, 2024
0d8ec81
More debugging
joehan Aug 20, 2024
766ebd2
I THINK IT WORKSSSSS
joehan Aug 22, 2024
0c99fce
Merge remote-tracking branch 'public/master' into jh-pglite
joehan Aug 28, 2024
5b4e0f6
Saving my progress at EoD
joehan Aug 28, 2024
36dd2be
--amend
joehan Aug 28, 2024
b65fb1c
Progress
joehan Aug 30, 2024
dab3523
clean up
joehan Sep 3, 2024
3c8964c
formatting
joehan Sep 3, 2024
0a4c9e3
Use extended query patch (thanks @gregnr!)
joehan Sep 9, 2024
13a8590
Merge remote-tracking branch 'public/master' into jh-pglite
joehan Sep 9, 2024
c86ce4e
Format and merge master
joehan Sep 9, 2024
890bcc2
PR fixes
joehan Sep 9, 2024
63bcc6b
Remove JSON comments
joehan Sep 9, 2024
5e7da7c
format
joehan Sep 9, 2024
a8d8aa6
Fxing test compilation issues
joehan Sep 9, 2024
3866a0e
Cleaning up build issues
joehan Sep 9, 2024
d96874d
PR fixes
joehan Sep 10, 2024
17601cc
Format and generate json schema
joehan Sep 10, 2024
dfd93f8
Patching in missing types for VSCode builds
joehan Sep 10, 2024
25c1bc0
More type fixing
joehan Sep 10, 2024
2b7cdeb
More type fixing
joehan Sep 11, 2024
78a4f47
Little bit mroe code review
joehan Sep 11, 2024
feeb1b6
Removed unused dep
joehan Sep 11, 2024
7a9d054
More pr fixes
joehan Sep 11, 2024
115224b
test:emualtors build fixed
joehan Sep 11, 2024
d959f67
Merging in launch
joehan Sep 11, 2024
f1e038e
PR fixes
joehan Sep 11, 2024
1439370
Fix port disagreement
joehan Sep 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ module.exports = {
// TODO(jamesdaniels): add this to overrides instead
ignorePatterns: [
"src/dynamicImport.js",
"src/emulator/dataconnect/pg-gateway",
"scripts/webframeworks-deploy-tests/nextjs/**",
"scripts/webframeworks-deploy-tests/angular/**",
"scripts/frameworks-tests/vite-project/**",
Expand Down
88 changes: 10 additions & 78 deletions firebase-vscode/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion firebase-vscode/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@
"@teamsupercell/typings-for-css-modules-loader": "^2.5.1",
"@types/glob": "^8.0.0",
"@types/mocha": "^10.0.1",
"@types/node": "16.x",
"@types/node": "20.x",
"@types/react": "^18.0.9",
"@types/react-dom": "^18.0.4",
"@types/vscode": "^1.69.0",
Expand Down
1 change: 1 addition & 0 deletions firebase-vscode/src/data-connect/toolkit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ export class DataConnectToolkit implements vscode.Disposable {
config,
configDir,
rc: RC,
autoconnectToPostgres: false,
enable_output_generated_sdk: true,
enable_output_schema_extensions: true,
};
Expand Down
4 changes: 2 additions & 2 deletions firebase-vscode/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
"typeRoots": ["node_modules/@types", "../src/types"],
"module": "es2020",
"moduleResolution": "node",
"target": "ES2020",
"target": "ES2017",
"outDir": "dist",
"lib": ["ES2020"],
"lib": ["ES2020", "DOM" ],
"jsx": "react",
"sourceMap": true,
"rootDirs": ["src", "../src", "common"],
Expand Down
11 changes: 11 additions & 0 deletions npm-shrinkwrap.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
]
},
"dependencies": {
"@electric-sql/pglite": "^0.2.0",
"@google-cloud/cloud-sql-connector": "^1.3.3",
"@google-cloud/pubsub": "^4.5.0",
"abort-controller": "^3.0.0",
Expand Down
6 changes: 6 additions & 0 deletions schema/firebase-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,12 @@
},
"port": {
"type": "number"
},
"postgresHost": {
"type": "string"
},
"postgresPort": {
"type": "number"
}
},
"type": "object"
Expand Down
6 changes: 4 additions & 2 deletions scripts/emulator-tests/tsconfig.dev.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
},
"include": [
"../../src/**/*",
"*",
]
"../../src/*",
"./*",
],
"exclude": [],
}
2 changes: 1 addition & 1 deletion src/dataconnect/webhook.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ export interface WebhookBody {
content?: string;
}

export const DEFAULT_PORT = "400322"; // 6 digit default used by vscode;
export const DEFAULT_PORT = "40000"; // 6 digit default used by vscode;

// If port in use, VSCode will pass a different port to the integrated term through env var
export const port = process.env.VSCODE_WEBHOOK_PORT || DEFAULT_PORT;
Expand Down
3 changes: 3 additions & 0 deletions src/emulator/controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -851,6 +851,9 @@ export async function startAll(
configDir,
rc: options.rc,
config: options.config,
autoconnectToPostgres: true,
postgresHost: options.config.get("emulators.dataconnect.postgresHost"),
postgresPort: options.config.get("emulators.dataconnect.postgresPort"),
enable_output_generated_sdk: true, // TODO: source from arguments
enable_output_schema_extensions: true,
});
Expand Down
6 changes: 6 additions & 0 deletions src/emulator/dataconnect/pg-gateway/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
The code in this directory is a very slightly modified version of https://github.com/supabase-community/pg-gateway/tree/next.
Full credit for this code goes to @gregnr and the other contributors on that repo.

Due to some known issues with how PGLite handles prepared statements, this versiom of pg-gateway includes middleware
to remove the extra Ready for Query messages that break schema migration. Once these underlying issues with PGLite are fixed,
we'll migrate to a normal dependency on pg-gateway.
34 changes: 34 additions & 0 deletions src/emulator/dataconnect/pg-gateway/auth/base-auth-flow.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import type { BufferReader } from '../buffer-reader';
import type { BufferWriter } from '../buffer-writer';
import type { ConnectionSignal } from '../connection';
import type { ConnectionState } from '../connection.types';

type BufferSource = ArrayBufferView | ArrayBuffer;

export interface AuthFlow {
createInitialAuthMessage(): Uint8Array | undefined;
handleClientMessage(message: BufferSource): AsyncGenerator<Uint8Array | ConnectionSignal>;
isCompleted: boolean;
}

export abstract class BaseAuthFlow implements AuthFlow {
protected reader: BufferReader;
protected writer: BufferWriter;
protected connectionState: ConnectionState;

constructor(params: {
reader: BufferReader;
writer: BufferWriter;
connectionState: ConnectionState;
}) {
this.reader = params.reader;
this.writer = params.writer;
this.connectionState = params.connectionState;
}

abstract createInitialAuthMessage(): Uint8Array | undefined;
abstract handleClientMessage(
message: BufferSource,
): AsyncGenerator<Uint8Array | ConnectionSignal>;
abstract get isCompleted(): boolean;
}
Loading
Loading