Introduce option to setup transaction before executing queries #3471
Conversation
Introduces a setup query config to have code executed transactionally with the query itself. Also, a way to define the setup in one place, and reuse it. Also, fix some issues with transacitonal code. This gets it working for embedded and jdbc-in-process
I also added an ExternalSingleServerConfig to help myself debug, and figured it was worth keeping.
...rc/main/java/com/apple/foundationdb/relational/server/jdbc/v1/TransactionRequestHandler.java
Outdated
Show resolved
Hide resolved
...onal-jdbc/src/main/java/com/apple/foundationdb/relational/jdbc/JDBCRelationalConnection.java
Outdated
Show resolved
Hide resolved
...ain/java/com/apple/foundationdb/relational/yamltests/configs/ExternalSingleServerConfig.java
Outdated
Show resolved
Hide resolved
| create table t1(id bigint, col1 bigint, primary key(id)) | ||
| create table table_t1(id bigint, col1 bigint, primary key(id)) | ||
| --- | ||
| transaction_setups: |
There was a problem hiding this comment.
I looked into using yaml's repeated nodes feature, but since each test_block is a separate "document" you can't really do that, so instead I'm adding our own reference section.
| create table table_t1(id bigint, col1 bigint, primary key(id)) | ||
| --- | ||
| transaction_setups: | ||
| t1_less_than_50: create temporary function t1() on commit drop function AS SELECT * FROM table_t1 where id < 50 |
There was a problem hiding this comment.
In theory, it wouldn't be hard to change this to allow an array of strings if there's a bunch of setup steps
Conflicted with the new Debugger query config
ScottDugas
changed the title
| maxRows:;initialVersionAtLeast:;initialVersionLessThan:;;mode:;repetition:;seed:;setup:;transaction_setups:; | ||
| setupReference:;check_cache:;connection_lifecycle:;steps:;preset:;statement_type:;!r;!in;!a;" /> |
There was a problem hiding this comment.
We seem pretty incosistent in general about whether things should be snake_case or camelCase
The bug was fixed in 4.4.7.0 so this no longer needs to be !current_version
| } else if (QueryConfig.QUERY_CONFIG_SETUP.equals(queryConfig.getConfigName())) { | ||
| Assert.that(!queryIsRunning, "Transaction setup should not be intermingled with query results"); | ||
| executor.addSetup(Matchers.string(Matchers.notNull(queryConfig.getVal(), "Setup Config Val"), | ||
| "Transaction setup")); |
There was a problem hiding this comment.
There seems to be some merit in enforcing "setup" configs to always appear consecutively and directly above "query"? I suppose that will be more visibly clear given what "setup" is supposed to do.
| final RelationalStatement statement = singleConnection.createStatement(); | ||
| statement.execute(setupStatement); |
There was a problem hiding this comment.
Hmm, while I see this to be working well for creating temporary functions, I am a unclear on the scope of SQL statements that can be used in "setup". For instance, UPDATE .... RETURNING ... won't work here as it requires slurping the result set. Now that I think of it, DQL works fine and so does DML except the case I mentioned above.
There was a problem hiding this comment.
I think we should either,
- restrict the scope of statements here (maybe to only CREATE TEMPORARY FUNCTION ....) by some sort of patern matching, or we can error if the
executereturns aResultSet(barring DQL completely). - or, we could softly specify in documentation about its specific usages and cautions.
Open to your thoughts on this!
| - setup: create temporary function t1() on commit drop function AS | ||
| SELECT * FROM table_t1 where id < 30; | ||
| - result: [] | ||
| - | ||
| # This query references the transaction setup from above. This behaves exactly the same as the inline version | ||
| # it just allows easier use of the same setup for many queries. | ||
| - query: select * from t1 where id > 15; | ||
| - setupReference: t1_less_than_50 | ||
| - result: [{id: 30, col1: 40}] |
There was a problem hiding this comment.
It will go to add cases here to showcase/tests that utilizes multiple setup configs (looks like that is supported!) and that the order of execution is preserved!
There was a problem hiding this comment.
Another case could be to have a mix of setup and setupReference intermingled
| # The map is basically from a name (e.g. t1_less_than_50) to a statement to run at the start of every query. | ||
| # They're referenced in the query using `setupReference`. Right now it only supports a single statement as a string, but | ||
| # it could be expanded to support an array in the future. | ||
| transaction_setups: |
There was a problem hiding this comment.
it seems like multiple transaction_setups are allowed, so they can be spread all over the file. However, since all of them are processed in parsing, the lifecycle of all the kv pairs is entire runtime. I think it will be good to disallow that or have an example citing this behavior.
| @@ -0,0 +1,30 @@ | |||
| /* | |||
| * SQLSupplier.java | |||
This adds the ability to specific a transaction setup to yamsql.
More specifically for an individual query you can have:
setup:which will be executed at the start of every transaction for that queryTo avoid writing the same setup repeatedly, it also adds a new block:
transaction_setups:which allows you to create references for the setups. Inside the query, it can now be referenced via the key, usingsetupReference:.This can be useful if you want to test with temporary functions, but still have all the benefits of randomized, and forced-continuations.
This currently only supports a single query in the setup, but it wouldn't be hard to extend it to take an array.