Error not handled when uploading a big form

[alxndrsn]

25000q-form.xml.txt

When the big form above is uploaded (with .txt extension removed), I see the following error:

RangeError: Maximum call stack size exceeded
    at sql (/odk/backend/node_modules/slonik/dist/src/factories/createSqlTag.js:36:29)
    at insertMany (/odk/backend/lib/util/db.js:157:13)
    at /odk/backend/lib/model/query/forms.js:72:15
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.transaction (/odk/backend/node_modules/slonik/dist/src/connectionMethods/transaction.js:22:24)
    at async Object.createConnection (/odk/backend/node_modules/slonik/dist/src/factories/createConnection.js:97:18)RangeError: Maximum call stack size exceeded
    at sql (/odk/backend/node_modules/slonik/dist/src/factories/createSqlTag.js:36:29)
    at insertMany (/odk/backend/lib/util/db.js:157:13)
    at /odk/backend/lib/model/query/forms.js:72:15
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.transaction (/odk/backend/node_modules/slonik/dist/src/connectionMethods/transaction.js:22:24)
    at async Object.createConnection (/odk/backend/node_modules/slonik/dist/src/factories/createConnection.js:97:18)::ffff:127.0.0.1 - - [29/Nov/2022:10:13:26 +0000] "POST /v1/projects/973/forms HTTP/1.1" 500 739
Error: Connection terminated
    at Connection.<anonymous> (/odk/backend/node_modules/pg/lib/client.js:132:36)
    at Object.onceWrapper (node:events:627:28)
    at Connection.emit (node:events:525:35)
    at Connection.emit (node:domain:489:12)
    at Socket.<anonymous> (/odk/backend/node_modules/pg/lib/connection.js:57:12)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at TCP.<anonymous> (node:net:757:14)node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);

Locally, this crashes my node process. On the server, the process does not crash, but the error is sent to Sentry.

An odk-central-frontend user sees:

The response body contains the full stacktrace (as per #549).

[matthew-white]

25,000 fields is a lot! I think that's larger than the big forms mentioned in this forum topic.

Based on the stack trace, this seems to be a general issue with insertMany() that could theoretically appear in other areas of the app as well. How about we have insertMany() throw a Problem if passed too many objects? Then at least the result will be a Problem instead of an unhandled error.

[sadiqkhoja]

There is an open issue related to this in Slonik: gajus/slonik#344

[srujner]

Tested with success!

[dbemke]

Tested with success!