[Reporting] 'preserve layout' PDF reports are not working

[tsullivan]

Version: 8.8.0 BC3

Method used to run Kibana: source / docker

Steps to reproduce:

1. Trial license
2. Use any sample data
3. Create a PDF report of the sample data dashboard. Do not use the "optimize for printing" checkbox.

Result:
Report fails with an error message: Failed with: x_content_parse_exception Caused by: stream_constraints_exception: String length (5009248) exceeds the maximum length (5000000) Root causes: x_content_parse_exception: [1:29] [UpdateRequest] failed to parse field [doc]"

Full error:

[2023-05-10T14:33:21.535-07:00][INFO ][plugins.screenshotting] taking screenshots
[2023-05-10T14:33:22.294-07:00][INFO ][plugins.screenshotting] screenshots taken: 1
[2023-05-10T14:33:22.336-07:00][INFO ][plugins.screenshotting] Compiling PDF using "preserve_layout" layout...
[2023-05-10T14:33:25.364-07:00][ERROR][plugins.reporting.runTask] Error: ReportingError(code: unknown_error) "Max attempts (1) reached for job lhi7w3fq0iki0719c02p2j0k. Failed with: x_content_parse_exception
        Caused by:
                stream_constraints_exception: String length (5009248) exceeds the maximum length (5000000)
        Root causes:
                x_content_parse_exception: [1:29] [UpdateRequest] failed to parse field [doc]"
    at mapToReportingError (map_to_reporting_error.ts:50:10)
    at Object.run (execute_report.ts:449:50)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at TaskManagerRunner.run (task_runner.ts:314:22)

Analysis:
The error comes from attempting to send the report contents into reporting plugin's content stream. The following code patch catches and logs the error:

diff --git a/x-pack/plugins/reporting/server/lib/content_stream.ts b/x-pack/plugins/reporting/server/lib/content_stream.ts
index 0228132200e..c8e38c4a440 100644
--- a/x-pack/plugins/reporting/server/lib/content_stream.ts
+++ b/x-pack/plugins/reporting/server/lib/content_stream.ts
@@ -289,11 +289,16 @@ export class ContentStream extends Duplex {
     const chunk = Buffer.concat(buffersToFlush);
     const content = this.encode(chunk);

-    if (!this.chunksWritten) {
-      await this.removeChunks();
-      await this.writeHead(content);
-    } else if (chunk.byteLength) {
-      await this.writeChunk(content);
+    try {
+      if (!this.chunksWritten) {
+        await this.removeChunks();
+        await this.writeHead(content);
+      } else if (chunk.byteLength) {
+        await this.writeChunk(content);
+      }
+    } catch (err) {
+      this.logger.error(err);
+      throw err;
     }

     if (chunk.byteLength) {

Question: why is this bug not caught in CI?

[elasticmachine]

Pinging @elastic/appex-sharedux (Team:SharedUX)

[tsullivan]

Possibly fixed by elastic/elasticsearch#96031

[tsullivan]

Tests are passing now in Kibana main. Still investigating the next release build candidate to see if the bug is fixed there.

[kfirpeled]

Hi @tsullivan, do you know how I can verify it is going to be part of the next BC before I trigger the job?

Currently, the next BC scheduled for tomorrow morning CST time

[kfirpeled]

Another thing, in case elastic/elasticsearch#96031 is a possible fix.
There's a BC that already includes it

[tsullivan]

Thanks @kfirpeled for the bump! BC4 just came out and you caught me right as I was testing this in BC4.

I confirm PDF reports are working now 🎉. Closing this issue.

[tsullivan]

Question: why is this bug not caught in CI?

Answer
The issue in Elasticsearch prevented report contents from being saved if the encoded contents are too large. The size limitation from the error was stated to be 5mb. The PDF binary is stored as a base64 string.

A 5mb string is 5242880 characters. Each character is used to represent 6 bits. A 5mb string of base64 can encode a 3.932mb PDF.

Most of the reporting tests do not create a PDF that large. However in the latest main, the "[Logs] Web Traffic" sample data dashboard does create a PDF that large. It was during testing that dashboard manually when we noticed this bug.

Shortly after, the root cause issue did cause many CI failures in a few reporting tests in other branches:

• Failing ES Forward Compatibility: Default CI Group #7 / reporting management app Listing of Reports "before each" hook for "Confirm single report deletion works" #157035
• Failing ES Promotion: FTR Configs #10 / reporting management app Listing of Reports "before each" hook for "Confirm single report deletion works" #156901
• Failing ES Promotion: FTR Configs #42 / Reporting APIs Reporting error codes adds warning text with cause of failure in report output #156902