Allow passing matrix filename as benchmark input

[upsj]

When running benchmarks by hand, I always have to type -input '[{"filename":"..."}' by hand. The MatrixMarket and Ginkgo binary formats can be differentiated from JSON files by the first character: % is MM, G is binary, everything else is potentially JSON. So I had the idea to pass the matrix filename directly as input for the benchmark, either with the same flag (like in this PR) or with a different flag.

Not sure whether this overloads the -input flag too much, but I wanted to suggest it anyways.

[MarcelKoch]

I don't know if -input was part of the last release, but perhaps just rename that to -input_matrix and expect it to be a filename. There may not be much benefit of -input [{'filename': '...'}] over echo [{filename': '...'}] | benchmark.

[upsj]

The main benefit of passing JSON input via flag is that you don't need to work with pipes, which simplifies things like e.g. debugging with gdb (where you can't easily pass anything into stdin)

[MarcelKoch]

From some stackoverflow (https://stackoverflow.com/a/8422306) it seems like you can do this:

> gdb ./exec
> run < input.json

And also, perhaps it's enough for the debugging to have one input matrix.

[upsj]

That doesn't allow changing the parameters in a debugging session, since you cannot do echo ... | run.

[MarcelKoch]

you can do

echo ... > input.json
gdb ./exec
run < input.json

Or do you want to change stdin during the run?

[upsj]

No, but having to switch to another terminal is always inconvenient. But gdb is only one example, automation in Python is another one - having to specify stdin streams complicates the subprocess call and makes integration with other tools harder

[MarcelKoch]

In newer python version (>=3.7) you can just pass a string to subprocess.run(args, input=some_input). But I might not understand your workflow anyway.
Still, I think it would be better to have this parameter for the very simple case of using exactly one input file.

[upsj]

I simply don't see a reason to remove the functionality when it can make scripting things much easier. Running benchmarks for multiple BLAS parameters on multiple GPUs using parallel is another example case where
parallel command -input ... :: parameters is simpler than parallel bash -c 'echo ... | command' :: parameters

[MarcelKoch]

TBH, I'm still not sure if it makes scripting easier. But anyway, perhaps just add this under a new parameter name, to not overload the old one.

[upsj]

The reason I reused the name is purely practical: The parameter would either need to be put into general.hpp and thus be inapplicable for blas/multivector/matrix_generator or be duplicated into individual functions or put into an entirely separate header. I'll use -input_matrix

[yhmtsai]

could you remind me about the json output? I remember something related to the json. like the intermediate output only in stderr not stdout, or the stderr does not contain json

[upsj]

This is how it will be after #1323, at the moment, there is still JSON in both stderr and stdout

[MarcelKoch]

LGTM.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

90.9% Coverage
0.0% Duplication

The version of Java (11.0.3) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here