MultiJson::ParseError when running with parallel_tests

[cburgmer]

Hey, similar to #370 we are seeing a MultiJson::ParseError every once in a while, making our builds flaky.

The output looks like

[…]

Finished in 2 minutes 1.6 seconds (files took 7.21 seconds to load)
610 examples, 0 failures
.
Randomized with seed 5955

.......Coverage report generated for (6/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
Coverage report generated for (2/8), (6/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
Coverage report generated for (2/8), (3/8), (6/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
........Coverage report generated for (2/8), (3/8), (4/8), (6/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
...Coverage report generated for  to /tmp/repo/coverage. 8873 / 9455 LOC (93.84%) covered.
Coverage report generated for (2/8), (3/8), (4/8), (5/8), (6/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
...Coverage report generated for (2/8), (3/8), (4/8), (5/8), (6/8), (7/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
/tmp/gems/bundler/ruby/2.0.0/gems/multi_json-1.11.2/lib/multi_json/adapter.rb:19:in `load'/tmp/gems/bundler/ruby/2.0.0/gems/multi_json-1.11.2/lib/multi_json/adapter.rb:19:in `load': JSON::ParserError (MultiJson::ParseError)
    from /tmp/gems/bundler/ruby/2.0.0/gems/multi_json-1.11.2/lib/multi_json.rb:119:in `load'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/json.rb:8:in `parse'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/last_run.rb:10:in `read'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/defaults.rb:66:in `block in <top (required)>'
: JSON::ParserError (MultiJson::ParseError)
    from /tmp/gems/bundler/ruby/2.0.0/gems/multi_json-1.11.2/lib/multi_json.rb:119:in `load'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/json.rb:8:in `parse'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/last_run.rb:10:in `read'
    from /tmp/gems/bundler/ruby/2.0.0/gems/simplecov-0.9.0/lib/simplecov/defaults.rb:66:in `block in <top (required)>'
..............................................

Finished in 2 minutes 13.2 seconds (files took 7.39 seconds to load)
123 examples, 0 failures

Randomized with seed 29299

Coverage report generated for (1/8), (2/8), (3/8), (4/8), (5/8), (6/8), (7/8), (8/8), RSpec to /tmp/repo/coverage. 123 / 234 LOC (42.0%) covered.
Tests Failed

I've anonymised the input, do ignore irrelevant coverage numbers.

The .last_run.json retrieved after the failure looks like

{
    "result": {
        "covered_percent": 97.28
    }
}

It looks like the coverage is compiled before all parallel runs have finished (do see the final dots after the stack trace), and I am assuming a concurrency issue.

We are running simplecov 0.9.0 together with parallel_tests 1.5.0. I'll upgrade to 0.10.0 and see if the failures are ongoing.

[bf4]

Thanks for the report. Do you have a merge_timeout set? do you set the SimpleCov.pid? Do you set the command name? Can you show any parts of your simplecov config (no need for groups or filters)?

[cburgmer]

We don't set merge_timeout or pid or command_name.

I've red through the README but from the documentation for parallel_tests it is unclear to me whether simplecov is able to correctly guess the instance of the parallel run or whether I have explicitly tell it. From the code it looks like the automagic feature exists.

Our config is as simple as

SimpleCov.start :rails do
  add_filter "/vendor/cache/"
  add_group "Events", "app/events/"
  ... more groups...
end

[cburgmer]

Upgrading to 0.10.0 does not seem to fix the issue.

[bforma]

We're running into similar problems when running with parallel_tests:

/var/lib/jenkins/.rbenv/versions/2.3.0/lib/ruby/2.3.0/json/common.rb:156:in `initialize': A JSON text must at least contain two octets! (JSON::ParserError)
    from /var/lib/jenkins/.rbenv/versions/2.3.0/lib/ruby/2.3.0/json/common.rb:156:in `new'
    from /var/lib/jenkins/.rbenv/versions/2.3.0/lib/ruby/2.3.0/json/common.rb:156:in `parse'
    from /var/lib/jenkins/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/simplecov-0.11.2/lib/simplecov/last_run.rb:12:in `read'
    from /var/lib/jenkins/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/simplecov-0.11.2/lib/simplecov/defaults.rb:80:in `block in <top (required)>'

Any ideas?

PS. using simplecov version 0.11.2