Parallel Cucumber and RSpec merge is overwriting earlier test

[dankohn]

I'm not able to merging my parallel RSpec and Cucumber tests. Merging non-parallel tests works fine. I do rm -r tmp coverage between each of the following:
rspec spec: 1885 / 2042 LOC (92.31%)
cucumber: 1698 / 1998 LOC (84.98%)
rspec spec && cucumber: 1999 / 2042 LOC (97.89%)
cucumber && rspec spec: 1999 / 2042 LOC (97.89%)
parallel_rspec spec && parallel_cucumber features: 1698 / 1998 LOC (84.98%)
parallel_cucumber features && parallel_rspec spec: 1885 / 2042 LOC (92.31%)

I get the same effect when I run from Rake. Any suggestions?

My .simplecov has a 20 minute timeout:

SimpleCov.start 'rails' do
  merge_timeout 1200
  add_group 'ActiveAdmin', 'app/activeadmin'
  add_group 'Validators', 'app/validators'
  add_filter '/config/'
  add_filter '/features/'
  add_filter '/lib/assets'
  add_filter '/lib/tasks'
  add_filter '/spec/'
  add_filter '/vendor/'
end

[bf4]

It appears your results are getting clobbered in the last two examples but not in the previous. What if you try something like

bundle exec ruby -rsimplecov -S parallel_cucumber features && bundle exec ruby -rsimplecov  -S parallel_rspec spec
# or better
bundle exec ruby -rsimplecov -S rake parallel:all

[dankohn]

The second half of the first statement is overwriting the first half, just as before.
bundle exec ruby -rsimplecov -S parallel_cucumber features && bundle exec ruby -rsimplecov -S parallel_rspec spec: 1885 / 2042 LOC (92.31%)

There is unfortunately no parallel:all Rake task. When I run a Rake task that invokes two other parallel Rake tasks, the second results clobber the first ones.

Anything else to try, please?

[bf4]

Maybe try setting the command_name from each task and process.... In
the .simplecov

[dankohn]

@bf4, I'm not quote sure where to put SimpleCov.command_name so that it's invoked only once for each test suite. The issue is that my Cucumber step definitions use RSpec expect statements, which I think means that the spec-helper.rb would be invoked twice.

I have one more data point that might point to a solution. I created the following Rake file:

task(:test).clear.enhance %w(rspec cucumber)

desc 'Calculate test coverage'
task coverage: :environment do
  sh 'rspec && cucumber'
end

rake coverage works correctly and I get 1999 / 2042 LOC (97.89%)

But when I do rake test, I get the following odd results:

$ rake test
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
........[Lots of dots].......
Finished in 7 minutes 1 second (files took 7.24 seconds to load)
1850 examples, 0 failures
Coverage report generated for RSpec to /Users/dan/Dropbox/Documents/dev/chewy/coverage. 1885 / 2042 LOC (92.31%) covered.
/Users/dan/.rbenv/versions/2.1.2/bin/ruby -S bundle exec cucumber  --profile default
Using the default profile...
........[Lots of dots].......
140 scenarios (140 passed)
690 steps (690 passed)
4m34.376s
Coverage report generated for Cucumber Features, RSpec to /Users/dan/Dropbox/Documents/dev/chewy/coverage. 1999 / 2042 LOC (97.89%) covered.
Coverage report generated for Cucumber Features, RSpec to /Users/dan/Dropbox/Documents/dev/chewy/coverage. 1698 / 1998 LOC (84.98%) covered.

What's surprising is that SimpleCov reports the correct merged results (97.89%) and then immediately overwrites them with the Cucumber-only results. But this only occurs when the spec and cucumber tasks are called separately by Rake, versus being invoked at the same time via the shell.

Any other ideas to try?

[colszowka]

Hey guys, only read that last mail, so maybe I'm off, but here is my suggestion: Put that command name call into one arbitrarily picked spec file. That'll let simplecov know for rspec, and skip for cukes.

In an ideal world I'd also suggest you just load rspec-expectantions itself in cucumber, and move any bootstrapping that happens in spec helper to some place that can be required by both.

On 24. Oktober 2014 22:15:58 MESZ, Dan Kohn notifications@github.com wrote:

@bf4, I'm not quote sure where to put SimpleCov.command_name so that
it's invoked only once for each test suite. The issue is that my
Cucumber step definitions use RSpec expect statements, which I think
means that the spec-helper.rb would be invoked twice.

I have one more data point that might point to a solution. I created
the following Rake file:

task(:test).clear.enhance %w(rspec cucumber)

desc 'Calculate test coverage'
task coverage: :environment do
 sh 'rspec && cucumber'
end

rake coverage works correctly and I get 1999 / 2042 LOC (97.89%)

But when I do rake test, I get the following odd results:

$ rake test
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
........[Lots of dots].......
Finished in 7 minutes 1 second (files took 7.24 seconds to load)
1850 examples, 0 failures
Coverage report generated for RSpec to
/Users/dan/Dropbox/Documents/dev/chewy/coverage. 1885 / 2042 LOC
(92.31%) covered.
/Users/dan/.rbenv/versions/2.1.2/bin/ruby -S bundle exec cucumber 
--profile default
Using the default profile...
........[Lots of dots].......
140 scenarios (140 passed)
690 steps (690 passed)
4m34.376s
Coverage report generated for Cucumber Features, RSpec to
/Users/dan/Dropbox/Documents/dev/chewy/coverage. 1999 / 2042 LOC
(97.89%) covered.
Coverage report generated for Cucumber Features, RSpec to
/Users/dan/Dropbox/Documents/dev/chewy/coverage. 1698 / 1998 LOC
(84.98%) covered.

What's surprising is that SimpleCov reports the correct merged results
(97.89%) and then immediately overwrites them with the Cucumber-only
results. But this only occurs when the spec and cucumber tasks are
called separately by Rake, versus being invoked at the same time via
the shell.

Any other ideas to try?

---

Reply to this email directly or view it on GitHub:
#339 (comment)

[bf4]

@dankohn See some examples in #340

[dankohn]

Thanks @bf4! Adding command_name "rails_app_#{$PROCESS_ID}" to my .simplecov fixed the problem of cucumber results overriding RSpec. My footer looks a little silly ("using RSpec, rails_app_73312, rails_app_73518, rails_app_73540, rails_app_73561, rails_app_73563, rails_app_73564, rails_app_73565, rails_app_73948, rails_app_73950, rails_app_73951, rails_app_73952, rails_app_74575") but the key thing is that the results are correct.

[bf4]

Closing as fixed