Using simplecov for Cucumber Selenium tests

[Vygo]

Hey, guys
We've been successfully using Simplecov for the rspec unit test code coverage coverage
Here is how are setup looks like

89% code coverage, everything is great
So last week we decided to use it to get some numbers for Cucumber/Capybara/Selenium test
Here is the setup:

We found the problem that code coverage is not being reposted. It looks like only the initialization of all ruby files happening.

Anyone can point me on what am I doing wrong, no matter how many of our UI tests I run the numbers don't update.

Version of simplecov 0.9.0
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-darwin13.0]
rails 4.2.0

[bf4]

Aside: what are you using spork for? or rcov? Also, behold, multiple formatters: https://github.com/colszowka/simplecov#using-multiple-formatters

So, there's a couple of issues here that boil down to using merge_timeout and possibly multiple command names. See https://github.com/colszowka/simplecov#timeout-for-merge and https://github.com/colszowka/simplecov#test-suite-names and perhaps #340 the issue that I might never finish

[Vygo]

We don't have a problem with second setup for our Unit tests, it works perfectly and there are no complains.
We have a problem with regression selenium tests using Cucumber (first setup), which as it shown does do anything else then initialization.

[avivalk]

Does anyone know what we are doing wrong? What is wrong with the Cucumber regression tests setup with SimpleCov that it only initializes all the Ruby files?

[bf4]

@avivalk are you and @Vygo on the same code?

[avivalk]

@bf4 , I am the one using it now and was hoping you could provide some guidance as to where the setup is wrong.
Thanks!

[bf4]

@avivalk That for following up. I made some suggestions above. Did you look at them? Did you look at the advice in the README for using browser tests? https://github.com/colszowka/simplecov#getting-started It's really hard to help without being able to reproduce it. Maybe you could make a test repo?

[avivalk]

@bf4 Were your suggestions above for the Cucumber integrations tests setup or the rspec unit test setup?
I am having issues with the cucumber integrations tests setup and I have followed the instructions in the readme.
I have the gem added, I have
require 'simplecov'
SimpleCov.start 'rails'
in the beginning of the env.rb file.

However, no matter how many tests I run, the code coverage is always the same number for initializing all the ruby files.

Am I missing something in the setup?
Thank you!

[bf4]

@avivalk see #393 (comment)

It's really hard to help without being able to reproduce it. Maybe you could make a test repo?

[avivalk]

@bf4 , here is a test repo I created https://github.com/avivalk/Tweeter
Two things to note though -

1. There is a different issue with the simplecov coverage where it is now showing 100% coverage when I run either one or two tests (and even one test should not give 100% coverage)
2. This repo does not have coverage for the rspec unit tests whereas the other repo that I am working with does - could the two coverages affect each other?

Please feel free to fork the Tweeter repo and play around and add to it.
Thank you!

[avivalk]

@bf4 have you looked at the Tweeter test repo I created? I'm pretty stuck with getting my code coverage and any help would be much appreciated!
Thank you!

[bf4]

I haven't yet, sorry. Juggling things... :(

[bf4]

@colszowka @xaviershay @sferik @amatsuda any of you want to take a look at this? Cucumber Selenium tests with a test app? I've not looked at it since late June.

[xaviershay]

I'll see if I can take a look on the weekend

[xaviershay]

Hi @avivalk @Vygo,
Sorry but this still isn't quite reproducible enough. I spent 10 minutes trying to get it working but failed. Specifically: you're missing relevant gems from the Gemfile that you require in env.rb. I started adding them, but there are too many listed in features/support/env.rb. I tried commenting out many of the requires, then also the RSpec.configure (why is this in your cucumber env?), but then the cukes were failing on a chromedriver executable thing. Then I gave up :(

Here is what we need from you to reopen: actual bash commands that will reproduce the problem. It should look something like the following:

cd /tmp
git clone git@github.com:avivalk/Tweeter.git
cd Tweeter
bundle
bundle exec features
# Document actual output
# Document expected output

Hopefully this doesn't come across as me being stubborn, I just have limited time for open source contributions! If this really is a simplecov issue I do want to get to the bottom of it, but we have many people using cucumber with no issues so a repro from you is critical if we're to solve this.

Thank you for your continued patience.
Xavier