-
Notifications
You must be signed in to change notification settings - Fork 16
Fix #195: Acceptance tests based on Cucumber #205
Conversation
$ bundle exec rake -T | ||
|
||
<a name="acceptance-tests"></a> | ||
## Acceptance tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Acceptance Tests
would be more appropriate here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? You mean a capital 'T'? I don't think so?
@hferentschik Implementation looks great 👍 . Need to test it locally. |
|
||
- [Objective](#objective) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@budhrg thanks for the review. I fixed the typo and made some cosmetic changes to the Rakefile and added a features:open_report open report task. Regarding the Gemfile.lock, I think I still prefer to lock the version for development and mitigate gem file incompatibilities via a CI job and dedicated version checks prior to releases. If we agree on the other changes, we can drop the last commit for now and apply the test changes first in order to move on. |
thanks
Let me know how it goes. |
@dharmit so what you I need to do now to get access to a CentOS CI server? |
Ok, I removed the checkin of the Gemfile.lock, we can take this discussion later. I also added support for testing against libvirt when available. Maybe we can get this merged now. After all this is "just" a test framework. We can improve/adjust over time. I just don't won't to have to rebase forever. |
38eb107
to
421327d
Compare
_build/boxes/adb-\<provider\>.box_ resp _build/boxes/cdk-\<provider\>.box_. You can | ||
either copy the box files manually or use the _get_adb_ resp. _get_cdk_ Rake tasks. | ||
|
||
Per default the VirtualBox provider is used. However, you can also run against Libvirt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we rewrite it as Virtualbox provider is used as default provider.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why, the default spelling is VirtualBox as far as I know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean to say Let's rewrite this whole sentence as:
VirtualBox is used as a default provider. However, you can also run against Libvirt
As starting sentence using Per default
doesn't sound good to me. However, I would like @bexelbie and @Preeticp to review this README since they are mostly handling docs for ADB.
You can contact us on: | ||
* IRC: #atomic and #nulecule on freenode | ||
* Mailing List: container-tools@redhat.com | ||
After cloing the repository, install the [Bundler](http://bundler.io/) gem: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spell correction s/cloing/cloning/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
For a reference, here is the bug report for running Aruba on Windows - cucumber/aruba#387. Seems the project is still quite active and there is definitely interest to fix this. It might be ok to go with the Cucmber + Aruba approach under the assumption/hope that there will be a fix to the problem soonish. |
# Run tests against CDK using Libvirt | ||
$ bundle exec rake features BOX=cdk PROVIDER=libvirt | ||
|
||
# Run against ADB and CDK |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
probably Run against ADB and/or CDK
. As one can give one or both names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but the example is for the 'AND' case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Made a small change
Pushed an update which allows to mix and match which features are run against which provider and box usinf env variables FEATURE, BOX and PROVIDER. See README. Also made IP part of the test scenario |
$ bundle exec rake features:open_report | ||
|
||
<a name="builds"></a> | ||
## Builds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section can be deleted, imho.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah :-) I wanted to ask about this. I do it right away!
README LGTM with note about possible duplicative section deletion. |
93b17c3
to
1e65f76
Compare
@hferentschik it looks like you took the oldest commit while squashing. Date of commit is commit 1e65f76 |
$ bundle exec rake features BOX=cdk PROVIDER=libvirt | ||
|
||
# Run against ADB and CDK (boxes are comma seperated) | ||
$ budle exec rake features BOX=cdk,adb |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo error @hferentschik
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
LGTM 👍
|
The CDK tests on libvirt worked fine for me . Trying out it on OSX
|
@LalatenduMohanty Could you run tests with both?
|
I don't have vagrant box for ADB on my local workstation (the way test wants it) and downloading going to take time :( |
@LalatenduMohanty ok will check with ADB - libvirt combination and merge this PR. |
Got a ADB 1.8.0 box and it seems to work
|
Right now, no. It could be done, but then you need different Vagrantfile (one w/ URL and one w/o) for each box or work out. Makes the setup more complicated and you most likely need to duplicate test code. Hence I opted against it. Effectively you are getting the ADB box from Hashicorp anyways. That's where it is downloaded from, if you use the download task. I am also a big fan of keeping everything w/i the project, which includes tmp artifacts, tmp files and directories, etc. If I delete pkg and build, there should be nothing left which got created by any development tasks.
Come on, since when do we care re-downloading something like this. I bet you you have downloaded the box more than once already. So yes, the current approach downloads and keeps the box file local to the project, but until you delete the build (run clean task) directory it will be used from there. I also think it is in general a bad idea to interact with anything in ~/.vagrant. For example, when experimenting next with moving config into the box Vagrantfile, I will modify the Vagrantfile in ~/.vagrant. If at the same time I would run tests against the CDK using the box in ~/.vagrant I might get unexpected results. Also definitely a -1 for trying to re-package the box. |
- Adding required dependencies - Adding tests - Updating REDAME on how to run tests
It is not exactly the same. Basically if I use atlas I get un-compressed box. But if I use cloud.c.o I get the whole box. |
|
Can we open a separate ticket for issues we are finding for OS X and get this PR merged for Linux platform? This should unblock us to have a CI job first and we keep improving the tests for other platforms as well? |
@navidshaikh lets do that. |
Well, I should have merged using command line, this github UI merge will add one more commit in history. :-( Next time! |
LOL, thanks anyway |
This button can be so tempting ;-) |
To execute the Cucumber tests:
Reports in build/features_report.html.
Alternatively one can manually copy the box files to the expected directory (see README)
I switched from vagrant-spec to Cucumber, since the former seems to be lest maintained and the syntax and reporting of Cucumber/Gherkin is much better.
Thoughts?