Backport PR jupyter#608: Fix Windows tests

Fixes jupyter#599 (I think)

@lgpage if you could see if this helps on conda forge that would be great -- I *think* the issue really is just spurious test cases so I added in a pytest plugin that will rerun failed tests. This fixed the test failures at least on the Jenkins build server, so I'm hoping it will on appveyor too.

(There are a few other changes in this PR that address issues I was having with getting the tests to run on Jenkins, but the main thing is adding the `pytest-rerunfailures` to the dev requirements and then adding `--rerun=1` to the pytest command).

dev-requirements-windows.txt

@@ -0,0 +1,9 @@
+pytest
+pytest-cov
+pytest-rerunfailures
+coverage
+selenium
+invoke
+sphinx
+codecov
+cov-core

dev-requirements.txt

@@ -1,11 +1,4 @@
-pytest
-pytest-cov
-coverage
-selenium
-invoke
-sphinx
-codecov
-cov-core
+-r dev-requirements-windows.txt
 pyenchant
 sphinxcontrib-spelling
 sphinx_rtd_theme

nbgrader/docs/source/contributor_guide/documentation.rst

@@ -30,6 +30,11 @@ the rst version of it to ``.gitignore``.
 Building documentation locally
 ------------------------------
 
+.. warning::
+
+  Building the docs is not currently well-supported on Windows. This is because one of
+  the dependencies (enchant) does not install easily in Windows.
+
 If you have made changes to the user guide or other notebooks that need to be
 executed, please make sure you re-run all the documentation before committing.
 While the documentation gets built automatically on Read The Docs, the notebooks do **not** get execute by Read The Docs -- they must be executed manually.

nbgrader/docs/source/contributor_guide/installation_developer.rst

@@ -15,6 +15,11 @@ nbgrader installs and builds with one command::
 
     pip install -r dev-requirements.txt -e .
 
+Currently, building docs is not supported on Windows because some of the dependencies (enchant)
+are not easily installable. Instead of the above command, run the following on windows::
+
+    pip install -r dev-requirements-windows.txt -e .
+
 Installing notebook extensions
 ------------------------------
 Install the notebook extensions. The ``--symlink`` option is recommended since it

tasks.py

@@ -45,7 +45,7 @@ def docs(ctx):
     run(ctx, 'make -C nbgrader/docs spelling')
 
 
-def _run_tests(ctx, mark=None, skip=None):
+def _run_tests(ctx, mark=None, skip=None, junitxml=None):
     if not WINDOWS:
         import distutils.sysconfig
         site = distutils.sysconfig.get_python_lib()
@@ -76,8 +76,11 @@ def _run_tests(ctx, mark=None, skip=None):
     if not WINDOWS:
         cmd.append('--cov nbgrader')
         cmd.append('--no-cov-on-fail')
+    if junitxml:
+        cmd.extend(['--junitxml', junitxml])
     cmd.append('-v')
     cmd.append('-x')
+    cmd.extend(['--rerun', '1'])
 
     marks = []
     if mark is not None:
@@ -95,21 +98,21 @@ def _run_tests(ctx, mark=None, skip=None):
 
 
 @task
-def tests(ctx, group='all', skip=None):
+def tests(ctx, group='all', skip=None, junitxml=None):
     if group == 'python':
-        _run_tests(ctx, mark="not formgrader and not nbextensions", skip=skip)
+        _run_tests(ctx, mark="not formgrader and not nbextensions", skip=skip, junitxml=junitxml)
 
     elif group == 'formgrader':
-        _run_tests(ctx, mark="formgrader", skip=skip)
+        _run_tests(ctx, mark="formgrader", skip=skip, junitxml=junitxml)
 
     elif group == 'nbextensions':
-        _run_tests(ctx, mark="nbextensions", skip=skip)
+        _run_tests(ctx, mark="nbextensions", skip=skip, junitxml=junitxml)
 
     elif group == 'docs':
         docs(ctx)
 
     elif group == 'all':
-        _run_tests(ctx, skip=skip)
+        _run_tests(ctx, skip=skip, junitxml=junitxml)
 
     else:
         raise ValueError("Invalid test group: {}".format(group))