Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different test names with different runs #35

Closed
redixin opened this issue Nov 18, 2015 · 0 comments
Closed

Different test names with different runs #35

redixin opened this issue Nov 18, 2015 · 0 comments

Comments

@redixin
Copy link
Contributor

redixin commented Nov 18, 2015

Lets take a look at this simple test:

import unittest
import mock
from ddt import ddt, data

@ddt
class Test(unittest.TestCase):
    @data(mock.Mock())
    def test_dict(self, value):
        pass

This lead to different test names with different runs:

test_dict_1__Mock_id__139806852956624__ (tests.test_d.Test) ... ok
test_dict_1__Mock_id__140027200111056__ (tests.test_d.Test) ... ok

This is serious problem when running tests by testr.

Testr generates a list of all tests and divides it into N parts, depends on number of CPUs in the system.
Then it forks N processes, and each process loads one list.
Tests with mock.Mock in arguments are silently skipped because of unique mock id in test name.

redixin pushed a commit to redixin/ddt that referenced this issue Nov 18, 2015
Remove hash randomization checking. Simply do not mess with
non-trivial values. Simple enumerated test names are much better
then long ugly names with many underscores.

Close datadriventests#35
@txels txels closed this as completed in #36 Nov 18, 2015
openstack-gerrit pushed a commit to openstack/rally that referenced this issue Nov 19, 2015
_get_or_create_network: create networks only if there is no networks
in context.

_get_or_create_subnets: create subnets only if there is no subnets
in context.

Unit tests are passing because of bug in ddt [0]
This issue was actually caught by rally-ci (rally-pg-py27-unit) but was
ignored by revievers.

Running tests without testr may be used as workaround:

    .tox/py27/bin/python -m unittest discover tests/unit

[0] datadriventests/ddt#35

Change-Id: Ib77bd4f7eeaec2ed3a712e1d56a20d1f87325264
openstack-gerrit pushed a commit to openstack/requirements that referenced this issue Dec 8, 2015
Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
openstack-gerrit pushed a commit to openstack/openstack that referenced this issue Dec 8, 2015
Project: openstack/requirements  a8f5079210290e1cc5f8d837c7f1204be457a6ed

Bump ddt version to >=1.0.1

Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
openstack-gerrit pushed a commit to openstack/openstack that referenced this issue Dec 8, 2015
Project: openstack/requirements  a8f5079210290e1cc5f8d837c7f1204be457a6ed

Bump ddt version to >=1.0.1

Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
tanaypf9 pushed a commit to tanaypf9/pf9-requirements that referenced this issue May 20, 2024
Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
tanaypf9 pushed a commit to tanaypf9/pf9-requirements that referenced this issue May 20, 2024
Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
tanaypf9 pushed a commit to tanaypf9/pf9-requirements that referenced this issue May 20, 2024
Some tests may be skipped silently when using earlier versions with
testr and Mock instances in data. More details may be found by the
following link: datadriventests/ddt#35

Change-Id: I78286cb941f5f562f8f425a7ff24b4ac724f375e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant