Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[TEST] Added large tensor tests to verify support for split ops #19357

Merged
merged 1 commit into from
Oct 16, 2020

Conversation

access2rohit
Copy link
Contributor

@access2rohit access2rohit commented Oct 15, 2020

Large tensor tests for split, hsplit, vsplit and dsplit

Description

(Brief description on what this PR is about)

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Testing

(pytest) ubuntu@ip-172-31-90-243 ~/workspace/incubator-mxnet (split_lt) $ python -m pytest -s --exitfirst --verbose tests/nightly/test_np_large_array.py::test_split
==================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.2, pluggy-0.13.1 -- /home/ubuntu/anaconda3/envs/pytest/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/workspace/incubator-mxnet, inifile: pytest.ini
plugins: timeout-1.4.2
timeout: 1200.0s
timeout method: signal
timeout func_only: False
collected 1 item

tests/nightly/test_np_large_array.py::test_split Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1334543772 to reproduce.
[23:27:40] ../src/storage/storage.cc:199: Using Pooled (Naive) StorageManager for CPU
[23:27:41] ../src/base.cc:84: Upgrade advisory: this mxnet has been built against cuDNN lib version 7501, which is older than the oldest version tested by CI (7600).  Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
PASSED

===================================================================================================================================== warnings summary ======================================================================================================================================
tests/nightly/test_np_large_array.py:92
  /home/ubuntu/workspace/incubator-mxnet/tests/nightly/test_np_large_array.py:92: DeprecationWarning: invalid escape sequence \
    '''

tests/nightly/test_np_large_array.py:1323
  /home/ubuntu/workspace/incubator-mxnet/tests/nightly/test_np_large_array.py:1323: DeprecationWarning: invalid escape sequence \
    '''

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================================================================================== 1 passed, 2 warnings in 38.17s ===============================================================================================================================

(pytest) ubuntu@ip-172-31-90-243 ~/workspace/incubator-mxnet (split_lt) $ python -m pytest -s --exitfirst --verbose tests/nightly/test_np_large_array.py::test_hsplit
==================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.2, pluggy-0.13.1 -- /home/ubuntu/anaconda3/envs/pytest/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/workspace/incubator-mxnet, inifile: pytest.ini
plugins: timeout-1.4.2
timeout: 1200.0s
timeout method: signal
timeout func_only: False
collected 1 item

tests/nightly/test_np_large_array.py::test_hsplit Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1388706242 to reproduce.
[23:31:02] ../src/storage/storage.cc:199: Using Pooled (Naive) StorageManager for CPU
[23:31:03] ../src/base.cc:84: Upgrade advisory: this mxnet has been built against cuDNN lib version 7501, which is older than the oldest version tested by CI (7600).  Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
PASSED

=============================================================================================================================== 1 passed in 159.57s (0:02:39) ===============================================================================================================================

(pytest) ubuntu@ip-172-31-90-243 ~/workspace/incubator-mxnet (split_lt) $ python -m pytest -s --exitfirst --verbose tests/nightly/test_np_large_array.py::test_vsplit
==================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.2, pluggy-0.13.1 -- /home/ubuntu/anaconda3/envs/pytest/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/workspace/incubator-mxnet, inifile: pytest.ini
plugins: timeout-1.4.2
timeout: 1200.0s
timeout method: signal
timeout func_only: False
collected 1 item

tests/nightly/test_np_large_array.py::test_vsplit Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=631816591 to reproduce.
[23:34:19] ../src/storage/storage.cc:199: Using Pooled (Naive) StorageManager for CPU
[23:34:21] ../src/base.cc:84: Upgrade advisory: this mxnet has been built against cuDNN lib version 7501, which is older than the oldest version tested by CI (7600).  Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
PASSED

===================================================================================================================================== warnings summary ======================================================================================================================================
tests/nightly/test_np_large_array.py:92
  /home/ubuntu/workspace/incubator-mxnet/tests/nightly/test_np_large_array.py:92: DeprecationWarning: invalid escape sequence \
    '''

tests/nightly/test_np_large_array.py:1323
  /home/ubuntu/workspace/incubator-mxnet/tests/nightly/test_np_large_array.py:1323: DeprecationWarning: invalid escape sequence \
    '''

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================================================================= 1 passed, 2 warnings in 95.36s (0:01:35) ==========================================================================================================================

(pytest) ubuntu@ip-172-31-90-243 ~/workspace/incubator-mxnet (split_lt) $ python -m pytest -s --exitfirst --verbose tests/nightly/test_np_large_array.py::test_dsplit
==================================================================================================================================== test session starts ====================================================================================================================================
platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.2, pluggy-0.13.1 -- /home/ubuntu/anaconda3/envs/pytest/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/workspace/incubator-mxnet, inifile: pytest.ini
plugins: timeout-1.4.2
timeout: 1200.0s
timeout method: signal
timeout func_only: False
collected 1 item

tests/nightly/test_np_large_array.py::test_dsplit Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=124339485 to reproduce.
[23:37:48] ../src/storage/storage.cc:199: Using Pooled (Naive) StorageManager for CPU
[23:37:49] ../src/base.cc:84: Upgrade advisory: this mxnet has been built against cuDNN lib version 7501, which is older than the oldest version tested by CI (7600).  Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
PASSED

=============================================================================================================================== 1 passed in 76.49s (0:01:16) ================================================================================================================================
(pytest) ubuntu@ip-172-31-90-243 ~/workspace/incubator-mxnet (split_lt)

@mxnet-bot
Copy link

Hey @access2rohit , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [unix-gpu, sanity, clang, website, windows-cpu, miscellaneous, centos-cpu, unix-cpu, edge, centos-gpu, windows-gpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@lanking520 lanking520 added the pr-awaiting-testing PR is reviewed and waiting CI build and test label Oct 15, 2020
@access2rohit
Copy link
Contributor Author

@mxnet-label-bot add [pr-awaiting-review]

@lanking520 lanking520 added pr-awaiting-review PR is waiting for code review and removed pr-awaiting-review PR is waiting for code review labels Oct 15, 2020
Copy link
Contributor

@Zha0q1 Zha0q1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left 1 comment, rest looks good! It looks like internally they share the same implementation SplitOpForwardImpl


@use_np
def test_dsplit():
inp = np.arange(INT_OVERFLOW, dtype=np.int64).reshape(INT_OVERFLOW, 1, 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use create_2d_np_tensor here?

Copy link
Contributor Author

@access2rohit access2rohit Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 3d tensor. We don't have N-Dim function. Plus not too many cases where we need 3d tensor for testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh ok that makes sense

@lanking520 lanking520 added pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Oct 16, 2020
@sandeep-krishnamurthy sandeep-krishnamurthy merged commit 7bbe928 into apache:master Oct 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants