Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cudaErrorIllegalAddress: an illegal memory access was encountered #136

Closed
felixmr1 opened this issue Oct 26, 2023 · 1 comment
Closed

cudaErrorIllegalAddress: an illegal memory access was encountered #136

felixmr1 opened this issue Oct 26, 2023 · 1 comment

Comments

@felixmr1
Copy link

felixmr1 commented Oct 26, 2023

After building the focal_cuda docker container and running the tests within it. I get this error:

[==========] Running 9 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 1 test from GICPTestBase
[ RUN      ] GICPTestBase.LoadCheck
[       OK ] GICPTestBase.LoadCheck (9 ms)
[----------] 1 test from GICPTestBase (9 ms total)

[----------] 8 tests from AlignmentTest2/AlignmentTest
[ RUN      ] AlignmentTest2/AlignmentTest.test/GICP_ST
[       OK ] AlignmentTest2/AlignmentTest.test/GICP_ST (317 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/GICP_MT
[       OK ] AlignmentTest2/AlignmentTest.test/GICP_MT (117 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/VGICP_ST
[       OK ] AlignmentTest2/AlignmentTest.test/VGICP_ST (211 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/VGICP_MT
[       OK ] AlignmentTest2/AlignmentTest.test/VGICP_MT (93 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/VGICP_CUDA_ST
/root/fast_gicp/src/test/gicp_test.cpp:164: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
FORWARD TEST
/root/fast_gicp/src/test/gicp_test.cpp:174: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
BACKWARD TEST
/root/fast_gicp/src/test/gicp_test.cpp:186: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
SWAP AND SET SOURCE TEST
/root/fast_gicp/src/test/gicp_test.cpp:198: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
SWAP AND SET TARGET TEST
[  FAILED  ] AlignmentTest2/AlignmentTest.test/VGICP_CUDA_ST, where GetParam() = (0x55a2afbf1209 pointing to "VGICP_CUDA", false) (5579 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/VGICP_CUDA_MT
/root/fast_gicp/src/test/gicp_test.cpp:164: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
FORWARD TEST
/root/fast_gicp/src/test/gicp_test.cpp:174: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
BACKWARD TEST
/root/fast_gicp/src/test/gicp_test.cpp:186: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
SWAP AND SET SOURCE TEST
/root/fast_gicp/src/test/gicp_test.cpp:198: Failure
Expected: (errors[0]) < (t_tol), actual: 0.497354 vs 0.05
SWAP AND SET TARGET TEST
[  FAILED  ] AlignmentTest2/AlignmentTest.test/VGICP_CUDA_MT, where GetParam() = (0x55a2afbf1209 pointing to "VGICP_CUDA", true) (69 ms)
[ RUN      ] AlignmentTest2/AlignmentTest.test/NDT_CUDA_ST
terminate called after throwing an instance of 'thrust::system::system_error'
  what():  parallel_for failed: cudaErrorIllegalAddress: an illegal memory access was encountered
Aborted (core dumped)

Has anyone gotten the same?

Output of nvidia-smi on host computer:

Thu Oct 26 15:35:02 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.113.01             Driver Version: 535.113.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3050 ...    Off | 00000000:01:00.0 Off |                  N/A |
| N/A   50C    P8               9W /  40W |    603MiB /  4096MiB |      2%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      3379      G   /usr/lib/xorg/Xorg                          373MiB |
|    0   N/A  N/A      3508      G   /usr/bin/gnome-shell                         93MiB |
|    0   N/A  N/A    475224      G   x-terminal-emulator                           8MiB |
|    0   N/A  N/A    479320      G   x-terminal-emulator                           8MiB |
|    0   N/A  N/A    488622    C+G   ...8017866,16359961026231572468,262144       14MiB |
|    0   N/A  N/A    717283      G   x-terminal-emulator                           8MiB |
|    0   N/A  N/A    725490      G   x-terminal-emulator                           8MiB |
|    0   N/A  N/A    737069      G   x-terminal-emulator                           8MiB |
+---------------------------------------------------------------------------------------+

My OS is pop_OS 22.04 - which essentially is ubuntu 22.04

@felixmr1
Copy link
Author

Should be fixed by: #137

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant