CUSPARSE_STATUS_INVALID_VALUE when using features_at_coordinates method #308
Labels
bug
Something isn't working
Comments
|
The code runs fine on and
|
|
Similar issue on #312 |
|
Thanks for checking this out Chris. I've edited my previous comment to include the full output of print_diagnostics. |
|
I've tried pytorch==1.7.1 to match your setup closely, but could not reproduce the error on It is very strange that there are two issues on this, but I could not reproduce this on all systems that I have. Do you have a different machine or could try a docker image nvcr.io/nvidia/pytorch:20.10-py3 to see if it works? |
|
I tried in another machine and I am getting the same error. Setup in this new machine is similar to the first one though: On the docker image you suggest, the code runs fine with no error. |
|
Can you try the latest MinkowskiEngine with debug flag? And then, can you please post the entire error message when you run the above code? |
|
Sure, this was the full output when installing from source with debug flag: |
|
Thanks for the quick reply. The log says Which says that the cusparse_handle is invalid. I created a branch |
|
Hi Chris, I'm also experiencing this issue after upgrading to the newest version (CPU version works fine). But I am able to get rid of this cusparse error using your Here is my environment: |
|
Same here, the test script runs fine for me using the |
|
Merged to the master. |
AlexeyGB
pushed a commit
to AlexeyGB/MinkowskiEngine
that referenced
this issue
Feb 23, 2021
…IDIA#308) (NVIDIA#315) * force initialize cusparse handle * replace all at::cuda::getCurrentCUDASparseHandle with custom func * change log
Tanazzah
pushed a commit
to Tanazzah/MinkowskiEngine
that referenced
this issue
Feb 9, 2024
…IDIA#308) (NVIDIA#315) * force initialize cusparse handle * replace all at::cuda::getCurrentCUDASparseHandle with custom func * change log
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I have found the following error when using features_at_coodinates method of a sparse tensor in gpu. This error does not happen when using cpu.
To Reproduce
Steps to reproduce the behavior.
Desktop (please complete the following information):
MinkowskiEngine.print_diagnostics())==========System==========
Linux-5.8.0-40-generic-x86_64-with-glibc2.29
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0]
==========Pytorch==========
1.7.1
torch.cuda.is_available(): True
==========NVIDIA-SMI==========
/usr/bin/nvidia-smi
Driver Version 460.32.03
CUDA Version 11.2
VBIOS Version 90.02.30.00.39
Image Version G001.0000.02.04
==========NVCC==========
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0
==========CC==========
/usr/bin/c++
c++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
==========MinkowskiEngine==========
0.5.0
MinkowskiEngine compiled with CUDA Support: True
NVCC version MinkowskiEngine is compiled: 11020
CUDART version MinkowskiEngine is compiled: 11020
The text was updated successfully, but these errors were encountered: