You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Will raise the following for the test tests/models/mamba2/test_modeling_mamba2.py::Mamba2ModelTest::test_model_parallel_beam_search
E RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
Mitigation
Move the smallest tensors of that operation to the device of the largest. Will do that soon, but if anyone wants to jump on the issue happy to review as well.
The text was updated successfully, but these errors were encountered:
Observed issue
Found out when running multi-gpu slow tests in #33560 .
Line 479 exactly of the mamba2 modeling file
transformers/src/transformers/models/mamba2/modeling_mamba2.py
Lines 472 to 480 in 8efc06e
Will raise the following for the test
tests/models/mamba2/test_modeling_mamba2.py::Mamba2ModelTest::test_model_parallel_beam_search
E RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
Mitigation
Move the smallest tensors of that operation to the device of the largest. Will do that soon, but if anyone wants to jump on the issue happy to review as well.
The text was updated successfully, but these errors were encountered: