Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ray: Core] Ray can hang when getting an ObjectRef from an unknown environment #28341

Open
pabloem opened this issue Sep 7, 2022 · 1 comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core fix-error-msg This issue has a bad error message that should be improved. observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling P1.5 Issues that will be fixed in a couple releases. It will be bumped once all P1s are cleared

Comments

@pabloem
Copy link
Contributor

pabloem commented Sep 7, 2022

What happened + What you expected to happen

In the following script, the last statement hangs:

ray.init()
myref = ray.put('mystringhere')
ray.shutdown()

ray.init()
ray.get(myref)   # This statement hangs

What I would expect is for it to eventually throw an exception.

Versions / Dependencies

I ran this on Ray 2.0 and a recent Linux Debian

Reproduction script

ray.init()
myref = ray.put('mystringhere')
ray.shutdown()

ray.init()
ray.get(myref)   # This statement hangs

Issue Severity

Medium: It is a significant difficulty but I can work around it.

@pabloem pabloem added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Sep 7, 2022
@scottsun94 scottsun94 added the observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling label Oct 13, 2022
@richardliaw richardliaw added the core Issues that should be addressed in Ray Core label Oct 29, 2022
@rkooo567
Copy link
Contributor

rkooo567 commented Nov 1, 2022

This is not supported. But I think we should raise a better error message

@rkooo567 rkooo567 added the fix-error-msg This issue has a bad error message that should be improved. label Nov 1, 2022
@hora-anyscale hora-anyscale added P1 Issue that should be fixed within a few weeks and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Dec 14, 2022
pabloem added a commit to pabloem/ray that referenced this issue Dec 15, 2022
Signed-off-by: Pablo E <pabloem@apache.org>

Fixes ray-project#28341. Generates an error instead of hanging on ray.get of unknown object

Signed-off-by: Pablo E <pabloem@apache.org>

format

Signed-off-by: Pablo E <pabloem@apache.org>

fix build error -Wunused-result

Signed-off-by: Pablo E <pabloem@apache.org>

Apply clang-format

Signed-off-by: Pablo E <pabloem@apache.org>

Fix jni import

fixup

Signed-off-by: Pablo E <pabloem@apache.org>
@rkooo567 rkooo567 added P1.5 Issues that will be fixed in a couple releases. It will be bumped once all P1s are cleared and removed P1 Issue that should be fixed within a few weeks labels Mar 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core fix-error-msg This issue has a bad error message that should be improved. observability Issues related to the Ray Dashboard, Logging, Metrics, Tracing, and/or Profiling P1.5 Issues that will be fixed in a couple releases. It will be bumped once all P1s are cleared
Projects
None yet
Development

No branches or pull requests

5 participants