-
Notifications
You must be signed in to change notification settings - Fork 740
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make GILOnceCell
threadsafe
#4512
base: main
Are you sure you want to change the base?
Conversation
c4c5dc3
to
ddffef3
Compare
A few questions:
|
Ok, on deeper review, I believe there's a difference between this behavior and When This actually addresses both of my questions. However, I think it makes the documentation a bit misleading: the deadlock prevention here has nothing to do with |
Yes exactly, this change preserves the existing runtime semantics while removing reliance on the GIL. Agreed the documentation is now out of date, will correct that. The name is also unfortunate now that there is no reliance on the GIL. I think that multiple concurrent calls to A possible alternative is to disable |
Actually, I started writing Instead, I wonder if an extension trait to add a helper method to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In init
, there are two lines:
let value = f()?;
let _ = self.set(py, value);
You can get behavior that's closer to the GIL build by wrapping the two lines in a Py_BEGIN/END_CRITICAL_SECTION
. Basically, for some (but not all) implementations of f()
, the GIL ensured that f()
was only called once. A Py_BEGIN/END_CRITICAL_SECTION
should ensure those same f
's are called only once without the risk of introducing a deadlock.
This changes
GILOnceCell
to be thread-safe. I do this by adding astd::sync::Once
to theGILOnceCell
, which blocks multiple writers from concurrently writing to the GIL (this is almost exactly howstd::sync::OnceLock
works).This comes at making accesses to the
GILOnceCell
cost an atomic load. I do this on all builds for simplicity of our implementation, so there is a bit of slowdown on the non-free threaded builds, but I don't think this will be catastrophic. I also think it's better to make the performance characteristics the same for consistency.I considered using a Python critical section here instead of the
Once
, but I came to realise that was not necessary for the short-lived lock around the write.cc @ngoldbaum
(cc @colesbury)