You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pickle protocol 4, introduced with Python 3.4, brings some nice enhancements. In particular, it turns out that the framing feature introduced in protocol 4 "with a potentially huge performance impact" really does have a big impact.
I'm using zodbpickle to gain access to pickle protocol 3 under Python 2 in RelStorage. Under Python 3, I use the builtin pickle and (under 3.4 and up) protocol 4.
In one test in RelStorage, protocol 4 takes about 5 seconds to handle an object, whereas protocol 3 takes about 30 seconds. (To workaround this in Python 2, I have to buffer a large object into memory; even using buffered IO from an SSD and with the data putatively in the buffer cache. Of course, this varies by system, but the slower the filesystem/stream the worse the problem).
It would be nice to have protocol 4 available to python 2.
I don't know how much work would be involved in the backport (and making sure that no_load stays supported). I personally don't have any concrete plans to do this work right now (maybe someday), but I thought I'd throw this feature request out there along with the motivating numbers in case it intrigues any one else.
The text was updated successfully, but these errors were encountered:
Pickle protocol 4, introduced with Python 3.4, brings some nice enhancements. In particular, it turns out that the framing feature introduced in protocol 4 "with a potentially huge performance impact" really does have a big impact.
I'm using zodbpickle to gain access to pickle protocol 3 under Python 2 in RelStorage. Under Python 3, I use the builtin pickle and (under 3.4 and up) protocol 4.
In one test in RelStorage, protocol 4 takes about 5 seconds to handle an object, whereas protocol 3 takes about 30 seconds. (To workaround this in Python 2, I have to buffer a large object into memory; even using buffered IO from an SSD and with the data putatively in the buffer cache. Of course, this varies by system, but the slower the filesystem/stream the worse the problem).
It would be nice to have protocol 4 available to python 2.
I don't know how much work would be involved in the backport (and making sure that no_load stays supported). I personally don't have any concrete plans to do this work right now (maybe someday), but I thought I'd throw this feature request out there along with the motivating numbers in case it intrigues any one else.
The text was updated successfully, but these errors were encountered: