-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache all query carousels #9807
Cache all query carousels #9807
Conversation
e95b08e
to
551f73b
Compare
openlibrary/core/cache.py
Outdated
# strip [ and ] from key | ||
a = self.json_encode(list(args))[1:-1] | ||
|
||
if kw: | ||
return a + "-" + self.json_encode(kw) | ||
return f"{hash(a)}${hash(self.json_encode(kw))}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we'll want to use an md5 hash function here, apparently python's hash
is non-deterministic between sessions.
8ef616b
to
d542bb0
Compare
Avoids caching the book page related works carousel, which is fetched after the page loads. Also prevents the homepage "Classic Books" carousel from being cached, as the homepage itself is cached for the same amount of time.
Hey @jimchamp! These changes are looking great and I'm excited for them. I think the feedback about tightening up the keys to use as few hash digests as possible is a good one as, at the lookup scale we're operating, I think it could make a difference. Otherwise, this seems to be working great on testing and I'm looking forward to getting it merged. Thank you for leading this! |
436d8da
to
7c09923
Compare
I think this one is good to be squashed and merged, just testing now Just so everyone has confidence and clarity with the solution, the main changes are:
|
Closes #9058
This branch does the following:
QueryCarousel
macro template to, by default, request a cached copy of the carousel's markupargs
andkwargs
are hasheduse_cache
keyword argument to theQueryCarousel
macro, which allows us to avoid caching select query carouselTechnical
The "Classic Books" carousel is not cached, as it appears on our cached homepage. Also, both the homepage and our macros are cached for the same amount of time (five minutes).
The "Related Works" carousel is always fetched asynchronously, and therefore not cached.
Cache key protocol
Cache keys will now be in the following
$
delimited form:If there are no
kwargs
, the key will not include the second$
character and the hashedkwargs
:Examples:
Cache key containing hashed key word arguments:
RawQueryCarousel.en$4587276919764249206$-1969269941747307046
Cache key when no key word arguments are provided:
ia.get_metadata$6232568801773938868
Unit test changes
The
test_encode_args
unit test has been removed. I'm not sure how to best test the updatedencode_args
. The tests that were removed asserted thatencode_args
returned a specific expected result. Now that we are hashing the arguments, testing by string comparisons will fail if the hash algorithm is ever updated.Testing
Once deployed, load some of our
/collections
pages. Note how long the initial load for each page takes. Then, refresh the pages. Expect the load time to decrease noticeably.Screenshot
Stakeholders