-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(gatsby-plugin-sharp): strip non related args when hashing resizing args #12129
Conversation
Updated metrics:On https://github.com/baobabKoodaa/blog/tree/temp384Without the changes 438 images. 66 duplicate images On using remarkwithout the changes 70 images. no duplicates I probably hadn't cleaned public before. |
Minor update: Each run is not deterministic on the number of images. Probably related to #11837 |
Another update:
|
f94bef0
to
73e7c93
Compare
I forced-push this PR rebasing to currnet master. i know jest tests will fail due to not updating snapshots. I am waiting for feedback and will then do it. |
some refactoring for stripping sharp args
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @oorestisime
So continuing on the work to make gatsby-plugin-sharp friendly on third-party packages (oorestisime/gatsby-remark-rehype-images#2) we now face this issue #12080
TL;DR images generated from transformer-sharp or and other plugins are the same (same sha256) but with different hash (depends on the available options). The problem is that all the plugins leak their options into the hash generation and even ones that are unrelated to the resizing processing (base64, generateTraceSVG, traceSVG etc)
I've been discussing this with @pieh for the last 2-3 hours :D
Proposed solution was to sanitize the args based on a whitelist, which is implemented on the PR. Which leads to the first question:
By doing so it can also potentially help lazy image processing @wardpeet by getting rid of unrelated options when persisting jobs.
Anyway current state of the PR is that it kind of works, i ve tested this in websites using my plugin and other that not. The issue here is that there are some differences in the number of images created when this modification is on or not.
I also verified that public folder doesn't contain duplicate images:
find public -regex ".*\.\(jpg\|webp\|png\|jpeg\)" -type f -exec sha256sum {} \; | cut -d ' ' -f 1 |sort| uniq -d | wc -l
This needs to be a bit more tested (and fix update snapshots etc)