Gatsby develop build fails when transforming images with sharp

[joesanchezjr]

Description

gatsby develop fails when transforming images with gatsby-transformer-sharp/gatsby-plugin-sharp with error:

VipsJpeg: Invalid SOS parameters for sequential JPEG
VipsJpeg: out of order read at line 0
VipsJpeg: Invalid SOS parameters for sequential JPEG

Likely that the error is related to incorrectly encoded Samsung JPEGs per this issue in the sharp repository. If the sharp plugins for Gatsby allowed the passing of an option for failOnError: false (as suggested in the issue), that might be enough to resolve this error. Or perhaps instead of that specific option, allowing the passing through of an object with any of the acceptable options as outlined in the sharp docs.

Steps to reproduce

This bug is very specific to sourcing and then transforming images taken on certain Samsung devices so reproducing might be difficult unfortunately. [Edit: see comment below]

Here are details on my setup:

I created a new Gatsby site with the using-gatsby-source-wordpress-experimental starter made by Tyler Barnes.

I set up the appropriate plugins.

I ran gatsby develop.

All images were sourced correctly and stored locally. After sourcing, images were being transformed and kept failing when reaching images photographed with a Samsung Galaxy S7.

Expected result

It was expected that all successfully sourced images would then be successfully transformed for use with Gatsby Image

Actual result

Images from Samsung devices failed transformation and subsequently ended the development build.

Notes on troubleshooting

When I was trying to solve this issue myself, I was hoping that either gatsby-plugin-sharp or gatsby-transformer-sharp would have the ability to pass through more options. I saw that the specific option I needed (and referenced in the sharp repo linked above) was not one of the ones I could pass, so I decided to directly manipulate the media that was causing an issue.

In my media library, it was a total of 5 images that needed to be re-encoded (by importing into Photoshop, then re-exported as a JPEG). This worked for my site, but it did not seem to be a scalable option. Unless this is fixed in the sharp plugins, I will need to do the same for my production site.

It's a very specific issue, but I'd rather not have a site with thousands of images crash on a development build because of a single image.

Also, I did a small non-comprehensive test by editing the gatsby-plugin-sharp/src/process-file.js file and changing line 83 to be: pipeline = sharp(file, { failOnError: false });. This also removed the error for my development build allowing it continue without crashing. The image itself was inaccessible by localFile.childImageSharp.fluid but I'd rather have that than not even being able to work on the site with gatsby develop, especially since I do receive a GraphQL error in my terminal and I'm not unaware of a broken image.

Environment

  System:
    OS: macOS 10.15.6
    CPU: (4) x64 Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 10.16.0 - ~/.nvm/versions/node/v10.16.0/bin/node
    Yarn: 1.22.5 - ~/.yarn/bin/yarn
    npm: 6.9.0 - ~/.nvm/versions/node/v10.16.0/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 85.0.4183.83
    Firefox: 79.0
    Safari: 13.1.2
  npmPackages:
    gatsby: ^2.24.7 => 2.24.53 
    gatsby-image: ^2.4.13 => 2.4.16 
    gatsby-plugin-chakra-ui: ^0.1.4 => 0.1.4 
    gatsby-plugin-netlify-cache: ^1.2.0 => 1.2.0 
    gatsby-plugin-react-svg: ^3.0.0 => 3.0.0 
    gatsby-plugin-sass: ^2.3.12 => 2.3.12 
    gatsby-plugin-sharp: ^2.6.19 => 2.6.31 
    gatsby-source-filesystem: ^2.3.7 => 2.3.27 
    gatsby-source-wordpress-experimental: ^1.3.8 => 1.3.8 
    gatsby-transformer-sharp: ^2.5.14 => 2.5.14 
    gatsby-wordpress-experimental-inline-images: ^0.0.3 => 0.0.3 
  npmGlobalPackages:
    gatsby-cli: 2.12.93

[joesanchezjr]

Update: I created a repository to reproduce this bug locally or if preferred, a sandbox

[sandren]

It seems a lot of websites have been experiencing build failures after the gatsby-plugin-sharp and gatsby-transformer-sharp plugins upgraded their sharp dependency from 0.25.1 to 0.25.4.

Try specifying specific versions of these two plugins, namely "gatsby-plugin-sharp": "2.6.23" and "gatsby-transformer-sharp": "2.5.2", in your package.json file and reinstalling your dependencies. It should force a downgrade to the last known good version of sharp.

If it resolves your build failure, then we've discovered another place where the sharp dependency upgrade is causing issues.

[joesanchezjr]

@sandren I downgraded my dependencies to what you suggested, but the result was the same, so it seems like this issue is unrelated to the issues happening on the latest version of the plugins.

[vladar]

I could reproduce this with your example, thanks! 👍 And can confirm that changing this line from:

gatsby/packages/gatsby-plugin-sharp/src/process-file.js

Line 64 in 9e49435

pipeline = sharp(file)

To:

pipeline = sharp(file, { failOnError: false })

Indeed makes the build succeed. Instead of exiting the process sharp throws this error which is then displayed in the console as GraphQL error. Also, the fluid piece of the data is replaced with null.

@wardpeet Do you see any pitfalls if we add this change?

P.S. One missing step for the repro is to actually run a GraphQL query containing sharp transforms, e.g.:

{
  allImageSharp {
    nodes {
      id
      parent {
        ... on File {
          absolutePath
        }
      }
      fluid {
        base64
      }
    }
  }
}

[joesanchezjr]

Any update on this? I would like to see a solution merged in soon. I'd be happy to open up a PR with that single line change if it's all that is needed. I'm not too familiar with other parts of the plugin, so I'm unsure if it would affect anything else.

[vladar]

We are happy to accept a PR! But this will be a breaking change (as now the build that was failing previously can pass unexpectedly). So it will require a major bump for the plugin.

Edit: we can expose failOnError as a plugin option instead to keep it backward compatible and avoid the major bump. Will it work for you?

[joesanchezjr]

Great! I'll take another look at the contributing docs on the Gatsby site and open up a PR. And yes, I believe that exposing it as a plugin option instead is a great idea!