[fsevents bug] Stuck at "source and transform nodes" / "createPagesStatefully" on MacOS

[ehowey]

Description

I am working on a theme, local build was working fine with no problems, and recently upgraded all the dependencies to Gatsby 2.14.0 and both gatsby develop and gatsby build hang at source and transform nodes in my local dev environment.

Interestingly it works and builds on Netlify. This would point to it being something on my system. I have deleted the node modules folders in the workspaces and the root workspace folder and done a fresh yarn command. I also deleted the yarn.lock and package.lock files...not sure if this would cause problems.

Steps to reproduce

Theme repo is here: Gatsby-Theme-Catalyst-Core

Starter repo is here: Gatsby-Starter-Catalyst-Core

I have this setup in a yarn workspace for development however the same issue occurs if you do a fresh install of the starter using gatsby new my-catalyst-starter-core https://github.com/ehowey/gatsby-starter-catalyst-core

Expected result

Build successfully

Actual result

yarn workspace v1.17.3
yarn run v1.17.3
$ gatsby develop
success open and validate gatsby-configs - 0.122 s
success load plugins - 1.964 s
success onPreInit - 0.073 s
success initialize cache - 0.056 s
success copy gatsby files - 0.242 s
success onPreBootstrap - 0.087 s
⠙ source and transform nodes

Environment

System:
OS: macOS High Sierra 10.13.6
CPU: (2) x64 Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 12.9.1 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.11.2 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 76.0.3809.100
Firefox: 67.0.2
Safari: 12.1.2
npmGlobalPackages:
gatsby-cli: 2.7.40

[ehowey]

Updated - I tested commenting out the gatsby-node.js file and the build progressed on one occasion, but then I tried again and doing this and it got stuck in the same spot again.

Is there any chance this is due to my ancient computer, 2010 Macbook Pro (upgraded to 8GB Ram and an SSD), hasn't stopped me yet but I know its days are limited. This is a hobby for me and I have young kids so the dollars haven't been there to spend on a new computer since this one has worked fine with the upgraded Ram and SSD.

[ehowey]

I have tried to revert back to gatsby 2.13.52 which was the last version I was stable on, I have also tried reverting to react 16.8.6.

Interestingly I had it build successfully one time when I reverted react to 16.8.6 but after that first time it hung up at the same spot which is really odd behaviour to me.

[ehowey]

I can also, rarely, for no reason I can discern get it to build fine. There does not seem to be a rhyme or reason for when this happens. I spent a couple of hours looking for patterns or errors that might be causing this but didn't find anything. I reviewed #6654 and tried some of the items in there to no avail.

[ehowey]

This is got to me one of the most odd bugs I have ever encountered because the behaviour seems to change without there being discernible change in the code. In some cases the build hangs at source and transform nodes(60% of time), in some cases it hangs at Create Pages Statefully(20%), in some cases it builds successfully (20%). I have had it display all three of these behaviours without changing a line of code.

I have replicated this behaviour in gatsby-starter-blog as well, which is really odd. Again it makes me think this is an issue on my end locally. Interestingly it only hangs at createPagesStatefully.

I am now starting to think that it has something to do with a library that got updated by homebrew automatically - which one I have no idea but it is the only thing I can think of which I have not tested.

Here is a list of the things I have tried so far:

• Changing node versions, tried 12, 10 and 8

• Reverting to a previous point in my commit history that was working - it still fails now.

• Commenting out plugins and areas of the gatsby-config

• Commenting out the contents of my gatsby-node.js

• Test it again on Netlify, still built successfully, which is what makes me think it must have something to do with my computer.

• Deleting the 4 pages in my src/pages directory and putting in a very basic index.mdx file

• Deleting all the node_module and lock files, reinstalling

• Restarting computer

• Creating a fresh yarn workspace with new clones of the theme/starter from Github.

• Testing gatsby-starter-blog, similar behaviour but it hangs on createPagesStatefully

[steinitz]

Hello,

I can do little but confirm that I'm seeing this issue on a number of gatsby starters. And, as you point out, it sometimes just decides to build or to stop building, hanging at either "source and transform nodes" or createPagesStatefully.

Quite frustrating and leading to trying the most outrageous attempts to fix it.

[georgiee]

I did not witness this problem but this sounds awkward and I would really like to know the reason for this behaviour on your side.

Preparation
You should try to use the node debugger to get to the root of the problem. A task in your package.json would look like this. If you are using VSCode you can enable "Auto Attach" to use the internal debugger together with this task (make sure you use the internal terminal for this purpose)

"start:debug": "node --inspect=127.0.0.1:9232 node_modules/.bin/gatsby develop",

Debugging will of course work with any IDE just make sure you attach your debugger correctly.

1. Variant: Minimal Reproductiion
You have mentioned createPagesStatefully as the source of the problem. If you are sure that's the source of the problem maybe you could create a very tiny project to reproduce it. Ditch all the starters, just use gatsby directly and use createPagesStatefully through gatsby-node.js with some examples that mimic the things your starters are doing. Then start gatsby and debug it through node inspect.

That way you might find where it hangs.

2. Alternative: Inside your project/starter
You could of course try to debug the problem inside your project with the same technique. But maybe you have multiple problems that are stacking and blur your view on the cause of problems. So I would always try to get a minimal reproduction before starting to debug.

Good luck.

[ehowey]

So...I ran into some weird behaviour with the lock files. Maybe someone who knows more could point me in the right direction. In the process of trying to get to a minimal working build I basically stripped down to a "hello world" Gatsby install and it still wasn't working which was really odd.

Where it got even more strange was that the actual gatsby-starter-hello-world does work and build fine on my computer. BUT...if I delete the lock file and get yarn to rebuild a new lock file then the build starts failing, hanging at source and transform nodes. I could get my stripped down and minimal implementation to build if I copied the lock file from "hello-world" and used this. So my current theory is that there is some kind of version problem in my lock files that is causing this issue but I am stuck here.

I also deleted all of my homebrew installs and reinstalled node, yarn, git, etc. Just to make sure it wasn't somekind of funny business there.

[rheinardkorf]

Thanks @ehowey for raising this.... I thought it was just me because it is quite intermittent (but happens more than 50% of the time). Did the same as you to try and debug the situation. When I hang on ⠙ source and transform nodes it usually means having to kill my terminal.

If I find something I'll let you know. And I'll watch this thread too.

[steinitz]

@georgiee - thank you for the --inspect info. I'm going to see if I can get node debugging working with WebStorm.

I also like your idea of a minimal reproduction. But that will take time while I understand Gatsby a little deeper.

Currently it's hanging on "source and transform nodes". Rarely, it makes it to createPagesStatefully and hangs there. Otherwise it builds normally.

[hbthen3rd]

I am currently facing the same problem after doing "yarn upgrade" to fix a vulnerable dependency. Here's my set up:

System:
OS: macOS 10.15
CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 12.8.1 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.10.3 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 68.0.2
Safari: 13.0
npmPackages:
gatsby: ^2.13.42 => 2.14.7
gatsby-background-image: ^0.8.3 => 0.8.6
gatsby-image: ^2.2.7 => 2.2.15
gatsby-plugin-manifest: ^2.2.4 => 2.2.10
gatsby-plugin-netlify: ^2.1.4 => 2.1.10
gatsby-plugin-netlify-cms: ^4.1.6 => 4.1.12
gatsby-plugin-offline: ^2.2.5 => 2.2.10
gatsby-plugin-react-helmet: ^3.1.2 => 3.1.5
gatsby-plugin-react-svg: ^2.1.1 => 2.1.2
gatsby-plugin-sass: ^2.1.3 => 2.1.12
gatsby-plugin-sharp: ^2.2.9 => 2.2.18
gatsby-plugin-typography: ^2.3.2 => 2.3.5
gatsby-plugin-webfonts: ^1.1.0 => 1.1.0
gatsby-remark-images: ^3.1.10 => 3.1.19
gatsby-remark-relative-images-v2: ^0.1.5 => 0.1.5
gatsby-remark-responsive-iframe: ^2.2.5 => 2.2.10
gatsby-source-filesystem: ^2.1.6 => 2.1.18
gatsby-transformer-remark: ^2.6.12 => 2.6.19
gatsby-transformer-sharp: ^2.2.5 => 2.2.12

[hbthen3rd]

I am currently facing the same problem after doing "yarn upgrade" to fix a vulnerable dependency. Here's my set up:

Update: I manage to fix things by restoring my old "yarn.lock" file.

[steinitz]

@hbthen3rd

Update: I manage to fix things by restoring my old "yarn.lock" file.

My experience is that the problem can go away for no clear reason only to return later. You may find the problem returning despite restoring yarn.lock. Keep us posted.

[ehowey]

Here is a minimal reproduction using gatsby-starter-hello-world:

https://github.com/ehowey/gatsby-test-lockfiles

The current lock file included in the repo is the one failing for me. I have also included copies in the repo of yarn-working.lock and yarn-notworking.lock. Hopefully that makes it easier for someone to troubleshoot.

My current environment:

System:
OS: macOS High Sierra 10.13.6
CPU: (2) x64 Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 10.16.3 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.10.3 - /usr/local/bin/npm
Languages:
Python: 2.7.10 - /usr/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 67.0.2
Safari: 12.1.2
npmPackages:
gatsby: ^2.13.73 => 2.15.0
npmGlobalPackages:
gatsby-cli: 2.7.40

[sharvit]

I am experiencing the same issue.

Some direction I have found:

1. Downgrading gatsby-plugin-sitemap from 2.2.9 to 2.2.6 solved the issue with yarn develop.

   • It is odd because gatsby-plugin-sitemap should not affect development build.

2. yarn build still doesn't work but when I remove gatsby-plugin-sitemap form my config, yarn build works again.

[ehowey]

@sharvit I can report that this did not work in my case. Glad it fixed it for you, I do think it has something ultimately to do with the lock files and some weird version issue deep in the bowels of the lock file.

I have managed to get back to a working build, most of the time, maybe 8/10 times by reverting to an old lock file and doing some copy and paste. I can now also CTRL+C to force quit the build if it is hanging which I couldn't do at one point in this process. I don't know what fixes it in the lock file but I think the clues are in the repository I posted above with two copies of a lock file, one that works and one that doesn't. This is an odd beast of a bug. Usually in computer land things reassuringly work or don't work.

[ehowey]

@steinitz Any luck on your end? I had what you mention happen. It seemed to get better, pretty good but not perfect and now is failing almost every time again. Pretty frustrating.

[steinitz]

@ehowey

As you might imagine, due to the on-again off-again nature of this problem, I'm hesitant to report. Here's a case in point:

After deleting the node_modules directory, deleting yarn.lock then running
npx yarn-upgrade-all
and
yarn install
all was well.

Then, just now, in response to your message, I tried
yarn start
Result: it hangs again at
source and transform nodes

I suppose there are two sensible things to do:

1. take @georgiee's advice to get Webstorm's debugging working with
   node --inspect and see if I can spot any obvious problems
2. set Gatsby aside for a while to see if some smart person solves the problem

[steinitz]

Update:

ctl-C'd the above stuck process (that didn't used to stop the stuck process which was doubly annoying).
Then yarn start got stuck on createPagesStatefully.
ctl-C'd again, another yarn start - stuck on source and transform nodes
ctl-C'd one more time (for the hell of it) - this time yarn start worked and it's running

Don't know what to make of that, but there it is.

[ehowey]

This is similar to what I am seeing, tonight it seems worse maybe building successfully 1/10 times or less. From a programming/coding perspective this is super odd behaviour. Anecdotally it seemed to be working better a few days ago at 2.15.1 then today at 2.15.6. I also wonder what commonalities we all share that is causing this bug to happen. Can you run the gatsby info --clipboard command and post it?

It obviously isn't super widespread otherwise there would be a flood of reports but it also isn't just me like I originally thought. We all seem to be using yarn as well, which is a requirement for me as I am working on themes in a yarn workspace. I am still able to reproduce this on the gatsby-starter-hello-world so I believe it is some dependency issue or conflict in the core gatsby files.

[steinitz]

@ehowey here's the gatsby info you requested

System:
OS: macOS 10.14.6
CPU: (4) x64 Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 12.9.1 - /usr/local/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.11.2 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 76.0.3809.132
Firefox: 68.0.1
Safari: 12.1.2
npmPackages:
gatsby: ^2.14.3 => 2.14.7
gatsby-image: ^2.2.14 => 2.2.15
gatsby-plugin-feed: ^2.3.9 => 2.3.9
gatsby-plugin-i18n: ^1.0.1 => 1.0.1
gatsby-plugin-less: ^3.0.2 => 3.0.4
gatsby-plugin-manifest: ^2.2.9 => 2.2.10
gatsby-plugin-offline: ^2.2.10 => 2.2.10
gatsby-plugin-react-helmet: ^3.1.5 => 3.1.5
gatsby-plugin-robots-txt: ^1.5.0 => 1.5.0
gatsby-plugin-sharp: ^2.2.18 => 2.2.18
gatsby-plugin-sitemap: ^2.2.9 => 2.2.9
gatsby-remark-images: ^3.1.19 => 3.1.19
gatsby-remark-prismjs: ^3.3.9 => 3.3.9
gatsby-source-filesystem: ^2.1.18 => 2.1.18
gatsby-transformer-remark: ^2.6.19 => 2.6.19
gatsby-transformer-sharp: ^2.2.12 => 2.2.12
npmGlobalPackages:
gatsby-cli: 2.7.40

[goblindegook]

I was having the same issue: site built flawlessly on Netlify, but hung on my development machine with both gatsby build and gatsby develop.

After playing around with package versions, I noticed that reverting gatsby-plugin-sitemap from version 2.2.10 to 2.2.9 fixed the problem for me. Oddly enough, some people here seem to have issues with 2.2.9, which could mean the problem resides somewhere else.

Edit: Spoke too soon, Gatsby still hangs at the "source and transform nodes" and "createPagesStatefully" steps, though much less frequently.

[steinitz]

@goblindegook this seems a common pattern with this particular issue. Because the issue comes and goes, seemingly with the weather, the time of day, time since boot up... you can believe something you've done has fixed it - only to have it recur.

[domLocalHeroes]

Experiencing this issue too, downgraded gatsby-plugin-sitemap to 2.2.6 and that seems to have fixed it

[namukang]

FWIW, I'm also experiencing this but use neither yarn nor gatsby-plugin-sitemap.

My gatsby info in case it helps:

  System:
    OS: macOS 10.14.6
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 12.9.1 - /usr/local/bin/node
    npm: 6.11.2 - /usr/local/bin/npm
  Languages:
    Python: 3.7.4 - /usr/local/opt/python/libexec/bin/python
  Browsers:
    Chrome: 76.0.3809.132
    Firefox: 68.0.1
    Safari: 12.1.2
  npmPackages:
    gatsby: ^2.15.1 => 2.15.1 
    gatsby-cli: ^2.7.41 => 2.7.41 
    gatsby-image: ^2.2.15 => 2.2.15 
    gatsby-plugin-catch-links: ^2.1.6 => 2.1.6 
    gatsby-plugin-google-tagmanager: ^2.1.7 => 2.1.7 
    gatsby-plugin-manifest: ^2.2.12 => 2.2.12 
    gatsby-plugin-offline: ^3.0.0 => 3.0.0 
    gatsby-plugin-react-helmet: ^3.1.5 => 3.1.5 
    gatsby-plugin-sass: ^2.1.12 => 2.1.12 
    gatsby-plugin-sharp: ^2.2.18 => 2.2.18 
    gatsby-remark-images: ^3.1.19 => 3.1.19 
    gatsby-source-filesystem: ^2.1.18 => 2.1.18 
    gatsby-transformer-remark: ^2.6.19 => 2.6.19 
    gatsby-transformer-sharp: ^2.2.12 => 2.2.12 
    gatsby-transformer-yaml: ^2.2.7 => 2.2.7 
  npmGlobalPackages:
    gatsby-cli: 2.7.41

[karlhorky]

For me, cleaning the cache with gatsby clean solved the issue.

[ehowey]

I am still doing some hunting, trying to figure this out. Still a problem for me. Does anyone know if this might relate to the switch from babel 7.0.0 -> 7.5.5. This switch happened around the time I started experiencing this bug and coincides with me beginning to see problems...I have been trying to get yarn resolutions to work in order to peg the babel versions at 7.0.0 but haven't had success with this yet.

[canvaspixels]

I'm seeing this too with 2.15.28 when I do gatsby build. Do i need to stop the gatsby develop in the other terminal? It's intermittently happening

[canvaspixels]

Happened again without the dev server running. I've got a simple blog from following the blog tutorial though.

[canvaspixels]

It seems to be almost every other run. I'm on a Mac btw

[RomanHotsiy]

@canvaspixels does resizing the terminal window unstuck your build? If so, please try this and let us know if that helped #17938 (comment)

[canvaspixels]

@RomanHotsiy that indeed sorts it! Thanks!

[Js-Brecht]

Hey everybody, the patched version of fsevents has been published. You should be able to simply delete your yarn.lock file, and run yarn again. Each of your dependencies should automatically pick up fsevents@2.1.1, which should resolve the issue.

[karlhorky]

Be careful - deleting your whole yarn.lock file can also cause other packages to be upgraded.

Another, more precise option if you're using yarn is to delete just the lines relating to fsevents in the yarn.lock and re-run yarn. This will cause only the affected package to be upgraded. So for example, I removed the following lines:

- 
- fsevents@^2.0.6:
-   version "2.0.7"
-   resolved "https://registry.yarnpkg.com/fsevents/-/fsevents-2.0.7.tgz#382c9b443c6cbac4c57187cdda23aa3bf1ccfc2a"
-   integrity sha512-a7YT0SV3RB+DjYcppwVDLtn13UQnmg0SWZS7ezZD0UjnLwXmy8Zm21GMVGLaFGimIqcvyMQaOJBrop8MyOp1kQ==

Another way of doing this is to use the resolutions feature of Yarn (https://yarnpkg.com/lang/en/docs/selective-version-resolutions/). By adding the following to your package.json and then running yarn, it will also upgrade transitive dependencies and update your yarn.lock file:

  "resolutions": {
    "fsevents": "^2.1.1"
  }

After you have done this and your lockfile is updated, you can remove the resolutions property from package.json again.

Edit: removed previous, incorrect version of instructions.

[Js-Brecht]

You can also run yarn upgrade chokidar@latest. That should rebuild the lock file with the correct version of fsevents, without touching anything else

[Js-Brecht]

Oh, wait. That's only if you have chokidar as a direct dependency 🙄 . Forgot. @karlhorky is right

[rubengmurray]

I'm using npm. Deleting node_modules, running npm i --save fsevents@2.1.1 and then npm i did the trick for me. Took 19s to startup and I'm using the gatsby-lumen-starter as a base.

-- Edit:

I actually finished what I was working on, pushed to Netlify and it was completely unable to build because of fsevents (error fsevents@2.1.1: The platform 'linux' is incompatible with this module).

I switched to yarn and had the same issue so I yarn removed fsevents entirely, and now local and netlify are both working...

[itdsntwork]

Got the same issue and could not track the reason. This happen to me both on mac and pc. Could try to relate this to internet speed, however this happened to me also when I was connected to high speed network. What seems to be working for me right now is to add this in my env

GATSBY_CONCURRENT_DOWNLOAD=5

and running using --v8-pool-size=128

[thomasthep]

In my case it seems to be my firewall prompt on osx. If I'm fast enough to allow or deny connections coming from outside, it'll succeed flawlessly. Problem is, the prompt dialog pops up for a split second and disapears as Gatsby continues and then fails hanging on one of the many steps mentioned above.
Options:

• disable firewall (not having that)
• whitelist Gatsby (not consistent across projects and upgrades)
• find a way to halt on prompt until user selects option
• default bind address to localhost (should not require prompt to accept incoming connections).

[Js-Brecht]

@thomasthep The default bind address for the dev server is localhost already. It doesn't even listen for external connections unless you tell it to use your out-facing IP address (or 0.0.0.0). And even then, it doesn't start the dev server until after the bootstrap / build are done. If it's the bootstrap that is causing the firewall prompt, then it would have more to do with what plugins you are using, because Gatsby does not reach out to the internet in its default state.

There shouldn't even be any connections "coming from outside", generally speaking; even when a plugin is collecting information from an external source, the connection originates from your local host, not from the external source, and that is usually accepted as an "established connection" by most firewalls, in my experience, since most firewalls are configured to accept any outgoing connections.

I could see this happening if your firewall is configured to block processes, rather than just connections. In that case, it would be Node that would need to be whitelisted.

Would need more information to really understand why that's happening to you. It would probably be more effective to open a new ticket, though. This one had to do with fsevents on MacOS causing issues, and that has been addressed, which is why it's closed now. Anything not related to fsevents really should have its own issue, to get the attention it deserves.

[jiv-e]

For the record this also can happen if you have your GraphQL server running in debug mode and it's stopped at a breakpoint.

[LpmRaven]

For the record: This started happening to me when I added gatsby-source-s3-image and the s3 bucket reached over 100 images. It goes through all 145 images at the source and transform nodes stage and then hangs there forever. It's still happening, I've tried the fixes above. Luckily, it works after 5-6 attempts so I am not blocked.

[saadnoor]

Weirdly resizing the terminal window worked for me

[calluna18]

For me, limiting concurrent downloads as described here seems to help.

I added the following line to my .env file. The default is 200.
GATSBY_CONCURRENT_DOWNLOAD=50

Not sure if it solves your problem but maybe it helps someone :)

[griessel-jc]

@rishabhaggarwal2 There's a similar issue that sounds like it might be what you're experiencing, where it will hang while retrieving data from an online source. Can you try using GATSBY_CONCURRENT_DOWNLOAD=10 gatsby develop?

Thank you, this fixed it for me. Since I was pulling a ton of content from a third party website it kept hanging on downloading the content. (97% - so close yet so far)

[navneet-kumar]

I'm using npm. Deleting node_modules, running npm i --save fsevents@2.1.1 and then npm i did the trick for me. Took 19s to startup and I'm using the gatsby-lumen-starter as a base.

-- Edit:

I actually finished what I was working on, pushed to Netlify and it was completely unable to build because of fsevents (error fsevents@2.1.1: The platform 'linux' is incompatible with this module).

I switched to yarn and had the same issue so I yarn removed fsevents entirely, and now local and netlify are both working...

I am having the same issue, it's building on fine on my mac and also on github runner which is a container running ubuntu-latest.

but when i am trying to deploy it on prod which is ubuntu 16.x LTS it stucks at this line -

$ npm ci
Killed    .........] \ reify:fsevents: timing reifyNode:node_modules/webpack-dev-server/node_modules/fsevents Completed in 1053ms

how can we remove the fsevents dependency completely ?

[theabdulmateen]

In my case I had been working with a custom graphql server which seemed to get "socket hang up" error, I discovered this when I was using netlify dev but due to proxy reasons I switched back to gatsby develop.

Restarting the custom server seemed to help.