Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many open files / UnknownContentCollectionError: site does not build #7073

Closed
1 task
fviolette opened this issue May 12, 2023 · 26 comments · Fixed by #7125
Closed
1 task

Too many open files / UnknownContentCollectionError: site does not build #7073

fviolette opened this issue May 12, 2023 · 26 comments · Fixed by #7125
Assignees
Labels
- P4: important Violate documented behavior or significantly impacts performance (priority)

Comments

@fviolette
Copy link
Contributor

fviolette commented May 12, 2023

What version of astro are you using?

2.4.5

Are you using an SSR adapter? If so, which one?

None

What package manager are you using?

pnpm

What operating system are you using?

Mac

What browser are you using?

Chrome

Describe the Bug

Site does not build using Astro 2.4.5 (with the latest MDX integration).
The issue seems to start with Astro 2.4.4.

I've used the docs site as an example.
My private site has the same issue.

EMFILE: too many open files, open '/home/projects/github-xwpzeh/src/pages/[lang]/[...slug].astro'

UnknownContentCollectionError: [object Object]
    at Module.getEntrySlug (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/astro@2.4.5_@types+node@18.6.4_sass@1.54.3/node_modules/astro/dist/content/utils.js:298:11)
    at async eval (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/astro@2.4.5_@types+node@18.6.4_sass@1.54.3/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:99:20)
    at async Promise.all (index 103)
    at async getStringifiedLookupMap (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/astro@2.4.5_@types+node@18.6.4_sass@1.54.3/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:90:3)
    at async Context.load (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/astro@2.4.5_@types+node@18.6.4_sass@1.54.3/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:52:36)
    at async Object.load (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/vite@4.3.2_@types+node@18.6.4_sass@1.54.3/node_modules/vite/dist/node/chunks/dep-7efa13d7.js:42887:32)
    at async loadAndTransform (file://file:///home/projects/github-xwpzeh/node_modules/.pnpm/vite@4.3.2_@types+node@18.6.4_sass@1.54.3/node_modules/vite/dist/node/chunks/dep-7efa13d7.js:53326:24)

The troubleshooting link provided is https://docs.astro.build/en/reference/errors/unknown-content-collection-error/

Link to Minimal Reproducible Example

https://stackblitz.com/edit/github-xwpzeh?file=package.json

Participation

  • I am willing to submit a pull request for this issue.
@ezzle

This comment was marked as outdated.

@ChrisChinchilla
Copy link

Also same for me, and the site used to build/serve. Same problem locally on macOS and using Netlify. I have a reasonable amount of content, but it used to work.

@ezzle
Copy link

ezzle commented May 16, 2023

More detailed error message :
`node:fs:601
handleErrorFromBinding(ctx);
^

Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\node_modules\vscode-oniguruma\release\onig.wasm'
at Object.openSync (node:fs:601:3)
at Object.readFileSync (node:fs:469:35)
at getOniguruma (D:\Workspace\astro-esse\node_modules\astro\node_modules\shiki\dist\index.js:1926:26)
at getHighlighter (D:\Workspace\astro-esse\node_modules\astro\node_modules\shiki\dist\index.js:2774:34)
at getViteErrorPayload (file:///D:/Workspace/astro-esse/node_modules/astro/dist/core/errors/dev/vite.js:92:29)
at Timeout._onTimeout (file:///D:/Workspace/astro-esse/node_modules/astro/dist/vite-plugin-astro-server/response.js:18:67)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7) {
}

Node.js v18.15.0`

@infovore
Copy link

Yep, same here. This bug comes in around 2.4.4 - 2.4.3 is entirely fine.

Simple reproduction: make a new Astro project with npm create, use the basic blog template, throw a few hundred posts into src/content/blog, and watch something have a meltdown. It looks like it might be related to Shiki.

@ChrisChinchilla
Copy link

I'm slowly digging into this… I seem to have things sort of working for serve now, but not yet for build. Will summarise my findings when done. Even working, the processes are much slower than before.

@infovore
Copy link

@ChrisChinchilla looking at the error messages and changes between 2.4.3 and 2.4.4, my gut says that #6916 might be the source.

@bluwy
Copy link
Member

bluwy commented May 16, 2023

Looks like my suggestion here did cause issues 😬 Perhaps making this part serial or using a promise queue would fix it.

await Promise.all(
contentGlob
// Ignore underscore files in lookup map
.filter((e) => !hasUnderscoreBelowContentDirectoryPath(pathToFileURL(e), contentDir))
.map(async (filePath) => {
const info = getEntryInfo({ contentDir, entry: filePath });
// Globbed entry outside a collection directory
// Log warning during type generation, safe to ignore in lookup map
if (info instanceof NoCollectionError) return;
const contentEntryType = contentEntryConfigByExt.get(extname(filePath));
if (!contentEntryType) return;
const { id, collection, slug: generatedSlug } = info;
const slug = await getEntrySlug({
id,
collection,
generatedSlug,
fs,
fileUrl: pathToFileURL(filePath),
contentEntryType,
});
filePathByLookupId[collection] = {
...filePathByLookupId[collection],
[slug]: rootRelativePath(root, filePath),
};
})
);

@bluwy bluwy added - P4: important Violate documented behavior or significantly impacts performance (priority) content-collections labels May 16, 2023
@ChrisChinchilla
Copy link

ChrisChinchilla commented May 16, 2023

Here's a brain dump of my notes and things I found that may or may not help others 🤷

Rogue .vscode folder

too many open files errors

https://facebook.github.io/watchman/docs/install#macos-file-descriptor-limits

Remove minor React component

Seemed to be posts folder, tried to debug which one

Renamed hash from imported medium posts

Change file casing to all lower case, remove extra special symbols

Too many updates to types at once? So try sere adding chunks of content at a time

That gets serve to run

Build still stuck and somewhat quiet about what's happening

Debug doesn't do much
Downgrade to Astro 2.4.4
icon and svgo issues
natemoo-re/astro-icon#93
natemoo-re/astro-icon#65

Then build works!

02:02:59 PM [build] Waiting for the @astrojs/image integration...
02:03:01 PM [@astrojs/image] "~/assets/images/default.png" image could not be fetched
Completed in 12.81s.

Completed in 61.75s.

@astrojs/sitemap: `sitemap-index.xml` is created.

02:03:12 PM [build] Waiting for the astro-compress integration...
Successfully compressed a total of 4 CSS files for 1010 Bytes.
Error: Cannot compress file /Users/chrisward/Library/Mobile Documents/com~apple~CloudDocs/OSS projects/chris-chinchilla/new-site/dist/blog/2015/2015-01-26-review-of-triumph-of-the-nerds-and-kobold-guide-to-board-game-design/index.html!
Error: Cannot compress file /Users/chrisward/Library/Mobile Documents/com~apple~CloudDocs/OSS projects/chris-chinchilla/new-site/dist/blog/2015/2015-03-22-add-your-suggestions-to-what-came-next/index.html!
Successfully compressed a total of 976 HTML files for 4.62 MB.
02:03:47 PM [build] 978 page(s) built in 1641.25s
02:03:47 PM [build] Complete!

@ChrisChinchilla
Copy link

ChrisChinchilla commented May 16, 2023

Going to now see if that gets a Netlify build to work, no…

It gets very close and then times out :(

@ematipico ematipico assigned ematipico and unassigned ematipico May 16, 2023
@fviolette
Copy link
Contributor Author

Works with 2.5.0 on my end.

@ezzle
Copy link

ezzle commented May 19, 2023

Still not working for me

New error after npm install astro@latest

Logs
[vite] Error when evaluating SSR module D:\Workspace\astro-esse\src\content\config.ts: failed to import "astro:content"
|- UnknownContentCollectionError: [object Object]
    at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11) 
    at async file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:110:22
    at async Promise.all (index 8189)
    at async getStringifiedLookupMap (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:86:3)
    at async Context.load (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:47:38)
    at async Object.load (file:///D:/Workspace/astro-esse/node_modules/vite/dist/node/chunks/dep-934dbc7c.js:42856:32)
    at async loadAndTransform (file:///D:/Workspace/astro-esse/node_modules/vite/dist/node/chunks/dep-934dbc7c.js:53282:24)

 error   [object Object]
  File:
    D:\Workspace\astro-esse\node_modules\astro\dist\content\utils.js:325:11
  Stacktrace:
UnknownContentCollectionError: [object Object]
    at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11) 
    at async handleEvent (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:194:27)
    at async runEvents (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:259:24)
    at async Object.init (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:61:5)
    at async attachListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:46:5)
    at async attachContentServerListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:27:5)
    at async dev (file:///D:/Workspace/astro-esse/node_modules/astro/dist/core/dev/dev.js:63:3)       
    at async runCommand (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:151:7)  
    at async cli (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:209:5)

@ChrisChinchilla
Copy link

Same, I also forgot to note that I could only get anything to work after changing the open file limit on macOS, as I can’t do that on Netlify, there I had to temporarily remove a load of content to get it to work. Not ideal…

@bluwy
Copy link
Member

bluwy commented May 19, 2023

I've made an attempt to fix this at #7125. Appreciate if you can try it with npm install astro@next--content-serial to see if it works.

@infovore
Copy link

Still not working for me

Well, as far as I can see, there's nothing in 2.5.0 that alters what I feel the cause of the issue was.

@ezzle

This comment was marked as outdated.

@bluwy

This comment was marked as outdated.

@ezzle
Copy link

ezzle commented May 19, 2023

Build OK, but very slow

Logs
This is a  content-serial-20230519093503  prerelease build
    Feedback? https://astro.build/issues

20:01:13 [content] Watching src/content/ for changes`

But access to pages too too too much slow.

And finally an error when trying to get a page

error [object Object]
File:
D:\Workspace\astro-esse\node_modules\astro\dist\content\utils.js:325:11
Code:
324 | } catch (e) {
> 325 | throw new AstroError(AstroErrorData.UnknownContentCollectionError, { cause: e });
| ^
326 | }
327 | const { slug: frontmatterSlug } = await contentEntryType.getEntryInfo({
328 | fileUrl,
Stacktrace:
UnknownContentCollectionError: [object Object]
at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11)
at async handleEvent (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:194:27)
at async runEvents (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:259:24)
at async Object.init (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:61:5)
at async attachListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:46:5)
at async attachContentServerListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:27:5)
at async dev (file:///D:/Workspace/astro-esse/node_modules/astro/dist/core/dev/dev.js:63:3)
at async runCommand (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:151:7)
at async cli (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:209:5)

</details>

@ezzle
Copy link

ezzle commented May 20, 2023

Build OK, but very slow

How Big is my project :

*.xml           238 MB            2/2 files     100%
*.mdx           129 MB  101539/101539 files     100%
*.png           831 KB          10/10 files     100%
*.md            448 KB          75/75 files     100%
*.ts            448 KB        105/105 files     100%

@ChrisChinchilla
Copy link

Will try later myself when back at a computer. Initially my project is about 8000 items of content, but even cutting it down to a fraction of that I an’t get it to build locally without upping the open file limit (on macOS anyway) and as stated, it all used to work totally fine and fairly speedily. Right now, building and serving is slow Lee than some old school SSGs like Jekyll.

@bluwy
Copy link
Member

bluwy commented May 22, 2023

Thanks for the feedback. I've cut another release for astro@next--content-serial using p-limit to run 10 at a time instead of serially, maybe that could fix it. If not, maybe we should revert the original PR.

Also, if you're using MDX, you can keep an eye on #7151 which should also improve things.

@ezzle
Copy link

ezzle commented May 22, 2023

Thanks for the new release. Unfortunately still not the green light

Logs
 astro  v0.0.0-content-serial-20230522100058 started in 613ms

  ┃ Local    http://localhost:3003/
  ┃ Network  use --host to expose

  ▶ This is a  content-serial-20230522100058  prerelease build
    Feedback? https://astro.build/issues

12:52:45 [content] Watching src/content/ for changes
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x2)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x3)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x4)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x5)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x6)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x7)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x8)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x9)
 error   EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx'
Error: EMFILE: too many open files, open 'D:\Workspace\astro-esse\src\content\essential\en\slot\realised_by_data_objects.mdx' (x10)
Sourcemap for "D:/Workspace/astro-esse/src/pages/essential/[...slug].astro" points to missing source files
 error   [object Object]
  File:
    D:\Workspace\astro-esse\node_modules\astro\dist\content\utils.js:325:11
  Stacktrace:
UnknownContentCollectionError: [object Object]
    at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11)        
    at async handleEvent (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:202:27)
    at async runEvents (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:267:24)
    at async Object.init (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:69:5)
    at async attachListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:46:5)
    at async attachContentServerListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:27:5)
    at async dev (file:///D:/Workspace/astro-esse/node_modules/astro/dist/core/dev/dev.js:63:3)
    at async runCommand (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:151:7)
    at async cli (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:209:5)

Best regards

Tshitshi

@bluwy
Copy link
Member

bluwy commented May 22, 2023

@ezzle which release had started causing issues for you? The original issue mentioned 2.4.4 which I'm pretty sure I've fixed the cause.

Given you have more than 100k files in src/content, I think you'll always be hitting the limit because Vite is also watching the files (see Vite's troubleshooting guide). However that link doesn't apply to Windows (and I'm not sure how to do the same on Windows), you can perhaps also try configuring vite.server.ignored in astro.config.js.

@ematipico
Copy link
Member

Given you have more than 100k files in src/content, I think you'll always be hitting the limit because Vite is also watching the files

When generating the content collections, should we turn off the watcher?

@ezzle
Copy link

ezzle commented May 22, 2023

@bluwy

Thank you for the support.

The issue in my case is related with the number of files in 'src/content'. It appears when >= +-4000K with any Astro version. I am very new in the Astro community, coming fron ApostropheCMS eco-system. I am hoping to achieve a more reactive & lean alternative with astro & alpine as base. Unfortunately, the capacity to serve efficiently >= 1000K files is mandatory. Do you think Astro can help meet that content volume requirement ?

The current astro version is 2.5.1. Previously I have been using the default version of the astro-doc project : ["astro": "^2.3.1"].

By the way uninstalling the patch is worst. I got the following error

npm run dev

> docs@0.0.7 dev
> astro dev

   astro  v2.5.1 started in 609ms

  ┃ Local    http://localhost:3003/
  ┃ Network  use --host to expose

14:44:37 [content] Watching src/content/ for changes
[object Object]
[object Object] (x2)
14:44:42 [vite] Error when evaluating SSR module D:\Workspace\astro-esse\src\content\config.ts: failed to import "astro:content"
|- UnknownContentCollectionError: [object Object]
    at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11)        
    at async file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:110:22
    at async Promise.all (index 8189)
    at async getStringifiedLookupMap (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:86:3)
    at async Context.load (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/vite-plugin-content-virtual-mod.js:47:38)
    at async Object.load (file:///D:/Workspace/astro-esse/node_modules/vite/dist/node/chunks/dep-4d3eff22.js:42892:32)
    at async loadAndTransform (file:///D:/Workspace/astro-esse/node_modules/vite/dist/node/chunks/dep-4d3eff22.js:53318:24)

 error   [object Object]
  File:
    D:\Workspace\astro-esse\node_modules\astro\dist\content\utils.js:325:11
  Stacktrace:
UnknownContentCollectionError: [object Object]
    at getEntrySlug (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/utils.js:325:11)        
    at async handleEvent (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:202:27)
    at async runEvents (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:267:24)
    at async Object.init (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/types-generator.js:69:5)
    at async attachListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:46:5)
    at async attachContentServerListeners (file:///D:/Workspace/astro-esse/node_modules/astro/dist/content/server-listeners.js:27:5)
    at async dev (file:///D:/Workspace/astro-esse/node_modules/astro/dist/core/dev/dev.js:63:3)
    at async runCommand (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:151:7)
    at async cli (file:///D:/Workspace/astro-esse/node_modules/astro/dist/cli/index.js:209:5)`

With the patch installed, the build terminates normally

` astro  v0.0.0-content-serial-20230522100058 started in 740ms

  ┃ Local    http://localhost:3003/
  ┃ Network  use --host to expose

  ▶ This is a  content-serial-20230522100058  prerelease build
    Feedback? https://astro.build/issues

14:46:13 [content] Watching src/content/ for changes
14:48:21 [content] Unsupported file types found. Prefix with an underscore (`_`) to ignore:
- essential-schema.ts
14:48:21 [content] Types generated
14:48:21 [astro] update D:/Workspace/astro-esse/.astro/types.d.ts

But installing the patch produce warning for conflicting dependencies version

npm install astro@next--content-serial
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: docs@0.0.7
npm WARN Found: astro@2.5.1
npm WARN node_modules/astro
npm WARN   peer astro@"^2.5.0" from @astrojs/markdown-remark@2.2.1
npm WARN   node_modules/@astrojs/mdx/node_modules/@astrojs/markdown-remark
npm WARN     @astrojs/markdown-remark@"^2.1.3" from @astrojs/mdx@0.18.4
npm WARN     node_modules/@astrojs/mdx
npm WARN   4 more (astro-auto-import, astro-og-canvas, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer astro@"^1.0.0 || ^2.0.0-beta" from astro-auto-import@0.2.1
npm WARN node_modules/astro-auto-import
npm WARN   dev astro-auto-import@"^0.2.1" from the root project
npm WARN ERESOLVE overriding peer dependency

Best regards

Tshitshi

@ezzle which release had started causing issues for you? The original issue mentioned 2.4.4 which I'm pretty sure I've fixed the cause.

Given you have more than 100k files in src/content, I think you'll always be hitting the limit because Vite is also watching the files (see Vite's troubleshooting guide). However that link doesn't apply to Windows (and I'm not sure how to do the same on Windows), you can perhaps also try configuring vite.server.ignored in astro.config.js.

@ChrisChinchilla
Copy link

@bluwy et all, the #7125 change fixed my issues. I get some dependency conflicts, but I guess that's expected and using --force gets things to work fine. Excellent, I look forward to a proper release!

@bluwy
Copy link
Member

bluwy commented May 23, 2023

When generating the content collections, should we turn off the watcher?

That makes sense. We're spinning up one-off Vite servers in some places and turning the watcher off could fix it. You can't shut it down entirely though since Vite always spins it up, but we can configure to ignore watching all files.

Do you think Astro can help meet that content volume requirement ?

We should be able to but seeing that many files is a first for me so maybe we're not doing something right at that scale. However, I think it's diverging from the original issue so I'd suggest opening another issue if you can find out which Astro version starts causing issues. The errors you get are because some other dependencies rely on the latest version of Astro, so you might need to downgrade the others too.

@bluwy et all, the #7125 change fixed my issues

Thanks for the update! I'll go ahead and get the fix out then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
- P4: important Violate documented behavior or significantly impacts performance (priority)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants