shinyFiles re-indexes folder contents slowing down performance.

[PhilipFaure]

Hi,

We are using your shinyFiles package extensively within an app we are developing. We really like shinyFiles, but we have one issue for which we can not find a work around. Often, users will be working with large hard drives (e.g. 4 Tb drives) and some folders on it may easily contain well over 200,000 images. With such folders, if an user clicks on or tries to open one of these folders using shinyFiles it dramatically slows down. The app we are developing is run using a containerised instance.

My question is whether there is a way to stop shinyFiles from indexing all the files in the content viewer panel, or some other way to speed it up? Perhaps can the viewer be turned off somehow to speed up performance, would that work?

I’ve looked extensively on stackoverflow and other sites for anyone else reporting a similar issue, but with no success.

Thank you in advance for your time,

Best Wishes,
Philip

[vnijs]

I assume you are talking explicitly about the shinDirChoose? There is currently no way to turn off the preview. I agree that it might make sense to try and limit the size of the preview.

[pantheracorp]

@vnijs @PhilipFaure – we are having the same issue for shinyDirChoose. Is there any plan to allow users to limit these previews, @vnijs? That may really help! Thanks

[vnijs]

I could add a max_files argument to shinyDirChoose. If you set that to, e.g., 1000 you wouldn't only see the first 1000 entries from fs::dir_ls. Would that work for you?

[pantheracorp]

Hi @vnijs, that would work, assuming it'll speed up the process? Thank you!

[vnijs]

Try the below and let me know if you see any issues:

remotes::install_github("thomasp85/shinyFiles@max_files")

[pantheracorp]

The recent update unfortunately increases the computation time, it's taking even longer (~ 50% longer) to drill down into directories when max_files = 10, compared to the default. Just for clarity, when using shinyDirChoose, is the function indexing or caching the entire folder contents? Just trying to get an idea of why there is such a delay. Many thanks

[vnijs]

I see no reason why the changes in the commit linked below could increase computation time. I'm afraid you will have to dig into the fileGetter function to explore what might address you specific situation

680c1c2

[sneumann]

Hi, just weighing in here, indeed for e.g. NFS mounted home directories things are painfully slow.
I'll happily trade speed for eye-candy.
Yours, Steffen