Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jarjar_runner.sh is unreliable on case-insensitive filesystems (Mac) #136

Open
thirtyseven opened this issue Jul 13, 2021 · 2 comments
Open
Labels
bug Something isn't working P4

Comments

@thirtyseven
Copy link

thirtyseven commented Jul 13, 2021

Issue

When jarjar_runner.sh processes a jar that contains multiple files that differ only in the case of their filename, they will be incorrectly flagged as duplicates when running on a case-insensitive filesystem like macOS's default APFS root volume.

BUILD.bazel:

jarjar_library(
        name="test_shaded",
        rules=[],
        jars=["test.jar"]
)

Steps to reproduce:

Download test.zip (just a zip with three empty files: test, TEST, and build-data.properties)

% mv test.zip test.jar
% bazel build :test_shaded
INFO: Analyzed target //:test_shaded (0 packages loaded, 0 targets configured).
INFO: Found 1 target...

ERROR: /Users/*snip*/BUILD.bazel:11:15: Action test_shaded.jar failed: (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Error: duplicate files in merged jar: test~
Target //:test_shaded failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.191s, Critical Path: 0.09s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

Possible solutions

One solution I have tested is to modify jarjar_runner.sh to extract the jars to a temporary sparse disk image created with a case-sensitive filesystem mounted to a tmp directory. However, I've found that this approach is "leaky", i.e. if you ctrl-C the bazel build before it finishes, the user will have a volume mounted that they then will have to manually unmount. I tried to solve this by adding a trap command to jarjar_runner.sh, however, it never seemed to get triggered, probably due to some subtlety of being called from process-wrapper.

Another solution would be to replace jarjar_runner.sh with a Python script that uses the stdlib zipfile library to interact with the jar files to avoid interacting with the native FS.

@ronshapiro
Copy link
Collaborator

I'm not sure what makes the most sense here without adding in a lot of complexity.

Could we create a separate subdirectory for each jar so that there won't be any conflicts?

@thirtyseven
Copy link
Author

Could we create a separate subdirectory for each jar so that there won't be any conflicts?

Not sure that would work, since the case I illustrated above only has one jar.

@amalloy amalloy added bug Something isn't working P4 labels Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P4
Projects
None yet
Development

No branches or pull requests

3 participants