Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dotnet workload install should normalize pack directory name to lower case #17438

Closed
grendello opened this issue Apr 15, 2021 · 17 comments
Closed
Assignees
Milestone

Comments

@grendello
Copy link

grendello commented Apr 15, 2021

dotnet workload install should normalize pack directory name to lower case

Original issue:

I discovered that even though the Xamarin.Android templates package (named Microsoft.Android.Templates.11.0.200-ci.main.216.nupkg) was installed in dotnet/template-packs, it wasn't considered by dotnet new. After running dotnet under strace, I found that it checks for the presence of the following packages in template-packs (apparently hardcoded?):

$ grep template-packs dotnet.log.381805 | grep lstat
lstat("dotnet/template-packs/microsoft.macos.templates.11.1.100-preview.3.1379.nupkg", 0x7f6864ce3350) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.macos.templates.11.1.100-preview.3.1379.nupkg", 0x7f6864ce33d0) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.android.templates.11.0.200-ci.main.216.nupkg", 0x7f6864ce3350) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.android.templates.11.0.200-ci.main.216.nupkg", 0x7f6864ce33d0) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.ios.templates.14.4.100-preview.3.1326.nupkg", 0x7f6864ce3350) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.ios.templates.14.4.100-preview.3.1326.nupkg", 0x7f6864ce33d0) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.maccatalyst.templates.14.3.100-preview.3.471.nupkg", 0x7f6864ce3350) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.maccatalyst.templates.14.3.100-preview.3.471.nupkg", 0x7f6864ce33d0) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.tvos.templates.14.3.100-preview.3.1379.nupkg", 0x7f6864ce3350) = -1 ENOENT (No such file or directory)
lstat("dotnet/template-packs/microsoft.tvos.templates.14.3.100-preview.3.1379.nupkg", 0x7f6864ce33d0) = -1 ENOENT (No such file or directory)

Copying our package to the dotnet/templates/ directory or renaming it to all lower case in dotnet/template-packs made dotnet new show the templates defined there.

I think for the best cross-platform support, dotnet new should perform a case-insensitive check for the presence of template packages.

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

pjcollins referenced this issue in pjcollins/android Apr 26, 2021
Context: https://github.com/dotnet/sdk/issues/16946

Copy our template workload pack to a lowercase destination so that it
can be detected and used when building and testing on Linux.
@marcpopMSFT marcpopMSFT transferred this issue from dotnet/sdk Apr 28, 2021
@wli3
Copy link

wli3 commented Apr 29, 2021

@DavidKarlas could you look into this? This impacts net6.0 workload work

@DavidKarlas
Copy link
Contributor

Sorry I didn’t start looking into this yet, but I plan this week so its ready for Preview 5.

@DavidKarlas
Copy link
Contributor

Root cause of problem is that we do NuGet PackageId.ToLower here: https://github.com/dotnet/sdk/blob/cb5b3ff/src/Resolvers/Microsoft.NET.Sdk.WorkloadManifestReader/WorkloadResolver.cs#L215

This should be fixed one of this two ways:
A) Stop doing ToLower(but what if original file has ".NUPKG" extension or missmatch between .json and .nupkg file name)
B) Do "path resolution"/"find" logic on case sensitive file systems this will happen in https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/commands/dotnet-new/OptionalWorkloadProvider.cs#L47

Since fix will be in SDK repo, transferring issue there, but keeping assignment on myself since I plan to fix this...

@mhutch since you wrote original .ToLower code, and since I assume you are familiar with .nupkg file names casing missmatches, what do you think is preferred way to fix this?

@DavidKarlas DavidKarlas transferred this issue from dotnet/templating May 5, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged Request triage from a team member label May 5, 2021
@mhutch
Copy link
Member

mhutch commented May 14, 2021

The ID in the on-disk path of the nupkg and extracted nupkg is expected to be lowercased to match what NuGet does (see ~/.nuget/.packages). This applies to all pack folders (packs, template-packs, library-packs) and OSes.

It's expected that whatever installs the pack files (e.g dotnet workload install) will do this transformation. Any pack that does not follow this has not been installed correctly.

@DavidKarlas
Copy link
Contributor

@dsplaisted @wli3 Is it OK if I close this issue? Do you want to open new one for dotnet workload install?

@wli3
Copy link

wli3 commented May 18, 2021

@DavidKarlas let me just rename this issue

@wli3 wli3 changed the title dotnet new should perform a case-insensitive search for templates dotnet workload install should normalize pack directory name to lower case May 18, 2021
@wli3 wli3 removed the untriaged Request triage from a team member label May 18, 2021
@wli3 wli3 added this to the 6.0.1xx milestone May 18, 2021
@mhutch
Copy link
Member

mhutch commented May 19, 2021

FWIW I'm not dead set on the lowercasing. If folks have strong opinions otherwise we could consider preserving original case. But either way there should be a consistent policy so that the casing of the paths returned from the manifest reader always matches the casing of the paths installed on disk.

@sfoslund
Copy link
Member

@grendello how did install the package to this directory, was it using the dotnet workload install command? I believe the current implementation takes this into account and uses the path provided by the resolver when laying down the packs. I'm not able to repro this with the workload install command.

@dsplaisted
Copy link
Member

Maybe it was maui-check?

@grendello
Copy link
Author

@grendello how did install the package to this directory, was it using the dotnet workload install command? I believe the current implementation takes this into account and uses the path provided by the resolver when laying down the packs. I'm not able to repro this with the workload install command.

It was installed by our internal code in Xamarin.Android. Until NET6 is released, we build and "install" packages with our runtime in the copy of the current dotnet6 version we're using (also installed by us). The nuget in question was created with CamelCase simply following the convention used for all of our packages.

@sfoslund
Copy link
Member

It sounds like the issue was in temporary code and shouldn't repro with the real workload installer then. Is there any action needed here?

@wli3
Copy link

wli3 commented May 24, 2021

If it is a temporary issue, we can close.

@wli3 wli3 closed this as completed May 24, 2021
@grendello
Copy link
Author

It sounds like the issue was in temporary code and shouldn't repro with the real workload installer then. Is there any action needed here?

I think there's a need to fix this. While I noticed it using temporary code, anyone can grab a template pack from the Internet and put it in the template-packs folder without changing its name case. That will result in the templates not being visible.

@sfoslund
Copy link
Member

anyone can grab a template pack from the Internet and put it in the template-packs folder without changing its name case

I imagine we don't want to encourage that scenario anyway.

@wli3
Copy link

wli3 commented May 25, 2021

Agree. That is not a supported scenario. Especially considering dotnet folder could be in program files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants