Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Workload Clean: Garbage Collection Component #30266

Merged
merged 49 commits into from
Mar 23, 2023
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
5bb3924
Add basic template code for dotnet workload clean
nagilson Jan 31, 2023
0089ec5
Register workload clean onto the list of workload commands available
nagilson Feb 2, 2023
f1e5978
Add basic garbage collection functionality
nagilson Feb 2, 2023
48b4d88
Refactor MSI garbage collection into separate smaller pieces
nagilson Feb 3, 2023
ef02ba7
Attempt to uninstall all workloads of the feature band and report VS …
nagilson Feb 4, 2023
c9a5dae
Code cleanup, add a scenario where clean can fail gracefully if dotne…
nagilson Feb 6, 2023
fd98b41
Display workloads from ALL Vs versions
nagilson Feb 6, 2023
493c864
GC MSI WKLOADS for ALL Feature Bands to clean other things that may …
nagilson Feb 7, 2023
4c22bf8
Delete installation packs for file based (msi seems to already do thi…
nagilson Feb 7, 2023
385a312
Observe that bandRecords are paths instead of bands
nagilson Feb 7, 2023
5b969bc
Clean only feature bands <= to the current one
nagilson Feb 7, 2023
c479465
[WIP] Add test infra code for workload clean file based
nagilson Feb 7, 2023
63803db
GC only for things of lesser FB and improve tests
nagilson Feb 8, 2023
9f3da52
Improve code quality, resx string clarity, and make tests for workloa…
nagilson Feb 8, 2023
4a97981
Delete workload installation records for clean --all
nagilson Feb 9, 2023
ec8889f
Fix test that wrote workload installation record for the wrong featur…
nagilson Feb 9, 2023
29c27bc
Update comment regarding install record version in test to be correct
nagilson Feb 9, 2023
4da1e91
MSI Clean now correctly removes packs and wkload install records
nagilson Feb 10, 2023
8d04323
Merge branch 'nagilson-workload-clean' of https://github.com/nagilson…
nagilson Feb 10, 2023
d33acd0
Update xlf
nagilson Feb 10, 2023
b650f96
update comment in workload GC to be more accurate and precise
nagilson Feb 10, 2023
ae65d10
Dont uninstall msi workload installation records in clean
nagilson Feb 14, 2023
7f0d9cc
Merge remote-tracking branch 'primary/main' into nagilson-workload-clean
nagilson Feb 14, 2023
4f81400
Fix gathering only vs workloads from .net sdk cli
nagilson Feb 15, 2023
2ec7ba2
Merge branch 'nagilson-workload-clean' of https://github.com/nagilson…
nagilson Feb 15, 2023
3e837f2
Undo removal of sandbox testing mode
nagilson Feb 15, 2023
cc2c84d
Dont call vsworkloads api in source build
nagilson Feb 15, 2023
da51ea7
Fix clean all to clean install records for all bands
nagilson Feb 15, 2023
a29d753
Clean up only existing workload file based records and also mark thin…
nagilson Feb 16, 2023
58a4936
Fix comment type
nagilson Feb 16, 2023
8e4ce30
Workload --info dont use manifest path incorrectly to find msi instal…
nagilson Feb 16, 2023
d19b939
Add logging to msi workload installation record removals
nagilson Feb 16, 2023
37d85aa
wip report vs workloads for other bands
nagilson Feb 23, 2023
f3f8307
Workload clean properly displays all VS versions workloads
nagilson Feb 23, 2023
7719db4
Uncomment line redacted for testing and improve tests to work in helix
nagilson Feb 23, 2023
9c81792
Describe how to uninstall VS workloads in a more clear fashion
nagilson Mar 2, 2023
3d309d1
Better describe how to uninstall vs workloads for users
nagilson Mar 2, 2023
24ce61f
Respond to all PR feedback for workload clean, EXCEPT doing it for al…
nagilson Mar 3, 2023
8f54148
File based GC no longer checks for feature band lower than or eq, jus…
nagilson Mar 6, 2023
006694e
Update info messages per @baronfel to improve customer clarity on wh…
nagilson Mar 6, 2023
874d8f9
Delete the code to delete pack records because it was a no op.
nagilson Mar 6, 2023
8fdb31a
MSI install also cleans for all feature bands now
nagilson Mar 6, 2023
aa68a49
erase msi worklod install records for > feature bands
nagilson Mar 6, 2023
6ce728d
Update workload clean tests to account for file based cleaning all bands
nagilson Mar 6, 2023
4ce329e
Respond to PR feedback to make code from worklod clean... cleaner!
nagilson Mar 7, 2023
b359556
Merge remote-tracking branch 'upstream/main' into nagilson-workload-c…
nagilson Mar 7, 2023
6e97f90
Respond to feedback. Don't print vs workloads in filebased, and do it…
nagilson Mar 8, 2023
bfff2e3
Respond to pr feedback
nagilson Mar 23, 2023
70dff83
Merge and fix bug inside of workload command parser from another pr t…
nagilson Mar 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/Cli/dotnet/commands/InstallingWorkloadCommand.cs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,6 @@ protected async Task<List<WorkloadDownload>> GetDownloads(IEnumerable<WorkloadId

protected IEnumerable<WorkloadId> GetInstalledWorkloads(bool fromPreviousSdk)
{
//var currentFeatureBand = new SdkFeatureBand(_installedFeatureBand.ToString());
if (fromPreviousSdk)
{
var priorFeatureBands = _workloadInstaller.GetWorkloadInstallationRecordRepository().GetFeatureBandsWithInstallationRecords()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ private static Command ConstructCommand()
command.AddCommand(WorkloadUninstallCommandParser.GetCommand());
command.AddCommand(WorkloadRepairCommandParser.GetCommand());
command.AddCommand(WorkloadRestoreCommandParser.GetCommand());
command.AddCommand(WorkloadCleanCommandParser.GetCommand());
command.AddCommand(WorkloadElevateCommandParser.GetCommand());

command.SetHandler((parseResult) => ProcessArgs(parseResult));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ public InstalledWorkloadsCollection AddInstalledVsWorkloads(IEnumerable<Workload
#if !DOT_NET_BUILD_FROM_SOURCE
if (OperatingSystem.IsWindows())
{
VisualStudioWorkloads.GetInstalledWorkloads(WorkloadResolver, _currentSdkFeatureBand,
installedWorkloads);
VisualStudioWorkloads.GetInstalledWorkloads(WorkloadResolver, installedWorkloads);
}
#endif
return installedWorkloads;
Expand Down
132 changes: 132 additions & 0 deletions src/Cli/dotnet/commands/dotnet-workload/clean/LocalizableStrings.resx
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema

Version 2.0

The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.

Example:

... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
<value>[base64 mime encoded serialized .NET Framework object]</value>
</data>
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>

There are any number of "resheader" rows that contain simple
name/value pairs.

Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.

The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:

Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.

mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.

mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.

mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
<xsd:element name="root" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="metadata">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="name" use="required" type="xsd:string" />
<xsd:attribute name="type" type="xsd:string" />
<xsd:attribute name="mimetype" type="xsd:string" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="assembly">
<xsd:complexType>
<xsd:attribute name="alias" type="xsd:string" />
<xsd:attribute name="name" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="data">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="resheader">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" />
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<resheader name="resmimetype">
<value>text/microsoft-resx</value>
</resheader>
<resheader name="version">
<value>2.0</value>
</resheader>
<resheader name="reader">
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<data name="CleanAllOptionDescription" xml:space="preserve">
<value>Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</value>
</data>
<data name="CommandDescription" xml:space="preserve">
<value>Remove orphaned workload components that may be on the machine.</value>
nagilson marked this conversation as resolved.
Show resolved Hide resolved
</data>
<data name="VSWorkloadNotRemoved" xml:space="preserve">
<value>Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</value>
nagilson marked this conversation as resolved.
Show resolved Hide resolved
</data>
<data name="InvalidWorkloadProcessPath" xml:space="preserve">
<value>The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</value>
</data>
</root>
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// Copyright (c) .NET Foundation and contributors. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
using System;
using System.CommandLine;
using System.IO;
using Microsoft.Deployment.DotNet.Releases;
using Microsoft.DotNet.Cli;
using Microsoft.DotNet.Cli.NuGetPackageDownloader;
using Microsoft.DotNet.Cli.Utils;
using Microsoft.DotNet.Configurer;
using Microsoft.DotNet.Workloads.Workload.Install;
using Microsoft.DotNet.Workloads.Workload.List;
using Microsoft.NET.Sdk.WorkloadManifestReader;

#nullable enable

namespace Microsoft.DotNet.Workloads.Workload.Clean
{
internal class WorkloadCleanCommand : WorkloadCommandBase
{
private readonly bool _cleanAll;

private readonly ReleaseVersion _sdkVersion;
private readonly IInstaller _workloadInstaller;
private readonly IWorkloadResolver _workloadResolver;

public WorkloadCleanCommand(
ParseResult parseResult,
IReporter? reporter = null,
IWorkloadResolver? workloadResolver = null,
string? dotnetDir = null,
string? version = null,
string? userProfileDir = null
) : base(parseResult, reporter: reporter)
{
_cleanAll = parseResult.GetValue(WorkloadCleanCommandParser.CleanAllOption);

string? dotnetPath = dotnetDir ?? Path.GetDirectoryName(Environment.ProcessPath);
if (dotnetPath == null)
{
throw new GracefulException(String.Format(LocalizableStrings.InvalidWorkloadProcessPath, Environment.ProcessPath ?? "null"));
}

userProfileDir = userProfileDir ?? CliFolderPathCalculator.DotnetUserProfileFolderPath;

_sdkVersion = WorkloadOptionsExtensions.GetValidatedSdkVersion(parseResult.GetValue(WorkloadUninstallCommandParser.VersionOption), version, dotnetPath, userProfileDir, true);
var sdkFeatureBand = new SdkFeatureBand(_sdkVersion);

var workloadManifestProvider = new SdkDirectoryWorkloadManifestProvider(dotnetPath, _sdkVersion.ToString(), userProfileDir);
_workloadResolver = workloadResolver ?? WorkloadResolver.Create(workloadManifestProvider, dotnetPath, _sdkVersion.ToString(), userProfileDir);
_workloadInstaller = WorkloadInstallerFactory.GetWorkloadInstaller(Reporter, sdkFeatureBand, _workloadResolver, Verbosity, userProfileDir, VerifySignatures, PackageDownloader, dotnetPath);
}

public override int Execute()
{
ExecuteGarbageCollection();
return 0;
}

private void ExecuteGarbageCollection()
nagilson marked this conversation as resolved.
Show resolved Hide resolved
{
if (_cleanAll)
{
_workloadInstaller.GarbageCollectInstalledWorkloadPacks(cleanAllPacks: true);
}
else
{
_workloadInstaller.GarbageCollectInstalledWorkloadPacks();
}
nagilson marked this conversation as resolved.
Show resolved Hide resolved

#if !DOT_NET_BUILD_FROM_SOURCE
if (OperatingSystem.IsWindows())
{
InstalledWorkloadsCollection vsWorkloads = new();
VisualStudioWorkloads.GetInstalledWorkloads(_workloadResolver, vsWorkloads, _cleanAll ? null : new SdkFeatureBand(_sdkVersion));
foreach (var vsWorkload in vsWorkloads.AsEnumerable())
{
Reporter.WriteLine(AnsiColorExtensions.Yellow(string.Format(LocalizableStrings.VSWorkloadNotRemoved, vsWorkload.Key)));
}
}
#endif
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright (c) .NET Foundation and contributors. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.
using System.CommandLine;
using Microsoft.DotNet.Workloads.Workload.Clean;
using LocalizableStrings = Microsoft.DotNet.Workloads.Workload.Clean.LocalizableStrings;

namespace Microsoft.DotNet.Cli
{
internal static class WorkloadCleanCommandParser
{

public static readonly Option<bool> CleanAllOption = new Option<bool>("--all", LocalizableStrings.CleanAllOptionDescription);

private static readonly Command Command = ConstructCommand();

public static Command GetCommand()
{
return Command;
}

private static Command ConstructCommand()
{
Command command = new Command("clean", LocalizableStrings.CommandDescription);

command.AddOption(CleanAllOption);

command.SetHandler((parseResult) => new WorkloadCleanCommand(parseResult).Execute());

return command;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="utf-8"?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 xliff-core-1.2-transitional.xsd">
<file datatype="xml" source-language="en" target-language="cs" original="../LocalizableStrings.resx">
<body>
<trans-unit id="CleanAllOptionDescription">
<source>Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</source>
<target state="new">Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</target>
<note />
</trans-unit>
<trans-unit id="CommandDescription">
<source>Remove orphaned workload components that may be on the machine.</source>
<target state="new">Remove orphaned workload components that may be on the machine.</target>
<note />
</trans-unit>
<trans-unit id="InvalidWorkloadProcessPath">
<source>The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</source>
<target state="new">The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</target>
<note />
</trans-unit>
<trans-unit id="VSWorkloadNotRemoved">
<source>Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</source>
<target state="new">Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</target>
<note />
</trans-unit>
</body>
</file>
</xliff>
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="utf-8"?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 xliff-core-1.2-transitional.xsd">
<file datatype="xml" source-language="en" target-language="de" original="../LocalizableStrings.resx">
<body>
<trans-unit id="CleanAllOptionDescription">
<source>Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</source>
<target state="new">Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</target>
<note />
</trans-unit>
<trans-unit id="CommandDescription">
<source>Remove orphaned workload components that may be on the machine.</source>
<target state="new">Remove orphaned workload components that may be on the machine.</target>
<note />
</trans-unit>
<trans-unit id="InvalidWorkloadProcessPath">
<source>The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</source>
<target state="new">The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</target>
<note />
</trans-unit>
<trans-unit id="VSWorkloadNotRemoved">
<source>Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</source>
<target state="new">Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</target>
<note />
</trans-unit>
</body>
</file>
</xliff>
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="utf-8"?>
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2" xsi:schemaLocation="urn:oasis:names:tc:xliff:document:1.2 xliff-core-1.2-transitional.xsd">
<file datatype="xml" source-language="en" target-language="es" original="../LocalizableStrings.resx">
<body>
<trans-unit id="CleanAllOptionDescription">
<source>Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</source>
<target state="new">Clean every workload pack, including non-orphaned ones. In addition, clean SDK feature bands (versions) that are less than or equal to the current running SDK, as opposed to only the current version.</target>
<note />
</trans-unit>
<trans-unit id="CommandDescription">
<source>Remove orphaned workload components that may be on the machine.</source>
<target state="new">Remove orphaned workload components that may be on the machine.</target>
<note />
</trans-unit>
<trans-unit id="InvalidWorkloadProcessPath">
<source>The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</source>
<target state="new">The path '{0}' of the process is the root path of the drive, which is not allowed, or it is invalid/inaccessible. Please run dotnet in a valid and available path.</target>
<note />
</trans-unit>
<trans-unit id="VSWorkloadNotRemoved">
<source>Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</source>
<target state="new">Workload '{0}' was not removed because it is installed and managed by Visual Studio. Please uninstall this workload through Visual Studio to fully remove it.</target>
<note />
</trans-unit>
</body>
</file>
</xliff>
Loading