Skip to content

TOML support #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,21 +46,32 @@ jobs:
$VERSION = "0.0.1-$COMMIT_HASH"
}

Write-Output "::set-output name=VERSION::$VERSION"
Write-Output "Version is $VERSION"

echo "VERSION=$VERSION" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8
echo "VERSION=$VERSION" >> $env:GITHUB_OUTPUT

- name: Build AggregateConfigBuildTask solution in Release mode
run: dotnet build src/AggregateConfigBuildTask.sln --configuration Release -warnaserror -p:Version=${{ steps.get_version.outputs.VERSION }}
run: dotnet build src/AggregateConfigBuildTask.sln --configuration Release -warnaserror -p:Version=${{ env.VERSION }}

- name: Run tests for AggregateConfigBuildTask solution
run: dotnet test src/AggregateConfigBuildTask.sln --configuration Release -warnaserror -p:Version=${{ steps.get_version.outputs.VERSION }} -p:CollectCoverage=true
id: run_tests
run: dotnet test src/AggregateConfigBuildTask.sln --configuration Release -warnaserror -p:Version=${{ env.VERSION }} -p:CollectCoverage=true

- name: Upload TestResults artifact
if: ${{ always() && steps.run_tests.conclusion != 'skipped' }}
uses: actions/upload-artifact@v4
with:
name: TestResults
path: src\UnitTests\TestResults

- name: Upload NuGetPackage artifact
uses: actions/upload-artifact@v4
with:
name: NuGetPackage
path: |
src/Task/bin/Release/AggregateConfigBuildTask.${{ steps.get_version.outputs.VERSION }}.nupkg
src/Task/bin/Release/AggregateConfigBuildTask.${{ steps.get_version.outputs.VERSION }}.snupkg
src/Task/bin/Release/AggregateConfigBuildTask.${{ env.VERSION }}.nupkg
src/Task/bin/Release/AggregateConfigBuildTask.${{ env.VERSION }}.snupkg

integration_tests:
needs: build
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ output*.json
_site/
docs/
src/.manifest
src/UnitTests/TestResults
.vscode
25 changes: 15 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
[![NuGet Version](https://img.shields.io/nuget/v/AggregateConfigBuildTask)](https://www.nuget.org/packages/AggregateConfigBuildTask) [![GitHub Build Status](https://img.shields.io/github/actions/workflow/status/richardsondev/AggregateConfigBuildTask/build.yml?branch=main
)](https://github.com/richardsondev/AggregateConfigBuildTask/actions/workflows/build.yml?query=branch%3Amain)

**AggregateConfigBuildTask** is a cross-platform MSBuild task that aggregates and transforms configuration files into more consumable formats like JSON, Azure ARM template parameters, YAML during the build process.
**AggregateConfigBuildTask** is a cross-platform MSBuild task that aggregates and transforms configuration files into more consumable formats like JSON, Azure ARM template parameters, YAML, and TOML during the build process.

## Features

- Merge multiple configuration files into a single output format (JSON, Azure ARM parameters, or YAML).
- Merge multiple configuration files into a single output format (JSON, Azure ARM parameters, YAML, or TOML).
- Support for injecting custom metadata (e.g., `ResourceGroup`, `Environment`) into the output.
- Optionally include the source file name in each configuration entry.
- Embed output files as resources in the assembly for easy inclusion in your project.
Expand Down Expand Up @@ -39,9 +39,9 @@ Alternatively, add the following line to your `.csproj` file:
| Parameter | Description | Supported Values | Default |
|----------|----------|----------|----------|
| **OutputFile**<br>*(Required)* | The file path to write output to. Should include the extension. | | |
| **OutputType**<br>*(Required)* | Specifies the format of the output file. | `Json`, `Arm`, `Yaml` | |
| **OutputType**<br>*(Required)* | Specifies the format of the output file. | `Json`, `Arm`, `Yaml`, `Toml` | |
| **InputDirectory**<br>*(Required)* | The directory containing the files that need to be aggregated. | | |
| **InputType** | Specifies the format of the input files. Refer to the [File Types](#file-types) table below for the corresponding file extensions that will be searched for. | `Json`, `Arm`, `Yaml` | `Yaml` |
| **InputType** | Specifies the format of the input files. Refer to the [File Types](#file-types) table below for the corresponding file extensions that will be searched for. | `Json`, `Arm`, `Yaml`, `Toml` | `Yaml` |
| **AddSourceProperty** | Adds a `source` property to each object in the output, specifying the filename from which the object originated. | `true`, `false` | `false` |
| **AdditionalProperties** | A set of custom top-level properties to include in the final output. Use `ItemGroup` syntax to define key-value pairs. See [below](#additional-properties) for usage details. | | |
| **IsQuietMode** | When true, only warning and error logs are generated by the task, suppressing standard informational output. | `true`, `false` | `false` |
Expand All @@ -54,6 +54,7 @@ The `InputDirectory` will be scanned for files based on the specified `InputType
|---------------|------------------------|
| `Json` | `.json` |
| `Arm` | `.json` |
| `Toml` | `.toml` |
| `Yaml` | `.yml`, `.yaml` |

## Usage
Expand Down Expand Up @@ -367,13 +368,17 @@ This project is licensed under the MIT License. See the [LICENSE](https://github

This project leverages the following third-party libraries that are bundled with the package:

- **[YamlDotNet](https://github.com/aaubry/YamlDotNet)**\
__Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Antoine Aubry and contributors__\
Used for YAML serialization and deserialization. YamlDotNet is distributed under the MIT License. For detailed information, refer to the [YamlDotNet License](https://github.com/aaubry/YamlDotNet/blob/master/LICENSE.txt).
- **[YamlDotNet](https://github.com/aaubry/YamlDotNet)**
_Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013, 2014 Antoine Aubry and contributors_
Used for YAML serialization and deserialization. YamlDotNet is distributed under the MIT License. For details, refer to the [YamlDotNet License](https://github.com/aaubry/YamlDotNet/blob/master/LICENSE.txt).

- **[YamlDotNet.System.Text.Json](https://github.com/IvanJosipovic/YamlDotNet.System.Text.Json)**\
__Copyright (c) 2022 Ivan Josipovic__\
Facilitates type handling for YAML serialization and deserialization, enhancing compatibility with System.Text.Json. This library is also distributed under the MIT License. For more details, see the [YamlDotNet.System.Text.Json License](https://github.com/IvanJosipovic/YamlDotNet.System.Text.Json/blob/main/LICENSE).
- **[YamlDotNet.System.Text.Json](https://github.com/IvanJosipovic/YamlDotNet.System.Text.Json)**
_Copyright (c) 2022 Ivan Josipovic_
Facilitates type handling for YAML serialization and deserialization with System.Text.Json. Distributed under the MIT License. For more details, see the [YamlDotNet.System.Text.Json License](https://github.com/IvanJosipovic/YamlDotNet.System.Text.Json/blob/main/LICENSE).

- **[Tommy](https://github.com/dezhidki/Tommy)**
_Copyright (c) 2020 Denis Zhidkikh_
Tommy provides support for working with TOML files in .NET applications. Distributed under the MIT License. See the full license at the [Tommy License](https://github.com/dezhidki/Tommy/blob/master/LICENSE).

## Contributing

Expand Down
29 changes: 15 additions & 14 deletions src/Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,31 @@
</PropertyGroup>
<!-- Build packages-->
<ItemGroup>
<PackageVersion Include="Microsoft.Build.Framework" Version="17.11.4" PrivateAssets="all" />
<PackageVersion Include="Microsoft.Build.Utilities.Core" Version="17.11.4" PrivateAssets="all" />
<PackageVersion Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="all" />
<PackageVersion Include="Microsoft.Build.Framework" Version="17.11.4" />
<PackageVersion Include="Microsoft.Build.Utilities.Core" Version="17.11.4" />
<PackageVersion Include="Microsoft.SourceLink.GitHub" Version="8.0.0" />
</ItemGroup>
<!-- Runtime packages -->
<ItemGroup>
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" PrivateAssets="all" />
<PackageVersion Include="System.Text.Json" Version="8.0.4" PrivateAssets="all" />
<PackageVersion Include="YamlDotNet" Version="16.1.0" PrivateAssets="all" GeneratePathProperty="true" />
<PackageVersion Include="YamlDotNet.System.Text.Json" Version="1.5.0" PrivateAssets="all" />
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" />
<PackageVersion Include="System.Text.Json" Version="8.0.4" />
<PackageVersion Include="Tommy" Version="3.1.2" />
<PackageVersion Include="YamlDotNet" Version="16.1.2" />
<PackageVersion Include="YamlDotNet.System.Text.Json" Version="1.5.0" />
</ItemGroup>
<!-- Test packages -->
<ItemGroup>
<PackageVersion Include="coverlet.collector" Version="6.0.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
<PackageVersion Include="coverlet.collector" Version="6.0.2" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageVersion Include="Moq" Version="4.20.72" />
<PackageVersion Include="MSTest.TestAdapter" Version="3.1.1" />
<PackageVersion Include="MSTest.TestFramework" Version="3.1.1" />
<PackageVersion Include="MSTest.TestAdapter" Version="3.6.0" />
<PackageVersion Include="MSTest.TestFramework" Version="3.6.0" />
</ItemGroup>
<!-- Global packages -->
<ItemGroup>
<GlobalPackageReference Include="AsyncFixer" Version="1.3.0" />
<GlobalPackageReference Include="AsyncFixer" Version="1.6.0" />
<GlobalPackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="8.0.0" />
<GlobalPackageReference Include="ReferenceTrimmer" Version="3.3.6" />
<GlobalPackageReference Include="Roslynator.Analyzers" Version="4.12.4" />
<GlobalPackageReference Include="Roslynator.Analyzers" Version="4.12.5" />
</ItemGroup>
</Project>
</Project>
54 changes: 30 additions & 24 deletions src/Task/AggregateConfig.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
using System.IO;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Threading.Tasks;
using AggregateConfigBuildTask.FileHandlers;
using Microsoft.Build.Framework;
using Task = Microsoft.Build.Utilities.Task;
Expand Down Expand Up @@ -109,30 +110,7 @@ public override bool Execute()
return false;
}

logger.LogMessage(MessageImportance.High, "Aggregating {0} to {1} in folder {2}", inputType, outputType, InputDirectory);

string directoryPath = Path.GetDirectoryName(OutputFile);
if (!fileSystem.DirectoryExists(directoryPath))
{
fileSystem.CreateDirectory(directoryPath);
}

var finalResult = ObjectManager.MergeFileObjects(InputDirectory, inputType, AddSourceProperty, fileSystem, logger).GetAwaiter().GetResult();

if (finalResult == null)
{
logger.LogError("No input was found! Check the input directory.");
return false;
}

var additionalPropertiesDictionary = JsonHelper.ParseAdditionalProperties(AdditionalProperties);
finalResult = ObjectManager.InjectAdditionalProperties(finalResult, additionalPropertiesDictionary, logger).GetAwaiter().GetResult();

var writer = FileHandlerFactory.GetFileHandlerForType(fileSystem, outputType);
writer.WriteOutput(finalResult, OutputFile);
logger.LogMessage(MessageImportance.High, "Wrote aggregated configuration file to {0}", OutputFile);

return true;
return Process(inputType, outputType).GetAwaiter().GetResult();
}
catch (Exception ex)
{
Expand All @@ -142,6 +120,34 @@ public override bool Execute()
}
}

private async Task<bool> Process(FileType inputType, FileType outputType)
{
logger.LogMessage(MessageImportance.High, "Aggregating {0} to {1} in folder {2}", inputType, outputType, InputDirectory);

string directoryPath = Path.GetDirectoryName(OutputFile);
if (!fileSystem.DirectoryExists(directoryPath))
{
fileSystem.CreateDirectory(directoryPath);
}

var finalResult = await ObjectManager.MergeFileObjects(InputDirectory, inputType, AddSourceProperty, fileSystem, logger).ConfigureAwait(false);

if (finalResult == null)
{
logger.LogError("No input was found! Check the input directory.");
return false;
}

var additionalPropertiesDictionary = JsonHelper.ParseAdditionalProperties(AdditionalProperties);
finalResult = await ObjectManager.InjectAdditionalProperties(finalResult, additionalPropertiesDictionary, logger).ConfigureAwait(false);

var writer = FileHandlerFactory.GetFileHandlerForType(fileSystem, outputType);
await writer.WriteOutput(finalResult, OutputFile).ConfigureAwait(false);
logger.LogMessage(MessageImportance.High, "Wrote aggregated configuration file to {0}", OutputFile);

return true;
}

private void EmitHeader()
{
var assembly = Assembly.GetExecutingAssembly();
Expand Down
2 changes: 2 additions & 0 deletions src/Task/AggregateConfigBuildTask.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,15 @@
<ItemGroup>
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" PrivateAssets="all" />
<PackageReference Include="System.Text.Json" PrivateAssets="all" />
<PackageReference Include="Tommy" PrivateAssets="all" />
<PackageReference Include="YamlDotNet" PrivateAssets="all" GeneratePathProperty="true" />
<PackageReference Include="YamlDotNet.System.Text.Json" PrivateAssets="all" />
</ItemGroup>

<ItemGroup>
<None Include="$(OutputPath)/Microsoft.Bcl.AsyncInterfaces.dll" Pack="true" PackagePath="/tasks/netstandard2.0/" />
<None Include="$(OutputPath)/System.Text.Json.dll" Pack="true" PackagePath="/tasks/netstandard2.0/" />
<None Include="$(OutputPath)/Tommy.dll" Pack="true" PackagePath="/tasks/netstandard2.0/" />
<None Include="$(OutputPath)/YamlDotNet.dll" Pack="true" PackagePath="/tasks/netstandard2.0/" />
<None Include="$(OutputPath)/YamlDotNet.System.Text.Json.dll" Pack="true" PackagePath="/tasks/netstandard2.0/" />
<None Include="build/AggregateConfigBuildTask.targets" Pack="true" PackagePath="/build/AggregateConfigBuildTask.targets" />
Expand Down
8 changes: 5 additions & 3 deletions src/Task/FileHandlers/ArmParametersFileHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@

namespace AggregateConfigBuildTask.FileHandlers
{
/// <inheritdoc/>
/// <summary>
/// Handles reading and writing ARM template parameter JSON files.
/// </summary>
public class ArmParametersFileHandler : IFileHandler
{
private readonly IFileSystem fileSystem;
Expand Down Expand Up @@ -51,7 +53,7 @@ public async ValueTask<JsonElement> ReadInput(string inputPath)
}

/// <inheritdoc/>
public void WriteOutput(JsonElement? mergedData, string outputPath)
public async Task WriteOutput(JsonElement? mergedData, string outputPath)
{
if (mergedData.HasValue && mergedData.Value.ValueKind == JsonValueKind.Object)
{
Expand All @@ -76,7 +78,7 @@ public void WriteOutput(JsonElement? mergedData, string outputPath)
["parameters"] = parameters
};
var jsonContent = JsonSerializer.Serialize(armTemplate, jsonOptions);
fileSystem.WriteAllText(outputPath, jsonContent);
await fileSystem.WriteAllTextAsync(outputPath, jsonContent).ConfigureAwait(false);
}
else
{
Expand Down
6 changes: 5 additions & 1 deletion src/Task/FileHandlers/FileHandlerFactory.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using System;
using System;
using System.Collections.Generic;

namespace AggregateConfigBuildTask.FileHandlers
Expand All @@ -25,6 +25,8 @@ internal static IFileHandler GetFileHandlerForType(IFileSystem fileSystem, FileT
return new YamlFileHandler(fileSystem);
case FileType.Arm:
return new ArmParametersFileHandler(fileSystem);
case FileType.Toml:
return new TomlFileHandler(fileSystem);
default:
throw new ArgumentException("Unsupported format");
}
Expand All @@ -46,6 +48,8 @@ internal static List<string> GetExpectedFileExtensions(FileType inputType)
return new List<string> { ".yml", ".yaml" };
case FileType.Arm:
return new List<string> { ".json" };
case FileType.Toml:
return new List<string> { ".toml" };
default:
throw new ArgumentException("Unsupported input type");
}
Expand Down
7 changes: 6 additions & 1 deletion src/Task/FileHandlers/FileTypeEnum.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
namespace AggregateConfigBuildTask
namespace AggregateConfigBuildTask
{
/// <summary>
/// Enum representing different file types supported for merging and processing.
Expand Down Expand Up @@ -29,5 +29,10 @@ public enum FileType
/// Alias for the <see cref="Yml"/> file type, for files with the .yaml extension.
/// </summary>
Yaml = Yml,

/// <summary>
/// Represents a TOML (Tom's Obvious, Minimal Language) file type.
/// </summary>
Toml = 3,
}
}
2 changes: 1 addition & 1 deletion src/Task/FileHandlers/IFileHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ public interface IFileHandler
/// </summary>
/// <param name="mergedData">The intermediate data in <see cref="JsonElement"/> format. Can be null.</param>
/// <param name="outputPath">The path to the output file where the data will be written.</param>
void WriteOutput(JsonElement? mergedData, string outputPath);
Task WriteOutput(JsonElement? mergedData, string outputPath);
}
}
8 changes: 5 additions & 3 deletions src/Task/FileHandlers/JsonFileHandler.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@

namespace AggregateConfigBuildTask.FileHandlers
{
/// <inheritdoc/>
/// <summary>
/// Handles reading and writing JSON files.
/// </summary>
public class JsonFileHandler : IFileHandler
{
readonly IFileSystem fileSystem;
Expand All @@ -25,10 +27,10 @@ public async ValueTask<JsonElement> ReadInput(string inputPath)
}

/// <inheritdoc/>
public void WriteOutput(JsonElement? mergedData, string outputPath)
public Task WriteOutput(JsonElement? mergedData, string outputPath)
{
var jsonContent = JsonSerializer.Serialize(mergedData, jsonOptions);
fileSystem.WriteAllText(outputPath, jsonContent);
return fileSystem.WriteAllTextAsync(outputPath, jsonContent);
}
}
}
Loading