diff --git a/docs/Producing effective SARIF.md b/docs/Producing effective SARIF.md index fdc5742e9..17336142c 100644 --- a/docs/Producing effective SARIF.md +++ b/docs/Producing effective SARIF.md @@ -6,7 +6,7 @@ This document is for creators of static analysis tools who want to produce SARIF Teams can use SARIF log files in many ways. They can view the results in an IDE extension such as the [SARIF extension for VS Code](https://marketplace.visualstudio.com/items?itemName=MS-SarifVSCode.sarif-viewer) or the [SARIF Viewer VSIX for Visual Studio](https://marketplace.visualstudio.com/items?itemName=WDGIS.MicrosoftSarifViewer), or in a [web-based viewer](https://microsoft.github.io/sarif-web-component/). They can import it into a static analysis results database, or use it to drive automatic bug fiing. Most important, developers use the information in a SARIF log file to understand and fix the problems it reports. -Because of this variety of usage scenarios, a SARIF log file that is useful one scenario might not be useful in another. Ideally, static analysis tools will provide options to let their users specify the output that meets their needs. +Because of this variety of usage scenarios, a SARIF log file that is useful in one scenario might not be useful in another. Ideally, static analysis tools will provide options to let their users specify the output that meets their needs. The [SARIF specification](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html) defines dozens of objects with hundreds of properties. It can be hard to decide which ones are important (aside from the few that the spec says are mandatory). What information is most helpful to developers? What information should you include if you want to file useful bugs from the SARIF log? @@ -14,15 +14,52 @@ On top of all that, the spec is written in format language that's hard to read. The purpose of this document is to cut through the confusion and provide clear guidance on what information your tool should include in a SARIF file, and how to make that information as helpful and usable as possible. -### Structural requirements +## Principles for producing effective SARIF + +This document contains dozens of individual rules and guidelines for producing effective SARIF, but they all derive from a handful of bedrock principles. + +### Readability/Understandability/Actionability + +The most important elements of a SARIF log file are the "result messages". A result messages must be readable, understandable, and actionable. It must describe exactly what went wrong, why it's a problem, and how to fix it -- and it must do all of that in one compact, well-written plain-text paragraph. (You can supply Markdown messages as well, but the plain-text message is required because not every SARIF consumer can interpret Markdown.) + +### Fitness for purpose + +The SARIF format has many optional properties. Some of them depend on what kind of analysis tool you are writing. For example, a Web analyzer will probably emit `run.webRequests` and `.webResponses`; a crash dump analyzer might emit `run.addresses`. + +Other optional properties are more or less useful depending how end users or downstream systems plan to use the logs. For example: +- If you plan to use SARIF log files as the input to an automated bug filing system, you'll want to populate `result.partialFingerprints` to make it easier to determine which results are new in each run. +- If you plan to view the results in an environment where you don't have access to the source code (for example, in a Web-based SARIF viewer), you'll want to populate `physicalLocation.contextRegion` so that the viewer can display a few lines of code around the actual location of the result. You might even want to populate `run.artifacts[].contents`, which contains the entire contents of the artifact that was analyzed. +- If you plan to use SARIF files as an input to a compliance system, you might want to populate `run.tool.driver.rules` with the complete set of rules that were run, even if most of them didn't produce any results. Similarly, you might want to populate `run.artifacts` with the complete list of files that were analyzed, even if most of them didn't contain any results. + +### Compactness + +SARIF files from large projects can be huge: multiple gigabytes in size, with over a million results. Even though a great deal of work has been and is being done to compress SARIF files and make them faster to access, it's still important not to unnecessarily increase log file size. + +Some optional SARIF properties can take up alot of space, most notably `artifact.contents`. + +In some cases, SARIF can represent the same information in multiple places in the log file. For example, a `result` object can (and usually does) specify the result's location with a URI, but that same URI appears in the `run.artifacts` array. Deciding which duplicative information to include is a trade-off between file size on the one hand and what we might call "local readability" on the other. + +In short, both "fitness for purpose" and "compactness" are important values, they are in tension with each other, and so it's important for analysis tools to provide flexibility in which properties they emit. + +Having said that, the SARIF MultiTool can "enrich" SARIF files with additional properties after the fact (especially if the MultiTool has access to the source code). So one possible strategy is for a tool to emit a minimal SARIF file, and rely on consumers to enrich it as necessary to address specific usage scenarios. + +### Serviceability + +SARIF files are often used in scenarios where it's important to know which tool, and which _version_ of the tool, produced the results. Therefore it's important for your tool to populate `run.tool.driver` with enough information to identify your tool and its version. + +### What's next + +The remainder of this document will present a set of specific rules and guidelines, all of them aimed at producing SARIF that conforms to these principles. + +## Structural requirements Many of SARIF's structural requirements are expressed in a [JSON schema](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/schemas/sarif-schema-2.1.0.json), but the schema can't express all the structural requirements. In addition to providing helpful, useful information, it's important for tools to produce SARIF that meet all the structural requirements, even the ones that the schema can't express. -### The SARIF validation tool +## The SARIF validation tool -The SARIF validation tool (the "validator") helps ensure that a SARIF file conforms to SARIF's structural requirements as well as to the guidelines for producing high-quality SARIF output. +The SARIF validation tool (the "validator") helps ensure that a SARIF file conforms to SARIF's structural requirements as well as to the guidelines for producing effective SARIF output. -#### What the validator does +### What the validator does Here's how the validator process a SARIF log file: @@ -33,11 +70,11 @@ Here's how the validator process a SARIF log file: The analysis rules in Step 3 fall into two categories: 1. Rules that detect structural problems that the schema can't express. -2. Rules that enforce guidelines for producing high quality SARIF. +2. Rules that enforce guidelines for producing effective SARIF. -The validator is incomplete: it does not enforce every structural condition in the spec, nor every guideline for producing high quality SARIF. We hope to continue to add analysis rules in both those areas. +The validator is incomplete: it does not enforce every structural condition in the spec, nor every guideline for producing effective SARIF. We hope to continue to add analysis rules in both those areas. -#### Installing and using the validator +### Installing and using the validator To install the validator, run the command ``` @@ -52,13 +89,13 @@ The SARIF Multitool can do many things besides validate a SARIF file (that's why sarif ``` -### How this document is organized +## How this document is organized This document expresses each structural requirement and guideline as a validator analysis rule. At the time of this writing, not all of those rules actually exist. Those that do not are labled "(NYI)". First come the rules that detect serious violations of the SARIF spec (rules which the validator would report as `"error"`). They have numbers in the range 1000-1999, for example, `SARIF1001.RuleIdentifiersMustBeValid`. -Then come the rules that detect either less serious violations of the SARIF spec (rules which the validator would report as `"warning"` or `"note"`). They have numbers in the range 2000-2999, for example, `SARIF2001.AuthorHighQualityMessages`. +Then come the rules that detect either less serious violations of the SARIF spec (rules which the validator would report as `"warning"` or `"note"`), or guidance based on integrating SARIF into a wide variety of static analysis tools. They have numbers in the range 2000-2999, for example, `SARIF2005.ProvideToolProperties`. Each rule has a description that describes its purpose, followed by one or more messages that can appear in a SARIF result object that reports a violation of this rule. Each message includes one or more replacement sequences (`{0}`, `{1}`, _etc._). The first one (`{0}`) is always a JSON path expression that describes the location of the result. For example, `/runs/0/results/0/locations/0/physicalLocation` specifies the `physicalLocation` property of the first location of the first result in the first run in the log file. @@ -72,90 +109,125 @@ Rules that describe violations of **SHALL**/**SHALL NOT** requirements of the [S #### Description -The two identity-related properties of a SARIF rule must be consistent. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([§3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([§3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If both 'name' and 'id' are opaque identifiers, omit the 'name' property. If both 'name' and 'id' are human-readable identifiers, then consider assigning an opaque identifier to each rule, but in the meantime, omit the 'name' property. +The two identity-related properties of a SARIF rule must be consistent. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If both 'name' and 'id' are opaque identifiers, omit the 'name' property. If both 'name' and 'id' are human-readable identifiers, then consider assigning an opaque identifier to each rule, but in the meantime, omit the 'name' property. #### Messages ##### `Default`: error -{0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([§3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([§3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If they are identical, the tool must omit the 'name' property. +{0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If they are identical, the tool must omit the 'name' property. --- -### Rule`SARIF1002.UrisMustBeValid` +### Rule `SARIF1002.UrisMustBeValid` #### Description +Specify a valid URI reference for every URI-valued property. + +URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, 'file' URIs must not include '..' segments. If symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. + #### Messages ##### `UrisMustConformToRfc3986`: error +{0}: The string '{1}' is not a valid URI reference. URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). + ##### `FileUrisMustConformToRfc8089`: error ##### `FileUrisMustNotIncludeDotDotSegments`: error ---- -### Rule`SARIF1003.UrisShouldUseConventionalForm` +{0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. -##### `FileUrisWithoutHostNameShouldUseTripleSlashForm`: warning +### Rule `SARIF1004.ExpressUriBaseIdsCorrectly` ---- +#### Description -### Rule`SARIF1004.ExpressUriBaseIdsCorrectly` +When using the 'uriBaseId' property, obey the requirements in the SARIF specification [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) that enable it to fulfill its purpose of resolving relative references to absolute locations. In particular: -#### Description +If an 'artifactLocation' object has a 'uriBaseId' property, its 'uri' property must be a relative reference, because if 'uri' is an absolute URI then 'uriBaseId' serves no purpose. + +Every URI reference in 'originalUriBaseIds' must resolve to an absolute URI in the manner described in the SARIF specification [3.14.14](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317498). #### Messages ##### `UriBaseIdRequiresRelativeUri`: error +{0}: This 'artifactLocation' object has a 'uriBaseId' property '{1}', but its 'uri' property '{2}' is an absolute URI. Since the purpose of 'uriBaseId' is to resolve a relative reference to an absolute URI, it is not allowed when the 'uri' property is already an absolute URI. + ##### `TopLevelUriBaseIdMustBeAbsolute`: error +{0}: The '{1}' element of 'originalUriBaseIds' has no 'uriBaseId' property, but its 'uri' property '{2}' is not an absolute URI. According to the SARIF specification, every such "top-level" entry in 'originalUriBaseIds' must specify an absolute URI, because the purpose of 'originalUriBaseIds' is to enable the resolution of relative references to absolute URIs. + ##### `UriBaseIdValueMustEndWithSlash`: error +{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that does not end with a slash. The trailing slash is required to minimize the likelihood of an error when concatenating URI segments together. + ##### `UriBaseIdValueMustNotContainDotDotSegment`: error +{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. + ##### `UriBaseIdValueMustNotContainQueryOrFragment`: error +{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a query or a fragment. This is not valid because the purpose of the 'uriBaseId' property is to help resolve a relative reference to an absolute URI by concatenating the relative reference to the absolute base URI. This won't work if the base URI contains a query or a fragment. + --- -### Rule`SARIF1005.UriMustBeAbsolute` +### Rule `SARIF1005.UriMustBeAbsolute` #### Description +Certain URIs are required to be absolute. For the most part, these are URIs that refer to http addresses, such as work items or rule help topics. + #### Messages ##### `Default`: error +{0}: The value of this property is required to be an absolute URI, but '{1}' is a relative URI reference. + --- -### Rule`SARIF1006.InvocationPropertiesMustBeConsistent` +### Rule `SARIF1006.InvocationPropertiesMustBeConsistent` #### Description +The properties of an 'invocation' object must be consistent. + +If the 'invocation' object specifies both 'startTimeUtc' and 'endTimeUtc', then 'endTimeUtc' must not precede 'startTimeUtc'. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal. + #### Messages ##### `EndTimeMustNotPrecedeStartTime`: error +{0}: The 'endTimeUtc' value '{1}' precedes the 'startTimeUtc' value '{2}'. The properties of an 'invocation' object must be internally consistent. + --- -### Rule`SARIF1007.RegionPropertiesMustBeConsistent` +### Rule `SARIF1007.RegionPropertiesMustBeConsistent` #### Description -SARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: with line and column numbers, with a character offset and count, or with a byte offset and count. The specification states certain constraints on these properties, both within each property groups (for example, the start line cannot be greater than end line) and between the groups (for example, if more than one group is present, they must independently specify the exact same portion of the file). See the SARIF specification ([§3.30](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317685)). +The properties of a 'region' object must be consistent. + +SARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: with line and column numbers, with a character offset and count, or with a byte offset and count. The specification states certain constraints on these properties, both within each property group (for example, the start line cannot be greater than end line) and between the groups (for example, if more than one group is present, they must independently specify the same portion of the file). See the SARIF specification ([3.30](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317685)). #### Messages ##### `EndLineMustNotPrecedeStartLine`: error +{0}: In this 'region' object, the 'endLine' property '{1}' is less than the 'startLine' property '{2}'. The properties of a 'region' object must be internally consistent. + ##### `EndColumnMustNotPrecedeStartColumn`: error +{0}: In this 'region' object, the 'endColumn' property '{1}' is less than the 'startColumn' property '{2}'. The properties of a 'region' object must be internally consistent. + ##### `RegionStartPropertyMustBePresent`: error +{0}: This 'region' object does not specify 'startLine', 'charOffset', or 'byteOffset'. As a result, it is impossible to determine whether this 'region' object describes a line/column text region, a character offset/length text region, or a binary region. + --- -### Rule`SARIF1008.PhysicalLocationPropertiesMustBeConsistent` +### Rule `SARIF1008.PhysicalLocationPropertiesMustBeConsistent` #### Description @@ -167,7 +239,7 @@ A SARIF 'physicalLocation' object has two related properties 'region' and 'conte 'contextRegion' provides users with a broader view of the result location. Typically, it consists of a range starting a few lines before 'region' and ending a few lines after. Again, if a SARIF viewer has access to the artifact, it can display it, and highlight the context region (perhaps in a lighter shade than the region itself). This isn't terribly useful since the user can already see the whole file, with the 'region' already highlighted. But if 'contextRegion' has a 'snippet' property, then even a viewer without access to the artifact can display a few lines of code surrounding the actual result, which is helpful to users. -If the SARIF validator reports that 'contextRegion' is present but 'region' is absent, then it's possible that the tool should have populated 'region' rather than 'contextRegion', or that it simply neglected to populate 'region'. If the validator reports that 'contextRegion' is not a proper superset of 'region', then it's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool should simply omit 'contextRegion'. +If the validator reports that 'contextRegion' is not a proper superset of 'region', then it's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool should simply omit 'contextRegion'. #### Messages @@ -181,19 +253,25 @@ If the SARIF validator reports that 'contextRegion' is present but 'region' is a --- -### Rule`SARIF1009.IndexPropertiesMustBeConsistentWithArrays` +### Rule `SARIF1009.IndexPropertiesMustBeConsistentWithArrays` #### Description +If an object contains a property that is used as an array index (an "index-valued property"), then that array must be present and must contain at least "index + 1" elements. + #### Messages ##### `TargetArrayMustExist`: error +{0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' does not exist. An index-valued property always refers to an array, so the array must be present. + ##### `TargetArrayMustBeLongEnough`: error +{0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' has fewer than {5} elements. An index-valued properties must be valid for the array that it refers to. + --- -### Rule`SARIF1010.RuleIdMustBeConsistent` +### Rule `SARIF1010.RuleIdMustBeConsistent` #### Description @@ -213,55 +291,69 @@ This validation rule is required because this constraint cannot be expressed in --- -### Rule`SARIF1011.ReferenceFinalSchema` +### Rule `SARIF1011.ReferenceFinalSchema` #### Description +The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + +The SARIF standard was developed over several years, and many intermediate versions of the schema were produced. Now that the standard is final, only the OASIS standard version of the schema is valid. + #### Messages ##### `Default`: error +{0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the '$schema' property with a URL that refers to the final version of the schema. + --- -### Rule`SARIF1012.MessagePropertiesMustBeConsistent` +### Rule `SARIF1012.MessageArgumentsMustBeConsistentWithRule` #### Description -Ensure consistency among the properties of a 'message' object. +The properties of a result's 'message' property must be consistent with the properties of the rule that the result refers to. -When a tool creates a 'message' object that uses the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{3}', then the 'arguments' array must contain 4 elements. +When a result's 'message' object uses the 'id' and 'arguments' properties (which, by the way, is recommended: see SARIF2002.ProvideMessageArguments), it must ensure that the rule actually defines a message string with that id, and that 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain at least 4 elements. #### Messages -##### `SupplyCorrectNumberOfArguments`: error +##### `SupplyEnoughMessageArguments`: error -{0}: The message with id '{1}' in rule '{2}' requires {3} arguments, but the 'arguments' array in this message object has only {4} elements. When a tool creates 'message' objects that use the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain 4 elements. +{0}: The message with id '{1}' in rule '{2}' requires '{3}' arguments, but the 'arguments' array in this message object has only '{4}' element(s). When a tool creates a result message that use the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain 4 elements. + +##### `MessageIdMustExist`: error + +{0}: This message object refers to the message with id '{1}' in rule '{2}', but that rule does not define a message with that id. When a tool creates a result message that uses the 'id' property, it must ensure that the specified rule actually has a message with that id. --- -## Rules that describe less serious violations +## Rules that describe less serious violations and guidelines for effective SARIF Rules that describe violations of **SHOULD**/**SHOULD NOT** requirements of the [SARIF specification](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html) have numbers between 2000 and 2999, and always have level `"warning"`. Rules that describe violations of SARIF recommendations or best practices also have numbers in this range. Some of those recommendations are expressed in the spec as **MAY** requirements; others are based on experience using the format. These rules have level `"warning"` or `"note"`, depending on the tool's opinion of the seriousness of the violation. +This section also includes guidelines which, although not found in the spec text, are based on experience in integrating SARIF into a wide variety of static analysis tools. + --- -### Rule`SARIF2001.AuthorHighQualityMessages` +### Rule `SARIF2001.TerminateMessagesWithPeriod` #### Description -#### Messages +Express plain text result messages as complete sentences and end each sentence with a period. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary. -##### `IncludeDynamicContent`: warning +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also `SARIF2014.ProvideDynamicMessageContent` and `SARIF2015.EnquoteDynamicMessageContent`. -##### `EnquoteDynamicContent`: warning +#### Messages + +##### `Default`: warning -##### `TerminateWithPeriod`: warning +{0}: In rule '{1}', the message with id '{2}' does not end in a period. Express plain text rule messages as complete sentences. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary. --- -### Rule`SARIF2002.UseMessageArguments` +### Rule `SARIF2002.ProvideMessageArguments` #### Description @@ -275,99 +367,211 @@ In result messages, use the 'message.id' and 'message.arguments' properties rath --- -### Rule`SARIF2003.ProduceEnrichedSarif` +### Rule `SARIF2003.ProvideVersionControlProvenance` #### Description +Provide 'versionControlProvenance' to record which version of the code was analyzed, and to enable paths to be expressed relative to the root of the repository. + #### Messages -##### `ProvideVersionControlProvenance`: note +##### `Default`: note -##### `ProvideCodeSnippets`: note +This run does not provide 'versionControlProvenance'. As a result, it is not possible to determine which version of code was analyzed, nor to map relative paths to their locations within the repository. -##### `ProvideContextRegion`: note +--- -##### `ProvideHelpUris`: note +### Rule `SARIF2004.OptimizeFileSize` -##### `EmbedFileContent`: note +#### Description ---- +Emit arrays only if they provide additional information. -### Rule`SARIF2004.OptimizeFileSize` +In several parts of a SARIF log file, a subset of information about an object appears in one place, and the full information describing all such objects appears in an array elsewhere in the log file. For example, each 'result' object has a 'ruleId' property that identifies the rule that was violated. Elsewhere in the log file, the array 'run.tool.driver.rules' contains additional information about the rules. But if the elements of the 'rules' array contained no information about the rules beyond their ids, then there might be no reason to include the 'rules' array at all, and the log file could be made smaller simply by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information. -#### Description +Similarly, most 'result' objects contain at least one 'artifactLocation' object. Elsewhere in the log file, the array 'run.artifacts' contains additional information about the artifacts that were analyzed. But if the elements of the 'artifacts' array contained not information about the artifacts beyond their locations, then there might be no reason to include the 'artifacts' array at all, and again the log file could be made smaller by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information. #### Messages ##### `EliminateLocationOnlyArtifacts`: warning -##### `DoNotIncludeExtraIndexedObjectProperties`: warning +{0}: The 'artifacts' array contains no information beyond the locations of the artifacts. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information. + +##### `EliminateIdOnlyRules`: warning + +{0}: The 'rules' array contains no information beyond the ids of the rules. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information. --- -### Rule`SARIF2005.ProvideHelpfulToolInformation` +### Rule `SARIF2005.ProvideToolProperties` #### Description +Provide information that makes it easy to identify the name and version of your tool. + +The tool's 'name' property should be no more than three words long. This makes it easy to remember and allows it to fit into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property. + +The tool should provide either or both of the 'version' and 'semanticVersion' properties. This enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions. + +If 'version' is used, facilitate comparison between versions by specifying a version number that starts with an integer, optionally followed by any desired characters. + #### Messages ##### `ProvideConciseToolName`: note +{0}: The tool name '{1}' contains {2} words, which is more than the recommended maximum of {3} words. A short tool name is easy to remember and fits into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property. + ##### `ProvideToolVersion`: warning +{0}: The tool '{1}' provides neither a 'version' property nor a 'semanticVersion' property. Providing a version enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions. + ##### `UseNumericToolVersions`: warning +{0}: The tool '{1}' contains the 'version' property '{2}', which is not numeric. To facilitate comparison between versions, specify a 'version' that starts with an integer, optionally followed by any desired characters. + --- -### Rule`SARIF2006.UrisShouldBeReachable` +### Rule `SARIF2006.UrisShouldBeReachable` #### Description +URIs that refer to locations such as rule help pages and result-related work items should be reachable via a GET request. + #### Messages ##### `Default`: warning +{0}: The URI '{1}' was not reachable via a GET request. + --- -### Rule`SARIF2007.ExpressPathsRelativeToRepoRoot` +### Rule `SARIF2007.ExpressPathsRelativeToRepoRoot` #### Description +Provide information that makes it possible to determine the repo-relative locations of files that contain analysis results. + +Each element of the 'versionControlProvenance' array is a 'versionControlDetails' object that describes a repository containing files that were analyzed. 'versionControlDetails.mappedTo' defines the file system location to which the root of that repository is mapped. If 'mappedTo.uriBaseId' is present, and if result locations are expressed relative to that 'uriBaseId', then the repo-relative location of each result can be determined. + #### Messages -##### `Default`: warning +##### `ProvideUriBaseIdForMappedTo`: warning + +{0}: The 'versionControlDetails' object that describes the repository '{1}' does not provide 'mappedTo.uriBaseId'. As a result, it will not be possible to determine the repo-relative location of files containing analysis results for this repository. + +##### `ExpressResultLocationsRelativeToMappedTo`: warning + +{0}: This result location does not provide any of the 'uriBaseId' values that specify repository locations: '{1}'. As a result, it will not be possible to determine the location of the file containing this result relative to the root of the repository that contains it. --- -### Rule`SARIF2008.ProvideSchema` +### Rule `SARIF2008.ProvideSchema` #### Description +A SARIF log file should contain, on the root object, a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + #### Messages ##### `Default`: warning +{0}: The SARIF log file does not contain a '$schema' property. Add a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + --- -### Rule`SARIF2009.UseConventionalSymbolicNames` +### Rule `SARIF2009.ConsiderConventionalIdentifierValues` #### Description -Adopt uniform naming conventions for the symbolic names that SARIF uses various contexts. +Adopt uniform naming conventions for rule ids. Many tools follow a conventional format for the 'reportingDescriptor.id' property: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001' for a diagnostic from the Roslyn C# compiler. For uniformity of experience across tools, we recommend this format. -Many tool use similar names for 'uriBaseId' symbols. We suggest 'REPOROOT' for the root of a repository, 'SRCROOT' for the root of the directory containing all source code, 'TESTROOT' for the root of the directory containing all test code (if your repository is organized in that way), and 'BINROOT' for the root of the directory containing build output (if your project places all build output in a common directory). - #### Messages ##### `UseConventionalRuleIds`: note -{0}: The 'name' property ' of the rule '{1}' does not follow the recommended format: a short string identifying the tool concatenated with a numeric rule number, for example, `CS2001`. Using a conventional format for the rule id provides a more uniform experience across tools. +{0}: The 'id' property of the rule '{1}' does not follow the recommended format: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001'. Using a conventional format for the rule id provides a more uniform experience across tools. + +--- + +### Rule `SARIF2010.ProvideCodeSnippets` + +#### Description + +Provide code snippets to enable users to see the code that triggered each result, even if they are not enlisted in the code. + +#### Messages + +##### `Default`: note + +{0}: The 'region' object in this result location does not provide a 'snippet' property. Providing a code snippet enables users to see the source code that triggered the result, even if they are not enlisted in the code. + +--- + +### Rule `SARIF2011.ProvideContextRegion` + +#### Description + +Provide context regions to enable users to see a portion of the code that surrounds each result, even if they are not enlisted in the code. + +##### `Default`: note + +{0}: This result location does not provide a 'contextRegion' property. Providing a context region enables users to see a portion of the code that surrounds the result, even if they are not enlisted in the code. + +--- + +### Rule `SARIF2012.ProvideHelpUris` + +#### Description + +For each rule, provide a URI where users can find detailed information about the rule. This information should include a detailed description of the invalid pattern, an explanation of why the pattern is poor practice (particularly in contexts such as security or accessibility where driving considerations might not be readily apparent), guidance for resolving the problem (including describing circumstances in which ignoring the problem altogether might be appropriate), examples of invalid and valid patterns, and special considerations (such as noting when a violation should never be ignored or suppressed, noting when a violation could cause downstream tool noise, and noting when a rule can be configured in some way to refine or alter the analysis). + +##### `Default`: note + +{0}: The rule '{1}' does not provide a help URI. Providing a URI where users can find detailed information about the rule helps users to understand the result and how they can best address it. + +--- + +### Rule `SARIF2013.ProvideEmbeddedFileContent` + +Provide embedded file content so that users can examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it. + +#### Description + +##### `Default`: note + +{0}: This run does not provide embedded file content. Providing embedded file content enables users to examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it. + +--- + +### Rule `SARIF2014.ProvideDynamicMessageContent` + +#### Description + +Include "dynamic content" (information that varies among results from the same rule) to makes your messages more specific, and to avoid the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable. -##### `UseConventionalUriBaseIdNames`: note +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2015.EnquoteDynamicMessageContent'. -{0}: The 'originalUriBaseIds' symbol '{1}' is not one of the conventional symbols. We suggest 'REPOROOT' for the root of a repository, 'SRCROOT' for the root of the directory containing all source code, 'TESTROOT' for the root of the directory containing all test code (if your repository is organized in that way), and 'BINROOT' for the root of the directory containing build output (if your project places all build output in a common directory). +#### Messages + +##### `Default`: note + +{0}: In rule '{1}', the message with id '{2}' does not include any dynamic content. Dynamic content makes your messages more specific and avoids the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable. --- + +### Rule `SARIF2015.EnquoteDynamicMessageContent` + +#### Description + +Place dynamic content in single quotes to set it off from the static text and to make it easier to spot. It's especially helpful when the dynamic content is a string that might contain spaces, and most especially when the string might be empty (and so would be invisible if it weren't for the quotes). We recommend single quotes for a less cluttered appearance, even though US English usage would require double quotes. + +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2014.ProvideDynamicMessageContent'. + +#### Messages + +##### `Default`: note + +{0}: In rule '{1}', the message with id '{2}' includes dynamic content that is not enclosed in single quotes. Enquoting dynamic content makes it easier to spot, and single quotes give a less cluttered appearance. \ No newline at end of file diff --git a/docs/Rule factoring.xlsx b/docs/Rule factoring.xlsx index 414421d4a..d72af51e7 100644 Binary files a/docs/Rule factoring.xlsx and b/docs/Rule factoring.xlsx differ diff --git a/src/ReleaseHistory.md b/src/ReleaseHistory.md index b7f6c9631..7c65af950 100644 --- a/src/ReleaseHistory.md +++ b/src/ReleaseHistory.md @@ -1,6 +1,7 @@ # SARIF Package Release History (SDK, Driver, Converters, and Multitool) ## **v2.3.1** [Sdk](https://www.nuget.org/packages/Sarif.Sdk/2.3.1) | [Driver](https://www.nuget.org/packages/Sarif.Driver/2.3.1) | [Converters](https://www.nuget.org/packages/Sarif.Converters/2.3.1) | [Multitool](https://www.nuget.org/packages/Sarif.Multitool/2.3.1) +* FEATURE: Revised and improved validation rules in `Sarif.Multitool`. * FEATURE: Properties serialization performance improved (~20% faster load when Results use Properties). * FEATURE: Allow result messages to be truncated for display. [#1915](https://github.com/microsoft/sarif-sdk/issues/1915) * BUGFIX: Rebase URI command now honors `--insert` and `--remove` arguments for injecting or eliding optional data (such as region snippets). diff --git a/src/Sarif.Multitool/Rules/RuleId.cs b/src/Sarif.Multitool/Rules/RuleId.cs index f2d83ee74..5ed687f2b 100644 --- a/src/Sarif.Multitool/Rules/RuleId.cs +++ b/src/Sarif.Multitool/Rules/RuleId.cs @@ -5,23 +5,39 @@ namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules { public static class RuleId { - public const string DoNotUseFriendlyNameAsRuleId = "SARIF1001"; - public const string UrisMustBeValid = "SARIF1003"; - public const string HashAlgorithmsMustBeUnique = "SARIF1006"; - public const string EndTimeMustNotBeBeforeStartTime = "SARIF1007"; - public const string MessagesShouldEndWithPeriod = "SARIF1008"; - public const string StepValuesMustFormOneBasedSequence = "SARIF1009"; - public const string EndLineMustNotBeLessThanStartLine = "SARIF1012"; - public const string EndColumnMustNotBeLessThanStartColumn = "SARIF1013"; - public const string UriBaseIdRequiresRelativeUri = "SARIF1014"; - public const string UriMustBeAbsolute = "SARIF1015"; - public const string ContextRegionRequiresRegion = "SARIF1016"; - public const string InvalidIndex = "SARIF1017"; - public const string InvalidUriInOriginalUriBaseIds = "SARIF1018"; - public const string RuleIdMustBePresentAndConsistent = "SARIF1019"; - public const string ReferToFinalSchema = "SARIF1020"; + public const string RuleIdentifiersMustBeValid = "SARIF1001"; + public const string UrisMustBeValid = "SARIF1002"; + public const string ExpressUriBaseIdsCorrectly = "SARIF1004"; + public const string UriMustBeAbsolute = "SARIF1005"; + + public const string InvocationPropertiesMustBeConsistent = "SARIF1006"; + public const string RegionPropertiesMustBeConsistent = "SARIF1007"; + public const string PhysicalLocationPropertiesMustBeConsistent = "SARIF1008"; + public const string IndexPropertiesMustBeConsistentWithArrays = "SARIF1009"; + public const string RuleIdMustBeConsistent = "SARIF1010"; + + public const string ReferenceFinalSchema = "SARIF1011"; + public const string MessageArgumentsMustBeConsistentWithRule = "SARIF1012"; + + public const string TerminateMessagesWithPeriod = "SARIF2001"; + public const string ProvideMessageArguments = "SARIF2002"; + public const string ProvideVersionControlProvenance = "SARIF2003"; + public const string OptimizeFileSize = "SARIF2004"; + public const string ProvideToolProperties = "SARIF2005"; + + public const string UrisShouldBeReachable = "SARIF2006"; + public const string ExpressPathsRelativeToRepoRoot = "SARIF2007"; + public const string ProvideSchema = "SARIF2008"; + public const string ConsiderConventionalIdentifierValues = "SARIF2009"; + public const string ProvideCodeSnippets = "SARIF2010"; + + public const string ProvideContextRegion = "SARIF2011"; + public const string ProvideHelpUris = "SARIF2012"; + public const string ProvideEmbeddedFileContent = "SARIF2013"; + public const string ProvideDynamicMessageContent = "SARIF2014"; + public const string EnquoteDynamicMessageContent = "SARIF2015"; // TEMPLATE: - // public const string RULEFRIENDLYNAME = "RULEID"; + // public const string RuleFriendlyName = "SARIFnnnn"; } } diff --git a/src/Sarif.Multitool/Rules/RuleResources.Designer.cs b/src/Sarif.Multitool/Rules/RuleResources.Designer.cs index ad680c1d8..b8eaf3b80 100644 --- a/src/Sarif.Multitool/Rules/RuleResources.Designer.cs +++ b/src/Sarif.Multitool/Rules/RuleResources.Designer.cs @@ -61,308 +61,652 @@ internal RuleResources() { } /// - /// Looks up a localized string similar to {0}: The name and id properties of rule "{1}" are the same.. + /// Looks up a localized string similar to {0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name [rest of string was truncated]";. /// - internal static string SARIF1001_Default { + internal static string SARIF1001_RuleIdentifiersMustBeValid_Error_Default_Text { get { - return ResourceManager.GetString("SARIF1001_Default", resourceCulture); + return ResourceManager.GetString("SARIF1001_RuleIdentifiersMustBeValid_Error_Default_Text", resourceCulture); } } /// - /// Looks up a localized string similar to Do not use the same string for a rule's id and name properties. The id property must be a stable, opaque identifer such as "SARIF1001". The name property should be a string that is understandable to an end user, such as "DoNotUserFriendlyNameAsRuleId".. + /// Looks up a localized string similar to The two identity-related properties of a SARIF rule must be consistent. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are pres [rest of string was truncated]";. /// - internal static string SARIF1001_DoNotUseFriendlyNameAsRuleIdDescription { + internal static string SARIF1001_RuleIdentifiersMustBeValid_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1001_DoNotUseFriendlyNameAsRuleIdDescription", resourceCulture); + return ResourceManager.GetString("SARIF1001_RuleIdentifiersMustBeValid_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The string "{1}" is not a valid URI reference.. + /// Looks up a localized string similar to {0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it.. /// - internal static string SARIF1003_Default { + internal static string SARIF1002_UrisMustBeValid_Error_FileUrisMustNotIncludeDotDotSegments_Text { get { - return ResourceManager.GetString("SARIF1003_Default", resourceCulture); + return ResourceManager.GetString("SARIF1002_UrisMustBeValid_Error_FileUrisMustNotIncludeDotDotSegments_Text", resourceCulture); } } /// - /// Looks up a localized string similar to Specify a valid URI reference for every URI-valued property.. + /// Looks up a localized string similar to {0}: The string '{1}' is not a valid URI reference. URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986).. /// - internal static string SARIF1003_UrisMustBeValid { + internal static string SARIF1002_UrisMustBeValid_Error_UrisMustConformToRfc3986_Text { get { - return ResourceManager.GetString("SARIF1003_UrisMustBeValid", resourceCulture); + return ResourceManager.GetString("SARIF1002_UrisMustBeValid_Error_UrisMustConformToRfc3986_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The array contains multiple objects with the value "{1}" for the algorithm property.. + /// Looks up a localized string similar to Specify a valid URI reference for every URI-valued property. + /// + ///URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, 'file' URIs must not include '..' segments. If symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it.. /// - internal static string SARIF1006_Default { + internal static string SARIF1002_UrisMustBeValid_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1006_Default", resourceCulture); + return ResourceManager.GetString("SARIF1002_UrisMustBeValid_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to In any given "file" object, every element of the "hashes" array must have a different value for its "algorithm" property. . + /// Looks up a localized string similar to {0}: The '{1}' element of 'originalUriBaseIds' has no 'uriBaseId' property, but its 'uri' property '{2}' is not an absolute URI. According to the SARIF specification, every such "top-level" entry in 'originalUriBaseIds' must specify an absolute URI, because the purpose of 'originalUriBaseIds' is to enable the resolution of relative references to absolute URIs.. /// - internal static string SARIF1006_HashAlgorithmsMustBeUnique { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_Error_TopLevelUriBaseIdMustBeAbsolute_Text { get { - return ResourceManager.GetString("SARIF1006_HashAlgorithmsMustBeUnique", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_Error_TopLevelUriBaseIdMustBeAbsolute_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The end time "{1}" is before the start time "{2}".. + /// Looks up a localized string similar to {0}: This 'artifactLocation' object has a 'uriBaseId' property '{1}', but its 'uri' property '{2}' is an absolute URI. Since the purpose of 'uriBaseId' is to resolve a relative reference to an absolute URI, it is not allowed when the 'uri' property is already an absolute URI.. /// - internal static string SARIF1007_Default { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdRequiresRelativeUri_Text { get { - return ResourceManager.GetString("SARIF1007_Default", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdRequiresRelativeUri_Text", resourceCulture); } } /// - /// Looks up a localized string similar to The end time of a run must not precede the start time. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal.. + /// Looks up a localized string similar to {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that does not end with a slash. The trailing slash is required to minimize the likelihood of an error when concatenating URI segments together.. /// - internal static string SARIF1007_EndTimeMustNotBeBeforeStartTime { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustEndWithSlash_Text { get { - return ResourceManager.GetString("SARIF1007_EndTimeMustNotBeBeforeStartTime", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustEndWithSlash_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The message "{1}" does not end with a period.. + /// Looks up a localized string similar to {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it.. /// - internal static string SARIF1008_Default { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainDotDotSegment_Text { get { - return ResourceManager.GetString("SARIF1008_Default", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainDotDotSegm" + + "ent_Text", resourceCulture); } } /// - /// Looks up a localized string similar to Messages should consist of one or more complete sentences, ending with a period.. + /// Looks up a localized string similar to {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a query or a fragment. This is not valid because the purpose of the 'uriBaseId' property is to help resolve a relative reference to an absolute URI by concatenating the relative reference to the absolute base URI. This won't work if the base URI contains a query or a fragment.. /// - internal static string SARIF1008_MessagesShouldEndWithPeriod { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainQueryOrFragment_Text { get { - return ResourceManager.GetString("SARIF1008_MessagesShouldEndWithPeriod", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainQueryOrFra" + + "gment_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The value of the "step" property should be {1}, but is {2}. The step values must form a 1-based sequence starting at the first threadFlowLocation in the thread flow.. + /// Looks up a localized string similar to When using the 'uriBaseId' property, obey the requirements in the SARIF specification [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) that enable it to fulfill its purpose of resolving relative references to absolute locations. In particular: + /// + ///If an 'artifactLocation' object has a 'uriBaseId' property, its 'uri' property must be a relative reference, because if 'uri' is an absolute URI then 'uriBaseId' serves no purpose. + /// + ///Every URI reference in 'originalUriBa [rest of string was truncated]";. /// - internal static string SARIF1009_InvalidStepValue { + internal static string SARIF1004_ExpressUriBaseIdsCorrectly_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1009_InvalidStepValue", resourceCulture); + return ResourceManager.GetString("SARIF1004_ExpressUriBaseIdsCorrectly_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The "step" property is absent. The "step" property is present on some but not all threadFlowLocations in this threadFlow. If the "step" property is used on any threadFlowLocation in a threadFlow, it must be present on every threadFlowLocation in that threadFlow.. + /// Looks up a localized string similar to {0}: The value of this property is required to be an absolute URI, but '{1}' is a relative URI reference.. /// - internal static string SARIF1009_StepNotPresentOnAllLocations { + internal static string SARIF1005_UriMustBeAbsolute_Error_Default_Text { get { - return ResourceManager.GetString("SARIF1009_StepNotPresentOnAllLocations", resourceCulture); + return ResourceManager.GetString("SARIF1005_UriMustBeAbsolute_Error_Default_Text", resourceCulture); } } /// - /// Looks up a localized string similar to If the "step" property is used on any threadFlowLocation in a threadFlow, then it must be present for every threadFlowLocation in the threadFlow, its value must be 1 for the first threadFlowLocation, and its value must increase by 1 for each succeeding threadFlowLocation.. + /// Looks up a localized string similar to Certain URIs are required to be absolute. For the most part, these are URIs that refer to http addresses, such as work items or rule help topics.. /// - internal static string SARIF1009_StepValuesMustFormOneBasedSequence { + internal static string SARIF1005_UriMustBeAbsolute_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1009_StepValuesMustFormOneBasedSequence", resourceCulture); + return ResourceManager.GetString("SARIF1005_UriMustBeAbsolute_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The value of the "endLine" property is {1}, which is less than the value of the "startLine" property, which is {2}.. + /// Looks up a localized string similar to {0}: The 'endTimeUtc' value '{1}' precedes the 'startTimeUtc' value '{2}'. The properties of an 'invocation' object must be internally consistent.. /// - internal static string SARIF1012_Default { + internal static string SARIF1006_InvocationPropertiesMustBeConsistent_Error_EndTimeMustNotPrecedeStartTime_Text { get { - return ResourceManager.GetString("SARIF1012_Default", resourceCulture); + return ResourceManager.GetString("SARIF1006_InvocationPropertiesMustBeConsistent_Error_EndTimeMustNotPrecedeStartTi" + + "me_Text", resourceCulture); } } /// - /// Looks up a localized string similar to The "endLine" property of a region object must not be less than the "startLine" property.. + /// Looks up a localized string similar to The properties of an 'invocation' object must be consistent. + /// + ///If the 'invocation' object specifies both 'startTimeUtc' and 'endTimeUtc', then 'endTimeUtc' must not precede 'startTimeUtc'. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal.. /// - internal static string SARIF1012_EndLineMustNotBeLessThanStartLine { + internal static string SARIF1006_InvocationPropertiesMustBeConsistent_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1012_EndLineMustNotBeLessThanStartLine", resourceCulture); + return ResourceManager.GetString("SARIF1006_InvocationPropertiesMustBeConsistent_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The value of the "endColumn" property is {1}, which is less than the value of the "startColumn" property, which is {2}.. + /// Looks up a localized string similar to {0}: In this 'region' object, the 'endColumn' property '{1}' is less than the 'startColumn' property '{2}'. The properties of a 'region' object must be internally consistent.. /// - internal static string SARIF1013_Default { + internal static string SARIF1007_RegionPropertiesMustBeConsistent_Error_EndColumnMustNotPrecedeStartColumn_Text { get { - return ResourceManager.GetString("SARIF1013_Default", resourceCulture); + return ResourceManager.GetString("SARIF1007_RegionPropertiesMustBeConsistent_Error_EndColumnMustNotPrecedeStartColu" + + "mn_Text", resourceCulture); } } /// - /// Looks up a localized string similar to The "endColumn" property of a region object must not be less than the "startColumn" property.. + /// Looks up a localized string similar to {0}: In this 'region' object, the 'endLine' property '{1}' is less than the 'startLine' property '{2}'. The properties of a 'region' object must be internally consistent.. /// - internal static string SARIF1013_EndColumnMustNotBeLessThanStartColumn { + internal static string SARIF1007_RegionPropertiesMustBeConsistent_Error_EndLineMustNotPrecedeStartLine_Text { get { - return ResourceManager.GetString("SARIF1013_EndColumnMustNotBeLessThanStartColumn", resourceCulture); + return ResourceManager.GetString("SARIF1007_RegionPropertiesMustBeConsistent_Error_EndLineMustNotPrecedeStartLine_T" + + "ext", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: This fileLocation object contains a "uriBaseId" property, which means that the value of the "uri" property must be a relative URI reference, but "{1}" is an absolute URI reference.. + /// Looks up a localized string similar to {0}: This 'region' object does not specify 'startLine', 'charOffset', or 'byteOffset'. As a result, it is impossible to determine whether this 'region' object describes a line/column text region, a character offset/length text region, or a binary region.. /// - internal static string SARIF1014_Default { + internal static string SARIF1007_RegionPropertiesMustBeConsistent_Error_RegionStartPropertyMustBePresent_Text { get { - return ResourceManager.GetString("SARIF1014_Default", resourceCulture); + return ResourceManager.GetString("SARIF1007_RegionPropertiesMustBeConsistent_Error_RegionStartPropertyMustBePresent" + + "_Text", resourceCulture); } } /// - /// Looks up a localized string similar to If a fileLocation object contains a "uriBaseId" property, the value of the "uri" property must be a relative URI reference.. + /// Looks up a localized string similar to The properties of a 'region' object must be consistent. + /// + ///SARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: with line and column numbers, with a character offset and count, or with a byte offset and count. The specification states certain constraints on these properties, both within each property group (for example, the start line cannot be greater than end line) and between the groups (for example, if more than one group is present, they must independently specify the sam [rest of string was truncated]";. /// - internal static string SARIF1014_UriBaseIdRequiresRelativeUri { + internal static string SARIF1007_RegionPropertiesMustBeConsistent_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1014_UriBaseIdRequiresRelativeUri", resourceCulture); + return ResourceManager.GetString("SARIF1007_RegionPropertiesMustBeConsistent_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The value of this property is required to be an absolute URI, but "{1}" is a relative URI reference.. + /// Looks up a localized string similar to {0}: This 'physicalLocation' object contains both a 'region' and a 'contextRegion' property, but 'contextRegion' is not a proper superset of 'region'. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. It's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool must omit 'contextRegion'.. /// - internal static string SARIF1015_Default { + internal static string SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionMustBeProperSupersetOfRegion_Text { get { - return ResourceManager.GetString("SARIF1015_Default", resourceCulture); + return ResourceManager.GetString("SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionMustBePro" + + "perSupersetOfRegion_Text", resourceCulture); } } /// - /// Looks up a localized string similar to Certain URIs are required to be absolute.. + /// Looks up a localized string similar to {0}: This 'physicalLocation' object contains a 'contextRegion' property, but it does not contain a 'region' property. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. If a tool associates only one region with a result, it must populate 'region', not 'contextRegion'.. /// - internal static string SARIF1015_UriMustBeAbsolute { + internal static string SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionRequiresRegion_Text { get { - return ResourceManager.GetString("SARIF1015_UriMustBeAbsolute", resourceCulture); + return ResourceManager.GetString("SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionRequiresR" + + "egion_Text", resourceCulture); } } /// - /// Looks up a localized string similar to If the "contextRegion" property is present, the "region" property must also be present.. + /// Looks up a localized string similar to Ensure consistency among the properties of a 'physicalLocation' object. + /// + ///A SARIF 'physicalLocation' object has two related properties 'region' and 'contextRegion'. If 'contextRegion' is present, then 'region' must also be present, and 'contextRegion' must be a "proper superset" of 'region'. That is, 'contextRegion' must completely contain 'region', and it must be larger than 'region'. To understand why this is so we must understand the roles of the 'region' and 'contextRegion' properties. + /// + ///'region' allo [rest of string was truncated]";. /// - internal static string SARIF1016_ContextRegionRequiresRegion { + internal static string SARIF1008_PhysicalLocationPropertiesMustBeConsistent_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1016_ContextRegionRequiresRegion", resourceCulture); + return ResourceManager.GetString("SARIF1008_PhysicalLocationPropertiesMustBeConsistent_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: This "physicalLocation" object contains a "contextRegion" property, but it does not contain a "region" property.. + /// Looks up a localized string similar to {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' has fewer than {5} elements. An index-valued properties must be valid for the array that it refers to.. /// - internal static string SARIF1016_Default { + internal static string SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustBeLongEnough_Text { get { - return ResourceManager.GetString("SARIF1016_Default", resourceCulture); + return ResourceManager.GetString("SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustBeLongEn" + + "ough_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: This "{1}" object contains a property "{2}" with value {3}, but either "{4}" is absent, or it has fewer than {5} elements.. + /// Looks up a localized string similar to {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' does not exist. An index-valued property always refers to an array, so the array must be present.. /// - internal static string SARIF1017_Default { + internal static string SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustExist_Text { get { - return ResourceManager.GetString("SARIF1017_Default", resourceCulture); + return ResourceManager.GetString("SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustExist_Te" + + "xt", resourceCulture); } } /// - /// Looks up a localized string similar to If an object contains a property that is used as an array index, then that array must be present and must contain at least "index + 1" elements.. + /// Looks up a localized string similar to If an object contains a property that is used as an array index (an "index-valued property"), then that array must be present and must contain at least "index + 1" elements.. /// - internal static string SARIF1017_InvalidIndex { + internal static string SARIF1009_IndexPropertiesMustBeConsistentWithArrays_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1017_InvalidIndex", resourceCulture); + return ResourceManager.GetString("SARIF1009_IndexPropertiesMustBeConsistentWithArrays_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to In the artifactLocation objects contained in run.originalUriBaseIds, if uriBaseId is absent, then uri must either be an absolute URI or it must be absent. Also, uri must end with a slash, so that it can safely be combined with the relative URIs in artifactLocation objects elsewhere in the log file.. + /// Looks up a localized string similar to {0}: This result contains neither of the properties 'ruleId' or 'rule.id'. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires at least one of these properties to be present.. /// - internal static string SARIF1018_InvalidUriInOriginalUriBaseIds { + internal static string SARIF1010_RuleIdMustBeConsistent_Error_ResultMustSpecifyRuleId_Text { get { - return ResourceManager.GetString("SARIF1018_InvalidUriInOriginalUriBaseIds", resourceCulture); + return ResourceManager.GetString("SARIF1010_RuleIdMustBeConsistent_Error_ResultMustSpecifyRuleId_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds does not end with a slash.. + /// Looks up a localized string similar to {0}: This result contains both the 'ruleId' property '{1}' and the 'rule.id' property '{2}', but they are not equal. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires that if both of these properties are present, they must be equal.. /// - internal static string SARIF1018_LacksTrailingSlash { + internal static string SARIF1010_RuleIdMustBeConsistent_Error_ResultRuleIdMustBeConsistent_Text { get { - return ResourceManager.GetString("SARIF1018_LacksTrailingSlash", resourceCulture); + return ResourceManager.GetString("SARIF1010_RuleIdMustBeConsistent_Error_ResultRuleIdMustBeConsistent_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds is not an absolute URI.. + /// Looks up a localized string similar to Every result must contain at least one of the properties 'ruleId' and 'rule.id'. If both are present, they must be equal. See the SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)).. /// - internal static string SARIF1018_NotAbsolute { + internal static string SARIF1010_RuleIdMustBeConsistent_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1018_NotAbsolute", resourceCulture); + return ResourceManager.GetString("SARIF1010_RuleIdMustBeConsistent_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The result contains both the ruleId property '{1}' and the rule.id property '{2}', and they are not equal.. + /// Looks up a localized string similar to {0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the '$schema' property with a URL that refers to the final version of the schema.. /// - internal static string SARIF1019_InconsistentResultRuleId { + internal static string SARIF1011_ReferenceFinalSchema_Error_Default_Text { get { - return ResourceManager.GetString("SARIF1019_InconsistentResultRuleId", resourceCulture); + return ResourceManager.GetString("SARIF1011_ReferenceFinalSchema_Error_Default_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The result contains neither result.ruleId nor result.rule.id.. + /// Looks up a localized string similar to The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + /// + ///The SARIF standard was developed over several years, and many intermediate versions of the schema were produced. Now that the standard is final, only the OASIS standard version of the schema is valid.. /// - internal static string SARIF1019_MissingResultRuleId { + internal static string SARIF1011_ReferenceFinalSchema_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1019_MissingResultRuleId", resourceCulture); + return ResourceManager.GetString("SARIF1011_ReferenceFinalSchema_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to In every result, at least one of the properties result.ruleId and result.rule.id must be present. If both are present, they must be equal.. + /// Looks up a localized string similar to {0}: This message object refers to the message with id '{1}' in rule '{2}', but that rule does not define a message with that id. When a tool creates a result message that uses the 'id' property, it must ensure that the specified rule actually has a message with that id.. /// - internal static string SARIF1019_RuleIdMustBePresentAndConsistent { + internal static string SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_MessageIdMustExist_Text { get { - return ResourceManager.GetString("SARIF1019_RuleIdMustBePresentAndConsistent", resourceCulture); + return ResourceManager.GetString("SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_MessageIdMustExist_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The $schema property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the $schema property with a URL that refers to the final version of the schema.. + /// Looks up a localized string similar to {0}: The message with id '{1}' in rule '{2}' requires '{3}' arguments, but the 'arguments' array in this message object has only '{4}' element(s). When a tool creates a result message that use the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must con [rest of string was truncated]";. /// - internal static string SARIF1020_ReferenceToOldSchemaVersion { + internal static string SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_SupplyEnoughMessageArguments_Text { get { - return ResourceManager.GetString("SARIF1020_ReferenceToOldSchemaVersion", resourceCulture); + return ResourceManager.GetString("SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_SupplyEnoughMessageArgum" + + "ents_Text", resourceCulture); } } /// - /// Looks up a localized string similar to The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.. + /// Looks up a localized string similar to The properties of a result's 'message' property must be consistent with the properties of the rule that the result refers to. + /// + ///When a result's 'message' object uses the 'id' and 'arguments' properties (which, by the way, is recommended: see SARIF2002.ProvideMessageArguments), it must ensure that the rule actually defines a message string with that id, and that 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highes [rest of string was truncated]";. /// - internal static string SARIF1020_ReferToFinalSchema { + internal static string SARIF1012_MessageArgumentsMustBeConsistentWithRule_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1020_ReferToFinalSchema", resourceCulture); + return ResourceManager.GetString("SARIF1012_MessageArgumentsMustBeConsistentWithRule_FullDescription_Text", resourceCulture); } } /// - /// Looks up a localized string similar to {0}: The SARIF log file does not contain a $schema property. Add a $schema property that refers to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.. + /// Looks up a localized string similar to Express plain text result messages as complete sentences and end each sentence with a period. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary. + /// + ///This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also `SARIF2014.ProvideDynamicMessageContent` and `SARIF2015.EnquoteDynamicMessageContent`.. /// - internal static string SARIF1020_SchemaReferenceMissing { + internal static string SARIF2001_TerminateMessagesWithPeriod_FullDescription_Text { get { - return ResourceManager.GetString("SARIF1020_SchemaReferenceMissing", resourceCulture); + return ResourceManager.GetString("SARIF2001_TerminateMessagesWithPeriod_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: In rule '{1}', the message with id '{2}' does not end in a period. Express plain text rule messages as complete sentences. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary.. + /// + internal static string SARIF2001_TerminateMessagesWithPeriod_Warning_Default_Text { + get { + return ResourceManager.GetString("SARIF2001_TerminateMessagesWithPeriod_Warning_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to In result messages, use the 'message.id' and 'message.arguments' properties rather than 'message.text'. This has several advantages. If 'text' is lengthy, using 'id' and 'arguments' makes the SARIF file smaller. If the rule metadata is stored externally to the SARIF log file, the message text can be improved (for example, by adding more text, clarifying the phrasing, or fixing typos), and the result messages will pick up the improvements the next time it is displayed. Finally, SARIF supports localizing mess [rest of string was truncated]";. + /// + internal static string SARIF2002_ProvideMessageArguments_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2002_ProvideMessageArguments_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'message' property of this result contains a 'text' property. Consider replacing it with 'id' and 'arguments' properties. This potentially reduces the log file size, allows the message text to be improved without modifying the log file, and enables localization.. + /// + internal static string SARIF2002_ProvideMessageArguments_Warning_Default_Text { + get { + return ResourceManager.GetString("SARIF2002_ProvideMessageArguments_Warning_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide 'versionControlProvenance' to record which version of the code was analyzed, and to enable paths to be expressed relative to the root of the repository.. + /// + internal static string SARIF2003_ProvideVersionControlProvenance_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2003_ProvideVersionControlProvenance_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: This run does not provide 'versionControlProvenance'. As a result, it is not possible to determine which version of code was analyzed, nor to map relative paths to their locations within the repository.. + /// + internal static string SARIF2003_ProvideVersionControlProvenance_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2003_ProvideVersionControlProvenance_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Emit arrays only if they provide additional information. + /// + ///In several parts of a SARIF log file, a subset of information about an object appears in one place, and the full information describing all such objects appears in an array elsewhere in the log file. For example, each 'result' object has a 'ruleId' property that identifies the rule that was violated. Elsewhere in the log file, the array 'run.tool.driver.rules' contains additional information about the rules. But if the elements of the 'rules' array [rest of string was truncated]";. + /// + internal static string SARIF2004_OptimizeFileSize_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2004_OptimizeFileSize_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'rules' array contains no information beyond the ids of the rules. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information.. + /// + internal static string SARIF2004_OptimizeFileSize_Warning_EliminateIdOnlyRules_Text { + get { + return ResourceManager.GetString("SARIF2004_OptimizeFileSize_Warning_EliminateIdOnlyRules_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'artifacts' array contains no information beyond the locations of the artifacts. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information.. + /// + internal static string SARIF2004_OptimizeFileSize_Warning_EliminateLocationOnlyArtifacts_Text { + get { + return ResourceManager.GetString("SARIF2004_OptimizeFileSize_Warning_EliminateLocationOnlyArtifacts_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide information that makes it easy to identify the name and version of your tool. + /// + ///The tool's 'name' property should be no more than three words long. This makes it easy to remember and allows it to fit into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property. + /// + ///The tool should provide either or both of the 'version' and 'semanticVersion' properties. This enables the log file consumer to determine whether the file was [rest of string was truncated]";. + /// + internal static string SARIF2005_ProvideToolProperties_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2005_ProvideToolProperties_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The tool name '{1}' contains {2} words, which is more than the recommended maximum of {3} words. A short tool name is easy to remember and fits into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property.. + /// + internal static string SARIF2005_ProvideToolProperties_Warning_ProvideConciseToolName_Text { + get { + return ResourceManager.GetString("SARIF2005_ProvideToolProperties_Warning_ProvideConciseToolName_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The tool '{1}' provides neither a 'version' property nor a 'semanticVersion' property. Providing a version enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions.. + /// + internal static string SARIF2005_ProvideToolProperties_Warning_ProvideToolVersion_Text { + get { + return ResourceManager.GetString("SARIF2005_ProvideToolProperties_Warning_ProvideToolVersion_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The tool '{1}' contains the 'version' property '{2}', which is not numeric. To facilitate comparison between versions, specify a 'version' that starts with an integer, optionally followed by any desired characters.. + /// + internal static string SARIF2005_ProvideToolProperties_Warning_UseNumericToolVersions_Text { + get { + return ResourceManager.GetString("SARIF2005_ProvideToolProperties_Warning_UseNumericToolVersions_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to URIs that refer to locations such as rule help pages and result-related work items should be reachable via an HTTP GET request.. + /// + internal static string SARIF2006_UrisShouldBeReachable_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2006_UrisShouldBeReachable_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The URI '{1}' was not reachable via an HTTP GET request.. + /// + internal static string SARIF2006_UrisShouldBeReachable_Warning_Default_Text { + get { + return ResourceManager.GetString("SARIF2006_UrisShouldBeReachable_Warning_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide information that makes it possible to determine the repo-relative locations of files that contain analysis results. + /// + ///Each element of the 'versionControlProvenance' array is a 'versionControlDetails' object that describes a repository containing files that were analyzed. 'versionControlDetails.mappedTo' defines the file system location to which the root of that repository is mapped. If 'mappedTo.uriBaseId' is present, and if result locations are expressed relative to that 'uriBaseId', then the repo [rest of string was truncated]";. + /// + internal static string SARIF2007_ExpressPathsRelativeToRepoRoot_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2007_ExpressPathsRelativeToRepoRoot_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: This result location does not provide any of the 'uriBaseId' values that specify repository locations: '{1}'. As a result, it will not be possible to determine the location of the file containing this result relative to the root of the repository that contains it.. + /// + internal static string SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ExpressResultLocationsRelativeToMappedTo_Text { + get { + return ResourceManager.GetString("SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ExpressResultLocationsRelativeTo" + + "MappedTo_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'versionControlDetails' object that describes the repository '{1}' does not provide 'mappedTo.uriBaseId'. As a result, it will not be possible to determine the repo-relative location of files containing analysis results for this repository.. + /// + internal static string SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ProvideUriBaseIdForMappedTo_Text { + get { + return ResourceManager.GetString("SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ProvideUriBaseIdForMappedTo_Text" + + "", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to A SARIF log file should contain, on the root object, a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.. + /// + internal static string SARIF2008_ProvideSchema_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2008_ProvideSchema_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The SARIF log file does not contain a '$schema' property. Add a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.. + /// + internal static string SARIF2008_ProvideSchema_Warning_Default_Text { + get { + return ResourceManager.GetString("SARIF2008_ProvideSchema_Warning_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Adopt uniform naming conventions for rule ids. + /// + ///Many tools follow a conventional format for the 'reportingDescriptor.id' property: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001' for a diagnostic from the Roslyn C# compiler. For uniformity of experience across tools, we recommend this format.. + /// + internal static string SARIF2009_ConsiderConventionalIdentifierValues_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2009_ConsiderConventionalIdentifierValues_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'id' property of the rule '{1}' does not follow the recommended format: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001'. Using a conventional format for the rule id provides a more uniform experience across tools.. + /// + internal static string SARIF2009_ConsiderConventionalIdentifierValues_Note_UseConventionalRuleIds_Text { + get { + return ResourceManager.GetString("SARIF2009_ConsiderConventionalIdentifierValues_Note_UseConventionalRuleIds_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide code snippets to enable users to see the code that triggered each result, even if they are not enlisted in the code.. + /// + internal static string SARIF2010_ProvideCodeSnippets_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2010_ProvideCodeSnippets_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The 'region' object in this result location does not provide a 'snippet' property. Providing a code snippet enables users to see the code that triggered the result, even if they are not enlisted in the code.. + /// + internal static string SARIF2010_ProvideCodeSnippets_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2010_ProvideCodeSnippets_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide context regions to enable users to see a portion of the code that surrounds each result, even if they are not enlisted in the code.. + /// + internal static string SARIF2011_ProvideContextRegion_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2011_ProvideContextRegion_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: This result location does not provide a 'contextRegion' property. Providing a context region enables users to see a portion of the code that surrounds the result, even if they are not enlisted in the code.. + /// + internal static string SARIF2011_ProvideContextRegion_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2011_ProvideContextRegion_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to For each rule, provide a URI where users can find detailed information about the rule. This information should include a detailed description of the invalid pattern, an explanation of why the pattern is poor practice (particularly in contexts such as security or accessibility where driving considerations might not be readily apparent), guidance for resolving the problem (including describing circumstances in which ignoring the problem altogether might be appropriate), examples of invalid and valid patterns, [rest of string was truncated]";. + /// + internal static string SARIF2012_ProvideHelpUris_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2012_ProvideHelpUris_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: The rule '{1}' does not provide a help URI. Providing a URI where users can find detailed information about the rule helps users to understand the result and how they can best address it.. + /// + internal static string SARIF2012_ProvideHelpUris_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2012_ProvideHelpUris_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Provide embedded file content so that users can examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it.. + /// + internal static string SARIF2013_ProvideEmbeddedFileContent_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2013_ProvideEmbeddedFileContent_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: This run does not provide embedded file content. Providing embedded file content enables users to examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it.. + /// + internal static string SARIF2013_ProvideEmbeddedFileContent_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2013_ProvideEmbeddedFileContent_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Include "dynamic content" (information that varies among results from the same rule) to makes your messages more specific, and to avoid the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable. + /// + ///This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2015.EnquoteDynamicMessageContent'.. + /// + internal static string SARIF2014_ProvideDynamicMessageContent_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2014_ProvideDynamicMessageContent_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: In rule '{1}', the message with id '{2}' does not include any dynamic content. Dynamic content makes your messages more specific and avoids the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable.. + /// + internal static string SARIF2014_ProvideDynamicMessageContent_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2014_ProvideDynamicMessageContent_Note_Default_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to Place dynamic content in single quotes to set it off from the static text and to make it easier to spot. It's especially helpful when the dynamic content is a string that might contain spaces, and most especially when the string might be empty (and so would be invisible if it weren't for the quotes). We recommend single quotes for a less cluttered appearance, even though US English usage would require double quotes. + /// + ///This is part of a set of authoring practices that make your rule messages more readable, [rest of string was truncated]";. + /// + internal static string SARIF2015_EnquoteDynamicMessageContent_FullDescription_Text { + get { + return ResourceManager.GetString("SARIF2015_EnquoteDynamicMessageContent_FullDescription_Text", resourceCulture); + } + } + + /// + /// Looks up a localized string similar to {0}: In rule '{1}', the message with id '{2}' includes dynamic content that is not enclosed in single quotes. Enquoting dynamic content makes it easier to spot, and single quotes give a less cluttered appearance.. + /// + internal static string SARIF2015_EnquoteDynamicMessageContent_Note_Default_Text { + get { + return ResourceManager.GetString("SARIF2015_EnquoteDynamicMessageContent_Note_Default_Text", resourceCulture); } } } diff --git a/src/Sarif.Multitool/Rules/RuleResources.resx b/src/Sarif.Multitool/Rules/RuleResources.resx index acca56dbb..43d28bfb1 100644 --- a/src/Sarif.Multitool/Rules/RuleResources.resx +++ b/src/Sarif.Multitool/Rules/RuleResources.resx @@ -117,106 +117,247 @@ System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 - - {0}: The name and id properties of rule "{1}" are the same. + + {0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If they are identical, the tool must omit the 'name' property. - - Do not use the same string for a rule's id and name properties. The id property must be a stable, opaque identifer such as "SARIF1001". The name property should be a string that is understandable to an end user, such as "DoNotUserFriendlyNameAsRuleId". + + The two identity-related properties of a SARIF rule must be consistent. The required 'id' property must be a "stable, opaque identifier" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If both 'name' and 'id' are opaque identifiers, omit the 'name' property. If both 'name' and 'id' are human-readable identifiers, then consider assigning an opaque identifier to each rule, but in the meantime, omit the 'name' property. - - {0}: The string "{1}" is not a valid URI reference. + + {0}: The string '{1}' is not a valid URI reference. URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). - - Specify a valid URI reference for every URI-valued property. + + Specify a valid URI reference for every URI-valued property. + +URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, 'file' URIs must not include '..' segments. If symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. + + + {0}: The 'endTimeUtc' value '{1}' precedes the 'startTimeUtc' value '{2}'. The properties of an 'invocation' object must be internally consistent. + + + The properties of an 'invocation' object must be consistent. + +If the 'invocation' object specifies both 'startTimeUtc' and 'endTimeUtc', then 'endTimeUtc' must not precede 'startTimeUtc'. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal. + + + {0}: In rule '{1}', the message with id '{2}' does not end in a period. Express plain text rule messages as complete sentences. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary. + + + Express plain text result messages as complete sentences and end each sentence with a period. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary. + +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also `SARIF2014.ProvideDynamicMessageContent` and `SARIF2015.EnquoteDynamicMessageContent`. + + + {0}: This 'artifactLocation' object has a 'uriBaseId' property '{1}', but its 'uri' property '{2}' is an absolute URI. Since the purpose of 'uriBaseId' is to resolve a relative reference to an absolute URI, it is not allowed when the 'uri' property is already an absolute URI. + + + When using the 'uriBaseId' property, obey the requirements in the SARIF specification [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) that enable it to fulfill its purpose of resolving relative references to absolute locations. In particular: + +If an 'artifactLocation' object has a 'uriBaseId' property, its 'uri' property must be a relative reference, because if 'uri' is an absolute URI then 'uriBaseId' serves no purpose. + +Every URI reference in 'originalUriBaseIds' must resolve to an absolute URI in the manner described in the SARIF specification [3.14.14](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317498). + + + {0}: The value of this property is required to be an absolute URI, but '{1}' is a relative URI reference. + + + Certain URIs are required to be absolute. For the most part, these are URIs that refer to http addresses, such as work items or rule help topics. + + + Ensure consistency among the properties of a 'physicalLocation' object. + +A SARIF 'physicalLocation' object has two related properties 'region' and 'contextRegion'. If 'contextRegion' is present, then 'region' must also be present, and 'contextRegion' must be a "proper superset" of 'region'. That is, 'contextRegion' must completely contain 'region', and it must be larger than 'region'. To understand why this is so we must understand the roles of the 'region' and 'contextRegion' properties. + +'region' allows both users and tools to distinguish similar results within the same artifact. If a SARIF viewer has access to the artifact, it can display it, and highlight the location identified by the analysis tool. If the region has a 'snippet' property, then even if the viewer doesn't have access to the artifact (which might be the case for a web-based viewer), it can still display the faulty code. + +'contextRegion' provides users with a broader view of the result location. Typically, it consists of a range starting a few lines before 'region' and ending a few lines after. Again, if a SARIF viewer has access to the artifact, it can display it, and highlight the context region (perhaps in a lighter shade than the region itself). This isn't terribly useful since the user can already see the whole file, with the 'region' already highlighted. But if 'contextRegion' has a 'snippet' property, then even a viewer without access to the artifact can display a few lines of code surrounding the actual result, which is helpful to users. + +If the validator reports that 'contextRegion' is not a proper superset of 'region', then it's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool should simply omit 'contextRegion'. + + + {0}: This 'physicalLocation' object contains a 'contextRegion' property, but it does not contain a 'region' property. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. If a tool associates only one region with a result, it must populate 'region', not 'contextRegion'. + + + {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' has fewer than {5} elements. An index-valued properties must be valid for the array that it refers to. + + + If an object contains a property that is used as an array index (an "index-valued property"), then that array must be present and must contain at least "index + 1" elements. + + + {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that does not end with a slash. The trailing slash is required to minimize the likelihood of an error when concatenating URI segments together. + + + {0}: The '{1}' element of 'originalUriBaseIds' has no 'uriBaseId' property, but its 'uri' property '{2}' is not an absolute URI. According to the SARIF specification, every such "top-level" entry in 'originalUriBaseIds' must specify an absolute URI, because the purpose of 'originalUriBaseIds' is to enable the resolution of relative references to absolute URIs. + + + {0}: This result contains both the 'ruleId' property '{1}' and the 'rule.id' property '{2}', but they are not equal. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires that if both of these properties are present, they must be equal. + + + {0}: This result contains neither of the properties 'ruleId' or 'rule.id'. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires at least one of these properties to be present. + + + Every result must contain at least one of the properties 'ruleId' and 'rule.id'. If both are present, they must be equal. See the SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)). + + + {0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the '$schema' property with a URL that refers to the final version of the schema. - - {0}: The array contains multiple objects with the value "{1}" for the algorithm property. + + The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + +The SARIF standard was developed over several years, and many intermediate versions of the schema were produced. Now that the standard is final, only the OASIS standard version of the schema is valid. + + + {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' does not exist. An index-valued property always refers to an array, so the array must be present. + + + A SARIF log file should contain, on the root object, a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + + + {0}: The SARIF log file does not contain a '$schema' property. Add a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. - - In any given "file" object, every element of the "hashes" array must have a different value for its "algorithm" property. + + {0}: In this 'region' object, the 'endColumn' property '{1}' is less than the 'startColumn' property '{2}'. The properties of a 'region' object must be internally consistent. + + + {0}: In this 'region' object, the 'endLine' property '{1}' is less than the 'startLine' property '{2}'. The properties of a 'region' object must be internally consistent. + + + The properties of a 'region' object must be consistent. + +SARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: with line and column numbers, with a character offset and count, or with a byte offset and count. The specification states certain constraints on these properties, both within each property group (for example, the start line cannot be greater than end line) and between the groups (for example, if more than one group is present, they must independently specify the same portion of the file). See the SARIF specification ([3.30](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317685)). - - {0}: The end time "{1}" is before the start time "{2}". + + {0}: This 'physicalLocation' object contains both a 'region' and a 'contextRegion' property, but 'contextRegion' is not a proper superset of 'region'. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. It's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool must omit 'contextRegion'. - - The end time of a run must not precede the start time. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal. + + Provide information that makes it easy to identify the name and version of your tool. + +The tool's 'name' property should be no more than three words long. This makes it easy to remember and allows it to fit into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property. + +The tool should provide either or both of the 'version' and 'semanticVersion' properties. This enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions. + +If 'version' is used, facilitate comparison between versions by specifying a version number that starts with an integer, optionally followed by any desired characters. - - {0}: The message "{1}" does not end with a period. + + {0}: The tool name '{1}' contains {2} words, which is more than the recommended maximum of {3} words. A short tool name is easy to remember and fits into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property. - - Messages should consist of one or more complete sentences, ending with a period. + + {0}: The tool '{1}' provides neither a 'version' property nor a 'semanticVersion' property. Providing a version enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions. - - {0}: The value of the "step" property should be {1}, but is {2}. The step values must form a 1-based sequence starting at the first threadFlowLocation in the thread flow. + + {0}: The tool '{1}' contains the 'version' property '{2}', which is not numeric. To facilitate comparison between versions, specify a 'version' that starts with an integer, optionally followed by any desired characters. - - {0}: The "step" property is absent. The "step" property is present on some but not all threadFlowLocations in this threadFlow. If the "step" property is used on any threadFlowLocation in a threadFlow, it must be present on every threadFlowLocation in that threadFlow. + + {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. - - If the "step" property is used on any threadFlowLocation in a threadFlow, then it must be present for every threadFlowLocation in the threadFlow, its value must be 1 for the first threadFlowLocation, and its value must increase by 1 for each succeeding threadFlowLocation. + + {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a query or a fragment. This is not valid because the purpose of the 'uriBaseId' property is to help resolve a relative reference to an absolute URI by concatenating the relative reference to the absolute base URI. This won't work if the base URI contains a query or a fragment. - - {0}: The value of the "endLine" property is {1}, which is less than the value of the "startLine" property, which is {2}. + + {0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it. - - The "endLine" property of a region object must not be less than the "startLine" property. + + Adopt uniform naming conventions for rule ids. + +Many tools follow a conventional format for the 'reportingDescriptor.id' property: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001' for a diagnostic from the Roslyn C# compiler. For uniformity of experience across tools, we recommend this format. - - {0}: The value of the "endColumn" property is {1}, which is less than the value of the "startColumn" property, which is {2}. + + {0}: The 'id' property of the rule '{1}' does not follow the recommended format: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001'. Using a conventional format for the rule id provides a more uniform experience across tools. - - The "endColumn" property of a region object must not be less than the "startColumn" property. + + {0}: This 'region' object does not specify 'startLine', 'charOffset', or 'byteOffset'. As a result, it is impossible to determine whether this 'region' object describes a line/column text region, a character offset/length text region, or a binary region. - - {0}: This fileLocation object contains a "uriBaseId" property, which means that the value of the "uri" property must be a relative URI reference, but "{1}" is an absolute URI reference. + + Emit arrays only if they provide additional information. + +In several parts of a SARIF log file, a subset of information about an object appears in one place, and the full information describing all such objects appears in an array elsewhere in the log file. For example, each 'result' object has a 'ruleId' property that identifies the rule that was violated. Elsewhere in the log file, the array 'run.tool.driver.rules' contains additional information about the rules. But if the elements of the 'rules' array contained no information about the rules beyond their ids, then there might be no reason to include the 'rules' array at all, and the log file could be made smaller simply by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information. + +Similarly, most 'result' objects contain at least one 'artifactLocation' object. Elsewhere in the log file, the array 'run.artifacts' contains additional information about the artifacts that were analyzed. But if the elements of the 'artifacts' array contained not information about the artifacts beyond their locations, then there might be no reason to include the 'artifacts' array at all, and again the log file could be made smaller by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information. - - If a fileLocation object contains a "uriBaseId" property, the value of the "uri" property must be a relative URI reference. + + {0}: The 'artifacts' array contains no information beyond the locations of the artifacts. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information. - - {0}: The value of this property is required to be an absolute URI, but "{1}" is a relative URI reference. + + In result messages, use the 'message.id' and 'message.arguments' properties rather than 'message.text'. This has several advantages. If 'text' is lengthy, using 'id' and 'arguments' makes the SARIF file smaller. If the rule metadata is stored externally to the SARIF log file, the message text can be improved (for example, by adding more text, clarifying the phrasing, or fixing typos), and the result messages will pick up the improvements the next time it is displayed. Finally, SARIF supports localizing messages into different languages, which is possible if the SARIF file contains 'message.id' and 'message.arguments', but not if it contains 'message.text' directly. - - Certain URIs are required to be absolute. + + {0}: The 'message' property of this result contains a 'text' property. Consider replacing it with 'id' and 'arguments' properties. This potentially reduces the log file size, allows the message text to be improved without modifying the log file, and enables localization. - - If the "contextRegion" property is present, the "region" property must also be present. + + Provide 'versionControlProvenance' to record which version of the code was analyzed, and to enable paths to be expressed relative to the root of the repository. - - {0}: This "physicalLocation" object contains a "contextRegion" property, but it does not contain a "region" property. + + {0}: This run does not provide 'versionControlProvenance'. As a result, it is not possible to determine which version of code was analyzed, nor to map relative paths to their locations within the repository. - - {0}: This "{1}" object contains a property "{2}" with value {3}, but either "{4}" is absent, or it has fewer than {5} elements. + + {0}: The 'rules' array contains no information beyond the ids of the rules. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information. - - If an object contains a property that is used as an array index, then that array must be present and must contain at least "index + 1" elements. + + URIs that refer to locations such as rule help pages and result-related work items should be reachable via an HTTP GET request. - - In the artifactLocation objects contained in run.originalUriBaseIds, if uriBaseId is absent, then uri must either be an absolute URI or it must be absent. Also, uri must end with a slash, so that it can safely be combined with the relative URIs in artifactLocation objects elsewhere in the log file. + + {0}: The URI '{1}' was not reachable via an HTTP GET request. - - {0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds does not end with a slash. + + Provide information that makes it possible to determine the repo-relative locations of files that contain analysis results. + +Each element of the 'versionControlProvenance' array is a 'versionControlDetails' object that describes a repository containing files that were analyzed. 'versionControlDetails.mappedTo' defines the file system location to which the root of that repository is mapped. If 'mappedTo.uriBaseId' is present, and if result locations are expressed relative to that 'uriBaseId', then the repo-relative location of each result can be determined. - - {0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds is not an absolute URI. + + {0}: The 'versionControlDetails' object that describes the repository '{1}' does not provide 'mappedTo.uriBaseId'. As a result, it will not be possible to determine the repo-relative location of files containing analysis results for this repository. - - {0}: The result contains both the ruleId property '{1}' and the rule.id property '{2}', and they are not equal. + + Provide code snippets to enable users to see the code that triggered each result, even if they are not enlisted in the code. + + + {0}: The 'region' object in this result location does not provide a 'snippet' property. Providing a code snippet enables users to see the code that triggered the result, even if they are not enlisted in the code. + + + Provide context regions to enable users to see a portion of the code that surrounds each result, even if they are not enlisted in the code. + + + {0}: This result location does not provide a 'contextRegion' property. Providing a context region enables users to see a portion of the code that surrounds the result, even if they are not enlisted in the code. + + + {0}: This message object refers to the message with id '{1}' in rule '{2}', but that rule does not define a message with that id. When a tool creates a result message that uses the 'id' property, it must ensure that the specified rule actually has a message with that id. + + + {0}: The message with id '{1}' in rule '{2}' requires '{3}' arguments, but the 'arguments' array in this message object has only '{4}' element(s). When a tool creates a result message that use the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain 4 elements. + + + The properties of a result's 'message' property must be consistent with the properties of the rule that the result refers to. + +When a result's 'message' object uses the 'id' and 'arguments' properties (which, by the way, is recommended: see SARIF2002.ProvideMessageArguments), it must ensure that the rule actually defines a message string with that id, and that 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain at least 4 elements. + + + {0}: This run does not provide embedded file content. Providing embedded file content enables users to examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it. + + + Include "dynamic content" (information that varies among results from the same rule) to makes your messages more specific, and to avoid the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable. + +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2015.EnquoteDynamicMessageContent'. + + + {0}: In rule '{1}', the message with id '{2}' does not include any dynamic content. Dynamic content makes your messages more specific and avoids the "wall of bugs" phenomenon, where hundreds of occurrences of the same message appear unapproachable. + + + Place dynamic content in single quotes to set it off from the static text and to make it easier to spot. It's especially helpful when the dynamic content is a string that might contain spaces, and most especially when the string might be empty (and so would be invisible if it weren't for the quotes). We recommend single quotes for a less cluttered appearance, even though US English usage would require double quotes. + +This is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2014.ProvideDynamicMessageContent'. - - {0}: The result contains neither result.ruleId nor result.rule.id. + + {0}: In rule '{1}', the message with id '{2}' includes dynamic content that is not enclosed in single quotes. Enquoting dynamic content makes it easier to spot, and single quotes give a less cluttered appearance. - - In every result, at least one of the properties result.ruleId and result.rule.id must be present. If both are present, they must be equal. + + For each rule, provide a URI where users can find detailed information about the rule. This information should include a detailed description of the invalid pattern, an explanation of why the pattern is poor practice (particularly in contexts such as security or accessibility where driving considerations might not be readily apparent), guidance for resolving the problem (including describing circumstances in which ignoring the problem altogether might be appropriate), examples of invalid and valid patterns, and special considerations (such as noting when a violation should never be ignored or suppressed, noting when a violation could cause downstream tool noise, and noting when a rule can be configured in some way to refine or alter the analysis). - - {0}: The $schema property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the $schema property with a URL that refers to the final version of the schema. + + {0}: The rule '{1}' does not provide a help URI. Providing a URI where users can find detailed information about the rule helps users to understand the result and how they can best address it. - - The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + + Provide embedded file content so that users can examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it. - - {0}: The SARIF log file does not contain a $schema property. Add a $schema property that refers to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files. + + {0}: This result location does not provide any of the 'uriBaseId' values that specify repository locations: '{1}'. As a result, it will not be possible to determine the location of the file containing this result relative to the root of the repository that contains it. \ No newline at end of file diff --git a/src/Sarif.Multitool/Rules/SARIF1001.DoNotUseFriendlyNameAsRuleId.cs b/src/Sarif.Multitool/Rules/SARIF1001.DoNotUseFriendlyNameAsRuleId.cs deleted file mode 100644 index 4a40628a6..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1001.DoNotUseFriendlyNameAsRuleId.cs +++ /dev/null @@ -1,43 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System; -using System.Collections.Generic; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class DoNotUseFriendlyNameAsRuleId : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1001_DoNotUseFriendlyNameAsRuleIdDescription - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Warning; - - /// - /// SARIF1001 - /// - public override string Id => RuleId.DoNotUseFriendlyNameAsRuleId; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1001_Default) - }; - - protected override void Analyze(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) - { - if (reportingDescriptor.Id != null && - reportingDescriptor.Name != null && - reportingDescriptor.Id.Equals(reportingDescriptor.Name, StringComparison.OrdinalIgnoreCase)) - { - LogResult( - reportingDescriptorPointer, - nameof(RuleResources.SARIF1001_Default), - reportingDescriptor.Id); - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1001.RuleIdentifiersMustBeValid.cs b/src/Sarif.Multitool/Rules/SARIF1001.RuleIdentifiersMustBeValid.cs new file mode 100644 index 000000000..2f369529b --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1001.RuleIdentifiersMustBeValid.cs @@ -0,0 +1,57 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class RuleIdentifiersMustBeValid : SarifValidationSkimmerBase + { + /// + /// SARIF2001 + /// + public override string Id => RuleId.RuleIdentifiersMustBeValid; + + /// + /// The two identity-related properties of a SARIF rule must be consistent. The required 'id' + /// property must be a "stable, opaque identifier" (the SARIF specification + /// ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) + /// explains the reasons for this). The optional 'name' property + /// ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) + /// is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' + /// are present, they must be different. If both 'name' and 'id' are opaque identifiers, + /// omit the 'name' property. If both 'name' and 'id' are human-readable identifiers, then + /// consider assigning an opaque identifier to each rule, but in the meantime, omit the 'name' + /// property. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1001_RuleIdentifiersMustBeValid_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1001_RuleIdentifiersMustBeValid_Error_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + protected override void Analyze(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) + { + if (reportingDescriptor.Id != null && + reportingDescriptor.Name != null && + reportingDescriptor.Id.Equals(reportingDescriptor.Name, StringComparison.OrdinalIgnoreCase)) + { + // {0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. + // The required 'id' property must be a "stable, opaque identifier" (the SARIF specification + // ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) + // explains the reasons for this). The optional 'name' property + // ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) + // is an identifer that is understandable to an end user. Therefore if both 'id' and + // 'name' are present, they must be different. If they are identical, the tool must + // omit the 'name' property. + LogResult( + reportingDescriptorPointer, + nameof(RuleResources.SARIF1001_RuleIdentifiersMustBeValid_Error_Default_Text), + reportingDescriptor.Id); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1003.UrisMustBeValid.cs b/src/Sarif.Multitool/Rules/SARIF1002.UrisMustBeValid.cs similarity index 54% rename from src/Sarif.Multitool/Rules/SARIF1003.UrisMustBeValid.cs rename to src/Sarif.Multitool/Rules/SARIF1002.UrisMustBeValid.cs index a599c7429..918d2cac5 100644 --- a/src/Sarif.Multitool/Rules/SARIF1003.UrisMustBeValid.cs +++ b/src/Sarif.Multitool/Rules/SARIF1002.UrisMustBeValid.cs @@ -4,31 +4,35 @@ using System; using System.Collections.Generic; using System.Linq; + using Microsoft.Json.Pointer; namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules { public class UrisMustBeValid : SarifValidationSkimmerBase { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1003_UrisMustBeValid - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - /// - /// SARIF1003 + /// SARIF1002 /// public override string Id => RuleId.UrisMustBeValid; - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1003_Default) + /// + /// Specify a valid URI reference for every URI-valued property. + /// + /// URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, + /// 'file' URIs must not include '..' segments. If symbolic links are present, '..' + /// might have different meanings on the machine that produced the log file and the + /// machine where an end user or a tool consumes it. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1002_UrisMustBeValid_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1002_UrisMustBeValid_Error_UrisMustConformToRfc3986_Text), + nameof(RuleResources.SARIF1002_UrisMustBeValid_Error_FileUrisMustNotIncludeDotDotSegments_Text) }; + public override FailureLevel DefaultLevel => FailureLevel.Error; + protected override void Analyze(SarifLog log, string logPointer) { AnalyzeUri(log.SchemaUri, logPointer.AtProperty(SarifPropertyName.Schema)); @@ -58,19 +62,6 @@ protected override void Analyze(ReportingDescriptor reportingDescriptor, string AnalyzeUri(reportingDescriptor.HelpUri, messageDescriptorPointer.AtProperty(SarifPropertyName.HelpUri)); } - protected override void Analyze(Run run, string runPointer) - { - if (run.OriginalUriBaseIds != null) - { - string originalUriBaseIdsPointer = runPointer.AtProperty(SarifPropertyName.OriginalUriBaseIds); - - foreach (string key in run.OriginalUriBaseIds.Keys) - { - AnalyzeUri(run.OriginalUriBaseIds[key].Uri, originalUriBaseIdsPointer.AtProperty(key).AtProperty(SarifPropertyName.Uri)); - } - } - } - protected override void Analyze(ToolComponent toolComponent, string toolComponentPointer) { AnalyzeUri(toolComponent.DownloadUri, toolComponentPointer.AtProperty(SarifPropertyName.DownloadUri)); @@ -83,16 +74,32 @@ protected override void Analyze(VersionControlDetails versionControlDetails, str private void AnalyzeUri(Uri uri, string pointer) { - AnalyzeUri(uri?.OriginalString, pointer); - } - - private void AnalyzeUri(string uri, string pointer) - { - if (uri != null) + string uriString = uri?.OriginalString; + if (uriString != null) { - if (!Uri.IsWellFormedUriString(uri, UriKind.RelativeOrAbsolute)) + if (!Uri.IsWellFormedUriString(uriString, UriKind.RelativeOrAbsolute)) + { + // {0}: The string '{1}' is not a valid URI reference. URIs must conform to + // [RFC 3986](https://tools.ietf.org/html/rfc3986). + LogResult( + pointer, + nameof(RuleResources.SARIF1002_UrisMustBeValid_Error_UrisMustConformToRfc3986_Text), + uriString); + } + + if (uri.IsAbsoluteUri && uri.IsFile) { - LogResult(pointer, nameof(RuleResources.SARIF1003_Default), uri); + if (uriString.Split('/').Any(x => x.Equals(".."))) + { + // {0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because + // if symbolic links are present, '..' might have different meanings on the + // machine that produced the log file and the machine where an end user or + // a tool consumes it. + LogResult( + pointer, + nameof(RuleResources.SARIF1002_UrisMustBeValid_Error_FileUrisMustNotIncludeDotDotSegments_Text), + uriString); + } } } } diff --git a/src/Sarif.Multitool/Rules/SARIF1004.ExpressUriBaseIdsCorrectly.cs b/src/Sarif.Multitool/Rules/SARIF1004.ExpressUriBaseIdsCorrectly.cs new file mode 100644 index 000000000..466883959 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1004.ExpressUriBaseIdsCorrectly.cs @@ -0,0 +1,141 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Linq; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ExpressUriBaseIdsCorrectly : SarifValidationSkimmerBase + { + /// + /// SARIF1004 + /// + public override string Id => RuleId.ExpressUriBaseIdsCorrectly; + + /// + /// When using the 'uriBaseId' property, obey the requirements in the SARIF specification + /// [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) + /// that enable it to fulfill its purpose of resolving relative references to absolute locations. + /// In particular: + /// + /// If an 'artifactLocation' object has a 'uriBaseId' property, its 'uri' property must be a + /// relative reference, because if 'uri' is an absolute URI then 'uriBaseId' serves no purpose. + /// + /// Every URI reference in 'originalUriBaseIds' must resolve to an absolute URI in the manner + /// described in the SARIF specification + /// [3.14.14] (https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317498). + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdRequiresRelativeUri_Text), + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_TopLevelUriBaseIdMustBeAbsolute_Text), + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustEndWithSlash_Text), + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainDotDotSegment_Text), + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainQueryOrFragment_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Error; + + protected override void Analyze(ArtifactLocation artifactLocation, string artifactLocationPointer) + { + if (artifactLocation.UriBaseId != null && artifactLocation.Uri.IsAbsoluteUri) + { + // {0}: This 'artifactLocation' object has a 'uriBaseId' property '{1}', but its + // 'uri' property '{2}' is an absolute URI. Since the purpose of 'uriBaseId' is + // to resolve a relative reference to an absolute URI, it is not allowed when + // the 'uri' property is already an absolute URI. + LogResult( + artifactLocationPointer, + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdRequiresRelativeUri_Text), + artifactLocation.UriBaseId, + artifactLocation.Uri.OriginalString); + } + } + + protected override void Analyze(Run run, string runPointer) + { + if (run.OriginalUriBaseIds != null) + { + string originalUriBaseIdsPointer = runPointer.AtProperty(SarifPropertyName.OriginalUriBaseIds); + + foreach (string uriBaseId in run.OriginalUriBaseIds.Keys) + { + AnalyzeOriginalUriBaseIdsEntry(uriBaseId, run.OriginalUriBaseIds[uriBaseId], originalUriBaseIdsPointer.AtProperty(uriBaseId)); + } + } + } + + private void AnalyzeOriginalUriBaseIdsEntry(string uriBaseId, ArtifactLocation artifactLocation, string pointer) + { + if (artifactLocation.Uri == null) { return; } + + // If it's not a well-formed URI of _any_ kind, then don't bother triggering this rule. + // Rule SARIF1003, UrisMustBeValid, will catch it. + // Check for well-formedness first, before attempting to create a Uri object, to + // avoid having to do a try/catch. Unfortunately Uri.TryCreate will return true + // even for a malformed URI string. + string uriString = artifactLocation.Uri.OriginalString; + if (uriString != null && Uri.IsWellFormedUriString(uriString, UriKind.RelativeOrAbsolute)) + { + var uri = new Uri(uriString, UriKind.RelativeOrAbsolute); + + if (artifactLocation.UriBaseId == null && !uri.IsAbsoluteUri) + { + // {0}: The '{1}' element of 'originalUriBaseIds' has no 'uriBaseId' property, but its 'uri' + // property '{2}' is not an absolute URI. According to the SARIF specification, every such + // "top-level" entry in 'originalUriBaseIds' must specify an absolute URI, because the purpose + // of 'originalUriBaseIds' is to enable the resolution of relative references to absolute URIs. + LogResult( + pointer, + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_TopLevelUriBaseIdMustBeAbsolute_Text), + uriBaseId, + uriString); + } + + if (!uriString.EndsWith("/")) + { + // {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that does not + // end with a slash. The trailing slash is required to minimize the likelihood of an error + // when concatenating URI segments together. + LogResult( + pointer, + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustEndWithSlash_Text), + uriBaseId, + uriString); + } + + if (uriString.Split('/').Any(x => x.Equals(".."))) + { + // {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains + // a '..' segment. This is dangerous because if symbolic links are present, '..' might have + // different meanings on the machine that produced the log file and the machine where an end + // user or a tool consumes it. + LogResult( + pointer, + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainDotDotSegment_Text), + uriBaseId, + uriString); + } + + if (uri.IsAbsoluteUri && (!string.IsNullOrEmpty(uri.Fragment) || !string.IsNullOrEmpty(uri.Query))) + { + // {0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a + // query or a fragment. This is not valid because the purpose of the 'uriBaseId' property is + // to help resolve a relative reference to an absolute URI by concatenating the relative + // reference to the absolute base URI. This won't work if the base URI contains a query or a + // fragment. + LogResult( + pointer, + nameof(RuleResources.SARIF1004_ExpressUriBaseIdsCorrectly_Error_UriBaseIdValueMustNotContainQueryOrFragment_Text), + uriBaseId, + uriString); + } + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1015.UriMustBeAbsolute.cs b/src/Sarif.Multitool/Rules/SARIF1005.UriMustBeAbsolute.cs similarity index 80% rename from src/Sarif.Multitool/Rules/SARIF1015.UriMustBeAbsolute.cs rename to src/Sarif.Multitool/Rules/SARIF1005.UriMustBeAbsolute.cs index 867891dd8..7c31c90db 100644 --- a/src/Sarif.Multitool/Rules/SARIF1015.UriMustBeAbsolute.cs +++ b/src/Sarif.Multitool/Rules/SARIF1005.UriMustBeAbsolute.cs @@ -4,31 +4,30 @@ using System; using System.Collections.Generic; using System.Linq; + using Microsoft.Json.Pointer; namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules { public class UriMustBeAbsolute : SarifValidationSkimmerBase { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1015_UriMustBeAbsolute - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - /// - /// SARIF1015 + /// SARIF1005 /// public override string Id => RuleId.UriMustBeAbsolute; - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1015_Default) + /// + /// Certain URIs are required to be absolute. For the most part, these are URIs that refer to http + /// addresses, such as work items or rule help topics. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1005_UriMustBeAbsolute_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1005_UriMustBeAbsolute_Error_Default_Text) }; + public override FailureLevel DefaultLevel => FailureLevel.Error; + protected override void Analyze(SarifLog log, string logPointer) { AnalyzeUri(log.SchemaUri, logPointer.AtProperty(SarifPropertyName.Schema)); @@ -82,7 +81,11 @@ private void AnalyzeUri(string uriString, string pointer) Uri uri = new Uri(uriString, UriKind.RelativeOrAbsolute); if (!uri.IsAbsoluteUri) { - LogResult(pointer, nameof(RuleResources.SARIF1015_Default), uriString); + // {0}: The value of this property is required to be an absolute URI, but '{1}' is a relative URI reference. + LogResult( + pointer, + nameof(RuleResources.SARIF1005_UriMustBeAbsolute_Error_Default_Text), + uriString); } } } diff --git a/src/Sarif.Multitool/Rules/SARIF1007.EndTimeMustNotBeBeforeStartTime.cs b/src/Sarif.Multitool/Rules/SARIF1006.InvocationPropertiesMustBeConsistent.cs similarity index 54% rename from src/Sarif.Multitool/Rules/SARIF1007.EndTimeMustNotBeBeforeStartTime.cs rename to src/Sarif.Multitool/Rules/SARIF1006.InvocationPropertiesMustBeConsistent.cs index d7a77b78d..f651213c4 100644 --- a/src/Sarif.Multitool/Rules/SARIF1007.EndTimeMustNotBeBeforeStartTime.cs +++ b/src/Sarif.Multitool/Rules/SARIF1006.InvocationPropertiesMustBeConsistent.cs @@ -4,31 +4,34 @@ using System; using System.Collections.Generic; using System.Globalization; + using Microsoft.Json.Pointer; namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules { - public class EndTimeMustNotBeBeforeStartTime : SarifValidationSkimmerBase + public class InvocationPropertiesMustBeConsistent : SarifValidationSkimmerBase { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1007_EndTimeMustNotBeBeforeStartTime - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; + /// + /// SARIF1006 + /// + public override string Id => RuleId.InvocationPropertiesMustBeConsistent; /// - /// SARIF1007 + /// The properties of an 'invocation' object must be consistent. + /// + /// If the 'invocation' object specifies both 'startTimeUtc' and 'endTimeUtc', then 'endTimeUtc' + /// must not precede 'startTimeUtc'. To allow for the possibility that the duration of the run + /// is less than the resolution of the string representation of the time, the start time and the + /// end time may be equal. /// - public override string Id => RuleId.EndTimeMustNotBeBeforeStartTime; + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1006_InvocationPropertiesMustBeConsistent_FullDescription_Text }; - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1007_Default) + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1006_InvocationPropertiesMustBeConsistent_Error_EndTimeMustNotPrecedeStartTime_Text) }; + public override FailureLevel DefaultLevel => FailureLevel.Error; + protected override void Analyze(Invocation invocation, string invocationPointer) { // Compare the start and end times only if both are present. @@ -38,9 +41,11 @@ protected override void Analyze(Invocation invocation, string invocationPointer) { string endTimePointer = invocationPointer.AtProperty(SarifPropertyName.EndTimeUtc); + // {0}: The 'endTimeUtc' value '{1}' precedes the 'startTimeUtc' value '{2}'. + // The properties of an 'invocation' object must be internally consistent. LogResult( endTimePointer, - nameof(RuleResources.SARIF1007_Default), + nameof(RuleResources.SARIF1006_InvocationPropertiesMustBeConsistent_Error_EndTimeMustNotPrecedeStartTime_Text), FormatDateTime(invocation.EndTimeUtc), FormatDateTime(invocation.StartTimeUtc)); } diff --git a/src/Sarif.Multitool/Rules/SARIF1007.RegionPropertiesMustBeConsistent.cs b/src/Sarif.Multitool/Rules/SARIF1007.RegionPropertiesMustBeConsistent.cs new file mode 100644 index 000000000..ace4324e5 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1007.RegionPropertiesMustBeConsistent.cs @@ -0,0 +1,95 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +using Microsoft.Json.Pointer; + +using Newtonsoft.Json.Linq; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class RegionPropertiesMustBeConsistent : SarifValidationSkimmerBase + { + /// + /// SARIF1007 + /// + public override string Id => RuleId.RegionPropertiesMustBeConsistent; + + /// + /// The properties of a 'region' object must be consistent. + /// + /// SARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: + /// with line and column numbers, with a character offset and count, or with a byte offset + /// and count.The specification states certain constraints on these properties, both within + /// each property group (for example, the start line cannot be greater than end line) and + /// between the groups(for example, if more than one group is present, they must independently + /// specify the same portion of the file). See the SARIF specification + /// ([3.30] (https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317685)). + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_EndLineMustNotPrecedeStartLine_Text), + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_EndColumnMustNotPrecedeStartColumn_Text), + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_RegionStartPropertyMustBePresent_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Error; + + protected override void Analyze(Region region, string regionPointer) + { + var jsonPointer = new JsonPointer(regionPointer); + JToken regionToken = jsonPointer.Evaluate(Context.InputLogToken); + + if (!region.IsBinaryRegion && + !region.IsLineColumnBasedTextRegion && + !region.IsOffsetBasedTextRegion) + { + // {0}: This 'region' object does not specify 'startLine', 'charOffset', or 'byteOffset'. + // As a result, it is impossible to determine whether this 'region' object describes + // a line/column text region, a character offset/length text region, or a binary region. + LogResult( + regionPointer, + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_RegionStartPropertyMustBePresent_Text)); + } + + if (regionToken.HasProperty(SarifPropertyName.EndLine) && + region.EndLine < region.StartLine) + { + string endLinePointer = regionPointer.AtProperty(SarifPropertyName.EndLine); + + // {0}: In this 'region' object, the 'endLine' property '{1}' is less than the 'startLine' + // property '{2}'. The properties of a 'region' object must be internally consistent. + LogResult( + endLinePointer, + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_EndLineMustNotPrecedeStartLine_Text), + region.EndLine.ToInvariantString(), + region.StartLine.ToInvariantString()); + } + + + if (RegionIsOnOneLine(region, regionToken) && + regionToken.HasProperty(SarifPropertyName.EndColumn) && + region.EndColumn < region.StartColumn) + { + string endColumnPointer = regionPointer.AtProperty(SarifPropertyName.EndColumn); + + // {0}: In this 'region' object, the 'endColumn' property '{1}' is less than the 'startColumn' + // property '{2}'. The properties of a 'region' object must be internally consistent. + LogResult( + endColumnPointer, + nameof(RuleResources.SARIF1007_RegionPropertiesMustBeConsistent_Error_EndColumnMustNotPrecedeStartColumn_Text), + region.EndColumn.ToInvariantString(), + region.StartColumn.ToInvariantString()); + } + } + + private static bool RegionIsOnOneLine(Region region, JToken regionToken) + { + return regionToken.HasProperty(SarifPropertyName.EndLine) + ? region.StartLine == region.EndLine + : true; + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1008.MessagesShouldEndWithPeriod.cs b/src/Sarif.Multitool/Rules/SARIF1008.MessagesShouldEndWithPeriod.cs deleted file mode 100644 index 2928999bc..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1008.MessagesShouldEndWithPeriod.cs +++ /dev/null @@ -1,92 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System; -using System.Collections.Generic; -using Microsoft.Json.Pointer; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class MessagesShouldEndWithPeriod : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1008_MessagesShouldEndWithPeriod - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Warning; - - /// - /// SARIF1008 - /// - public override string Id => RuleId.MessagesShouldEndWithPeriod; - - protected override IEnumerable MessageResourceNames - { - get - { - return new string[] - { - nameof(RuleResources.SARIF1008_Default) - }; - } - } - - protected override void Analyze(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) - { - AnalyzeMessageStrings(reportingDescriptor.MessageStrings, reportingDescriptorPointer); - } - - private void AnalyzeMessageStrings( - IDictionary messageStrings, - string reportingDescriptorPointer) - { - if (messageStrings != null) - { - string messageStringsPointer = reportingDescriptorPointer.AtProperty(SarifPropertyName.MessageStrings); - - foreach (string key in messageStrings.Keys) - { - MultiformatMessageString messageString = messageStrings[key]; - - string messageStringPointer = messageStringsPointer.AtProperty(key); - - AnalyzeMessageString(messageString.Text, messageStringPointer, SarifPropertyName.Text); - } - } - } - - protected override void Analyze(MultiformatMessageString multiformatMessageString, string multiformatMessageStringPointer) - { - AnalyzeMessageString(multiformatMessageString.Text, multiformatMessageStringPointer, SarifPropertyName.Text); - } - - protected override void Analyze(Message message, string messagePointer) - { - AnalyzeMessageString(message.Text, messagePointer, SarifPropertyName.Text); - } - - private void AnalyzeMessageString( - string messageString, - string messagePointer, - string propertyName) - { - if (!string.IsNullOrEmpty(messageString) && DoesNotEndWithPeriod(messageString)) - { - string textPointer = messagePointer.AtProperty(propertyName); - - LogResult( - textPointer, - nameof(RuleResources.SARIF1008_Default), - messageString); - } - } - - private static bool DoesNotEndWithPeriod(string message) - { - return message != null && !message.EndsWith(".", StringComparison.Ordinal); - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1008.PhysicalLocationPropertiesMustBeConsistent.cs b/src/Sarif.Multitool/Rules/SARIF1008.PhysicalLocationPropertiesMustBeConsistent.cs new file mode 100644 index 000000000..ef1fbedf3 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1008.PhysicalLocationPropertiesMustBeConsistent.cs @@ -0,0 +1,86 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class PhysicalLocationPropertiesMustBeConsistent : SarifValidationSkimmerBase + { + /// + /// SARIF1008 + /// + public override string Id => RuleId.PhysicalLocationPropertiesMustBeConsistent; + + /// + /// Ensure consistency among the properties of a 'physicalLocation' object. + /// + /// A SARIF 'physicalLocation' object has two related properties 'region' and 'contextRegion'. + /// If 'contextRegion' is present, then 'region' must also be present, and 'contextRegion' + /// must be a "proper superset" of 'region'. That is, 'contextRegion' must completely contain + /// 'region', and it must be larger than 'region'. To understand why this is so we must + /// understand the roles of the 'region' and 'contextRegion' properties. + /// + /// 'region' allows both users and tools to distinguish similar results within the same + /// artifact. If a SARIF viewer has access to the artifact, it can display it, and highlight + /// the location identified by the analysis tool.If the region has a 'snippet' property, + /// then even if the viewer doesn't have access to the artifact (which might be the case + /// for a web-based viewer), it can still display the faulty code. + /// + /// 'contextRegion' provides users with a broader view of the result location. Typically, + /// it consists of a range starting a few lines before 'region' and ending a few lines after. + /// Again, if a SARIF viewer has access to the artifact, it can display it, and highlight + /// the context region (perhaps in a lighter shade than the region itself). This isn't + /// terribly useful since the user can already see the whole file, with the 'region' + /// already highlighted. But if 'contextRegion' has a 'snippet' property, then even a + /// viewer without access to the artifact can display a few lines of code surrounding + /// the actual result, which is helpful to users. + /// + /// If the validator reports that 'contextRegion' is not a proper superset of 'region', + /// then it's possible that the tool reversed 'region' and 'contextRegion'. If 'region' + /// and 'contextRegion' are identical, the tool should simply omit 'contextRegion'. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1008_PhysicalLocationPropertiesMustBeConsistent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionRequiresRegion_Text), + nameof(RuleResources.SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionMustBeProperSupersetOfRegion_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Error; + + protected override void Analyze(PhysicalLocation physicalLocation, string physicalLocationPointer) + { + if (physicalLocation.ContextRegion == null) + { + return; + } + + if (physicalLocation.Region == null) + { + // {0}: This 'physicalLocation' object contains a 'contextRegion' property, but it + // does not contain a 'region' property. This is invalid because the purpose of + // 'contextRegion' is to provide a viewing context around the 'region' which is the + // location of the result. If a tool associates only one region with a result, it + // must populate 'region', not 'contextRegion'. + LogResult( + physicalLocationPointer, + nameof(RuleResources.SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionRequiresRegion_Text)); + return; + } + + if (!physicalLocation.ContextRegion.IsProperSupersetOf(physicalLocation.Region)) + { + // {0}: This 'physicalLocation' object contains both a 'region' and a 'contextRegion' + // property, but 'contextRegion' is not a proper superset of 'region'. This is invalid + // because the purpose of 'contextRegion' is to provide a viewing context around the + // 'region' which is the location of the result. It's possible that the tool reversed + // 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the + // tool must omit 'contextRegion'. + LogResult( + physicalLocationPointer, + nameof(RuleResources.SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Error_ContextRegionMustBeProperSupersetOfRegion_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1017.InvalidIndex.cs b/src/Sarif.Multitool/Rules/SARIF1009.IndexPropertiesMustBeConsistentWithArrays.cs similarity index 75% rename from src/Sarif.Multitool/Rules/SARIF1017.InvalidIndex.cs rename to src/Sarif.Multitool/Rules/SARIF1009.IndexPropertiesMustBeConsistentWithArrays.cs index e1e7a4a80..2e78765c7 100644 --- a/src/Sarif.Multitool/Rules/SARIF1017.InvalidIndex.cs +++ b/src/Sarif.Multitool/Rules/SARIF1009.IndexPropertiesMustBeConsistentWithArrays.cs @@ -5,27 +5,27 @@ namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules { - public class InvalidIndex : SarifValidationSkimmerBase + public class IndexPropertiesMustBeConsistentWithArrays : SarifValidationSkimmerBase { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1017_InvalidIndex - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; + /// + /// SARIF1009 + /// + public override string Id => RuleId.IndexPropertiesMustBeConsistentWithArrays; /// - /// SARIF1017 + /// If an object contains a property that is used as an array index (an "index-valued + /// property"), then that array must be present and must contain at least "index + 1" + /// elements. /// - public override string Id => RuleId.InvalidIndex; + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1009_IndexPropertiesMustBeConsistentWithArrays_FullDescription_Text }; - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1017_Default) + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustExist_Text), + nameof(RuleResources.SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustBeLongEnough_Text) }; + public override FailureLevel DefaultLevel => FailureLevel.Error; + protected override void Analyze(Address address, string addressPointer) { ValidateArrayIndex( @@ -168,11 +168,34 @@ private void ValidateArrayIndex( string propertyName, string arrayName) { + if (index == -1) + { + return; + } + + if (container == null) + { + // {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' does + // not exist. An index-valued property always refers to an array, so the array must + // be present. + LogResult( + jsonPointer, + nameof(RuleResources.SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustExist_Text), + objectName, + propertyName, + index.ToInvariantString(), + arrayName); + return; + } + if (!IndexIsValid(index, container)) { + // {0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' has + // fewer than {5} elements. An index-valued properties must be valid for the array + // that it refers to. LogResult( jsonPointer, - nameof(RuleResources.SARIF1017_Default), + nameof(RuleResources.SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Error_TargetArrayMustBeLongEnough_Text), objectName, propertyName, index.ToInvariantString(), @@ -182,6 +205,6 @@ private void ValidateArrayIndex( } private static bool IndexIsValid(int index, IList container) - => index == -1 || (index >= 0 && container?.Count > index); + => index >= 0 && container.Count > index; } } diff --git a/src/Sarif.Multitool/Rules/SARIF1010.RuleIdMustBeConsistent.cs b/src/Sarif.Multitool/Rules/SARIF1010.RuleIdMustBeConsistent.cs new file mode 100644 index 000000000..2bc02af28 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1010.RuleIdMustBeConsistent.cs @@ -0,0 +1,63 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class RuleIdMustBeConsistent : SarifValidationSkimmerBase + { + /// + /// SARIF1010 + /// + public override string Id => RuleId.RuleIdMustBeConsistent; + + /// + /// Every result must contain at least one of the properties 'ruleId' and 'rule.id'. + /// If both are present, they must be equal. See the SARIF specification ([§3.27.5] + /// (https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)). + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1010_RuleIdMustBeConsistent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1010_RuleIdMustBeConsistent_Error_ResultRuleIdMustBeConsistent_Text), + nameof(RuleResources.SARIF1010_RuleIdMustBeConsistent_Error_ResultMustSpecifyRuleId_Text) + }; + public override FailureLevel DefaultLevel => FailureLevel.Error; + + protected override void Analyze(Result result, string resultPointer) + { + AnalyzeRuleId(result, resultPointer); + } + + private void AnalyzeRuleId(Result result, string pointer) + { + // At least one of result.ruleId or result.rule.id must be present + if (string.IsNullOrWhiteSpace(result.RuleId) && string.IsNullOrWhiteSpace(result.Rule?.Id)) + { + // {0}: This result contains neither of the properties 'ruleId' or 'rule.id'. The SARIF specification + // ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) + // requires at least one of these properties to be present. + LogResult( + pointer, + nameof(RuleResources.SARIF1010_RuleIdMustBeConsistent_Error_ResultMustSpecifyRuleId_Text)); + } + // if both are present, they must be equal. + else if (!string.IsNullOrWhiteSpace(result.RuleId) + && !string.IsNullOrWhiteSpace(result.Rule?.Id) + && !result.RuleId.Equals(result.Rule?.Id, StringComparison.OrdinalIgnoreCase)) + { + // {0}: This result contains both the 'ruleId' property '{1}' and the 'rule.id' property + // {2}', but they are not equal. The SARIF specification ([§3.27.5] + // (https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) + // requires that if both of these properties are present, they must be equal. + LogResult( + pointer, + nameof(RuleResources.SARIF1010_RuleIdMustBeConsistent_Error_ResultRuleIdMustBeConsistent_Text), + result.RuleId, + result.Rule?.Id); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1011.ReferenceFinalSchema.cs b/src/Sarif.Multitool/Rules/SARIF1011.ReferenceFinalSchema.cs new file mode 100644 index 000000000..c01a44bb4 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1011.ReferenceFinalSchema.cs @@ -0,0 +1,65 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ReferenceFinalSchema : SarifValidationSkimmerBase + { + /// + /// SARIF1011 + /// + public override string Id => RuleId.ReferenceFinalSchema; + + /// + /// The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This + /// enables IDEs to provide Intellisense for SARIF log files. + /// + /// The SARIF standard was developed over several years, and many intermediate versions of + /// the schema were produced.Now that the standard is final, only the OASIS standard version + /// of the schema is valid. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1011_ReferenceFinalSchema_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1011_ReferenceFinalSchema_Error_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Error; + + + protected override void Analyze(SarifLog log, string logPointer) + { + AnalyzeSchema(log.SchemaUri, logPointer.AtProperty(SarifPropertyName.Schema)); + } + + private void AnalyzeSchema(Uri schemaUri, string schema) + { + if (schemaUri == null) + { + // If SchemaUri is not present, it will be caught by SARIF2008 rule. + return; + } + + if (!schemaUri.OriginalString.EndsWith(VersionConstants.StableSarifVersion) + && !schemaUri.OriginalString.EndsWith($"{VersionConstants.StableSarifVersion}.json") + && !schemaUri.OriginalString.EndsWith(VersionConstants.SchemaVersionAsPublishedToSchemaStoreOrg) + && !schemaUri.OriginalString.EndsWith($"{VersionConstants.SchemaVersionAsPublishedToSchemaStoreOrg}.json")) + { + // {0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF + // 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading + // your analysis tool to produce the final version. If this file does in fact conform to the + // final version of the schema, upgrade the tool to populate the '$schema' property with a URL + // that refers to the final version of the schema. + LogResult( + schema, + nameof(RuleResources.SARIF1011_ReferenceFinalSchema_Error_Default_Text), + schemaUri.OriginalString); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1012.EndLineMustNotBeLessThanStartLine.cs b/src/Sarif.Multitool/Rules/SARIF1012.EndLineMustNotBeLessThanStartLine.cs deleted file mode 100644 index df8ed6189..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1012.EndLineMustNotBeLessThanStartLine.cs +++ /dev/null @@ -1,48 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System.Collections.Generic; -using Microsoft.Json.Pointer; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class EndLineMustNotBeLessThanStartLine : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1012_EndLineMustNotBeLessThanStartLine - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1012 - /// - public override string Id => RuleId.EndLineMustNotBeLessThanStartLine; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1012_Default) - }; - - protected override void Analyze(Region region, string regionPointer) - { - var jsonPointer = new JsonPointer(regionPointer); - Newtonsoft.Json.Linq.JToken regionToken = jsonPointer.Evaluate(Context.InputLogToken); - - if (regionToken.HasProperty(SarifPropertyName.EndLine) && - region.EndLine < region.StartLine) - { - string endLinePointer = regionPointer.AtProperty(SarifPropertyName.EndLine); - - LogResult( - endLinePointer, - nameof(RuleResources.SARIF1012_Default), - region.EndLine.ToInvariantString(), - region.StartLine.ToInvariantString()); - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1012.MessageArgumentsMustBeConsistentWithRule.cs b/src/Sarif.Multitool/Rules/SARIF1012.MessageArgumentsMustBeConsistentWithRule.cs new file mode 100644 index 000000000..c33015e49 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF1012.MessageArgumentsMustBeConsistentWithRule.cs @@ -0,0 +1,106 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.RegularExpressions; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class MessageArgumentsMustBeConsistentWithRule : SarifValidationSkimmerBase + { + /// + /// SARIF1012 + /// + public override string Id => RuleId.MessageArgumentsMustBeConsistentWithRule; + + /// + /// The properties of a result's 'message' property must be consistent with the properties + /// of the rule that the result refers to. + /// + /// When a result's 'message' object uses the 'id' and 'arguments' properties (which, by the + /// way, is recommended: see SARIF2002.ProvideMessageArguments), it must ensure that the rule + /// actually defines a message string with that id, and that 'arguments' array has enough + /// elements to provide values for every replacement sequence in the message specified by 'id'. + /// For example, if the highest numbered replacement sequence in the specified message string + /// is '{3}', then the 'arguments' array must contain at least 4 elements. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF1012_MessageArgumentsMustBeConsistentWithRule_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_MessageIdMustExist_Text), + nameof(RuleResources.SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_SupplyEnoughMessageArguments_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Error; + + private static readonly Regex s_replacementSequenceRegex = new Regex(@"\{(?\d+)\}", RegexOptions.Compiled | RegexOptions.CultureInvariant); + private IList currentRules; + private Run run; + + protected override void Analyze(Run run, string runPointer) + { + this.run = run; + this.currentRules = run.Tool.Driver?.Rules; + } + + protected override void Analyze(Result result, string resultPointer) + { + // If message.id is present, check that a message with that id exists in the rule. + if (!string.IsNullOrEmpty(result.Message.Id)) + { + ReportingDescriptor rule = result.GetRule(this.run); + + if (this.currentRules == null + || rule.MessageStrings?.ContainsKey(result.Message.Id) == false) + { + // {0}: This message object refers to the message with id '{1}' in rule '{2}', + // but that rule does not define a message with that id. When a tool creates a + // result message that uses the 'id' property, it must ensure that the specified + // rule actually has a message with that id. + LogResult( + resultPointer, + nameof(RuleResources.SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_MessageIdMustExist_Text), + result.Message.Id, + result.ResolvedRuleId(run) ?? "null"); + return; + } + + // A message with the specified key is present in the rule. Check if the result supplied enough arguments. + string messageText = rule.MessageStrings[result.Message.Id].Text; + int numArgsRequired = GetNumArgsRequired(messageText); + int numArgsPresent = result.Message.Arguments?.Count ?? 0; + if (numArgsRequired > numArgsPresent) + { + // {0}: The message with id '{1}' in rule '{2}' requires '{3}' arguments, but the + // 'arguments' array in this message object has only '{4}' element(s). When a tool + // creates a result message that use the 'id' and 'arguments' properties, it must + // ensure that the 'arguments' array has enough elements to provide values for every + // replacement sequence in the message specified by 'id'. For example, if the highest + // numbered replacement sequence in the specified message string is '{{3}}', then + // the 'arguments' array must contain 4 elements. + LogResult( + resultPointer, + nameof(RuleResources.SARIF1012_MessageArgumentsMustBeConsistentWithRule_Error_SupplyEnoughMessageArguments_Text), + result.Message.Id, + result.ResolvedRuleId(run) ?? "null", + numArgsRequired.ToString(), + result.Message.Arguments.Count.ToString()); + } + } + } + + private int GetNumArgsRequired(string text) + { + int max = -1; + foreach (Match match in s_replacementSequenceRegex.Matches(text)) + { + int index = int.Parse(match.Groups["index"].Value); + max = Math.Max(max, index); + } + + return max + 1; + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF1013.EndColumnMustNotBeLessThanStartColumn.cs b/src/Sarif.Multitool/Rules/SARIF1013.EndColumnMustNotBeLessThanStartColumn.cs deleted file mode 100644 index 8928fa4f0..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1013.EndColumnMustNotBeLessThanStartColumn.cs +++ /dev/null @@ -1,57 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System.Collections.Generic; -using Microsoft.Json.Pointer; -using Newtonsoft.Json.Linq; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class EndColumnMustNotBeLessThanStartColumn : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1013_EndColumnMustNotBeLessThanStartColumn - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1013 - /// - public override string Id => RuleId.EndColumnMustNotBeLessThanStartColumn; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1013_Default) - }; - - protected override void Analyze(Region region, string regionPointer) - { - var jsonPointer = new JsonPointer(regionPointer); - JToken regionToken = jsonPointer.Evaluate(Context.InputLogToken); - - if (RegionIsOnOneLine(region, regionToken) && - regionToken.HasProperty(SarifPropertyName.EndColumn) && - region.EndColumn < region.StartColumn) - { - string endColumnPointer = regionPointer.AtProperty(SarifPropertyName.EndColumn); - - LogResult( - endColumnPointer, - nameof(RuleResources.SARIF1013_Default), - region.EndColumn.ToInvariantString(), - region.StartColumn.ToInvariantString()); - } - } - - private static bool RegionIsOnOneLine(Region region, JToken regionToken) - { - return regionToken.HasProperty(SarifPropertyName.EndLine) - ? region.StartLine == region.EndLine - : true; - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1014.UriBaseIdRequiresRelativeUri.cs b/src/Sarif.Multitool/Rules/SARIF1014.UriBaseIdRequiresRelativeUri.cs deleted file mode 100644 index c2821dd12..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1014.UriBaseIdRequiresRelativeUri.cs +++ /dev/null @@ -1,41 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System.Collections.Generic; -using Microsoft.Json.Pointer; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class UriBaseIdRequiresRelativeUri : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1014_UriBaseIdRequiresRelativeUri - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1014 - /// - public override string Id => RuleId.UriBaseIdRequiresRelativeUri; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1014_Default) - }; - - protected override void Analyze(ArtifactLocation fileLocation, string fileLocationPointer) - { - if (fileLocation.UriBaseId != null && fileLocation.Uri.IsAbsoluteUri) - { - LogResult( - fileLocationPointer.AtProperty(SarifPropertyName.Uri), - nameof(RuleResources.SARIF1014_Default), - fileLocation.Uri.OriginalString); - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1016.ContextRegionRequiresRegion.cs b/src/Sarif.Multitool/Rules/SARIF1016.ContextRegionRequiresRegion.cs deleted file mode 100644 index e4061b932..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1016.ContextRegionRequiresRegion.cs +++ /dev/null @@ -1,37 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System.Collections.Generic; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class ContextRegionRequiresRegion : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1016_ContextRegionRequiresRegion - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1016 - /// - public override string Id => RuleId.ContextRegionRequiresRegion; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1016_Default) - }; - - protected override void Analyze(PhysicalLocation physicalLocation, string physicalLocationPointer) - { - if (physicalLocation.ContextRegion != null && physicalLocation.Region == null) - { - LogResult(physicalLocationPointer, nameof(RuleResources.SARIF1016_Default)); - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1018.InvalidUriInOriginalUriBaseIds.cs b/src/Sarif.Multitool/Rules/SARIF1018.InvalidUriInOriginalUriBaseIds.cs deleted file mode 100644 index 8ff9b4ba5..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1018.InvalidUriInOriginalUriBaseIds.cs +++ /dev/null @@ -1,71 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System; -using System.Collections.Generic; -using Microsoft.Json.Pointer; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class InvalidUriInOriginalUriBaseIds : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1018_InvalidUriInOriginalUriBaseIds - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1018 - /// - public override string Id => RuleId.InvalidUriInOriginalUriBaseIds; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1018_NotAbsolute), - nameof(RuleResources.SARIF1018_LacksTrailingSlash) - }; - - protected override void Analyze(Run run, string runPointer) - { - if (run.OriginalUriBaseIds != null) - { - string originalUriBaseIdsPointer = runPointer.AtProperty(SarifPropertyName.OriginalUriBaseIds); - - foreach (string uriBaseId in run.OriginalUriBaseIds.Keys) - { - AnalyzeOriginalUriBaseIdsEntry(uriBaseId, run.OriginalUriBaseIds[uriBaseId], originalUriBaseIdsPointer.AtProperty(uriBaseId)); - } - } - } - - private void AnalyzeOriginalUriBaseIdsEntry(string uriBaseId, ArtifactLocation artifactLocation, string pointer) - { - if (artifactLocation.Uri == null) { return; } - - // If it's not a well-formed URI of _any_ kind, then don't bother triggering this rule. - // Rule SARIF1003, UrisMustBeValid, will catch it. - // Check for well-formedness first, before attempting to create a Uri object, to - // avoid having to do a try/catch. Unfortunately Uri.TryCreate will return true - // even for a malformed URI string. - string uriString = artifactLocation.Uri.OriginalString; - if (uriString != null && Uri.IsWellFormedUriString(uriString, UriKind.RelativeOrAbsolute)) - { - var uri = new Uri(uriString, UriKind.RelativeOrAbsolute); - - if (artifactLocation.UriBaseId == null && !uri.IsAbsoluteUri) - { - LogResult(pointer, nameof(RuleResources.SARIF1018_NotAbsolute), uriString, uriBaseId); - } - - if (!uriString.EndsWith("/")) - { - LogResult(pointer, nameof(RuleResources.SARIF1018_LacksTrailingSlash), uriString, uriBaseId); - } - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1019.RuleIdMustBePresentAndConsistent.cs b/src/Sarif.Multitool/Rules/SARIF1019.RuleIdMustBePresentAndConsistent.cs deleted file mode 100644 index c28585125..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1019.RuleIdMustBePresentAndConsistent.cs +++ /dev/null @@ -1,52 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using System; -using System.Collections.Generic; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class RuleIdMustBePresentAndConsistent : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1019_RuleIdMustBePresentAndConsistent - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1019 - /// - public override string Id => RuleId.RuleIdMustBePresentAndConsistent; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1019_InconsistentResultRuleId), - nameof(RuleResources.SARIF1019_MissingResultRuleId) - }; - - protected override void Analyze(Result result, string resultPointer) - { - AnalyzeRuleId(result, resultPointer); - } - - private void AnalyzeRuleId(Result result, string pointer) - { - // At least one of result.ruleId or result.rule.id must be present - if (string.IsNullOrWhiteSpace(result.RuleId) && string.IsNullOrWhiteSpace(result.Rule?.Id)) - { - LogResult(pointer, nameof(RuleResources.SARIF1019_MissingResultRuleId)); - } - // if both are present, they must be equal. - else if (!string.IsNullOrWhiteSpace(result.RuleId) - && !string.IsNullOrWhiteSpace(result.Rule?.Id) - && !result.RuleId.Equals(result.Rule?.Id, StringComparison.OrdinalIgnoreCase)) - { - LogResult(pointer, nameof(RuleResources.SARIF1019_InconsistentResultRuleId), result.RuleId, result.Rule?.Id); - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF1020.ReferToFinalSchema.cs b/src/Sarif.Multitool/Rules/SARIF1020.ReferToFinalSchema.cs deleted file mode 100644 index 0981fc923..000000000 --- a/src/Sarif.Multitool/Rules/SARIF1020.ReferToFinalSchema.cs +++ /dev/null @@ -1,55 +0,0 @@ -// Copyright (c) Microsoft. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using Microsoft.Json.Pointer; -using System; -using System.Collections.Generic; - -namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules -{ - public class ReferToFinalSchema : SarifValidationSkimmerBase - { - private readonly MultiformatMessageString _fullDescription = new MultiformatMessageString - { - Text = RuleResources.SARIF1020_ReferToFinalSchema - }; - - public override MultiformatMessageString FullDescription => _fullDescription; - - public override FailureLevel DefaultLevel => FailureLevel.Error; - - /// - /// SARIF1020 - /// - public override string Id => RuleId.ReferToFinalSchema; - - protected override IEnumerable MessageResourceNames => new string[] - { - nameof(RuleResources.SARIF1020_ReferenceToOldSchemaVersion), - nameof(RuleResources.SARIF1020_SchemaReferenceMissing) - }; - - protected override void Analyze(SarifLog log, string logPointer) - { - AnalyzeSchema(log.SchemaUri, logPointer); - } - - private void AnalyzeSchema(Uri schemaUri, string pointer) - { - if (schemaUri == null) - { - LogResult(pointer, nameof(RuleResources.SARIF1020_SchemaReferenceMissing)); - return; - } - - if (!schemaUri.OriginalString.EndsWith(VersionConstants.StableSarifVersion) - && !schemaUri.OriginalString.EndsWith($"{VersionConstants.StableSarifVersion}.json") - && !schemaUri.OriginalString.EndsWith(VersionConstants.SchemaVersionAsPublishedToSchemaStoreOrg) - && !schemaUri.OriginalString.EndsWith($"{VersionConstants.SchemaVersionAsPublishedToSchemaStoreOrg}.json")) - { - LogResult(pointer.AtProperty(SarifPropertyName.Schema), nameof(RuleResources.SARIF1020_ReferenceToOldSchemaVersion), schemaUri.OriginalString); - return; - } - } - } -} diff --git a/src/Sarif.Multitool/Rules/SARIF2001.TerminateMessagesWithPeriod.cs b/src/Sarif.Multitool/Rules/SARIF2001.TerminateMessagesWithPeriod.cs new file mode 100644 index 000000000..6c3c14137 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2001.TerminateMessagesWithPeriod.cs @@ -0,0 +1,90 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class TerminateMessagesWithPeriod : SarifValidationSkimmerBase + { + /// + /// SARIF2001 + /// + public override string Id => RuleId.TerminateMessagesWithPeriod; + + /// + /// Express plain text result messages as complete sentences and end each sentence with a period. + /// This guidance does not apply to Markdown messages, which might include formatting that makes + /// the punctuation unnecessary. + /// + /// This is part of a set of authoring practices that make your rule messages more readable, + /// understandable, and actionable.See also `SARIF2014.ProvideDynamicMessageContent` and + /// `SARIF2015.EnquoteDynamicMessageContent`. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2001_TerminateMessagesWithPeriod_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2001_TerminateMessagesWithPeriod_Warning_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + protected override void Analyze(Tool tool, string toolPointer) + { + if (tool.Driver != null) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (toolComponent.Rules != null) + { + string rulesPointer = toolDriverPointer.AtProperty(SarifPropertyName.Rules); + for (int i = 0; i < toolComponent.Rules.Count; i++) + { + AnalyzeReportingDescriptor(toolComponent.Rules[i], rulesPointer.AtIndex(i)); + } + } + } + + private void AnalyzeReportingDescriptor(ReportingDescriptor rule, string reportingDescriptorPointer) + { + if (rule.MessageStrings != null) + { + string messageStringsPointer = reportingDescriptorPointer.AtProperty(SarifPropertyName.MessageStrings); + foreach (KeyValuePair message in rule.MessageStrings) + { + AnalyzeMessageString(rule.Id, message.Value.Text, message.Key, messageStringsPointer.AtProperty(message.Key)); + } + } + } + + private void AnalyzeMessageString(string ruleId, string messageString, string messageKey, string messagePointer) + { + if (string.IsNullOrEmpty(messageString)) + { + return; + } + + string textPointer = messagePointer.AtProperty(SarifPropertyName.Text); + + if (!messageString.EndsWith(".", StringComparison.Ordinal)) + { + // {0}: In rule '{1}', the message with id '{2}' does not end in a period. Express plain + // text rule messages as complete sentences. This guidance does not apply to Markdown + // messages, which might include formatting that makes the punctuation unnecessary. + LogResult( + textPointer, + nameof(RuleResources.SARIF2001_TerminateMessagesWithPeriod_Warning_Default_Text), + ruleId, + messageKey); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2002.ProvideMessageArguments.cs b/src/Sarif.Multitool/Rules/SARIF2002.ProvideMessageArguments.cs new file mode 100644 index 000000000..f603256ca --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2002.ProvideMessageArguments.cs @@ -0,0 +1,50 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideMessageArguments : SarifValidationSkimmerBase + { + /// + /// SARIF2002 + /// + public override string Id => RuleId.ProvideMessageArguments; + + /// + /// In result messages, use the 'message.id' and 'message.arguments' properties rather than + /// 'message.text'. This has several advantages. If 'text' is lengthy, using 'id' and 'arguments' + /// makes the SARIF file smaller. If the rule metadata is stored externally to the SARIF log file, + /// the message text can be improved (for example, by adding more text, clarifying the phrasing, + /// or fixing typos), and the result messages will pick up the improvements the next time it is + /// displayed. Finally, SARIF supports localizing messages into different languages, which is + /// possible if the SARIF file contains 'message.id' and 'message.arguments', but not if it contains + /// 'message.text' directly. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2002_ProvideMessageArguments_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2002_ProvideMessageArguments_Warning_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + protected override void Analyze(Result result, string resultPointer) + { + if (string.IsNullOrEmpty(result.Message.Id)) + { + // {0}: The 'message' property of this result contains a 'text' property. Consider replacing + // it with 'id' and 'arguments' properties. This potentially reduces the log file size, allows + // the message text to be improved without modifying the log file, and enables localization. + LogResult( + resultPointer.AtProperty(SarifPropertyName.Message), + nameof(RuleResources.SARIF2002_ProvideMessageArguments_Warning_Default_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2003.ProvideVersionControlProvenance.cs b/src/Sarif.Multitool/Rules/SARIF2003.ProvideVersionControlProvenance.cs new file mode 100644 index 000000000..1f8bc9ffc --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2003.ProvideVersionControlProvenance.cs @@ -0,0 +1,40 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideVersionControlProvenance : SarifValidationSkimmerBase + { + /// + /// SARIF2003 + /// + public override string Id => RuleId.ProvideVersionControlProvenance; + + /// + /// Provide 'versionControlProvenance' to record which version of the code was analyzed, + /// and to enable paths to be expressed relative to the root of the repository. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2003_ProvideVersionControlProvenance_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2003_ProvideVersionControlProvenance_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + protected override void Analyze(Run run, string runPointer) + { + if (run.VersionControlProvenance == null || run.VersionControlProvenance.Count == 0) + { + // {0}: This run does not provide 'versionControlProvenance'. As a result, it is + // not possible to determine which version of code was analyzed, nor to map + // relative paths to their locations within the repository. + LogResult( + runPointer, + nameof(RuleResources.SARIF2003_ProvideVersionControlProvenance_Note_Default_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2004.OptimizeFileSize.cs b/src/Sarif.Multitool/Rules/SARIF2004.OptimizeFileSize.cs new file mode 100644 index 000000000..0fcb38ac0 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2004.OptimizeFileSize.cs @@ -0,0 +1,154 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Linq; + +using Microsoft.Json.Pointer; + +using Newtonsoft.Json.Linq; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class OptimizeFileSize : SarifValidationSkimmerBase + { + /// + /// SARIF2004 + /// + public override string Id => RuleId.OptimizeFileSize; + + /// + /// Emit arrays only if they provide additional information. + /// + /// In several parts of a SARIF log file, a subset of information about an object appears + /// in one place, and the full information describing all such objects appears in an array + /// elsewhere in the log file. For example, each 'result' object has a 'ruleId' property + /// that identifies the rule that was violated. Elsewhere in the log file, the array + /// 'run.tool.driver.rules' contains additional information about the rules. But if the + /// elements of the 'rules' array contained no information about the rules beyond their ids, + /// then there might be no reason to include the 'rules' array at all, and the log file + /// could be made smaller simply by omitting it. In some scenarios (for example, when + /// assessing compliance with policy), the 'rules' array might be used to record the full + /// set of rules that were evaluated. In such a scenario, the 'rules' array should be retained + /// even if it contains only id information. + /// + /// Similarly, most 'result' objects contain at least one 'artifactLocation' object. Elsewhere + /// in the log file, the array 'run.artifacts' contains additional information about the artifacts + /// that were analyzed. But if the elements of the 'artifacts' array contained not information + /// about the artifacts beyond their locations, then there might be no reason to include the + /// 'artifacts' array at all, and again the log file could be made smaller by omitting it. In + /// some scenarios (for example, when assessing compliance with policy), the 'artifacts' array + /// might be used to record the full set of artifacts that were analyzed. In such a scenario, + /// the 'artifacts' array should be retained even if it contains only location information. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2004_OptimizeFileSize_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2004_OptimizeFileSize_Warning_EliminateLocationOnlyArtifacts_Text), + nameof(RuleResources.SARIF2004_OptimizeFileSize_Warning_EliminateIdOnlyRules_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + protected override void Analyze(Run run, string runPointer) + { + AnalyzeLocationOnlyArtifacts(run, runPointer); + AnalyzeIdOnlyRules(run, runPointer); + } + + private void AnalyzeLocationOnlyArtifacts(Run run, string runPointer) + { + // We only verify first item in the results and artifacts array, + // since tools will typically generate similar nodes. + // This approach may cause occasional false negatives. + ArtifactLocation firstLocationInArtifactsArray = run.Artifacts?.FirstOrDefault()?.Location; + ArtifactLocation firstlocationInResults = run.Results?.FirstOrDefault()?.Locations?.FirstOrDefault()?.PhysicalLocation?.ArtifactLocation; + + if (firstLocationInArtifactsArray == null || firstlocationInResults == null) + { + return; + } + + string firstArtifactPointer = runPointer + .AtProperty(SarifPropertyName.Artifacts).AtIndex(0); + + string firstResultLocationPointer = runPointer + .AtProperty(SarifPropertyName.Results).AtIndex(0) + .AtProperty(SarifPropertyName.Locations).AtIndex(0) + .AtProperty(SarifPropertyName.PhysicalLocation) + .AtProperty(SarifPropertyName.ArtifactLocation); + + if (HasResultLocationsWithUriAndIndex(firstResultLocationPointer) && HasLocationOnlyArtifacts(firstArtifactPointer)) + { + // {0}: The 'artifacts' array contains no information beyond the locations of the + // artifacts. Removing this array might reduce the log file size without losing + // information. In some scenarios (for example, when assessing compliance with policy), + // the 'artifacts' array might be used to record the full set of artifacts that were + // analyzed. In such a scenario, the 'artifacts' array should be retained even if it + // contains only location information. + LogResult( + firstArtifactPointer, + nameof(RuleResources.SARIF2004_OptimizeFileSize_Warning_EliminateLocationOnlyArtifacts_Text)); + } + } + + private bool HasResultLocationsWithUriAndIndex(string resultPointer) + { + JToken resultToken = resultPointer.ToJToken(Context.InputLogToken); + return + resultToken.HasProperty(SarifPropertyName.Uri) && + resultToken.HasProperty(SarifPropertyName.Index); + } + + private bool HasLocationOnlyArtifacts(string artifactPointer) + { + JToken artifactToken = artifactPointer.ToJToken(Context.InputLogToken); + return + artifactToken.HasProperty(SarifPropertyName.Location) && + artifactToken.Children().Count() == 1; + } + + private void AnalyzeIdOnlyRules(Run run, string runPointer) + { + // We only verify first item in the rules array, + // since tools will typically generate similar nodes. + // This approach may cause occasional false negatives. + // Also, `tool` and `driver` are mandatory fields, hence + // null check in not required. + ReportingDescriptor firstRule = run.Tool.Driver.Rules?.FirstOrDefault(); + + if (firstRule == null) + { + return; + } + + string firstRulePointer = runPointer + .AtProperty(SarifPropertyName.Tool) + .AtProperty(SarifPropertyName.Driver) + .AtProperty(SarifPropertyName.Rules) + .AtIndex(0); + + if (HasIdOnlyRules(firstRulePointer)) + { + // {0}: The 'rules' array contains no information beyond the ids of the rules. + // Removing this array might reduce the log file size without losing information. + // In some scenarios (for example, when assessing compliance with policy), the + // 'rules' array might be used to record the full set of rules that were evaluated. + // In such a scenario, the 'rules' array should be retained even if it contains + // only id information. + LogResult( + firstRulePointer, + nameof(RuleResources.SARIF2004_OptimizeFileSize_Warning_EliminateIdOnlyRules_Text)); + } + } + + private bool HasIdOnlyRules(string rulePointer) + { + JToken ruleToken = rulePointer.ToJToken(Context.InputLogToken); + return + ruleToken.HasProperty(SarifPropertyName.Id) && + ruleToken.Children().Count() == 1; + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2005.ProvideToolProperties.cs b/src/Sarif.Multitool/Rules/SARIF2005.ProvideToolProperties.cs new file mode 100644 index 000000000..5d7b32c54 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2005.ProvideToolProperties.cs @@ -0,0 +1,112 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideToolProperties : SarifValidationSkimmerBase + { + /// + /// SARIF2005 + /// + public override string Id => RuleId.ProvideToolProperties; + + /// + /// Provide information that makes it easy to identify the name and version of your tool. + /// + /// The tool's 'name' property should be no more than three words long. This makes it easy + /// to remember and allows it to fit into a narrow column when displaying a list of results. + /// If you need to provide more information about your tool, use the 'fullName' property. + /// + /// The tool should provide either or both of the 'version' and 'semanticVersion' properties. + /// This enables the log file consumer to determine whether the file was produced by an up + /// to date version, and to avoid accidentally comparing log files produced by different tool + /// versions. + /// + /// If 'version' is used, facilitate comparison between versions by specifying a version number + /// that starts with an integer, optionally followed by any desired characters. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2005_ProvideToolProperties_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_ProvideToolVersion_Text), + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_ProvideConciseToolName_Text), + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_UseNumericToolVersions_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + private static readonly Regex s_versionRegex = new Regex(@"^\d+.*", RegexOptions.Compiled | RegexOptions.CultureInvariant); + + protected override void Analyze(Tool tool, string toolPointer) + { + if (tool.Driver != null) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (!string.IsNullOrEmpty(toolComponent.Name)) + { + const int MaxWords = 3; + int wordCount = toolComponent.Name.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries).Length; + if (wordCount > MaxWords) + { + string driverNamePointer = toolDriverPointer.AtProperty(SarifPropertyName.Name); + + // {0}: The tool name '{1}' contains {2} words, which is more than the recommended + // maximum of {3} words. A short tool name is easy to remember and fits into a + // narrow column when displaying a list of results. If you need to provide more + // information about your tool, use the 'fullName' property. + LogResult( + driverNamePointer, + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_ProvideConciseToolName_Text), + toolComponent.Name, + wordCount.ToString(), + MaxWords.ToString()); + } + } + + if (string.IsNullOrWhiteSpace(toolComponent.Version) && string.IsNullOrWhiteSpace(toolComponent.SemanticVersion)) + { + // {0}: The tool '{1}' provides neither a 'version' property nor a 'semanticVersion' + // property. Providing a version enables the log file consumer to determine whether + // the file was produced by an up to date version, and to avoid accidentally comparing + // log files produced by different tool versions. + LogResult( + toolDriverPointer, + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_ProvideToolVersion_Text), + toolComponent.Name); + } + else + { + if (!string.IsNullOrWhiteSpace(toolComponent.Version)) + { + AnalyzeVersion(toolComponent.Name, toolComponent.Version, toolDriverPointer.AtProperty(SarifPropertyName.Version)); + } + } + } + + private void AnalyzeVersion(string name, string version, string pointer) + { + if (!s_versionRegex.IsMatch(version)) + { + // {0}: The tool '{1}' contains the 'version' property '{2}', which is not numeric. + // To facilitate comparison between versions, specify a 'version' that starts with + // an integer, optionally followed by any desired characters. + LogResult( + pointer, + nameof(RuleResources.SARIF2005_ProvideToolProperties_Warning_UseNumericToolVersions_Text), + name, + version); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2006.UrisShouldBeReachable.cs b/src/Sarif.Multitool/Rules/SARIF2006.UrisShouldBeReachable.cs new file mode 100644 index 000000000..a3d11f445 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2006.UrisShouldBeReachable.cs @@ -0,0 +1,110 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Net.Http; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class UrisShouldBeReachable : SarifValidationSkimmerBase + { + /// + /// SARIF2006 + /// + public override string Id => RuleId.UrisShouldBeReachable; + + /// + /// URIs that refer to locations such as rule help pages and result-related work items + /// should be reachable via an HTTP GET request. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2006_UrisShouldBeReachable_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2006_UrisShouldBeReachable_Warning_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + private static readonly HttpClient s_httpClient = new HttpClient(); + private static readonly Dictionary s_checkedUris = new Dictionary(); + + protected override void Analyze(SarifLog log, string logPointer) + { + AnalyzeUri(log.SchemaUri, logPointer.AtProperty(SarifPropertyName.Schema)); + } + + protected override void Analyze(Result result, string resultPointer) + { + if (result.WorkItemUris != null) + { + Uri[] workItemUris = result.WorkItemUris.ToArray(); + string workItemUrisPointer = resultPointer.AtProperty(SarifPropertyName.WorkItemUris); + + for (int i = 0; i < workItemUris.Length; ++i) + { + AnalyzeUri(workItemUris[i], workItemUrisPointer.AtIndex(i)); + } + } + } + + protected override void Analyze(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) + { + AnalyzeUri(reportingDescriptor.HelpUri, reportingDescriptorPointer.AtProperty(SarifPropertyName.HelpUri)); + } + + protected override void Analyze(ToolComponent toolComponent, string toolComponentPointer) + { + AnalyzeUri(toolComponent.DownloadUri, toolComponentPointer.AtProperty(SarifPropertyName.DownloadUri)); + } + + protected override void Analyze(VersionControlDetails versionControlDetails, string versionControlDetailsPointer) + { + AnalyzeUri(versionControlDetails.RepositoryUri, versionControlDetailsPointer.AtProperty(SarifPropertyName.RepositoryUri)); + } + + private void AnalyzeUri(Uri uri, string pointer) + { + AnalyzeUri(uri?.OriginalString, pointer); + } + + private void AnalyzeUri(string uriString, string pointer) + { + // If it's not a well-formed URI, or if it's not absolute, then don't bother triggering this rule. + // If it's not well-formed, SARIF1003.UrisMustBeValid will catch it, and if it's not absolute, + // SARIF1005.UriMustBeAbsolute will catch it. + // + // Check for well-formedness first, before attempting to create a Uri object, to + // avoid having to do a try/catch. Unfortunately Uri.TryCreate will return true + // even for a malformed URI string. + if (uriString != null && Uri.IsWellFormedUriString(uriString, UriKind.Absolute)) + { + // Ok, it's a well-formed absolute URI. If it's not reachable, _now_ we can report it. + Uri uri = new Uri(uriString, UriKind.Absolute); + if (!IsUriReachable(uri.AbsoluteUri)) + { + // {0}: The URI '{1}' was not reachable via an HTTP GET request. + LogResult( + pointer, + nameof(RuleResources.SARIF2006_UrisShouldBeReachable_Warning_Default_Text), + uriString); + } + } + } + + private bool IsUriReachable(string uriString) + { + if (s_checkedUris.ContainsKey(uriString)) + { + return s_checkedUris[uriString]; + } + + HttpResponseMessage httpResponseMessage = s_httpClient.GetAsync(uriString).GetAwaiter().GetResult(); + s_checkedUris.Add(uriString, httpResponseMessage.IsSuccessStatusCode); + return httpResponseMessage.IsSuccessStatusCode; + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2007.ExpressPathsRelativeToRepoRoot.cs b/src/Sarif.Multitool/Rules/SARIF2007.ExpressPathsRelativeToRepoRoot.cs new file mode 100644 index 000000000..f9710e93c --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2007.ExpressPathsRelativeToRepoRoot.cs @@ -0,0 +1,105 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; +using System.Linq; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ExpressPathsRelativeToRepoRoot : SarifValidationSkimmerBase + { + /// + /// SARIF2007 + /// + public override string Id => RuleId.ExpressPathsRelativeToRepoRoot; + + /// + /// Provide information that makes it possible to determine the repo-relative locations of + /// files that contain analysis results. + /// + /// Each element of the 'versionControlProvenance' array is a 'versionControlDetails' object + /// that describes a repository containing files that were analyzed. 'versionControlDetails.mappedTo' + /// defines the file system location to which the root of that repository is mapped. If + /// 'mappedTo.uriBaseId' is present, and if result locations are expressed relative to that + /// 'uriBaseId', then the repo-relative location of each result can be determined. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2007_ExpressPathsRelativeToRepoRoot_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ExpressResultLocationsRelativeToMappedTo_Text), + nameof(RuleResources.SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ProvideUriBaseIdForMappedTo_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + private HashSet uriBaseIds; + + protected override void Analyze(Run run, string runPointer) + { + this.uriBaseIds = new HashSet(); + + if (run.VersionControlProvenance != null) + { + string versionControlProvenancePointer = runPointer.AtProperty(SarifPropertyName.VersionControlProvenance); + + for (int i = 0; i < run.VersionControlProvenance.Count; i++) + { + string versionControlDetailsPointer = versionControlProvenancePointer.AtIndex(i); + if (run.VersionControlProvenance[i].MappedTo == null || string.IsNullOrWhiteSpace(run.VersionControlProvenance[i].MappedTo.UriBaseId)) + { + // {0}: The 'versionControlDetails' object that describes the repository '{1}' + // does not provide 'mappedTo.uriBaseId'. As a result, it will not be possible + // to determine the repo-relative location of files containing analysis results + // for this repository. + LogResult( + versionControlDetailsPointer, + nameof(RuleResources.SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ProvideUriBaseIdForMappedTo_Text), + run.VersionControlProvenance[i].RepositoryUri.OriginalString); + } + else + { + this.uriBaseIds.Add(run.VersionControlProvenance[i].MappedTo.UriBaseId); + } + } + } + } + + protected override void Analyze(Result result, string resultPointer) + { + if (result.Locations != null && this.uriBaseIds.Any()) + { + string locationsPointer = resultPointer.AtProperty(SarifPropertyName.Locations); + for (int i = 0; i < result.Locations.Count; i++) + { + AnalyzeLocation(result.Locations[i], locationsPointer.AtIndex(i)); + } + } + } + + private void AnalyzeLocation(Location location, string locationPointer) + { + if (this.uriBaseIds.Any() && location.PhysicalLocation != null) + { + string physicalLocation = locationPointer.AtProperty(SarifPropertyName.PhysicalLocation); + if (location.PhysicalLocation.ArtifactLocation != null) + { + string artifactLocation = physicalLocation.AtProperty(SarifPropertyName.ArtifactLocation); + if (string.IsNullOrWhiteSpace(location.PhysicalLocation.ArtifactLocation.UriBaseId) + || !this.uriBaseIds.Contains(location.PhysicalLocation.ArtifactLocation.UriBaseId)) + { + // {0}: This result location does not provide any of the 'uriBaseId' values + // that specify repository locations: {1}. As a result, it will not be possible + // to determine the location of the file containing this result relative to the + // root of the repository that contains it. + LogResult( + artifactLocation, + nameof(RuleResources.SARIF2007_ExpressPathsRelativeToRepoRoot_Warning_ExpressResultLocationsRelativeToMappedTo_Text), + string.Join(", ", this.uriBaseIds)); + } + } + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2008.ProvideSchema.cs b/src/Sarif.Multitool/Rules/SARIF2008.ProvideSchema.cs new file mode 100644 index 000000000..d9613069b --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2008.ProvideSchema.cs @@ -0,0 +1,41 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideSchema : SarifValidationSkimmerBase + { + /// + /// SARIF2008 + /// + public override string Id => RuleId.ProvideSchema; + + /// + /// A SARIF log file should contain, on the root object, a '$schema' property that refers to + /// the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide + /// Intellisense for SARIF log files. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2008_ProvideSchema_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2008_ProvideSchema_Warning_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Warning; + + protected override void Analyze(SarifLog log, string logPointer) + { + if (!Context.InputLogToken.HasProperty("$schema")) + { + // {0}: The SARIF log file does not contain a '$schema' property. Add a '$schema' + // property that refers to the final, OASIS standard version of the SARIF 2.1.0 + // schema. This enables IDEs to provide Intellisense for SARIF log files. + LogResult( + logPointer, + nameof(RuleResources.SARIF2008_ProvideSchema_Warning_Default_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2009.ConsiderConventionalIdentifierValues.cs b/src/Sarif.Multitool/Rules/SARIF2009.ConsiderConventionalIdentifierValues.cs new file mode 100644 index 000000000..1294d0cd4 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2009.ConsiderConventionalIdentifierValues.cs @@ -0,0 +1,78 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ConsiderConventionalIdentifierValues : SarifValidationSkimmerBase + { + /// + /// SARIF2009 + /// + public override string Id => RuleId.ConsiderConventionalIdentifierValues; + + /// + /// Adopt uniform naming conventions for rule ids. + /// + /// Many tools follow a conventional format for the 'reportingDescriptor.id' property: + /// a short string identifying the tool concatenated with a numeric rule number, for + /// example, 'CS2001' for a diagnostic from the Roslyn C# compiler. For uniformity of + /// experience across tools, we recommend this format. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2009_ConsiderConventionalIdentifierValues_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2009_ConsiderConventionalIdentifierValues_Note_UseConventionalRuleIds_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + private static readonly Regex s_conventionalIdRegex = new Regex(@"^[A-Z]{1,5}[0-9]{1,4}$", RegexOptions.Compiled | RegexOptions.CultureInvariant); + + protected override void Analyze(Run run, string runPointer) + { + AnalyzeTool(run.Tool, runPointer.AtProperty(SarifPropertyName.Tool)); + } + + private void AnalyzeTool(Tool tool, string toolPointer) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (toolComponent.Rules != null) + { + string rulesPointer = toolDriverPointer.AtProperty(SarifPropertyName.Rules); + for (int i = 0; i < toolComponent.Rules.Count; i++) + { + AnalyzeReportingDescriptor(toolComponent.Rules[i], rulesPointer.AtIndex(i)); + } + } + } + + private void AnalyzeReportingDescriptor(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) + { + if (string.IsNullOrWhiteSpace(reportingDescriptor.Id)) + { + return; + } + + if (!s_conventionalIdRegex.IsMatch(reportingDescriptor.Id)) + { + // {0}: The 'id' property of the rule '{1}' does not follow the recommended format: + // a short string identifying the tool concatenated with a numeric rule number, for + // example, 'CS2001'. Using a conventional format for the rule id provides a more + // uniform experience across tools. + LogResult( + reportingDescriptorPointer.AtProperty(SarifPropertyName.Id), + nameof(RuleResources.SARIF2009_ConsiderConventionalIdentifierValues_Note_UseConventionalRuleIds_Text), + reportingDescriptor.Id); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2010.ProvideCodeSnippets.cs b/src/Sarif.Multitool/Rules/SARIF2010.ProvideCodeSnippets.cs new file mode 100644 index 000000000..0871df0ea --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2010.ProvideCodeSnippets.cs @@ -0,0 +1,69 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideCodeSnippets : SarifValidationSkimmerBase + { + /// + /// SARIF2010 + /// + public override string Id => RuleId.ProvideCodeSnippets; + + /// + /// Provide code snippets to enable users to see the code that triggered each result, + /// even if they are not enlisted in the code. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2010_ProvideCodeSnippets_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2010_ProvideCodeSnippets_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + protected override void Analyze(Result result, string resultPointer) + { + if (result.Locations != null) + { + string locationsPointer = resultPointer.AtProperty(SarifPropertyName.Locations); + for (int i = 0; i < result.Locations.Count; i++) + { + AnalyzeResultLocation(result.Locations[i], locationsPointer.AtIndex(i)); + } + } + } + + private void AnalyzeResultLocation(Location location, string locationPointer) + { + AnalyzeRegion( + location.PhysicalLocation?.Region, + locationPointer + .AtProperty(SarifPropertyName.PhysicalLocation) + .AtProperty(SarifPropertyName.Region)); + + AnalyzeRegion( + location.PhysicalLocation?.ContextRegion, + locationPointer + .AtProperty(SarifPropertyName.PhysicalLocation) + .AtProperty(SarifPropertyName.ContextRegion)); + } + + private void AnalyzeRegion(Region region, string regionPointer) + { + if (region != null && region.Snippet == null) + { + // {0}: The 'region' object in this result location does not provide a 'snippet' + // property. Providing a code snippet enables users to see the source code that + // triggered the result, even if they are not enlisted in the code. + LogResult( + regionPointer, + nameof(RuleResources.SARIF2010_ProvideCodeSnippets_Note_Default_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2011.ProvideContextRegion.cs b/src/Sarif.Multitool/Rules/SARIF2011.ProvideContextRegion.cs new file mode 100644 index 000000000..c213679d3 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2011.ProvideContextRegion.cs @@ -0,0 +1,59 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideContextRegion : SarifValidationSkimmerBase + { + /// + /// SARIF2011 + /// + public override string Id => RuleId.ProvideContextRegion; + + /// + /// Provide context regions to enable users to see a portion of the code that surrounds + /// each result, even if they are not enlisted in the code. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2011_ProvideContextRegion_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2011_ProvideContextRegion_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + protected override void Analyze(Result result, string resultPointer) + { + if (result.Locations != null) + { + string locationsPointer = resultPointer.AtProperty(SarifPropertyName.Locations); + for (int i = 0; i < result.Locations.Count; i++) + { + AnalyzeLocation(result.Locations[i], locationsPointer.AtIndex(i)); + } + } + } + + private void AnalyzeLocation(Location location, string locationPointer) + { + if (location.PhysicalLocation?.Region != null) + { + if (location.PhysicalLocation?.ContextRegion == null) + { + string physicalLocationPointer = locationPointer.AtProperty(SarifPropertyName.PhysicalLocation); + + // {0}: This result location does not provide a 'contextRegion' property. Providing + // a context region enables users to see a portion of the code that surrounds the + // result, even if they are not enlisted in the code. + LogResult( + physicalLocationPointer, + nameof(RuleResources.SARIF2011_ProvideContextRegion_Note_Default_Text)); + } + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2012.ProvideHelpUris.cs b/src/Sarif.Multitool/Rules/SARIF2012.ProvideHelpUris.cs new file mode 100644 index 000000000..12d8d6c03 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2012.ProvideHelpUris.cs @@ -0,0 +1,78 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideHelpUris : SarifValidationSkimmerBase + { + /// + /// SARIF2012 + /// + public override string Id => RuleId.ProvideHelpUris; + + /// + /// For each rule, provide a URI where users can find detailed information about the rule. + /// This information should include a detailed description of the invalid pattern, an + /// explanation of why the pattern is poor practice (particularly in contexts such as + /// security or accessibility where driving considerations might not be readily apparent), + /// guidance for resolving the problem (including describing circumstances in which ignoring + /// the problem altogether might be appropriate), examples of invalid and valid patterns, + /// and special considerations (such as noting when a violation should never be ignored or + /// suppressed, noting when a violation could cause downstream tool noise, and noting when + /// a rule can be configured in some way to refine or alter the analysis). + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2012_ProvideHelpUris_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2012_ProvideHelpUris_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + protected override void Analyze(Run run, string runPointer) + { + AnalyzeTool(run.Tool, runPointer.AtProperty(SarifPropertyName.Tool)); + } + + private void AnalyzeTool(Tool tool, string toolPointer) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (toolComponent.Rules != null) + { + string rulesPointer = toolDriverPointer.AtProperty(SarifPropertyName.Rules); + for (int i = 0; i < toolComponent.Rules.Count; i++) + { + AnalyzeReportingDescriptor(toolComponent.Rules[i], rulesPointer.AtIndex(i)); + } + } + } + + private void AnalyzeReportingDescriptor(ReportingDescriptor reportingDescriptor, string reportingDescriptorPointer) + { + if (reportingDescriptor.HelpUri == null) + { + string ruleMoniker = reportingDescriptor.Id; + if (!string.IsNullOrWhiteSpace(reportingDescriptor.Name)) + { + ruleMoniker += $".{reportingDescriptor.Name}"; + } + + // {0}: The rule '{1}' does not provide a help URI. Providing a URI where users can + // find detailed information about the rule helps users to understand the result and + // how they can best address it. + LogResult( + reportingDescriptorPointer, + nameof(RuleResources.SARIF2012_ProvideHelpUris_Note_Default_Text), + ruleMoniker); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2013.ProvideEmbeddedFileContent.cs b/src/Sarif.Multitool/Rules/SARIF2013.ProvideEmbeddedFileContent.cs new file mode 100644 index 000000000..5731c37e1 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2013.ProvideEmbeddedFileContent.cs @@ -0,0 +1,45 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System.Collections.Generic; +using System.Linq; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideEmbeddedFileContent : SarifValidationSkimmerBase + { + /// + /// SARIF2013 + /// + public override string Id => RuleId.ProvideEmbeddedFileContent; + + /// + /// Provide embedded file content so that users can examine results in their full context + /// without having to enlist in the source repository. Embedding file content in a SARIF + /// log file can dramatically increase its size, so consider the usage scenario when you + /// decide whether to provide it. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2013_ProvideEmbeddedFileContent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2013_ProvideEmbeddedFileContent_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + protected override void Analyze(Run run, string runPointer) + { + if (run.Artifacts != null && run.Artifacts.All(artifact => artifact.Contents == null)) + { + // {0}: This run does not provide embedded file content. Providing embedded file + // content enables users to examine results in their full context without having + // to enlist in the source repository. Embedding file content in a SARIF log file + // can dramatically increase its size, so consider the usage scenario when you + // decide whether to provide it. + LogResult( + runPointer, + nameof(RuleResources.SARIF2013_ProvideEmbeddedFileContent_Note_Default_Text)); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2014.ProvideDynamicMessageContent.cs b/src/Sarif.Multitool/Rules/SARIF2014.ProvideDynamicMessageContent.cs new file mode 100644 index 000000000..8f20533d9 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2014.ProvideDynamicMessageContent.cs @@ -0,0 +1,94 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class ProvideDynamicMessageContent : SarifValidationSkimmerBase + { + /// + /// SARIF2014 + /// + public override string Id => RuleId.ProvideDynamicMessageContent; + + /// + /// Include "dynamic content" (information that varies among results from the same rule) to + /// makes your messages more specific, and to avoid the "wall of bugs" phenomenon, where + /// hundreds of occurrences of the same message appear unapproachable. + /// + /// This is part of a set of authoring practices that make your rule messages more readable, + /// understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and + /// 'SARIF2015.EnquoteDynamicMessageContent'. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2014_ProvideDynamicMessageContent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2014_ProvideDynamicMessageContent_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + private static readonly Regex s_dynamicContentRegex = new Regex(@"\{[0-9]+\}", RegexOptions.Compiled | RegexOptions.CultureInvariant); + + protected override void Analyze(Tool tool, string toolPointer) + { + if (tool.Driver != null) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (toolComponent.Rules != null) + { + string rulesPointer = toolDriverPointer.AtProperty(SarifPropertyName.Rules); + for (int i = 0; i < toolComponent.Rules.Count; i++) + { + AnalyzeReportingDescriptor(toolComponent.Rules[i], rulesPointer.AtIndex(i)); + } + } + } + + private void AnalyzeReportingDescriptor(ReportingDescriptor rule, string reportingDescriptorPointer) + { + if (rule.MessageStrings != null) + { + string messageStringsPointer = reportingDescriptorPointer.AtProperty(SarifPropertyName.MessageStrings); + foreach (KeyValuePair message in rule.MessageStrings) + { + AnalyzeMessageString(rule.Id, message.Value.Text, message.Key, messageStringsPointer.AtProperty(message.Key), SarifPropertyName.Text); + AnalyzeMessageString(rule.Id, message.Value.Markdown, message.Key, messageStringsPointer.AtProperty(message.Key), SarifPropertyName.Markdown); + } + } + } + + private void AnalyzeMessageString(string ruleId, string messageString, string messageKey, string messagePointer, string propertyName) + { + if (string.IsNullOrEmpty(messageString)) + { + return; + } + + string pointer = messagePointer.AtProperty(propertyName); + + if (!s_dynamicContentRegex.IsMatch(messageString)) + { + // {0}: In rule '{1}', the message with id '{2}' does not include any dynamic content. + // Dynamic content makes your messages more specific and avoids the "wall of bugs" + // phenomenon, where hundreds of occurrences of the same message appear unapproachable. + LogResult( + pointer, + nameof(RuleResources.SARIF2014_ProvideDynamicMessageContent_Note_Default_Text), + ruleId, + propertyName, + messageKey); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SARIF2015.EnquoteDynamicMessageContent.cs b/src/Sarif.Multitool/Rules/SARIF2015.EnquoteDynamicMessageContent.cs new file mode 100644 index 000000000..b72d0dd46 --- /dev/null +++ b/src/Sarif.Multitool/Rules/SARIF2015.EnquoteDynamicMessageContent.cs @@ -0,0 +1,94 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Text.RegularExpressions; + +using Microsoft.Json.Pointer; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool.Rules +{ + public class EnquoteDynamicMessageContent : SarifValidationSkimmerBase + { + /// + /// SARIF2015 + /// + public override string Id => RuleId.EnquoteDynamicMessageContent; + + /// + /// Place dynamic content in single quotes to set it off from the static text and to make it easier + /// to spot. It's especially helpful when the dynamic content is a string that might contain spaces, + /// and most especially when the string might be empty (and so would be invisible if it weren't for + /// the quotes). We recommend single quotes for a less cluttered appearance, even though US English + /// usage would require double quotes. + /// + /// This is part of a set of authoring practices that make your rule messages more readable, + /// understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and + /// 'SARIF2014.ProvideDynamicMessageContent'. + /// + public override MultiformatMessageString FullDescription => new MultiformatMessageString { Text = RuleResources.SARIF2015_EnquoteDynamicMessageContent_FullDescription_Text }; + + protected override IEnumerable MessageResourceNames => new string[] { + nameof(RuleResources.SARIF2015_EnquoteDynamicMessageContent_Note_Default_Text) + }; + + public override FailureLevel DefaultLevel => FailureLevel.Note; + + private static readonly Regex s_nonEnquotedDynamicContextRegex = new Regex(@"(^|[^'])\{[0-9]+\}", RegexOptions.Compiled | RegexOptions.CultureInvariant); + + protected override void Analyze(Tool tool, string toolPointer) + { + if (tool.Driver != null) + { + AnalyzeToolDriver(tool.Driver, toolPointer.AtProperty(SarifPropertyName.Driver)); + } + } + + private void AnalyzeToolDriver(ToolComponent toolComponent, string toolDriverPointer) + { + if (toolComponent.Rules != null) + { + string rulesPointer = toolDriverPointer.AtProperty(SarifPropertyName.Rules); + for (int i = 0; i < toolComponent.Rules.Count; i++) + { + AnalyzeReportingDescriptor(toolComponent.Rules[i], rulesPointer.AtIndex(i)); + } + } + } + + private void AnalyzeReportingDescriptor(ReportingDescriptor rule, string reportingDescriptorPointer) + { + if (rule.MessageStrings != null) + { + string messageStringsPointer = reportingDescriptorPointer.AtProperty(SarifPropertyName.MessageStrings); + foreach (KeyValuePair message in rule.MessageStrings) + { + AnalyzeMessageString(rule.Id, message.Value.Text, message.Key, messageStringsPointer.AtProperty(message.Key)); + } + } + } + + private void AnalyzeMessageString(string ruleId, string messageString, string messageKey, string messagePointer) + { + if (string.IsNullOrEmpty(messageString)) + { + return; + } + + string textPointer = messagePointer.AtProperty(SarifPropertyName.Text); + + if (s_nonEnquotedDynamicContextRegex.IsMatch(messageString)) + { + // {0}: In rule '{1}', the message with id '{2}' includes dynamic content that is not + // enclosed in single quotes. Enquoting dynamic content makes it easier to spot, and + // single quotes give a less cluttered appearance. + LogResult( + textPointer, + nameof(RuleResources.SARIF2015_EnquoteDynamicMessageContent_Note_Default_Text), + ruleId, + messageKey); + } + } + } +} diff --git a/src/Sarif.Multitool/Rules/SarifValidationSkimmerBase.cs b/src/Sarif.Multitool/Rules/SarifValidationSkimmerBase.cs index 2287c3e0b..10f50764a 100644 --- a/src/Sarif.Multitool/Rules/SarifValidationSkimmerBase.cs +++ b/src/Sarif.Multitool/Rules/SarifValidationSkimmerBase.cs @@ -1028,6 +1028,16 @@ private void Visit(Run run, string runPointer) Visit(run.WebResponses[i], webResponsesPointer.AtIndex(i)); } } + + if (run.OriginalUriBaseIds != null) + { + string originalUriBaseIdsPointer = runPointer.AtProperty(SarifPropertyName.OriginalUriBaseIds); + + foreach (string uriBaseId in run.OriginalUriBaseIds.Keys) + { + Visit(run.OriginalUriBaseIds[uriBaseId], originalUriBaseIdsPointer.AtProperty(uriBaseId)); + } + } } private void Visit(Stack stack, string stackPointer) diff --git a/src/Sarif.Multitool/Sarif.Multitool.csproj b/src/Sarif.Multitool/Sarif.Multitool.csproj index e0243d487..cdd531653 100644 --- a/src/Sarif.Multitool/Sarif.Multitool.csproj +++ b/src/Sarif.Multitool/Sarif.Multitool.csproj @@ -87,6 +87,9 @@ PreserveNewest + + PreserveNewest + diff --git a/src/Sarif.Multitool/SarifPropertyName.cs b/src/Sarif.Multitool/SarifPropertyName.cs index 948b44da3..4baa72f35 100644 --- a/src/Sarif.Multitool/SarifPropertyName.cs +++ b/src/Sarif.Multitool/SarifPropertyName.cs @@ -17,6 +17,7 @@ public static class SarifPropertyName public const string ArtifactChanges = "artifactChanges"; public const string CodeFlows = "codeFlows"; public const string Configuration = "configuration"; + public const string ContextRegion = "contextRegion"; public const string Conversion = "conversion"; public const string ConversionSources = "conversionSources"; public const string Description = "description"; @@ -51,6 +52,7 @@ public static class SarifPropertyName public const string Markdown = "markdown"; public const string Message = "message"; public const string MessageStrings = "messageStrings"; + public const string Name = "name"; public const string Nodes = "nodes"; public const string NotificationConfigurationOverrides = "notificationConfigurationOverrides"; public const string Notifications = "notifications"; @@ -74,6 +76,7 @@ public static class SarifPropertyName public const string RunGraphIndex = "runGraphIndex"; public const string Runs = "runs"; public const string Schema = "$schema"; + public const string SemanticVersion = "semanticVersion"; public const string ShortDescription = "shortDescription"; public const string Stacks = "stacks"; public const string Stdin = "stdin"; @@ -90,6 +93,7 @@ public static class SarifPropertyName public const string ToolConfigurationNotifications = "toolConfigurationNotifications"; public const string ToolExecutionNotifications = "toolExecutionNotifications"; public const string Uri = "uri"; + public const string Version = "version"; public const string VersionControlProvenance = "versionControlProvenance"; public const string WebRequest = "webRequest"; public const string WebRequests = "webRequests"; diff --git a/src/Sarif.Multitool/StringExtensions.cs b/src/Sarif.Multitool/StringExtensions.cs new file mode 100644 index 000000000..f0bbf145f --- /dev/null +++ b/src/Sarif.Multitool/StringExtensions.cs @@ -0,0 +1,18 @@ +// Copyright (c) Microsoft. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Microsoft.Json.Pointer; +using Newtonsoft.Json.Linq; + +namespace Microsoft.CodeAnalysis.Sarif.Multitool +{ + internal static class StringExtensions + { + public static JToken ToJToken(this string pointer, JToken parentToken) + { + var jsonPointer = new JsonPointer(pointer); + JToken jToken = jsonPointer.Evaluate(parentToken); + return jToken; + } + } +} diff --git a/src/Sarif.Multitool/default.configuration.xml b/src/Sarif.Multitool/default.configuration.xml new file mode 100644 index 000000000..158f5cb1a --- /dev/null +++ b/src/Sarif.Multitool/default.configuration.xml @@ -0,0 +1,12 @@ + + + + + + + + + + diff --git a/src/Sarif/Core/Region.cs b/src/Sarif/Core/Region.cs index e67219992..744eb9c76 100644 --- a/src/Sarif/Core/Region.cs +++ b/src/Sarif/Core/Region.cs @@ -1,27 +1,184 @@ // Copyright (c) Microsoft. All Rights Reserved. // Licensed under the MIT license. See LICENSE file in the project root for full license information. +using System; +using System.Runtime.CompilerServices; + namespace Microsoft.CodeAnalysis.Sarif { public partial class Region { - public bool IsBinaryRegion + public bool IsBinaryRegion => this.ByteOffset >= 0; + + public bool IsLineColumnBasedTextRegion => this.StartLine >= 1; + + public bool IsOffsetBasedTextRegion => this.CharOffset >= 0; + + public override string ToString() + { + return this.FormatForVisualStudio(); + } + + public void PopulateDefaults() { - get + if (this.IsLineColumnBasedTextRegion) + { + this.PopulateLineColumnBasedTextDefaults(); + } + + if (this.IsOffsetBasedTextRegion) { - // Is this right? What about an insertion point right after a BOM in a text file?? - // Do we need to just bite the bullet and make these Nullable type so that we have a - // clear indicator of whether the region is binary vs. textual? I tend to think so. - return - this.StartLine == 0 && - this.CharLength == 0 && - this.CharOffset == 0; + this.PopulateOffsetBasedTextDefaults(); + } + + if (this.IsBinaryRegion) + { + this.PopulateBinaryDefaults(); } } - public override string ToString() + private void PopulateLineColumnBasedTextDefaults() { - return this.FormatForVisualStudio(); + if (this.EndLine == 0) + { + this.EndLine = this.StartLine; + } + + if (this.StartColumn == 0) + { + this.StartColumn = 1; + } + + if (this.EndColumn == 0) + { + this.EndColumn = int.MaxValue; + } + } + + private void PopulateOffsetBasedTextDefaults() + { + if (this.CharLength == -1) + { + this.CharLength = 0; + } + } + + private void PopulateBinaryDefaults() + { + if (this.ByteLength == -1) + { + this.ByteLength = 0; + } + } + + public bool IsProperSupersetOf(Region subRegion) + { + this.PopulateDefaults(); + subRegion.PopulateDefaults(); + + if (this.IsLineColumnBasedTextRegion && + subRegion.IsLineColumnBasedTextRegion && + !IsLineColumnBasedTextRegionProperSupersetOf(subRegion)) + { + return false; + } + + if (this.IsOffsetBasedTextRegion && + subRegion.IsOffsetBasedTextRegion && + !IsOffsetBasedTextRegionProperSupetSetOf(subRegion)) + { + return false; + } + + if (this.IsBinaryRegion && + subRegion.IsBinaryRegion && + !IsBinaryRegionProperSupersetOf(subRegion)) + { + return false; + } + + // if we reach here, the region and context region have been expressed as different property sets, + // and it is not possible to judge validity without looking at the actual content. + // It is a potential false negative. + return true; + } + + private bool IsLineColumnBasedTextRegionProperSupersetOf(Region subRegion) + { + if (this.StartLine > subRegion.StartLine || this.EndLine < subRegion.EndLine) + { + return false; + } + + if (this.StartLine == subRegion.StartLine && this.StartColumn > subRegion.StartColumn) + { + return false; + } + + if (this.EndLine == subRegion.EndLine && this.EndColumn < subRegion.EndColumn) + { + return false; + } + + if (this.StartLine == subRegion.StartLine && + this.EndLine == subRegion.EndLine && + this.StartColumn == subRegion.StartColumn && + this.EndColumn == subRegion.EndColumn) + { + return false; + } + + return true; + } + + private bool IsBinaryRegionProperSupersetOf(Region subRegion) + { + if (this.ByteOffset > subRegion.ByteOffset) + { + return false; + } + + if (GetByteEndOffset(this) < GetByteEndOffset(subRegion)) + { + return false; + } + + if (this.ByteOffset == subRegion.ByteOffset && this.ByteLength <= subRegion.ByteLength) + { + return false; + } + + return true; + } + + private bool IsOffsetBasedTextRegionProperSupetSetOf(Region subRegion) + { + if (this.CharOffset > subRegion.CharOffset) + { + return false; + } + + if (GetCharEndOffset(this) < GetCharEndOffset(subRegion)) + { + return false; + } + + if (this.CharOffset == subRegion.CharOffset && this.CharLength <= subRegion.CharLength) + { + return false; + } + + return true; + } + + private static int GetCharEndOffset(Region region) + { + return region.CharOffset + region.CharLength; + } + + private static int GetByteEndOffset(Region region) + { + return region.ByteOffset + region.ByteLength; } } } diff --git a/src/Sarif/ExtensionMethods.cs b/src/Sarif/ExtensionMethods.cs index 8a01ff42c..5e675f068 100644 --- a/src/Sarif/ExtensionMethods.cs +++ b/src/Sarif/ExtensionMethods.cs @@ -344,19 +344,7 @@ public static string GetMessageText(this Result result, ReportingDescriptor rule internal static string GetFormattedMessage(string formatString, string[] arguments) { - string formattedMessage = null; - -#if DEBUG - int argumentsCount = arguments.Length; - for (int i = 0; i < argumentsCount; i++) - { - // If this assert fires, there are too many arguments for the specifier - // or there is an argument is skipped or not consumed in the specifier - Debug.Assert(formatString.Contains("{" + i.ToString(CultureInfo.InvariantCulture) + "}")); - } -#endif - - formattedMessage = string.Format(CultureInfo.InvariantCulture, formatString, arguments); + string formattedMessage = string.Format(CultureInfo.InvariantCulture, formatString, arguments); return formattedMessage ?? string.Empty; } diff --git a/src/Sarif/FileRegionsCache.cs b/src/Sarif/FileRegionsCache.cs index 76f2b4f6b..201eaac95 100644 --- a/src/Sarif/FileRegionsCache.cs +++ b/src/Sarif/FileRegionsCache.cs @@ -132,7 +132,7 @@ private void PopulatePropertiesFromCharOffsetAndLength(NewLineIndex newLineIndex { Assert(!region.IsBinaryRegion); Assert(region.StartLine == 0); - Assert(region.CharLength > 0 || region.CharOffset > 0); + Assert(region.CharLength >= 0 || region.CharOffset >= 0); int startLine, startColumn, endLine, endColumn; diff --git a/src/Sarif/RuleUtilities.cs b/src/Sarif/RuleUtilities.cs index d14f9d0ef..8fa2c425d 100644 --- a/src/Sarif/RuleUtilities.cs +++ b/src/Sarif/RuleUtilities.cs @@ -3,6 +3,7 @@ using System; using System.Collections.Generic; +using System.Linq; using System.Resources; namespace Microsoft.CodeAnalysis.Sarif @@ -132,6 +133,24 @@ public static string NormalizeRuleMessageId(string ruleMessageId, string ruleId, throw new ArgumentNullException(nameof(ruleMessageId)); } + // Per our convention, all message ids in validation rules + // should follow this naming scheme: + // + // ____ + // + // The message ids present in sarif file should follow this scheme: + // + // _ + // + string[] messageComponents = ruleMessageId.Split('_'); + if (messageComponents.Count() == 5) + { + return $"{messageComponents[2]}_{messageComponents[3]}"; + } + + // TODO: Once we have migrated all the our messages to follow the convention, + // we should remove the rest of the conditions. + // The following code is for backward compatibility. if (!string.IsNullOrEmpty(ruleId) && ruleMessageId.StartsWith(ruleId + "_", StringComparison.Ordinal)) { ruleMessageId = ruleMessageId.Substring(ruleId.Length + 1); diff --git a/src/Test.FunctionalTests.Sarif/Multitool/ValidateCommandTests.cs b/src/Test.FunctionalTests.Sarif/Multitool/ValidateCommandTests.cs index fe853af52..2bde15661 100644 --- a/src/Test.FunctionalTests.Sarif/Multitool/ValidateCommandTests.cs +++ b/src/Test.FunctionalTests.Sarif/Multitool/ValidateCommandTests.cs @@ -4,12 +4,18 @@ using System; using System.Collections.Generic; using System.IO; +using System.Linq; + using FluentAssertions; + using Microsoft.CodeAnalysis.Sarif.Multitool; using Microsoft.CodeAnalysis.Sarif.Multitool.Rules; using Microsoft.CodeAnalysis.Sarif.Visitors; + using Moq; + using Newtonsoft.Json; + using Xunit; using Xunit.Abstractions; @@ -23,6 +29,20 @@ public ValidateCommandTests(ITestOutputHelper outputHelper, bool testProducesSar protected override string IntermediateTestFolder => @"Multitool"; + private class TestParameters + { + internal bool Verbose { get; } + internal string ConfigFileName { get; } + + internal TestParameters(bool verbose = false, string configFileName = null) + { + Verbose = verbose; + ConfigFileName = configFileName ?? Path.Combine(Directory.GetCurrentDirectory(), "default.configuration.xml"); + } + } + + private static readonly TestParameters s_defaultTestParameters = new TestParameters(); + [Fact] public void JSON1001_SyntaxError() => RunTest("JSON1001.SyntaxError.sarif"); @@ -32,35 +52,44 @@ public void JSON1002_DeserializationError() => RunTest("JSON1002.DeserializationError.sarif"); [Fact] - public void SARIF1001_DoNotUseFriendlyNameAsRuleId_Valid() - => RunTest(MakeValidTestFileName(RuleId.DoNotUseFriendlyNameAsRuleId, nameof(RuleId.DoNotUseFriendlyNameAsRuleId))); + public void SARIF1001_RuleIdentifiersMustBeValid_Valid() + => RunTest(MakeValidTestFileName(RuleId.RuleIdentifiersMustBeValid, nameof(RuleId.RuleIdentifiersMustBeValid))); [Fact] - public void SARIF1001_DoNotUseFriendlyNameAsRuleId_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.DoNotUseFriendlyNameAsRuleId, nameof(RuleId.DoNotUseFriendlyNameAsRuleId))); + public void SARIF1001_RuleIdentifiersMustBeValid_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.RuleIdentifiersMustBeValid, nameof(RuleId.RuleIdentifiersMustBeValid))); [Fact] - public void SARIF1003_UrisMustBeValid_Valid() + public void SARIF1002_UrisMustBeValid_Valid() => RunTest(MakeValidTestFileName(RuleId.UrisMustBeValid, nameof(RuleId.UrisMustBeValid))); [Fact] - public void SARIF1003_UrisMustBeValid_Invalid() + public void SARIF1002_UrisMustBeValid_Invalid() => RunTest(MakeInvalidTestFileName(RuleId.UrisMustBeValid, nameof(RuleId.UrisMustBeValid))); [Fact] - public void SARIF1007_EndTimeMustNotBeBeforeStartTime_Valid() - => RunTest(MakeValidTestFileName(RuleId.EndTimeMustNotBeBeforeStartTime, nameof(RuleId.EndTimeMustNotBeBeforeStartTime))); + public void SARIF1004_ExpressUriBaseIdsCorrectly_Valid() + => RunTest(MakeValidTestFileName(RuleId.ExpressUriBaseIdsCorrectly, nameof(RuleId.ExpressUriBaseIdsCorrectly))); + + [Fact] + public void SARIF1004_ExpressUriBaseIdsCorrectly_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ExpressUriBaseIdsCorrectly, nameof(RuleId.ExpressUriBaseIdsCorrectly))); + + [Fact] + public void SARIF1005_UriMustBeAbsolute_Valid() + => RunTest(MakeValidTestFileName(RuleId.UriMustBeAbsolute, nameof(RuleId.UriMustBeAbsolute))); [Fact] - public void SARIF1007_EndTimeMustNotBeBeforeStartTime_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.EndTimeMustNotBeBeforeStartTime, nameof(RuleId.EndTimeMustNotBeBeforeStartTime))); + public void SARIF1005_UriMustBeAbsolute_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.UriMustBeAbsolute, nameof(RuleId.UriMustBeAbsolute))); + [Fact] - public void SARIF1008_MessagesShouldEndWithPeriod_Valid() - => RunTest(MakeValidTestFileName(RuleId.MessagesShouldEndWithPeriod, nameof(RuleId.MessagesShouldEndWithPeriod))); + public void SARIF1006_InvocationPropertiesMustBeConsistent_Valid() + => RunTest(MakeValidTestFileName(RuleId.InvocationPropertiesMustBeConsistent, nameof(RuleId.InvocationPropertiesMustBeConsistent))); [Fact] - public void SARIF1008_MessagesShouldEndWithPeriod_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.MessagesShouldEndWithPeriod, nameof(RuleId.MessagesShouldEndWithPeriod))); + public void SARIF1006_InvocationPropertiesMustBeConsistent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.InvocationPropertiesMustBeConsistent, nameof(RuleId.InvocationPropertiesMustBeConsistent))); /****************** * This set of tests constructs a full file path that exceeds MAX_PATH when running in some AzureDevOps build and test @@ -69,76 +98,210 @@ public void SARIF1008_MessagesShouldEndWithPeriod_Invalid() * test file names in TestData\Inputs and TestData\ExpectedOutputs. ******************/ [Fact] - public void SARIF1012_EndLineMustNotBeLessThanStartLine_Valid() - => RunTest(MakeValidTestFileName(RuleId.EndLineMustNotBeLessThanStartLine, "EndLineMustNotBeLessThanStart")); + public void SARIF1007_RegionPropertiesMustBeConsistent_Valid() + => RunTest(MakeValidTestFileName(RuleId.RegionPropertiesMustBeConsistent, "RegionPropertiesMustBeConsistent")); [Fact] - public void SARIF1012_EndLineMustNotBeLessThanStartLine_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.EndLineMustNotBeLessThanStartLine, "EndLineMustNotBeLessThanStart")); + public void SARIF1007_RegionPropertiesMustBeConsistent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.RegionPropertiesMustBeConsistent, "RegionPropertiesMustBeConsistent")); + + /********** END PROBLEMATIC TESTS*******/ + [Fact] - public void SARIF1013_EndColumnMustNotBeLessThanStartColumn_Valid() - => RunTest(MakeValidTestFileName(RuleId.EndColumnMustNotBeLessThanStartColumn, "EndColumnMustNotBeLessThanStart")); + public void SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Valid() + => RunTest(MakeValidTestFileName(RuleId.PhysicalLocationPropertiesMustBeConsistent, nameof(RuleId.PhysicalLocationPropertiesMustBeConsistent))); [Fact] - public void SARIF1013_EndColumnMustNotBeLessThanStartColumn_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.EndColumnMustNotBeLessThanStartColumn, "EndColumnMustNotBeLessThanStart")); - /********** END PROBLEMATIC TESTS*******/ + public void SARIF1008_PhysicalLocationPropertiesMustBeConsistent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.PhysicalLocationPropertiesMustBeConsistent, nameof(RuleId.PhysicalLocationPropertiesMustBeConsistent))); [Fact] - public void SARIF1014_UriBaseIdRequiresRelativeUri_Valid() - => RunTest(MakeValidTestFileName(RuleId.UriBaseIdRequiresRelativeUri, nameof(RuleId.UriBaseIdRequiresRelativeUri))); + public void SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Valid() + => RunTest(MakeValidTestFileName(RuleId.IndexPropertiesMustBeConsistentWithArrays, nameof(RuleId.IndexPropertiesMustBeConsistentWithArrays))); [Fact] - public void SARIF1014_UriBaseIdRequiresRelativeUri_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.UriBaseIdRequiresRelativeUri, nameof(RuleId.UriBaseIdRequiresRelativeUri))); + public void SARIF1009_IndexPropertiesMustBeConsistentWithArrays_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.IndexPropertiesMustBeConsistentWithArrays, nameof(RuleId.IndexPropertiesMustBeConsistentWithArrays))); [Fact] - public void SARIF1015_UriMustBeAbsolute_Valid() - => RunTest(MakeValidTestFileName(RuleId.UriMustBeAbsolute, nameof(RuleId.UriMustBeAbsolute))); + public void SARIF1010_RuleIdMustBeConsistent_Valid() + => RunTest(MakeValidTestFileName(RuleId.RuleIdMustBeConsistent, nameof(RuleId.RuleIdMustBeConsistent))); [Fact] - public void SARIF1015_UriMustBeAbsolute_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.UriMustBeAbsolute, nameof(RuleId.UriMustBeAbsolute))); + public void SARIF1010_RuleIdMustBeConsistent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.RuleIdMustBeConsistent, nameof(RuleId.RuleIdMustBeConsistent))); [Fact] - public void SARIF1016_ContextRegionRequiresRegion_Valid() - => RunTest(MakeValidTestFileName(RuleId.ContextRegionRequiresRegion, nameof(RuleId.ContextRegionRequiresRegion))); + public void SARIF1011_ReferenceFinalSchema_Valid() + => RunTest(MakeValidTestFileName(RuleId.ReferenceFinalSchema, nameof(RuleId.ReferenceFinalSchema))); [Fact] - public void SARIF1016_ContextRegionRequiresRegion_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.ContextRegionRequiresRegion, nameof(RuleId.ContextRegionRequiresRegion))); + public void SARIF1011_ReferenceFinalSchema_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ReferenceFinalSchema, nameof(RuleId.ReferenceFinalSchema))); [Fact] - public void SARIF1017_InvalidIndex_Valid() - => RunTest(MakeValidTestFileName(RuleId.InvalidIndex, nameof(RuleId.InvalidIndex))); + public void SARIF1012_MessageArgumentsMustBeConsistentWithRule_Valid() + => RunTest(MakeValidTestFileName(RuleId.MessageArgumentsMustBeConsistentWithRule, nameof(RuleId.MessageArgumentsMustBeConsistentWithRule))); [Fact] - public void SARIF1017_InvalidIndex_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.InvalidIndex, nameof(RuleId.InvalidIndex))); + public void SARIF1012_MessageArgumentsMustBeConsistentWithRule_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.MessageArgumentsMustBeConsistentWithRule, nameof(RuleId.MessageArgumentsMustBeConsistentWithRule))); [Fact] - public void SARIF1018_InvalidUriInOriginalUriBaseIds_Valid() - => RunTest(MakeValidTestFileName(RuleId.InvalidUriInOriginalUriBaseIds, nameof(RuleId.InvalidUriInOriginalUriBaseIds))); + public void SARIF2001_TerminateMessagesWithPeriod_Valid() + => RunTest(MakeValidTestFileName(RuleId.TerminateMessagesWithPeriod, nameof(RuleId.TerminateMessagesWithPeriod))); [Fact] - public void SARIF1018_InvalidUriInOriginalUriBaseIds_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.InvalidUriInOriginalUriBaseIds, nameof(RuleId.InvalidUriInOriginalUriBaseIds))); + public void SARIF2001_TerminateMessagesWithPeriod_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.TerminateMessagesWithPeriod, nameof(RuleId.TerminateMessagesWithPeriod))); [Fact] - public void SARIF1019_RuleIdMustBePresentAndConsistent_Valid() - => RunTest(MakeValidTestFileName(RuleId.RuleIdMustBePresentAndConsistent, nameof(RuleId.RuleIdMustBePresentAndConsistent))); + public void SARIF2002_ProvideMessageArguments_Valid() + => RunTest(MakeValidTestFileName(RuleId.ProvideMessageArguments, nameof(RuleId.ProvideMessageArguments)), + parameter: new TestParameters(configFileName: "enable2002.configuration.xml")); [Fact] - public void SARIF1019_RuleIdMustBePresentAndConsistent_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.RuleIdMustBePresentAndConsistent, nameof(RuleId.RuleIdMustBePresentAndConsistent))); + public void SARIF2002_ProvideMessageArguments_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ProvideMessageArguments, nameof(RuleId.ProvideMessageArguments)), + parameter: new TestParameters(configFileName: "enable2002.configuration.xml")); [Fact] - public void SARIF1020_SchemaMustBePresentAndConsistent_Valid() - => RunTest(MakeValidTestFileName(RuleId.ReferToFinalSchema, nameof(RuleId.ReferToFinalSchema))); + public void SARIF2003_ProvideVersionControlProvenance_Valid() + => RunTest(MakeValidTestFileName(RuleId.ProvideVersionControlProvenance, nameof(RuleId.ProvideVersionControlProvenance)), + parameter: new TestParameters(verbose: true)); [Fact] - public void SARIF1020_SchemaMustBePresentAndConsistent_Invalid() - => RunTest(MakeInvalidTestFileName(RuleId.ReferToFinalSchema, nameof(RuleId.ReferToFinalSchema))); + public void SARIF2003_ProvideVersionControlProvenance_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ProvideVersionControlProvenance, nameof(RuleId.ProvideVersionControlProvenance)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2004_OptimizeFileSize_Valid() + => RunTest(MakeValidTestFileName(RuleId.OptimizeFileSize, nameof(RuleId.OptimizeFileSize))); + + [Fact] + public void SARIF2004_OptimizeFileSize_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.OptimizeFileSize, nameof(RuleId.OptimizeFileSize))); + + [Fact] + public void SARIF2005_ProvideToolProperties_Valid() + => RunTest(MakeValidTestFileName(RuleId.ProvideToolProperties, nameof(RuleId.ProvideToolProperties))); + + [Fact] + public void SARIF2005_ProvideToolProperties_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ProvideToolProperties, nameof(RuleId.ProvideToolProperties))); + + [Fact] + public void SARIF2006_UrisShouldBeReachable_Valid() + => RunTest(MakeValidTestFileName(RuleId.UrisShouldBeReachable, nameof(RuleId.UrisShouldBeReachable)), + parameter: new TestParameters(configFileName: "enable2006.configuration.xml")); + + [Fact] + public void SARIF2006_UrisShouldBeReachable_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.UrisShouldBeReachable, nameof(RuleId.UrisShouldBeReachable)), + parameter: new TestParameters(configFileName: "enable2006.configuration.xml")); + + [Fact] + public void SARIF2007_ExpressPathsRelativeToRepoRoot_Valid() + => RunTest(MakeValidTestFileName(RuleId.ExpressPathsRelativeToRepoRoot, nameof(RuleId.ExpressPathsRelativeToRepoRoot)), + parameter: new TestParameters(configFileName: "enable2007.configuration.xml")); + [Fact] + public void SARIF2007_ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid() + => RunTest("SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif", + parameter: new TestParameters(configFileName: "enable2007.configuration.xml")); + + [Fact] + public void SARIF2007_ExpressPathsRelativeToRepoRoot_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ExpressPathsRelativeToRepoRoot, nameof(RuleId.ExpressPathsRelativeToRepoRoot)), + parameter: new TestParameters(configFileName: "enable2007.configuration.xml")); + + [Fact] + public void SARIF2008_ProvideSchema_Valid() + => RunTest(MakeValidTestFileName(RuleId.ProvideSchema, nameof(RuleId.ProvideSchema))); + + [Fact] + public void SARIF2008_ProvideSchema_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ProvideSchema, nameof(RuleId.ProvideSchema))); + + [Fact] + public void SARIF2009_ConsiderConventionalIdentifierValues_Valid() + => RunTest( + MakeValidTestFileName(RuleId.ConsiderConventionalIdentifierValues, nameof(RuleId.ConsiderConventionalIdentifierValues)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2009_ConsiderConventionalIdentifierValues_Invalid() + => RunTest( + MakeInvalidTestFileName(RuleId.ConsiderConventionalIdentifierValues, nameof(RuleId.ConsiderConventionalIdentifierValues)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2010_ProvideCodeSnippets_Valid() + => RunTest( + MakeValidTestFileName(RuleId.ProvideCodeSnippets, nameof(RuleId.ProvideCodeSnippets)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2010_ProvideCodeSnippets_Invalid() + => RunTest( + MakeInvalidTestFileName(RuleId.ProvideCodeSnippets, nameof(RuleId.ProvideCodeSnippets)), + parameter: new TestParameters(verbose: true, configFileName: "disable2011.configuration.xml")); + + [Fact] + public void SARIF2011_ProvideContextRegion_Valid() + => RunTest( + MakeValidTestFileName(RuleId.ProvideContextRegion, nameof(RuleId.ProvideContextRegion)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2011_ProvideContextRegion_Invalid() + => RunTest( + MakeInvalidTestFileName(RuleId.ProvideContextRegion, nameof(RuleId.ProvideContextRegion)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2012_ProvideHelpUris_Valid() + => RunTest( + MakeValidTestFileName(RuleId.ProvideHelpUris, nameof(RuleId.ProvideHelpUris)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2012_ProvideHelpUris_Invalid() + => RunTest( + MakeInvalidTestFileName(RuleId.ProvideHelpUris, nameof(RuleId.ProvideHelpUris)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2013_ProvideEmbeddedFileContent_Valid() + => RunTest( + MakeValidTestFileName(RuleId.ProvideEmbeddedFileContent, nameof(RuleId.ProvideEmbeddedFileContent)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2013_ProvideEmbeddedFileContent_Invalid() + => RunTest( + MakeInvalidTestFileName(RuleId.ProvideEmbeddedFileContent, nameof(RuleId.ProvideEmbeddedFileContent)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2014_ProvideDynamicMessageContent_Valid() + => RunTest(MakeValidTestFileName(RuleId.ProvideDynamicMessageContent, nameof(RuleId.ProvideDynamicMessageContent)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2014_ProvideDynamicMessageContent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.ProvideDynamicMessageContent, nameof(RuleId.ProvideDynamicMessageContent)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2015_EnquoteDynamicMessageContent_Valid() + => RunTest(MakeValidTestFileName(RuleId.EnquoteDynamicMessageContent, nameof(RuleId.EnquoteDynamicMessageContent)), + parameter: new TestParameters(verbose: true)); + + [Fact] + public void SARIF2015_EnquoteDynamicMessageContent_Invalid() + => RunTest(MakeInvalidTestFileName(RuleId.EnquoteDynamicMessageContent, nameof(RuleId.EnquoteDynamicMessageContent)), + parameter: new TestParameters(verbose: true)); private const string ValidTestFileNameSuffix = "_Valid.sarif"; private const string InvalidTestFileNameSuffix = "_Invalid.sarif"; @@ -163,11 +326,18 @@ protected override string ConstructTestOutputFromInputResource(string inputResou // All SARIF rule prefixes require update to current release. // All rules with JSON prefix are low level syntax/deserialization checks. - // We can't transform these test inputs as that operation fixes up erros in the file. - // Also, don't transform the tests for SARIF1020, because that rule examines the actual contents of the $schema - // property, so we can't change it. + // We can't transform these test inputs as that operation fixes up errors in the file. + // Also, don't transform the tests for SARIF1011 or SARIF2008, because these rules + // examine the actual contents of the $schema property. + + string[] shouldNotTransform = { "SARIF1011", "SARIF2008" }; + bool updateInputsToCurrentSarif = ruleUnderTest.StartsWith("SARIF") - && ruleUnderTest != "SARIF1020" ? true : false; + && !shouldNotTransform.Contains(ruleUnderTest); + + TestParameters testParameters = parameter != null + ? (TestParameters)parameter + : s_defaultTestParameters; var validateOptions = new ValidateOptions { @@ -177,7 +347,9 @@ protected override string ConstructTestOutputFromInputResource(string inputResou Quiet = true, UpdateInputsToCurrentSarif = updateInputsToCurrentSarif, PrettyPrint = true, - Optimize = true + Optimize = true, + Verbose = testParameters.Verbose, + ConfigurationFilePath = testParameters.ConfigFileName }; var mockFileSystem = new Mock(); @@ -188,6 +360,7 @@ protected override string ConstructTestOutputFromInputResource(string inputResou mockFileSystem.Setup(x => x.ReadAllText(inputLogFilePath)).Returns(v2LogText); mockFileSystem.Setup(x => x.ReadAllText(It.IsNotIn(inputLogFilePath))).Returns(path => File.ReadAllText(path)); mockFileSystem.Setup(x => x.WriteAllText(It.IsAny(), It.IsAny())); + mockFileSystem.Setup(x => x.FileExists(validateOptions.ConfigurationFilePath)).Returns(true); var validateCommand = new ValidateCommand(mockFileSystem.Object); @@ -200,7 +373,8 @@ protected override string ConstructTestOutputFromInputResource(string inputResou returnCode.Should().Be(0); - SarifLog actualLog = JsonConvert.DeserializeObject(File.ReadAllText(actualLogFilePath)); + string actualLogFileContents = File.ReadAllText(actualLogFilePath); + SarifLog actualLog = JsonConvert.DeserializeObject(actualLogFileContents); // First, we'll strip any validation results that don't originate with the rule under test var newResults = new List(); diff --git a/src/Test.FunctionalTests.Sarif/Test.FunctionalTests.Sarif.csproj b/src/Test.FunctionalTests.Sarif/Test.FunctionalTests.Sarif.csproj index b3996eff6..23f859bc4 100644 --- a/src/Test.FunctionalTests.Sarif/Test.FunctionalTests.Sarif.csproj +++ b/src/Test.FunctionalTests.Sarif/Test.FunctionalTests.Sarif.csproj @@ -352,6 +352,21 @@ + + PreserveNewest + + + PreserveNewest + + + PreserveNewest + + + PreserveNewest + + + PreserveNewest + PreserveNewest diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1001.SyntaxError.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1001.SyntaxError.sarif index b460b332d..066e3b781 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1001.SyntaxError.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1001.SyntaxError.sarif @@ -27,6 +27,32 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1002.DeserializationError.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1002.DeserializationError.sarif index 4f60ba5bc..0b4c7460f 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1002.DeserializationError.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/JSON1002.DeserializationError.sarif @@ -27,6 +27,32 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif deleted file mode 100644 index b206144a8..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif +++ /dev/null @@ -1,95 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1001", - "name": "DoNotUseFriendlyNameAsRuleId", - "shortDescription": { - "text": "Do not use the same string for a rule's id and name properties." - }, - "fullDescription": { - "text": "Do not use the same string for a rule's id and name properties. The id property must be a stable, opaque identifer such as \"SARIF1001\". The name property should be a string that is understandable to an end user, such as \"DoNotUserFriendlyNameAsRuleId\"." - }, - "messageStrings": { - "Default": { - "text": "{0}: The name and id properties of rule \"{1}\" are the same." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1001", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].tool.driver.rules[0]", - "RULE0001" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 10, - "startColumn": 13 - } - } - } - ] - }, - { - "ruleId": "SARIF1001", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].tool.driver.rules[1]", - "RULE0002" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 14, - "startColumn": 13 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif deleted file mode 100644 index c5d894f0a..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif new file mode 100644 index 000000000..03c98ee64 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif @@ -0,0 +1,121 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1001", + "name": "RuleIdentifiersMustBeValid", + "shortDescription": { + "text": "The two identity-related properties of a SARIF rule must be consistent." + }, + "fullDescription": { + "text": "The two identity-related properties of a SARIF rule must be consistent. The required 'id' property must be a \"stable, opaque identifier\" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If both 'name' and 'id' are opaque identifiers, omit the 'name' property. If both 'name' and 'id' are human-readable identifiers, then consider assigning an opaque identifier to each rule, but in the meantime, omit the 'name' property." + }, + "messageStrings": { + "Error_Default": { + "text": "{0}: The rule '{1}' has a 'name' property that is identical to its 'id' property. The required 'id' property must be a \"stable, opaque identifier\" (the SARIF specification ([3.49.3](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317839)) explains the reasons for this). The optional 'name' property ([3.49.7](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317843)) is an identifer that is understandable to an end user. Therefore if both 'id' and 'name' are present, they must be different. If they are identical, the tool must omit the 'name' property." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1001", + "ruleIndex": 0, + "message": { + "id": "Error_Default", + "arguments": [ + "runs[0].tool.driver.rules[0]", + "RULE0001" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 11, + "startColumn": 13 + } + } + } + ] + }, + { + "ruleId": "SARIF1001", + "ruleIndex": 0, + "message": { + "id": "Error_Default", + "arguments": [ + "runs[0].tool.driver.rules[1]", + "RULE0002" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 15, + "startColumn": 13 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif new file mode 100644 index 000000000..8b22c2a2b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Invalid.sarif similarity index 60% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Invalid.sarif index 4276722b0..60f3ecbac 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Invalid.sarif @@ -8,36 +8,36 @@ "name": "SARIF Functional Testing", "rules": [ { - "id": "SARIF1003", + "id": "SARIF1002", "name": "UrisMustBeValid", "shortDescription": { "text": "Specify a valid URI reference for every URI-valued property." }, "fullDescription": { - "text": "Specify a valid URI reference for every URI-valued property." + "text": "Specify a valid URI reference for every URI-valued property.\r\n\r\nURIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, 'file' URIs must not include '..' segments. If symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it." }, "messageStrings": { - "Default": { - "text": "{0}: The string \"{1}\" is not a valid URI reference." + "Error_UrisMustConformToRfc3986": { + "text": "{0}: The string '{1}' is not a valid URI reference. URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986)." + }, + "Error_FileUrisMustNotIncludeDotDotSegments": { + "text": "{0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it." } }, "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" }, { - "id": "SARIF1020", - "name": "ReferToFinalSchema", + "id": "SARIF1011", + "name": "ReferenceFinalSchema", "shortDescription": { - "text": "The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema." + "text": "The '$schema' property must refer to the final version of the SARIF 2.1.0 schema." }, "fullDescription": { - "text": "The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." + "text": "The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.\r\n\r\nThe SARIF standard was developed over several years, and many intermediate versions of the schema were produced. Now that the standard is final, only the OASIS standard version of the schema is valid." }, "messageStrings": { - "ReferenceToOldSchemaVersion": { - "text": "{0}: The $schema property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the $schema property with a URL that refers to the final version of the schema." - }, - "SchemaReferenceMissing": { - "text": "{0}: The SARIF log file does not contain a $schema property. Add a $schema property that refers to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." + "Error_Default": { + "text": "{0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the '$schema' property with a URL that refers to the final version of the schema." } }, "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" @@ -47,24 +47,50 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], "artifacts": [ { "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1003.UrisMustBeValid_Invalid.sarif", + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1002.UrisMustBeValid_Invalid.sarif", "uriBaseId": "TEST_DIR" } } ], "results": [ { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "$schema", "ht%tp://json.schemastore.org/sarif-2.0.0" @@ -85,14 +111,14 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ - "runs[0].originalUriBaseIds.SRCROOT.uri", - "fi%le:///c:/Code/sarif-sdk/src" + "runs[0].results[0].workItemUris[0]", + "ht&tp://example.com/my-project/issues/42" ] }, "locations": [ @@ -102,22 +128,22 @@ "index": 0 }, "region": { - "startLine": 35, - "startColumn": 49 + "startLine": 58, + "startColumn": 54 } } } ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ - "runs[0].results[0].workItemUris[0]", - "ht&tp://example.com/my-project/issues/42" + "runs[0].results[0].analysisTarget.uri", + "fi&le:///c:/src/file.c" ] }, "locations": [ @@ -127,22 +153,22 @@ "index": 0 }, "region": { - "startLine": 57, - "startColumn": 54 + "startLine": 55, + "startColumn": 43 } } } ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_FileUrisMustNotIncludeDotDotSegments", "arguments": [ - "runs[0].results[0].analysisTarget.uri", - "fi&le:///c:/src/file.c" + "runs[0].results[1].analysisTarget.uri", + "file:///c:/src/src2/src3/../../file.c" ] }, "locations": [ @@ -152,19 +178,19 @@ "index": 0 }, "region": { - "startLine": 54, - "startColumn": 43 + "startLine": 68, + "startColumn": 58 } } } ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].artifacts[0].location.uri", "fi%le:///c:/src/file.c" @@ -177,7 +203,7 @@ "index": 0 }, "region": { - "startLine": 41, + "startLine": 42, "startColumn": 43 } } @@ -185,11 +211,11 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].tool.driver.downloadUri", "ht%tp://www.example.com/tools/codescanner/download.html" @@ -202,7 +228,7 @@ "index": 0 }, "region": { - "startLine": 9, + "startLine": 10, "startColumn": 82 } } @@ -210,11 +236,11 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].tool.driver.notifications[0].helpUri", "ht%tp://www.example.com/rules/msg0001.html" @@ -227,7 +253,7 @@ "index": 0 }, "region": { - "startLine": 13, + "startLine": 14, "startColumn": 69 } } @@ -235,11 +261,11 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].tool.driver.notifications[1].helpUri", "ht%tp://www.example.com/rules/msg0002.html" @@ -252,7 +278,7 @@ "index": 0 }, "region": { - "startLine": 17, + "startLine": 18, "startColumn": 69 } } @@ -260,11 +286,11 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].tool.driver.rules[0].helpUri", "ht%tp://www.example.com/rules/tst0001.html" @@ -277,7 +303,7 @@ "index": 0 }, "region": { - "startLine": 23, + "startLine": 24, "startColumn": 69 } } @@ -285,11 +311,11 @@ ] }, { - "ruleId": "SARIF1003", + "ruleId": "SARIF1002", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_UrisMustConformToRfc3986", "arguments": [ "runs[0].versionControlProvenance[0].repositoryUri", "ht%tps://example.com/my-project" @@ -302,7 +328,7 @@ "index": 0 }, "region": { - "startLine": 30, + "startLine": 31, "startColumn": 60 } } @@ -310,11 +336,36 @@ ] }, { - "ruleId": "SARIF1020", + "ruleId": "SARIF1002", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_UrisMustConformToRfc3986", + "arguments": [ + "runs[0].originalUriBaseIds.SRCROOT.uri", + "fi%le:///c:/Code/sarif-sdk/src" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 36, + "startColumn": 49 + } + } + } + ] + }, + { + "ruleId": "SARIF1011", "ruleIndex": 1, "level": "error", "message": { - "id": "ReferenceToOldSchemaVersion", + "id": "Error_Default", "arguments": [ "$schema", "ht%tp://json.schemastore.org/sarif-2.0.0" diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Valid.sarif new file mode 100644 index 000000000..bf0927b07 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1002.UrisMustBeValid_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1002.UrisMustBeValid_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif new file mode 100644 index 000000000..fa1418469 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif @@ -0,0 +1,857 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1002", + "name": "UrisMustBeValid", + "shortDescription": { + "text": "Specify a valid URI reference for every URI-valued property." + }, + "fullDescription": { + "text": "Specify a valid URI reference for every URI-valued property.\r\n\r\nURIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986). In addition, 'file' URIs must not include '..' segments. If symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it." + }, + "messageStrings": { + "Error_UrisMustConformToRfc3986": { + "text": "{0}: The string '{1}' is not a valid URI reference. URIs must conform to [RFC 3986](https://tools.ietf.org/html/rfc3986)." + }, + "Error_FileUrisMustNotIncludeDotDotSegments": { + "text": "{0}: The 'file' URI '{1}' contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + }, + { + "id": "SARIF1004", + "name": "ExpressUriBaseIdsCorrectly", + "shortDescription": { + "text": "When using the 'uriBaseId' property, obey the requirements in the SARIF specification [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) that enable it to fulfill its purpose of resolving relative references to absolute locations." + }, + "fullDescription": { + "text": "When using the 'uriBaseId' property, obey the requirements in the SARIF specification [3.4.4](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317431) that enable it to fulfill its purpose of resolving relative references to absolute locations. In particular:\r\n\r\nIf an 'artifactLocation' object has a 'uriBaseId' property, its 'uri' property must be a relative reference, because if 'uri' is an absolute URI then 'uriBaseId' serves no purpose.\r\n\r\nEvery URI reference in 'originalUriBaseIds' must resolve to an absolute URI in the manner described in the SARIF specification [3.14.14](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317498)." + }, + "messageStrings": { + "Error_UriBaseIdRequiresRelativeUri": { + "text": "{0}: This 'artifactLocation' object has a 'uriBaseId' property '{1}', but its 'uri' property '{2}' is an absolute URI. Since the purpose of 'uriBaseId' is to resolve a relative reference to an absolute URI, it is not allowed when the 'uri' property is already an absolute URI." + }, + "Error_TopLevelUriBaseIdMustBeAbsolute": { + "text": "{0}: The '{1}' element of 'originalUriBaseIds' has no 'uriBaseId' property, but its 'uri' property '{2}' is not an absolute URI. According to the SARIF specification, every such \"top-level\" entry in 'originalUriBaseIds' must specify an absolute URI, because the purpose of 'originalUriBaseIds' is to enable the resolution of relative references to absolute URIs." + }, + "Error_UriBaseIdValueMustEndWithSlash": { + "text": "{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that does not end with a slash. The trailing slash is required to minimize the likelihood of an error when concatenating URI segments together." + }, + "Error_UriBaseIdValueMustNotContainDotDotSegment": { + "text": "{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a '..' segment. This is dangerous because if symbolic links are present, '..' might have different meanings on the machine that produced the log file and the machine where an end user or a tool consumes it." + }, + "Error_UriBaseIdValueMustNotContainQueryOrFragment": { + "text": "{0}: The '{1}' element of 'originalUriBaseIds' has a 'uri' property '{2}' that contains a query or a fragment. This is not valid because the purpose of the 'uriBaseId' property is to help resolve a relative reference to an absolute URI by concatenating the relative reference to the absolute base URI. This won't work if the base URI contains a query or a fragment." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1002", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_FileUrisMustNotIncludeDotDotSegments", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_DOTDOT.uri", + "file:///C:/rules/dir1/dir2/../" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 115, + "startColumn": 49 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_TopLevelUriBaseIdMustBeAbsolute", + "arguments": [ + "runs[0].originalUriBaseIds.REPO_ROOT", + "REPO_ROOT", + "repository" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 101, + "startColumn": 22 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustEndWithSlash", + "arguments": [ + "runs[0].originalUriBaseIds.REPO_ROOT", + "REPO_ROOT", + "repository" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 101, + "startColumn": 22 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustEndWithSlash", + "arguments": [ + "runs[0].originalUriBaseIds.SOURCE_ROOT", + "SOURCE_ROOT", + "src" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 107, + "startColumn": 24 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustNotContainDotDotSegment", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_DOTDOT", + "BAD_URI_CONTAINING_DOTDOT", + "file:///C:/rules/dir1/dir2/../" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 114, + "startColumn": 38 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustEndWithSlash", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_QUERY", + "BAD_URI_CONTAINING_QUERY", + "http://www.contoso.com/catalog/shownew.htm?date=today" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 120, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustNotContainQueryOrFragment", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_QUERY", + "BAD_URI_CONTAINING_QUERY", + "http://www.contoso.com/catalog/shownew.htm?date=today" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 120, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustEndWithSlash", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_FRAGMENT", + "BAD_URI_CONTAINING_FRAGMENT", + "http://www.contoso.com/index.htm#search" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 126, + "startColumn": 40 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdValueMustNotContainQueryOrFragment", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_CONTAINING_FRAGMENT", + "BAD_URI_CONTAINING_FRAGMENT", + "http://www.contoso.com/index.htm#search" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 126, + "startColumn": 40 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].conversion.analysisToolLogFiles[0]", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 84, + "startColumn": 11 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].analysisTarget", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 155, + "startColumn": 29 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].attachments[0].artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 217, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 162, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 193, + "startColumn": 47 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].provenance.conversionSources[0].artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 226, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].stacks[0].frames[0].location.physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 175, + "startColumn": 43 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].relatedLocations[0].physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 208, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].results[0].fixes[0].artifactChanges[0].artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 237, + "startColumn": 39 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].artifacts[0].location", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 142, + "startColumn": 23 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].executableLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 55, + "startColumn": 33 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].responseFiles[0]", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 15, + "startColumn": 13 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].stdin", + "%SRCROOT%", + "file:///c:/log/in.txt" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 59, + "startColumn": 20 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].stdout", + "%SRCROOT%", + "file:///c:/log/out.txt" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 63, + "startColumn": 21 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].stderr", + "%SRCROOT%", + "file:///c:/log/err.txt" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 67, + "startColumn": 21 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].stdoutStderr", + "%SRCROOT%", + "file:///c:/log/out-err.txt" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 71, + "startColumn": 27 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].toolExecutionNotifications[0].locations[0].physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 25, + "startColumn": 41 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].invocations[0].toolConfigurationNotifications[0].locations[0].physicalLocation.artifactLocation", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 42, + "startColumn": 41 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].versionControlProvenance[0].mappedTo", + "%SRCROOT%", + "file:///C:/src/file.c" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 94, + "startColumn": 23 + } + } + } + ] + }, + { + "ruleId": "SARIF1004", + "ruleIndex": 1, + "level": "error", + "message": { + "id": "Error_UriBaseIdRequiresRelativeUri", + "arguments": [ + "runs[0].originalUriBaseIds.BAD_URI_HAS_BASEID_BUT_ALSO_ABSOLUTE_URI", + "SOURCE_ROOT", + "file:///C:/rules/" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 132, + "startColumn": 53 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif new file mode 100644 index 000000000..004c51dc0 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif similarity index 69% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif index 427b001d6..9ada20ce7 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif @@ -8,17 +8,17 @@ "name": "SARIF Functional Testing", "rules": [ { - "id": "SARIF1015", + "id": "SARIF1005", "name": "UriMustBeAbsolute", "shortDescription": { "text": "Certain URIs are required to be absolute." }, "fullDescription": { - "text": "Certain URIs are required to be absolute." + "text": "Certain URIs are required to be absolute. For the most part, these are URIs that refer to http addresses, such as work items or rule help topics." }, "messageStrings": { - "Default": { - "text": "{0}: The value of this property is required to be an absolute URI, but \"{1}\" is a relative URI reference." + "Error_Default": { + "text": "{0}: The value of this property is required to be an absolute URI, but '{1}' is a relative URI reference." } }, "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" @@ -28,24 +28,50 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], "artifacts": [ { "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1015.UriMustBeAbsolute_Invalid.sarif", + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1005.UriMustBeAbsolute_Invalid.sarif", "uriBaseId": "TEST_DIR" } } ], "results": [ { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "$schema", "json.schemastore.org/sarif-2.1.0-rtm.5" @@ -66,11 +92,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].results[0].workItemUris[0]", "example.com/my-project/issues/42" @@ -83,7 +109,7 @@ "index": 0 }, "region": { - "startLine": 41, + "startLine": 42, "startColumn": 46 } } @@ -91,11 +117,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].tool.driver.downloadUri", "tools/codescanner/download.html" @@ -108,7 +134,7 @@ "index": 0 }, "region": { - "startLine": 9, + "startLine": 10, "startColumn": 58 } } @@ -116,11 +142,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].tool.driver.notifications[0].helpUri", "www.example.com/notifications/msg0001.html" @@ -133,7 +159,7 @@ "index": 0 }, "region": { - "startLine": 13, + "startLine": 14, "startColumn": 69 } } @@ -141,11 +167,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].tool.driver.notifications[1].helpUri", "www.example.com/notifications/msg0002.html" @@ -158,7 +184,7 @@ "index": 0 }, "region": { - "startLine": 17, + "startLine": 18, "startColumn": 69 } } @@ -166,11 +192,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].tool.driver.rules[0].helpUri", "www.example.com/rules/tst0001.html" @@ -183,7 +209,7 @@ "index": 0 }, "region": { - "startLine": 23, + "startLine": 24, "startColumn": 61 } } @@ -191,11 +217,11 @@ ] }, { - "ruleId": "SARIF1015", + "ruleId": "SARIF1005", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_Default", "arguments": [ "runs[0].versionControlProvenance[0].repositoryUri", "example.com/my-project" @@ -208,7 +234,7 @@ "index": 0 }, "region": { - "startLine": 30, + "startLine": 31, "startColumn": 51 } } diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Valid.sarif new file mode 100644 index 000000000..16a05afca --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1005.UriMustBeAbsolute_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1005.UriMustBeAbsolute_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..22a66f904 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif @@ -0,0 +1,99 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1006", + "name": "InvocationPropertiesMustBeConsistent", + "shortDescription": { + "text": "The properties of an 'invocation' object must be consistent." + }, + "fullDescription": { + "text": "The properties of an 'invocation' object must be consistent.\r\n\r\nIf the 'invocation' object specifies both 'startTimeUtc' and 'endTimeUtc', then 'endTimeUtc' must not precede 'startTimeUtc'. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal." + }, + "messageStrings": { + "Error_EndTimeMustNotPrecedeStartTime": { + "text": "{0}: The 'endTimeUtc' value '{1}' precedes the 'startTimeUtc' value '{2}'. The properties of an 'invocation' object must be internally consistent." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1006", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndTimeMustNotPrecedeStartTime", + "arguments": [ + "runs[0].invocations[0].endTimeUtc", + "2016-08-25T21:26:40.051Z", + "2016-08-25T21:26:42.049Z" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 16, + "startColumn": 50 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif new file mode 100644 index 000000000..325de483f --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif deleted file mode 100644 index b83d6bb48..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif +++ /dev/null @@ -1,73 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1007", - "name": "EndTimeMustNotBeBeforeStartTime", - "shortDescription": { - "text": "The end time of a run must not precede the start time." - }, - "fullDescription": { - "text": "The end time of a run must not precede the start time. To allow for the possibility that the duration of the run is less than the resolution of the string representation of the time, the start time and the end time may be equal." - }, - "messageStrings": { - "Default": { - "text": "{0}: The end time \"{1}\" is before the start time \"{2}\"." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1007", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].endTimeUtc", - "2016-08-25T21:26:40.051Z", - "2016-08-25T21:26:42.049Z" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 15, - "startColumn": 50 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif deleted file mode 100644 index aaa781812..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..2841230b1 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif @@ -0,0 +1,311 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1007", + "name": "RegionPropertiesMustBeConsistent", + "shortDescription": { + "text": "The properties of a 'region' object must be consistent." + }, + "fullDescription": { + "text": "The properties of a 'region' object must be consistent.\r\n\r\nSARIF can specify a 'region' (a contiguous portion of a file) in a variety of ways: with line and column numbers, with a character offset and count, or with a byte offset and count. The specification states certain constraints on these properties, both within each property group (for example, the start line cannot be greater than end line) and between the groups (for example, if more than one group is present, they must independently specify the same portion of the file). See the SARIF specification ([3.30](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317685))." + }, + "messageStrings": { + "Error_EndLineMustNotPrecedeStartLine": { + "text": "{0}: In this 'region' object, the 'endLine' property '{1}' is less than the 'startLine' property '{2}'. The properties of a 'region' object must be internally consistent." + }, + "Error_EndColumnMustNotPrecedeStartColumn": { + "text": "{0}: In this 'region' object, the 'endColumn' property '{1}' is less than the 'startColumn' property '{2}'. The properties of a 'region' object must be internally consistent." + }, + "Error_RegionStartPropertyMustBePresent": { + "text": "{0}: This 'region' object does not specify 'startLine', 'charOffset', or 'byteOffset'. As a result, it is impossible to determine whether this 'region' object describes a line/column text region, a character offset/length text region, or a binary region." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndColumnMustNotPrecedeStartColumn", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation.region.endColumn", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 28, + "startColumn": 32 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndColumnMustNotPrecedeStartColumn", + "arguments": [ + "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.region.endColumn", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 73, + "startColumn": 42 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndColumnMustNotPrecedeStartColumn", + "arguments": [ + "runs[0].results[0].stacks[0].frames[0].location.physicalLocation.region.endColumn", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 48, + "startColumn": 38 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndColumnMustNotPrecedeStartColumn", + "arguments": [ + "runs[0].results[0].relatedLocations[0].physicalLocation.region.endColumn", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 95, + "startColumn": 32 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndLineMustNotPrecedeStartLine", + "arguments": [ + "runs[0].results[1].locations[0].physicalLocation.region.endLine", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 118, + "startColumn": 30 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndLineMustNotPrecedeStartLine", + "arguments": [ + "runs[0].results[1].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.region.endLine", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 161, + "startColumn": 40 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndLineMustNotPrecedeStartLine", + "arguments": [ + "runs[0].results[1].stacks[0].frames[0].location.physicalLocation.region.endLine", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 137, + "startColumn": 36 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_EndLineMustNotPrecedeStartLine", + "arguments": [ + "runs[0].results[1].relatedLocations[0].physicalLocation.region.endLine", + "1", + "2" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 182, + "startColumn": 30 + } + } + } + ] + }, + { + "ruleId": "SARIF1007", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_RegionStartPropertyMustBePresent", + "arguments": [ + "runs[0].results[1].relatedLocations[1].physicalLocation.region" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 194, + "startColumn": 27 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif new file mode 100644 index 000000000..a831edab8 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif deleted file mode 100644 index cf34b9473..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif +++ /dev/null @@ -1,431 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1008", - "name": "MessagesShouldEndWithPeriod", - "shortDescription": { - "text": "Messages should consist of one or more complete sentences, ending with a period." - }, - "fullDescription": { - "text": "Messages should consist of one or more complete sentences, ending with a period." - }, - "messageStrings": { - "Default": { - "text": "{0}: The message \"{1}\" does not end with a period." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].attachments[0].description.text", - "attachment description without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 136, - "startColumn": 63 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].attachments[0].rectangles[0].message.text", - "rectangle message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 147, - "startColumn": 62 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].locations[0].message.text", - "location message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 75, - "startColumn": 57 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].locations[0].physicalLocation.region.message.text", - "region message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 69, - "startColumn": 59 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].codeFlows[0].message.text", - "codeFlow message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 101, - "startColumn": 57 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].codeFlows[0].threadFlows[0].message.text", - "threadFlow message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 107, - "startColumn": 63 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.message.text", - "ThreadFlow location message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 123, - "startColumn": 78 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].message.text", - "result message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 57, - "startColumn": 51 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].stacks[0].frames[0].location.message.text", - "frame message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 90, - "startColumn": 60 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].stacks[0].message.text", - "stack message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 83, - "startColumn": 54 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].fixes[0].description.text", - "fix description without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 157, - "startColumn": 56 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].toolExecutionNotifications[0].message.text", - "toolNotification message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 35, - "startColumn": 65 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].toolConfigurationNotifications[0].message.text", - "configurationNotification message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 43, - "startColumn": 74 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].tool.driver.rules[0].messageStrings.Default.text", - "rule message without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 22, - "startColumn": 55 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].tool.driver.rules[0].shortDescription.text", - "Short rule description without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 13, - "startColumn": 63 - } - } - } - ] - }, - { - "ruleId": "SARIF1008", - "ruleIndex": 0, - "message": { - "id": "Default", - "arguments": [ - "runs[0].tool.driver.rules[0].fullDescription.text", - "Short full rule description without period" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 17, - "startColumn": 68 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif deleted file mode 100644 index ae8ab95b6..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..154b490cf --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif @@ -0,0 +1,484 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1008", + "name": "PhysicalLocationPropertiesMustBeConsistent", + "shortDescription": { + "text": "Ensure consistency among the properties of a 'physicalLocation' object." + }, + "fullDescription": { + "text": "Ensure consistency among the properties of a 'physicalLocation' object.\r\n\r\nA SARIF 'physicalLocation' object has two related properties 'region' and 'contextRegion'. If 'contextRegion' is present, then 'region' must also be present, and 'contextRegion' must be a \"proper superset\" of 'region'. That is, 'contextRegion' must completely contain 'region', and it must be larger than 'region'. To understand why this is so we must understand the roles of the 'region' and 'contextRegion' properties.\r\n\r\n'region' allows both users and tools to distinguish similar results within the same artifact. If a SARIF viewer has access to the artifact, it can display it, and highlight the location identified by the analysis tool. If the region has a 'snippet' property, then even if the viewer doesn't have access to the artifact (which might be the case for a web-based viewer), it can still display the faulty code.\r\n\r\n'contextRegion' provides users with a broader view of the result location. Typically, it consists of a range starting a few lines before 'region' and ending a few lines after. Again, if a SARIF viewer has access to the artifact, it can display it, and highlight the context region (perhaps in a lighter shade than the region itself). This isn't terribly useful since the user can already see the whole file, with the 'region' already highlighted. But if 'contextRegion' has a 'snippet' property, then even a viewer without access to the artifact can display a few lines of code surrounding the actual result, which is helpful to users.\r\n\r\nIf the validator reports that 'contextRegion' is not a proper superset of 'region', then it's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool should simply omit 'contextRegion'." + }, + "messageStrings": { + "Error_ContextRegionRequiresRegion": { + "text": "{0}: This 'physicalLocation' object contains a 'contextRegion' property, but it does not contain a 'region' property. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. If a tool associates only one region with a result, it must populate 'region', not 'contextRegion'." + }, + "Error_ContextRegionMustBeProperSupersetOfRegion": { + "text": "{0}: This 'physicalLocation' object contains both a 'region' and a 'contextRegion' property, but 'contextRegion' is not a proper superset of 'region'. This is invalid because the purpose of 'contextRegion' is to provide a viewing context around the 'region' which is the location of the result. It's possible that the tool reversed 'region' and 'contextRegion'. If 'region' and 'contextRegion' are identical, the tool must omit 'contextRegion'." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionRequiresRegion", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 21, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[1].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 41, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[2].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 69, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[3].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 93, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[4].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 117, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[5].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 141, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[6].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 164, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[7].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 189, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[8].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 213, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[9].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 237, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[10].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 261, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[11].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 285, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[12].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 308, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[13].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 332, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[14].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 356, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[15].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 380, + "startColumn": 35 + } + } + } + ] + }, + { + "ruleId": "SARIF1008", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ContextRegionMustBeProperSupersetOfRegion", + "arguments": [ + "runs[0].results[16].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 404, + "startColumn": 35 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif new file mode 100644 index 000000000..067f8f50b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif similarity index 70% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif index a9c5f7e70..074d7bb59 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif @@ -8,17 +8,20 @@ "name": "SARIF Functional Testing", "rules": [ { - "id": "SARIF1017", - "name": "InvalidIndex", + "id": "SARIF1009", + "name": "IndexPropertiesMustBeConsistentWithArrays", "shortDescription": { - "text": "If an object contains a property that is used as an array index, then that array must be present and must contain at least \"index + 1\" elements." + "text": "If an object contains a property that is used as an array index (an \"index-valued property\"), then that array must be present and must contain at least \"index + 1\" elements." }, "fullDescription": { - "text": "If an object contains a property that is used as an array index, then that array must be present and must contain at least \"index + 1\" elements." + "text": "If an object contains a property that is used as an array index (an \"index-valued property\"), then that array must be present and must contain at least \"index + 1\" elements." }, "messageStrings": { - "Default": { - "text": "{0}: This \"{1}\" object contains a property \"{2}\" with value {3}, but either \"{4}\" is absent, or it has fewer than {5} elements." + "Error_TargetArrayMustExist": { + "text": "{0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' does not exist. An index-valued property always refers to an array, so the array must be present." + }, + "Error_TargetArrayMustBeLongEnough": { + "text": "{0}: This '{1}' object contains a property '{2}' with value {3}, but '{4}' has fewer than {5} elements. An index-valued properties must be valid for the array that it refers to." } }, "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" @@ -28,24 +31,50 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], "artifacts": [ { "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1017.InvalidIndex_Invalid.sarif", + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif", "uriBaseId": "TEST_DIR" } } ], "results": [ { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].results[0]", "result", @@ -62,7 +91,7 @@ "index": 0 }, "region": { - "startLine": 137, + "startLine": 138, "startColumn": 9 } } @@ -70,11 +99,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].results[0].locations[0].physicalLocation.address", "address", @@ -91,7 +120,7 @@ "index": 0 }, "region": { - "startLine": 150, + "startLine": 151, "startColumn": 28 } } @@ -99,11 +128,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].results[0].locations[0].physicalLocation.artifactLocation", "artifactLocation", @@ -120,7 +149,7 @@ "index": 0 }, "region": { - "startLine": 153, + "startLine": 154, "startColumn": 37 } } @@ -128,18 +157,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].locations[0].logicalLocations[0]", "logicalLocation", "index", "0", - "runs[0].logicalLocations", - "1" + "runs[0].logicalLocations" ] }, "locations": [ @@ -149,7 +177,7 @@ "index": 0 }, "region": { - "startLine": 158, + "startLine": 159, "startColumn": 17 } } @@ -157,18 +185,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].locations[0].logicalLocations[0]", "logicalLocation", "parentIndex", "1", - "runs[0].logicalLocations", - "2" + "runs[0].logicalLocations" ] }, "locations": [ @@ -178,7 +205,7 @@ "index": 0 }, "region": { - "startLine": 158, + "startLine": 159, "startColumn": 17 } } @@ -186,18 +213,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0]", "threadFlowLocation", "index", "0", - "runs[0].threadFlowLocations", - "1" + "runs[0].threadFlowLocations" ] }, "locations": [ @@ -207,7 +233,7 @@ "index": 0 }, "region": { - "startLine": 170, + "startLine": 171, "startColumn": 21 } } @@ -215,18 +241,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].webRequest", "webRequest", "index", "0", - "runs[0].webRequests", - "1" + "runs[0].webRequests" ] }, "locations": [ @@ -236,7 +261,7 @@ "index": 0 }, "region": { - "startLine": 177, + "startLine": 178, "startColumn": 37 } } @@ -244,18 +269,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].webResponse", "webResponse", "index", "0", - "runs[0].webResponses", - "1" + "runs[0].webResponses" ] }, "locations": [ @@ -265,7 +289,7 @@ "index": 0 }, "region": { - "startLine": 180, + "startLine": 181, "startColumn": 38 } } @@ -273,11 +297,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].results[0].provenance", "resultProvenance", @@ -294,7 +318,7 @@ "index": 0 }, "region": { - "startLine": 205, + "startLine": 206, "startColumn": 25 } } @@ -302,11 +326,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].results[0].graphTraversals[0]", "graphTraversal", @@ -323,7 +347,7 @@ "index": 0 }, "region": { - "startLine": 196, + "startLine": 197, "startColumn": 13 } } @@ -331,18 +355,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].graphTraversals[1]", "graphTraversal", "runGraphIndex", "0", - "runs[0].graphTraversals", - "1" + "runs[0].graphTraversals" ] }, "locations": [ @@ -352,7 +375,7 @@ "index": 0 }, "region": { - "startLine": 200, + "startLine": 201, "startColumn": 13 } } @@ -360,18 +383,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].webRequest", "webRequest", "index", "0", - "runs[0].webRequests", - "1" + "runs[0].webRequests" ] }, "locations": [ @@ -381,7 +403,7 @@ "index": 0 }, "region": { - "startLine": 213, + "startLine": 214, "startColumn": 25 } } @@ -389,18 +411,17 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustExist", "arguments": [ "runs[0].results[0].webResponse", "webResponse", "index", "0", - "runs[0].webResponses", - "1" + "runs[0].webResponses" ] }, "locations": [ @@ -410,7 +431,7 @@ "index": 0 }, "region": { - "startLine": 216, + "startLine": 217, "startColumn": 26 } } @@ -418,11 +439,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].addresses[0]", "address", @@ -439,7 +460,7 @@ "index": 0 }, "region": { - "startLine": 247, + "startLine": 248, "startColumn": 9 } } @@ -447,11 +468,11 @@ ] }, { - "ruleId": "SARIF1017", + "ruleId": "SARIF1009", "ruleIndex": 0, "level": "error", "message": { - "id": "Default", + "id": "Error_TargetArrayMustBeLongEnough", "arguments": [ "runs[0].artifacts[0]", "artifact", @@ -468,7 +489,7 @@ "index": 0 }, "region": { - "startLine": 129, + "startLine": 130, "startColumn": 9 } } diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif new file mode 100644 index 000000000..4e42941a8 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..76bb963de --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif @@ -0,0 +1,126 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1010", + "name": "RuleIdMustBeConsistent", + "shortDescription": { + "text": "Every result must contain at least one of the properties 'ruleId' and 'rule.id'." + }, + "fullDescription": { + "text": "Every result must contain at least one of the properties 'ruleId' and 'rule.id'. If both are present, they must be equal. See the SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643))." + }, + "messageStrings": { + "Error_ResultRuleIdMustBeConsistent": { + "text": "{0}: This result contains both the 'ruleId' property '{1}' and the 'rule.id' property '{2}', but they are not equal. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires that if both of these properties are present, they must be equal." + }, + "Error_ResultMustSpecifyRuleId": { + "text": "{0}: This result contains neither of the properties 'ruleId' or 'rule.id'. The SARIF specification ([§3.27.5](https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html#_Toc34317643)) requires at least one of these properties to be present." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1010.RuleIdMustBeConsistent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1010", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ResultMustSpecifyRuleId", + "arguments": [ + "runs[0].results[0]" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 13, + "startColumn": 9 + } + } + } + ] + }, + { + "ruleId": "SARIF1010", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_ResultRuleIdMustBeConsistent", + "arguments": [ + "runs[0].results[1]", + "TST0001", + "TST0002" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 18, + "startColumn": 9 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif new file mode 100644 index 000000000..96a32255d --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1010.RuleIdMustBeConsistent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif new file mode 100644 index 000000000..ab0c99d63 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif @@ -0,0 +1,98 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1011", + "name": "ReferenceFinalSchema", + "shortDescription": { + "text": "The '$schema' property must refer to the final version of the SARIF 2.1.0 schema." + }, + "fullDescription": { + "text": "The '$schema' property must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files.\r\n\r\nThe SARIF standard was developed over several years, and many intermediate versions of the schema were produced. Now that the standard is final, only the OASIS standard version of the schema is valid." + }, + "messageStrings": { + "Error_Default": { + "text": "{0}: The '$schema' property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the '$schema' property with a URL that refers to the final version of the schema." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1011.ReferenceFinalSchema_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1011", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_Default", + "arguments": [ + "$schema", + "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.4.json" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 2, + "startColumn": 88 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Valid.sarif new file mode 100644 index 000000000..de820b02e --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1011.ReferenceFinalSchema_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1011.ReferenceFinalSchema_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif deleted file mode 100644 index bdaba0c76..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif new file mode 100644 index 000000000..f3d57881b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif @@ -0,0 +1,130 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF1012", + "name": "MessageArgumentsMustBeConsistentWithRule", + "shortDescription": { + "text": "The properties of a result's 'message' property must be consistent with the properties of the rule that the result refers to." + }, + "fullDescription": { + "text": "The properties of a result's 'message' property must be consistent with the properties of the rule that the result refers to.\r\n\r\nWhen a result's 'message' object uses the 'id' and 'arguments' properties (which, by the way, is recommended: see SARIF2002.ProvideMessageArguments), it must ensure that the rule actually defines a message string with that id, and that 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain at least 4 elements." + }, + "messageStrings": { + "Error_MessageIdMustExist": { + "text": "{0}: This message object refers to the message with id '{1}' in rule '{2}', but that rule does not define a message with that id. When a tool creates a result message that uses the 'id' property, it must ensure that the specified rule actually has a message with that id." + }, + "Error_SupplyEnoughMessageArguments": { + "text": "{0}: The message with id '{1}' in rule '{2}' requires '{3}' arguments, but the 'arguments' array in this message object has only '{4}' element(s). When a tool creates a result message that use the 'id' and 'arguments' properties, it must ensure that the 'arguments' array has enough elements to provide values for every replacement sequence in the message specified by 'id'. For example, if the highest numbered replacement sequence in the specified message string is '{{3}}', then the 'arguments' array must contain 4 elements." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF1012", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_SupplyEnoughMessageArguments", + "arguments": [ + "runs[0].results[0]", + "DoesExist", + "TEST1001", + "2", + "1" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 26, + "startColumn": 9 + } + } + } + ] + }, + { + "ruleId": "SARIF1012", + "ruleIndex": 0, + "level": "error", + "message": { + "id": "Error_MessageIdMustExist", + "arguments": [ + "runs[0].results[1]", + "DoesNotExist", + "TEST1001" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 36, + "startColumn": 9 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif new file mode 100644 index 000000000..73c984219 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif deleted file mode 100644 index 6893145f0..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif +++ /dev/null @@ -1,151 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1012", - "name": "EndLineMustNotBeLessThanStartLine", - "shortDescription": { - "text": "The \"endLine\" property of a region object must not be less than the \"startLine\" property." - }, - "fullDescription": { - "text": "The \"endLine\" property of a region object must not be less than the \"startLine\" property." - }, - "messageStrings": { - "Default": { - "text": "{0}: The value of the \"endLine\" property is {1}, which is less than the value of the \"startLine\" property, which is {2}." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1012", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].locations[0].physicalLocation.region.endLine", - "1", - "2" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 26, - "startColumn": 30 - } - } - } - ] - }, - { - "ruleId": "SARIF1012", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.region.endLine", - "1", - "2" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 69, - "startColumn": 40 - } - } - } - ] - }, - { - "ruleId": "SARIF1012", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].stacks[0].frames[0].location.physicalLocation.region.endLine", - "1", - "2" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 45, - "startColumn": 36 - } - } - } - ] - }, - { - "ruleId": "SARIF1012", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].relatedLocations[0].physicalLocation.region.endLine", - "1", - "2" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 90, - "startColumn": 30 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif deleted file mode 100644 index 99aea32af..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif deleted file mode 100644 index c755231b8..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif +++ /dev/null @@ -1,522 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1014", - "name": "UriBaseIdRequiresRelativeUri", - "shortDescription": { - "text": "If a fileLocation object contains a \"uriBaseId\" property, the value of the \"uri\" property must be a relative URI reference." - }, - "fullDescription": { - "text": "If a fileLocation object contains a \"uriBaseId\" property, the value of the \"uri\" property must be a relative URI reference." - }, - "messageStrings": { - "Default": { - "text": "{0}: This fileLocation object contains a \"uriBaseId\" property, which means that the value of the \"uri\" property must be a relative URI reference, but \"{1}\" is an absolute URI reference." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].conversion.analysisToolLogFiles[0].uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 84, - "startColumn": 42 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].analysisTarget.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 115, - "startColumn": 42 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].attachments[0].artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 177, - "startColumn": 46 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].locations[0].physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 122, - "startColumn": 48 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 153, - "startColumn": 58 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].provenance.conversionSources[0].artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 186, - "startColumn": 48 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].stacks[0].frames[0].location.physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 135, - "startColumn": 54 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].relatedLocations[0].physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 168, - "startColumn": 48 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].fixes[0].artifactChanges[0].artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 197, - "startColumn": 50 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].artifacts[0].location.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 102, - "startColumn": 42 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].executableLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 55, - "startColumn": 42 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].responseFiles[0].uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 15, - "startColumn": 44 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].stdin.uri", - "file:///c:/log/in.txt" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 59, - "startColumn": 42 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].stdout.uri", - "file:///c:/log/out.txt" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 63, - "startColumn": 43 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].stderr.uri", - "file:///c:/log/err.txt" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 67, - "startColumn": 43 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].stdoutStderr.uri", - "file:///c:/log/out-err.txt" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 71, - "startColumn": 47 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].toolExecutionNotifications[0].locations[0].physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 25, - "startColumn": 52 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].invocations[0].toolConfigurationNotifications[0].locations[0].physicalLocation.artifactLocation.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 42, - "startColumn": 52 - } - } - } - ] - }, - { - "ruleId": "SARIF1014", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].versionControlProvenance[0].mappedTo.uri", - "file:///C:/src/file.c" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 94, - "startColumn": 42 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif deleted file mode 100644 index d810bac03..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Valid.sarif deleted file mode 100644 index e63d6b9f7..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1015.UriMustBeAbsolute_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1015.UriMustBeAbsolute_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif deleted file mode 100644 index 3c82780cc..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif +++ /dev/null @@ -1,71 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1016", - "name": "ContextRegionRequiresRegion", - "shortDescription": { - "text": "If the \"contextRegion\" property is present, the \"region\" property must also be present." - }, - "fullDescription": { - "text": "If the \"contextRegion\" property is present, the \"region\" property must also be present." - }, - "messageStrings": { - "Default": { - "text": "{0}: This \"physicalLocation\" object contains a \"contextRegion\" property, but it does not contain a \"region\" property." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1016.ContextRegionRequiresRegion_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1016", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "Default", - "arguments": [ - "runs[0].results[0].locations[0].physicalLocation" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 20, - "startColumn": 35 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif deleted file mode 100644 index e94a4d9d1..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1016.ContextRegionRequiresRegion_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif deleted file mode 100644 index 82e99fb1e..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif +++ /dev/null @@ -1,128 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1018", - "name": "InvalidUriInOriginalUriBaseIds", - "shortDescription": { - "text": "In the artifactLocation objects contained in run.originalUriBaseIds, if uriBaseId is absent, then uri must either be an absolute URI or it must be absent." - }, - "fullDescription": { - "text": "In the artifactLocation objects contained in run.originalUriBaseIds, if uriBaseId is absent, then uri must either be an absolute URI or it must be absent. Also, uri must end with a slash, so that it can safely be combined with the relative URIs in artifactLocation objects elsewhere in the log file." - }, - "messageStrings": { - "NotAbsolute": { - "text": "{0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds is not an absolute URI." - }, - "LacksTrailingSlash": { - "text": "{0}: The URI '{1}' belonging to the '{2}' element of run.originalUriBaseIds does not end with a slash." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1018", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "NotAbsolute", - "arguments": [ - "runs[0].originalUriBaseIds.PROJECT_ROOT", - "project", - "PROJECT_ROOT" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 12, - "startColumn": 25 - } - } - } - ] - }, - { - "ruleId": "SARIF1018", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "LacksTrailingSlash", - "arguments": [ - "runs[0].originalUriBaseIds.PROJECT_ROOT", - "project", - "PROJECT_ROOT" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 12, - "startColumn": 25 - } - } - } - ] - }, - { - "ruleId": "SARIF1018", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "LacksTrailingSlash", - "arguments": [ - "runs[0].originalUriBaseIds.SOURCE_ROOT", - "src", - "SOURCE_ROOT" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 18, - "startColumn": 24 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif deleted file mode 100644 index 56ae676bf..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif deleted file mode 100644 index 744aa1027..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif +++ /dev/null @@ -1,100 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1019", - "name": "RuleIdMustBePresentAndConsistent", - "shortDescription": { - "text": "In every result, at least one of the properties result.ruleId and result.rule.id must be present." - }, - "fullDescription": { - "text": "In every result, at least one of the properties result.ruleId and result.rule.id must be present. If both are present, they must be equal." - }, - "messageStrings": { - "InconsistentResultRuleId": { - "text": "{0}: The result contains both the ruleId property '{1}' and the rule.id property '{2}', and they are not equal." - }, - "MissingResultRuleId": { - "text": "{0}: The result contains neither result.ruleId nor result.rule.id." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1019", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "MissingResultRuleId", - "arguments": [ - "runs[0].results[0]" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 12, - "startColumn": 9 - } - } - } - ] - }, - { - "ruleId": "SARIF1019", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "InconsistentResultRuleId", - "arguments": [ - "runs[0].results[1]", - "TST0001", - "TST0002" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 17, - "startColumn": 9 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif deleted file mode 100644 index 8da8319d2..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Invalid.sarif deleted file mode 100644 index 2840dca02..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Invalid.sarif +++ /dev/null @@ -1,75 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing", - "rules": [ - { - "id": "SARIF1020", - "name": "ReferToFinalSchema", - "shortDescription": { - "text": "The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema." - }, - "fullDescription": { - "text": "The $schema property should be present, and must refer to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." - }, - "messageStrings": { - "ReferenceToOldSchemaVersion": { - "text": "{0}: The $schema property value '{1}' does not refer to the final version of the SARIF 2.1.0 schema. If you are using an earlier version of the SARIF format, consider upgrading your analysis tool to produce the final version. If this file does in fact conform to the final version of the schema, upgrade the tool to populate the $schema property with a URL that refers to the final version of the schema." - }, - "SchemaReferenceMissing": { - "text": "{0}: The SARIF log file does not contain a $schema property. Add a $schema property that refers to the final version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." - } - }, - "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" - } - ] - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1020.ReferToFinalSchema_Invalid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [ - { - "ruleId": "SARIF1020", - "ruleIndex": 0, - "level": "error", - "message": { - "id": "ReferenceToOldSchemaVersion", - "arguments": [ - "$schema", - "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.4.json" - ] - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "index": 0 - }, - "region": { - "startLine": 2, - "startColumn": 88 - } - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Valid.sarif deleted file mode 100644 index 4188f7e6c..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1020.ReferToFinalSchema_Valid.sarif +++ /dev/null @@ -1,28 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "SARIF Functional Testing" - } - }, - "invocations": [ - { - "executionSuccessful": true - } - ], - "artifacts": [ - { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1020.ReferToFinalSchema_Valid.sarif", - "uriBaseId": "TEST_DIR" - } - } - ], - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif new file mode 100644 index 000000000..12f41a1dc --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif @@ -0,0 +1,98 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2001", + "name": "TerminateMessagesWithPeriod", + "shortDescription": { + "text": "Express plain text result messages as complete sentences and end each sentence with a period." + }, + "fullDescription": { + "text": "Express plain text result messages as complete sentences and end each sentence with a period. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary.\r\n\r\nThis is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also `SARIF2014.ProvideDynamicMessageContent` and `SARIF2015.EnquoteDynamicMessageContent`." + }, + "messageStrings": { + "Warning_Default": { + "text": "{0}: In rule '{1}', the message with id '{2}' does not end in a period. Express plain text rule messages as complete sentences. This guidance does not apply to Markdown messages, which might include formatting that makes the punctuation unnecessary." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2001", + "ruleIndex": 0, + "message": { + "id": "Warning_Default", + "arguments": [ + "runs[0].tool.driver.rules[0].messageStrings.NoPeriod.text", + "TST0001", + "NoPeriod" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 18, + "startColumn": 111 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif new file mode 100644 index 000000000..bcc4c18ca --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2001.TerminateMessagesWithPeriod_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Invalid.sarif new file mode 100644 index 000000000..13b0daf19 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Invalid.sarif @@ -0,0 +1,88 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2002", + "name": "ProvideMessageArguments", + "shortDescription": { + "text": "In result messages, use the 'message.id' and 'message.arguments' properties rather than 'message.text'." + }, + "fullDescription": { + "text": "In result messages, use the 'message.id' and 'message.arguments' properties rather than 'message.text'. This has several advantages. If 'text' is lengthy, using 'id' and 'arguments' makes the SARIF file smaller. If the rule metadata is stored externally to the SARIF log file, the message text can be improved (for example, by adding more text, clarifying the phrasing, or fixing typos), and the result messages will pick up the improvements the next time it is displayed. Finally, SARIF supports localizing messages into different languages, which is possible if the SARIF file contains 'message.id' and 'message.arguments', but not if it contains 'message.text' directly." + }, + "messageStrings": { + "Warning_Default": { + "text": "{0}: The 'message' property of this result contains a 'text' property. Consider replacing it with 'id' and 'arguments' properties. This potentially reduces the log file size, allows the message text to be improved without modifying the log file, and enables localization." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2002.ProvideMessageArguments_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2002", + "ruleIndex": 0, + "message": { + "id": "Warning_Default", + "arguments": [ + "runs[0].results[0].message" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 15, + "startColumn": 22 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Valid.sarif new file mode 100644 index 000000000..15dc4ed37 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2002.ProvideMessageArguments_Valid.sarif @@ -0,0 +1,46 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2002.ProvideMessageArguments_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif new file mode 100644 index 000000000..0361642c2 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif @@ -0,0 +1,97 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2003", + "name": "ProvideVersionControlProvenance", + "shortDescription": { + "text": "Provide 'versionControlProvenance' to record which version of the code was analyzed, and to enable paths to be expressed relative to the root of the repository." + }, + "fullDescription": { + "text": "Provide 'versionControlProvenance' to record which version of the code was analyzed, and to enable paths to be expressed relative to the root of the repository." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: This run does not provide 'versionControlProvenance'. As a result, it is not possible to determine which version of code was analyzed, nor to map relative paths to their locations within the repository." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2003.ProvideVersionControlProvenance_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2003", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0]" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 5, + "startColumn": 5 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif new file mode 100644 index 000000000..f22a97e22 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2003.ProvideVersionControlProvenance_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Invalid.sarif new file mode 100644 index 000000000..178859edb --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Invalid.sarif @@ -0,0 +1,122 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2004", + "name": "OptimizeFileSize", + "shortDescription": { + "text": "Emit arrays only if they provide additional information." + }, + "fullDescription": { + "text": "Emit arrays only if they provide additional information.\r\n\r\nIn several parts of a SARIF log file, a subset of information about an object appears in one place, and the full information describing all such objects appears in an array elsewhere in the log file. For example, each 'result' object has a 'ruleId' property that identifies the rule that was violated. Elsewhere in the log file, the array 'run.tool.driver.rules' contains additional information about the rules. But if the elements of the 'rules' array contained no information about the rules beyond their ids, then there might be no reason to include the 'rules' array at all, and the log file could be made smaller simply by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information.\r\n\r\nSimilarly, most 'result' objects contain at least one 'artifactLocation' object. Elsewhere in the log file, the array 'run.artifacts' contains additional information about the artifacts that were analyzed. But if the elements of the 'artifacts' array contained not information about the artifacts beyond their locations, then there might be no reason to include the 'artifacts' array at all, and again the log file could be made smaller by omitting it. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information." + }, + "messageStrings": { + "Warning_EliminateLocationOnlyArtifacts": { + "text": "{0}: The 'artifacts' array contains no information beyond the locations of the artifacts. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'artifacts' array might be used to record the full set of artifacts that were analyzed. In such a scenario, the 'artifacts' array should be retained even if it contains only location information." + }, + "Warning_EliminateIdOnlyRules": { + "text": "{0}: The 'rules' array contains no information beyond the ids of the rules. Removing this array might reduce the log file size without losing information. In some scenarios (for example, when assessing compliance with policy), the 'rules' array might be used to record the full set of rules that were evaluated. In such a scenario, the 'rules' array should be retained even if it contains only id information." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2004.OptimizeFileSize_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2004", + "ruleIndex": 0, + "message": { + "id": "Warning_EliminateLocationOnlyArtifacts", + "arguments": [ + "runs[0].artifacts[0]" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 18, + "startColumn": 9 + } + } + } + ] + }, + { + "ruleId": "SARIF2004", + "ruleIndex": 0, + "message": { + "id": "Warning_EliminateIdOnlyRules", + "arguments": [ + "runs[0].tool.driver.rules[0]" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 11, + "startColumn": 13 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Valid.sarif new file mode 100644 index 000000000..eaf8ea4c4 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2004.OptimizeFileSize_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2004.OptimizeFileSize_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Invalid.sarif new file mode 100644 index 000000000..167d72840 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Invalid.sarif @@ -0,0 +1,130 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2005", + "name": "ProvideToolProperties", + "shortDescription": { + "text": "Provide information that makes it easy to identify the name and version of your tool." + }, + "fullDescription": { + "text": "Provide information that makes it easy to identify the name and version of your tool.\r\n\r\nThe tool's 'name' property should be no more than three words long. This makes it easy to remember and allows it to fit into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property.\r\n\r\nThe tool should provide either or both of the 'version' and 'semanticVersion' properties. This enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions.\r\n\r\nIf 'version' is used, facilitate comparison between versions by specifying a version number that starts with an integer, optionally followed by any desired characters." + }, + "messageStrings": { + "Warning_ProvideToolVersion": { + "text": "{0}: The tool '{1}' provides neither a 'version' property nor a 'semanticVersion' property. Providing a version enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions." + }, + "Warning_ProvideConciseToolName": { + "text": "{0}: The tool name '{1}' contains {2} words, which is more than the recommended maximum of {3} words. A short tool name is easy to remember and fits into a narrow column when displaying a list of results. If you need to provide more information about your tool, use the 'fullName' property." + }, + "Warning_UseNumericToolVersions": { + "text": "{0}: The tool '{1}' contains the 'version' property '{2}', which is not numeric. To facilitate comparison between versions, specify a 'version' that starts with an integer, optionally followed by any desired characters." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2005.ProvideToolProperties_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2005", + "ruleIndex": 0, + "message": { + "id": "Warning_ProvideConciseToolName", + "arguments": [ + "runs[0].tool.driver.name", + "SARIF Functional Testing Long", + "4", + "3" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 8, + "startColumn": 49 + } + } + } + ] + }, + { + "ruleId": "SARIF2005", + "ruleIndex": 0, + "message": { + "id": "Warning_UseNumericToolVersions", + "arguments": [ + "runs[0].tool.driver.version", + "SARIF Functional Testing Long", + "x" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 9, + "startColumn": 24 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Valid.sarif new file mode 100644 index 000000000..9bab411ba --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2005.ProvideToolProperties_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2005.ProvideToolProperties_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif similarity index 50% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif index 21dbb2c98..81b812019 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif @@ -8,17 +8,17 @@ "name": "SARIF Functional Testing", "rules": [ { - "id": "SARIF1013", - "name": "EndColumnMustNotBeLessThanStartColumn", + "id": "SARIF2006", + "name": "UrisShouldBeReachable", "shortDescription": { - "text": "The \"endColumn\" property of a region object must not be less than the \"startColumn\" property." + "text": "URIs that refer to locations such as rule help pages and result-related work items should be reachable via an HTTP GET request." }, "fullDescription": { - "text": "The \"endColumn\" property of a region object must not be less than the \"startColumn\" property." + "text": "URIs that refer to locations such as rule help pages and result-related work items should be reachable via an HTTP GET request." }, "messageStrings": { - "Default": { - "text": "{0}: The value of the \"endColumn\" property is {1}, which is less than the value of the \"startColumn\" property, which is {2}." + "Warning_Default": { + "text": "{0}: The URI '{1}' was not reachable via an HTTP GET request." } }, "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" @@ -28,28 +28,44 @@ }, "invocations": [ { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], "executionSuccessful": true } ], "artifacts": [ { "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif", + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2006.UrisShouldBeReachable_Invalid.sarif", "uriBaseId": "TEST_DIR" } } ], "results": [ { - "ruleId": "SARIF1013", + "ruleId": "SARIF2006", "ruleIndex": 0, - "level": "error", "message": { - "id": "Default", + "id": "Warning_Default", "arguments": [ - "runs[0].results[0].locations[0].physicalLocation.region.endColumn", - "1", - "2" + "runs[0].results[0].workItemUris[0]", + "https://example.com/my-project/issues/42" ] }, "locations": [ @@ -59,23 +75,21 @@ "index": 0 }, "region": { - "startLine": 27, - "startColumn": 32 + "startLine": 33, + "startColumn": 54 } } } ] }, { - "ruleId": "SARIF1013", + "ruleId": "SARIF2006", "ruleIndex": 0, - "level": "error", "message": { - "id": "Default", + "id": "Warning_Default", "arguments": [ - "runs[0].results[0].codeFlows[0].threadFlows[0].locations[0].location.physicalLocation.region.endColumn", - "1", - "2" + "runs[0].tool.driver.downloadUri", + "http://www.example.com/tools/codescanner/download.html" ] }, "locations": [ @@ -85,23 +99,21 @@ "index": 0 }, "region": { - "startLine": 72, - "startColumn": 42 + "startLine": 10, + "startColumn": 81 } } } ] }, { - "ruleId": "SARIF1013", + "ruleId": "SARIF2006", "ruleIndex": 0, - "level": "error", "message": { - "id": "Default", + "id": "Warning_Default", "arguments": [ - "runs[0].results[0].stacks[0].frames[0].location.physicalLocation.region.endColumn", - "1", - "2" + "runs[0].tool.driver.rules[0].helpUri", + "http://www.example.com/rules/tst0001.html" ] }, "locations": [ @@ -111,23 +123,21 @@ "index": 0 }, "region": { - "startLine": 47, - "startColumn": 38 + "startLine": 14, + "startColumn": 68 } } } ] }, { - "ruleId": "SARIF1013", + "ruleId": "SARIF2006", "ruleIndex": 0, - "level": "error", "message": { - "id": "Default", + "id": "Warning_Default", "arguments": [ - "runs[0].results[0].relatedLocations[0].physicalLocation.region.endColumn", - "1", - "2" + "runs[0].versionControlProvenance[0].repositoryUri", + "https://example.com/my-project" ] }, "locations": [ @@ -137,8 +147,8 @@ "index": 0 }, "region": { - "startLine": 94, - "startColumn": 32 + "startLine": 21, + "startColumn": 59 } } } diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Valid.sarif new file mode 100644 index 000000000..10b639277 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2006.UrisShouldBeReachable_Valid.sarif @@ -0,0 +1,46 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2006.UrisShouldBeReachable_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif new file mode 100644 index 000000000..d0c58db3f --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif @@ -0,0 +1,164 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2007", + "name": "ExpressPathsRelativeToRepoRoot", + "shortDescription": { + "text": "Provide information that makes it possible to determine the repo-relative locations of files that contain analysis results." + }, + "fullDescription": { + "text": "Provide information that makes it possible to determine the repo-relative locations of files that contain analysis results.\r\n\r\nEach element of the 'versionControlProvenance' array is a 'versionControlDetails' object that describes a repository containing files that were analyzed. 'versionControlDetails.mappedTo' defines the file system location to which the root of that repository is mapped. If 'mappedTo.uriBaseId' is present, and if result locations are expressed relative to that 'uriBaseId', then the repo-relative location of each result can be determined." + }, + "messageStrings": { + "Warning_ExpressResultLocationsRelativeToMappedTo": { + "text": "{0}: This result location does not provide any of the 'uriBaseId' values that specify repository locations: '{1}'. As a result, it will not be possible to determine the location of the file containing this result relative to the root of the repository that contains it." + }, + "Warning_ProvideUriBaseIdForMappedTo": { + "text": "{0}: The 'versionControlDetails' object that describes the repository '{1}' does not provide 'mappedTo.uriBaseId'. As a result, it will not be possible to determine the repo-relative location of files containing analysis results for this repository." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2007", + "ruleIndex": 0, + "message": { + "id": "Warning_ProvideUriBaseIdForMappedTo", + "arguments": [ + "runs[0].versionControlProvenance[1]", + "https://example.com/microsoft/repo2/" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 33, + "startColumn": 9 + } + } + } + ] + }, + { + "ruleId": "SARIF2007", + "ruleIndex": 0, + "message": { + "id": "Warning_ProvideUriBaseIdForMappedTo", + "arguments": [ + "runs[0].versionControlProvenance[2]", + "https://example.com/microsoft/repo3/" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 39, + "startColumn": 9 + } + } + } + ] + }, + { + "ruleId": "SARIF2007", + "ruleIndex": 0, + "message": { + "id": "Warning_ExpressResultLocationsRelativeToMappedTo", + "arguments": [ + "runs[0].results[0].locations[1].physicalLocation.artifactLocation", + "PROJECT_ROOT, OTHER_ROOT" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 83, + "startColumn": 37 + } + } + } + ] + }, + { + "ruleId": "SARIF2007", + "ruleIndex": 0, + "message": { + "id": "Warning_ExpressResultLocationsRelativeToMappedTo", + "arguments": [ + "runs[0].results[0].locations[2].physicalLocation.artifactLocation", + "PROJECT_ROOT, OTHER_ROOT" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 91, + "startColumn": 37 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif new file mode 100644 index 000000000..883824799 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif @@ -0,0 +1,46 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif new file mode 100644 index 000000000..160930102 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif @@ -0,0 +1,46 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Invalid.sarif new file mode 100644 index 000000000..40861b074 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Invalid.sarif @@ -0,0 +1,96 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2008", + "name": "ProvideSchema", + "shortDescription": { + "text": "A SARIF log file should contain, on the root object, a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema." + }, + "fullDescription": { + "text": "A SARIF log file should contain, on the root object, a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." + }, + "messageStrings": { + "Warning_Default": { + "text": "{0}: The SARIF log file does not contain a '$schema' property. Add a '$schema' property that refers to the final, OASIS standard version of the SARIF 2.1.0 schema. This enables IDEs to provide Intellisense for SARIF log files." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2008.ProvideSchema_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2008", + "ruleIndex": 0, + "message": { + "id": "Warning_Default", + "arguments": [ + "" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 1, + "startColumn": 1 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Valid.sarif new file mode 100644 index 000000000..2f63a3c23 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2008.ProvideSchema_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2008.ProvideSchema_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif new file mode 100644 index 000000000..138d825bb --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif @@ -0,0 +1,98 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2009", + "name": "ConsiderConventionalIdentifierValues", + "shortDescription": { + "text": "Adopt uniform naming conventions for rule ids." + }, + "fullDescription": { + "text": "Adopt uniform naming conventions for rule ids.\r\n\r\nMany tools follow a conventional format for the 'reportingDescriptor.id' property: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001' for a diagnostic from the Roslyn C# compiler. For uniformity of experience across tools, we recommend this format." + }, + "messageStrings": { + "Note_UseConventionalRuleIds": { + "text": "{0}: The 'id' property of the rule '{1}' does not follow the recommended format: a short string identifying the tool concatenated with a numeric rule number, for example, 'CS2001'. Using a conventional format for the rule id provides a more uniform experience across tools." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2009", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_UseConventionalRuleIds", + "arguments": [ + "runs[0].tool.driver.rules[0].id", + "WRONG00001" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 13, + "startColumn": 32 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif new file mode 100644 index 000000000..63047de07 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif new file mode 100644 index 000000000..beec5b767 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif @@ -0,0 +1,201 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2010", + "name": "ProvideCodeSnippets", + "shortDescription": { + "text": "Provide code snippets to enable users to see the code that triggered each result, even if they are not enlisted in the code." + }, + "fullDescription": { + "text": "Provide code snippets to enable users to see the code that triggered each result, even if they are not enlisted in the code." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: The 'region' object in this result location does not provide a 'snippet' property. Providing a code snippet enables users to see the code that triggered the result, even if they are not enlisted in the code." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2011' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2010.ProvideCodeSnippets_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2010", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation.region" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 30, + "startColumn": 27 + } + } + } + ] + }, + { + "ruleId": "SARIF2010", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation.contextRegion" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 33, + "startColumn": 34 + } + } + } + ] + }, + { + "ruleId": "SARIF2010", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[1].locations[0].physicalLocation.contextRegion" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 59, + "startColumn": 34 + } + } + } + ] + }, + { + "ruleId": "SARIF2010", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[2].locations[0].physicalLocation.region" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 79, + "startColumn": 27 + } + } + } + ] + }, + { + "ruleId": "SARIF2010", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[3].locations[0].physicalLocation.region" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 105, + "startColumn": 27 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Valid.sarif new file mode 100644 index 000000000..687cb20cc --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2010.ProvideCodeSnippets_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2010.ProvideCodeSnippets_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Invalid.sarif new file mode 100644 index 000000000..059499e1a --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Invalid.sarif @@ -0,0 +1,97 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2011", + "name": "ProvideContextRegion", + "shortDescription": { + "text": "Provide context regions to enable users to see a portion of the code that surrounds each result, even if they are not enlisted in the code." + }, + "fullDescription": { + "text": "Provide context regions to enable users to see a portion of the code that surrounds each result, even if they are not enlisted in the code." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: This result location does not provide a 'contextRegion' property. Providing a context region enables users to see a portion of the code that surrounds the result, even if they are not enlisted in the code." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2011.ProvideContextRegion_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2011", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].results[0].locations[0].physicalLocation" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 26, + "startColumn": 35 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Valid.sarif new file mode 100644 index 000000000..33ddbddc1 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2011.ProvideContextRegion_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2011.ProvideContextRegion_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Invalid.sarif new file mode 100644 index 000000000..1f9a0e8b0 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Invalid.sarif @@ -0,0 +1,98 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2012", + "name": "ProvideHelpUris", + "shortDescription": { + "text": "For each rule, provide a URI where users can find detailed information about the rule." + }, + "fullDescription": { + "text": "For each rule, provide a URI where users can find detailed information about the rule. This information should include a detailed description of the invalid pattern, an explanation of why the pattern is poor practice (particularly in contexts such as security or accessibility where driving considerations might not be readily apparent), guidance for resolving the problem (including describing circumstances in which ignoring the problem altogether might be appropriate), examples of invalid and valid patterns, and special considerations (such as noting when a violation should never be ignored or suppressed, noting when a violation could cause downstream tool noise, and noting when a rule can be configured in some way to refine or alter the analysis)." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: The rule '{1}' does not provide a help URI. Providing a URI where users can find detailed information about the rule helps users to understand the result and how they can best address it." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2012.ProvideHelpUris_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2012", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].tool.driver.rules[0]", + "SARIF2009.ConsiderConventionalIdentifierValues" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 11, + "startColumn": 13 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Valid.sarif new file mode 100644 index 000000000..e5384b399 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2012.ProvideHelpUris_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2012.ProvideHelpUris_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif new file mode 100644 index 000000000..6eff865bc --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif @@ -0,0 +1,97 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2013", + "name": "ProvideEmbeddedFileContent", + "shortDescription": { + "text": "Provide embedded file content so that users can examine results in their full context without having to enlist in the source repository." + }, + "fullDescription": { + "text": "Provide embedded file content so that users can examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: This run does not provide embedded file content. Providing embedded file content enables users to examine results in their full context without having to enlist in the source repository. Embedding file content in a SARIF log file can dramatically increase its size, so consider the usage scenario when you decide whether to provide it." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2013", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0]" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 5, + "startColumn": 5 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif new file mode 100644 index 000000000..acc8b45e7 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2013.ProvideEmbeddedFileContent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif new file mode 100644 index 000000000..bfbc7727b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif @@ -0,0 +1,100 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2014", + "name": "ProvideDynamicMessageContent", + "shortDescription": { + "text": "Include \"dynamic content\" (information that varies among results from the same rule) to makes your messages more specific, and to avoid the \"wall of bugs\" phenomenon, where hundreds of occurrences of the same message appear unapproachable." + }, + "fullDescription": { + "text": "Include \"dynamic content\" (information that varies among results from the same rule) to makes your messages more specific, and to avoid the \"wall of bugs\" phenomenon, where hundreds of occurrences of the same message appear unapproachable.\r\n\r\nThis is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2015.EnquoteDynamicMessageContent'." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: In rule '{1}', the message with id '{2}' does not include any dynamic content. Dynamic content makes your messages more specific and avoids the \"wall of bugs\" phenomenon, where hundreds of occurrences of the same message appear unapproachable." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2014.ProvideDynamicMessageContent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2014", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].tool.driver.rules[0].messageStrings.NoPlaceholders.text", + "TST0001", + "text", + "NoPlaceholders" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 18, + "startColumn": 74 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif new file mode 100644 index 000000000..80b5aca7c --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2014.ProvideDynamicMessageContent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif new file mode 100644 index 000000000..d6c0e64f9 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif @@ -0,0 +1,99 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "rules": [ + { + "id": "SARIF2015", + "name": "EnquoteDynamicMessageContent", + "shortDescription": { + "text": "Place dynamic content in single quotes to set it off from the static text and to make it easier to spot." + }, + "fullDescription": { + "text": "Place dynamic content in single quotes to set it off from the static text and to make it easier to spot. It's especially helpful when the dynamic content is a string that might contain spaces, and most especially when the string might be empty (and so would be invisible if it weren't for the quotes). We recommend single quotes for a less cluttered appearance, even though US English usage would require double quotes.\r\n\r\nThis is part of a set of authoring practices that make your rule messages more readable, understandable, and actionable. See also 'SARIF2001.TerminateMessagesWithPeriod' and 'SARIF2014.ProvideDynamicMessageContent'." + }, + "messageStrings": { + "Note_Default": { + "text": "{0}: In rule '{1}', the message with id '{2}' includes dynamic content that is not enclosed in single quotes. Enquoting dynamic content makes it easier to spot, and single quotes give a less cluttered appearance." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "SARIF2015", + "ruleIndex": 0, + "level": "note", + "message": { + "id": "Note_Default", + "arguments": [ + "runs[0].tool.driver.rules[0].messageStrings.NotEnquoted.text", + "TST0001", + "NotEnquoted" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 18, + "startColumn": 91 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif new file mode 100644 index 000000000..89d2f964e --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif @@ -0,0 +1,54 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing" + } + }, + "invocations": [ + { + "toolConfigurationNotifications": [ + { + "message": { + "text": "Rule 'SARIF2002' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2006' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + }, + { + "message": { + "text": "Rule 'SARIF2007' was explicitly disabled by the user. As result, this tool run cannot be used for compliance or other auditing processes that require a comprehensive analysis." + }, + "descriptor": { + "id": "WRN999.RuleExplicitlyDisabled" + } + } + ], + "executionSuccessful": true + } + ], + "artifacts": [ + { + "location": { + "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF2015.EnquoteDynamicMessageContent_Valid.sarif", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif similarity index 94% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif index a0cff6461..939a50b79 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Invalid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "rules": [ { "id": "RULE0001", diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif similarity index 94% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif index 9d7ca95d9..92d0dad79 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.DoNotUseFriendlyNameAsRuleId_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1001.RuleIdentifiersMustBeValid_Valid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "rules": [ { "id": "RULE0001", diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Invalid.sarif similarity index 79% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Invalid.sarif index 2199099e8..df8ff5344 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Invalid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "downloadUri": "ht%tp://www.example.com/tools/codescanner/download.html", "notifications": [ { @@ -56,6 +57,19 @@ "workItemUris": [ "ht&tp://example.com/my-project/issues/42" ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result: file uri has dot dot segment." + }, + "analysisTarget": { + "uri": "file:///c:/src/src2/src3/../../file.c" + }, + "workItemUris": [ + "http://example.com/my-project/issues/42" + ] } ], "columnKind": "utf16CodeUnits" diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Valid.sarif similarity index 98% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Valid.sarif index 0c14791be..301df35e2 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1003.UrisMustBeValid_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1002.UrisMustBeValid_Valid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "downloadUri": "http://www.example.com/tools/codescanner/download.html", "rules": [ { diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif similarity index 79% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif index bdd3c9425..7bcf29a61 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Invalid.sarif @@ -5,7 +5,48 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" + } + }, + "originalUriBaseIds": { + "REPO_ROOT": { + "uri": "repository", + "description": { + "text": "This artifactLocation has no uriBaseId, so its uri, if present, must be absolute. But it isn't. It also doesn't end with a slash." + } + }, + "SOURCE_ROOT": { + "uri": "src", + "uriBaseId": "REPO_ROOT", + "description": { + "text": "This is a regression test for Bug #1862, where we were not checking for a trailing slash on relative URIs." + } + }, + "BAD_URI_CONTAINING_DOTDOT": { + "uri": "file:///C:/rules/dir1/dir2/../", + "description": { + "text": "Uri contains dot dot segment." + } + }, + "BAD_URI_CONTAINING_QUERY": { + "uri": "http://www.contoso.com/catalog/shownew.htm?date=today", + "description": { + "text": "Uri contains a query." + } + }, + "BAD_URI_CONTAINING_FRAGMENT": { + "uri": "http://www.contoso.com/index.htm#search", + "description": { + "text": "URI contains a fragment." + } + }, + "BAD_URI_HAS_BASEID_BUT_ALSO_ABSOLUTE_URI": { + "uri": "file:///C:/rules/", + "uriBaseId": "SOURCE_ROOT", + "description": { + "text": "URI contains a fragment." + } } }, "invocations": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif similarity index 88% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif index d989aceeb..c0a6d2336 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1014.UriBaseIdRequiresRelativeUri_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1004.ExpressUriBaseIdsCorrectly_Valid.sarif @@ -5,7 +5,28 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" + } + }, + "originalUriBaseIds": { + "REPO_ROOT": { + "description": { + "text": "This artifactLocation has neither a uri nor a uriBaseId. This is fine." + } + }, + "TEST_ROOT": { + "uri": "file:///C:/tests/", + "description": { + "text": "This artifactLocation has no uriBaseId, so its uri, if present, must be absolute." + } + }, + "SOURCE_ROOT": { + "uri": "src/", + "uriBaseId": "REPO_ROOT", + "description": { + "text": "This artifactLocation has a uriBaseId so its uri must be relative." + } } }, "invocations": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif similarity index 97% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif index ecbc57b73..9f266e89a 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Invalid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "downloadUri": "tools/codescanner/download.html", "notifications": [ { diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Valid.sarif similarity index 97% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Valid.sarif index d19ec690e..c833fbdb2 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1015.UriMustBeAbsolute_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1005.UriMustBeAbsolute_Valid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "CodeScanner", + "version": "1.0", "downloadUri": "http://www.example.com/tools/codescanner/download.html", "rules": [ { diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif similarity index 89% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif index 16271fe34..8a3fe411f 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Invalid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" } }, "invocations": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif similarity index 94% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif index 6b94b42a4..ad85553c5 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.EndTimeMustNotBeBeforeStartTime_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1006.InvocationPropertiesMustBeConsistent_Valid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" } }, "invocations": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..a7b79d681 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Invalid.sarif @@ -0,0 +1,209 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "results": [ + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "[No message provided]." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "startColumn": 2, + "endColumn": 1 + } + }, + "message": { + "text": "endColumn less than startColumn in result location." + } + } + ], + "stacks": [ + { + "frames": [ + { + "location": { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "startColumn": 2, + "endColumn": 1 + } + }, + "message": { + "text": "endColumn less than startColumn in stackFrame." + } + } + } + ] + } + ], + "codeFlows": [ + { + "threadFlows": [ + { + "locations": [ + { + "location": { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "startColumn": 2, + "endColumn": 1 + } + }, + "message": { + "text": "endColumn less than startColumn in threadFlow location." + } + } + } + ] + } + ] + } + ], + "relatedLocations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "startColumn": 2, + "endColumn": 1 + } + }, + "message": { + "text": "endColumn less than startColumn in related location." + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Result message." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "endLine": 1 + } + }, + "message": { + "text": "endLine less than startLine in result location." + } + } + ], + "stacks": [ + { + "frames": [ + { + "location": { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "endLine": 1 + } + }, + "message": { + "text": "endLine less than startLine in stackFrame." + } + } + } + ] + } + ], + "codeFlows": [ + { + "threadFlows": [ + { + "locations": [ + { + "location": { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "endLine": 1 + } + }, + "message": { + "text": "endLine less than startLine in threadFlow location." + } + } + } + ] + } + ] + } + ], + "relatedLocations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 2, + "endLine": 1 + } + }, + "message": { + "text": "endLine less than startLine in related location." + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "endLine": 1, + "charLength": 9 + } + }, + "message": { + "text": "region start properties are absent." + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif similarity index 61% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif index 1f490b966..c3ab50129 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1007.RegionPropertiesMustBeConsistent_Valid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" } }, "results": [ @@ -77,6 +78,56 @@ } } ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Result message." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 1, + "endLine": 1 + } + }, + "message": { + "text": "endLine equals startLine." + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 1, + "endLine": 2 + } + }, + "message": { + "text": "endLine greater than startLine." + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/src/file.c" + }, + "region": { + "startLine": 1 + } + }, + "message": { + "text": "endLine not specified." + } + } + ] } ], "columnKind": "utf16CodeUnits" diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif deleted file mode 100644 index 62eef513b..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Invalid.sarif +++ /dev/null @@ -1,182 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "CodeScanner", - "rules": [ - { - "id": "TST0001", - "shortDescription": { - "text": "Short rule description without period", - "markdown": "_Rich_ short rule description without period" - }, - "fullDescription": { - "text": "Short full rule description without period", - "markdown": "_Rich_ full rule description without period" - }, - "messageStrings": { - "Default": { - "text": "rule message without period", - "markdown": "_Rich_ rule message without period" - } - } - } - ] - } - }, - "invocations": [ - { - "toolExecutionNotifications": [ - { - "message": { - "text": "toolNotification message without period", - "markdown": "_Rich_ toolNotification message without period" - } - } - ], - "toolConfigurationNotifications": [ - { - "message": { - "text": "configurationNotification message without period", - "markdown": "_Rich_ configuratioNotification message without period" - } - } - ], - "executionSuccessful": true - } - ], - "results": [ - { - "ruleId": "TST0001", - "ruleIndex": 0, - "level": "error", - "message": { - "text": "result message without period", - "markdown": "_Rich_ result message without period" - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 1, - "message": { - "text": "region message without period", - "markdown": "_Rich_ region message without period" - } - } - }, - "message": { - "text": "location message without period", - "markdown": "_Rich_ location message without period" - } - } - ], - "stacks": [ - { - "message": { - "text": "stack message without period", - "markdown": "_Rich_ stack message without period" - }, - "frames": [ - { - "location": { - "message": { - "text": "frame message without period", - "markdown": "_Rich_ frame message without period" - } - } - } - ] - } - ], - "codeFlows": [ - { - "message": { - "text": "codeFlow message without period", - "markdown": "_Rich_ codeFlow message without period" - }, - "threadFlows": [ - { - "message": { - "text": "threadFlow message without period", - "markdown": "_Rich_ threadFlow message without period" - }, - "locations": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 42, - "startColumn": 12 - } - }, - "message": { - "text": "ThreadFlow location message without period", - "markdown": "_Rich_ ThreadFlow location message without period" - } - } - } - ] - } - ] - } - ], - "attachments": [ - { - "description": { - "text": "attachment description without period", - "markdown": "_Rich_ attachment description without period" - }, - "artifactLocation": { - "uri": "file:///c:/output/screenshot.png" - }, - "rectangles": [ - { - "left": 100.0, - "right": 100.0, - "message": { - "text": "rectangle message without period", - "markdown": "_Rich_ rectangle message without period" - } - } - ] - } - ], - "fixes": [ - { - "description": { - "text": "fix description without period", - "markdown": "_Rich_ fix description without period" - }, - "artifactChanges": [ - { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "replacements": [ - { - "deletedRegion": { - "byteOffset": 0, - "byteLength": 1 - } - } - ] - } - ] - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif deleted file mode 100644 index 0a446b6e2..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.MessagesShouldEndWithPeriod_Valid.sarif +++ /dev/null @@ -1,182 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "CodeScanner", - "rules": [ - { - "id": "TST0001", - "shortDescription": { - "text": "Short rule description with period.", - "markdown": "_Rich_ short rule description with period." - }, - "fullDescription": { - "text": "Short full rule description with period.", - "markdown": "_Rich_ full rule description with period." - }, - "messageStrings": { - "Default": { - "text": "rule message with period.", - "markdown": "_Rich_ rule message with period." - } - } - } - ] - } - }, - "invocations": [ - { - "toolExecutionNotifications": [ - { - "message": { - "text": "toolNotification message with period.", - "markdown": "_Rich_ toolNotification message with period." - } - } - ], - "toolConfigurationNotifications": [ - { - "message": { - "text": "configurationNotification message with period.", - "markdown": "_Rich_ configuratioNotification message with period." - } - } - ], - "executionSuccessful": true - } - ], - "results": [ - { - "ruleId": "TST0001", - "ruleIndex": 0, - "level": "error", - "message": { - "text": "result message with period.", - "markdown": "_Rich_ result message with period." - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 1, - "message": { - "text": "region message with period.", - "markdown": "_Rich_ region message with period." - } - } - }, - "message": { - "text": "location message with period.", - "markdown": "_Rich_ location message with period." - } - } - ], - "stacks": [ - { - "message": { - "text": "stack message with period.", - "markdown": "_Rich_ stack message with period." - }, - "frames": [ - { - "location": { - "message": { - "text": "frame message with period.", - "markdown": "_Rich_ frame message with period." - } - } - } - ] - } - ], - "codeFlows": [ - { - "message": { - "text": "codeFlow message with period.", - "markdown": "_Rich_ codeFlow message with period." - }, - "threadFlows": [ - { - "message": { - "text": "threadFlow message with period.", - "markdown": "_Rich_ threadFlow message with period." - }, - "locations": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 42, - "startColumn": 12 - } - }, - "message": { - "text": "ThreadFlow location message with period.", - "markdown": "_Rich_ ThreadFlow location message with period." - } - } - } - ] - } - ] - } - ], - "attachments": [ - { - "description": { - "text": "attachment description with period.", - "markdown": "_Rich_ attachment description with period." - }, - "artifactLocation": { - "uri": "file:///c:/output/screenshot.png" - }, - "rectangles": [ - { - "left": 100.0, - "right": 100.0, - "message": { - "text": "rectangle message with period.", - "markdown": "_Rich_ rectangle message with period." - } - } - ] - } - ], - "fixes": [ - { - "description": { - "text": "fix description with period.", - "markdown": "_Rich_ fix description with period." - }, - "artifactChanges": [ - { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "replacements": [ - { - "deletedRegion": { - "byteOffset": 0, - "byteLength": 1 - } - } - ] - } - ] - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif new file mode 100644 index 000000000..9eb86c41d --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Invalid.sarif @@ -0,0 +1,423 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "results": [ + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - context region is present without region." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "contextRegion": { + "startLine": 2, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - partially overlapping region and contextregion (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 1, + "startColumn": 5, + "endLine": 10, + "endColumn": 15 + }, + "contextRegion": { + "startLine": 1, + "startColumn": 1, + "endLine": 10, + "endColumn": 5 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - non-overlapping region and contextregion (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 5, + "endLine": 10 + }, + "contextRegion": { + "startLine": 2, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - identical region and contextregion (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 2, + "endLine": 4 + }, + "contextRegion": { + "startLine": 2, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - contextregion smaller than region (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 2, + "endLine": 5 + }, + "contextRegion": { + "startLine": 3, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - default populated endLine (same as startLine) (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 5 + }, + "contextRegion": { + "startLine": 5, + "endLine": 5 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - default populated endColumn (int.maxvalue) (text and line/column based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 2, + "startColumn": 5 + }, + "contextRegion": { + "startLine": 2, + "startColumn": 5, + "endColumn": 8 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - partially overlapping region and contextregion (text and offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "charOffset": 5, + "charLength": 25 + }, + "contextRegion": { + "charOffset": 10, + "charLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - non-overlapping region and contextregion (text and offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "charOffset": 10, + "charLength": 25 + }, + "contextRegion": { + "charOffset": 37, + "charLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - identical region and contextregion (text and offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "charOffset": 10, + "charLength": 25 + }, + "contextRegion": { + "charOffset": 10, + "charLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - contextregion smaller than region (text and offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "charOffset": 5, + "charLength": 20 + }, + "contextRegion": { + "charOffset": 10, + "charLength": 5 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - default populated charLength (0) (text and offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "charOffset": 5, + "charLength": 20 + }, + "contextRegion": { + "charOffset": 5 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - partially overlapping region and contextregion (binary offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "byteOffset": 5, + "byteLength": 25 + }, + "contextRegion": { + "byteOffset": 10, + "byteLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - non-overlapping region and contextregion (binary offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "byteOffset": 10, + "byteLength": 25 + }, + "contextRegion": { + "byteOffset": 35, + "byteLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - identical region and contextregion (binary offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "byteOffset": 10, + "byteLength": 25 + }, + "contextRegion": { + "byteOffset": 10, + "byteLength": 25 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - contextregion smaller than region (binary offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "byteOffset": 10, + "byteLength": 25 + }, + "contextRegion": { + "byteOffset": 15, + "byteLength": 5 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Bad result - default populated byteLength (0) (binary offset based)." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "byteOffset": 5, + "byteLength": 20 + }, + "contextRegion": { + "byteOffset": 5 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif similarity index 93% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif index 31117c2aa..f9f7fefef 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1008.PhysicalLocationPropertiesMustBeConsistent_Valid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" } }, "results": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif similarity index 99% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif index dbac3ddfb..e462f3410 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Invalid.sarif @@ -6,6 +6,7 @@ "tool": { "driver": { "name": "SARIF Functional Testing", + "version": "1.0", "notifications": [ { "id": "CNOT0001" diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif similarity index 90% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif index 3ad60c2eb..ae79d29cd 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1017.InvalidIndex_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1009.IndexPropertiesMustBeConsistentWithArrays_Valid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "Sarif Functional Testing" + "name": "Sarif Functional Testing", + "version": "1.0" } }, "logicalLocations": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif similarity index 89% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif index c50700ea7..931d33d56 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Invalid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "Sarif Functional Testing" + "name": "Sarif Functional Testing", + "version": "1.0" } }, "results": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif similarity index 92% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif index 57b98035b..ac78f46a6 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1019.RuleIdMustBePresentAndConsistent_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1010.RuleIdMustBeConsistent_Valid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "Sarif Functional Testing" + "name": "Sarif Functional Testing", + "version": "1.0" } }, "results": [ diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif similarity index 77% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif index 212c74e5e..68fa7f1b0 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Invalid.sarif @@ -5,7 +5,8 @@ { "tool": { "driver": { - "name": "Sarif Functional Testing" + "name": "Sarif Functional Testing", + "version": "1.0" } }, "results": [], diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Valid.sarif new file mode 100644 index 000000000..08e831d4b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1011.ReferenceFinalSchema_Valid.sarif @@ -0,0 +1,16 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Sarif Functional Testing", + "version": "1.0" + } + }, + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif deleted file mode 100644 index b738f5d6a..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.EndLineMustNotBeLessThanStart_Invalid.sarif +++ /dev/null @@ -1,107 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "CodeScanner" - } - }, - "results": [ - { - "ruleId": "TST0001", - "level": "error", - "message": { - "text": "[No message provided]." - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "startColumn": 2, - "endColumn": 1 - } - }, - "message": { - "text": "endColumn less than startColumn in result location." - } - } - ], - "stacks": [ - { - "frames": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "startColumn": 2, - "endColumn": 1 - } - }, - "message": { - "text": "endColumn less than startColumn in stackFrame." - } - } - } - ] - } - ], - "codeFlows": [ - { - "threadFlows": [ - { - "locations": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "startColumn": 2, - "endColumn": 1 - } - }, - "message": { - "text": "endColumn less than startColumn in threadFlow location." - } - } - } - ] - } - ] - } - ], - "relatedLocations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "startColumn": 2, - "endColumn": 1 - } - }, - "message": { - "text": "endColumn less than startColumn in related location." - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif new file mode 100644 index 000000000..82c78fc4b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Invalid.sarif @@ -0,0 +1,50 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test 1001 full description." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "results": [ + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID" + ] + } + }, + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "message": { + "id": "DoesNotExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID" + ] + } + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif new file mode 100644 index 000000000..1e5cefa32 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1012.MessageArgumentsMustBeConsistentWithRule_Valid.sarif @@ -0,0 +1,41 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test 1001 full description." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "results": [ + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID", + "SRCINVALID" + ] + } + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif deleted file mode 100644 index aded91e74..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Invalid.sarif +++ /dev/null @@ -1,103 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "CodeScanner" - } - }, - "results": [ - { - "ruleId": "TST0001", - "level": "error", - "message": { - "text": "Result message." - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "endLine": 1 - } - }, - "message": { - "text": "endLine less than startLine in result location." - } - } - ], - "stacks": [ - { - "frames": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "endLine": 1 - } - }, - "message": { - "text": "endLine less than startLine in stackFrame." - } - } - } - ] - } - ], - "codeFlows": [ - { - "threadFlows": [ - { - "locations": [ - { - "location": { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "endLine": 1 - } - }, - "message": { - "text": "endLine less than startLine in threadFlow location." - } - } - } - ] - } - ] - } - ], - "relatedLocations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 2, - "endLine": 1 - } - }, - "message": { - "text": "endLine less than startLine in related location." - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif deleted file mode 100644 index f007b3de4..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1013.EndColumnMustNotBeLessThanStart_Valid.sarif +++ /dev/null @@ -1,66 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "CodeScanner" - } - }, - "results": [ - { - "ruleId": "TST0001", - "level": "error", - "message": { - "text": "Result message." - }, - "locations": [ - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 1, - "endLine": 1 - } - }, - "message": { - "text": "endLine equals startLine." - } - }, - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 1, - "endLine": 2 - } - }, - "message": { - "text": "endLine greater than startLine." - } - }, - { - "physicalLocation": { - "artifactLocation": { - "uri": "file:///c:/src/file.c" - }, - "region": { - "startLine": 1 - } - }, - "message": { - "text": "endLine not specified." - } - } - ] - } - ], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif deleted file mode 100644 index 689a2661b..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Invalid.sarif +++ /dev/null @@ -1,30 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "Sarif Functional Testing" - } - }, - "originalUriBaseIds": { - "PROJECT_ROOT": { - "uri": "project", - "description": { - "text": "This artifactLocation has no uriBaseId, so its uri, if present, must be absolute. But it isn't. It also doesn't end with a slash." - } - }, - "SOURCE_ROOT": { - "uri": "src", - "uriBaseId": "PROJECT_ROOT", - "description": { - "text": "This is a regression test for Bug #1862, where we were not checking for a trailing slash on relative URIs." - } - } - }, - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif deleted file mode 100644 index d2f4aa6e2..000000000 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1018.InvalidUriInOriginalUriBaseIds_Valid.sarif +++ /dev/null @@ -1,35 +0,0 @@ -{ - "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", - "version": "2.1.0", - "runs": [ - { - "tool": { - "driver": { - "name": "Sarif Functional Testing" - } - }, - "originalUriBaseIds": { - "PROJECT_ROOT": { - "description": { - "text": "This artifactLocation has neither a uri nor a uriBaseId. This is fine." - } - }, - "RULES_ROOT": { - "uri": "file:///C:/rules/", - "description": { - "text": "This artifactLocation has no uriBaseId, so its uri, if present, must be absolute." - } - }, - "SOURCE_ROOT": { - "uri": "src/", - "uriBaseId": "PROJECT_ROOT", - "description": { - "text": "This artifactLocation has a uriBaseId so its uri must be relative. But that is not enforced by SARIF1018; it will be enforced by a separate rule." - } - } - }, - "results": [], - "columnKind": "utf16CodeUnits" - } - ] -} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif new file mode 100644 index 000000000..aa2339625 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Invalid.sarif @@ -0,0 +1,28 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "NoPeriod": { + "text": "This message contains enquoted dynamic content '{0}' but does not end with a period" + } + } + } + ] + } + }, + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif new file mode 100644 index 000000000..e2ad2b10b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2001.TerminateMessagesWithPeriod_Valid.sarif @@ -0,0 +1,28 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "Default": { + "text": "This string has enquoted dynamic content '{0}' and '{1}', and ends with a period." + } + } + } + ] + } + }, + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Invalid.sarif new file mode 100644 index 000000000..5c43a9d7f --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Invalid.sarif @@ -0,0 +1,23 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3" + } + }, + "results": [ + { + "ruleId": "TEST1001", + "message": { + "text": "Test message." + } + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Valid.sarif new file mode 100644 index 000000000..1bc2e64c6 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2002.ProvideMessageArguments_Valid.sarif @@ -0,0 +1,41 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test message." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "results": [ + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID", + "SRCINVALID" + ] + } + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif similarity index 78% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif index a3e5bae05..41573e196 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1020.ReferToFinalSchema_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Invalid.sarif @@ -5,10 +5,10 @@ { "tool": { "driver": { - "name": "Sarif Functional Testing" + "name": "CodeScanner", + "version": "1.0" } }, - "results": [], "columnKind": "utf16CodeUnits" } ] diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif new file mode 100644 index 000000000..ddcbf72c4 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2003.ProvideVersionControlProvenance_Valid.sarif @@ -0,0 +1,20 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Invalid.sarif new file mode 100644 index 000000000..f8172d4c0 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Invalid.sarif @@ -0,0 +1,51 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Code Analyser", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001" + } + ] + } + }, + "artifacts": [ + { + "location": { + "uri": "local/test/code/src/file1.cs", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "level": "error", + "message": { "text": "Bad result: artifacts array does not provide any additional useful information." }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0, + "uri": "local/test/code/src/file1.cs", + "uriBaseId": "TEST_DIR" + }, + "region": { + "startLine": 2, + "startColumn": 53 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Valid.sarif new file mode 100644 index 000000000..ab7a6b108 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2004.OptimizeFileSize_Valid.sarif @@ -0,0 +1,50 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Code Analyser", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "name": "a test violation." + } + ] + } + }, + "artifacts": [ + { + "location": { + "uri": "local/test/code/src/file1.cs", + "uriBaseId": "TEST_DIR" + } + } + ], + "results": [ + { + "ruleId": "TEST1001", + "ruleIndex": 0, + "level": "error", + "message": { "text": "Good result: artifacts array provides additional useful information." }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "index": 0 + }, + "region": { + "startLine": 2, + "startColumn": 53 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Invalid.sarif new file mode 100644 index 000000000..d29df6f6d --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Invalid.sarif @@ -0,0 +1,16 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing Long", + "version": "x" + } + }, + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Valid.sarif new file mode 100644 index 000000000..624cffd40 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2005.ProvideToolProperties_Valid.sarif @@ -0,0 +1,16 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3s" + } + }, + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif new file mode 100644 index 000000000..c833fbdb2 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Invalid.sarif @@ -0,0 +1,40 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0", + "downloadUri": "http://www.example.com/tools/codescanner/download.html", + "rules": [ + { + "id": "TST0001", + "helpUri": "http://www.example.com/rules/tst0001.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://example.com/my-project" + } + ], + "results": [ + { + "ruleId": "TST0001", + "ruleIndex": 0, + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "workItemUris": [ + "https://example.com/my-project/issues/42" + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Valid.sarif new file mode 100644 index 000000000..d31c98543 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2006.UrisShouldBeReachable_Valid.sarif @@ -0,0 +1,40 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0", + "downloadUri": "https://github.com/microsoft/sarif-sdk", + "rules": [ + { + "id": "TST0001", + "helpUri": "https://github.com/microsoft/sarif-sdk" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "results": [ + { + "ruleId": "TST0001", + "ruleIndex": 0, + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "workItemUris": [ + "https://github.com/microsoft/sarif-sdk" + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif new file mode 100644 index 000000000..7f53bc66a --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Invalid.sarif @@ -0,0 +1,110 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test 1001 full description." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "originalUriBaseIds": { + "PROJECT_ROOT": { + "uri": "file:///C:/project/" + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://example.com/microsoft/repo1/", + "mappedTo": { + "uri": ".", + "uriBaseId": "PROJECT_ROOT" + } + }, + { + "repositoryUri": "https://example.com/microsoft/repo2/", + "mappedTo": { + "uri": "." + } + }, + { + "repositoryUri": "https://example.com/microsoft/repo3/" + }, + { + "repositoryUri": "https://example.com/microsoft/submodule", + "mappedTo": { + "uri": "sub/", + "uriBaseId": "PROJECT_ROOT" + } + }, + { + "repositoryUri": "https://example.com/microsoft/other", + "mappedTo": { + "uri": "other/", + "uriBaseId": "OTHER_ROOT" + } + } + ], + "results": [ + { + "ruleId": "TEST1001", + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID", + "SRCINVALID" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/file.cs", + "uriBaseId": "PROJECT_ROOT" + } + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/file.cs", + "uriBaseId": "TEST_ROOT" + } + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "file:///c:/project/file.cs" + } + } + }, + { + "physicalLocation": { + "artifactLocation": { + "uri": "sub/src/file.cs", + "uriBaseId": "PROJECT_ROOT" + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif new file mode 100644 index 000000000..1480ffa9e --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_Valid.sarif @@ -0,0 +1,59 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test 1001 full description." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk/", + "mappedTo": { + "uri": "sarif-sdk/", + "uriBaseId": "SOURCE_ROOT" + } + } + ], + "results": [ + { + "ruleId": "TEST1001", + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID", + "SRCINVALID" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "project/file.cs", + "uriBaseId": "SOURCE_ROOT" + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif new file mode 100644 index 000000000..f0dbbfcc7 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2007.ExpressPathsRelativeToRepoRoot_WithoutVersionControlProvenance_Valid.sarif @@ -0,0 +1,50 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2.3", + "rules": [ + { + "id": "TEST1001", + "fullDescription": { + "text": "Test 1001 full description." + }, + "messageStrings": { + "DoesExist": { + "text": "'{0}': Placeholder '{1}'." + } + } + } + ] + } + }, + "results": [ + { + "ruleId": "TEST1001", + "message": { + "id": "DoesExist", + "arguments": [ + "runs[0].originalUriBaseIds.SRCINVALID", + "SRCINVALID" + ] + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "project/file.cs", + "uriBaseId": "SOURCE_ROOT" + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Invalid.sarif new file mode 100644 index 000000000..6ba149900 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Invalid.sarif @@ -0,0 +1,15 @@ +{ + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Sarif Functional Testing", + "version": "1.0" + } + }, + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Valid.sarif new file mode 100644 index 000000000..08e831d4b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2008.ProvideSchema_Valid.sarif @@ -0,0 +1,16 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Sarif Functional Testing", + "version": "1.0" + } + }, + "results": [], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif new file mode 100644 index 000000000..3fcb84434 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Invalid.sarif @@ -0,0 +1,33 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0", + "downloadUri": "http://www.example.com/tools/codescanner/download.html", + "rules": [ + { + "id": "WRONG00001", + "name": "Wrong Rule 00001", + "helpUri": "http://www.example.com/rules/tst0001.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "originalUriBaseIds": { + "SRCINVALID": { + "uri": "file:///c:/Code/sarif-sdk/src/" + } + }, + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif new file mode 100644 index 000000000..c2fb1d0d8 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2009.ConsiderConventionalIdentifierValues_Valid.sarif @@ -0,0 +1,32 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0", + "downloadUri": "http://www.example.com/tools/codescanner/download.html", + "rules": [ + { + "id": "TST0001", + "helpUri": "http://www.example.com/rules/tst0001.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "originalUriBaseIds": { + "SRCROOT": { + "uri": "file:///c:/Code/sarif-sdk/src/" + } + }, + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif new file mode 100644 index 000000000..380906160 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Invalid.sarif @@ -0,0 +1,112 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "results": [ + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3 + }, + "contextRegion": { + "startLine": 2, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3, + "snippet": {"text": "snippet here."} + }, + "contextRegion": { + "startLine": 2, + "endLine": 4 + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3 + }, + "contextRegion": { + "startLine": 2, + "endLine": 4, + "snippet": { "text": "context snippet here." } + } + } + } + ] + }, + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3 + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Valid.sarif new file mode 100644 index 000000000..3bd82600c --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2010.ProvideCodeSnippets_Valid.sarif @@ -0,0 +1,47 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "results": [ + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3, + "snippet": {"text": "region code snippet"} + }, + "contextRegion": { + "startLine": 2, + "endLine": 4, + "snippet": { "text": "context region code snippet. More context here." } + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Invalid.sarif similarity index 67% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Invalid.sarif index e43ae2c84..99513ce8f 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF1016.ContextRegionRequiresRegion_Invalid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Invalid.sarif @@ -5,9 +5,15 @@ { "tool": { "driver": { - "name": "CodeScanner" + "name": "CodeScanner", + "version": "1.0" } }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], "results": [ { "ruleId": "TST0001", @@ -21,9 +27,9 @@ "artifactLocation": { "uri": "src/test.c" }, - "contextRegion": { - "startLine": 2, - "endLine": 4 + "region": { + "startLine": 3, + "snippet": {"text": "region snippet"} } } } diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Valid.sarif new file mode 100644 index 000000000..881c33052 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2011.ProvideContextRegion_Valid.sarif @@ -0,0 +1,47 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "CodeScanner", + "version": "1.0" + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "results": [ + { + "ruleId": "TST0001", + "level": "error", + "message": { + "text": "Some testing occurred." + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "src/test.c" + }, + "region": { + "startLine": 3, + "snippet": { "text": "region code snippet" } + }, + "contextRegion": { + "startLine": 2, + "endLine": 4, + "snippet": { "text": "context region code snippet. More context here." } + } + } + } + ] + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Invalid.sarif new file mode 100644 index 000000000..83373b26d --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Invalid.sarif @@ -0,0 +1,26 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.0.0", + "rules": [ + { + "id": "SARIF2009", + "name": "ConsiderConventionalIdentifierValues" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Valid.sarif new file mode 100644 index 000000000..5c76c8a14 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2012.ProvideHelpUris_Valid.sarif @@ -0,0 +1,27 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.0.0", + "rules": [ + { + "id": "SARIF2009", + "name": "ConsiderConventionalIdentifierValues", + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif similarity index 53% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif index 3b1b4164e..d4c9b65e9 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1017.InvalidIndex_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Invalid.sarif @@ -5,23 +5,22 @@ { "tool": { "driver": { - "name": "SARIF Functional Testing" + "name": "SARIF Functional Testing", + "version": "1.2" } }, - "invocations": [ + "artifacts": [ { - "executionSuccessful": true + "description": { + "text": "no content" + } } ], - "artifacts": [ + "versionControlProvenance": [ { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1017.InvalidIndex_Valid.sarif", - "uriBaseId": "TEST_DIR" - } + "repositoryUri": "https://github.com/microsoft/sarif-sdk" } ], - "results": [], "columnKind": "utf16CodeUnits" } ] diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif similarity index 52% rename from src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Valid.sarif rename to src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif index 5b6352a38..3838e2a34 100644 --- a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/ExpectedOutputs/SARIF1003.UrisMustBeValid_Valid.sarif +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2013.ProvideEmbeddedFileContent_Valid.sarif @@ -5,23 +5,22 @@ { "tool": { "driver": { - "name": "SARIF Functional Testing" + "name": "SARIF Functional Testing", + "version": "1.2" } }, - "invocations": [ + "artifacts": [ { - "executionSuccessful": true + "contents": { + "text": "sample content" + } } ], - "artifacts": [ + "versionControlProvenance": [ { - "location": { - "uri": "FunctionalTestOutput.ValidateCommand/Inputs.SARIF1003.UrisMustBeValid_Valid.sarif", - "uriBaseId": "TEST_DIR" - } + "repositoryUri": "https://github.com/microsoft/sarif-sdk" } ], - "results": [], "columnKind": "utf16CodeUnits" } ] diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif new file mode 100644 index 000000000..21fac9840 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Invalid.sarif @@ -0,0 +1,34 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "NoPlaceholders": { + "text": "This message does not contain dynamic content." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif new file mode 100644 index 000000000..b527a5b7f --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2014.ProvideDynamicMessageContent_Valid.sarif @@ -0,0 +1,34 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "Default": { + "text": "This string has enquoted dynamic content '{0}' and '{1}', and ends with a period." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif new file mode 100644 index 000000000..2fa22b71b --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Invalid.sarif @@ -0,0 +1,34 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "NotEnquoted": { + "text": "This message contains dynamic content {0} that is not enquoted." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif new file mode 100644 index 000000000..b527a5b7f --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/TestData/Multitool/ValidateCommand/Inputs/SARIF2015.EnquoteDynamicMessageContent_Valid.sarif @@ -0,0 +1,34 @@ +{ + "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "SARIF Functional Testing", + "version": "1.2", + "rules": [ + { + "id": "TST0001", + "fullDescription": { + "text": "This is a test." + }, + "messageStrings": { + "Default": { + "text": "This string has enquoted dynamic content '{0}' and '{1}', and ends with a period." + } + }, + "helpUri": "http://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html" + } + ] + } + }, + "versionControlProvenance": [ + { + "repositoryUri": "https://github.com/microsoft/sarif-sdk" + } + ], + "columnKind": "utf16CodeUnits" + } + ] +} \ No newline at end of file diff --git a/src/Test.FunctionalTests.Sarif/default.configuration.xml b/src/Test.FunctionalTests.Sarif/default.configuration.xml new file mode 100644 index 000000000..284b7c7de --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/default.configuration.xml @@ -0,0 +1,15 @@ + + + + + + + + + + + + + diff --git a/src/Test.FunctionalTests.Sarif/disable2011.configuration.xml b/src/Test.FunctionalTests.Sarif/disable2011.configuration.xml new file mode 100644 index 000000000..b5621d897 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/disable2011.configuration.xml @@ -0,0 +1,18 @@ + + + + + + + + + + + + + + + + diff --git a/src/Test.FunctionalTests.Sarif/enable2002.configuration.xml b/src/Test.FunctionalTests.Sarif/enable2002.configuration.xml new file mode 100644 index 000000000..dafa7d860 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/enable2002.configuration.xml @@ -0,0 +1,13 @@ + + + + + + + + + + diff --git a/src/Test.FunctionalTests.Sarif/enable2006.configuration.xml b/src/Test.FunctionalTests.Sarif/enable2006.configuration.xml new file mode 100644 index 000000000..036373a82 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/enable2006.configuration.xml @@ -0,0 +1,13 @@ + + + + + + + + + + diff --git a/src/Test.FunctionalTests.Sarif/enable2007.configuration.xml b/src/Test.FunctionalTests.Sarif/enable2007.configuration.xml new file mode 100644 index 000000000..8131fe048 --- /dev/null +++ b/src/Test.FunctionalTests.Sarif/enable2007.configuration.xml @@ -0,0 +1,13 @@ + + + + + + + + + + diff --git a/src/Test.UnitTests.Sarif/Core/RegionTests.cs b/src/Test.UnitTests.Sarif/Core/RegionTests.cs new file mode 100644 index 000000000..ef4aa1467 --- /dev/null +++ b/src/Test.UnitTests.Sarif/Core/RegionTests.cs @@ -0,0 +1,128 @@ +// Copyright (c) Microsoft. All Rights Reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using FluentAssertions; +using Microsoft.CodeAnalysis.Sarif; +using Xunit; + +namespace Microsoft.CodeAnalysis.Test.UnitTests.Sarif.Core +{ + public class RegionTests + { + [Fact] + public void Region_PopulateDefaults_EndLineColumnDefaultsPopulatedCorrectly() + { + var region = new Region + { + StartLine = 5, + StartColumn = 20 + }; + + region.PopulateDefaults(); + + region.StartLine.Should().Be(5); + region.StartColumn.Should().Be(20); + region.EndLine.Should().Be(5); + region.EndColumn.Should().Be(int.MaxValue); + } + + [Fact] + public void Region_PopulateDefaults_StartColumnDefaultsPopulatedCorrectly() + { + var region = new Region + { + StartLine = 5, + }; + + region.PopulateDefaults(); + + region.StartLine.Should().Be(5); + region.StartColumn.Should().Be(1); + region.EndLine.Should().Be(5); + region.EndColumn.Should().Be(int.MaxValue); + } + + [Fact] + public void Region_PopulateDefaults_TextOffsetDefaultsPopulatedCorrectly() + { + var region = new Region + { + CharOffset = 5 + }; + + region.PopulateDefaults(); + + region.CharOffset.Should().Be(5); + region.CharLength.Should().Be(0); + } + + [Fact] + public void Region_PopulateDefaults_BinaryOffsetDefaultsPopulatedCorrectly() + { + var region = new Region + { + ByteOffset = 15 + }; + + region.PopulateDefaults(); + + region.ByteOffset.Should().Be(15); + region.ByteLength.Should().Be(0); + } + + [Fact] + public void Region_IsBinaryRegion_ComputesCorrectly() + { + var region = new Region + { + ByteOffset = 15 + }; + + region.IsLineColumnBasedTextRegion.Should().BeFalse(); + region.IsOffsetBasedTextRegion.Should().BeFalse(); + region.IsBinaryRegion.Should().BeTrue(); + } + + [Fact] + public void Region_IsLineColumnBasedTextRegion_ComputesCorrectly() + { + var region = new Region + { + StartLine = 23 + }; + + region.IsLineColumnBasedTextRegion.Should().BeTrue(); + region.IsOffsetBasedTextRegion.Should().BeFalse(); + region.IsBinaryRegion.Should().BeFalse(); + } + + [Fact] + public void Region_IsOffsetBasedTextRegion_ComputesCorrectly() + { + var region = new Region + { + CharOffset = 15 + }; + + region.IsLineColumnBasedTextRegion.Should().BeFalse(); + region.IsOffsetBasedTextRegion.Should().BeTrue(); + region.IsBinaryRegion.Should().BeFalse(); + } + + [Fact] + public void Region_HasAllRegionTypes_ComputesPropertiesCorrectly() + { + var region = new Region + { + StartLine = 23, + CharOffset = 15, + ByteOffset = 15 + + }; + + region.IsLineColumnBasedTextRegion.Should().BeTrue(); + region.IsOffsetBasedTextRegion.Should().BeTrue(); + region.IsBinaryRegion.Should().BeTrue(); + } + } +} \ No newline at end of file diff --git a/src/Test.UnitTests.Sarif/FileRegionsCacheTests.cs b/src/Test.UnitTests.Sarif/FileRegionsCacheTests.cs index 8203e0d70..2fb0a9e42 100644 --- a/src/Test.UnitTests.Sarif/FileRegionsCacheTests.cs +++ b/src/Test.UnitTests.Sarif/FileRegionsCacheTests.cs @@ -55,19 +55,19 @@ public TestCaseData(Region inputRegion, Region outputRegion) private const string LINES_2_AND_3 = "efg\r\nhijk"; private const string CARRIAGE_RETURN_NEW_LINE = "\r\n"; - private readonly static Region s_Insertion_Beginning_Of_Binary_File = + private readonly static Region s_Insertion_Beginning_Of_OffsetBased_Text_File = new Region() { - Snippet = null, - StartLine = 0, - StartColumn = 0, - EndLine = 0, - EndColumn = 0, + Snippet = new ArtifactContent() { Text = INSERTION_POINT }, + StartLine = 1, + StartColumn = 1, + EndLine = 1, + EndColumn = 1, CharOffset = 0, CharLength = 0 }; - private readonly static Region s_Insertion_Beginning_Of_Text_File = + private readonly static Region s_Insertion_Beginning_Of_LineColumnBased_Text_File = new Region() { Snippet = new ArtifactContent() { Text = INSERTION_POINT }, @@ -79,6 +79,20 @@ public TestCaseData(Region inputRegion, Region outputRegion) CharLength = 0 }; + private readonly static Region s_Insertion_Beginning_Of_Binary_File = + new Region() + { + Snippet = null, + StartLine = 0, + StartColumn = 0, + EndLine = 0, + EndColumn = 0, + CharOffset = -1, + CharLength = 0, + ByteOffset = 0, + ByteLength = 0 + }; + private readonly static Region s_Insertion_End_Of_File = new Region() { @@ -272,15 +286,19 @@ public TestCaseData(Region inputRegion, Region outputRegion) private static readonly ReadOnlyCollection s_specExampleTestCases = new ReadOnlyCollection(new TestCaseData[] { - // Insertion point at beginning of binary file - new TestCaseData(outputRegion : s_Insertion_Beginning_Of_Binary_File, + // Insertion point at beginning of an offset based text file + new TestCaseData(outputRegion : s_Insertion_Beginning_Of_OffsetBased_Text_File, inputRegion: new Region() { CharOffset = 0}), - // Insertion point at beginning of text file, can only + // Insertion point at beginning of a line/column based text file, can only // be denoted by use of startLine - new TestCaseData(outputRegion : s_Insertion_Beginning_Of_Text_File, + new TestCaseData(outputRegion : s_Insertion_Beginning_Of_LineColumnBased_Text_File, inputRegion: new Region() { StartLine = 1, StartColumn = 1, EndColumn = 1, CharOffset = 0 }), + // Insertion point at beginning of a based binary file + new TestCaseData(outputRegion : s_Insertion_Beginning_Of_Binary_File, + inputRegion: new Region() { ByteOffset = 0}), + new TestCaseData(outputRegion : s_Insertion_End_Of_File, inputRegion: new Region() { CharOffset = 20 }),