Enhance validation for graph checks error messages #142

Aymen-Soussi-01 · 2025-07-17T10:00:04Z

Adding Attribute explanation to the graph checks in the metamodel
Extract the explanation attribute and add it to the log when we have errors for graph checks
Assert to ensure the explanation attribute is there
Adapt the RST Tests to the new log error format output

close: #115

…ssages

github-actions · 2025-07-17T10:01:19Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //src:license-check

Status: ✅ Passed

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 3013d59b-b0a5-4cad-96d5-301ddd062e9a
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: src
Loading: 0 packages loaded
    currently loading: src
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //src:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //src:license-check (103 packages loaded, 10 targets configured)

Analyzing: target //src:license-check (145 packages loaded, 1577 targets configured)

Analyzing: target //src:license-check (153 packages loaded, 2652 targets configured)

Analyzing: target //src:license-check (158 packages loaded, 2687 targets configured)

Analyzing: target //src:license-check (162 packages loaded, 4820 targets configured)

INFO: Analyzed target //src:license-check (163 packages loaded, 4946 targets configured).
[8 / 13] Creating runfiles tree bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/score_dash_license_checker~/tool/formatters/dash_format_converter.runfiles [for tool]; 0s local ... (2 actions, 1 running)
INFO: Found 1 target...
Target //src:license.check.license_check up-to-date:
  bazel-bin/src/license.check.license_check
  bazel-bin/src/license.check.license_check.jar
INFO: Elapsed time: 14.989s, Critical Path: 0.32s
INFO: 13 processes: 4 disk cache hit, 9 internal.
INFO: Build completed successfully, 13 total actions
INFO: Running command line: bazel-bin/src/license.check.license_check src/formatted.txt -review -project automotive.score -repo https://github.com/eclipse-score/docs-as-code -token otyhZ4eaRYK1tKLNNF-Y
[main] INFO Querying Eclipse Foundation for license data for 83 items.
[main] INFO Found 58 items.
[main] INFO Querying ClearlyDefined for license data for 25 items.
[main] INFO Found 25 items.
[main] INFO Vetted license information was found for all content. No further investigation is required.

github-actions · 2025-07-17T10:03:21Z

The created documentation from the pull request is available at: docu-html

MaximilianSoerenPollak · 2025-07-17T12:48:35Z

src/extensions/score_metamodel/checks/graph_checks.py

    oper: dict[str, Any] = {
        "and": operator.and_,
        "or": operator.or_,
-        "not": operator.not_,
+        "not": lambda x: not x,


Can you expaline why this is needed?

When I was checking if there is other changes I need to make in graph_checks.py to make the tests works I found that Python’s operator.not_ actually expects a single operand, but its behavior can be a bit inconsistent with non-boolean types.
Using lambda x: not x is a safer and more explicit way to handle the logical not operation on evaluated conditions.
I’ve replaced operator.not_ with lambda x: not x for clarity and correctness.

Can you give me some examples?

'operator.not_' works only with boolean inputs. It expects a direct boolean value like True or False.
However, in our function, the evaluated condition might not be a pure boolean — it might be a string, number, or even a None, which Python treats as "falsy" or "truthy" in logical contexts.
Here are some Examples:

```' # Example 1 print(operator.not_(True)) # ✅ Works → False print(operator.not_(False)) # ✅ Works → True # Example 2 - Non-boolean print(operator.not_("status")) # ✅ Works → False print(operator.not_("")) # ✅ Works → True # BUT... # Example 3 - What if we accidentally give a list or dict? print(operator.not_({"key": "value"})) # ❌ Technically works (False), but semantically risky '''

'operator.not_ 'doesn’t do type checking, and in more complex condition trees, it's harder to ensure only boolean values are passed.
In our recursive eval_need_condition, the return value might not always be strictly boolean, leading to unclear behavior or silent logic errors.

'lambda x: not x' is safer as it ensures you're explicitly controlling the context — Python's native not works on any type and follows standard truthy/falsy semantics just like Python developers expect. It also reads more clearly when debugging or tracing the logic.

MaximilianSoerenPollak · 2025-07-17T12:48:44Z

src/extensions/score_metamodel/checks/graph_checks.py

+    if cond == "not":
+        if not isinstance(vals, list) or len(vals) != 1:
+            raise ValueError("Operator 'not' requires exactly one operand.")
+        return oper["not"](eval_need_condition(need, vals[0], log))


Same as above

For sure it's a suggestion and I can roll back the changes if needed.

MaximilianSoerenPollak · 2025-07-17T12:49:26Z

src/extensions/score_metamodel/checks/graph_checks.py

                parent_ids = need[parent_relation]
+                if not isinstance(parent_ids, list):


I don't quiet understand this?

What's the pourpes of this?

Just to ensure it's a list, as we are going to loop through it, but I have removed this if statement now

Theoretically it makes sense, though if this weren't a list then we have much bigger problems, as that means the Metamodel loaded wrong.
If you truly want you can make this as an 'assert' here.

I will just keep it simple and remove the if there is no need for an assert I guess

MaximilianSoerenPollak · 2025-07-17T12:50:23Z

src/extensions/score_metamodel/metamodel.yaml

@@ -714,13 +714,15 @@ graph_checks:
      condition: safety == QM
    check:
      satisfies: safety == QM
+    explanation: An ASIL requirement must link at least one parent/upstream ASIL requirement for correct decomposition. Please ensure the parent’s safety level is QM and its status is valid.


Status valid is not checked here.
Though it will be in the future as far as I know. So I think we should keep this Explanation for now

MaximilianSoerenPollak · 2025-07-17T12:50:57Z

src/extensions/score_metamodel/metamodel.yaml

  # If need-req is `ASIL_B`, parent must be `QM` or `ASIL_B`.
  req_safety_linkage_asil_b:
    needs:
      include: comp_req, feat_req
      condition: safety == ASIL_B
    check:
      satisfies: safety != ASIL_D
+    explanation: An ASIL requirement must link at least one parent/upstream ASIL requirement for correct decomposition. Please ensure the parent’s safety level is not ASIL_D and its status is valid.


Missing that this only applies when current requirement is ASIL_B, not whne it's QM

MaximilianSoerenPollak · 2025-07-17T12:52:50Z

src/extensions/score_metamodel/metamodel.yaml

@@ -732,3 +734,4 @@ graph_checks:
      condition: safety == ASIL_B
    check:
      mitigates: safety != QM
+    explanation: An ASIL requirement must link at least one parent/upstream ASIL requirement for correct decomposition. Please ensure the parent’s safety level is not QM and its status is valid.


Doesn't make sense.

Suggested change

explanation: An ASIL requirement must link at least one parent/upstream ASIL requirement for correct decomposition. Please ensure the parent’s safety level is not QM and its status is valid.

explanation: An ASIL_B safety requirement must link to a ASIL_B requirement. Please ensure that the linked requirements safety level is not QM and it's status is valid.

but here also we it can be equal or higher so must link to ASIL_B or ASIL_D

ASIL_D is no longer available to be choosen.

MaximilianSoerenPollak

Some questions. See Comments

Edit: Also, formatting missing.

…ion description

MaximilianSoerenPollak

Some comments

Aymen-Soussi-01 added 3 commits July 17, 2025 10:01

add explanation to graph checks

86a1b1c

Merge branch 'main' into enhance-validation-for-graph-checks-error-me…

2980d56

…ssages

Correct tests and adapt them to checks

aa57b64

Aymen-Soussi-01 requested review from AlexanderLanin, MaximilianSoerenPollak and dcalavrezo-qorix as code owners July 17, 2025 10:00

github-project-automation bot added this to Infrastructure Jul 17, 2025

github-project-automation bot moved this to Draft in Infrastructure Jul 17, 2025

Aymen-Soussi-01 requested a review from nicu1989 July 17, 2025 10:00

Aymen-Soussi-01 self-assigned this Jul 17, 2025

MaximilianSoerenPollak reviewed Jul 17, 2025

View reviewed changes

Aymen-Soussi-01 added 2 commits July 17, 2025 18:23

Correct ruff format errors and rollback the function eval_need_condit…

de7d8dc

…ion description

Fix review comments

76605f8

MaximilianSoerenPollak reviewed Jul 18, 2025

View reviewed changes

		parent_ids = need[parent_relation]
		if not isinstance(parent_ids, list):

	explanation: An ASIL requirement must link at least one parent/upstream ASIL requirement for correct decomposition. Please ensure the parent’s safety level is not QM and its status is valid.
	explanation: An ASIL_B safety requirement must link to a ASIL_B requirement. Please ensure that the linked requirements safety level is not QM and it's status is valid.

Enhance validation for graph checks error messages #142

Are you sure you want to change the base?

Enhance validation for graph checks error messages #142

Conversation

Aymen-Soussi-01 commented Jul 17, 2025

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

github-actions bot commented Jul 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaximilianSoerenPollak left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaximilianSoerenPollak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

MaximilianSoerenPollak left a comment •

edited

Loading