Skip to content

test: enhance testing logging #4097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 14, 2025
Merged

test: enhance testing logging #4097

merged 3 commits into from
Jul 14, 2025

Conversation

germa89
Copy link
Collaborator

@germa89 germa89 commented Jul 14, 2025

Description

As the title. Improve logging while testing.

Issue linked

NA but related to #1300.

Checklist

Summary by Sourcery

Improve testing logging and debugging, introduce controlled DPF backend testing, and adjust test fixtures for enhanced output and error handling.

New Features:

  • Add test_dpf_backend function to enable DPF backend testing via TEST_DPF_BACKEND environment variable.

Enhancements:

  • Replace log_test_start with unified log_test supporting start and end markers and switched to mapdl.com for consistent messaging.
  • Extend Mapdl reconnection and exit routines with additional DEBUG and WARNING log statements.
  • Raise explicit exceptions when test error representations lack expected attributes.

Tests:

  • Set TEST_DPF_BACKEND flag and adjust run_before_and_after_tests fixture to emit start and end logs conditionally.
  • Change mapdl fixture loglevel to DEBUG and configure log_apdl and mapdl_output paths based on DEBUG_TESTING and ON_LOCAL.

@Copilot Copilot AI review requested due to automatic review settings July 14, 2025 09:49
@germa89 germa89 requested a review from a team as a code owner July 14, 2025 09:49
@germa89 germa89 self-assigned this Jul 14, 2025
@ansys-reviewer-bot
Copy link
Contributor

Thanks for opening a Pull Request. If you want to perform a review write a comment saying:

@ansys-reviewer-bot review

Copy link
Contributor

sourcery-ai bot commented Jul 14, 2025

Reviewer's Guide

Improved test logging and debugging capabilities by consolidating logging functions, introducing a DPF backend flag, refining error messages, enhancing MAPDL session lifecycle logs, and increasing fixture verbosity.

File-Level Changes

Change Details Files
Added DPF backend testing flag
  • Introduce test_dpf_backend() function for environment-based backend selection
  • Expose TEST_DPF_BACKEND in conftest for conditional test runs
tests/common.py
tests/conftest.py
Refactored test logging functions and message formatting
  • Merge start/end logging into a single log_test(mapdl, end) function
  • Replace mapdl._run calls with mapdl.com for muted logging
  • Add end-phase invocation of log_test to mark test completion
tests/common.py
tests/conftest.py
Enhanced MAPDL session lifecycle logs
  • Add LOG.debug statements before and after reconnect attempts
  • Log exit and relaunch actions in is_exited and restart_mapdl
  • Emit warnings on forced exit failures
tests/common.py
Improved test error reporting parsing
  • Distinguish reprcrash and errorstring in get_xpassed_message
  • Raise an exception for unrecognized longrepr formats
tests/conftest.py
Increased fixture verbosity
  • Change default mapdl fixture loglevel from ERROR to DEBUG
  • Conditionally enable log_apdl and mapdl_output based on DEBUG_TESTING and local CI flags
tests/conftest.py
Cleaned up test imports
  • Add missing constants (IS_SMP, ON_CI, etc.) to test_mapdl imports
  • Remove redundant import lines and reorder for clarity
tests/test_mapdl.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@germa89
Copy link
Collaborator Author

germa89 commented Jul 14, 2025

@pyansys-ci-bot LGTM.

Copy link
Contributor

@pyansys-ci-bot pyansys-ci-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approving this PR because germa89 said so in here 😬

LGTM

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves test logging by introducing a unified log_test function with start/end markers, adds a TEST_DPF_BACKEND flag, and increases the default log level to DEBUG during Mapdl launches.

  • Renamed log_test_start to log_test with an optional end parameter and updated its usage.
  • Added a test_dpf_backend helper and TEST_DPF_BACKEND constant to toggle DPF backend tests.
  • Changed default Mapdl launch loglevel from ERROR to DEBUG.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
tests/test_mapdl.py Imported new flags (IS_SMP, ON_CI, ON_LOCAL, QUICK_LAUNCH_SWITCHES, requires) and removed a duplicate import
tests/conftest.py Replaced log_test_start with log_test, added TEST_DPF_BACKEND, enhanced error handling in get_error_message, and updated loglevel
tests/common.py Added the test_dpf_backend function to parse TEST_DPF_BACKEND env var
Comments suppressed due to low confidence (3)

tests/common.py:172

  • [nitpick] The new test_dpf_backend function lacks a docstring. Please add a brief description explaining its purpose and expected inputs/outputs.
def test_dpf_backend() -> bool:

tests/conftest.py:64

  • You import IS_SMP in tests/test_mapdl.py but conftest.py doesn’t define an IS_SMP constant. Consider adding IS_SMP = is_smp() alongside the other environment-based constants.
TEST_DPF_BACKEND = test_dpf_backend()

tests/conftest.py:403

  • Raising a generic exception for unknown longrepr types may abort the entire test suite unexpectedly. Consider falling back to a safe default message or logging a warning instead of throwing.
                raise Exception(str(rep.longrepr))

@github-actions github-actions bot added the maintenance General maintenance of the repo (libraries, cicd, etc) label Jul 14, 2025
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @germa89 - I've reviewed your changes - here's some feedback:

  • Rename helper function test_dpf_backend to avoid pytest test discovery (e.g. is_dpf_backend_enabled) to prevent accidental execution as a test.
  • Consider adding a custom pytest CLI option (via pytest_addoption) to toggle DPF backend tests rather than relying on environment variables for better configurability.
  • Split the unified log_test function into distinct log_test_start and log_test_end functions to improve readability and avoid boolean flags.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Rename helper function `test_dpf_backend` to avoid pytest test discovery (e.g. `is_dpf_backend_enabled`) to prevent accidental execution as a test.
- Consider adding a custom pytest CLI option (via pytest_addoption) to toggle DPF backend tests rather than relying on environment variables for better configurability.
- Split the unified `log_test` function into distinct `log_test_start` and `log_test_end` functions to improve readability and avoid boolean flags.

## Individual Comments

### Comment 1
<location> `tests/common.py:223` </location>
<code_context>
     return elements


-def log_test_start(mapdl: Mapdl) -> None:
+def log_test(mapdl: Mapdl, end=False) -> None:
     """Print the current test to the MAPDL log file and console output."""
     test_name = os.environ.get(
</code_context>

<issue_to_address>
Consider adding or updating tests for the new log_test() function, especially for the 'end' parameter.

Ensure tests cover both start and end logging behaviors, as well as edge cases like long test names and missing environment variables.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@germa89 germa89 changed the title maint: enhance testing logging test: enhance testing logging Jul 14, 2025
@germa89 germa89 enabled auto-merge (squash) July 14, 2025 09:51
@github-actions github-actions bot added enhancement Improve any current implemented feature and removed maintenance General maintenance of the repo (libraries, cicd, etc) labels Jul 14, 2025
@germa89
Copy link
Collaborator Author

germa89 commented Jul 14, 2025

@pyansys-ci-bot LGTM.

Copy link

codecov bot commented Jul 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.79%. Comparing base (a9d37dc) to head (5e80675).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4097      +/-   ##
==========================================
- Coverage   91.82%   91.79%   -0.03%     
==========================================
  Files         187      187              
  Lines       15033    15033              
==========================================
- Hits        13804    13800       -4     
- Misses       1229     1233       +4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@germa89 germa89 merged commit cab7691 into main Jul 14, 2025
45 checks passed
@germa89 germa89 deleted the maint/improving-testing-logging branch July 14, 2025 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improve any current implemented feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants