Commits on Oct 1, 2024

1. fix(celery): close celery.apply spans even without after_task_publi… …

   …sh, when using apply_async (#10676)
   
   The instrumentation for the Celery integration relies on various [Celery
   signals ](https://docs.celeryq.dev/en/stable/userguide/signals.html) in
   order to start and end the span when calling on `apply_async`.
   
   The integration can fail if the expected signals don't trigger, which
   can lead to broken context propagation (and unexpected traces).
   
   **Example:**
   - dd-trace-py expects the signal `before_task_publish` to start the span
   then `after_task_publish` to close the span. If the `after_task_publish`
   signal never gets called (which can happen if a Celery exception occurs
   while processing the app), then the span won't finish.
   - The same thing above can also happen to `task_prerun` and
   `task_postrun`.
   
   **Solution**
   
   This PR patches `apply_async` so that there is a check to see if there
   is a span lingering around and closes it when `apply_task` is called.
   
   If an internal exception happens, the error will be marked on the
   `celery.apply` span.
   
   To track this, I added new logs in debug mode:
   > The after_task_publish signal was not called, so manually closing span
   
   and
   > The task_postrun signal was not called, so manually closing span
   
   There's a related PR #10848
   that works to improve how we extract information based on the protocols,
   that also affects when spans get closed or not.
   
   Special Thanks:
   - Thanks to @tabgok for going through this with me in great detail!
   - @timmc-edx for helping us track it down!
   
   [APMS-13158]
   
   ## Checklist
   - [x] PR author has checked that all the criteria below are met
   - The PR description includes an overview of the change
   - The PR description articulates the motivation for the change
   - The change includes tests OR the PR description describes a testing
   strategy
   - The PR description notes risks associated with the change, if any
   - Newly-added code is easy to change
   - The change follows the [library release note
   guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
   - The change includes or references documentation updates if necessary
   - Backport labels are set (if
   [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
   
   ## Reviewer Checklist
   - [x] Reviewer has checked that all the criteria below are met
   - Title is accurate
   - All changes are related to the pull request's stated goal
   - Avoids breaking
   [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
   changes
   - Testing strategy adequately addresses listed risks
   - Newly-added code is easy to change
   - Release note makes sense to a user of the library
   - If necessary, author has acknowledged and discussed the performance
   implications of this PR as reported in the benchmarks PR comment
   - Backport labels are set in a manner that is consistent with the
   [release branch maintenance
   policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
   
   APMS-13158
   
   [APMS-13158]:
   https://datadoghq.atlassian.net/browse/APMS-13158?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
   
   ---------
   
   Co-authored-by: Emmett Butler <723615+emmettbutler@users.noreply.github.com>
   (cherry picked from commit 0d28e08)

   

   wantsui authored and github-actions[bot] committed Oct 1, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for f285b51
   • Browse repository at this point

   Copy the full SHA

   f285b51 View commit details

   Browse the repository at this point in the history

2. Merge branch '2.12' into backport-10676-to-2.12

   

   erikayasuda authored Oct 1, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 87fec0a
   • Browse repository at this point

   Copy the full SHA

   87fec0a View commit details

   Browse the repository at this point in the history