[v1.x] Port #19378 from master

[josephevans]

Remove cleanup on side threads (#19378)

• Remove cleanup on side threads

[mxnet-bot]

Hey @josephevans , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

• To trigger all jobs: @mxnet-bot run ci [all]
• To trigger specific jobs: @mxnet-bot run ci [job1, job2]

---

CI supported jobs: [unix-gpu, windows-cpu, website, unix-cpu, clang, centos-gpu, sanity, miscellaneous, edge, windows-gpu, centos-cpu]

---

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.