[Snapshot V2] Remove orphan timestamps post create snapshot completion

[gbbafna]

Description

We update pinned timestamps before finalizing the snapshot . In case post pinning timestamp, finalizing snapshot can fail . This leads to orphan pinned timestamps. This will leak data in remote store.

In this change, post every create snapshot operation, we find the orphan pinned timestamps and then unpin it .

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 0503f63: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for d045995: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 1cfb6c7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 14f0323: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 36.00000% with 32 lines in your changes missing coverage. Please review.

Project coverage is 72.08%. Comparing base (4e3a6d0) to head (14f0323).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...ava/org/opensearch/snapshots/SnapshotsService.java	24.24%	24 Missing and 1 partial ⚠️
...remotestore/RemoteStorePinnedTimestampService.java	58.82%	6 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16148      +/-   ##
============================================
+ Coverage     71.91%   72.08%   +0.17%     
- Complexity    64491    64590      +99     
============================================
  Files          5289     5289              
  Lines        301509   301559      +50     
  Branches      43557    43563       +6     
============================================
+ Hits         216818   217370     +552     
+ Misses        66892    66393     -499     
+ Partials      17799    17796       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.