Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] after upgrade to OpenSearch 2.0, finished tasks no longer get stored in .tasks index #5376

Closed
pbruski opened this issue Nov 24, 2022 · 0 comments · Fixed by #5412
Closed

[BUG] after upgrade to OpenSearch 2.0, finished tasks no longer get stored in .tasks index #5376

pbruski opened this issue Nov 24, 2022 · 0 comments · Fixed by #5412
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing

Comments

@pbruski
Copy link

pbruski commented Nov 24, 2022

Describe the bug
We've upgraded from OpenSearch 1.3 to OpenSearch 2.0. The index mapping of .tasks index didn't get updated to include the cancelled field. Since the mapping on this index is strict, results of finished tasks no longer get stored in the index and are gone from OS as soon as the task is finished. This makes it impossible to track the results of long running tasks.

To Reproduce
Steps to reproduce the behavior:

  1. Install OS 1.3
  2. Run a longer update by query operation (e.g. scripted) (or run it async to get the task id)
  3. Issue a GET on /_tasks, make note of the task id
  4. Issue a GET on /_tasks/taskId. You will get a result while the task is running and after it has finished.
  5. Upgrade to 2.0
  6. Repeat steps 2-4
  7. The task results will be available while the task is running and will disappear right after the task is finished.

Expected behavior
The result of the task is visible even after the task is finished.
@pbruski pbruski added bug Something isn't working untriaged labels Nov 24, 2022
pbruski pushed a commit to pbruski/OpenSearch that referenced this issue Nov 24, 2022
Upgrade the mappings on .tasks index if needed opensearch-project#5376
1cabd1c
pbruski pushed a commit to pbruski/OpenSearch that referenced this issue Nov 24, 2022
Upgrade the mappings on .tasks index if needed (opensearch-project#5376)
5bfde88
pbruski pushed a commit to pbruski/OpenSearch that referenced this issue Nov 24, 2022
Upgrade the mappings on .tasks index if needed (opensearch-project#5376)
39d1e4a 
Signed-off-by: Przemek Bruski <pbruskispam@op.pl>
@pbruski pbruski mentioned this issue Nov 24, 2022
Closed
6 tasks
@adnapibar adnapibar added Indexing Indexing, Bulk Indexing and anything related to indexing and removed untriaged labels Nov 29, 2022
andrross added a commit to andrross/OpenSearch that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatiblity bug with stored Tasks
d674d14 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
@andrross andrross mentioned this issue Nov 30, 2022
Merged
6 tasks
andrross added a commit to andrross/OpenSearch that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks
319aecc 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (#5412)
4616dfa 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes #5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit to andrross/OpenSearch that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (opensearch-project#5412)
665491a 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
Signed-off-by: Andrew Ross <andrross@amazon.com>
@andrross andrross mentioned this issue Nov 30, 2022
Merged
6 tasks
andrross added a commit to andrross/OpenSearch that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (opensearch-project#5412)
8e11adf 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit that referenced this issue Nov 30, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (#5412) (#5415)
17b9077 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes #5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit to andrross/OpenSearch that referenced this issue Dec 3, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (opensearch-project#5412)
8dfd0ff 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
andrross added a commit to andrross/OpenSearch that referenced this issue Dec 3, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (opensearch-project#5412)
284dcde 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes opensearch-project#5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross added a commit that referenced this issue Dec 3, 2022
@andrross
Fix 1.x compatibility bug with stored Tasks (#5412) (#5440)
f2f809e 
When the new 'cancelled' field was introduced it was a miss not to
increment the version number on the mapping definitions for the .tasks
index. This commit fixes that oversight, as well as modifies the
existing backward compatiblity test to ensure that it will catch future
mistakes like this one.

Closes #5376

Signed-off-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit 4616dfa)
Signed-off-by: Andrew Ross <andrross@amazon.com>

Signed-off-by: Andrew Ross <andrross@amazon.com>
@ketanv3 ketanv3 mentioned this issue Jan 10, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working Indexing Indexing, Bulk Indexing and anything related to indexing
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix 1.x compatibility bug with stored Tasks andrross/OpenSearch
2 participants
@pbruski @adnapibar