Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add includeFields parameter to the method extractFieldNamesTypes #376

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

gaobinlong
Copy link
Contributor

@gaobinlong gaobinlong commented Jul 30, 2024

Description

This is a follow-up PR of #348, in that PR, we moved and refactored the method extractFieldNamesTypes, but the new method has some impact on the PPLTool, in order to remove the impact, this PR add a new parameter includeFields to that method, when it's set to false, the subfields under a text type field will not be resolved and written to the result.

For example, this is a text type field with subfields:

"c": {
      "type": "text",
      "fields": {
        "d": {
          "type":"keyword"
        }
      }
    }

, if includeFields is true, the flattened mapping result contains:

"c": "text",
"c.d": "keyword"

, but if includeFields is false, the flattened mapping result only contains:

"c": "text",

.

In addition, for object field, we do not extract the top level field name and type into the final result since it's useless for both PPLTool and CreateAnomalyDetectorTool actually.
For example, this is an object field with setting type explicitly:

"a": {
      "type": "object",
      "properties": {
        "b": {
          "type":"keyword"
        }
      }
    }

, the flattened mapping result contains:

"a.b": "keyword"

, but doesn't contain:

"a": "object"

.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: gaobinlong <gbinlong@amazon.com>
@gaobinlong
Copy link
Contributor Author

@zane-neo @xinyual please help to review this PR, thanks!

@zane-neo zane-neo merged commit 5a9dbcd into opensearch-project:main Aug 2, 2024
7 of 8 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 2, 2024
* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Remove empty line

Signed-off-by: gaobinlong <gbinlong@amazon.com>

---------

Signed-off-by: gaobinlong <gbinlong@amazon.com>
(cherry picked from commit 5a9dbcd)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
zane-neo pushed a commit that referenced this pull request Aug 2, 2024
… (#378)

* Add includeFields parameter to the method extractFieldNamesTypes



* Remove empty line



---------


(cherry picked from commit 5a9dbcd)

Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 2, 2024
* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Remove empty line

Signed-off-by: gaobinlong <gbinlong@amazon.com>

---------

Signed-off-by: gaobinlong <gbinlong@amazon.com>
(cherry picked from commit 5a9dbcd)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
zane-neo pushed a commit that referenced this pull request Aug 2, 2024
… (#379)

* Add includeFields parameter to the method extractFieldNamesTypes



* Remove empty line



---------


(cherry picked from commit 5a9dbcd)

Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gaobinlong added a commit to gaobinlong/skills that referenced this pull request Sep 3, 2024
* Add CreateAnomalyDetectorTool

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Optimize exception

Signed-off-by: gaobinlong <gbinlong@amazon.com>

---------

Signed-off-by: gaobinlong <gbinlong@amazon.com>

Fix warning and format issue for CreateAnomalyDetectorTool (opensearch-project#358)

Signed-off-by: gaobinlong <gbinlong@amazon.com>

Add includeFields parameter to the method extractFieldNamesTypes (opensearch-project#376)

* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <gbinlong@amazon.com>

* Remove empty line

Signed-off-by: gaobinlong <gbinlong@amazon.com>

---------

Signed-off-by: gaobinlong <gbinlong@amazon.com>

Optimize the prompt for create anomaly detector tool

Signed-off-by: gaobinlong <gbinlong@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants