Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port feature/agent_framework commits to main branch #297

Merged
merged 27 commits into from
Dec 18, 2023

Conversation

dbwiddis
Copy link
Member

@dbwiddis dbwiddis commented Dec 15, 2023

We want to preserve the original PR commit history

Do NOT "Squash and Merge".  Choose the "Rebase and Merge" option.

Description

Backports all commits from feature/agent_framework branch (except those that are already on main or were CI-specific for the feature branch/2.11 development).

Issues Resolved

Fixes #264

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

owaiskazi19 and others added 5 commits December 15, 2023 09:24
opensearch-project#198)

* Initial register agent workflow step

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Added tools step

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed ClassCastException

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled exception for Instant

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Added type Instant for WorklowNode Parser

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Removed created and last updated time

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Addressed parsing error

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled parsing of Long values for Instant

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled nested object for llm key

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled parsing error

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Another attempt to fix parsing error for llm

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Another attemp to fix XContent

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed Parsing error

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Added tests for toolstep and javadocs

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Undo CI changes

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Addressing PR comments

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Addressing PR comments

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled interface changes

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Addressed conflicts

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Added TODO

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

---------

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
…ct#211)

* Add Delete Connector Step

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add eclipse core runtime version resolution

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use JDK17 for spotless

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add Delete Connector Step

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add eclipse core runtime version resolution

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use JDK17 for spotless

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fetch connector ID from appropriate previous node output

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix tests

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Test that actual ID is properly passed

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Update to current setup-java version

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Remove unneeded argument captors

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
…ch-project#231)

* adding new resources created format and adding enum for resource types

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* remove spotless from java 17

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* add action listener to update resource created

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* fixing UT

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* changed exception type

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

---------

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>
… workflow steps (opensearch-project#235)

* Flattened llm field of register agent

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled optional modelId

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled modelId for llm

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Parsing for parameters field of tools

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Handled test case failures

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed spotless failure

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

---------

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
…s step output (opensearch-project#234)

* Util method to get required inputs

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Implement parsing in some of the steps

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Handle parsing exceptions in the future

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Improve exception handling

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* More steps using the new input parsing

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Update Delete Connector Step with parsing util

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
dbwiddis and others added 11 commits December 15, 2023 11:25
…tory (opensearch-project#243)

Actually make the WorkflowStepFactory a Factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Add Delete Model Step

Signed-off-by: Daniel Widdis <widdis@gmail.com>
…lSteps (opensearch-project#242)

* Add agentId to parameters map for root agent

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Modified ToolStep with new Util method

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Integrated RegisterAgentStep with new Util method

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Spotless fixes

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Removed TODO

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

---------

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
Adding state index update on agent

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>
…pensearch-project#245)

* Added retry functionality for DeployModel

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed timeout and exception issues

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Addressed PR Comments

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

---------

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
* fixing status api bug with new deploy task

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* changed to getName()

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

---------

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>
Delete Agent Step

Signed-off-by: Daniel Widdis <widdis@gmail.com>
…search-project#262)

Includ workflow id and current node id in the exception message during registe agent step

Signed-off-by: Jackie Han <jkhanjob@gmail.com>
…oject#265)

change thread queue to 100 and fix headers bug

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>
add deploy model resource

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>
@dbwiddis dbwiddis force-pushed the feature-branch branch 2 times, most recently from 6f81cf0 to 77a3599 Compare December 15, 2023 22:27
dbwiddis and others added 6 commits December 15, 2023 14:28
…ch-project#231)

* adding new resources created format and adding enum for resource types

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* remove spotless from java 17

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* add action listener to update resource created

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* fixing UT

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

* changed exception type

Signed-off-by: Amit Galitzky <amgalitz@amazon.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
template by workflow id (opensearch-project#273)

* renaming status API implementation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding GetWorkflow API

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* addressing PR comments

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding todo reminder

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
…h-project#284)

* Modifying workflow state index mapping and resources created

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding Search workflow state API

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding rest unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Transport unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Moving resourceType determination outside of the resources created class

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
…ensearch-project#282)

* replace dryrun parameter with provision in create workflow

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* test

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* test

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* test

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* Combine create api with provision api by adding a provision param

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* cleanup

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* keep dryrun option in create workflow

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* cleanup

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

* keep both dryRun and provision parameter

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

---------

Signed-off-by: Jackie Han <jkhanjob@gmail.com>
dbwiddis and others added 5 commits December 15, 2023 14:29
* Deprovision REST and Transport Actions

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix errors you find actually running the code

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add test for Rest deprovision action

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Initial copypaste of Deprovision Transport Action Test

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add some delays to let deletions propagate, reset workflow state

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Improved deprovisioning results and status updates

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fix bug in resource created parsing

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Completed test implementations

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Fixes after rebase

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
…arch-project#290)

* Adding installed plugins validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Adding failure success unit tests

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Combining graph and installed plugin validation

Signed-off-by: Joshua Palis <jpalis@amazon.com>

* Removing stray comment

Signed-off-by: Joshua Palis <jpalis@amazon.com>

---------

Signed-off-by: Joshua Palis <jpalis@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
…h-project#295)

* Always consume the workflow_id param

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Delegate no-content error message to BaseRestHandler

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Don't lose FlowFrameworkException status

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
@dbwiddis dbwiddis merged commit 0f23dff into opensearch-project:main Dec 18, 2023
21 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/flow-framework/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/flow-framework/backport-2.x
# Create a new branch
git switch --create backport/backport-297-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0f23dff293a5b18868b004546ff2613f887f558e
# Push it to GitHub
git push --set-upstream origin backport/backport-297-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/flow-framework/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-297-to-2.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport PRs to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CHORE] Port Feature Branch commits to main and then 2.x
5 participants