Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpk/transform: support javascript and typescript #18078

Merged
merged 10 commits into from
May 2, 2024

Conversation

rockwotj
Copy link
Contributor

@rockwotj rockwotj commented Apr 25, 2024

Support JS and TS for Data Transforms.

We still have work to do here, mainly a deploy pipeline for the JS buildpack, otherwise I have proof this is working end to end.

Closes: CORE-2370 and CORE-2373

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.1.x
  • v23.3.x
  • v23.2.x

Release Notes

Features

  • Support JavaScript and TypeScript as languages for Data Transforms.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
to be valid json 🤦

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
@github-actions github-actions bot added area/rpk area/wasm WASM Data Transforms labels Apr 25, 2024
@rockwotj rockwotj self-assigned this Apr 26, 2024
@rockwotj rockwotj marked this pull request as ready for review April 26, 2024 20:49
@rockwotj rockwotj requested review from andrewhsu and oleiman and removed request for a team April 26, 2024 20:49
r-vasquez
r-vasquez previously approved these changes Apr 29, 2024
Copy link
Contributor

@r-vasquez r-vasquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rpk bits LGTM, just a couple of small comments that can help us improve an (unlikely) error message.

src/go/rpk/pkg/cli/transform/build.go Outdated Show resolved Hide resolved
src/go/rpk/pkg/cli/transform/build.go Show resolved Hide resolved
src/go/rpk/pkg/cli/transform/template/js_template.go Outdated Show resolved Hide resolved
To support JS and TS we need:

1) To generate an NPM project with esbuild and (optional) typescript
2) A new buildpack that has our JS VM and can package that with the
   source

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
This script will build the binaries that RPK needs when turning JS ->
Wasm. The resulting tarballs need to be hosted somewhere (S3?) for RPK
to download.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Using this other type of comment allows LSPs to show the docs inline.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Specifically check we can't find the binary, and print the error if it
is less expected.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Copy link
Member

@ivotron ivotron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GHA workflow changes LGTM

@ivotron ivotron removed the request for review from andrewhsu April 30, 2024 17:38
Copy link
Member

@oleiman oleiman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. presumably we want to re-enable LTO at this point?

set_property(TARGET redpanda_js_transform PROPERTY INTERPROCEDURAL_OPTIMIZATION ON)
endif()
endif()
# TODO: Rename LTO builds when #18077 is merged
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#18077 has merged, re-enable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes will do in a followup, I would like to get this in for now

@rockwotj rockwotj merged commit d1d3af5 into redpanda-data:dev May 2, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants