-
Notifications
You must be signed in to change notification settings - Fork 506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Merged by Bors] - feat: generate SmartModules using SMDK #2630
[Merged by Bors] - feat: generate SmartModules using SMDK #2630
Conversation
_out: Arc<O>, | ||
_fluvio: &Fluvio, | ||
) -> Result<()> { | ||
let template_path = TemplatePath { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neither this struct
nor GenerateArgs
implements Default trait. So I had to provide initial values for both.
I'll open a PR to the generate crate to see if they allow us to have a Default
implementation.
Addresses #2631 |
see: #2632 |
Updated #2631 per new Smdk spec |
Awesome! Here I'm also taking care of the build step by spawning a |
Great! Will migrate to |
e0fd9fd
to
154a5c9
Compare
@sehz now we have support for testing the SmartModule build using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also attach running output of working?
44cbfe5
to
68c785f
Compare
Provides support to generate new SmartModules using `cargo-generate` as a libary. Resolve: infinyon#2621
a8b337e
to
54a1733
Compare
37e5fd2
to
f8e2e84
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, can you add output from generating->build->test including init?
I believe template need handling for init
Current template has support for
But I dont find any effect from such option in the https://github.com/infinyon/fluvio/blob/master/smartmodule/cargo_template/src/lib.rs |
That needs to be added. See regex project |
b6511c1
to
4752f3d
Compare
Handling for |
@sehz im stick to the RegExp project version instead of using the current cargo-generate template for this generation. Also added |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Nice work!
bors r+ |
Provides support to generate new SmartModules using `cargo-generate` as a libary. Resolve: #2621 --- ## Examples ### Generates SmartModule Projects using `smdk generate` ```bash ➜ Desktop smdk generate log-filter Generating new SmartModule project: log-filter 🔧 Destination: /Users/esteban/Desktop/log-filter ... 🔧 Generating template ... ✔ 🤷 Which type of SmartModule would you like? · filter ✔ 🤷 Want to use SmartModule parameters? · false 🤷 SmartModule Version [default: git = "https://github.com/infinyon/fluvio.git"]: git = "https://github.com/infinyon/fluvio.git" ✔ 🤷 Want to use SmartModule init? · true Ignoring: /var/folders/0g/l702s31x2mj4zdlw2_43rpyr0000gn/T/.tmpwQ4Yql/cargo-generate.toml [1/5] Done: Cargo.toml [2/5] Done: README.md [3/5] Done: Smart.toml [4/5] Done: src/lib.rs [5/5] Done: src 🔧 Moving generated files into: `/Users/esteban/Desktop/log-filter`... 💡 Initializing a fresh Git repository ✨ Done! New project created /Users/esteban/Desktop/log-filter ``` ### Generated Project ```bash total 24 drwxr-xr-x 7 esteban staff 224B Oct 2 13:54 . drwx------+ 6 esteban staff 192B Oct 2 13:54 .. drwxr-xr-x 9 esteban staff 288B Oct 2 13:54 .git -rw-r--r-- 1 esteban staff 358B Oct 2 13:54 Cargo.toml -rw-r--r-- 1 esteban staff 1.8K Oct 2 13:54 README.md -rw-r--r-- 1 esteban staff 152B Oct 2 13:54 Smart.toml drwxr-xr-x 3 esteban staff 96B Oct 2 13:54 src ``` ### Generated `src/lib.rs` with the provided options ```rust use fluvio_smartmodule::dataplane::smartmodule::{SmartModuleExtraParams, SmartModuleInitError}; use once_cell::sync::OnceCell; use fluvio_smartmodule::eyre; use fluvio_smartmodule::{smartmodule, Result, Record}; #[smartmodule(filter)] pub fn filter(record: &Record) -> Result<bool> { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } static CRITERIA: OnceCell<String> = OnceCell::new(); #[smartmodule(init)] fn init(params: SmartModuleExtraParams) -> Result<()> { // You can refer to the example SmartModules in Fluvio's GitHub Repository // https://github.com/infinyon/fluvio/tree/master/smartmodule if let Some(key) = params.get("key") { CRITERIA.set(key.clone()).map_err(|err| eyre!("failed setting key: {:#?}", err)) } else { Err(SmartModuleInitError::MissingParam("key".to_string()).into()) } } ``` ### Building SMDK Project ```bash ➜ Desktop cd ./log-filter ➜ log-filter git:(main) ✗ smdk build Updating git repository `https://github.com/infinyon/fluvio.git` Updating crates.io index Compiling proc-macro2 v1.0.46 ``` ### Testing SM Project using `smdk test` ``` ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "testing" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 0 records outputed ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "tasty" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 1 records outputed tasty ```
Build failed: |
Mac VM failures |
bors r+ |
Provides support to generate new SmartModules using `cargo-generate` as a libary. Resolve: #2621 --- ## Examples ### Generates SmartModule Projects using `smdk generate` ```bash ➜ Desktop smdk generate log-filter Generating new SmartModule project: log-filter 🔧 Destination: /Users/esteban/Desktop/log-filter ... 🔧 Generating template ... ✔ 🤷 Which type of SmartModule would you like? · filter ✔ 🤷 Want to use SmartModule parameters? · false 🤷 SmartModule Version [default: git = "https://github.com/infinyon/fluvio.git"]: git = "https://github.com/infinyon/fluvio.git" ✔ 🤷 Want to use SmartModule init? · true Ignoring: /var/folders/0g/l702s31x2mj4zdlw2_43rpyr0000gn/T/.tmpwQ4Yql/cargo-generate.toml [1/5] Done: Cargo.toml [2/5] Done: README.md [3/5] Done: Smart.toml [4/5] Done: src/lib.rs [5/5] Done: src 🔧 Moving generated files into: `/Users/esteban/Desktop/log-filter`... 💡 Initializing a fresh Git repository ✨ Done! New project created /Users/esteban/Desktop/log-filter ``` ### Generated Project ```bash total 24 drwxr-xr-x 7 esteban staff 224B Oct 2 13:54 . drwx------+ 6 esteban staff 192B Oct 2 13:54 .. drwxr-xr-x 9 esteban staff 288B Oct 2 13:54 .git -rw-r--r-- 1 esteban staff 358B Oct 2 13:54 Cargo.toml -rw-r--r-- 1 esteban staff 1.8K Oct 2 13:54 README.md -rw-r--r-- 1 esteban staff 152B Oct 2 13:54 Smart.toml drwxr-xr-x 3 esteban staff 96B Oct 2 13:54 src ``` ### Generated `src/lib.rs` with the provided options ```rust use fluvio_smartmodule::dataplane::smartmodule::{SmartModuleExtraParams, SmartModuleInitError}; use once_cell::sync::OnceCell; use fluvio_smartmodule::eyre; use fluvio_smartmodule::{smartmodule, Result, Record}; #[smartmodule(filter)] pub fn filter(record: &Record) -> Result<bool> { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } static CRITERIA: OnceCell<String> = OnceCell::new(); #[smartmodule(init)] fn init(params: SmartModuleExtraParams) -> Result<()> { // You can refer to the example SmartModules in Fluvio's GitHub Repository // https://github.com/infinyon/fluvio/tree/master/smartmodule if let Some(key) = params.get("key") { CRITERIA.set(key.clone()).map_err(|err| eyre!("failed setting key: {:#?}", err)) } else { Err(SmartModuleInitError::MissingParam("key".to_string()).into()) } } ``` ### Building SMDK Project ```bash ➜ Desktop cd ./log-filter ➜ log-filter git:(main) ✗ smdk build Updating git repository `https://github.com/infinyon/fluvio.git` Updating crates.io index Compiling proc-macro2 v1.0.46 ``` ### Testing SM Project using `smdk test` ``` ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "testing" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 0 records outputed ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "tasty" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 1 records outputed tasty ```
Build failed: |
bors r+ |
Provides support to generate new SmartModules using `cargo-generate` as a libary. Resolve: #2621 --- ## Examples ### Generates SmartModule Projects using `smdk generate` ```bash ➜ Desktop smdk generate log-filter Generating new SmartModule project: log-filter 🔧 Destination: /Users/esteban/Desktop/log-filter ... 🔧 Generating template ... ✔ 🤷 Which type of SmartModule would you like? · filter ✔ 🤷 Want to use SmartModule parameters? · false 🤷 SmartModule Version [default: git = "https://github.com/infinyon/fluvio.git"]: git = "https://github.com/infinyon/fluvio.git" ✔ 🤷 Want to use SmartModule init? · true Ignoring: /var/folders/0g/l702s31x2mj4zdlw2_43rpyr0000gn/T/.tmpwQ4Yql/cargo-generate.toml [1/5] Done: Cargo.toml [2/5] Done: README.md [3/5] Done: Smart.toml [4/5] Done: src/lib.rs [5/5] Done: src 🔧 Moving generated files into: `/Users/esteban/Desktop/log-filter`... 💡 Initializing a fresh Git repository ✨ Done! New project created /Users/esteban/Desktop/log-filter ``` ### Generated Project ```bash total 24 drwxr-xr-x 7 esteban staff 224B Oct 2 13:54 . drwx------+ 6 esteban staff 192B Oct 2 13:54 .. drwxr-xr-x 9 esteban staff 288B Oct 2 13:54 .git -rw-r--r-- 1 esteban staff 358B Oct 2 13:54 Cargo.toml -rw-r--r-- 1 esteban staff 1.8K Oct 2 13:54 README.md -rw-r--r-- 1 esteban staff 152B Oct 2 13:54 Smart.toml drwxr-xr-x 3 esteban staff 96B Oct 2 13:54 src ``` ### Generated `src/lib.rs` with the provided options ```rust use fluvio_smartmodule::dataplane::smartmodule::{SmartModuleExtraParams, SmartModuleInitError}; use once_cell::sync::OnceCell; use fluvio_smartmodule::eyre; use fluvio_smartmodule::{smartmodule, Result, Record}; #[smartmodule(filter)] pub fn filter(record: &Record) -> Result<bool> { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } static CRITERIA: OnceCell<String> = OnceCell::new(); #[smartmodule(init)] fn init(params: SmartModuleExtraParams) -> Result<()> { // You can refer to the example SmartModules in Fluvio's GitHub Repository // https://github.com/infinyon/fluvio/tree/master/smartmodule if let Some(key) = params.get("key") { CRITERIA.set(key.clone()).map_err(|err| eyre!("failed setting key: {:#?}", err)) } else { Err(SmartModuleInitError::MissingParam("key".to_string()).into()) } } ``` ### Building SMDK Project ```bash ➜ Desktop cd ./log-filter ➜ log-filter git:(main) ✗ smdk build Updating git repository `https://github.com/infinyon/fluvio.git` Updating crates.io index Compiling proc-macro2 v1.0.46 ``` ### Testing SM Project using `smdk test` ``` ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "testing" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 0 records outputed ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "tasty" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 1 records outputed tasty ```
Build failed: |
jeez. Murphy's law strikes... |
bors r+ |
Provides support to generate new SmartModules using `cargo-generate` as a libary. Resolve: #2621 --- ## Examples ### Generates SmartModule Projects using `smdk generate` ```bash ➜ Desktop smdk generate log-filter Generating new SmartModule project: log-filter 🔧 Destination: /Users/esteban/Desktop/log-filter ... 🔧 Generating template ... ✔ 🤷 Which type of SmartModule would you like? · filter ✔ 🤷 Want to use SmartModule parameters? · false 🤷 SmartModule Version [default: git = "https://github.com/infinyon/fluvio.git"]: git = "https://github.com/infinyon/fluvio.git" ✔ 🤷 Want to use SmartModule init? · true Ignoring: /var/folders/0g/l702s31x2mj4zdlw2_43rpyr0000gn/T/.tmpwQ4Yql/cargo-generate.toml [1/5] Done: Cargo.toml [2/5] Done: README.md [3/5] Done: Smart.toml [4/5] Done: src/lib.rs [5/5] Done: src 🔧 Moving generated files into: `/Users/esteban/Desktop/log-filter`... 💡 Initializing a fresh Git repository ✨ Done! New project created /Users/esteban/Desktop/log-filter ``` ### Generated Project ```bash total 24 drwxr-xr-x 7 esteban staff 224B Oct 2 13:54 . drwx------+ 6 esteban staff 192B Oct 2 13:54 .. drwxr-xr-x 9 esteban staff 288B Oct 2 13:54 .git -rw-r--r-- 1 esteban staff 358B Oct 2 13:54 Cargo.toml -rw-r--r-- 1 esteban staff 1.8K Oct 2 13:54 README.md -rw-r--r-- 1 esteban staff 152B Oct 2 13:54 Smart.toml drwxr-xr-x 3 esteban staff 96B Oct 2 13:54 src ``` ### Generated `src/lib.rs` with the provided options ```rust use fluvio_smartmodule::dataplane::smartmodule::{SmartModuleExtraParams, SmartModuleInitError}; use once_cell::sync::OnceCell; use fluvio_smartmodule::eyre; use fluvio_smartmodule::{smartmodule, Result, Record}; #[smartmodule(filter)] pub fn filter(record: &Record) -> Result<bool> { let string = std::str::from_utf8(record.value.as_ref())?; Ok(string.contains('a')) } static CRITERIA: OnceCell<String> = OnceCell::new(); #[smartmodule(init)] fn init(params: SmartModuleExtraParams) -> Result<()> { // You can refer to the example SmartModules in Fluvio's GitHub Repository // https://github.com/infinyon/fluvio/tree/master/smartmodule if let Some(key) = params.get("key") { CRITERIA.set(key.clone()).map_err(|err| eyre!("failed setting key: {:#?}", err)) } else { Err(SmartModuleInitError::MissingParam("key".to_string()).into()) } } ``` ### Building SMDK Project ```bash ➜ Desktop cd ./log-filter ➜ log-filter git:(main) ✗ smdk build Updating git repository `https://github.com/infinyon/fluvio.git` Updating crates.io index Compiling proc-macro2 v1.0.46 ``` ### Testing SM Project using `smdk test` ``` ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "testing" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 0 records outputed ➜ log-filter git:(main) ✗ smdk test --params "key=a" --text "tasty" project name: "log-filter" loading module at: target/wasm32-unknown-unknown/release-lto/log_filter.wasm 1 records outputed tasty ```
Pull request successfully merged into master. Build succeeded: |
Provides support to generate new SmartModules using
cargo-generate
as a libary.Resolve: #2621
Examples
Generates SmartModule Projects using
smdk generate
Generated Project
total 24 drwxr-xr-x 7 esteban staff 224B Oct 2 13:54 . drwx------+ 6 esteban staff 192B Oct 2 13:54 .. drwxr-xr-x 9 esteban staff 288B Oct 2 13:54 .git -rw-r--r-- 1 esteban staff 358B Oct 2 13:54 Cargo.toml -rw-r--r-- 1 esteban staff 1.8K Oct 2 13:54 README.md -rw-r--r-- 1 esteban staff 152B Oct 2 13:54 Smart.toml drwxr-xr-x 3 esteban staff 96B Oct 2 13:54 src
Generated
src/lib.rs
with the provided optionsBuilding SMDK Project
Testing SM Project using
smdk test