Ability for "aggregate-init" to read the last record in a stream at connector init #3120
Labels
enhancement
New feature or request
features/smartmodule
good first issue
Good for newcomers
help wanted
Good issue for community involvement
no-issue-activity
When using an aggregate with producers, the aggregate needs and initial value. However, connectors don't currently have the ability to keep state, tough it has access to the data stream.
We should implement an
aggregate-init
function that gives the user access to the last record in the stream to initialize the aggregate value. As a references, there was a "custom" implementation of this in one of the legacy connectors:https://github.com/infinyon/fluvio-connectors/blob/main/examples/github-stars/http-connector.yaml
This function should be available to aggregate smartmodules template in place of the parameter init.
Without this function, I end-up with this:
{"forks":134,"stars":1723}
{"forks":134,"stars":1723}
{"forks":134,"stars":1723,"old_forks":0,"old_stars":0}
{"forks":134,"stars":1723}
{"forks":134,"stars":1723} (edited)
every time the producer connector restarts.
The text was updated successfully, but these errors were encountered: