-
Notifications
You must be signed in to change notification settings - Fork 36
Open
Labels
Prio1Medium priority, needs attention, order of 2-3 weeksMedium priority, needs attention, order of 2-3 weekscfp-datamodelCodeflare pipelines datamodelCodeflare pipelines datamodelgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededuser-storyUser storyUser story
Description
Overview
As a CFP user, I would like to split a dataset (e.g., np array, pandas dataframe) into smaller objects that can then be fed into other nodes/pipeline. This is especially useful when we have compute intensive tasks and would like to parallelize it easily.
Acceptance Criteria
- Design for splitter, should be simple and intuitive
- Implementation as an extension to the
Node
construct - Tests
Questions
- What type of semantics does the splitter node define?
Assumptions
Reference
Metadata
Metadata
Assignees
Labels
Prio1Medium priority, needs attention, order of 2-3 weeksMedium priority, needs attention, order of 2-3 weekscfp-datamodelCodeflare pipelines datamodelCodeflare pipelines datamodelgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededuser-storyUser storyUser story