Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeline_agg #636

Merged
merged 3 commits into from
Dec 6, 2022
Merged

Add timeline_agg #636

merged 3 commits into from
Dec 6, 2022

Conversation

syvb
Copy link
Member

@syvb syvb commented Nov 25, 2022

Adds a timeline_agg aggregate that can be used in the same ways as state_agg but also allows getting the entire state timeline with state_timeline.

Note that this PR returns start times and end times instead of tstzranges. pgx 0.6.0 will add support for tstzranges so when that release comes out I'll update the relevant functions to return ranges.

This partially addresses #619 (adding rollup and supporting integer values will be separate PRs to make them easier to review).

@syvb syvb changed the title Add timeline_agg and rollup for state aggregates Add timeline_agg for state_agg Nov 28, 2022
@syvb syvb changed the title Add timeline_agg for state_agg Add timeline_agg Nov 28, 2022
@syvb syvb marked this pull request as ready for review November 28, 2022 14:11
extension/src/state_aggregate.rs Outdated Show resolved Hide resolved
extension/src/state_aggregate.rs Show resolved Hide resolved
Copy link
Contributor

@WireBaron WireBaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of nits about the layout of the StateAgg structure. Feel free to ignore these comments, they're probably both in the high effort, low reward category.

#[derive(Clone, Debug, Deserialize, Eq, FlatSerializable, PartialEq, Serialize)]
#[repr(C)]
pub struct TimeInState {
start_time: i64,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpick, but it seems a bit wasteful to be always storing this when, for every value but the first, it will be the same as the end_time for the previous value.


#[derive(Clone, Debug, Deserialize, Eq, FlatSerializable, PartialEq, Serialize)]
#[repr(C)]
pub struct DurationInState {
duration: i64, // TODO BRIAN is i64 or u64 the right type
duration: i64,
state_beg: u32,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We really should be able to store an array of strings in the same order as their duration information, though we don't currently have a flat serializable implementation for this. Even with that we'd have to store some index in each TimeInState though, which would have to take up 64 bytes to be aligned, so it's not a big win.

Copy link
Contributor

@WireBaron WireBaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to see if we can be a bit more efficient in our flattened format, but this looks good for an experimental release.

@syvb
Copy link
Member Author

syvb commented Dec 6, 2022

bors r+

@bors
Copy link
Contributor

bors bot commented Dec 6, 2022

Build succeeded:

@bors bors bot merged commit 1484bd2 into main Dec 6, 2022
@bors bors bot deleted the sv/state_agg-more branch December 6, 2022 22:18
@syvb syvb restored the sv/state_agg-more branch December 6, 2022 22:19
@syvb syvb deleted the sv/state_agg-more branch December 6, 2022 22:19
bors bot added a commit that referenced this pull request Dec 6, 2022
640: Support rollup and integer states state/timeline aggs r=Smittyvb a=Smittyvb

- Adds `rollup` for `state_agg` and `timeline_agg`.
- Supports integer states for `state_agg` and `timeline_agg`. An aggregate can have integer states or string states, but not both. For a few functions this would mean having two functions with the same argument types but differing return types, so in that case I made a separate function for integer-valued aggregates (e.g. `state_int_timeline`).

Ideally this should have been two separate PRs, but my changes to `rollup` and the rest of the `state_agg` code have become intertwined.

(this should be merged after #636)

Co-authored-by: Smitty <smitty@timescale.com>
Co-authored-by: Smittyvb <smitty@timescale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants