Provide VideoFrame presentation timestamp data from encoded h.264/5 stream data.

[C-Lunn]

The spec states that the VideoFrame object has a timestamp property, defined as:

The presentation duration, given in microseconds. The duration is copied from the EncodedVideoChunk corresponding to this VideoFrame.

However, for at least h.264 and h.265 video, this is incorrect -- it is not necessarily the presentation timestamp, but merely the number passed into the EncodedVideoChunk: it's more of a numerical identifier for the frame. You can, for example, feed 0 in for every frame and frames will still come out in the correct order, but with their so-called 'presentation' timestamp all set to 0.

My issue is such: I have a bunch of EncodedVideoChunks that have the decode timestamp set, but not the presentation timestamp, and so the frames that come out of the decoder are in order, but with a timestamp that jumps around. As I understand it, the h.264/5 decoder does know the actual presentation timestamp, so if this information could be provided, it would be very useful.

I cannot simply use a keyframe + delta frames approach, as the video sources I am working with may have dropped frames and/or timecode discontinuities.

If the decoder could provide this presentation timestamp, it would definitely be useful in my use case. if I've missed something obvious, please feel free to point that out to me, too.

Many thanks.

[sandersdan]

The H.264 decoder does not know the presentation timestamp, unless it is provided explicitly. What it does know is the PicOrderCnt which is just the ordering of the frames.

FWIW, I do not recommend using all-zeros as timestamps; Chrome's implementation uses timestamps to link inputs to outputs and then copy metadata such as duration. It's best if the timestamps are unique. Timestamps are more important for encoding, where they can affect rate control.