flow-docs

HLS DVR manifest and FFMpeg program time usage

FFMpeg allows you to include program time information in the HLS (HTTP Live Streaming) “DVR” manifest file for each “chunk”. In practice, this means that each slice of the stream has a timestamp, allowing for more accurate extraction of timing information.

HLS Manifest and Program Date Time

The HLS manifest file is a text file that contains the information the client needs to play the stream. This information includes the available bitrate, the location of the stream slices (chunks), and optionally other metadata.

The EXT-X-PROGRAM-DATE-TIME tag, which indicates the program time, has been available since HLS version 1, making it usable in all HLS versions. This tag appears in the manifest file in the following format:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PROGRAM-DATE-TIME:2023-05-08T14:00:00Z
#EXTINF:8,
fileSequence0.ts
#EXT-X-PROGRAM-DATE-TIME:2023-05-08T14:00:08Z
#EXTINF:8,
fileSequence1.ts
#EXT-X-PROGRAM-DATE-TIME:2023-05-08T14:00:16Z
#EXTINF:8,
fileSequence2.ts
...

Backend: FFMpeg settings

First, you need to make sure that FFMpeg is configured correctly to add the program time information to the HLS manifest file. To do this, you can use the following option:

// ...

ff.addOption('-hls_flags', 'program_date_time')

// ...

This option tells FFMpeg to add program time information to each chunk.

Frontend: Using program time

Once the program time information is added to the HLS manifest file, the frontend can use this information to accurately time the data in the stream.

Simple example: timeupdate event

// ...

let firstDvrTime = null

// ...

hls.on(Hls.Events.MANIFEST_LOADED, (event, data) => {
	firstDvrTime = data.levels[0].details.fragments[0].programDateTime
})

// ...

video.addEventListener('timeupdate', () => {
	const gameTime = new Date(firstDvrTime).getTime() / 1000 + video.currentTime
	console.log('Game Time:', gameTime)
})

// ...

In this example, we read the media’s current playback time (accessed as currentTime) within the timeupdate event listener and add it to the first program time. This calculation gives the relative game time.

Advanced example: requestVideoFrameCallback method

// ...

const onFrame = (pts, { mediaTime }) => {
	console.log(pts) // useless in this usecase
	console.log(new Date(mediaTime * 1000 + firstDvrTime))
	video.requestVideoFrameCallback(onFrame)
}

video.requestVideoFrameCallback(onFrame)
// ...

In this case, we get the mediaTime value directly from the metadata provided by the requestVideoFrameCallback. This is more accurate than the timeupdate event, as it is called for each frame, not just at certain intervals.

These examples demonstrate how to add the first program timestamp in the stream to the current playback time to determine the relative game time.

resources