A media player looks like a solved problem until you write one. Decoding is a library call. Drawing a frame is a library call. The work is in the machinery between them — threads, timing, and the queue that connects the two.
Three threads, not one
The naive version decodes and draws in the same loop, and stutters the moment a frame takes longer than its slot. The structure that works separates concerns:
- Demux. Read the container, split it into packet streams, push them into per-stream queues.
- Decode. One worker per stream, turning packets into frames.
- Present. Take decoded frames and put them on screen at the right moment.
The queues between these stages are where the design lives. Too shallow and any hiccup starves the presenter. Too deep and seeking becomes sluggish because you must discard everything already buffered.
Audio is the clock
The single most useful thing to internalise: synchronise video to audio, not the other way around.
Audio hardware consumes samples at a fixed, unforgiving rate. Fall behind and the listener hears a click — obvious and unacceptable. A video frame arriving a few milliseconds late is invisible.
So the audio device becomes the master clock. Each decoded video frame carries a presentation timestamp; the presenter compares it against the audio clock and either waits, shows it, or drops it.
Qt's part
Qt supplies the window, the event loop, and the widget toolkit — and, crucially, one build that runs on Linux, macOS and Windows.
Two things to get right:
- Never block the GUI thread. Decoding on the main thread freezes the interface. Workers communicate back through queued signals, which Qt marshals across threads for you.
- Render through the GPU. Uploading each frame as a texture and letting the GPU handle scaling and colour conversion is dramatically cheaper than doing it on the CPU, and it is what makes high-resolution playback feasible on ordinary hardware.
The parts nobody demos
The demo plays a file. The product handles the rest:
- Seeking, which means flushing every queue and every decoder without leaving stale frames behind.
- Formats where the container lies about duration.
- Files whose audio and video start at different timestamps.
- Hardware decoders that are available, then abruptly are not.
None of this is conceptually hard. All of it is where the time goes.