Codec architecture: H.264, HEVC and AV1 internals
Modern video codecs are all variations on one idea: predict a block, transform the residual, quantise it, and encode what remains with an entropy coder that exploits statistical structure.
Understanding that shared spine makes the differences between standards legible. H.264, formally MPEG-4 Part 10 or AVC, established the block-based hybrid model in the form most engineers still reason about. HEVC, also called H.265, extended it with larger prediction units and a more elaborate partitioning tree. AV1 rewrote much of the toolset with a royalty-free mandate and added tools such as film grain synthesis and advanced motion modelling.
GOP structure and the three frame types
A group of pictures begins at an instantaneous decoder refresh point, the I-frame, which is coded without reference to any other frame. Because it is self-contained, it is a seek point. After it come predicted frames. A P-frame references frames that precede it in display order. A B-frame references frames on both sides, which lets it describe motion more cheaply but introduces reordering, because decoding order and presentation order no longer coincide.
The pattern of frame types is the GOP structure, and it is the primary lever on random access granularity. A closed GOP starting every second gives streaming clients a predictable seek grid and keeps error propagation bounded. A long GOP trades seek flexibility for a better compression ratio, because more frames are described as deltas rather than from scratch. The keyframe strip that titles this site's navigation is a deliberate visual pun on that same structure.
| Type | References | Seekable | Relative size | Typical role |
|---|---|---|---|---|
I | none | yes | largest | GOP anchor, seek point |
P | past frames | no | medium | forward prediction |
B | past and future | no | smallest | bidirectional prediction |
Prediction, transform and quantisation
Intra prediction guesses a block's content from neighbouring samples already coded in the same frame, using directional modes. Inter prediction describes a block as a translated copy of a region in a reference frame, encoded as a motion vector plus a residual. The decoder reconstructs the prediction and adds the residual back.
The residual is transformed into a frequency domain, historically with a discrete cosine transform and in newer standards with integer approximations plus additional transform variants. Low frequencies cluster in the top-left corner, which is where most of the energy sits. Quantisation then divides those coefficients by a step size, discarding precision in exchange for fewer bits. This step, not the transform, dominates perceived quality loss, because coarse quantisation removes the high-frequency detail that the eye uses to judge sharpness.
Entropy coding
The quantised coefficients and syntax elements are encoded with an entropy coder. H.264 offers context-adaptive variable-length coding and the more efficient context-adaptive binary arithmetic coding. HEVC uses a single binary arithmetic coder. AV1 introduces multi-symbol arithmetic coding. In every case the goal is the same: assign short codes to probable symbols and long codes to improbable ones, adapting the probabilities to the local statistics of the bitstream.
Profiles and levels
A profile names a set of coding tools, and a level names the resource envelope a decoder must meet. Profiles let an encoder guarantee a feature set the decoder handles, for instance the main profile of H.264 with its 8-bit 4:2:0 support. Levels bound resolution, frame rate, bitrate and buffer size so that hardware can be designed to a defined ceiling. Devices advertise the combinations they support, and a stream outside an advertised combination may simply fail to decode.
What AV1 changes
AV1's superblock partitioning is more flexible than the fixed macroblock grid of older standards, allowing large coding blocks to be subdivided adaptively where detail demands it. Its motion model includes more reference frames and can warp a predicted block to follow rotation and perspective. Film grain synthesis separates random noise from structured content, transmitting a parameterised grain model instead of spending bits on each noisy pixel. The net effect is a meaningful compression improvement at the cost of more computation in both encoder and decoder, which is why hardware support has advanced alongside adoption.
Summary
The three standards share a spine: predict, transform, quantise, entropy-code. They differ in the granularity of prediction, the flexibility of partitioning and the richness of the motion model. Those differences map directly to the compression gains published in every serious comparison, and they explain why quality knobs carry different numeric ranges across codecs, a theme developed in CRF and RF quality.