Glossary

Video and codecs

Almost every confusing thing about video files comes from one distinction — between the box and what is in it. These are the words for both, and for the operations that move things between them.

Container

wrapper, format

The file structure that holds video, audio and subtitle tracks together and describes where each one is. MP4, MKV, AVI and WebM are containers. A container does not compress anything.

Think of it as a filing box with a contents label. The box says: there is a video track here, encoded this way; two audio tracks there, in these languages; subtitles at this offset. It carries no opinion about how any of them were compressed.

This is why the same video can exist in an MP4 and an MKV with identical picture quality and identical file size, and why one of those two opens on an iPhone while the other does not. The difference is the box, not the contents.

Codec

coder-decoder

The method used to compress a video or audio stream, and to decompress it again for playback. H.264, HEVC, AV1, AAC and MP3 are codecs. A codec is not a file format.

The name is a contraction of coder-decoder. Encoding throws away detail in a way calculated to be hard to notice, and decoding reconstructs an approximation of the original from what was kept. The better the codec, the less noticeable the loss at a given size.

A device plays a file when it can read the container *and* decode every codec inside it. Both halves are required, and when a file opens but shows nothing, the second half is what failed.

H.264

AVC, MPEG-4 Part 10

The video codec that everything can decode. Published in 2003, still the safest choice for compatibility, and hardware-accelerated on every phone, computer and TV made in the last fifteen years.

H.264 won so thoroughly that "will this play everywhere" and "is this H.264" are nearly the same question. It is not the most efficient codec available — HEVC and AV1 both do more with fewer bits — but nothing else comes close on breadth of support.

It has profiles, and one of them causes trouble on Apple devices: High 10, often written Hi10P, encodes ten bits per colour channel instead of eight. Hardware decoders are eight-bit only, so a Hi10P file plays sound with no picture on a device that has no software fallback.

HEVC

H.265

H.264's successor, roughly half the size for the same quality. Every Apple device from 2015 onward decodes it in hardware, and it is what an iPhone records in.

HEVC is what made 4K practical on phones: the same picture at roughly half the bitrate. Apple adopted it early and thoroughly, so on Apple hardware it is as safe a choice as H.264.

Elsewhere it is messier. The patent licensing around HEVC was contested enough that web browsers were slow to support it, which is precisely the gap AV1 was created to fill.

AV1

A royalty-free codec published in 2018, more efficient than HEVC and, unlike it, free of licensing disputes. Hardware decoding only arrived on Apple devices with the iPhone 15 Pro.

AV1 exists because HEVC's licensing made the largest streaming and web browser companies want an alternative they could ship without paying anyone. It is now what a great deal of web video is delivered in.

The catch is decoding cost. Without hardware support, AV1 is decoded in software, which is possible but drains a battery visibly and can stutter on high resolutions. On older devices, an H.264 version of the same video is the better file.

Bitrate

How much data one second of video or audio uses, usually in megabits per second. It is the main thing that decides both file size and how much a device has to work.

Resolution gets the attention, but bitrate decides how a file actually looks. A 4K file at a low bitrate looks worse than a 1080p file at a high one, because the encoder had to discard more to reach its target.

It is also what a stuttering file is usually about. When a video plays on a computer and stutters on a phone, nothing is wrong with the file: it is simply asking for more decoding work per second than that chip wants to do.

Remux

rewrap

Moving the existing video and audio streams from one container into another without re-encoding them. Lossless, and fast enough to be limited by disk speed rather than by the processor.

A remux copies streams; it does not touch a single frame. An hour-long MKV holding H.264 and AAC becomes an MP4 in seconds, bit-for-bit identical in picture and sound.

It only works when the target container accepts what the source holds — MP4 will take H.264 and AAC, and will not take VP9 or Vorbis. When people say a conversion was "instant", they remuxed. When it took an hour, something was re-encoded.

Transcode

re-encode, convert

Decoding a stream and encoding it again, usually into a different codec or at a lower bitrate. Slow, and lossy every time — quality never comes back.

Transcoding is what people mean by "converting a video", and it is a genuinely expensive operation: every frame is decoded, processed and compressed again. It is the right answer when the codec truly is unsupported, or when a file must be made smaller.

It is the wrong answer to "my phone will not open this", when a remux would have done, and it is a poor answer to a file whose only problem is a DTS audio track — replacing one track leaves the picture untouched.

Hardware decoding

Decoding video on a dedicated chip rather than the main processor. Far faster, far cooler and far kinder to a battery — but it only handles the codecs it was built for.

Every phone contains a fixed-function video decoder that does H.264 and HEVC extremely efficiently. Anything it does not support falls to software, which works but costs battery and heat, and may not keep up at high resolutions.

This is why format support on a phone is not a single yes or no. A device may play a codec perfectly at 1080p in software and fail the same codec at 4K, and neither result says anything about the file being wrong.

10-bit

Hi10P, High 10

Video storing ten bits per colour channel instead of eight, for smoother gradients. In HEVC it is normal and well supported; in H.264 it is the single most common cause of a file that has sound but no picture on an iPhone.

More bits per channel means finer steps between shades, which removes the banding you see in dark skies and slow fades. For HDR it is a requirement, not a refinement.

The problem is H.264 specifically. Ten-bit H.264 is outside what hardware decoders implement, and Apple's system player has no software path for it, so the audio plays and the picture never arrives. Anime releases use it heavily, which is why this comes up far more often than the format's overall share would suggest.

Keyframe

I-frame

A frame stored in full rather than as a difference from the frame before it. Seeking can only land on one, which is why a video sometimes jumps to a slightly different moment than you asked for.

Video compression works by describing most frames as changes from their neighbours. Only keyframes stand alone, and they typically arrive every two to ten seconds.

When you drag a scrub bar, a player finds the nearest keyframe and starts there. Widely spaced keyframes make a file smaller and seeking coarser; that trade-off is decided at encoding time and cannot be changed afterwards without re-encoding.

The other subjects

Streaming and IPTV Files, downloads and transfer Privacy and encryption Every term