Amara GrahamI had a few people reach out and ask for more information about lossy and lossless compression, particularly related to videos and video codecs.
I'll admit, I've run into this topic before when working with audio files for Speech to Text (Opus, in a webm container if you are capturing from the browser), but I learned exactly the bare minimum to get results. I'm probably just a set above bare minimum with video at this point.
This won't be your most comprehensive look into video compression as there are other resources that probably do that much better.
Let's start with the basics.
Compression 101
One of the most common ways you encounter compression is when packaging your enormous powerpoint presentation into a zip file.
Through voodoo computer magic your presentation goes from huge to almost manageable and you can send it as an email attachment. The person on the receiving end is able to download the zip, uncompress it (or unzip it), and they get your presentation in all the enormously huge goodness.
This is your brain.
This is your brain on compression.
Kidding but not really. We got rid of the redundant info, the spaces between the vehicles. Whatever, it's a metaphor. Make your file smaller and easier to send across the network with the information you really want to convey.
Lossless Compression
That voodoo computer magic I mentioned before I went on my quest for a pictorial metaphor is really a set of algorithms, aka encoding methods or codecs, that identify places where you can use reference instead of repetition. Remember back to your 'pass by value' vs. 'pass by reference' day if you went through formal computer science education. References are cheaper because we can always refer to that location for the same value, even though you might have the same graph on 3 slides because you are trying to prove a point.
This methodology keeps all the original information in the file and is consider lossless compression - we lost no information during compression.
Languages and written text are easy to compress because they are filled with patterns. Although American English is a mess, I promise it still has patterns.
Lossy Compression
The problem is, not everything is text based, and our powerpoint example was maybe not the best, mainly because it was probably filled with images, gifs, and maybe other non-text items.
Images, audio files, video- they all have significantly less patterns. So our algorithms instead have to make some choices, assumptions, getting rid of similar or extraneous information.
This results in the file being changed, potentially losing information in the process. You literally threw information away, why would you expect to get it back?
Now, theoretically, our algorithms can be really good at compressing a particularly kind of file like an image, but you may see an issue with color as the algorithm chose a single value for all your green grass or all your blue sky.
We want a smaller file size, but we also want to make sure we don't degrade the experience in a noticeable way.
But I Want High Quality Streaming Video
Of course you do, which is why you are going to go with lossy compression.
We need a lossy codec that picks the right stuff to throw away and not degrade your experience. This is hard, and when something is hard for a computer it can be slow.
Encoding or decoding, literally the work your codec is doing, is the slowest part of the video process and you'll need to make decisions about what you choose to tradeoff during this process. Lower bitrate codecs may have more distortion, which might be ok for you.
More Resources
In case you want to learn more about video codecs, compression, and encoding I've included some links below. Just remember a codec is software or hardware that does the encoding or decoding. There are two kinds of compression codecs - lossy and lossless.
Lists, if you are into that:
- https://en.wikipedia.org/wiki/Comparison_of_video_codecs
- https://en.wikipedia.org/wiki/List_of_open-source_codecs
Overviews, maybe even with graphics:
