Blog

Revenge of the Codecs Part 2 – The War for 4K Streaming Profits

One codec to rule them all…

For many years, the happy Hobbits of Netflixshire, and adjacent suburbs of Hulu, HBO, and Amazon, have been serving massive dollops of delicious HD content to all visitors. The cooks (content creators) and the Shire are paid handsomely, but the Halflings of Codec receive only a small delivery fee. 

However, darkness has shadowed Middle-Earth as all eye the rise of Profits, which are far more Precious than ever before. Large shires are breaking apart as the Apple-ings, Lords of the Mouse, Servants of the CBS Eye and others set up their own content Shires. Codec Halflings, seeing 4K as their last chance to enrich their coffers, are demanding increased tariffs on 4K streams, thus beginning the War of the 4K – One Codec to Rule Them All.

With apologies to J. R. R. Tolkien, this parable summarizes the state of consumer streaming. There are no Orcs – just companies pursuing competing profit goals.

Years ago, when Netflix was in its infancy, the MPEG 4/AVC H.264 codec was developed for HD video. The patent pool, MPEG LA, set up a clearly defined royalty process, costs and cap for annual usage, which amounts to pocket change for large-scale streaming suppliers like Netflix today. Even so, Chrome and Firefox browsers don’t support H.264 (or HEVC), saving costs and advancing Google’s own VP9/AV1 codecs.

When HEVC (Highly Efficient Video Codec) H.265 for 4K streaming was developed, everyone got greedy, and things got messy. Now there are three independent patent pools and many others not in a pool, all insisting on a piece of the profits. As a result, HEVC adoption has been problematic, as companies can’t predict usage and costs.

What’s even more confusing is that HEVC is really a temporary codec, to be replaced by VVC (Versatile Video Codec) H.266 in a year or so. Turns out HEVC isn’t as Highly Efficient as planned. VVC is designed to be 30-50% more efficient, and promises to provide a better patent process – though experts expect the same patent mess as HEVC.

The streaming giants, such as Netflix, Amazon, Facebook, Intel, Google, Apple, Microsoft and more, formed the Alliance for Open Media to create a new 4K codec based on Google’s VP9, called AV1. It’s royalty-free, and the Alliance has the resources to fight off claims from patent pools. There’s one set up already, but yeah, good luck with that. 

Consumers won’t sense the battle in the background – 4K will still look like 4K. The big winners will be the streaming vendors, as they can encode all the streams using one codec. Generally, Web videos are typically VP9/AV1, and device streaming tends to be HEVC. Not that people are watching much 4K anyhow – more than 90% of streaming content is HD H.264. That will change over time. It’s interesting that Samsung is developing AI-assisted 8K upscaling as they bring out more 8K TVs.  Looks like they expect content to be a mix of HD and 4K for the foreseeable future.

Who wins? Easy question, as anyone who recalls the Microsoft Explorer/Netscape Wars knows – free always wins.

Revenge of the Codecs Part 1 – JPEG XS

The standard vehicle for IP video switching, Motion JPEG 2000, is a new technology to many AV integrators but is quite old, dating back to 2001. The codec, initially designed for compressing video for archiving, found a broader application today for IP video switching and distribution.

Its architecture is perfect for the application, as the stream is a continuous series of JPEG images, it’s easy to switch between frames. With 20:1 compression, a 4K 60 stream can be carried over a standard 1G network. The advanced intoPIX codec, used by Dante AV and Crestron, can process video with a low latency of 10 milliseconds. And to top it off, MJP2000 is royalty-free. What could be better?

Well, there’s a wish list of new features from 4K/8K video producers and broadcasters. They want uncompressed performance with the bandwidth savings of compression, largely so they can use their existing 1G IP and 3G SDI copper infrastructure. A codec designed for 4K/8K video from the ground up, and a lighter approach to compression with less demanding processing than the old MJP2 engine. Most importantly, latency in the microsecond range. And royalty-free (there is a nominal cost from the codec vendor, but very low compared to Dolby or MPEG). 

Welcome to JPEG XS

That’s the specs for the new JPEG XS, which stands for eXtra Small (bandwidth, processing and name). The codec will be finalized this year, available next year for CPU, FPGA and ASIC processors. 

As most AV IP Switching codecs run on an FPGA processor, vendors could switch to the new codec with an update, gaining uncompressed level video quality and microsecond latency. intoPIX is one provider, so this could be an easy step for Dante and Crestron. This is a very competitive space, and the JPEG XS can separate the winners from the honorable mentions.

In the long term, equipment costs may reduce, as JPEG XS can run on lower-cost processors. 

So keep an eye out – this transition could happen as fast as, well, JPEG XS.

Audinate Drops the InfoComm Bomb – Dante Now Routes 4K Video as well as Audio

A small one-paragraph blurb in the InfoComm 2018 Show Daily is the biggest story of the show – Audinate announces Dante AV. A perfectly timed article, “Making the Case for AV-0ver-IP,” in the June 2019 Sound & Communication magazine, written by Audinate’s Brad Price, reveals a bigger picture. 

The short story is this – Dante AV accepts up to 4K60 4:4:4 10-bit video and 8 audio tracks, routing AV streams over 1G IP as easily as Dante audio, all in sync with a master clock. Audinate makes the new Dante AV interface that’s inside, and other vendors add Motion JPEG 2000 encoding/decoding as well as pass-through RS-232 and IR. Network routing, discovery, master clock, and control is handled by Dante, with assured interoperability between suppliers’ transceivers. 

The bigger vision for the new innovation is presented with clarity in the article, written before Dante AV was revealed at the show. AV-over-IP has great potential, but as in the early days of network audio, manufacturers have created a mish-mosh of choice, an irregular, branded mix of codecs, networking schemes, control commands and proprietary features. Pixel Perfect vs PURE3? What does that even mean?

For the AV-over-IP to grow, a consistent standard is required. To meet the needs of both the AV and IP communities,  a standard must be set that answers requirements for security, scalability, interoperability, performance, and market adoption. An ecosystem made of many partners offers more options than domination by one brand, or niches owned by a few. 

For many reasons, AV-via-Dante can be a strong foundation for that ecosystem. After all, they did emerge as the defacto standard for IP audio – why not AV? 

Crestron is the market leader for many good reasons, but Dante-friendly Q-Sys programming is quickly gaining ground. Other major players will arise shortly – Yamaha is already an Audinate AV partner, with products in development. As Crestron plays well with other standards, they could incorporate Dante AV as well. 

However all this plays out, Dante AV should open the door to new innovation in AV-over-IP!

“The Road to ATSC 3.0” Needs…..a Road.

The NAB Show is over, and another round of ATSC 3.0 glad tidings and joy begins. Broadcasters are eager packing for their trip to 3.0Land, vendors are gleefully announcing their new 3.0Gear, and TV stations are excited about future 3.0 opportunities because we’re on “The Road to ATSC 3.0”. All that’s missing to make the journey is….the road – and the motivation to ride.

Ten years ago, the U.S. government created a path to transition the country from analog TV to digital by allowing stations to broadcast HD from a second channel. Despite all the announcements and adapter coupons, it took five years longer than planned to complete. Even then, TV stations were barraged with thousands of phone calls asking, “Where is my TV?”

Closing the Road

The motivation behind that transition was the Deficit Reduction Act, which reaped billions by selling off TV channels 52-69 to cell phone vendors. But the FCC didn’t stop there -in 2018 auctioning off another band of spectrum, earning about $22 Billion, and packing TV broadcasters into channels 2-36.

Do the math, There are 30 TV channels in Dallas, soon to be packed into 36, leaving only 6 channels left for alternate ATSC 3.0 broadcasts. There’s talk of sharing several stations into several ATSC 1.0 channels for those who can’t or won’t switch, but 30 can’t be squeezed into 6.

In contrast, the first country to standardize on ATSC 3.0. Sound Korea has set aside the 700 MHz band for the new channels – the same band the FCC just took away from broadcasters. So, no, there isn’t a physical road to ATSC 3.0 in the U.S.

Finding Motivation

The road to ATSC 3.0 also requires motivation to make the journey. ATSC 3.0 is great technology, better than anyone could ask. The problem is, consumers aren’t asking; they don’t understand the question, and they didn’t ask for the last Digital Transition. Lacking a market trend, the FCC mandated the first transition, but they’re not doing that today.

The FCC is saying, “OK, we have an ATSC 3.0 standard, but the rest is on you, broadcasters. We won’t give you the spectrum for an orderly transition. We won’t fund ATSC 3.0 adapters. We won’t pay for new broadcasting equipment. We won’t mandate the shift, ATSC 1.0 remains the only national standard. We won’t force TV vendors to include ATSC 3.0 tuners. But go ahead, make it happen if you can.”

Take a DVB Chill Pill

In contrast, the rest of the world, who uses the DVB-T HD standard, has taken a more intelligent path. Countries have been slowly adding 4K DVB-T2 channels since 2010. Like the U.S. countries have sold spectrum for cell use – but kept space open for 4K expansion. Enough 4K channels are available that many TVs now include DVB-T and DVB-T2 tuners. And they’ve been smart enough to keep Dolby’s greedy hands off the audio tracks.

The question for me is, as ATSC 3.0 is based on the DVB-T broadcast technology (ATSC 3.0 is not based on ATSC’s 8-VSB) – why didn’t we open the door to DVB-T2 10 years ago and join the global community?

Nah, won’t happen, we’re ‘Mericans. Maybe the way to go is adopt a European-style mindset for transition. Roll out ATSC 3.0 and see what happens.

Peace out, I’m gonna watch some 4K on my $40 Fire Stick…..

What is Motion JPEG 2000?

You’ve seen the technology in use with many AV over IP video systems, but what exactly is Motion JPEG 2000?

We’re familiar with JPEG images, found all over on the Web and on our phones and tablets. JPEG was created in 1992 by the Joint Photographic Experts Group, who followed up with JPEG 2000 in Y2K. As a new standard, it was glorious. Also known as JP2, advanced wavelet encoding could save images in a lossless or lossy version. While JP2 offered improved performance, it never gained acceptance – the difference wasn’t enough to challenge the popularity of JPEG.



ve seen the technology in use for many AV over IP video systems, but what exactly is Motion JPEG 2000?

You’ve seen JPEG images, found all over on the Web and on your cell phone. JPEG was created in 1992 by the Joint Photographic Experts Group, who followed up with JPEG 2000 in Y2K. As a new standard, it was glorious. Also known as JP2, advanced wavelet encoding can save the images in a lossless or lossy version. While JP2 offered improved performance, it never gained acceptance – the difference wasn’t enough to challenge the popularity of JPEG.

Going Into Motion

Acceptance was much improved when the group upgraded the legacy Motion JPEG to use JP2 technology. A Motion JPEG 2000 video stream is composed of a series of JP2 images at 24, 30, or 60 frames per second. The first adopters were archivists like the Smithsonian, who discovered they could compress videos at a 3:1 ratio without losing original quality. Video editors preferred MJP2 over MPEG, because it was easy to cut between individual frames. When you go to a digital movie theater, you’re watching a high-end version of MJP2.

Adoption for network streaming was a different story, as MJP2 streams are massive, and MPEG H.264 performs better for highly compressed streams. However, with the advent of dedicated AV 1G/10G networks, MJP2 has found a new life for AV over IP, as:

  • Video can switch between frames, assuring zero latency
  • Compression shrinks bandwidth to fit the network – 3:1 compression for 10G, 20:1 compression for 1G networks
  • The codec is free, with zero royalties

Motion JPEG 2000 by Any Other Name

Some products will list MJP2 by name, others may use the codec under a brand name, and some use a variant of MJP2. I suspect branded codecs are really Motion JPEG 2000 – what vendor would invest a million or so for a new codec that works the same as the free codec?

  • Crestron NVX Pixel Perfect – MJP2 with tweaked settings
  • Extron PURE3 – it’s not a patent, but a trademark for sending compressed streams (likely MJP2), audio and control through a network
  • SVDoE – “Proprietary” but compression ratios like MJP2
  • Atlona VC-2 – Britain’s BBC’s Dirac codec is similar to MPEG but uses JP2 instead of JPEG for the I-frame in the GOP. Dirac Pro (SMPTE VP-2), designed for broadcast production, can handle 4-8K video and encode video with only I-frames, essentially the same as MJP2.

1GB Versus 10GB – Is There Really a Debate?

An article in AV Technology Feb/March 2019, “1Gb Versus 10Gb” states that the market is migrating from central matrix switching to employing generic Ethernet switches while desiring the same values of HDBaseT – uncompressed video and zero-latency switching. The author felt the SDVoE should be compared more with HDBaseT – and that’s not wrong. Both are IP solutions, with one using an internal 10G switch, while the other uses an external switch. But the story never really delves into the 1G side of the debate.

It’s ironic that the same publication features a full-page Crestron NVX ad that states,”YES. The 1GB VS 10GB Debate is Over.” Obviously, there is some debate, especially between the two statements that are essentially ads.

Let’s face it – all 4K-capable switching systems are using compression – it’s the only way to fit the signal into the pipe. If the audience can’t see the difference between 3:1 compressed (10G) and 20:1 compressed (1G) 4K video – then there is no debate. Well, of course, there is debate – one would expect quality and latency differences between 3:1, 20:1, or 8:1 compression . The importance of those differences depends on the application.

Crestron believes there is a place for both 1G and 10G switching, and is the current leader of the pack. There are several key reasons for this:

  • On the 10G side, new Crestron DM systems (and other 18G HDBaseT systems) feature VESA DSC compression for 4K60 4:4:4 video, while lower resolutions are uncompressed – a more common-sense way to keep HDBaseT values intact.
  • On the 1G side of the switch, Crestron’s NVX compresses all streams by 20:1, which will pass all variations of 8-bit and 10-bit 4K. As with most other IP Switch systems, NVX employs Motion JPEG 2000 (MJP2).
  • On the 1G side of the switch, Crestron’s NVX compresses all streams by 20:1, which will pass all variations of 8-bit and 10-bit 4K. As with most other IP Switch systems, NVX employs Motion JPEG 2000 (MJP2).
  • 1G applications are popular as existing Category wiring is likely 5e/6 UTP, so 1G switching is the only way to go without running all new cable (10G can be run over Cat 5e/6, but over a short distance) or fiber.
  • NVX is more than video, it’s designed as part of the full Crestron ecosystem, including control, audio, XiO Cloud, and Air Media.

But That’s not the Only 1G Option

Aurora Multimedia’ also supports both 10G and 1G IP switching solutions, both based on the same SDVoE technology. The VLX system is an interesting contender for 1G Cat 5e/6 video distribution. Its design philosophy rests on the reality that commercial video systems will never employ entertainment-level 4K content. By limiting resolution to 4K60 4:2:0/4K30 4:4:4 resolution, the system can offer SDVoE signal quality, a host of key integration features, and simplified 1G networking – all at a low transceiver cost. Features include:

  • Low 8:1 compression, 1.5 frame latency
  • Compact transceiver boxes and wall panels
  • 1G POE networking (9W)
  • 2-way IR and 1-way RS-232 control (An internal channel list can be created for simple IR wireless control)
  • Host and device USB 1.1/ 2.0 ports
  • Optional Dante audio routing

Aurora Multimedia and Crestron are only two of many providers of 1G IP Switching systems. I’ve mentioned the NVX and VLX systems as they represent two different approaches to the same application.

Is the Market Transitioning?

In terms of competitor’s hopes to gain a piece of Crestron’s market share – not so much. Crestron’s reputation, relationship and integration is a high hurdle to jump over. Talking with integrators, many systems are self-contained switching applications that work perfectly well with DM HDBaseT. If the IP switch is in the same rack as the video sources – it’s the same thing, anyhow.

Not that there aren’t good options available from others. AMX SVSI and Extron NAV enjoy solid control ecosystems as well. Kramer has a 1G system driven by cloud-sourced control. WyreStorm offers a surprising array of MPEG, JPEG 2000, 18G HDBaseT, and SVDoE switching systems.

Video in Transition- Hauling a 4K load Over a 1080p Bridge

New technology brings new challenges and solutions in AV design

The transition from 3:4 analog to 16:9 digital HD was reasonably painless. For meeting rooms and living rooms, HD screens had had little impact on room design – they were about as tall as 3:4 screens, with more horizontal real estate.

Converting analog video to HD was a greater challenge for twisted-pair video distribution. “Video Voodoo” problems arose as the precise kind of wire twist affected distance, resolution and performance. Those problems faded away with the introduction of HDBaseT, a chipset that converted digital video into packets. Think of HDBaseT UTP cable as an 8G bridge, easily transporting 3G 1080p video, audio, control data and Ethernet for up to 300 feet.

Enter 4K, an easy step for TV vendors, not so easy for twisted-pair video. That 8G bridge now has to carry a 10-16G load, more with 4:4:4 color and 10-bit HDR. Not happening, so something has to change – lighten the load, strengthen the bridge, or a bit of both.

Building the Bridge with PAM

PAM stands for Pulse-Amplitude Modulation, that expresses data using amplitude in a series of signals. PAM-5, used by 1G Ethernet uses values of -2, -1,0,+1 and +2 VDC to represent bits of data. Each of the four twisted pairs carry up to 125 mbps, summing up to 1G of data. 10G Ethernet, both HDBaseT and IP Switching, employs PAM-16, encoding 16 levels with added physics. The limiting factor in PAM technology is RF noise, as it prevents the receiver from sensing the right levels. This is a key factor in 10G Ethernet, as the level steps are much smaller. For this reason, shielded cable should be used for all 10G applications. 2.5G/5G Ethernet uses a lighter version of PAM-16 designed for Cat 5e/6 UTP applications. Of course, optical fiber is always the best carrier, as there is no RF noise to interfere with the signal.

It’s interesting to note that AV platforms we view as different – AV over IP, QAM digital cable, 8-VSB off-air digital channels, and HDBaseT – are all based on variations of PAM technology.

Compression – Lightening the Load

Full 18G 4K60 HDR video can’t travel over a 10G bridge, so the stream has to be compressed to fit. There are two technologies in use today:

  • VESA Display Stream Compression (DSC) in newer HDBaseT systems, creates a visually lossless stream, typically at a 3:1 ratio, reducing an 18G 4K60 HDR stream to 6G with little loss in quality. Examples include Crestron DM 4KZ and WyreStorm 18G HDBaseT systems.
  • Motion JPEG 2000 was designed for video storage, reducing files by 3:1 without losing original quality. With today’s higher bandwidth, the codec has a new application in AV over IP technology:
    • 10G IP Switching. Employs 3:1 compression to deliver mathematically lossless video. In use today in SVDoE and Extron systems.
    • 1G IP Switching. Employs 20:1 compression to deliver visually lossless streams over 1G Ethernet networks. Available from Crestron, WyreStorm, Extron, Kramer and many others.

VESA DSC has a key advantage – compression only affects streams starting at 4K60 4:4:4, other streams are uncompressed.

The good news is there are solutions in place for the challenges in 4K video distribution.

  • Sites with existing Cat 5e/6 UTP wiring can transport 4K video using 1G IP switching gear, and companies like WyreStorm and others have HDBaseT extenders that can transport 4K video over Cat 6, with some resolution and distance limitations.
  • New installations have many options for 4K distribution over Cat 6a/7a STP cable or fiber.