Google nuke thyself: Mountain View's H.264 righteous flame-out
How MPEG LA tamed a giant - and will again
Posted in Developer, 31st May 2013 09:05 GMT
Back in 2010, champions of a free web were ecstatic over Google's plan to seed the internet with a patent-free video. VP8 was going to crush the patent-heavy H.264, now celebrating its 10th birthday [1]. Or so we were told.
In May of 2010, Google open-sourced [2] VP8, the video compression codec component to the audio-visual WebM format, which it had bought and developed earlier that year, and threw open the WebM Project to all comers.
Mozilla and Opera – who’d kept H.264 out of their browsers - were very much onboard, saying their browsers would play VP8 while AMD, ARM, NVIDIA, and Qualcomm joined the WebM Project [3] working with others to build and patch code delivering hardware acceleration for the codec. Journalists were euphoric, producing articles like this [4]: Google opens VP8 codec, aims to nuke H.264 with WebM.
But three years later, the party’s over – and it's Google and VP8 that were nuked. Mozilla – maker of the web’s third biggest browser – has swallowed H.264 in Firefox and Mountain View in March signed up to a licence for technologies in H.264.
Google is also a $50bn trans-national machine run so smoothly that its tax affairs seemingly operate on a level beyond the grasp of ordinary governments. The company has its foot on the nexus of our web and our culture.
What thwarted Google was MPEG LA [5], a group that’s likely little known by the average square-eyed netizen. MPEG LA administers more than 5,500 licensees spanning families and generations of media patents. The group was created by a cluster of highly motivated technology companies - all MPEG-2 patent-holders - in 1997 to outsource the charging and enforcing of their patents.
Most ordinary punters won’t have heard of H.264 but they’ll have certainly experienced it. H.264 is a block-oriented motion compression technology used to play video offline and online. It's used on services from YouTube to Apple’s iTunes store - anything played by Adobe’s Flash Player uses H.264 - so that means most film, TV, porn and videos of cats playing the piano on the web. It’s the video compression technology used in Blu-ray players, too. H.264 was built using patented technology from Microsoft, Apple, the Fraunhofer Institute and others.
H.264 has taken over the web quickly: drafted in May 2003 there are today 1,600 licensees versus 2,000 for MPEG-2, which dates back to 1996.
Frank Sinton, chief executive of video indexing and search site MeFeedia, tells us succinctly that WebM has been, er, crushed. “H.264 has kind of won,” Sinton says.
MeFeedia in 2011 surveyed [6] the millions of video feeds it indexes and serves and found H.264 played 80 per cent of HTML5 video, while WebM accounts for 2 per cent. MeFeedia didn’t bother researching the subject again. “That’s the reason we haven’t updated the numbers [since 2011] - the [H.264] codec [covers] so many browsers and devices.”
It was MPEG LA that helped ensure H.264's continued dominance and that started the grenade rolling under Google's feet, with an action that meant Google's VP8 web takeover was in trouble from the very start.
MPEG LA patent pools collectively represent companies who believe their patents are being used in various technologies. An H.264 pool would have been used by MPEG LA to enforce its group members' patents. By February the following year, it had called on patent-holders to submit evidence of patents they believed were used in Google’s codec.
The pressure seemed to pay off and - as is so often the case in the world of patents and alleged infringements - it was the behind-the-scenes maneouvring ahead of any court action that paid, and saw Google sign a licensing deal.
Under the agreement, Google agreed to license “techniques that may be essential” to VP8 while MPEG LA agreed to call off the dogs, saying it’ll discontinue efforts to form the threatened VP8 patent pool.
Was it saber-rattling that pulled Google to heel?
MPEG LA’s Horn says: "No". “Patent pools get too much of the blame and too much of the credit,” he told The Reg on the anniversary of H.264.
“When I was asked: ‘Is VP8 patent-free?’ I said: ‘I doubt it because the tools that go into it are well known to be covered by patents.’ That got things fired up in the market. I didn’t do that as a threat - it might have been perceived that way. My point is if there’s a lot of IP rights in the market, then MPEG LA has an opportunity and a service to perform [by] forming a pool. The market may have taken that as a shot across the bow that MPEG LA is threatening us, but I would say it’s not true.”
The exec refused to discuss what formal talks took place with Google. Yet Horn concedes the threat did bring Google to the table. We also asked Google for its side but the company declined to comment.
Google had tried to sidestep the matter by parking its codec [8] under a BSD licence. It was a unilateral decision, and one that clearly failed to satisfy MPEG LA.
According to Horn, it is MPEG LA - rather than individual patent holders like Apple or Microsoft - that takes the lead in forming patent pools. So it would have been MPEG LA that acted in this case against Google, without prompting. “We don’t discuss [pending action] with owners. When we form a pool, MPEG LA takes a decision based on feedback from the market,” Horn says.
Underlining the point: Steve Jobs in 2010 said a patent pool was going after Theora, the free lossy video compression format. “All video codecs are covered by patents. A patent pool is being assembled to go after Theora and other 'open source' codecs now,” Jobs wrote [9]. No MPEG LA patent pool was formed.
“Maybe I delude myself,” Horn continues. “But I don’t think we are under anybody’s influence around here - whether it’s our owners or patents-holders… We believe you have to be principled in the administration of these programs.
"Just because you are a patent-holder here doesn’t mean you get a free pass over here because you don’t take a licence. I define neutrality as having our own self-interest. Obviously, we are a business and we like to make a profit. The way we like to do that is providing a service to the market that people are willing to pay for.”
MPEG LA is owned by 10 companies including Cisco, Sony, General Electric and Google, thanks to its purchase of the phone business owned by Motorola. These owners are not necessarily patent-holders in MPEG LA’s pools and MPEG LA is not a standards body, so it does not create specifications.
The group was formed in 1997 by eight companies that wanted to turn the then-new MPEG 2 standard into a viable business by charging others to use it. The original line-up included Fujitsu, Panasonic, Sony and Philips.
MPEG LA doesn't create technology standards like MPEG 2 or H.264, but it does charge for them. MPEG 2 was the work of the ISO/IEC Moving Picture Experts Group (MPEG) and H.264 came from the ITU's Video Coding Experts Group and MPEG. The difference is the “LA” part - standing for Licensing Authority.
Now the ITU and MPEG group that built H.264, the Joint Collaborative Team on Video Coding, is cooking up H.265 - or High Efficiency Video Coding (HEVC). This idea is H.265 will improve video picture quality without demanding fatter bitrates from the devices' networks. This should make it better suited for use on tiny-footprint devices like smartphones, and tablets.
H.265 will improve coding efficiency to reduce the bitrate requirement – the goal is for a 50 per cent compression gain over H.264.
Like H.264, it’ll be MPEG LA that's the money-making end of the business for the tech companies whose patents get baked into H.265. “We are facilitating a pool for H.265,” according to Horn, who said the plan was for meetings in April and May with a licence to be issued this year.
If history is a guide, H.265 should be propelled to a rapid takeover. H.264's ascendancy was assisted by the fact that Apple and Microsoft, two of the most influential companies in business and consumer tech, were involved.
Apple and Microsoft not only helped build H.264, they made it ubiquitous through their browsers, servers, operating systems, tablets and phones. MPEG LA enforced the hegemony through patent licensing, seeing off potential rivals like VP8 from Google.
Horn won’t be drawn on why H.264 has taken over or MPEG LA's role: “These are decisions made by the marketplace based on numerous factors important to individual users/adopters,” he says. “We are agnostic on those choices.”
But with the muscle of MPEG LA in its corner, H.265 is likely to keep its rivals - tech giants or no - in line. ®
In May of 2010, Google open-sourced [2] VP8, the video compression codec component to the audio-visual WebM format, which it had bought and developed earlier that year, and threw open the WebM Project to all comers.
Mozilla and Opera – who’d kept H.264 out of their browsers - were very much onboard, saying their browsers would play VP8 while AMD, ARM, NVIDIA, and Qualcomm joined the WebM Project [3] working with others to build and patch code delivering hardware acceleration for the codec. Journalists were euphoric, producing articles like this [4]: Google opens VP8 codec, aims to nuke H.264 with WebM.
But three years later, the party’s over – and it's Google and VP8 that were nuked. Mozilla – maker of the web’s third biggest browser – has swallowed H.264 in Firefox and Mountain View in March signed up to a licence for technologies in H.264.
Wait, YouTube's owner couldn't make it work?
On paper Google could – and should – have succeeded. It owns YouTube, a video service with so may users that Google reckons that, averaged out, every single person on the planet has watched 140 video views. It therefore had a critical mass of web video. Android runs most smartphones on the planet, meaning Google has the means to deliver royalty-free WebM video.Google is also a $50bn trans-national machine run so smoothly that its tax affairs seemingly operate on a level beyond the grasp of ordinary governments. The company has its foot on the nexus of our web and our culture.
What thwarted Google was MPEG LA [5], a group that’s likely little known by the average square-eyed netizen. MPEG LA administers more than 5,500 licensees spanning families and generations of media patents. The group was created by a cluster of highly motivated technology companies - all MPEG-2 patent-holders - in 1997 to outsource the charging and enforcing of their patents.
Most ordinary punters won’t have heard of H.264 but they’ll have certainly experienced it. H.264 is a block-oriented motion compression technology used to play video offline and online. It's used on services from YouTube to Apple’s iTunes store - anything played by Adobe’s Flash Player uses H.264 - so that means most film, TV, porn and videos of cats playing the piano on the web. It’s the video compression technology used in Blu-ray players, too. H.264 was built using patented technology from Microsoft, Apple, the Fraunhofer Institute and others.
H.264 has taken over the web quickly: drafted in May 2003 there are today 1,600 licensees versus 2,000 for MPEG-2, which dates back to 1996.
Frank Sinton, chief executive of video indexing and search site MeFeedia, tells us succinctly that WebM has been, er, crushed. “H.264 has kind of won,” Sinton says.
MeFeedia in 2011 surveyed [6] the millions of video feeds it indexes and serves and found H.264 played 80 per cent of HTML5 video, while WebM accounts for 2 per cent. MeFeedia didn’t bother researching the subject again. “That’s the reason we haven’t updated the numbers [since 2011] - the [H.264] codec [covers] so many browsers and devices.”
It was MPEG LA that helped ensure H.264's continued dominance and that started the grenade rolling under Google's feet, with an action that meant Google's VP8 web takeover was in trouble from the very start.
Google grabs On2 the next thing
In May 2010, MPEG LA’s chief executive Larry Horn said [7] his group was looking into forming a patent pool against VP8 - this was after Google had bought On2 Technologies whose technologies became VP8.MPEG LA patent pools collectively represent companies who believe their patents are being used in various technologies. An H.264 pool would have been used by MPEG LA to enforce its group members' patents. By February the following year, it had called on patent-holders to submit evidence of patents they believed were used in Google’s codec.
The pressure seemed to pay off and - as is so often the case in the world of patents and alleged infringements - it was the behind-the-scenes maneouvring ahead of any court action that paid, and saw Google sign a licensing deal.
Under the agreement, Google agreed to license “techniques that may be essential” to VP8 while MPEG LA agreed to call off the dogs, saying it’ll discontinue efforts to form the threatened VP8 patent pool.
Was it saber-rattling that pulled Google to heel?
MPEG LA’s Horn says: "No". “Patent pools get too much of the blame and too much of the credit,” he told The Reg on the anniversary of H.264.
“When I was asked: ‘Is VP8 patent-free?’ I said: ‘I doubt it because the tools that go into it are well known to be covered by patents.’ That got things fired up in the market. I didn’t do that as a threat - it might have been perceived that way. My point is if there’s a lot of IP rights in the market, then MPEG LA has an opportunity and a service to perform [by] forming a pool. The market may have taken that as a shot across the bow that MPEG LA is threatening us, but I would say it’s not true.”
'Maybe I delude myself... but I don’t think we are under anybody’s influence'
“We put out a call for patents [on VP8],” said MPEG LA's Horn. "That initiated a discussion with Google about delivering licences to them."The exec refused to discuss what formal talks took place with Google. Yet Horn concedes the threat did bring Google to the table. We also asked Google for its side but the company declined to comment.
Google had tried to sidestep the matter by parking its codec [8] under a BSD licence. It was a unilateral decision, and one that clearly failed to satisfy MPEG LA.
According to Horn, it is MPEG LA - rather than individual patent holders like Apple or Microsoft - that takes the lead in forming patent pools. So it would have been MPEG LA that acted in this case against Google, without prompting. “We don’t discuss [pending action] with owners. When we form a pool, MPEG LA takes a decision based on feedback from the market,” Horn says.
Underlining the point: Steve Jobs in 2010 said a patent pool was going after Theora, the free lossy video compression format. “All video codecs are covered by patents. A patent pool is being assembled to go after Theora and other 'open source' codecs now,” Jobs wrote [9]. No MPEG LA patent pool was formed.
“Maybe I delude myself,” Horn continues. “But I don’t think we are under anybody’s influence around here - whether it’s our owners or patents-holders… We believe you have to be principled in the administration of these programs.
"Just because you are a patent-holder here doesn’t mean you get a free pass over here because you don’t take a licence. I define neutrality as having our own self-interest. Obviously, we are a business and we like to make a profit. The way we like to do that is providing a service to the market that people are willing to pay for.”
MPEG LA is owned by 10 companies including Cisco, Sony, General Electric and Google, thanks to its purchase of the phone business owned by Motorola. These owners are not necessarily patent-holders in MPEG LA’s pools and MPEG LA is not a standards body, so it does not create specifications.
The group was formed in 1997 by eight companies that wanted to turn the then-new MPEG 2 standard into a viable business by charging others to use it. The original line-up included Fujitsu, Panasonic, Sony and Philips.
MPEG LA doesn't create technology standards like MPEG 2 or H.264, but it does charge for them. MPEG 2 was the work of the ISO/IEC Moving Picture Experts Group (MPEG) and H.264 came from the ITU's Video Coding Experts Group and MPEG. The difference is the “LA” part - standing for Licensing Authority.
Now the ITU and MPEG group that built H.264, the Joint Collaborative Team on Video Coding, is cooking up H.265 - or High Efficiency Video Coding (HEVC). This idea is H.265 will improve video picture quality without demanding fatter bitrates from the devices' networks. This should make it better suited for use on tiny-footprint devices like smartphones, and tablets.
H.265 will improve coding efficiency to reduce the bitrate requirement – the goal is for a 50 per cent compression gain over H.264.
Like H.264, it’ll be MPEG LA that's the money-making end of the business for the tech companies whose patents get baked into H.265. “We are facilitating a pool for H.265,” according to Horn, who said the plan was for meetings in April and May with a licence to be issued this year.
If history is a guide, H.265 should be propelled to a rapid takeover. H.264's ascendancy was assisted by the fact that Apple and Microsoft, two of the most influential companies in business and consumer tech, were involved.
Apple and Microsoft not only helped build H.264, they made it ubiquitous through their browsers, servers, operating systems, tablets and phones. MPEG LA enforced the hegemony through patent licensing, seeing off potential rivals like VP8 from Google.
Horn won’t be drawn on why H.264 has taken over or MPEG LA's role: “These are decisions made by the marketplace based on numerous factors important to individual users/adopters,” he says. “We are agnostic on those choices.”
But with the muscle of MPEG LA in its corner, H.265 is likely to keep its rivals - tech giants or no - in line. ®
Links
- http://www.theregister.co.uk/2013/05/30/happy_birthdy_h264_codec/
- http://www.theregister.co.uk/2010/05/19/google_chrome_announcement/
- http://www.webmproject.org/
- http://arstechnica.com/business/2010/05/google-opens-vp8-codec-aims-to-nuke-h264-with-webm/
- http://www.mpegla.com/main/Pages/AboutHistory.aspx
- http://blog.mefeedia.com/html5-dec-2011
- http://allthingsd.com/20100520/googles-royalty-free-webm-video-may-not-be-royalty-free-for-long/
- http://blog.webmproject.org/2010/06/changes-to-webm-open-source-license.html
- http://hugoroy.eu/jobs-os.en.html
