From Wired:

The Trials and Tribulations of HTML Video in the Post-Flash Era

• By Ryan Paul - Ars Technica

Adobe reversed course on its Flash strategy after a recent round of layoffs and restructuring, concluding that HTML5 is the future of rich Internet content on mobile devices. Adobe now says it doesn’t intend to develop new mobile ports of its Flash player browser plugin, though existing implementations will continue to be maintained.

Adobe’s withdrawal from the mobile browser space means that HTML5 is now the path forward for developers who want to reach everyone and deliver an experience that works across all screens. The strengths and limitations of existing standards will now have significant implications for content creators who want to deliver video content on the post-flash web.

Apple’s decision to block third-party browser plugins like Flash on its iOS devices played a major role in compelling web developers to build standards-based fallbacks for their existing Flash content. This trend will be strengthened when Microsoft launches Windows 8 with a version of Internet Explorer that doesn’t support plugins in the platform’s new standard Metro environment.

Flash still has a significant presence on the Internet, but it’s arguably a legacy technology that will decline in relevance as mobile experiences become increasingly important. The faster pace of development and shorter release cycles in the browser market will allow open standards to mature faster and gain critical mass more quickly than before. In an environment where standards-based technologies are competitive for providing rich experiences, proprietary vendor-specific plugins like Flash will be relegated to playing a niche role.

Our use of the phrase post-Flash isn’t intended to mean that Flash is dead or going to die soon. We simply mean that it’s no longer essential to experiencing the full web. The HTML5 fallback experiences on many Flash-heavy sites still don’t provide feature parity with the Flash versions, but the gap is arguably shrinking — and will continue to shrink even more rapidly in the future.

Strengths and weaknesses of HTML5 video

HTML5 has much to offer for video delivery, as the HTML5 video element seamlessly meshes with the rest of the page DOM and is easy to manipulate through JavaScript. This means that HTML5 video offers significantly better native integration with page content than it has ever been possible to achieve with Flash. The open and inclusive nature of the standards process will also make it possible for additional parties to contribute to expanding the feature set.

A single company no longer dictates what can be achieved with video, and your video content is no longer isolated to a rectangle embedded in a page. HTML5 breaks down the barriers between video content and the rest of the web, opening the door for more innovation in content presentation. Three are some really compelling demonstrations out there that showcase the use of video in conjunction with WebGL and other modern web standards. For example, the video shader demo from the 3 Dreams of Black interactive film gives you a taste of what’s possible.

Of course, transitioning video delivery in the browser from Flash to HTML5 will also pose some major challenges for content creators. The standards aren’t fully mature yet and there are still a number of features that aren’t supported or widely available across browsers.

For an illustration of how deep the problems run, you need only look at Mozilla’s Firefox Livepromotional website, which touts the organization’s commitment to the open web and shows live streaming videos of Red Panda cubs from the Knoxville Zoo. The video is streamed with Flash instead of using standards-based open web technologies.

Flash is required to see the Red Panda cubs on Mozilla's website

In an FAQ attached to the site, Mozilla says that it simply couldn’t find a high-volume live-streaming solution based on open codecs and open standards. If Mozilla can’t figure out how to stream its cuddly mascot with open standards, it means there is still work to do.

Two of the major technical issues faced by HTML5 video adopters are the lack of adequate support for adaptive streaming and the lack of consensus surrounding codecs. There is currently an impasse between backers of the popular H.264 codec and Google’s royalty-free VP8 codec. There’s no question that a royalty-free video format is ideal for the web, but the matter of whether VP8 is truly unencumbered by patents — and also meets the rest of the industry’s technical requirements — is still in dispute.

There is another major issue that hasn’t been addressed yet by open web standards that could prove even more challenging: content protection. The vast majority of Flash video content on the Internet doesn’t use any kind of DRM and is trivially easy to download. Flash does, however, provide DRM capabilities and there are major video sites that rely on that technology in order to protect the content they distribute.

Can DRM be made to play nice with open standards?

DRM is almost always bad for regular end users and its desirability is highly debatable, but browser vendors will have to support it in some capacity in order to make HTML5 video a success. Many of the content creators who license video material to companies like Netflix and Hulu contractually stipulate a certain degree of content protection.

Mozilla&’s Robert O’Callahan raised the issue of HTML5 video DRM in a recent blog entry shortly after Adobe’s announcement regarding mobile Flash. He expressed some concern that browser vendors will look for a solution that is expedient rather than inclusive, to the detriment of the open web.

“The problem is that some big content providers insist on onerous DRM that necessarily violates some of our open web principles (such as web content being equally usable on any platform, based on royalty-free standards, and those standards being implementable in free software),” O’Callahan wrote. “We will probably get into a situation where web video distributors will be desperate for an in-browser strong DRM solution ASAP, and most browser vendors (who don’t care all that much about those principles) will step up to give them whatever they want, leaving Mozilla in another difficult position. I wish I could see a reasonable solution, but right now I can’t. It seems even harder than the codec problem.”

O’Callahan also pointed out in his blog entry that the upcoming release of Windows 8, which will not support browser plugins in its Metro environment, means that the lack of DRM support in standards-based web video is no longer just a theoretical concern. Microsoft may need to furnish a solution soon, or risk frustrating users who want to watch commercial video content on the web in Windows 8 without installing additional apps or leaving the Metro shell.

Netflix stands behind DASH

Flash evangelists may feel that the limitations of HTML5 video and the problems that content creators are sure to face during the transition are a vindication of the proprietary plugin model. But the advantages of a truly open, vendor-neutral, and standards-based video solution that can span every screen really dwarf the challenges. That is why major stakeholders are going to be willing to gather around the table to try find a way to make it work.

Netflix already uses HTML5 to build the user interfaces of some of its embedded applications, including the one on the PS3. The company has soundly praised the strengths of a standards-based web technology stack and has found that there are many advantages. But the DRM issue and the lack of suitably robust support for adaptive streaming have prevented Netflix from dropping its Silverlight-based player in regular web browsers.

The company has committed to participating in the effort to make HTML5 a viable choice for all video streaming. Netflix believes that the new Dynamic Adaptive Streaming over HTTP (DASH) standard being devised by the Motion Picture Experts Group (MPEG) will address many of the existing challenges and pave the way for ubiquitous adoption of HTML5 for streaming Internet video.

DASH, which is expected to be ratified as an official standard soon, has critical buy-in from many key industry players besides Netflix, including Microsoft and Apple. An early DASH playback implementations is already available as a plugin for the popular VLC video application.

The DASH standard makes video streaming practical over HTTP and addresses the many technical requirements of high-volume streaming companies like Netflix, but it doesn’t directly address the issue of DRM by itself. DASH can be implemented in a manner that is conducive to supporting DRM, however.

DASH and DRM

Ericsson Research, which is involved in the DASH standardization effort, has done some worthwhilepreliminary research to evaluate the viability of DRM on DASH. Ericsson produced a proof-of-concept implementation that uses DRM based on the Marlin rights management framework. Marlin, which was originally created by a coalition of consumer electronics vendors, is relatively open compared to alternate DRM technologies and makes use of many existing open standards. But Marlin is still fundamentally DRM and suffers from many of the same drawbacks, and adopters have to obtain a license from the Marlin Trust Management Organization, which holds the keys.

The architecture of the Marlin rights management framework

Ericsson explains in its research that it chose to experiment with Marlin for their proof-of-concept implementation because it’s available and mature — other similar DRM schemes could also easily be adopted. Existing mainstream DRM schemes would all likely pose the same challenges, however, and it’s unlikely that such solutions will be viewed as acceptable by Mozilla. More significantly, an implementation of HTML5 video that relies on this kind of DRM would undermine some of the key values and advantages of openness that are intrinsic to the open web.

The ease with which solutions like Marlin can be implemented on top of HTML5 will create pressure for mainstream browser vendors to adopt them quickly. This could result in the same kind of fragmentation that exists today surrounding codecs. As O’Callahan said, it’s easy to see this issue becoming far more contentious and challenging to overcome than the codec issue.

What next?

The transition to HTML5 and standards-based technology for video delivery will bring many advantages to the web. There are some great examples that show what can be achieved when developers really capitalize on the strengths of the whole open web stack. The inclusiveness of the standards process will also give a voice to additional contributors who want to expand the scope of what can be achieved with video on the web.

There are still some major obstacles that must be overcome in order for the profound potential of standards-based web video to be fully realized in the post-Flash era. Open standards still don’t deliver all of the functionality that content creators and distributors will require in order to drop their existing dependence on proprietary plugins. Supplying acceptable content protection mechanisms will prove to be a particularly bitter challenge.

Despite the barriers ahead, major video companies like Netflix recognize the significant advantages of HTML5 and are willing to collaborate with other stakeholders to make HTML5 video a success. The big question that remains unanswered is whether that goal can be achieved without compromising the critically important values of the open web.