Neural networks upscaling of 1896

Denis Shiryaev upscaled 60 fps 4k version of 1896 movie "Arrival of a Train at La Ciotat" with several neural networks

https://www.youtube.com/watch?v=3RYNThid23g





https://www.reddit.com/r/videos/comments/eyoxfb/oc_i_have_made_60_fps_4k_version_of_1896_movie/

Upscaled and resounded version of a classic B&W movie: Arrival of a Train at La Ciotat, The Lumière Brothers, 1896

Source used to upscale: https://youtu.be/MT-70ni4Ddo

Algorithms that were used:
 ››› To upscale to 4k – Gigapixel AI – Topaz Labs https://topazlabs.com/gigapixel-ai/
 ››› To add FPS – Dain, https://sites.google.com/view/wenbobao/dain 

This is the interesting Bit:
Depth-Aware Video Frame Interpolation
Wenbo Bao*, Wei-Sheng Lai#, Chao Ma*, Xiaoyun Zhang*, Zhiyong Gao*, Ming-Hsuan Yang#&

*Shanghai Jiao Tong University,     #University of California, Merced,   &Google

Abstract
Video frame interpolation aims to synthesize non-existent frames in-between the original frames. While significant advances have been made from the deep convolutional neural networks, the quality of interpolation is often reduced due to large object motion or occlusion. In this work, we propose to explicitly detect the occlusion by exploring the depth cue in frame interpolation. Specifically, we develop a depth-aware flow projection layer to synthesize intermediate flows that preferably sample closer objects than farther ones. In addition, we learn hierarchical features as the contextual  information. The proposed model then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels for synthesizing the output frame. Our model is compact, efficient, and fully differentiable to optimize all the components. We conduct extensive experiments to analyze the effect of the depth-aware flow projection layer and hierarchical contextual features. Quantitative and qualitative results demonstrate that the proposed model performs favorably against state-of-the-art frame interpolation methods on a wide variety of datasets. 


 Update: Colorized by DeOldify Neural Network version of this video: https://youtu.be/EqbOhqXHL7E





This is an upscaled version of the gorgeous video by Bard Canning of curiosity descent uploaded on Sep 13, 2012,
 source: https://youtu.be/Esj5juUzhpU

 The video was upscaled with Gigapixel AI software to 4K, frame by frame, 60 FPS was achieved with After Effect frame blending. I made this video for fun, to spread the love of space traveling. Here is my telegram channel: http://t.me/denissexy Here is comparison: https://gfycat.com/diligentgianteaste... Here is a tutorial how to upscale things in Russian language: https://vc.ru/76580 x

Sony Promotes 4K Resolution for Security Applications

Sony publishes Youtube video showing 4K technology for security cameras:



Another Sony video demos 5-axis optical image stabilization operation in Alpha 7-II DSLR, said to be the first in a full-frame camera.

Tata Sky demos 4K TV with FIFA World Cup quarter-final match

MUMBAI: The era of India being a country that lags behind the more developed nations is gone. Indian  DTH operators in the country are waking up, ancticipating  future desires of Indian TV viewers. Which is why one of the oldest and largest DTH operators Tata Sky, is keeping itself and its subscribers up to date with the  introduction of its 4k Ultra HD service.
 
The first quarter of 2015 will see Tata Sky roll out its set top boxes (STBs) for its 4k Ultra HD service. The STBs are being made by Technicolor and are based on the high efficiency video coding (HEVC) format that 4k uses the world over. However, prices of the STB are not yet known as the DTH operator wants to reveal it closer to the commercial launch date.
 
At a demonstration event in Mumbai, Tata Sky along with Sony Six unveiled the 4k UHD service with a live uninterrupted broadcast of the France versus Germany FIFA World Cup 2014 quarter final. The FIFA 4k signal rights had been bought by Sony Six while the cost of acquisition and transmission was split 50:50 by the broadcaster and the operator. The telecast took place live via satellite transmission.  
 
Indiantelevision.com readers will note that Videocon d2h had previewed its 4K service on 3 July 2014 too, a day before Tata Sky..
 
Even though most industry professionals  claim that it is too early to launch 4k Ultra HD in India, Tata Sky chief commercial officer Vikram Mehra feels otherwise. “When we launched HD in 2010, people doubted us saying it won’t work. As you keep enhancing TV experience, the viewer will appreciate it. So, Ultra HD is our endeavour in that area. A technology becomes popular when mainline pay-TV adopts it.”
 
Mehra feels that movies and sports will be the initial drivers for 4k technology and talks with broadcasters have already commenced. In three to four years, he expects it to be the way of life for most Indians TV viewers. Even though currently there is a lack of 4k content, he feels that it isn’t too far away. “We will make a product that makes sense to the customer. Customers will not buy STBs to adorn their homes. They will buy it for content,” he says.
 
Sony Six business head Prasana Krishnan feels that more sporting events should be produced in 4k soon, probably in the next year or so. “4K is the future of broadcasting, specially sports. We saw technology move from SD to HD and now HD has become standard. 4k is next,” he says. Krishnan is aware that a full-fledged 4k channel can’t be expected very soon. Big sporting events will have 4k production on crucial days such as the finals, while the run up matches will be in HD and SD.
 
The 4k screening of the FIFA World Cup 2014 quarter finals that took place in Mumbai yesterday, 4 July saw the Sony Six logo being superimposed on the FIFA feed on an 84 inch Sony television set that is soon to be launched in India and across the world. 4k screens give four times the clarity as compared to an HD feed.
 
Mehra  was not willing to venture any guesstimates about how much uptake the Tata Sky 4k service will have when it launches next year as he feels making futuristic statements is of no use. However, the fact that the DTH operator is confident about its prospects is clear from the contract that it has signed with Technicolor, according to which deliveries of 4K set top boxes in "volume" are expected to commence in early 2015.

SOURCE: IndianTelevision

The Continuing Evolution of Video Codecs

From a LinkedIn discussion in HEVC / H.265.

Sinan Othman, Ph.D. Said:  

The current dominant codec for Internet video is H.264, but there are influential companies dedicated to alternatives. With the advent of 4K (aka Ultra HD), new codecs will be useful. H.265 (aka HEVC) and VP9 could split the market.

While commenting on: 

The Continuing Evolution of Video Codecs

http://www.cedmagazine.com/articles/2014/07/the-continuing-evolution-of-video-codecs

HEVC, H.265, H.264, UHD, UHDTV, VP9, 4K, U;tra HD

Here's the 4K IMAX Camera That's Going to Make 3D Movies Awesome Again

http://gizmodo.com/heres-the-4k-imax-camera-thats-going-to-make-3d-movies-1588929194

Sony shows (and tells) us why 4K on a phone isn't crazy

http://www.engadget.com/2014/03/21/sony-xperia-z2-4k/

Looks like 4K cell phone cameras are here, but I have not found anyone making a cell phone 4K display yet.

This is something the VR world needs like the Oculus Rift and Samsung an other face mounted screens based HMD's.

Blackmagic Production Camera 4K Now Shipping for $2,995 | digphoto.org

http://www.digphoto.org/content/blackmagic-production-camera-4k-now-shipping-2995/71719796

4K Cameras at CES 2014!

http://youtu.be/L6RW1KaNuL0

EIZO DuraVision FDH3601 4K x 2K resolution across a 36.4" screen.

http://www.eizo.com/global/products/duravision/fdh3601/index.html

4K x 2K in a Single Glance

The DuraVision FDH3601 displays 8.8 megapixels (4096 x 2160 native resolution) across its 36.4-inch screen. This is enough pixels to display the content of several desktop monitors but without the distractive bezels between screens.

Monitor for Air Traffic Control and Geophysical Services

The DuraVision FDH3601 ideal for fields like air traffic control and geophysical services requiring large monitors that can display a high volume of information. The DuraVision FDH3601 enhances EIZO's selection of primary control monitors for ATC by offering new ways of displaying information. For example, it can display one 4K x 2K image across the entire screen or it can display a 2K x 2K image and auxiliary information simultaneously.

Wide Dimming Range

Using the latest white LED backlight technology the DuraVision FDH3601 offers a brightness range from 30 cd/m² up to 700 cd/m², making it ideal for almost any ambient lighting environment.

Fine Rendering of Detail

With an extremely tight pixel pitch of 0.1995, text and images such as contours on a map can be displayed in fine detail. The tight pixel pitch allows the monitor to display the equivalent of four full HD monitors on its 36.4-inch screen without bezels to divide the screens.

Presence Sensor for Power Savings

EIZO's EcoView Sense feature unites convenience with energy savings by ensuring that the monitor automatically conserves power when it is not in use. EcoView Sense is an integrated presence sensor that prompts the monitor to switch to power save mode when it detects your absence, and then resume normal operation when you return. To prevent the screen from reactivating when it shouldn't, EcoView Sense differentiates between your presence and a still object like an empty chair or general office activity in the background.

Zero Watts When Turned Off

When the monitor is switched off via its AC adapter's main power switch it consumes no electricity at all.

Brightness and Color Uniformity with DUE

Fluctuations in brightness and chroma on different parts of the screen are characteristic of LCD panels. Generally, the larger the screen, the more noticeable these uniformity errors become. EIZO's Digital Uniformity Equalizer (DUE) helps to compensate for these errors, resulting in a much more uniform image.

DVI-D and DisplayPort Inputs

The DuraVision FDH3601 includes two dual-link DVI-D and two DisplayPort connectors for connecting to up to four computers. A USB hub with one upstream and two downstream ports is also included.

Smooth Rendering of Color Tones and 10-Bit Simultaneous Display

A 16-bit look-up table offers a palette of about 278 trillion colors from which the most appropriate 16.7 million are displayed. This produces smooth reproduction of color and grayscale tones. Furthermore, with the DisplayPort inputs, the monitor can display simultaneous 10-bit color *. This is more than 1 billion colors shown simultaneously which is 64 times greater than the 16.7 million colors of 8-bit display. The result is even smoother display of gradations and reduced Delta-E between two adjacent colors.

• A graphics board and software which support 10-bit output are also necessary for 10-bit display.

Ergonomic Stand

The DuraVision FDH 3601 comes with EIZO's ergonomic FlexStand 2. It offers very versatile positioning with 100 mm height adjustment, 25° tilt, and 344° swivel. The monitor can be removed from the stand and mounted on a VESA-compatible arm or wall mount.

On-Screen Button Guide

An on-screen button guide appears above the buttons to indicate what each one is for. This is convenient when using the monitor in a dimly-lit working environment. Furthermore, the brightness of the power lamp can be adjusted in eight stages including shutting off the lamp.

Mercury-Free, White LED Backlight

The white LED backlight provides very stable brightness even at low settings like the monitor's minimum brightness of 30 cd/m². Unlike a CCFL backlight, an LED backlight is mercury free so it will have less impact on the environment when it is disposed of.

Wide Viewing Angles

The DuraVision FDH3601 uses an LCD panel that has viewing angles of 176° and exhibits minimal color shift so two or more people can view the screen at once.

Preset Modes

Five preset modes are included: sRGB, Text, and three customizable modes where users can input their preferred values for brightness, color temperature, and gamma.

MIT researchers build ultrahigh-definition Quad HD (4K) TV chip

From:  http://www.kurzweilai.net/mit-researchers-build-ultrahigh-definition-quad-hd-4k-tv-chip

At the International Solid-State Circuits Conference this week, MIT researchers unveiled their own Quad HD video chip design.

Quad HD is also known as 4K and ultrahigh-definition (UHD). The new Quad HD video standard enables a fourfold increase in the resolution of TV screens.

At the Consumer Electronics Show (CES) in January, several manufacturers debuted new UHD models.

There is no UHD content yet, but the Japanese government plans to launch the world's first 4K TV broadcast in July 2014, from communications satellites, followed by satellite broadcasting and ground digital broadcasting, NBC News Gadget Box has reported.

Nonetheless, 4K TVs are now on sale by Japanese makers including Sony, Panasonic and Sharp. Other manufacturers include South Korea's LG Electronics.

HEVC

[+]

A key to efficient video compression is predicting future video frames on the basis of past ones. This diagram concerns "intra angular prediction."

(Credit: ISO)

UHD also requires a new video-coding standard, known as high-efficiency video coding, or HEVC (aka or H.265).

Although the MIT chip isn't intended for commercial release, its developers believe that the challenge of implementing HEVC algorithms in silicon helps illustrate design principles that could be broadly useful.

Moreover, "because now we have the chip with us, it is now possible for us to figure out ways in which different types of video data actually interact with hardware," says Mehul Tikekar, an MIT graduate student in electrical engineering and computer science and one of the paper's co-authors.

How HEVC works

Like older coding standards, the HEVC standard exploits the fact that in successive frames of video, most of the pixels stay the same. Rather than transmitting entire frames, it's usually enough for broadcasters to transmit just the moving pixels, saving a great deal of bandwidth. The first step in the encoding process is thus to calculate "motion vectors" — mathematical descriptions of the motion of objects in the frame.

On the receiving, end, however, that description will not yield a perfectly faithful image, as the orientation of a moving object and the way it's illuminated can change as it moves. So the next step is to add a little extra information to correct motion estimates that are based solely on the vectors. Finally, to save even more bandwidth, the motion vectors and the corrective information are run through a standard data-compression algorithm, and the results are sent to the receiver.

The new chip performs this process in reverse. It was designed by researchers in the lab of Anantha Chandrakasan, the Joseph F. and Nancy P. Keithley Professor of Electrical Engineering and head of the MIT Department of Electrical Engineering and Computer Science.

The chip's first trick for increasing efficiency is to "pipeline" the decoding process: a chunk of data is decompressed and passed to a motion-compensation circuit, but as soon as the motion compensation begins, the decompression circuit takes in the next chunk of data. After motion compensation is complete, the data passes to a circuit that applies the corrective data and, finally, to a filtering circuit that smooths out whatever rough edges remain.

Fine-tuning

Pipelining is fairly standard in most video chips, but the MIT researchers developed a couple of other tricks to further improve efficiency. The application of the corrective data, for instance, is a single calculation known as matrix multiplication. A matrix is just a big grid of numbers; in matrix multiplication, numbers in the rows of one matrix are multiplied by numbers in the columns of another, and the results are added together to produce entries in a new matrix.

"We observed that the matrix has some patterns in it," Tikekar explains. In the new standard, a 32-by-32 matrix, representing a 32-by-32 block of pixels, is multiplied by another 32-by-32 matrix, containing corrective information. In principle, the corrective matrix could contain 1,024 different values. But the MIT researchers observed that, in practice, "there are only 32 unique numbers," Tikekar says. "So we can efficiently implement one of these [multiplications] and then use the same hardware to do the rest."

Similarly, Chiraag Juvekar, another graduate student in Chandrakasan's group, developed a more efficient way to store video data in memory. The "naive way," he explains, would be to store the values of each row of pixels at successive memory addresses. In that scheme, the values of pixels that are next to each other in a row would also be adjacent in memory, but the value of the pixels below them would be far away.

In video decoding, however, "it is highly likely that if you need the pixel on top, you also need the pixel right below it," Juvekar says. "So we optimize the data into small square blocks that are stored together. When you access something from memory, you not only get the pixels on the right and left, but you also get the pixels on the top and bottom in the same request."

Chandrakasan's group specializes in low-power devices, and in ongoing work, the researchers are trying to reduce the power consumption of the chip even further, to prolong the battery life of quad-HD cell phones or tablet computers.

One design modification they plan to investigate, Tikekar says, is the use of several smaller decoding pipelines that work in parallel. Reducing the computational demands on each group of circuits would also reduce the chip's operating voltage.

You Can Edit 4K Video on a MacBook Air (With a Red Rocket)

http://www.tomshardware.com/news/macbook-air-video-4k-video-red-rocket,14584.html

Since you can't even play 4K video on any PC, most of us would not automatically come up with an idea to edit 12-megapixel resolution video on a Macbook Air.

However, a demo published by Dave Helmly, in charge of tech sales of Adobe's pro audio and video products, shows that it is possible.

4K video editing

Helmly used a flagship Macbook Air with an Intel 1.8 GHz i7-2677M processor, which retails for $1700. The 4K video quickly overwhelmed the dual-core processor and its four threads. However, the video, apparently recorded with a Red One camera, ran in real-time via a Thunderbolt-connected Red Rocket accelerator and transcoder card. The Red Rocket card is key to getting 4K video to be played at 30 fps on a PC and can be purchased for $4,750.

It's important to note that Helmly went with the Apple hardware, but on the OS side it's running Windows 7 in boot camp. Helmly explained to Gizmodo on his choice:

No real reason I chose to show it running under Windows 7 other than I've been surprised that we haven't seen any demos of Windows running TB before now. It actually works pretty good in it's current beta state. That said the Mac OS kicks @ss running the same config. Please keep in mind that half the battle is getting alpha/beta Windows 7 64bit drivers for each TB device. All TB devices need drivers at some level. All necessary Mac OS driver are already shipping.

The Mac + TB is really last years news and we all want more TB peripherals to start shipping and to start showing Intel and PC makers that there is lots of interest on both sides and it will benefit all TB users. I have no preference on OS and use both everyday.

4K video is still an emerging standard and out of reach for mainstream customers. The Red One camera, which supports recording formats up to 4.5K at 4480x1920 pixels, is currently sold for $25,000. YouTube began supporting 4K videos in mid-2010. An example can be seen below (but you'll have to blow it up to original size if you want to see it in all its glory).

Surf NYC 4K Video