Computational Imaging and Microscopy

This is an excellent talk, and takes one of the most complex subjects and breaks it down simply from the beginning.  

     14:38 in to the video. 

  It took be the better part of 25 years to learn the secret of zooming in to a license plate from an impossibly Zoomed image.  Something that when shown in Sci-Fi TV shows in the 70's and 80's - 90's...  I just assumed was Bullshit.  

 I eventually learned the secret from one of the top digital imaging experts that's I've known 25 years, and well after his retirement.  He implemented it on super top secret hardware for satellite imaging, when I was still in grade school.  It used the fact that every pixel in your image represents a sine cardinal from the whole image.  Also known as Sinc function it is the continuous inverse Fourier transform of a rectangular pulse and can be thought of a Gaussian modulated sine wave, although I am not really sure if these are the exact equivalent though I see it implemented in RF applications like this. 

In his case the Lens had to be extremely well understood, and in the end it generated convolution filters that were able to be ran quickly and efficiently. 

What is interesting is to see this generalized in to a generic computational imaging problem.  It may actually yield better results with more information, but most likely will be much more computation. 

Computational imaging involves the joint design of imaging system hardware and software, optimizing across the entire pipeline from acquisition to reconstruction. This talk will describe new methods for computational microscopy with coded illumination, based on a simple and inexpensive hardware modification of a commercial microscope. Traditionally, one must trade field-of-view for resolution; with our methods we can have both, resulting in Gigapixel-scale images with resolution beyond the diffraction limit of the system. Our reconstruction algorithms are based on large-scale nonlinear non-convex optimization procedures for phase retrieval. Laura Waller leads the Computational Imaging Lab, which develops new methods for optical imaging, with optics and computational algorithms designed jointly. She holds the Ted Van Duzer Endowed Professorship and is a Senior Fellow at the Berkeley Institute of Data Science (BIDS), with affiliations in Bioengineering and Applied Sciences & Technology. Laura was a Postdoctoral Researcher and Lecturer of Physics at Princeton University from 2010-2012 and received BS, MEng and PhD degrees from MIT in 2014, 2015 and 2010, respectively. She is a Moore Foundation Data-Driven Investigator, Bakar fellow, Distinguished Graduate Student Mentoring awardee, NSF CAREER awardee and Packard Fellow.

Bus Passenger Counting System

Bus Passenger Counting System

The people counting function could be integrated in mobile DVR or cameras. 

You can choose the solution as you like.

If you're interested in the system, pls kindly let me know. TKS. My skype: levy-he. My email: levy.he@jasanwit.com

Augmented Pixels: Indoor Navigation Platform for Drones

Drones are notoriously difficult to handle indoors: hard to control and prevent crashing into walls or people.

Augmented Pixels has been actively developing technology (including SLAM) to ensure safe flights as well as intuitive and easy navigation using Augmented Reality.

They came up with a platform that significantly reduces accident rates and minimizes the effect of "human factor". Moreover, it is possible to program the drone to fly around and land by itself.

The prospects for this technology include a wide range of use cases (e.g. inspection of premises for security, creation of 360-degree tours, etc.).Augmented Pixels is located in Palo Alto, CA. 

http://augmentedpixels.com

Let's build a humanoid robot with computer vision

---------- Forwarded message ----------
From: Hack A Robot <info@meetup.com>
Date: Saturday, August 22, 2015
Subject: Invitation: Let's build a humanoid robot with computer vision

New Meetup

Let's build a humanoid robot with computer vision

Hack A Robot

Added by Thomas Lee

Thursday, August 27, 2015
6:30 PM

South bay area
TBD
South bay area, CA

(Venue of this event is TBD.  Looking for suggestions/ offers to host this event as well) Earlier this year, I created Hackabot Nano (a feature-rich Arduino compatible wheeled robot). It was crowdfunded through Kickstarter. The robot was p...

Learn more

Computational cameras

Computational Cameras: Convergence of Optics and Processing

 Changyin Zhou, Student Member, IEEE, and Shree K. Nayar, Member, IEEE

Abstract—A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras has been demonstrated to encode more useful visual information in the captured images, as compared with conventional cameras. In this paper, we survey computational cameras from two perspectives. First, we present a taxonomy of computational camera designs according to the coding approaches, including object side coding, pupil plane coding, sensor side coding, illumination coding, camera arrays and clusters, and unconventional imaging systems. Second, we use the abstract notion of light field representation as a general tool to describe computational camera designs, where each camera can be formulated as a projection of a high-dimensional light field to a 2-D image sensor. We show how individual optical devices transform light fields and use these transforms to illustrate how different computational camera designs (collections of optical devices) capture and encode useful visual information. Index Terms—Computer vision, imaging, image processing, optics.


http://www1.cs.columbia.edu/CAVE/publications/pdfs/Zhou_TIP11.pdf

BPG image format judged awesome versus JPEG

http://phys.org/news/2014-12-bpg-image-format-awesome-jpeg.html#ajTabs

New optical technique extracts audio from video

http://phys.org/news/2014-12-walls-optical-technique-audio-video.html


Those formerly silent walls can "talk" now: Researchers have demonstrated a simple optical technique by which audio information can be extracted from high-speed video recordings. The method uses an image-matching process based on vibration from sound waves, and is reported in an article appearing in the November issue of the journal Optical Engineering, published by SPIE, the international society for optics and photonics.

"One of the intriguing aspects of the paper is the ability to recover spoken words from a video of objects in the room," said journal Associate Editor Reiner Eschbach, a Research Fellow at Xerox Corp. "The paper shows that the sound creates minute vibrations in objects and that these vibrations ― given the right equipment ― can be picked up from a video signal. This is an interesting foray into a new application space and will, in my view, trigger interesting research in the field,"

The article, "Audio extraction from silent high-speed video using an optical technique," was authored by Zhaoyang Wang, Hieu Nguyen, and Jason Quisberth of the Department of Engineering of the Catholic University of America, and is available from the SPIE Digital Library.

The technique is based on the fact that sound waves are mechanical waves that cause air to vibrate when traveling, the paper notes. That vibration through air can cause vibration of objects located in its traveling path, especially if the objects are lightweight, thin, and flexible, such as a piece of paper. The vibrations, although usually with small amplitudes, can be detected and analyzed algorithmically, and audio reconstructed based on those calculations.

The authors used a subset-based image-correlation approach to detect the motions of points on the surface of an object, capturing target images with a high-speed camera and applying the Gauss-Newton algorithm and a few other measures to achieve very fast and highly accurate image matching. Because the detected vibrations are directly related to sound waves, a simple model was used to reconstruct the original audio information of the sound waves.

While other recent work in the area reports on more sophisticated techniques to compute motion signals, the authors chose a simpler image-matching approach to measure vibration. Because light can travel through air considerably farther than sound and can pass through glass, they anticipate that the technique may find applications such as the passive detection of conversations inside of a building from a far distance, Wang said. "We are currently improving the technique to increase its accuracy and sensitivity, make the measurements in real-time, and remove interference from other sources."

Explore further: Algorithm recovers speech from vibrations of potato-chip bag filmed through soundproof glass

~ $300 for stereo video on PI.

Dual Camera input PI $215 

http://www.newark.com/raspberry-pi/rpi-compute-dev-kit/raspberry-pi-compute-dev-board/dp/62X6560?ost=62X6560&COM=rpimain-topban-compmod

Camera Boards are $30 each. 

http://www.adafruit.com/products/1367

I think this price is mostly driven by OpenCV users and other hobbiests driving up prices to be too high to use internal to all but high ticket products. 

ICatch Technology

http://www.icatchtek.com  iCatch Technology

http://www.sunplus.com


ICatch Technology makes Digital video imaging platform chipsets used in smart phones, tablet PCs, digital cameras and digital video recorders and other products include digital, analog, algorithms, software and system design products and services.

founded in 2009, focuses on digital video imaging platform chip design. With its own video core technologies, patents and R&D team.

They have ten years of experience in the digital imaging industry, production and sales of more than one hundred million systems on chip.

iCDSP

iCDSP is iCatch’s image processing pipeline. It reaches 240M pixels/sec data throughput with in-pipe LDC (Lens Distortion Correction) and AHD (Advanced High-ISO De-noise) functions. Combined with the iCatch’s high speed JPEG encoder, the iCatch DSC SoC has a high burst capture performance of 16M@15fps. The images can be captured, processed, compressed and stored to the storage media until the storage card full. There is no limitation on the DRAM density.

iCDSP key features

» Super resolution

» Advanced High-ISO de-noise

» Wide dynamic range
» Multi-frame noise reduction 

» Face detection 

They make the video controller in my past article, Car Keychain Micro Video Camera's

They also have their own RTOS iCatOS although no public data is available that I could find on it.

Lecture 06 - Optical Flow

A Sidewalk Disappearing Act

Good article on how Google is using image processing of multiple camera views to remove people from Google map images. 

http://www.technologyreview.com/news/420187/a-sidewalk-disappearing-act/

This is also know as diminished Reality: 

http://videotechnology.blogspot.com/2010/10/diminished-reality.html

This can be done to live Full HD  images in real time, so you can't trust your CCTV cameras from any group with this level of technology that could be done for $99 of off the shelf chips. 

Video based mouse, Celluon Evomouse

http://www.celluon.com/products_em_overview.php


http://youtu.be/UZWLwjbJZ-s

Transcribing Piano Rolls with Python

http://hackaday.com/2014/04/12/transcribing-piano-rolls-with-python/

Perforated rolls of paper, called piano rolls, are used to input songs into player pianos. The image above was taken from a YouTube video showing a player piano playing a Gershwin tune called Limehouse Nights. There’s no published sheet music for the song, so [Zulko] decided to use Python to transcribe it.

First off the video was downloaded from YouTube. This video was processed withMoviePy library to create a single image plotting the notes. Using a Fourier Transform, the horizontal spacing between notes was found. This allowed the image to be reduced so that one pixel corresponded with one key.

With that done, each column could be assigned to a specific note on the piano. That takes care of the pitches, but the note duration requires more processing. The Fourier Transform is applied again to determine the length of a quarter note. With this known, the notes can be quantized, and a note duration can be applied to each.

Once the duration and notes are known, it’s time to export sheet music. LilyPond, an open source language for music notation, was used. This converts ASCII text into a sheet music PDF. The final result is a playable score of the piece.

Smart i 360 Degree Seamless Surround View Digital Video Recorder.

Published on Aug 3, 2013

360 Bird-Eye View of entire vehicle surrounding using the latest technology. Records up to 28 hours of continuous recording with a 28GB SD card. For more enquiries, kindly send you email to eric.sim@smart-i.us. We are looking for distributors.

smart-i.us

Fujitsu 360° Wraparound View System

This is the beginnings of a free viewpoint video system.

Getting a textured 3D scan from just a webcam

Getting a textured 3D scan from just a webcam

Here’s an oldie but a goodie that passed us up the first time it went around the Internet. [Qi Pan], (former) PhD student at Cambridge, made a 3D modeling program using only a simple webcam. Not only does this make very fast work of building 3D models, the real texture is also rendered on the virtual object.
The project is called ProFORMA, and to get some idea of exactly how fast it is, the model of a church seen above was captured and rendered in a little over a minute. To get the incredible speed of ProFORMA, [Qi] had his webcam take a series of keyframes. When the model is rotated about 10°, another keyframe is taken and the corners are triangulated with some very fancy math.
Even though [Qi]‘s project is from 2009, it seems like it would be better than the ReconstructMe, the Kinect-able 3D scanning we saw a while ago. There’s a great video of [Qi] modeling a papercraft church after the break, but check out the actual paper for a better idea of how ProFORMA works.

Real-time depth smoothing for the Kinect

From Hack-A-Day:

[Karl] set out to improve the depth image that the Kinect camera is able to feed into a computer. He’s come up with a pre-processing package which smooths the depth data in real-time.

There are a few problems here, one is that the Kinect has a fairly low resolution, it is also depth limited to a range of about 8 meters from the device (an issue we hadn’t considered when looking at Kinect-based mapping solutions). But the drawbacks of those shortcomings can be mitigated by improving the data that it does collect. [Karl's] approach is twofold: pixel filtering, and averaging of movement.

The pixel filtering works with the depth data to help clarify the outlines of objects. Weighted moving average is used to help reduce the amount of flickering areas rendered from frame to frame. [Karl] included a nice GUI with the code which lets you tweak the filter settings until they’re just right. See a demo of that interface in the clip after the break and let us know what you might use this for by leaving a comment.

Medical Imaging With a Hacked LCD Projector

From Slashdot:

Medical Imaging With a Hacked LCD Projector

Posted by Soulskill on Tuesday January 03, @07:20PM 
from the convergent-technology dept.

An anonymous reader writes"Grad students at UC Irvine have built a spatial frequency domain imaging system using parts from a cheap LCD projector and a digital camera. The system can be used to check the level of bruising or oxygenation in layers of tissue that aren't visible to the naked eye, according to an article in Chemical and Engineering News. An accompanying video shows the series of patterned pulses that the improvised imaging system makes in order to read hemoglobin and fat levels below the surface of the skin. A more sophisticated version of the imaging system is being commercialized by a startup within UC Irvine, called Modulated Imaging. The article and video also describe infrared brain scanners that can non-invasively check for brain bleeds, and multiphoton microscopes that produce stunning images of live skin cells."

Video based air harp

From Hack a day:
http://hackaday.com/2011/12/12/get-ready-to-play-some-wicked-air-harp

Who needs a tactile interface when you can wave your hands in the air to make music? Air String makes that possible and surprisingly it does so without the use of a Kinect sensor.

In the image above, you can see that two green marker caps are used as plectra to draw music out of the non-existent strings. Judiciously perched atop that Analysis and Design of Digital Systems with VHDL textbook is a camcorder recording an image of the player. This signal is processed by an FPGA (hence the textbook) in real-time, and shown on the monitor seen to the right. A set of guides are overlaid on the image, so the player knows where to pluck to get the notes she is expecting.

The program is designed to pick up on bright green colors as the inputs. It works like a charm as you can see in the video after the break. The team of Cornell students responsible for the project also mention a few possible improvements like adding a distance sensor (ultrasonic rangefinder?) so that depth can be used for the dynamics of the sound.

Real Time Face Substitution

This is a sign of things to come.
You can see this is very raw and has lots of problems, but it's close and soon will be nearly perfect.



Arturo Castro put together this technical demo for face swapping technique.

The application works in real time and it's developed using the opensource framework for creative coding openFrameworks: openFrameworks.cc

Most of the "magic" happens with Jason Saragih' c++ library for face tracking. The face tracking library returns a mesh that matches the contour of the eyes, nose, mouth and other facial features.

That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson's image clone code: methodart.blogspot.com/​ gives it the blending effect that can be seen in the final footage.

Also uising Kyle McDonald's ofxFaceTracker addon for openframeworks github.com/​kylemcdonald/​ofxFaceTracker which wraps Jason's library for easier use.



Read story:
Watch This Dude’s Face Become Other FacesReal Time Face Substitution Should Be Renamed Real Time Horror Show