Video based air harp

From Hack a day:
http://hackaday.com/2011/12/12/get-ready-to-play-some-wicked-air-harp

Who needs a tactile interface when you can wave your hands in the air to make music? Air String makes that possible and surprisingly it does so without the use of a Kinect sensor.

In the image above, you can see that two green marker caps are used as plectra to draw music out of the non-existent strings. Judiciously perched atop that Analysis and Design of Digital Systems with VHDL textbook is a camcorder recording an image of the player. This signal is processed by an FPGA (hence the textbook) in real-time, and shown on the monitor seen to the right. A set of guides are overlaid on the image, so the player knows where to pluck to get the notes she is expecting.

The program is designed to pick up on bright green colors as the inputs. It works like a charm as you can see in the video after the break. The team of Cornell students responsible for the project also mention a few possible improvements like adding a distance sensor (ultrasonic rangefinder?) so that depth can be used for the dynamics of the sound.