Python webcam theremin

josomebody · 2010-12-29 20:26:11

I hacked together some code last night to use my webcam as kind of a theremin. It borrows heavily from a couple of tutorials dealing with the pygame camera module and a technique to generate a tone and dump it into a pygame sound object. It's really dirty right now, but it works. If anybody wants to play with it or improve upon it I'd be happy to post the code and my sources it's based on.

igor_b · 2010-12-29 20:32:06

code would be nice

josomebody · 2010-12-29 20:57:07

http://lists.canonical.org/pipermail/kr … 00465.html
http://www.pygame.org/docs/tut/camera/CameraIntro.html
I borrowed very heavily from the above sources to get started.

camtrack.py

import pygame
import pygame.camera
from pygame.locals import *

pygame.mixer.pre_init(22050, -16, 1, 4096)
pygame.init()
pygame.camera.init()


class Capture(object):
    def __init__(self):
        self.size = (640,480)
        self.display = pygame.display.set_mode(self.size, 0)
        self.clist = pygame.camera.list_cameras()
        self.cam = pygame.camera.Camera(self.clist[0], self.size)
        self.cam.start()
        self.snapshot = pygame.surface.Surface(self.size, 0, self.display)
        self.thresholded = pygame.surface.Surface(self.size, 0, self.display)

    def get(self):
        if self.cam.query_image():
            self.snapshot = self.cam.get_image(self.snapshot)
            self.snapshot = pygame.transform.flip(self.snapshot,True,False)
        mask = pygame.mask.from_threshold(self.snapshot, self.ccolor, (30,30,30))
        self.display.blit(self.snapshot, (0,0))
        connected = mask.connected_component()
        if mask.count() > 100:
            coord = mask.centroid()
            pygame.draw.circle(self.display, (0,255,0), coord, max(min(50,mask.count()/400),5))
        else:
             coord = (0,0)
        pygame.display.flip()
        return coord

    def calibrate(self):
        self.snapshot = pygame.transform.flip(self.cam.get_image(self.snapshot),True,False)
        self.display.blit(self.snapshot, (0,0))
        crect = pygame.draw.rect(self.display, (255, 0, 0), (145,105,30,30), 4)
        self.ccolor = pygame.transform.average_color(self.snapshot, crect)
        self.display.fill(self.ccolor, (0,0,50,50))
        pygame.display.flip()

That module deals with the camera, then the main file is as follows:

from camtrack import *

import pygame, time, random, Numeric, pygame, pygame.sndarray
sample_rate = 44100

pygame.sndarray.use_arraytype('numeric')

def sine_array_onecycle(hz, peak):
    "Compute one cycle of an N-Hz sine wave with given peak amplitude."
    length = sample_rate / float(hz)
    omega = Numeric.pi * 2 / length
    xvalues = Numeric.arange(int(length)) * omega
    return ((peak * Numeric.sin(xvalues))+(peak/2*Numeric.sin(2*xvalues))).astype(Numeric.Int16)
def sine_array(hz, peak, n_samples = sample_rate):
    """Compute N samples of a sine wave with given frequency and peak amplitude.

    Defaults to one second."""
    return Numeric.resize(sine_array_onecycle(hz, peak), (n_samples,))
def play_for(sample_array, ms):
    "Play given samples, as a sound, for N ms."
    sound = pygame.sndarray.make_sound(sample_array)
    sound.play(0)
    return sound

c = Capture()
quit = False
c.calibrate()
sound = []
for i in range(4):
    sound.append(play_for(sine_array(1,0),44100))
i=0
j=3
while quit == False:
    framestart = pygame.time.get_ticks()
    for event in pygame.event.get():
        if event.type == KEYDOWN:
            if event.key == K_ESCAPE:
                quit = True
            elif event.key == K_c:
                c.calibrate()
    f, amp = c.get()
    f *= 880.0/320
    amp *= 65536/960
    if f>0 and amp>0:
        sound[j].stop()
        sound[i] = play_for(sine_array(f,amp,44100),1)
        j=i
        i+=1
        if i>=4:
            i=0
    frameend = pygame.time.get_ticks()
    frametime = frameend-framestart

If you run the second file, your cam should immediately come up at sound should start playing. There will be a green blob on the display that may or may not be moving. If you hit 'c', a red box will come up toward the upper left, and whatever's in that box will get tracked. You can hit the 'c' key over and over to get the right object tracked.
Up and down controls the volume, the lower the louder, and left to right controls the pitch, the farther to the right, the higher. I've found that it's easiest to make it track a pen light or a highlighter of a weird color, but your hand should work if you can get far enough away, are wearing a long-sleeved shirt, and there's nothing else in the image too close to your skin tone.

This clode is really sloppy, sorry. It does run on my machine though.

igor_b · 2010-12-29 21:12:30

cool! it works! thanks!

josomebody · 2010-12-29 21:14:04

I've been trying to figure out a good way to smooth out the tone, but other than magically making it run faster I haven't figured anything out yet.

greenmanwitch · 2011-01-01 00:46:58

Sufiĉe mojosa, dankon.

Pretty cool, thanks.

linkmaster03 · 2011-01-01 07:16:02

So, what exactly does this do? A description would be helpful.

greenmanwith: Hura, alia esperantisto! Estas bonege vidi unu el ni ekster ##esperanto@freenode.

REDAKTO: Fek al vi, kaja. Eliru tuj.

Last edited by linkmaster03 (2011-01-01 09:44:28)

josomebody · 2011-01-01 18:38:00

So, what exactly does this do? A description would be helpful.

It's a type of theremin, a musical instrument. http://en.wikipedia.org/wiki/Theremin http://eo.wikipedia.org/wiki/Teremino

This one uses your webcam instead of antennae, tracking a designated object instead of using electrical capacitance between the antennae and your hands. The horizontal position of the designated object on camera controls the pitch, and the vertical position controls the volume.

linkmaster03 · 2011-01-02 04:12:15

Awesome! I tried it and it worked (requires python2, python-pygame, and python-numeric for those wondering). The tone, as you said, is a bit choppy, but not a bad proof of concept. Great idea.

Last edited by linkmaster03 (2011-01-02 04:12:42)

Arch Linux

#1 2010-12-29 20:26:11

Python webcam theremin

#2 2010-12-29 20:32:06

Re: Python webcam theremin

#3 2010-12-29 20:57:07

Re: Python webcam theremin

#4 2010-12-29 21:12:30

Re: Python webcam theremin

#5 2010-12-29 21:14:04

Re: Python webcam theremin

#6 2011-01-01 00:46:58

Re: Python webcam theremin

#7 2011-01-01 07:16:02

Re: Python webcam theremin

#8 2011-01-01 18:38:00

Re: Python webcam theremin

#9 2011-01-02 04:12:15

Re: Python webcam theremin

Board footer