Quidgets < 3 Gstreamer, and the Return of Photobomb

I'm finally settling back into the groove of some of my side projects. I guess I'm handling the new position a bit better as time goes on, and feel that I can spend some free time working on somethings that I want to do, not just things that I feel that I should do.

So, these side projects I do for fun, and they are the most fun when they combine together in sweet ways. During the dead of winter, I spent a bunch of weeks working on Photobomb . On of the features that I added was that you could add an image directly from your web cam. To do this, I used the PyGame Web Cam API, essentially because I saw the API, and knew that I would be able to use it relatively easily, which in fact turned out to be the case.

It also turned out that not everyone had PyGame already installed on their systems. As a result installing Photobomb meant a 25+ Megabyte download, most of which was PyGame. So I was advised to use GStreamer instead. I got started on this conversion back in April, by creating a simple web cam display application using gstreamer. I ran into a series of roadblocks, one such roadblock was removed by Chris Halse Rogers of desktop team fame, who knew why it kept crashing (basically, I was trying to access the xid of a widget that wasn't yet realized).

But I soon had a pipeline together that could display the web cam, but I could not figure out how to modify it so that it could save out a picture whilst still displaying the web cam output. I finally hopped into #gstreamer to see if someone could give me a pointer. Well, it turns out that someone already wrote a pipeline called caemerabin that does everything I need for the web cam, and more.

Well, it turned out that the documentation was out of sync with the current API. This isn't too surprising, as camerabin is still in gstreamer0.01-plugins-bad, and the API is actually improved by the changes. But I was struggling to understand camerabin, so I went back to #gstreamer. Often in IRC, someone will volunteer to spend some time helping you out with a problem. thiagoss (who I think might be this guy) really helped me out. I'm not sure, but I think he may actually be a primary author of camerabin. Anyway, he set me straight on a couple of things, namely:

1. use $gst-inspect camerabin to see what properties and methods the GStreamer elements really support (if they are out of sync with the docs).

2. use GST_DEBUG=2 to run your gstreamer apps, as this puts more warnings in your output.

Well, between these 2 tips, I was quickly able to realize that my WebCamBox widget would not much more than some Gtk/Gstreamer app boiler plate, with a wrapper around camerabin.

So, for example the "take picture" function just creates a time stamp, then tells the camerabin instance to emit a "capture-start" signal.

      stamp = str(datetime.datetime.now())
extension = ".png"
directory = os.environ["HOME"] + _("/Pictures/")
self.filename = directory + self.filename_prefix + stamp + extension
self.camerabin.set_property("filename", self.filename)
self.camerabin.emit("capture-start")
return self.filename

Then in on_message, I capture the message that it is done, and fire a signal:

       t = message.type
if t == gst.MESSAGE_ELEMENT:
 if message.structure.get_name() == "image-captured":
     #work around to keep the camera working after lots
     #of pictures are taken
     self.camerabin.set_state(gst.STATE_NULL)
     self.camerabin.set_state(gst.STATE_PLAYING)

     self.emit("image-captured", self.filename)

Play, Pause, and Start were trivially easy to implement:

        self.camerabin.set_state(gst.STATE_PLAYING)
self.camerabin.set_state(gst.STATE_PAUSED)
self.camerabin.set_state(gst.STATE_NULL)

Like I say, there is also some boiler plate to instantiate the camera and associate it with a gtk.DrawingArea. It took me a lot of iterations to get it working, as you can see from all of these pictures of me working on it ...

The net result is that it's now pretty easy to create an app with a web cam in it. Here's all the code for the WebCamBox test app.

if __name__ == "__main__":
"""creates a test WebCamBox"""
import quickly.prompts

#create and show a test window
win = gtk.Window(gtk.WINDOW_TOPLEVEL)
win.set_title("WebCam Test Window")
win.connect("destroy",gtk.main_quit)
win.show()

#create a top level container
vbox = gtk.VBox(False, 10)
vbox.show()
win.add(vbox)

mb = WebCamBox()
mb.video_frame_rate = 30
vbox.add(mb)
mb.show()
mb.play()

mb.connect("image-captured", __image_captured)
play_butt = gtk.Button("Play")
pause_butt = gtk.Button("Pause")
stop_butt = gtk.Button("Stop")
pic_butt = gtk.Button("Picture")

play_butt.connect("clicked", lambda x:mb.play())
play_butt.show()
mb.pack_end(play_butt, False)

pause_butt.connect("clicked", lambda x:mb.pause())
pause_butt.show()
mb.pack_end(pause_butt, False)

stop_butt.connect("clicked", lambda x:mb.stop())
stop_butt.show()
mb.pack_end(stop_butt, False)

pic_butt.connect("clicked", lambda x:mb.take_picture())
pic_butt.show()
mb.pack_end(pic_butt, False)

gtk.main()

Almost all of it is standard code for creating widgets. I love that doing functions like play, pause, stop, and take a picture can be handled in lambdas. So much easier!

So my last step was to drop it into Photobomb. All I had to do was modify the CameraPage class that I had already set up for the PyGame based version.

import gtk
from quickly.widgets.web_cam_box import WebCamBox
from ImageListPage import ImageListPage

class CameraPage(ImageListPage):
def __init__(self):
   gtk.VBox.__init__(self,False, 0)
   self.__camera = WebCamBox()
   self.__camera.connect("image-captured",self.image_captured)
   self.__camera.show()
   self.__camera.set_size_request(128, 128)
   self.pack_start(self.__camera, False, False)

   button = gtk.Button("Take Picture")
   button.show()
   button.connect("clicked", lambda x:self.__camera.take_picture())
   self.pack_start(button, False, False)


def image_captured(self, widget, path):
   self.emit("clicked",path)

def on_selected(self):
   self.__camera.play()

def on_deselected(self):
   self.__camera.stop()

Well, almost all I had to do. I discovered a bug where if the WebCamBox is not actually visible, it locks up Photobomb if you try to show it. For now, I've worked around this by putting the CameraPage as the first and open page, so it just works. However, I suspect the bug is due to the xid not being available when camerabin tries to display on it. I think with a little thought, I can block this condition, and perhaps not let the camera play if it's not ready yet.

Anyway, I ended up with a few lines of wrapper code, around my wrapper code, and it all works thanks to the efforts of the folks working on camerabin!

camerabin has a whole lot of functionality that I haven't wrapped yet. It takes video, including audio! Also, it looks like you can change encoders, and drop in filters and such into the pipeline. To handle this for now, WebCamBox exposes a "camerabin" public property, so if you are using the widget, you won't run into a wall.