Using PyGame in a PyGtk App

I've written a few games in pygame, but never quite finished them. Generally because the complexity of adding user interactivity for things outside the essential game play. While pygame is a sweet sprite library, it has no widget toolkit, so things like collection a string from the user, having menus, etc... are all incredibly tedious to program. Adding something like a high score screen is as much effort as game programming itself.

Pygtk, however, has a rich widget toolkit, but it's hard to use it from within a pygame app. This is difficult in part because Gtk needs a gtk.main loop, but a typical pygame app is using it's own loop with clock.tick(). As a result, there are threads locking each other out, and generally craziness ensues.

But, it turns out, you can embed a pygame surface within a pygtk app, and it's actually pretty easy. So you can use just the one gtk.main loop, all interupt programming, access to the whole Gtk toolkit, and also all the easy loading of images, playing of sounds, collision detection, and other sprite functionality of pygame. It's really the best of all possible worlds.

I wrote a minimal demo script to show how to do this. The "game" is just a single sprite that you can move with the keyboard. You can see the screen in the screenshot at the top of this post that there is a gtk menu being activated!

I didn't intend this to be a tutorail on pygame, but rather integrating pygame in pygtk, so all the "game" does is let you move a sprite around.

It's probably easiest to look at the whole script, but let me point out some of the steps involved. I'll assume that you've already set up a gtk.Window with your menus and such. So, the first step is to set up a gtk.DrawingArea that will become the pygame surface. It's normal code to add it to your window:

      da = gtk.DrawingArea()

    da.set_size_request(300,300)

    da.show()

    vbox.pack_end(da)

    da.connect("realize",self._realized)

This last line, connecting to the realize signal, is important. It's important because you use the gtk.gdk.window.id to associate the drawing area with pygame. The drawing area's window won't have an id until it's been rendered by gtk. So the then in the handler for the realize event, we set up the association with the drawing area, and also start actually drawing the game:

def _realized(self, widget, data=None):

    os.putenv('SDL_WINDOWID', str(widget.window.xid))   

    pygame.init()

    pygame.display.set_mode((300, 300), 0, 0)

    self.screen = pygame.display.get_surface()

    gobject.timeout_add(200, self.draw)

The first three lines essentially set up the drawing area as the pygame surface. The fourth line creates a global object for a pygame.screen object that will be used in the draw function. Finally, the draw function gets caslled every 200 milliseconds via a gobject timeout. This is as much easier way of setting up the game loop than the normal pygtk way of blocking the clock for a certain number of ticks, as gobject.timeout_add plays nicely with the gtk.main loop.

In a real game, you'd probably put an "update" function on the timeout to do things like check for collisions, random events, and etc... (You'd also probably use a much smaller time out period than 200 milliseconds). However, our game only has one sprite, and it doesn't do anything but move so we don't need that functionality. But how do we tell it move? In pygame, every time through the main loop you pull all the events off the event stack. With pygtk, you use signals to handle the input when it occurs.

To do this, in the __init__ function for the window, we set up our pygame objects, and then connect to the key-press signal:

      #set up the pygame objects

    self.image = pygame.image.load("sprite.png")

    self.background = pygame.image.load("background.png")

    self.x = 150

    self.y = 150



    #collect key press events

    self.connect("key-press-event", self.key_pressed)

Then in the signal handler, we manipulate those objects. Typically, you'd be storing the x and y coordinates in a sprite object, but since there is only one sprite to track in this "game" we can handle it more simply. So in the key_press handler, we just detect if an arrow key was pressed, and we adjust the game data appropriately:

def key_pressed(self, widget, event, data=None):

    if event.keyval == 65361:

        self.x -= 5

    elif event.keyval == 65362:

        self.y -= 5

    elif event.keyval == 65363:

        self.x += 5

    elif event.keyval == 65364:

        self.y += 5

Then, every 200 milliseconds, the timeout comes buy and tells pygame to draw everything using normal pygame functions:

def draw(self):

    self.screen.blit(self.background,[0,0])

    rect = self.image.get_rect()

    rect.x = self.x

    rect.y = self.y

    self.screen.blit(self.image, rect)

    pygame.display.flip()



    return True

Since pygame is getting update in gobject timeout and normal Gtk signal handlers, and not in it's own gameloop, gtk.main is free to run and handle menus, and even display dialogs:

A Photobomb Sale!

Photobomb showed up in the Software Center yesterday. As I prepare a space in my garage for the Rolls Royce I intend to buy from the proceeds, I took a moment to check in on sales, and, in fact, someone bought it! The MyApps portal has a really nice overview screen for managing multiple apps for sale.

Now, I strongly suspect that a friend bought it to make me feel better, but still, it gave me a chance to check out the nice graphs that myapps makes for developers. Of course, I expect Photobomb sales to stress test their math and charting capabilities due to an overwhelming crush of sales, but it's nice to see what they are shooting for.


I'm really impressed with the work that Canonical ISD has done here. I'd love to see this work made available to people who want to land non-commercial apps on a stable release of Ubuntu soon. Currently, it only supports commercial apps.

Speaking of which, while I am selling Photobomb, please keep in mind that it is still Free. That means that anyone who buys it gets all the software freedoms (it is GPL3, afterall). Furthermore, if you don't want to buy it, you can get Photobomb for free my PPA. Seriously, it you want to use it, but don't want to buy it, no worries, running it from my PPA is no problem for me.

Cha-ching!

So, today I uploaded a special version of Photobomb to my PPA. It's special because I consider this my first *complete* release. There are some things that I would like to be better in it. For example:

• I wish you could drag into from the desktop or other apps.
• I wish that the app didn't block the UI when you click on the Gwibber tab when Gwibber isn't working.
• I wish the local files view had a watch on the current directory so that it refreshed automatically.
• I wish inking was smoother.
• I wish you could choose the size of the image that you are making.
• I wish that you could multi-select in it.

But alas, if I wait until it has everything and no bugs, I'll never release.

So, I am releasing Photobomb in my PPA. It is a free app. You can use it for free, and it's Free software. So, enjoy.

However, I am *also* releasing it commercially into the software center. That means that you can choose to install from my PPA for free, or you can buy it from the Software Center. I guess the only real difference will be that I could break it in my PPA, and I won't generally plan to provide lots of support, but if you bought it, I'd feel a bit more like I should strive to fix your bugs and stuff.

The code is GPL v3, so people can enhance it, or generally do whatever they think is useful for them (including giving it a way, or using it to make money).

I found it remarkably easy to submit photobomb to the Software Center. I just used the myapps.ubuntu portal, and it all went very smoothly. Really just a matter of filling in some forms. Of course, since I used Quickly to build Photobomb, Quickly took care of the packaging for me, so that simplified it loads.

I'll keep everyone posted on how it goes!

The End of an Era?

Today I committed and pushed a change to Photobomb to remove functionality! The web tab used to allow users to search the web for images. So you could search for "cat" and get the most popular kitteh pix ready to mix into our images. No longer. You can still plug in a web address, and Photobomb will it's best to find an image there, but no more web searches.

A couple of weeks ago, Google retired the Google API I was using, so I switched to Yahoo!, only to find that the Yahoo! one got retired a week or so ago.

Both services switched to newer APIs. The thing is, both Google and Yahoo! now charge an application for using those APIs. Yup, I would have to pay actual money to use the services.

I really don't have a problem with them doing that. I mean, someone has to pay to keep those services running. And I suppose if I charged a bit for Photobomb, I could use that to fund paying for the services. However, at the moment, I don't really feel like dealing with setting up an account and dealing with all that, in addition to recoding to the new APIs.

So, I guess the era of the web giants competing to get me to use their "free" services has drawn to a close. I suppose it's a bit like a bursting bubble. Goodbye free web-services :/

Coding Copy and Paste Functionality

Copy and Paste Features
A well designed application should probably support Copy and Paste. While it sounds simple, it actually entails a few features.

Copying from Your Application to Other Applications
This means that your application should offer up data formats that other applications can read. So a user should be able to copy in your application, and paste into another one. This may entail converting your internal data format into a new one that other programs will expect.

Copying from Other Applications into Your Application
This means reading other applications data formats and converting it your applications internal representation.

Copying and Pasting within Your Application or between Instances of Your Application
This entails marshaling all the data that your application needs. If a user can run multiple instances of your Application, it won't be sufficient to simply copy an instance of an object, since one instance won't have access to what is in the memory of another instances.

How Does Copy and Paste Work?
You start out by accessing the desktop's clipboard. Then, you tell that clipboard what kind of data your app can put on the clipboard, and then what kind of data your app can take off of the clipboard. Then, you right some functions to handle the following situations:

If the user selects a Copy command in your application, you tell the clipboard that data is available (but you don't actually put the data on the clipboard until the user selects paste).
If the user selects Paste in your application, you ask the clipboard for the data, and the application with the data then supplies it to the clipboard. Sometimes the application supplying the data is your application, and sometimes it is a different application.
If the user selects Paste in a different application, then the clipboard might ask your application to supply the data.

Set Up Variables
So, let's get to the code. First thing, we need to create a reference to desktop's clipboard, and also create a variable to store a reference to whatever item may be copied.

     self.clipboard = gtk.Clipboard()
self.clipboard_item = None

I put this in finish_initializing.

Handling the Copy Command
Then I wrote a function called "copy_menu" which is called when the user selects "Copy" fromt he menu. the first thing this function does is tells the clipboard what kinds of data is supported by passing in a list of tuples. Each tuple has a string that specifies the type, info regarding drag and drop, and a number that you can make up to specify what the type again. For Photobomb, only the strings are important. You need to use types that are commonly understood on the Linux desktop so that apps know that you can give them data they care about. I chose "UTF8_STRING" for putting strings on the clipboard, and the mime type "image/png" for images. For example, Gedit recongizes UTF8_STRING, and Gimp recognizes image/png, so Photobomb can copy data into both of those apps. Only Photobomb recognizes "PhotobombItem", of course. These strings are called "targets" in Gtk.

After specifying what data types you will support, you then tell the clipboard about those data types, along with what function to call if the user tries to paste, and what function to call when the clipboard gets cleared. Finally, the function stores a reference to the currently selected item. It's important to store this reference seperately, as the selection could change before the user pastes.

   def copy_menu(self, widget, data=None):
paste_data = [("UTF8_STRING", 0, 0),
        ("image/png", 0, 1),
        ("PhotobombItem", 0, 2)]
self.clipboard.set_with_data(paste_data, self.format_for_clipboard, clipboard_cleared)
self.clipboard_item = self.selected_item  

Handling Programs Pasting from Photobomb
After telling the clipboard what kind of data you can put on it, the user may try to actually Paste! If this happens, you have to figure out what kind of data they are asking for, and then put the correct kind of data onto the clipboard. To figure out what is being asked for, you check the target. This info is stored in the selectiondata argument, that gets passed in.

Putting Text on the Clipboard
When an application is asking for a string, life is really simple. Every PhotobombItem has a text property, so it's simple to simply set the text of the clipboard with this data.

   def format_for_clipboard(self, clipboard, selectiondata, info, data=None):
if selectiondata.get_target() == "UTF8_STRING":
self.clipboard.set_text(self.clipboard_item.text)

Putting an Image on the Clipboard
But what about when the app has asked to paste an image? The clipboard only supports text, so you can't paste an image, right? Actually, the contents of any binary stream or file can be converted to text, and that text can be put on the clipboard. However, you don't want to simply set the text for the clipboard, as binary data can have characters in it that will confuse a program that thinks it's a normal string. For example, when you read in a file of binary data, you can set that to a string in Python, but other programs not written in Python will probably get have errors if they try to treat it as a string type.

In fact, the clipboard will reject text that is binary data. So, if you have binary data, you need to:

1. convert it into text
2. set the selectiondata with the proper target

Photobomb has a function called "image_for_item" that returns image data as binary text, so Phothobomb can just call that function and put the resulting text into the selection data. The "8" in the second parameter tells the selectiondata object essentially boils down to text.

     if selectiondata.get_target() == "image/png":
  selectiondata.set("image/png", 8, self.image_for_item(self.clipboard_item))

Putting a PhotobombItem on the Clipboard
If an instance of Photobomb wants to paste, we need to provide enough information to create a proper PhotobombItem. A PhotobombItem knows a lot about itself, for example, it's clipping path, it's rotation, it's scale, etc... When copyng and pasting within Photobbomb or between instances of Photobomb, we don't want to lose all of that data.

So how do we paste a PhotobombItem if we can only put text on the clipboard? Python has built in support for turning objects into strings. This is called "pickling". So, in the simplest case, we could just convert the Python object to a string, put that string onto the clipboard, and then when convert it from the string back to a Python object.

So, first, import the pickle module:
import pickle

Then you can use "dumps" to dump the object to string:

   if selectiondata.get_target() == "PhotobombItem":
 s = pickle.dumps(self.clipboard_item)
 self.clipboard.set_text(s)

Sadly, this doesn't work for Photobomb because the GooCanvas classes that PhotobombItems derive from don't support pickling. So we can't do this in the simple way. However, what we *can* do, is create a dictionary with enough information to create identical PhotobombItems, and then pickle that dictionary. For photobomb, this gets a bit annoying complex and is not relevant to the blog posting, so I snipped out most of the code. You can always look at the full code on launchpad, of course.

This is, unfortunately, a lot more involved. However, it works just fine:

   if selectiondata.get_target() == "PhotobombItem":
 d = {"type":type(self.clipboard_item)}
 d["clip"] = self.clipboard_item._clip_path
 #snipped out all the other properties that need to be addeed to the dicitonary
 s = pickle.dumps(d)
 self.clipboard.set_text(s)

You don't have to use pickling. For example, you could represent this dictionary in JSON instead. However, pickling is the native serialization formatting for Python, so it's easiest to use it when you can.

Now Photobomb can support copying and pasting into text editors, like Gedit, and image editors, such as the Gimp, as well as into Photobomb itself.

Handling Pasting in Photobomb
The essential paste function is run when the Photobomb user uses Photobomb's Paste command. This function tries to figure out if there are any interesting data types on the clipboard, and then uses them. Typically, such a function should ask for the richest and most intersting data it can handle first, and then less rich data types in order. For Photobomb, the richest data is a PhotobombItem, or course. Then an image, and then finally text. You may have noticed above that when a user pastes from a clipboard, it runs a function in some application. There is no gauruntee that these functions will be fast. For this reason, it's best to use the asyncronous "request" methods on the clipboard.

So, the paste function simply tries to figure out which is the best data type to paste, if any, and then asks the clipboard to call the correct function when the data is ready.

 def paste_menu(self, widget, data=None):
targets = self.clipboard.wait_for_targets()
if "PhotobombItem" in targets:
  self.clipboard.request_contents("PhotobombItem", self.paste_photobomb_item)
  return
for t in targets:
  if t.lower().find("image") > -1:
    self.clipboard.request_image(self.paste_image)
    return
for t in targets:
  if t.lower().find("string") > -1 or t.lower().find("text") > -1:
    self.clipboard.request_text(self.paste_text)
    return

In the cases where another applications has put an image or text onto the clipboard, it's easy to use functions already built into Photobomb to paste those data types. Notice that the return functions already have all the data ready in the form of text or the pixbuf.

   def paste_text(self, clipboard, text, data=None):
  self.add_new_text(text)
def paste_image(self, clipboard, pixbuf, data=None):
  self.add_image_from_pixbuf(self, pixbuf)

When an instance of Photobomb has put the data on the clipboard, it's a bit more complex, because Photobomb has to de-serialize the dictionary, and then create the proper item. Again, I snipped out most of the code for using the de-serialized dictionary, because it just unnecessarily complicated the sample:

   def paste_photobomb_item(self, clipboard, selectiondata, data=None):
  item_dict = pickle.loads(clipboard.wait_for_text())
  if item_dict["type"] is PhotobombPath:
    new_item = PhotobombPath(self.__goo_canvas,
                item_dict["path"],
                item_dict["width"], None)
  #snip out all the code to handle all of the other properties
  new_item.set_clip_path(item_dict["clip"])
  new_item.translate(10,10)
  new_item.set_property("parent", self.__root)
  self.z_order.append(new_item)
  self.add_undo(new_item, None)

Persisting the Clipboard Item
You may have noticed that an item isn't actually put on the clipboard until the user tries to paste. This is a great optimization, because pasting complex or big things can take a lot of time, and there is no point on the user waiting for a copy command to finish if they are never going to actually paste with it. However, what if the user selects "Copy" in your app, and then quits your app? How is the item going to get pasted?

Gtk handles this by letting you "store" data on the clipboard. This can actually be used at any time in your application, but to handle this particular situation, Photobomb just includes a few lines in it's on_destroy function. This causes the desktop's clipboard to store all of the data types, so all the functions for servicing a paste command get called, and the desktop's clipboard holds onto the data as long as it needs.

   def on_destroy(self, widget, data=None):
   """on_destroy - called when the PhotobombWindow is close. """
   #clean up code for saving application state should be added here
   paste_data = [("UTF8_STRING", 0, 0),
      ("image/png", 0, 1),
      ("PhotobombItem", 0, 2)]
   self.clipboard.set_can_store(paste_data)
   self.clipboard.store()
gtk.main_quit()