Python and recovering Firefox sessions

First let me confess, I am a tab-aholic…

I tend to have several Firefox windows open at once, with several tabs in each. And I leave them open for… well… weeks. And usually that’s fine – it restores everything if I have to reboot or restart Firefox, and all is well. Except when it doesn’t.

Sometimes, for whatever reason, the sessions aren’t restored, and all of those tabs, as in “hey, I wasn’t quite done with that!”, are lost.

When this happened to me last night, I took a look in my Firefox profile directory and discovered something about the way Firefox stores it’s sessions. I’m sure it’s well-known to many, but it was news to me.

When I looked at sessionstore.bak (the automatically created backup of sessionstore.js) I saw something like:

({"windows":[{"tabs":[{"entries":[{"url":"http://stoa.canterburyschool.org:9080
/stoa/courses/Faculty/TechnologyGuides/VoIPandSkypeinForeignLanguage
/document_edit_form","title":": VoIP and Skype in Foreign Language","ID":2528274837}...

Hmmmm... I know that's Javascript, but it also looks a lot like a Python dictionary. Why not see if Python could eval() it to a dictionary? My first try failed on the unknown identifiers "true" and "false", but after defining true and false as equal to Python's True and False, it worked. Now I had a nested set of dictionaries containing all of my previous session's info. Not bad.

[Edit: Not being a regular JSON user, and since the file extension was .js, not .json, I never considered that this might be JSON. It turns that it's not QUITE JSON and the Python JSON library won't decode it (without at least stripping off the leading and trailing parentheses). Check out this discussion for more info.]

From there it was easy to create some code that would grab the URL's from an unrecoverable session and put them into an HTML page for easy access later.

Warning: this code uses eval() on arbitrary strings loaded from an arbitrary file, so it is not secure - use with great caution. If it wipes your hard drive, sells your house and absconds with the money, and kicks your dog on the way out, you have been warned.

In any case, below is my session recovery code, which I've already used a couple of times.

---

import sys
""" this code is released to the public domain and meant only for illustration. It is not
    warranted as being safe or suitable for any particular purpose at all.

    in particular, this program evaluates (and potentially executes) arbitrary code,
    so use with caution.
"""

def get_session(infilename, outfilename):
    true = True
    false = False
    sessionstring = open(infilename).read()
    session = eval(sessionstring)

    outfile = open(outfilename, "w")
    for window in session['windows']:
        outfile.write("---------------Window-----------------<p>\n")
        outfile.write("<ul>\n")
        for tab in window['tabs']:
          outfile.write( '<li><a href="%s" \="">%s</a> - %s</li> \n' %
                         (tab['entries'][0]['url'],tab['entries'][0]['url'],
                          tab['entries'][0]['title']))
        outfile.write("</ul>\n")
    outfile.close()

if __name__ == '__main__':
    infile = sys.argv[1]
    if len(sys.argv) == 3:
        outfile = sys.argv[2]
    else:
        outfile = "sessionsave.html"

    get_session(infile, outfile)

Robots and Python, first in a series

It must just be something in the air, but robots have become much more a part of my life the past couple of months. First of all, I bought an iRobot Create to experiment with for possible use in class. If you’re not familiar with the Create, it’s really a Roomba with the vacuum removed and an additional serial interface with can be used with an optional control unit or to interact with various sensor inputs.

I was so  impressed with the Create that I took advantage of a black Friday two-for-one sale and got both an interior Roomba and the shop-vac version. We’ve been fairly happy with those two as they’ve bounced around the house, picking up some of the mountains of dog hair generated by our three aussies.

In  the meantime, I was teaching a C programming elective at school and we decided to try to use an Arduino and some of the hardware from an old kit to create a line following robot. It wasn’t pretty, but after several struggles, they got it working and made it follow a line.

So all of this robotic fooling around gave me an idea. I wanted to mount a computer and web cam on the Create and see if I could drive it using Python. Then, I would see if we could use it in teaching at all.

My first idea was to use an eee 900 that I had. It was having hard drive (SSD) problems, but it would run just fine off of a USB stick with Ubuntu on it, and it already had a web cam.

My eee pc 901 perched on the iRobot Create.

To keep things simple, I’d just enable remote desktop sharing, use a VNC viewer to connect, and run Cheese to see the camera output. That would give me visuals from the bot’s point of view.

The Create (and most later Roomba’s) have a serial interface for controlling them, which takes a somewhat unusual connector. Fortunately that correct cable was included with the Create, going from standard 9-pin serial to the bot. Of course, the eee doesn’t have a serial port, but an old Belkin USB to serial converter I had from my Palm and Visor days seemed to make a connection. However, my attempts to use a terminal program (Miniterm) got nowhere. I’d see the bot boot up, but it ignored all of my commands. The problem was that the terminal program was sending the numeric command codes as text, not numbers. Finally, I switched to using Python and the serial module and the bot reacted.

However, controling the bot with raw codes is no fun so instead, I used iRobot’s opeinterface.py to connect to the bot. This took some tweaking of the code, since it was looking only for Windows style numbered comm ports rather than Linux device files. But once that was fixed, I could ssh to the eee, connect to the bot and run Python, and drive the bot from my desktop.

It really didn’t take long to put everything together. In the course of an evening, I had the eee perched on the Create, everything hooked up and a remote connection. Since I was still pretty ignorant of the Create’s operation, I just used the Python shell, imported openinterface.py and confined myself to the drive_straight(), stop() and turn_in_place() methods.

Driving was a bit cumbersome. For one thing the screen updates were slow, and for another, the weight of the eee was enough to give the Create just a little bit of trouble in getting over thresholds, carpet edges, etc. In any case, I managed to drive it from one room to another, much to my amusement and the puzzlement of the dogs. Seeing a giant dog snout fill up the field of view was enough to make me lose control and crash into a dog crate.