Command your shiny Android in Python

I am the happy owner of an Android phone. I love it. But what I love even more, is rolling some sweet Python on my precious.

In case you have a vanilla phone like I do, you will need to follow these steps:

  • download and install the Android SDK for your platform
  • add the SDK tools to your system path, for instance on Windows run this in a shell:
> set PATH=%PATH%;path_to_android_sdk\tools
  • activate USB debugging on your phone (Parameters > Applications > Development > USB Debugging)
  • setup your system to detect your device as explained here (on Windows, it means installing the usb driver from the SDK)
  • check that your phone is detected correctly by running adb get-state (if everything is ok, the result should be ‘device’)
  • check out the Android Scripting Environment svn repository on your computer (see this page)

Your system is almost configured, let’s install the necessary applications on the phone:

  • install the Android Scripting Environment on your phone
  • (optionally) install the Text-to-Speech library (by Charles Chen) on your phone if you want to use it in your scripts, via the Android Market
  • run ASE on your phone, type menu and start a Python shell, you will notice a line like ‘export AP_PORT = “49508”‘.
  • setup port forwarding on your system with the value you just noted:
> set AP_PORT=49508
> adb forward tcp:%AP_PORT% tcp:%AP_PORT%
  • go to the ASE directory on your computer containing the module
> cd path_to_ASE\python\ase
  • run your favorite Python 2.6 interpreter and enjoy the magic:
> python
Python 2.6 (r26:66721, Oct  2 2008, 11:35:03) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import android
>>> droid = android.Android()
# now your phone is ready to acknowledge your awesomeness
>>> droid.speak('hello, my master')
{u'result': None, u'id': 2, u'error': None}
# or alternatively, if you don't have the TTS library installed:
>>> droid.makeToast('hello, my master')
{u'result': None, u'id': 1, u'error': None}

Now you have all the information you need to start exploring the API and roll  more interesting examples.


Python Idioms: + versus join

I was told to use ”.join([]) instead of the ‘+’ operator in Python. However a (bad) benchmark showed ‘+’ to be a lot faster. I think it is reasonable to say that in some cases ‘+’ is faster, here is my test:

def test0(b, c, d, e, f):
    for i in xrange(10**7):
        a = b + c + d + e + f

def test1():
    l = ['hello ', 'world ', 'with ', '+ ', 'operator']
    for i in xrange(10**7):
        a = ''
	for j in l:
            a += j

def test2():
    l = ['hello', 'world', 'with', 'join', 'function']
    for i in xrange(10**7):
	a = ' '.join(l)

test0('hello ', 'world ', 'with ', '+ ', 'operator')

And the result of the test:

$ python -m cProfile -s cumulative
hello world with + operator
hello world with + operator
hello world with join function

   10000007 function calls in 14.968 CPU seconds
   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    6.838    6.838    6.838    6.838
        1    2.683    2.683    5.113    5.113
        1    3.016    3.016    3.016    3.016

So clearly the worst way of using ‘+’ is when iterating over a list of strings and accumulating the concatenations in a variable (function test1). But there is nothing wrong with performing multiple ‘+’ operations in a single line and then storing the result in a variable (function test0).

A quick look at the bytecode of the function confirms this intuition, we can see a bunch of LOADs and ADDs and only one STORE:

>>> import dis
>>> dis.dis(test0)
             19 LOAD_FAST                0 (b)
             22 LOAD_FAST                1 (c)
             25 BINARY_ADD          
             26 LOAD_FAST                2 (d)
             29 BINARY_ADD          
             30 LOAD_FAST                3 (e)
             33 BINARY_ADD          
             34 LOAD_FAST                4 (f)
             37 BINARY_ADD          
             38 STORE_FAST               6 (a)

The test was performed with Python 2.5.4 on a Debian sid. Would be nice to see if the results hold for new versions of the Python interpreter.

UPDATED: Don’t use set.add() in python when running cProfile

UPDATE: As pointed out in the comments, my original conclusion was wrong and is due to the effect of the profiler on the performance. When using the time command instead with Python 3.2 on OS X Lion (averaged over 3 runs), the version with set.add takes 1.37s versus 4.85s for the version with set union (results are 2.17s and 7.12s respectively with Python 2.7.1). Sorry for the “d’oh” moment. Take-home lesson: use a profiler to count events, not to time them.

Use the union operator instead, it’s 2 to 3x faster on my machine.

$ cat
def mister():
    s = set([1,2,3])
    for i in range(10**7):

def hankey():
    s = set([1,2,3])
    for i in range(10**7):
        s |= set([1])

$ python -m cProfile
10000008 function calls in 28.257 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
 1 11.507 11.507 22.978 22.978 1 5.138 5.138 5.278 5.278

(tested on Python 2.5, 2.6 and 3.1 on Windows and Linux)

Python XML for Real Men Cheat Sheet

Howdy again,

Now let’s do some serious XML output in Python using cElementTree.

# this is included in Python 2.5+
from xml.etree.cElementTree import ElementTree, Element, dump

# let's create the root element
root = Element("teabag")

# give it a child with an attribute
child1 = Element("spam")
child1.attrib["name"] = "value"

# and a child with text content
child2 = Element("eggs")
child2.text = "spam and eggs"

# print the whole thing to stdout

# or to a file

See the author’s website for downloads and usage information of cElementTree.

Using this API, I was able to create a 47 Mb XML file in a few minutes, burning roughly 300 Mb of heap space. This XML file represents a graph of graphs, namely the control flow graph of each function of IDA Pro. Here are some screenshots, using yEd for the visualization part:

Python XML Cheat Sheet

[UPDATED] Finally xml.dom.minidom sucks balls, it can burn up to hundreds/gigs of megabytes of sweet memory when working with “large” xml files (10 Mb or more). See this post for a really lightweight implementation.


Here is a quick reference of how to create an XML document and output it in Python.

import xml.dom.minidom

# create the document
doc = xml.dom.minidom.Document()

# populate it with an element
root = doc.createElement("teabag")

# time to give some children to the root element, one with an attribute for instance
child1 = doc.createElement("spam")
child1.setAttribute("name", "value")

# and another one with some text
child2 = doc.createElement("eggs")
text = doc.createTextNode("spam and eggs!")

# let's get the output, as a string
print doc.toprettyxml()

# you're supposed to get the following output:
#<?xml version="1.0" ?>
#    <spam name="value"/>
#    <eggs>
#        spam and eggs!
#    </eggs>

How nice is that ? Yep, a lot.