PySnippet

Open links from Text-elements in QtQuick

2011-10-12T10:49:00.000+02:00

I just finished yet another blog post on my companies blog site. It's about opening external application when clicking on links in QML Text-element.

Read about it here

Using QtCreator for Arduino development

2011-09-22T15:34:00.000+02:00

I just finished a blog post on my company's blog site about using QtCreator as development environment for the Arduino board.

Please head over here to read all about it.

System 76: Linux Devices and Better Business Values

2011-09-03T10:39:00.000+02:00

Guest post by Nadia Jones. Look at the end of the post for more info about Nadia.

Tothe average computer user there are two types of operating systems:Apple and Microsoft. While for the mass market this basically standstrue, computer programmers, developers, and techies also recognizeLinux as one of the great computer operating systems. Though Linux isnot as widespread or popular as Apple or Microsoft, it is just aswell-respected (if not more so). There are several reasons that Linuxjust hasn't become a mass success like other operating systems have.While there are very few companies that produce computers with theLinux OS already installed, System76does.

Notonly is System76 one of the few companies that sells laptops withLinux preinstalled, they are one of the onlycompanies that sells them at reasonably competitive prices. No doubt,these computers will likely cost more than your average Dell becauseSystem76 is such a small outfit, but the prices remain extremelyreasonable. Selling laptops, desktops, and servers all outfitted withUbuntu, System76 is a firm believer in Linux as an operating system.As System76 explains, Linux offers stability, security, and the"freedom to use the software you like, the way you like to useit". One of the other big perks about using Linux and all of itsmachines is that they are completely free. System 76 will provide youwith preinstalled applications such as open office suite, Totem andXine movie players, GIMP image manipulation, and several others.

Moreover,System76 is unique in that they actually do something with yourdevice before they send it to you. Rather than simply installingUbuntu on the laptop you order and then sending it to you, theSystem76 people give you their own drivers and special configurationsto make your machine run as smoothly and flawlessly as possible.Along with special attention paid to each individual product theyship out, System76 also pays special attention to each of theircustomers. Because they are such a small company, they are able tooffer exceptional customer service. If there is any sort of softwareor hardware issue, you can email the company and you will receive apersonal email back from someone involved with building yourcomputer.

Whilenot everything about these computers and this company is going towork impeccably, their focus on customer service and better businessis noteworthy. System76 offers a unique system with customizableapproaches. Furthermore, System76 is dedicated to eco-friendlybusiness values, making them not only reliable, but also current andforward thinking. For Linux enthusiasts and newbies alike, System76is certainly worth a look-see.

AuthorBio:

Thisis a guest post by Nadia Joneswho blogs at onlinecollege about education,college, student, teacher, money saving, movie related topics. Youcan reach her at nadia.jones5 @ gmail.com.

tvmatchen for Nokia N9

2011-08-13T09:26:00.010+02:00

I just released a video of my port of tvmatchen for Nokia devices (Symbian^3 and now N9). Tvmatchen is a tv-guide application for live sports and is only available in Swedish.

NokiaGadgets actually wrote a couple of lines about the application, even though the app is in Swedish. The review was very positive and the author wants to see more apps looking like this one. It does feel good in a coders heart when someone writes something positive about your app :)

Link to the article.

And of course the video

Nokia N9 and Python

2011-06-22T09:13:00.002+02:00

Yet another great day to be a python coder! The new Harmattan/MeeGo device, Nokia N9, will be packed with support for Python!

For more info, look here

Yes, it seems that you can publish your python apps on ovi... Nokia Store too.

QtQuick - From Bling Bling To Blink Blink

2011-06-19T14:44:00.000+02:00

Probably most of you already know that QtQuick is used to create amazing blingalicous UIs without too much effort. The declarative QML language makes it very easy to create object instances and setup property bindings, animation, states etc. Lately I’ve been tinkering with my Velleman K8055 USB experiment board and wanted to create a nice UI to control the board. It didn’t take too long until I was thinking a little bit out side of the box and started experimenting with controlling the board with QtQuick but without a UI. I created a couple of QML components on the C++ side to control the input/outputs of the board and quickly realized the possibilities QML provides even for non-UI applications.

Basically what I did was to create a declarative engine and not a declarative view. By doing this you can have a QML file interpreted and access to the object instances created by the engine from within your C++ code. I see a lot of potential and possibilities, one thing that comes to my mind straight away is to utilize QML as a Dependency Injection container :)

QDeclarativeEngine *engine = new QDeclarativeEngine;
QDeclarativeComponent component(engine, QUrl("qrc:/example1.qml"));
Board *board = qobject_cast<board *>(component.create());

Example code to interpret a QML file

The source code is available at k8055tinker

Check out the video below which demonstrates my code:

I've also tagged this post with Python because it can be of interest for the planet python readers since this can also be accomplished with PySide.

I wish that I had Jessie's girl...

2011-05-21T12:42:00.000+02:00

Rick Springfield had a great hit with a song called Jessie's Girl and the same track is now used for the Nokia N9 teaser ad which popped up a couple of days ago.

I really hope this is a MeeGo device, or to be honest, I just hope it's a Linux + Qt/QtQuick enabled device with some MeeGo compliance.

I've had quite a lot of fun with QtQuick lately (both for desktop and my N8) and I really enjoy using it. It just open so much possibilities and lets you be creative.

Another thing that would be awesome is if Nokia actually puts PySide, the Nokia sponsored Qt-bindings for Python, on the handset. Having Python as a rapid back-end language combined with QtQuick for the view is just too sweet :)

Grr... I hate not knowing, but I just have to wait and see.

I won the "Travelling salesman" competition!

2011-04-20T15:58:00.000+02:00

Nokia held a mobile app competition at its Nordic blog which I won!

For more info, look here

Tips! Kdenlive - A great video editor

2011-04-13T13:08:00.000+02:00

I've just started to take a look on how to editing videos and was searching the net to find some tools for Linux.

After trying a couple I found out about the KDE application Kdenlive from some blog post and I must say that this tools looks very promising. It did crash once when I tried it out but I can live with that since I'm not a heavy user of video editing. I just need something for creating youtube videos occasionally.

In addition to the great app there's also a bunch of video tutorials here to help you getting started.

So if you're in the same position as me I think you should take a look at Kdenlive

PySide and QML

2011-03-29T09:38:00.000+02:00

It has been out there for a while but I haven't had time to try it out, PySide 1.0 with QML support!

This really gives you a über rapid app development environment. The power of Python combined with the awesomeness of the declarative language QML from the Qt framework.

Take a look and feel free to play with the example below:


from PySide import QtCore
from PySide import QtGui
from PySide import QtDeclarative

class Message(QtDeclarative.QDeclarativeItem):

    messageChanged = QtCore.Signal()

    def __init__(self, parent = None):
        QtDeclarative.QDeclarativeItem.__init__(self, parent)
        self._msg = u''

    def getMessage(self):
        return self._msg

    def setMessage(self, value):
        if self._msg != value:
            print "Setting message property to", value
            self._msg = value
            self.messageChanged.emit()
        else:
            print "Message property already set to", value

    message = QtCore.Property(unicode, getMessage, setMessage, notify=messageChanged)


def main():
    app = QtGui.QApplication([])

    QtDeclarative.qmlRegisterType(Message, "utils", 1, 0, "Message")

    win = QtDeclarative.QDeclarativeView()
    win.setSource("main.qml")
    win.setWindowTitle("Hello World")
    win.setResizeMode(QtDeclarative.QDeclarativeView.SizeRootObjectToView)

    win.show()
    app.exec_()

if __name__ == "__main__":
    main()


import Qt 4.7 // or QtQuick 1.0 if applicable

import utils 1.0

Rectangle {
    id: main

    signal clicked

    color: "black"
    width: 360; height: 360

    Message {
        id: msg

        message: "Click Me!"
        onMessageChanged: label.font.pixelSize = 40
    }

    Text {
        id: label

        anchors.centerIn: parent

        text: msg.message
        color: "white"
        font.pixelSize: 25

 Behavior on font.pixelSize {
            NumberAnimation { duration: 800; easing.type: Easing.OutBounce }
        }

        MouseArea {
            anchors.fill: parent
            onClicked: msg.message = "Hello World"
        }
    }

}

For more information point your browser to http://www.pyside.org and download the package for your OS.

You'll also find useful information at http://developer.qt.nokia.com/wiki/Category:LanguageBindings::PySide

Happy hacking!

Guest blog post at Nokia Nordic Blog

2011-02-01T20:24:00.000+01:00

I've been invited to write a guest blog post at Nokia Nordic blog. Please check it out if you're interested in Qt-development.

http://blogs.nokia.com/nordicblog/news/meet-mario-the-man-behind-the-qt-app-tvmatchen/#date=2011-02-01,mode=month

Nokia Certified Qt Developer

2010-11-29T20:37:00.000+01:00

Now I finally received my Qt certification diploma by mail. Actually, I passed the exam when I attended the Qt DevDays 2010 (october) in Munich. See if I'll go for the advanced exam some day.

Nokia Rules!

2010-11-12T11:52:00.000+01:00

There's a conference in the southern part of Sweden (Malmoe) called Oredev which I have been attending this week. I was interested in a talk about MeeGo, meet some Nokia people and to attend a two days Qt course.

Nokia had a booth and I was passing by to check out if they had some Qt stickers which I needed for a demonstration (a simple QML app) I was going to show at my employers booth. To my disappointment they had no stickers but instead I had a great chat with one of guys.

We had an interesting and giving chat about Qt, apps, OviStore, Nokia and MeeGo. Suddenly the guy grabs a package and hands it to me, to my surprise I just realized that he gave me a Nokia N8! He smiled at me and told me to make my app available on OviStore.

Now finally Nokia has given me the opportunity to develop apps for mobile devices since I can use Qt (I already made an app w/o owning a device).

I can't wait for MeeGo devices to show up.

Thanks Nokia!

Getting Interactive With PyQt

2010-11-06T17:08:00.000+01:00

One of the cooler features with PyQt (IMHO) is the possibility to write Qt code interactively. It isn't actually something new and I'm sure it's been there for a while but some of you might not be aware of it. I find it very useful when I want to learn new things in Qt, so without further ado, here's a short introduction on using it.

Start an interactive python session by executing python in a shell:

$> python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Let's create a simle top-level window (I wont add the '>>>' in case you would like to copy/paste the code):

from PyQt4.QtCore import *
from PyQt4.QtGui import *
app = QApplication([])
window = QWidget()
window.resize(400, 300)
window.show()

Now you should have a window similar to the screen shot below:

Ok, let's create a push button:

button = QPushButton("Click Me!", window)

Hmm, no button shows up... Why? Well, that's because we need to make it visible after we have added it to the window:

button.show()

Ahh, sweet! We have added the button to the window interactively:

Let's put the button at the center of the window by using a layout manager:

layout = QHBoxLayout(window)
layout.addWidget(button)

And finally, let's see how 'layout.addStretch()' affects the layout:

layout.addStretch()

You can easily add a slot to the button's clicked signal:

def mySlot():
    print "Button Clicked"
button.clicked.connect(mySlot)

As you can see, it's an easy way to try out thing in Qt.

Thanks for reading!

Qt DevDays 2010

2010-09-27T20:10:00.000+02:00

I'm attending Qt DevDays 2010 and really looking forward to it.

I haven't decided yet which tracks I'll attend, there are so many interesting but I definitely go for some Quick-talks. I believe I'll have a great time.

And that's not all, in November I'll also attend a two day Mobile Qt course at Öredev with topics such as Qt Mobility API, Gesture framework, Hybrid JavaScript apps and last but not least Quick. Looks very promising.

PyQt and Signal overloads

2010-09-11T09:23:00.000+02:00

I almost spent half an hour trying to figure out why my code didn't work.

I used a QSignalMapper to connect a couple of clickable widgets to emit just one signal, widgetClicked(int index), instead of having multiple signals, one for each widget. I connected the widgetClicked signal to the mapped signal of the QSignalMapper class and did the required set up for the signal mapper, something like the following snippet:

self._signalMapper = QSignalMapper(self)
self._signalMapper.mapped.connect(self.widgetClicked)

for index, widget in enumerate(buttons):
    widget.clicked.connect(self._signalMapper.map)
    self._signalMapper.setMapping(widget, index)

Everything worked fine until I changed the signature of widgetClicked to pass a string instead of an int, it didn't work anymore. I only got the following print out: TypeError: connect() failed between 'mapped' and 'widgetClicked' and had no clue why because the mapped signal has a couple of overloads:

void mapped(int i)
void mapped(const QString & text)
void mapped(QWidget * widget)
void mapped(QObject * object)

I did also try changing widgetClicked to not receive any arguments and that worked, which was expected because the receiver is not required to match all arguments of a signal. Of course, it also worked if I changed back to int...

What the #@! is going on? Naturally, I had no option of searching for answers since I was sitting on the train back home from work and had no Internet connection.

Then it come to me, Python is not aware of the C++ signature overload. Python doesn't support overloading. Python/PyQt can't set up the right connection between widgetClicked and mapped. But how do you specify which of the overloads you want to connect to? By indexing:

# This connects your slot/signal to the string-overload
signalMapper.mapped[str].connect(...)

The int overload will be used default by PyQt, that's why my code worked when I connected widgetClicked to mapped w/o specifying which overload I wanted.

IcecastRadio - Qt widget/Qt Quick example Icecast player

2010-09-05T15:03:00.000+02:00

I've been toying with modeling a Qt application which supports both a traditional widget-based view and a quick-based (qml) view.

About IcecastRadio
The intention of this project/application is to demonstrate how important
it can be to design a model for your application. If the model is correctly
designed, it can be used from both traditional widget based UIs and
modern UIs built with Qt Quick without too much effort.

The advantage is of course that you can provide both a desktop application which integrates with the native look and feel of the OS and at the same time have a totally different experience UI-wise, for example in an embedded device by only changing the view-layer.

IcecastRadio is actually two applications, QtCast and QuickCast. QtCast implements the widget-based view and QuickCast the quick-based view. Both application depends on a third entity, the model. You'll see this division in the source code in form of three projects, qtcast, quickcast and model.

The application was successfully compiled against Qt 4.7-beta 1 and 2.

Source code
You'll find the source code at http://gitorious.org/icecastradio. The player code is released under the MIT license, clone and have fun. Please read the README file found in the repository for more information.

Screenshots
Must have...

Remark
The application lacks a lot of features that could be expected from a radio player, for example there is no way of sorting the list of stations nor any filtering can be applied. The intention was not to create a full blown Icecast player but to demonstrate how to create a model that is usable from both a widget-based application and a quick-based application.

Patches are welcome :)

Python Koans - A Great Way to Learn Python!

2010-09-01T21:48:00.000+02:00

I just found out about Python Koans by Greg Malcolm (thanks dude) after listening to the from python import podcast podcast (which I find amusing, thanks guys).

It's an awesome way to learn Python. Instead of just reading tutorials and/or books you learn Python by coding.

The interactive tutorial is built around unit-tests and you advance and gain new skills by passing tests and it's really funny. You do learn a lot about the Python language when doing the Koans so I recommend it even if you've been using Python for a while.

Another cool thing is that you learn how to do unit testing in Python, if you're not already familiar with it.

My python fix

2010-08-03T23:53:00.000+02:00

It has been a while since I did something in Python. I've been coding C++, Qt and Qt Quick lately and I do enjoy it, especially Quick. I really like the declarative way of describing UIs.

But I still need my shot of Python now and then. Often I end up just doing small non useful snippets like list comprehensions stuff. I just love them, they are so beautiful and they make me happy :)


>>> # Find the longest words
>>> words = "Some random words in a text string"
>>> max([(len(w), w) for w in words.split()], key=lambda x:x[0])[1]

QSignalMapper - at your service

2010-06-10T20:06:00.000+02:00

The QSignalMapper class is very useful, for example, in situations were you have a couple of QPushButtons and you want to connect the clicked-signal of each button to a single slot and still be able to identify the sender. It's possible to find out which object emitted a signal by calling sender() inside the slot, but I don't think that's good practice because you introduce a strong coupling between the signaling object and the slot.

A better solution is to use the QSignalMapper class which re-emits the signal with an additional configurable parameter. The parameter can be one of the following types: int, QString, QObject or QWidget.

So without further ado, here's a self describing example:


#
# Example showing how QSignalMapper can be used to manage an arbitrary
# numbers of parameterless signals and re-emit them with an argument
# identifying the sender.
#
# Each signal is associated in the QSignalMapper with either an int,
# QString, QObject or a QWidget which is passed as argument to the slot
# connected to the QSignalMapper.
#
from PyQt4.QtCore import QSignalMapper, pyqtSignal
from PyQt4.QtGui import QApplication, QFrame, QGridLayout, QPushButton

class Grid(QFrame):
    """ Lay out widgets in a grid. """

    clicked =  pyqtSignal(int)
    """
    This signal will be emitted when on one of the grid items is clicked.
    The grid item's index will be passed as argument.
    """

    def __init__(self, items, colCount, parent=None):
        """
        Create Grid and setup QSignalMapper.

        Connect each item's 'clicked' signal and use the sequence index
        as map value which will be passed as argument when the Grid's
        clicked signal is emitted.

        items: sequence with widgets having a 'void clicked()' signal
        colCount: column count for each row
        parent: parent widget, default None
        """
        super(Grid, self).__init__(parent)

        # Create a grid layout.
        layout = QGridLayout()
        self.setLayout(layout)

        # Create the signal mapper.
        signalMapper = QSignalMapper(self)

        for cnt, item in enumerate(items):
            # Setup mapping for the item. In this case, the
            # mapping is the sequence index of the item.
            signalMapper.setMapping(item, cnt)

            # Connect the item's 'clicked' signal to the signal
            # mapper's 'map' slot.
            # The 'map' slot will emit the 'mapped' signal
            # when invoked and the mapping set previously will be
            # passed as an argument to the slot/signal that is
            # connected to the signal mapper's 'mapped' signal.
            item.clicked.connect(signalMapper.map)

            # Add the widget to the grid layout
            layout.addWidget(item, cnt / colCount, cnt % colCount)

        # Forward the signal mapper's 'mapped' signal via the Grid's
        # 'clicked' signal. This will handle all widgets' 'clicked'
        # ssignals.
        signalMapper.mapped.connect(self.clicked)

if __name__ == '__main__':
    import sys
    app = QApplication(sys.argv)

    # Create grid items
    items = (QPushButton('Open'), QPushButton('Close'),
             QPushButton('Read'), QPushButton('Write'),
             QPushButton('Delete'))

    win = Grid(items, 2)

    # Handle grid clicks here
    @win.clicked.connect
    def clicked(index):
        print "Button at:", index, " - text:", items[index].text()

    win.show()
    sys.exit(app.exec_())

Hope this was to any help.

ABCs - Abstract Base Classes

2010-05-26T20:59:00.001+02:00

What's ABCs?
The abc (abstract base class) module was introduced in version 2.6 of Python. The module provides the infrastructure for defining an abstract base class and can typically be used in situations where frameworks need to specify a bunch of interfaces to be implemented to ensure the functionality. An example can be a database application where the framework requires you to provide a Data Access Object with basic CRUD operations and therefore defines an interface, call it UserDao, with the following methods:

UserDao

getById - Return a user entity with the specified id
store - store a user entity
remove - remove a user entity from the persistent storage

To ensure that developers follows the protocol an abc is introduced.

The UserDao abc

The following example will show you basic usage of the abc module. You should read the module documentation for more details.

Let's create the UserDao abc:


from abc import ABCMeta, abstractmethod

class UserDao:
    """
    This interface should be implemented to provide the framework with
    basic user CRUD operations.
    """
    __metaclass__ = ABCMeta

    @abstractmethod
    def getById(self, userId):
        """ Return the user with the user id. """

    @abstractmethod
    def store(self, user):
        """ Persist the user. """

    @abstractmethod
    def remove(self, user):
        """ Remove the user from the persistent storage. """

If we try to create an instance of this class we get the following error:


>>> dao = UserDao()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Can't instantiate abstract class UserDao with abstract methods getById, remove, store

In other words, we have a well defined interface to support CRUD operations for User entities and we also have an easy to read documentation of what's expected from a UserDao. If we would rely on duck typing instead of defining the abc, we would need to document the expected behavior for the UserDao elsewhere. I think that the biggest advantage of using ABCs is that you have the interface definition and documentation in one place.

Just to prevent flaming :) I enjoy duck typing and use it a lot but I think that ABCs are useful is some situations.

Implementing the abc

Implementing the UserDao can be done by sub-classing the UserDao.


class User(object):
    def __init__(self):
        self.id = None
        self.name = None

class UserDaoImpl(UserDao):

    def getById(self, userId):
        user = User()
        user.id = userId
        user.name = 'A User'
        return user

    def store(self, user):
        print "Storing: ", user.id, "-", user.name

    def remove(self, user):
        print "Removing: ", user.id

And to complete the example:


userDao = UserDaoImpl()
user = userDao.getById(10)
user.name = "New Name"
userDao.store(user)

Wrap up
I haven't investigated if ABCs introduces any performance overhead but I can't really see why they should. One advantage of using ABCs in frameworks is that you can have the documentation and interface details as a simple Python class. Another advantage is that Python will issue an error if a sub-class doesn't implement all abstract methods/properties of the abc, which is probably the author's intention.

Network manager problems on Kubuntu 10.04

2010-05-04T21:34:00.001+02:00

I installed 10.04 as soon as it was released and it worked right out of the box. But suddenly today the network manager refused to start.

Well, I could see the icon in the systray but it didn't show any network interfaces (I have eth0 and wlan0).

To get me an IP I had to run dhclient manually from the terminal which was successful. Finally I could google for help.

Google pointed me to this blog post where the author had run into the same problem.

Apparently there is a bug filed for this issue, I'm just too lazy to search in the bug database :)

Just wanted to share this with you if you bump into it.

Unit testing with Qt

2010-04-16T21:01:00.000+02:00

It's very easy to do unit testing with Qt. If you have any previous experience withJUnit you shouldn't have any difficulties using QTestLib.

All that is required to create a unit test, besides adding 'QT += testlib' to your .pro file, is that the test class is a sub-class of QObject and each test function is declared as a private slot. To run all your test functions in a class simply call QTest::qExec() from the main function.

The following snippet shows what a unit test looks like:

#include <QtTest/QTest>

class MyTest : public QObject
{
Q_OBJECT
public:
    explicit MyTest(QObject *parent = 0) : QObject(parent) {}

private slots:
    /** Test function 1 */
    void test1()
    {
        QCOMPARE('a', 'a');
    }

    /** Test function 2 */
    void test2()
    {
        QVERIFY(10 != 11);
    }

};
int main(int argc, char *argv[])
{
    // Simple test cases doesn't require a
    // Q[Core]Application instance.
    MyTest t;
    QTest::qExec(&t);
}
// The next line is only required if we have the
// definition and implementation in the same file
// as in this case.
#include "qunittest.moc"

The test above produces the following output when executed:

Config: Using QTest library 4.6.2, Qt 4.6.2
PASS   : MyTest::initTestCase()
PASS   : MyTest::test1()
PASS   : MyTest::test2()
PASS   : MyTest::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped

QVERIFY evaluates the expression and continues the execution if the expression evaluates to true.If the expression evaluates to false a message is appended to the test log and the test functionis stopped. QCOMPARE is similar to QVERIFY and gives a more verbose output.In addition to the test functions the framework will look for the following four functions (must alsobe declared as private slots):

initTestCase() - Executed before any test function
cleanupTestCase() - Executed after the last test function
init() - Executed before each test function
cleanup() - Executed after each test function

Ok, now we know the basics, lets go on and check how to do data driven tests.

Data driven tests are used when you want to exercise some piece of code withdifferent values and is supported by the Qt testlib. To feed test data to a test functionadd a function with the same name as the test function appended with a '_data' suffix. So ifthe test function is named testFeature, the function that serves the test with datashould be named testFeature_data. The test data is setup in a table. Two functions areused to create this table, addColumn and newRow. addColumn is used to define elementsin the table and newRow adds data. QFETCH is used to fetch data from the table.

Lets create a simple 'capitalize' function that takes a string, capitalizes the first character and returns the result as a new string. We want to test the function with different data such as an empty string, first character is lower case and first character is upper case.

#include <QtTest/QTest>

class MyTest : public QObject
{
Q_OBJECT
public:
    explicit MyTest(QObject *parent = 0) : QObject(parent) {}

    QString capitalize(const QString &text)
    {
        QString result = text;
        result[0] = text[0].toUpper();
        return result;
    }

private slots:
    void testCapitalize_data()
    {
        QTest::addColumn("string");
        QTest::addColumn("expected");

        QTest::newRow("empty string") << "" << "";
        QTest::newRow("lower case") << "lower case" << "Lower case";
        QTest::newRow("upper case") << "Upper case" << "Upper case";
    }

    void testCapitalize()
    {
        QFETCH(QString, string);
        QFETCH(QString, expected);
        QCOMPARE(capitalize(string), expected);
    }
};
// Macro that implements main
QTEST_APPLESS_MAIN(MyTest);
#include "qunittest.moc"

Running the test gives the following output:

********* Start testing of MyTest *********
Config: Using QTest library 4.6.2, Qt 4.6.2
PASS   : MyTest::initTestCase()
QFATAL : MyTest::testCapitalize(empty string) ASSERT: "i >= 0 && i < size()" in file ../../../../opt/qtsdk-2010.02/qt/include/QtCore/qstring.h, line 690
FAIL!  : MyTest::testCapitalize(empty string) Received a fatal error.
   Loc: [Unknown file(0)]
Totals: 1 passed, 1 failed, 0 skipped
********* Finished testing of MyTest *********

Oops, forgot to check if the string is empty. We fix that by adding the following check at the beginning of the capitalize function:

if (text.isEmpty()) {
    return text;
}

Running the test again:

********* Start testing of MyTest *********
Config: Using QTest library 4.6.2, Qt 4.6.2
PASS   : MyTest::initTestCase()
PASS   : MyTest::testCapitalize()
PASS   : MyTest::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of MyTest *********

Ok, this was a quick introduction to get you started. To find out more about the test lib read the manual, api docs, and the tutorial. You'll find examples on how to test your GUI by simulating mouse clicks, key presses and how to do simple benchmarking.

Ternary operation

2010-03-19T07:55:00.001+01:00

I'm a big fan of using ternary operations when possible but Python didn't introduced them until the 2.5 release, see Wikipedia page. Before the 2.5 release there existed options like using the and and or operator

x and y or z

or a lambda construction, but none if them are considered foolproof (see the wiki page for explanation).So how do you write the ternary operation in python?


a, b = 1, 2

# non ternary operation
if a > b:
    z = a
else:
    z = b

# rewritten to

z = a if a > b else b

Clean, readable and simple!

QTabWidget - Hiding the bar

2010-02-20T08:49:00.000+01:00

It has been a while since my last post. I've been sick, my son has been sick and so has my girlfriend. Now I'm back and have been coding on a private project for a couple of days.

While I was toying and trying some ideas I found out how to create tab bars with Qt that are only visible if you have more than one tab. This is quite popular in browser for example. So without further ado, here's the code:

from PyQt4 import QtGui

class TabWidget(QtGui.QTabWidget):
    def __init__(self, parent=None):
        super (TabWidget, self).__init__(parent)
        self.setTabsClosable(True)
        self.tabCloseRequested.connect(self.removeTab)

    def tabInserted(self, index):
        self.tabBar().setVisible(self.count() > 1)

    def tabRemoved(self, index):
        self.tabBar().setVisible(self.count() > 1)

As you can see, it isn't rocket science. I did some googling but couldn't find anything about it, that's why I want to share my findings. The magic is to only hide the TabBar associated with the TabWidget, not the TabWidget itself.

You can use the following snippet to see the TabWidget in action:

import sys
from PyQt4 import QtGui

class TabWidget(QtGui.QTabWidget):
    def __init__(self, parent=None):
        super (TabWidget, self).__init__(parent)
        self.setTabsClosable(True)
        self.tabCloseRequested.connect(self.removeTab)

    def tabInserted(self, index):
        self.tabBar().setVisible(self.count() > 1)

    def tabRemoved(self, index):
        self.tabBar().setVisible(self.count() > 1)

qApp = QtGui.QApplication(sys.argv)

tab = TabWidget()

button = QtGui.QPushButton('Hello')
@button.clicked.connect
def clicked():
    tab.addTab(QtGui.QLabel('Hello'), 'Hello')

tab.addTab(button, 'Button')

layout = QtGui.QHBoxLayout()
layout.addWidget(tab)

window = QtGui.QWidget()
window.setLayout(layout)
window.resize(600, 400)
window.show()

qApp.exec_()

Named tuple

2010-01-30T13:51:00.000+01:00

I recently read a post on planet python and the author mentioned something about named tuple which made my curious.

So what's a named tuple?
The namedtuple was introduced in Python 2.6. A named tuple is created using a factory function from the collections module and it extends the basic tuple by assigning a name to each position in a tuple but can still be used as a regular tuple. This makes it possible to access fields by name instead of an index. The named tuple should not require more memory, according to the documentation, than regular tuples since they don't have a per instance dictionary.

The factory function signature is:

collections.namedtuple(typename, field_names[, verbose])

The first argument specifies the name of the new type, the second argument is a string (space or comma separated) containing the field names and finally if verbose is true the factory function will also print the class generated.

Enough theory, I'll show you an example.

Example
Say you have a tuple containing username and password. To access the username you get the item at position zero and the password is accessed at position one:

credential = ('mario', 'secret')
print 'Username:', credential[0]
print 'Password:', credential[1]

There's nothing wrong with this code but the tuple isn't self-documented. You have to find and read the documentation about the positioning of the fields in the tuple. This is where named tuple can enter the scene. We can rewrite the previous example as following:

import collections
# Create a new sub-tuple named Credential
Credential = collections.namedtuple('Credential', 'username, password')

credential = Credential(username='mario', password='secret')

print 'Username:', credential.username
print 'Password:', credential.password

Nice, don't you agree?

If you are interested of what the code looks like for the newly created Credential-type you can add verbose=True to the argument list when creating the type, in this particular case we get the following output:

import collections
Credential = collections.namedtuple('Credential', 'username, password', verbose=True)

class Credential(tuple):                                     
        'Credential(username, password)'                     

        __slots__ = () 

        _fields = ('username', 'password') 

        def __new__(_cls, username, password):
            return _tuple.__new__(_cls, (username, password)) 

        @classmethod
        def _make(cls, iterable, new=tuple.__new__, len=len):
            'Make a new Credential object from a sequence or iterable'
            result = new(cls, iterable)                               
            if len(result) != 2:                                      
                raise TypeError('Expected 2 arguments, got %d' % len(result))
            return result

        def __repr__(self):
            return 'Credential(username=%r, password=%r)' % self

        def _asdict(t):
            'Return a new dict which maps field names to their values'
            return {'username': t[0], 'password': t[1]}

        def _replace(_self, **kwds):
            'Return a new Credential object replacing specified fields with new values'
            result = _self._make(map(kwds.pop, ('username', 'password'), _self))
            if kwds:
                raise ValueError('Got unexpected field names: %r' % kwds.keys())
            return result

        def __getnewargs__(self):
            return tuple(self)

        username = _property(_itemgetter(0))
        password = _property(_itemgetter(1))

The named tuple doesn't only provide access to fields by name but also contains helper functions such as the _make() function which helps creating an Credential instance from a sequence or iterable. For example:

cred_tuple = ('mario', 'secret')
credential = Credential._make(cred_tuple)

There are more interesting use-cases and examples in the documentation, so I suggest that you take a peek.

Comments
I think the named tuple is useful. They remove the error-prone indexing in tuples by providingaccess to fields by name without adding any memory overhead. They are also regular Python classes which means you can do anything you can do with classes.

More fun with QWebKit

2010-01-20T21:31:00.000+01:00

In the previous post I wrote about calling python methods and accessingproperties from JavaScript executed in QWebKit. In this post I'll showyou how to do the other way around, calling a JavaScript function fromPython.

I would like to thank Rich Moore for commenting on the previous post and pointing out that you should re-add your python object every time the javaScriptWindowObjectCleared() signal is emitted.

Actually, I think this is almost to simple to do a post about so I'll try doing something fun/useful with it.

QWebFrame contains a public slot:

QVariant evaluateJavaScript(const QString & scriptSource)

You simply pass the JavaScript-snippet to the function. The snippet will be evaluated using the frame as context and returns the result of the last executed statement. This makes it possible to 'click' submit buttons, fill form fields and other interesting stuff in the currently loaded frame. I'll give you an example on how you can create an auto-login GMail widget. The widget will automatically login the user by populating the login form with the user's credentials and 'click' the login button.

#!/usr/bin/env python
from PyQt4 import QtCore
from PyQt4 import QtGui
from PyQt4 import QtWebKit

javaScriptLogin = """
document.getElementsByName('Email').item(0).value='{email}';
document.getElementsByName('Passwd').item(0).value='{password}';
document.getElementsByName('signIn').item(0).click(); void(0);
"""

class GmailWebView(QtWebKit.QWebView):
    def __init__(self, parent=None):
        super(GmailWebView, self).__init__(parent)
        self.loggedIn = False

    def login(self, url, email, password):
        """Login to gmail."""
        self.url = QtCore.QUrl(url)
        self.email = email
        self.password = password  
        self.loadFinished.connect(self._loadFinished)
        self.load(self.url)

    def createWindow(self, windowType):
        """Load links in the same web-view."""
        return self

    def _loadFinished(self):
        if self.loggedIn:
            self.loadFinished.disconnect(self._loadFinished)

        self.loggedIn = True
        jscript = javaScriptLogin.format(email=self.email, password=self.password)
        self.page().mainFrame().evaluateJavaScript(jscript)

    def contextMenuEvent(self, event):
        """Add a 'Back to GMail' entry."""
        menu = self.page().createStandardContextMenu()
        menu.addSeparator()
        action = menu.addAction('Back to GMail')
        @action.triggered.connect
        def backToGMail():
            self.load(self.url)
        menu.exec_(QtGui.QCursor.pos())

def main():
    import sys
    qApp = QtGui.QApplication(sys.argv)

    # Prevents me from posting my password on the blog :)
    password, ok = QtGui.QInputDialog.getText(None, "Password request", "Enter password", QtGui.QLineEdit.Password)
    if not ok:
        return

    gmailWebView = GmailWebView()
    gmailWebView.login('https://mail.google.com/mail',
                       'mario.boikov',
                       password)
    gmailWebView.show()

    sys.exit(qApp.exec_())

if __name__ == "__main__":
    main()

This is just a quick hack, it lacks a bunch of checkings...

Maybe it's time to wipe the dust off my GMonitor-plasmoid and add support for opening my account in a new window based on the GMail widget above.

Calling Python from JavaScript in PyQt's QWebkit

2010-01-16T08:31:00.000+01:00

QtWebKit makes it very easy to expose methods and properties implemented in Python to JavaScript. Qt will automatically expose Qt-slots and Qt-properties to a JavaScript when a QObject is made available in the frame's JavaScript context.

I think the code speaks for itself

import sys
from PyQt4 import QtCore, QtGui, QtWebKit

"""Html snippet."""
html = """
<html><body>
  <center>
  <script language="JavaScript">
    document.write('<p>Python ' + pyObj.pyVersion + '</p>')
  </script>
  <button onClick="pyObj.showMessage('Hello from WebKit')">Press me</button>
  </center>
</body></html>
"""

class StupidClass(QtCore.QObject):
    """Simple class with one slot and one read-only property."""

    @QtCore.pyqtSlot(str)
    def showMessage(self, msg):
        """Open a message box and display the specified message."""
        QtGui.QMessageBox.information(None, "Info", msg)

    def _pyVersion(self):
        """Return the Python version."""
        return sys.version

    """Python interpreter version property."""
    pyVersion = QtCore.pyqtProperty(str, fget=_pyVersion)

def main():
    app = QtGui.QApplication(sys.argv)

    myObj = StupidClass()

    webView = QtWebKit.QWebView()
    # Make myObj exposed as JavaScript object named 'pyObj'
    webView.page().mainFrame().addToJavaScriptWindowObject("pyObj", myObj)
    webView.setHtml(html)

    window = QtGui.QMainWindow()
    window.setCentralWidget(webView)
    window.show()

    sys.exit(app.exec_())

if __name__ == "__main__":
    main()

Some references:

New-style PyQt Signals and Slots

2010-01-14T20:22:00.001+01:00

I was to lazy to take a look at the new-style signal and slot support which was introduced in PyQt 4.5 until yesterday. I did know that there were something called new-style signals and slots but that was the end of the story. Now I have taken the time and I think it's a cleaner solution than the old-style.

I'll just give you a short intro to whet your appetite, find all details here yourself.

import sys
from PyQt4 import QtCore
from PyQt4 import QtGui

def clicked():
    print "Button Clicked"

qApp = QtGui.QApplication(sys.argv)

button = QtGui.QPushButton("Click Me")
QtCore.QObject.connect(button, QtCore.SIGNAL('clicked()'), clicked)
button.show()

sys.exit(qApp.exec_())

This is the old way of connecting a signal to a slot. To use the new-style support just replace line 11 with following code

button.clicked.connect(clicked)

The new-style support introduces an attribute with the same name as the signal, in this case clicked.

If you need to define your own signal you'll do something like this (off the top of my head):

class X(QtCore.QObject):
    mySignal = QtCore.pyqtSignal(int)

    def emitMySignal(self):
        self.mySignal.emit(100)

And the old way:

class X(QtCore.QObject):
    def emitMySignal(self):
        self.emit(QtCore.SIGNAL('mySignal'), 100)

IMHO the new-style support is more pythonic and you don't have to specify your signals as strings when connecting. If you use pydev (eclipse) you'll also have completion support for signals.

Digikam - Light table

2010-01-08T12:47:00.002+01:00

This post is somewhat related to the post about ctypes and gphoto.

Background
I consider myself a pre-amateur photographer and that means I'm taking a lot of shots just to learn how the camera works. I own a Canon EOS 1000D and have recently moved from the fully automatic exposure modes to the advanced exposure modes, and to be more specific, the programmed auto exposure mode.

So, I needed an application that could load a couple of pictures, showing two pictures side-by-side for visual comparison and at the same time presenting the more important meta-data attributes such as focal length, exposure time, ISO and aperture. I considered doing such application myself and that's why I came across and started to explore gphoto. But luckily, I managed to stop myself and took yet another look at what Digikam offers. I'm a die-hard KDE-fan and have been using digikam, which is a KDE photo manager application, for quite some time to organize my photos but never actually used any other features except removing red-eyes.

To my surprise digikam included exactly the feature I was looking for! It's called the Light Table.

The Light Table
Digikam allows you to select a couple of pictures which can be placed onto the light table, as shown in the screenshot below.

By choosing Place onto Light Table a new window is opened with, in this case, the three pictures loaded and two of them shown side-by-side.

The light table also displays the meta-data for the respective picture (sorry about that some of the attributes are in Swedish) that are currently shown. You can synchronize operations like zooming and panning. When you zoom, both pictures will be zoomed to the same level and when panning, both pictures will be panned to the same area, really nice.

This was exactly what I was looking for. Now I can take a couple of shots with different settings, for example, changing the white balance, ISO and/or other parameters and compare the result rather easily. I get both a visual and a meta-data diff in the same view.

I just wanted to share my findings and hope this post was to some use for others that are looking for something similar.

Let's timeit!

2010-01-05T14:52:00.000+01:00

Every now and then you might want to time snippets just to make sure that you choose the more efficient solution. In those cases you can use the timeit module to measure execution time for snippets.

Introduction
It's very easy to setup and measure the execution time for a snippet with timeit. The module contains a class, Timer, which is used to perform the measurement. The class has one constructor and three methods:

Timer([stmt='pass'[, setup='pass'[, timer=<timer function>]]]) - stmt is the statement to be timed and setup is called once before executing the main statement. A timer function can be specified and default is time.time() for all platforms but windows which is set to time.clock() instead (according to my timeit.py)
timeit([number=1000000]) - Executes the main statement passed to the constructor number of times and returns the result in seconds as a float.
repeat([repeat=3[, number=1000000]]) - Convenience function that calls timeit(number) repeat times. Returns a list with the results.
print_exc([file=None]) - Helper to print a traceback from the timed snippet.

Starting with Python 2.6 the timeit module also defines two convenience functions, timeit.timeit() and timeit.repeat(). They are basically wrappers around the Timer class.

Example
Suppose that I would like to create a list containing 100 'c':s like this ['c', 'c', ...]. There are at least two ways of doing this:

lst = ['c'] * 100
# or
lst = ['c' for i in xrange(100)]

Which one should I choose? Well, let's execute both statements with timeit and measure the execution time.

>>> import timeit
>>> t = timeit.Timer(stmt="lst = ['c'] * 100")
>>> print t.timeit()
1.10580182076
>>> t = timeit.Timer(stmt="lst = ['c' for x in xrange(100)]")
>>> print t.timeit()
7.66900897026

Ok, I think I'll stick with the first snippet :)

The result returned is the total execution time in seconds. In this particular case when we are executing the snippet 1000000 times the result is also the execution time in microseconds for one single pass (1000000*exe_time/1000000 == exe_time).

Normally, the timeit module doesn't have access to things that you have defined in your module. If you would like to measure a function that you have defined in your module you can specify the import statement in the setup parameter:

>>> def create_lst(size):
...    return ['c'] * size
...
>>> t = timeit.Timer(stmt="create_lst(100)", setup="from __main__ import create_lst")
>>> print t.timeit()
1.21339488029

This will introduce a little overhead since the create_lst() function is called in the measurement loop instead of just executing an inlined snippet.

Note: Timer.timeit() will by default disable garbage collection during timing. To enable GC you can pass 'gc.enable()' as a setup statement.

I find the timeit module as a simple and convenient way to measure execution time for small snippets.

When ctypes comes to the rescue

2009-12-29T21:13:00.000+01:00

I recently purchased a DSLR (Canon 1000D) and almost at the same time I read an article about remote controlling your camera using gphoto. I tried it out and thought it was cool. During the holidays I had some time over to spend on hacking and wanted to try controlling my camera from Python. To my disappointment there were no Python bindings included with Ubuntu for gphoto. I did some googling but couldn't find any pre-compiled bindings, what to do?

Well, I could always try doing it with ctypes.

From the docs:

ctypes is a foreign function library for Python. It provides C compatibledata types, and allows calling functions in DLLs or shared libraries. It can beused to wrap these libraries in pure Python.

Note: I haven't done any serious stuff with ctypes nor gphoto before so if you find any errors etc please post a comment. 

It amazed my how easy it is to use ctypes. Here's a snippet that will take a picture (from the first camera found), download the image to local storage and then delete it from the camera's storage.

import ctypes
import os

# gphoto structures
""" From 'gphoto2-camera.h'
typedef struct {
        char name [128];
        char folder [1024];
} CameraFilePath;
"""
class CameraFilePath(ctypes.Structure):
    _fields_ = [('name', (ctypes.c_char * 128)),
                ('folder', (ctypes.c_char * 1024))]

# gphoto constants
# Defined in 'gphoto2-port-result.h'
GP_OK = 0
# CameraCaptureType enum in 'gphoto2-camera.h'
GP_CAPTURE_IMAGE = 0
# CameraFileType enum in 'gphoto2-file.h'
GP_FILE_TYPE_NORMAL = 1

# Load library
gp = ctypes.CDLL('libgphoto2.so.2')

# Init camera
context = gp.gp_context_new()
camera = ctypes.c_void_p()
gp.gp_camera_new(ctypes.pointer(camera))
gp.gp_camera_init(camera, context)

# Capture image
cam_path = CameraFilePath()
gp.gp_camera_capture(camera,
                     GP_CAPTURE_IMAGE,
                     ctypes.pointer(cam_path),
                     context)

# Download and delete
cam_file = ctypes.c_void_p()
fd = os.open('image.jpg', os.O_CREAT | os.O_WRONLY)
gp.gp_file_new_from_fd(ctypes.pointer(cam_file), fd)
gp.gp_camera_file_get(camera,
                      cam_path.folder,
                      cam_path.name,
                      GP_FILE_TYPE_NORMAL,
                      cam_file,
                      context)
gp.gp_camera_file_delete(camera,
                         cam_path.folder,
                         cam_path.name,
                         context)
gp.gp_file_unref(cam_file)

# Release the camera
gp.gp_camera_exit(camera, context)
gp.gp_camera_unref(camera)

Ok, remember that I haven't done any wrapper or anything, it almost looks like 'C' code. I have also skipped all error checking for brevity.

You can always use a c_void_p if you don't need access to the data in Python (if you only need to pass a pointer between foreign functions). I'm using c_void_p instead of defining ctypes structures for gphoto's data types such as Camera and CameraFile. I still had to define CameraFilePath since I needed access to the data in Python.

I really like ctypes because I don't have to maintain code in C to access native functionality. Maybe you won't get the same performance as with traditional bindings but in this particular case it's not an issue.

Hello Planet Python!

2009-12-22T09:04:00.000+01:00

I just got added to Planet Python and want to give the readers a quick introduction to myself.

My name is Mario and I live in Sweden. I've been working with Java development since 1999, mainly with enterprise systems. In 2007 I switched to embedded development and primary focusing on embedded Linux systems.

Early this year (2009) I got curious about developing KDE/QT applications since I've been using the KDE desktop for years now. I didn't want to use Java nor C++ for desktop development, I needed a language which was simple to use, feature rich and well supported by QT and KDE. There were two candidates for the job, Ruby and Python, guess which one I choosed.

I really like Python and it feels fun coding again, it's my first choice of programming language now.

On my blog you'll find posts about everything regarding Python, language features, APIs, frameworks, etc. I hope you'll enjoy reading.

try/for/while else... else what?

2009-12-15T08:03:00.002+01:00

The Python language actually has support for else-clauses in some compound statements such as try, for and while.

So how do you use them?

I'll illustrate the for usage with an example:

def has_only_alpha_string(lst):
    for s in lst:
        if not s.isalpha():
            print '[{0}] contains non alphabetic character'.format(s)
            break
    else:
        print 'All strings contains only alphabetic characters'

lst = ['Hello', 'World!']

only_alpha_string(lst)

If lst is empty or exhausted the execution will continue in the else-clause. If the break is executed, the else-clause will not be executed. The same applies to while statements, the else is only executed if no break is executed within the while statement.

Note: It's valid to have a for/while-else without a break. In that case the else-clause will always be executed. 

The first time I tried it I got it all wrong (before actually reading the documentation). I expected that the else-clause should be executed only if the list sequence was empty or exhausted, but it's the other way around.

What about try-else?

For me, at least, the try-else is probably a bit more intuitive.

try:
    f = None
    f = open('a_file', 'r')
except IOError as err:
    print err
else:
    print f.read()
finally:
    if f:
        print "Closing file..."
        f.close()

If the open call is successful the else-clause is executed and of course if the open call raises an IOError exception the else-clause isn't executed. Both open and read can raise IOError exceptions, the finally-clause will always be executed even if an exception occurs in the else-clause. In this particular case I'm only interested in catching the exception raised by open. If read raises an exception, it should be forwarded to the caller.

I could also write the code as following:

try:
    f = None
    f = open('a_file', 'r')
except IOError as err:
    print err
    # return or os.exit()

print f.read()
print "Closing file..."
f.close()

The problem with this solution is that file wouldn't be closed if read raises an exception.

Hope this post made the else-clause thing in combination with try/for/while more clear.

Profiling your Python code

2009-12-08T21:28:00.002+01:00

Python provides support for deterministic profiling of your application. It's very easy to setup and use. To start profiling your code, two python modules are used:

cProfile - the application that collects profiling data
pstats - the application that makes the profling data human readable

You can read more about Python profiling stuff at The Python Profilers page.

I'll show an example of how you can use the profiler.

Say I need to calculate the sum of all odd numbers from zero to an arbitrary positive value. My initial code might end up something like this:

def odd_numbers(max):
    """ Returns a list with all odd numbers between 0 to max (inclusive) """
    l = list()
    for i in xrange(max+1):
        if (i & 1):
            l.append(i)
    return l

def sum_odd_numbers(max):
    """ Sum all odd numbers between 0 to max (inclusive) """
    odd_nbrs = odd_numbers(max)

    res = 0
    for odd in odd_nbrs:
        res += odd
    return res

def main():
    # Run this 100 times to make it measurable
    for i in xrange(100):
        print sum_odd_numbers(1024)

if __name__ == '__main__':
    main()

Now I want to find out where my code spend most if its time to help me optimize the code if possible. To profile this snippet I run:

$ python -m cProfile sum_odd.py

This will output some statistics about the code (try it), but I'll show you a more handy way to browse and examine the profile dump.

$ python -m cProfile -o profile_dump sum_odd.py

This will output the profiling statistics to a file (in non-human readable format) which can be loaded and examined with pstats.Start pstats and browse the profile dump:

$ python -m pstats
Welcome to the profile statistics browser.                    
% help                                                        

Documented commands (type help topic):
========================================
EOF  add  callees  callers  quit  read  reverse  sort  stats  strip

Undocumented commands:
======================
help

% read profile_dump
profile_dump% stats
Tue Dec  8 20:55:41 2009    profile_dump

         51405 function calls in 0.186 CPU seconds

   Random listing order was used

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    51200    0.082    0.000    0.082    0.000 {method 'append' of 'list' objects}
      100    0.099    0.001    0.181    0.002 main.py:1(odd_numbers)
        1    0.000    0.000    0.185    0.185 main.py:1(module)
        1    0.000    0.000    0.186    0.186 {execfile}
      100    0.004    0.000    0.184    0.002 main.py:9(sum_odd_numbers)
        1    0.000    0.000    0.186    0.186 string:1(module)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.001    0.001    0.185    0.185 main.py:18(main)


profile_dump%

This will of course give the same information as if you just executed cProfile without specifying an output file. The advantage of using pstats interactively is that you can view the data in different ways.

Now I want to find out in which function we spend most time. This can be done by using the sort command:

profile_dump% sort
Valid sort keys (unique prefixes are accepted):
stdname -- standard name
nfl -- name/file/line
pcalls -- call count
file -- file name
calls -- call count
time -- internal time
line -- line number
cumulative -- cumulative time
module -- file name
name -- function name
profile_dump% sort time
profile_dump% stats
Tue Dec  8 20:55:41 2009    profile_dump

         51405 function calls in 0.186 CPU seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.099    0.001    0.181    0.002 main.py:1(odd_numbers)
    51200    0.082    0.000    0.082    0.000 {method 'append' of 'list' objects}
      100    0.004    0.000    0.184    0.002 main.py:9(sum_odd_numbers)
        1    0.001    0.001    0.185    0.185 main.py:18(main)
        1    0.000    0.000    0.186    0.186 {execfile}
        1    0.000    0.000    0.185    0.185 main.py:1(module)
        1    0.000    0.000    0.186    0.186 string:1(module)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Nice! We can see that most time is spent in the odd_numbers function. The time key specifies that we would like to sort the data by the time spent in a function (exclusive calls to other functions).

Time to optimize, change the odd_numbers function to the following snippet:


def odd_numbers(max):
    """ Returns a list with all odd numbers between 0 to max (inclusive) """
    return [i for i in xrange(max+1) if (i & 1)]

Now profile the code and load the dump in pstats:


profile_dump% stats
Tue Dec  8 21:20:19 2009    profile_dump

         205 function calls in 0.020 CPU seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.015    0.000    0.015    0.000 main.py:1(odd_numbers)
      100    0.004    0.000    0.019    0.000 main.py:5(sum_odd_numbers)
        1    0.001    0.001    0.020    0.020 main.py:14(main)
        1    0.000    0.000    0.020    0.020 {execfile}
        1    0.000    0.000    0.020    0.020 main.py:1(module)
        1    0.000    0.000    0.020    0.020 string:1(module)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Wow! Not that bad, we decreased the number of function calls from 51405 to 205. We also decreased the total time spent in the application from 0.186 to 0.020 CPU seconds by writting proper Python code :)

FUSE - Filesystem in Userspace part 3 (final)

2009-12-01T21:09:00.002+01:00

Finally, as I promised, the last blog post on implementing file systems using FUSE.

I've created a file system, shoutcastfs, which enables you to mount the Shoutcast Radio directory as a file system. The genres are represented as directories and stations as files. Each file contains the station's playlist and the files are suffixed with .pls which makes it possible to load the playlist in a media player such as Amarok by double-clicking the file.

Of course, I'm using pyshoutcast (Python shoutcast API) to access the shoutcast service.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import errno
import fuse
import stat
import os
import shoutcast

fuse.fuse_python_api = (0, 2)

_shoutcastApi = shoutcast.ShoutCast()

class RootInfo(fuse.Stat):
    def __init__(self):
        fuse.Stat.__init__(self)
        self.st_mode = stat.S_IFDIR | 0755
        self.st_nlink = 2
        self._genres = {}

    @property
    def genres(self):
        if not self._genres:
            for g in _shoutcastApi.genres():
                self._genres[g] = GenreInfo(g)
        return self._genres

class GenreInfo(fuse.Stat):
    def __init__(self, name):
        fuse.Stat.__init__(self)
        self.st_mode = stat.S_IFDIR | 0755
        self.st_nlink = 2
        self.name = name
        self._stations = {}

    @property
    def stations(self):
        if not self._stations:
            for s in _shoutcastApi.stations(self.name):
                name = '{0}.pls'.format(s[0])
                name = name.replace('/', '|')
                self._stations[name] = StationInfo(name, s[1])
        return self._stations

class StationInfo(fuse.Stat):
    def __init__(self, name, station_id):
        fuse.Stat.__init__(self)
        self.st_mode = stat.S_IFREG | 0644
        self.st_nlink = 1
        # Hope no playlist exceeds this size
        self.st_size = 4096
        self.name = name
        self.station_id = station_id
        self._content = None

    @property
    def content(self):
        if self._content is None:
            self._content = _shoutcastApi.tune_in(self.station_id).read()
        return self._content

class ShoutcastFS(fuse.Fuse):
    def __init__(self, *args, **kw):
        fuse.Fuse.__init__(self, *args, **kw)
        self.root = RootInfo()

    def split_path(self, path):
        """ Returns genre and station """
        if path == '/':
            return (None, None)

        parts = path.split('/')[1:]
        if len(parts) == 1:
            return (parts[0], None)
        else:
            return parts

    def getattr(self, path):
        genre, station = self.split_path(path)

        if genre is None:
            stat = self.root
        else:
            stat = self.root.genres.get(genre)
            if not stat:
                return -errno.ENOENT

            if station:
                stat = stat.stations.get(station)
                if not stat:
                    return -errno.ENOENT
        return stat

    def readdir(self, path, offset):
        yield fuse.Direntry('.')
        yield fuse.Direntry('..')

        if path == '/':
            entries = self.root.genres.keys()
        else:
            entries = self.root.genres[path[1:]].stations.keys()

        for e in entries:
            yield fuse.Direntry(e)

    def open(self, path, flags):
        # Only support for 'READ ONLY' flag
        access_flags = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
        if flags & access_flags != os.O_RDONLY:
            return -errno.EACCES
        else:
            return 0

    def read(self, path, size, offset):
        genre, station = self.split_path(path)
        info = self.root.genres[genre].stations[station]
        if offset < info.st_size:
            if offset + size > info.st_size:
                size = info.st_size - offset
            return info.content[offset:offset+size]
        else:
            return ''

if __name__ == '__main__':
    fs = ShoutcastFS()
    fs.parse(errex=1)
    fs.multithreaded = False
    fs.main()

To try the file system, run:

$ # Download shoutcast.py
$ wget http://github.com/mariob/pyshoutcast/raw/master/src/shoutcast.py
$ mkdir mnt
$ ./shoutcastfs mnt
$ cd mnt/
$ ls
...A list of genres...
$ cd Samba
$ ls
...A list of 'Samba' stations...
$ cat [station name]
...Playlist data...

Have fun!

Fedora 12

2009-11-23T21:07:00.000+01:00

For the moment I'm evaluating Fedora 12. I was a bit disappointed on the Kubuntu 9.10 release, the distribution felt a little bit slow and tiered. The only thing I'm satisfied with is the boot time, it starts quite fast.

The KDE4 desktop in Fedora feels kind of snappier than KDE in Kubuntu. I'm not sure if it's because Fedora is compiled for i686 and not i386 or if the team have done any optimizations. I didn't enabled desktop effects in neither distribution. I installed the nvidia proprietary driver in Kubuntu but use Nouveau (open source driver) in Fedora. I might try the proprietary driver in Fedora to see if I gain any speed.

There's one bad thing I found in Fedora in comparision with Kubuntu, KPackageKit. It takes ages before starting compared to the Kubuntu version so I'm using yum from the shell prompt instead. Well, to be honest, I'm not a heavy user of UI front-ends to package management systems. In Kubuntu I prefer using aptitude instead of KPackageKit.

I think I'll give Fedora a chance and try it out for a while.

FUSE - Filesystem in Userspace part 2

2009-11-19T07:48:00.004+01:00

This is the second post on implementing file systems using FUSE. In this post I'll show you how to create a read-only file system that is backed by an XML file.Each tag in the XML file is either a file or a directory. If the tag is representing a directory, it should have a is_dir="1" attribute added. When the tag represents a file it should have its content between the tags. The following XML snippet shows an example:


<root is_dir="1">

  <a is_dir="1">
    <file1.txt>File content for this file</file1.txt>
    <hello>Hello World</hello>
  </a>

  <empty_dir is_dir="1" />

</root>

The example models the following directory tree:


/
|-- a
|   |-- file1.txt
|   `-- hello
`-- empty_dir

There are some limitations in having an XML file representing a file system. You can't for example have filenames starting with a dot because tags are not allowed to start with a dot in XML.

On to the code, the following snippet implements the read-only file system which is backed by an XML file:


#!/usr/bin/env python
# -*- coding: utf-8 -*-
import errno
import fuse
import stat
import os
import time
import xml.etree.ElementTree as etree

fuse.fuse_python_api = (0, 2)

# Use same timestamp for all files
_file_timestamp = int(time.time())

class MyStat(fuse.Stat):
    """
    Convenient class for Stat objects.
    Set up the stat object with appropriate
    values depending on constructor args.
    """
    def __init__(self, is_dir, size):
        fuse.Stat.__init__(self)
        if is_dir:
            self.st_mode = stat.S_IFDIR | 0555
            self.st_nlink = 2
        else:
            self.st_mode = stat.S_IFREG | 0444
            self.st_nlink = 1
            self.st_size = size
        self.st_atime = _file_timestamp
        self.st_mtime = _file_timestamp
        self.st_ctime = _file_timestamp

class MyFS(fuse.Fuse):
    def __init__(self, xml_tree, *args, **kw):
        fuse.Fuse.__init__(self, *args, **kw)
        self.tree = xml_tree

    def getattr(self, path):
        # We do not support 'dot' files
        # since xml tags cannot start with a dot.
        if path.find('/.') != -1:
            return -errno.ENOENT

        entry = self.tree.find(path)

        if entry is None:
            return -errno.ENOENT
        else:
            is_dir = entry.get('is_dir', False)
            size = entry.text and len(entry.text.strip()) or 0
            return MyStat(is_dir, size)

    def readdir(self, path, offset):
        yield fuse.Direntry('.')
        yield fuse.Direntry('..')
        for e in self.tree.find(path).getchildren():
            yield fuse.Direntry(e.tag)

    def open(self, path, flags):
        # Only support for 'READ ONLY' flag
        access_flags = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
        if flags & access_flags != os.O_RDONLY:
            return -errno.EACCES
        else:
            return 0

    def read(self, path, size, offset):
        entry = self.tree.find(path)
        content = entry.text and entry.text.strip() or ''
        file_size = len(content)
        if offset < file_size:
            if offset + size > file_size:
                size = file_size - offset
            return content[offset:offset+size]
        else:
            return ''

if __name__ == '__main__':
    tree = etree.parse('tree.xml')

    fs = MyFS(tree)
    fs.parse(errex=1)
    fs.main()

I'm using the ElementTree XML API to parse the XML file. The ElementTree API is very easy to use and supports finding elements by specifying a path which fits very well in this context. For example, the following snippet shows how to get the file1.txt element from the example XML file above and extract the content between the tags:


import xml.etree.ElementTree as etree
tree = etree.parse('tree.xml')
el = tree.find('/a/file1.txt')
print el.text.strip() # Remove any white-spaces

The code hasn't change that much since the part 1 post. I've added two new methods which adds support for opening and reading files. The MyStat class is only used as an convenient class to help creating appropriate stat objects. You might notice that I don't do a lot of checking in the code, this is because FUSE do a lot of them for me, this page list some of the assumptions you can make when implementing file system using FUSE.

Did you believe it would be this easy to create a mountable file system that uses an XML file for the layout? I didn't.

In the next (and final) post about FUSE I think I'll create some kind of 'use a service on the Internet' file system which can be useful and not just another toy fs.

FUSE - Filesystem in Userspace part 1

2009-11-15T20:50:00.005+01:00

Here's the first part in a series of posts on implementing file systems using FUSE.

Creating a file system can seem to be an intimidating task but with FUSE it's very easy and it gets even easier with python bindings. Any decent Linux distribution should include support for FUSE and be configured such that regular users (or a group) can mount file systems written with FUSE without having root access.

If you are running Ubuntu you need to install the python-fuse package which contains the Python bindings for FUSE.

- Enough, show me the code

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import errno
import fuse
import stat
import time

fuse.fuse_python_api = (0, 2)

class MyFS(fuse.Fuse):
    def __init__(self, *args, **kw):
        fuse.Fuse.__init__(self, *args, **kw)

    def getattr(self, path):
        st = fuse.Stat()
        st.st_mode = stat.S_IFDIR | 0755
        st.st_nlink = 2
        st.st_atime = int(time.time())
        st.st_mtime = st.st_atime
        st.st_ctime = st.st_atime

        if path == '/':
            pass
        else:
            return - errno.ENOENT
        return st

if __name__ == '__main__':
    fs = MyFS()
    fs.parse(errex=1)
    fs.main()

The example above shows how easy it is to create a mountable file system, it does nothing but is still a valid file system. The MyFS class implements a file system with no entries, it will always return ENOENT (No such file or directory) except for the root path. You should take a look at the FUSE Python Reference page which contains details about FusePython and, among other things, describes the stat object returned from the getattr method.

Paste the example to a file, MyFS.py, and make the file executable:

$ chmod 755 MyFS.py

Now create a directory which will be used as mount point and mount the file system:

$ mkdir myfs
$ ls -l
totalt 4
drwxr-xr-x 2 mario mario 4096 2009-11-15 15:59 myfs
-rwxr-xr-x 1 mario mario  648 2009-11-15 15:59 MyFS.py
$ # This will mount the file system, try ./MyFS.py --help
$ ./MyFS.py myfs
$ ls -l
totalt 0
drwxr-xr-x 2 root  root    0 2009-11-15 16:10 myfs
-rwxr-xr-x 1 mario mario 648 2009-11-15 15:59 MyFS.py

Here we can see that the owner, size and time stamp changed on myfs after we mounted the file system. You can cd into myfs, but if you try to list the directory (file system root) you'll get an error:


$ cd myfs
$ ls
ls: reading directory .: Function not implemented

You got this error because we haven't implemented the readdir method which is invoked when listing a directory. To make this work we need to add a readdir method which returns (at least) the '.' and '..' entries. Add the following snippet to the class:


    def readdir(self, path, offset):
        for e in '.', '..':
            yield fuse.Direntry(e)

Before you continue you should unmount the file system:


$ fusermount -u myfs

This will unmount your FUSE file system as a regular user and doesn't require root access.

Retry to mount and cd into the myfs directory:


$ ./MyFS.py myfs
$ cd myfs
$ ls -a
.  ..

Alright, I think I'll stop for today.Oh, just one more thing... You won't get any print outputs nor tracebacks from your code if you don't start the file system in foreground mode. This is done by adding '-f' as argument when starting the file system.


$ ./MyFS.py -f myfs

It can be hard to tell why something doesn't work if you can't see tracebacks and print outputs.

GMonitor

2009-11-05T22:47:00.101+01:00

Ah, I've finally managed to make my new fresh Kubuntu 9.10 installation usable. Now it's time to do some KDE stuff and I'll show you how easy it's to create a plasmoid (applet/widget) for KDE4.

I've created a simple applet which shows the number of unread mail in a Gmail inbox. We already knowhow to fetch the list with unread mail, which I have explained in a previous blog post. Now I'll create a nice applet showing the count. I'm going to keep it assimple as possible, writing hard-coded values, skip error handling andso on to avoid making things more complicated than necessary.

Thereare several good tutorials at techbase.kde.org which explains plasmaprogramming. They cover the basics and explains things in detail soI'll only make references to the pages instead of repeating what'salready been written.

The first tutorial you should read (youdon't have to read it right now, you can read it after you tried doingthe applet) is Python Plasma Getting Started. The tutorial coverssetting up a simple plasmoid, packaging, installing and running.

I'll name the plasmoid GMonitor because it only monitors the mailbox andshows the number of unread mail, it will not actually try to do somekind of notification (yet?!).

I've put the code on github, you can clone it with:

$ git clone git://github.com/mariob/gmonitor.git

You'll find a Makefile in the git which can be useful when doing plasmoids. The Makefile supports installing, un-installing, viewing, packaging and updating the plasmoid.

The directory tree looks like this:

gmonitor/
|-- Makefile
|-- README
|-- contents
|   `-- code
|       `-- main.py
`-- metadata.desktop

The gmonitor directory is the project home and contents/code contains the python source. The metadata.desktop file will be explained below.

To be able to install and run your applet you need to provide a metadata.desktop file to plasma. The metadata.desktop file containsimportant information about the applet and you can read more about thefile in the Plasma Getting Started tutorial.

Feel free to change the file. The Name specifies the applet nameand the Icon field gives the name to the icon to associated with thisapplet. These two fields are typically shown when listing applets inthe 'Add Widget' dialog. There are two important fields whichmust be present for plasma to run your applet, the X-Plasma-API fieldspecifies which script-engine to be used and theX-Plasma-MainScript field which script to be executed. It's also good to know that the X-KDE-PluginInfo-Name fields is used as a plasmoid identification, so it should be unique.

Some notes about the implementation. There are two classes, GMonitor and MailFrame (I know, poor name). The MaiFrame class has an icon and a label. When the mail count is set to zero (by calling setCount(cnt)), the icon is disabled, grayed and the label is set to 'No new mail'. When the mail count is set to a value greater than zero, the icon is enabled, colored and the label is set to 'Count: x', where x is the number of mail in the inbox. See the images below.

Sample of what the applet looks like

When the icon is enabled and clicked, a 'clicked()' signal is emitted. The GMonitor class connects the openBrowser() method to the 'clicked()' signal. The openBrowser() method opens a default browser in KDE to load the google mail url. The GMonitor also sets up a timer to fetch the feed in 60 second interval. The fetchFeed() method uses the KIO framework in KDE to download the content from a URL and parseFeed() parses the downloaded feed and emits a 'mailcount' signal.

The implementation depends on feedparser which can be installed on Ubuntu by running:

$ sudo aptitude install python-feedparser

You can also download and put the feedparser.py file inside the code directory.

Not that bad. It's actually possible to create a very simple 'good looking' Gmail monitor in less than 100 lines of code (excluding comments) with Python and KDE.

To view this plasmoid without installing it run (while standing in the gmonitor folder):

$ make view

This might only work on KDE4.3, not sure if the plasmoidviewer in 4.2 supports running plasmoids without first installing them. If it fails you should install the plasmoid and run plasmoidviewer yourself (or modify the makefile):

$ make install
$ plasmoidviewer pysnippet-gmonitor

If you never heard of Qt signals and slots you should read this introduction on the topic. For example, a button can 'emit' a signal when it's clicked. A slot (function/method) can be connected to a signal and each time the signal is emitted the function/method will be called.

You can find the Python KDE4.3 API docs here. Unfortunately, I don't find them as good as the Qt docs.

When running the applet, KDE will ask for the username/password when it tries to connect to Google. You can check the 'remember the password' checkbox which will make KDE cache the password as long as the desktop session is alive. If you choose not to, you'll have to enter your username/password each time the code tries to fetch the feed. A better solution is to use KWallet.

Ok, hope you enjoyed it and start doing cool plasmoids!

Kubuntu 9.10

2009-11-02T21:52:00.000+01:00

I've installed Kubuntu 9.10 and I'm busy configuring it to a usable state. For the moment I don't have too much spare time but I'm almost done.

I hope I'll have time to do a post about creating plasmoids (KDE4 desktop widgets) in the next couple of days. I think I'll do a gmonitor applet which will show the number of unread mail in your Gmail inbox.

DIY: Google Mail Notifier

2009-10-28T21:50:00.004+01:00

It's quite easy to create your own GMail notifier. Your inbox, gmail or domain, can be monitored by fetching and parsing an atom feed.

Following code snippet prints new (unread) mail in your inbox:

# -*- coding: utf-8 -*-
import feedparser
import urllib2

# Create http basic auth handler
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password('New mail feed', 'https://mail.google.com/',
                          'USERNAME@DOMAIN', 'PASSWORD')

# Open url using the auth handler
opener = urllib2.build_opener(auth_handler)
feed_file = opener.open('https://mail.google.com/mail/feed/atom/')

# Parse feed using feedparser
d = feedparser.parse(feed_file)

# Print mail count and mails
print 'Mail count:', d.feed.fullcount

for entry in d.entries:
        print '----------------------------------------------'
 print 'Author: ', entry.author
 print 'Subject:', entry.title
 print 'Summary:', entry.summary

Just change USERNAME and PASSWORD. You also need to download feedparser which is free to use, or write one yourself.

Have fun!

ELCE 2009 presentations

2009-10-28T09:44:00.000+01:00

The presentations from ELCE 2009 are now available from here. Some of the presentations have not been posted yet.

pyshoutcast v0.7.1

2009-10-26T21:19:00.000+01:00

pyshoutcast version 0.7.1 has just been released. It implements a Python shoutcast API which makes it possible to list genres, stations, do search, etc, using the shoutcast yellow pages service.

Unfortunately, I just found a forum post stating that Amarok and VLC have removed support for shoutcast due to licensing issues. I'm not sure why because I can't find any info about it, I'll try to find out more details.

Anyway, to try it out you can start a interactive python prompt and paste the following code:

>>> import shoutcast
>>> api = shoutcast.ShoutCast()
>>> api.random()

'with' statement

2009-10-21T12:56:00.007+02:00

Python have a great number of cool language features and one of them is the with statement. The with statement can be used in numerous ways but I think that the most common usage is to track and release some kind of resource and at the same time encapsulate the code block with a try/finally clause.

For example:

with open('my_file.txt') as f:
    for line in f:
        print line

which could be translated to something like this:

f = open('my_file.txt')
try:
    for line in f:
        print line
finally:
    f.close()

and in a general form:

with <EXPRESSION> [as <VAR>]:
    <BLOCK>

The with statement guarantees that the file will be closed no matter what happens, for example an exception is thrown or a return is executed inside the for statement etc. The result from the expression that is evaluated after the with keyword will be bound to the variable specified after the as keyword. The as part is optional and I'll show you a use case later.

But wait a minute, how does the with keyword know how to close the file? Well, the magic is called 'Context Manager'.

A context manager is a class which defines two methods, __enter__() and __exit__(). The expression in the with statement must evaluate to a context manager else you'll get an AttributeError. The __enter__ method is called when the with statement is evaluated and should return the value which should be assigned to the variable specified after the as keyword. After the block has finished executing the __exit__ method is called. The __exit__ method will be passed any exception occurred while executing the block and can decide if the exception should be muted by returning True or be propagated to the caller by returning False.

Let's look at the following example (not a good design, just an example):

class TransactionContextManager(object):
    def __init__(self, obj):
        self.obj = obj

    def __enter__(self):
        self.obj.begin()
        return self.obj

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type:
            self.obj.rollback()
        else:
            self.obj.commit()
        return False

class Transaction(object):
    def begin(self):
        print "Start transaction"

    def query(self, sql):
        print "SQL:", sql

    def rollback(self):
        print "Rollback transaction"

    def commit(self):
        print "Commit transaction"

print "Before 'with'"

with TransactionContextManager(Transaction()) as tx:
    tx.query("SELECT * FROM users")

print "After 'with'"

The code produces the following output:

Before 'with'
Start transaction
SQL: SELECT * FROM users
Commit transaction
After 'with'

It's easy to trace the execution but I'll explain it anyway.

The code starts by printing "Before 'with'".
The with statement is executed and instances of TransactionContextManager and Transaction are created. The context manager's __enter__ method is invoked which will invoke the Transaction instance's begin method. The begin method prints "Start transaction" and finally the __enter__ method returns the Transaction instance.
The return value from the context manager's __enter__ method is bound to the variable tx.
The block is executed and the query method is called.
The query method prints "SQL: SELECT * FROM users".
The block is done executing and the context manager's __exit__ method is invoked. Since no exception occurred the commit method is invoked on the Transaction instance. The __exit__ method returns False which means that any exceptions should be propagated to the caller, none in this case.
The code ends executing by printing "After 'with'".

Let's change the Transaction class so that it throws an exception from the query method. This should execute the rollback method instead of the commit method. The exception should also be propagated to the caller since we return False from the __exit__ method.

def query(self, sql):
    raise Exception("SQL Error occurred")

And the output looks like following:

Before 'with'
Start transaction
Rollback transaction
Traceback (most recent call last):
  File "transaction.py", line 32, in 
    tx.query("SELECT * FROM users")
  File "transaction.py", line 21, in query
    raise Exception("SQL Error occurred")
Exception: SQL Error occurred

Here we see that the rollback method is called instead of commit. The rollback method is called because the code inside the with block raised an exception and a valid exc_type argument was passed to the __exit__ method. We can also see that the exception is propagated to the caller because we return False from the __exit__ method.

If we change the __exit__ method to return True instead of False we get the following output:

Before 'with'
Start transaction
Rollback transaction
After 'with'

As expected, we don't see the exception anymore. Note that the __exit__ method is not called if an exception occurs before the with statement starts executing the block.

Alright, I hope you agree with me that the with statement is very useful. Now I'll show you a final example where we don't need to bind the return value from the context manager to a variable. I've re-factored the previous example and hopefully made a better design.

class Connection(object):
    def __init__(self):
        # Default behaviour is to do auto commits
        self._auto_commit = True

    def connect(self):
        pass

    def auto_commit(self):
        return self._auto_commit

    def set_auto_commit(self, mode):
        print "Auto commit:", mode
        self._auto_commit = mode

    def rollback(self):
        print "Rollback transaction"

    def commit(self):
        print "Commit transaction"

    def executeQuery(self, sql):
        # Handle auto commit here if it's enabled
        print "SQL:", sql

class Transaction(object):
    def __init__(self, conn):
        self.conn = conn
        self.auto_commit = False

    def __enter__(self):
        self.auto_commit = self.conn.auto_commit()
        self.conn.set_auto_commit(False)
        return self.conn

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type:
            self.conn.rollback()
        else:
            self.conn.commit()

        self.conn.set_auto_commit(self.auto_commit)
        return False

conn = Connection()
conn.connect()

with Transaction(conn):
    conn.executeQuery("SELECT a user FROM users")
    conn.executeQuery("INSERT some data INTO users")

The output:

Auto commit: False
SQL: SELECT a user FROM users
SQL: INSERT some data INTO users
Commit transaction
Auto commit: True

As the example shows, we don't need to use the return value from the context manager to still have use of them. We only want to be sure that the queries inside the with block is executed in the same transaction and rollbacked if an exception occurs or commited if successful. Without the with statement the code would be implemented something like the following snippet:

conn = Connection()
conn.connect()
auto_comm = conn.auto_commit()

try:
    conn.set_auto_commit(False)
    conn.executeQuery("SELECT a user FROM users")
    conn.executeQuery("INSERT some data INTO users")
    conn.commit()
except:
    conn.rollback()
    raise
finally:
    conn.set_auto_commit(auto_comm)

Well, for me the with statement seems more clean and convenient to use than the last example.

Back home from ELCE 2009

2009-10-18T19:19:00.000+02:00

It has been a great visit to Grenoble. I'm pleased with the conference because most of the sessions were held by developers, which I prefer, and not marketing people. The major focus seemed to be on build systems and reducing boot time. I learned a lot about tools which can give you hints on performance issues etc. Some celebrities also showed up like Jon Masters and Alessandro Rubini.

I assume that the slides from the sessions will be available on the home page so you can take a look at them yourself.

Adeneo had a cool demo of booting up a Linux system with a Qt-app doing 3D (software rendering) in less than 5 seconds. Unfortunately I can't give you a clip since the new blogger editor doesn't support movie clips, or am I wrong?. A picture will do.

The conference had only a little exhibiton hall (which was ok for me) and here's a picture of Free Electrons' booth.

The food was a success! A lot of different finger food and of course wine. Here's an example of what the dessert looked like the second day.

The restroom has a really cool designed washbasin.

As I mentioned before I'm pleased with the conference and I believe I'll attend again next year.

I'm going to ELCE 2009

2009-10-13T20:27:00.002+02:00

I'll be visiting the Embedded LInux Conference Europe 2009 which is held in Grenoble, France. It's a two day conference, 15-16 October. I'm looking forward to attend the sessions about build systems and other cool low-level kernel stuff.

Hope I'll meet some interesting people.

KDE Hello World

2009-10-10T23:05:00.002+02:00

I have plans to do a web radio player (I wonder if we'll ever see this app). My intention is to learn Qt/KDE programming in Python. I've played around with Qt a bit and I'm starting to get comfy. I've decided to try doing the player as a KDE app, just to learn more about KDE. Qt is great for doing cross-platform stuff but I want to take a look at what KDE offers on top of Qt. You can find docs, tutorials and more on KDE TechBase.The following snippet shows the minimum amount of code needed to create a typical KDE app in Python.

from PyKDE4 import kdecore
from PyKDE4 import kdeui
import sys

def createAboutData():
    """
    Create a KAboutData with information about this application.
    """
    return kdecore.KAboutData(
            # Program name used internally
            "hello",
            
            # Catalog name
            "",                       
            
            # Displayable program name
            kdecore.ki18n("Hello World app"),
            
            # Program version
            "0.1.0",
            
            # Short description about the program
            kdecore.ki18n("A simple KDE Hello World app"),
            
            # Program license
            kdecore.KAboutData.License_BSD,
            
            # Copyright statement
            kdecore.ki18n ("(c) 2009 Mario Boikov"),
            
            # Free form text
            kdecore.ki18n("Free form text\nsupporting newlines"),
            
            # Home page address for this program
            "http://www.pysnippet.com",
            
            # Bug report email address
            "mario@beblue.org",
            )

def main():
    about = createAboutData()
    kdecore.KCmdLineArgs.init(sys.argv, about)

    app = kdeui.KApplication()

    # INSERT APP CODE HERE

    # Start event loop
    sys.exit(app.exec_())

if __name__ == '__main__':
    main()

Nothing special will happen if you execute this application, we need some kind of window/widget to display something. To accomplish this we can create a KMainWindow with a KPushButton. We'll also add code to print Hello World when the button is clicked. The following snippet will create a window with a push button:

from PyKDE4 import kdeui
from PyKDE4 import kdecore
from PyQt4 import QtCore

class MainWindow(kdeui.KMainWindow):
    def __init__(self):
        super(MainWindow, self).__init__()

        # Create widget with vertical box layout manager
        layout = kdeui.KVBox()

        # Create push button and add to layout
        button = kdeui.KPushButton(kdecore.i18n("Push Me!"), layout)

        # Connect to button's 'clicked' signal 
        QtCore.QObject.connect(button, QtCore.SIGNAL("clicked()"),
                               self.buttonClicked)

        self.setCentralWidget(layout)

    def buttonClicked(self):
        print "Hello World!"

You might wonder why I use ki18n() when creating the KAboutData and i18n() for the MainWindow. Both functions do translations, i18n() requires an instance of KApplication before use but ki18n() don't.

To actually show the window we have to instantiate it and make it visible, the "INSERT APP CODE HERE" comment should be replaced with the following lines:

    mainWindow = MainWindow()
    mainWindow.show()

As you can see, it doesn't take too much effort to create a simple app in KDE. There's almost more code to create the KAboutData than doing the window with a push button.Here's a screen shot of what the window looks like on my Kubuntu installation:

If we compare Qt and KDE there aren't too much differences, at least with this example. The code below implements the same Hello World app made with Qt:

import sys
from PyQt4 import QtCore, QtGui

class MainWindow(QtGui.QMainWindow):
    def __init__(self, title):
        super(MainWindow, self).__init__()
        self.setWindowTitle(title)

        centralWidget = QtGui.QWidget()

        button = QtGui.QPushButton("Push Me!")

        layout = QtGui.QVBoxLayout()
        layout.addWidget(button)

        centralWidget.setLayout(layout)

        QtCore.QObject.connect(button, QtCore.SIGNAL("clicked()"),
                               self.buttonClicked)

    self.setCentralWidget(centralWidget)

    def buttonClicked(self):
        print "Hello World!"

def main():
    app = QtGui.QApplication(sys.argv)

    mainWindow = MainWindow("Hello World App")
    mainWindow.show()

    sys.exit(app.exec_())

if __name__ == "__main__":
    main()

The biggest difference is how the layout is handled and the absence of About Data.

Python and unit testing

2009-10-07T21:20:00.021+02:00

Writing unit tests in python is really easy. It's the dynamic properties of the language which makes it very simple to mock class methods, instance methods and functions. This makes the task of writing isolated unit tests easy when you practice test-driven development.

Say you have the following class:


import urllib2

class UrlDownloader(object):
    def download(self, url):
        """
        Read the content from the url.
        The content is returned as a string object.
        """
        return urllib2.urlopen(url).read()

You actually don't want to download something from the net when you do a unit test for this class. You would like to have a fixed string returned from the read() method. This can be hard to achieve in some languages (for example Java) but that's not true for Python. Python supports redefining class methods, functions and other things in run-time. The following snippet shows how easy it's to mock the urlopen() function in the urllib2 module to return a mocked file-like object:

import urllib2
import unittest

class MockFileObject(object):
    def read(self):
        return "DATA"

def mock_open(url):
    return MockFileObject()

class UrlDownloaderTest(unittest.TestCase):
    def testDownload(self):
        urllib2.urlopen = mock_open
        dl = UrlDownloader()
        self.assertEqual(dl.download(""), "DATA")

if __name__ == '__main__':
    unittest.main()

In this unit test we redefine the urlopen-function (remember, this is not a member of an instance, it's a plain function) with our own which returns a mock file-like object. It's not that hard to imagen all the possibilities this gives when writing tests.There are of course frameworks that will help you with creating mocks etc, you could try The Python Mock Module which should cover most of the use cases.

File-like objects

2009-10-02T13:45:00.068+02:00

A Python file-like object can be seen as an abstraction to input/output streams. They are actually similar to Java's java.io.Reader/java.io.Writer abstraction (I'm coming from Java). The typical usage of file-like objects can be found in frameworks and APIs. The file-like object provides a generic way of reading/writing data and would fit an HTML-parser for example. The parser is not interested of where the data come from (files, sockets, etc), only the content, so naturally the implementation should not depend on how to open resources etc.

Basic methods that need to be implemented by a file-like object are:

read([size]) - read at most size bytes from the file.
readLine([size]) - read one entire line from the file.
write(str) - write a string to the file.
close() - close the file.

You should read the documentation of the function/method that uses the file-like object, often they specify which methods that are expected to be implemented by the file-like object. If the file-like object is only used for reading, no write-functions need to be supported. More information about file-like objects can be found here.

Ok, so why should you use file-like objects in your code?

Let's take a look at the ConfigParser class which can be used to read/write configuration data. You can expect to find methods for reading and writing configuration data using files but the class does also include support for file-like objects, which is great. For example it's very easy to write unittests for this class, thanks to the support of file-like objects, without having to create real files with test data. To create a file-like object from a string we can use the StringIO module. The module have both read and write support.

Here's some code:

import unittest
import StringIO
import ConfigParser

class ConfigParserTest(unittest.TestCase):

    def testHasSection(self):
        """
        has_section() should return True if the section exists
        and False if is doesn't.
        """
        data = StringIO.StringIO("[My section]\nSetting=1\n")
        config = ConfigParser.ConfigParser()
        config.readfp(data)
        self.assertTrue(config.has_section("My section"))
        self.assertFalse(config.has_section("Not a section"))

    def testMissingHeader(self):
        """
        readfp() should raise MissingSectionHeaderError
        when parsing a file without section headers.
        """
        data = StringIO.StringIO("\nSetting=1\n")

        config = ConfigParser.ConfigParser()
        self.assertRaises(ConfigParser.MissingSectionHeaderError,
                          config.readfp, data)

if __name__ == "__main__":
    unittest.main()

And a simple write configuration test:

import unittest
import StringIO
import ConfigParser

class WriteConfigTest(unittest.TestCase):

    def testWriteConfig(self):
        """
        Test write a simple config
        """
        data = StringIO.StringIO()
        config = ConfigParser.ConfigParser()
        config.add_section("My Section")
        config.set("My Section", "my_option", "my_value")
        config.write(data)

        expected = "[My Section]\nmy_option = my_value\n\n"
        self.assertEquals(expected, data.getvalue())

if __name__ == "__main__":
    unittest.main()

These unittests are of course only examples and do not fully test the ConfigParser class.

It's easy to see that file-like objects are very handy and fits well in frameworks and APIs.There are of course many advantages of using an abstraction when reading/writing data, some of them are:

Implementations do not depend on real files, socket, etc, only on a file-like object.
Easier to fake data in unittests.
Caller responsible of opening/closing the resource.
Less error-handling in framework code, delegate exceptions to the caller.

The downside with the current file-like object abstraction is that it's tailored for text I/O. In Python 2.6 and 3.x a new module have been added named io. The module originates from the New I/O PEP. The module enriches Python with an three-layer solution, the raw I/O layer, the buffered I/O layer and finally the text I/O layer. The raw and buffered layers deals with units of bytes and the text layer deals with units of characters. There are some things you should be aware of if you use the new I/O library in Python 2.6, read more about it here.

I've implemented a shoutcast playlist parser, plsparser, using the ConfigParser class. The parser is implemented as a Python generator function. Actually, I might do a post about generator functions someday.

Hello World

2009-10-01T12:46:00.005+02:00

What would be more appropriate than a simple Hello World as the first code snippet!

def main():
    print "Hello World"

if __name__ == "__main__":
    main()