Fedora 12

For the moment I'm evaluating Fedora 12. I was a bit disappointed on the Kubuntu 9.10 release, the distribution felt a little bit slow and tiered. The only thing I'm satisfied with is the boot time, it starts quite fast.

The KDE4 desktop in Fedora feels kind of snappier than KDE in Kubuntu. I'm not sure if it's because Fedora is compiled for i686 and not i386 or if the team have done any optimizations. I didn't enabled desktop effects in neither distribution. I installed the nvidia proprietary driver in Kubuntu but use Nouveau (open source driver) in Fedora. I might try the proprietary driver in Fedora to see if I gain any speed.

There's one bad thing I found in Fedora in comparision with Kubuntu, KPackageKit. It takes ages before starting compared to the Kubuntu version so I'm using yum from the shell prompt instead. Well, to be honest, I'm not a heavy user of UI front-ends to package management systems. In Kubuntu I prefer using aptitude instead of KPackageKit.

I think I'll give Fedora a chance and try it out for a while.

FUSE - Filesystem in Userspace part 2

This is the second post on implementing file systems using FUSE. In this post I'll show you how to create a read-only file system that is backed by an XML file. Each tag in the XML file is either a file or a directory. If the tag is representing a directory, it should have a is_dir="1" attribute added. When the tag represents a file it should have its content between the tags. The following XML snippet shows an example:

<root is_dir="1">

  <a is_dir="1">
    <file1.txt>File content for this file</file1.txt>
    <hello>Hello World</hello>
  </a>

  <empty_dir is_dir="1" />

</root>

The example models the following directory tree:

/
|-- a
|   |-- file1.txt
|   `-- hello
`-- empty_dir

There are some limitations in having an XML file representing a file system. You can't for example have filenames starting with a dot because tags are not allowed to start with a dot in XML.

On to the code, the following snippet implements the read-only file system which is backed by an XML file:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import errno
import fuse
import stat
import os
import time
import xml.etree.ElementTree as etree

fuse.fuse_python_api = (0, 2)

# Use same timestamp for all files
_file_timestamp = int(time.time())

class MyStat(fuse.Stat):
    """
    Convenient class for Stat objects.
    Set up the stat object with appropriate
    values depending on constructor args.
    """
    def __init__(self, is_dir, size):
        fuse.Stat.__init__(self)
        if is_dir:
            self.st_mode = stat.S_IFDIR | 0555
            self.st_nlink = 2
        else:
            self.st_mode = stat.S_IFREG | 0444
            self.st_nlink = 1
            self.st_size = size
        self.st_atime = _file_timestamp
        self.st_mtime = _file_timestamp
        self.st_ctime = _file_timestamp

class MyFS(fuse.Fuse):
    def __init__(self, xml_tree, *args, **kw):
        fuse.Fuse.__init__(self, *args, **kw)
        self.tree = xml_tree

    def getattr(self, path):
        # We do not support 'dot' files
        # since xml tags cannot start with a dot.
        if path.find('/.') != -1:
            return -errno.ENOENT

        entry = self.tree.find(path)

        if entry is None:
            return -errno.ENOENT
        else:
            is_dir = entry.get('is_dir', False)
            size = entry.text and len(entry.text.strip()) or 0
            return MyStat(is_dir, size)

    def readdir(self, path, offset):
        yield fuse.Direntry('.')
        yield fuse.Direntry('..')
        for e in self.tree.find(path).getchildren():
            yield fuse.Direntry(e.tag)

    def open(self, path, flags):
        # Only support for 'READ ONLY' flag
        access_flags = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
        if flags & access_flags != os.O_RDONLY:
            return -errno.EACCES
        else:
            return 0

    def read(self, path, size, offset):
        entry = self.tree.find(path)
        content = entry.text and entry.text.strip() or ''
        file_size = len(content)
        if offset < file_size:
            if offset + size > file_size:
                size = file_size - offset
            return content[offset:offset+size]
        else:
            return ''

if __name__ == '__main__':
    tree = etree.parse('tree.xml')

    fs = MyFS(tree)
    fs.parse(errex=1)
    fs.main()

I'm using the ElementTree XML API to parse the XML file. The ElementTree API is very easy to use and supports finding elements by specifying a path which fits very well in this context. For example, the following snippet shows how to get the file1.txt element from the example XML file above and extract the content between the tags:

import xml.etree.ElementTree as etree
tree = etree.parse('tree.xml')
el = tree.find('/a/file1.txt')
print el.text.strip() # Remove any white-spaces

The code hasn't change that much since the part 1 post. I've added two new methods which adds support for opening and reading files. The MyStat class is only used as an convenient class to help creating appropriate stat objects. You might notice that I don't do a lot of checking in the code, this is because FUSE do a lot of them for me, this page list some of the assumptions you can make when implementing file system using FUSE.

Did you believe it would be this easy to create a mountable file system that uses an XML file for the layout? I didn't.

In the next (and final) post about FUSE I think I'll create some kind of 'use a service on the Internet' file system which can be useful and not just another toy fs.

FUSE - Filesystem in Userspace part 1

Here's the first part in a series of posts on implementing file systems using FUSE.

Creating a file system can seem to be an intimidating task but with FUSE it's very easy and it gets even easier with python bindings. Any decent Linux distribution should include support for FUSE and be configured such that regular users (or a group) can mount file systems written with FUSE without having root access.

If you are running Ubuntu you need to install the python-fuse package which contains the Python bindings for FUSE.

- Enough, show me the code

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import errno
import fuse
import stat
import time

fuse.fuse_python_api = (0, 2)

class MyFS(fuse.Fuse):
    def __init__(self, *args, **kw):
        fuse.Fuse.__init__(self, *args, **kw)

    def getattr(self, path):
        st = fuse.Stat()
        st.st_mode = stat.S_IFDIR | 0755
        st.st_nlink = 2
        st.st_atime = int(time.time())
        st.st_mtime = st.st_atime
        st.st_ctime = st.st_atime

        if path == '/':
            pass
        else:
            return - errno.ENOENT
        return st

if __name__ == '__main__':
    fs = MyFS()
    fs.parse(errex=1)
    fs.main()

The example above shows how easy it is to create a mountable file system, it does nothing but is still a valid file system. The MyFS class implements a file system with no entries, it will always return ENOENT (No such file or directory) except for the root path. You should take a look at the FUSE Python Reference page which contains details about FusePython and, among other things, describes the stat object returned from the getattr method.

Paste the example to a file, MyFS.py, and make the file executable:

$ chmod 755 MyFS.py

Now create a directory which will be used as mount point and mount the file system:

$ mkdir myfs
$ ls -l
totalt 4
drwxr-xr-x 2 mario mario 4096 2009-11-15 15:59 myfs
-rwxr-xr-x 1 mario mario  648 2009-11-15 15:59 MyFS.py
$ # This will mount the file system, try ./MyFS.py --help
$ ./MyFS.py myfs
$ ls -l
totalt 0
drwxr-xr-x 2 root  root    0 2009-11-15 16:10 myfs
-rwxr-xr-x 1 mario mario 648 2009-11-15 15:59 MyFS.py

Here we can see that the owner, size and time stamp changed on myfs after we mounted the file system. You can cd into myfs, but if you try to list the directory (file system root) you'll get an error:

$ cd myfs
$ ls
ls: reading directory .: Function not implemented

You got this error because we haven't implemented the readdir method which is invoked when listing a directory. To make this work we need to add a readdir method which returns (at least) the '.' and '..' entries. Add the following snippet to the class:

    def readdir(self, path, offset):
        for e in '.', '..':
            yield fuse.Direntry(e) 

Before you continue you should unmount the file system:

$ fusermount -u myfs

This will unmount your FUSE file system as a regular user and doesn't require root access.

Retry to mount and cd into the myfs directory:

$ ./MyFS.py myfs
$ cd myfs
$ ls -a
.  ..

Alright, I think I'll stop for today. Oh, just one more thing... You won't get any print outputs nor tracebacks from your code if you don't start the file system in foreground mode. This is done by adding '-f' as argument when starting the file system.

$ ./MyFS.py -f myfs

It can be hard to tell why something doesn't work if you can't see tracebacks and print outputs.