By romkey, CC BY-NC 2.0

Woodchuck

Woodchuck is a research project investigating how to improve data availability on mobile devices.

Mobile devices should provide users with the data they want whenever and whereever they are. The current best approximation of this is fetching data on demand. This is inadequate: cellular coverage is not ubiquitous, data transfers are limited, and reading from the network consumes significantly more energy than accessing data stored locally. Further, accessing data stored remotely has higher latency than local data.

We are investigating how to predict when and where data will be needed and how to hoard it opportunistically. We are focusing on subscription content, such as podcasts, place shifted video and social network status updates. The main challenges include: scheduling transmission while respecting multiple optimization goals, application integration, modeling user and data stream behavior, and managing prefetched data.

We are conducting a user behavior study to improve our understanding of what data users access, when and where they access and how they access it.

People

Software

Binary packages are available for the N900 (Maemo5).

Clicking on the following links from the web browser on your N900 will download an appropriate installer:

You can install the packages listed below from the command line using apt-get. Either add the following repository manually or install the woodchuck server, whose installer adds this repository automatically.

deb http://hssl.cs.jhu.edu/~neal/woodchuck fremantle maemo
  • Programs:

    • VCS Sync (about): vcssync
  • Libraries to interface with a Woodchuck server:

    • PyWoodchuck: A Python interface for Woodchuck (documentation).
    • libgwoodchuck: A glib-based C interface for Woodchuck (documentation).

Source Code

Woodchuck includes a number of components including a data logger for the Maemo platform (smart-storage-logger), an implementation of the Woodchuck server (murmeltier), a C library for interacting with a Woodchuck server from a glib-based application (libgwoodchuck) and python modules for interacting with a Woodchuck server (woodchuck.py and pywoodchuck.py).

The latest releast is available here. You can get the latest development version by running:

  git clone http://hssl.cs.jhu.edu/~neal/woodchuck.git

You can also view the code online.

Mailing Lists

There are two mailing lists, woodchuck-devel@nongnu.org and woodchuck-bugs@nongnu.org. woodchuck-devel@nongnu.org is a general-purpose list for questions regarding Woodchuck development and porting. woodchuck-bugs is for bug reports. If you think you've found a bug (or have a feature request), please read how to report a bug.

Documentation

API documentation is available in HTML and PDF format. Here are some quick links to relevant parts of the HTML documentation:

News

An archive of all blog posts is also available.

RSS Atom Add a new post titled:

I was recently hunting down a slightly annoying usability bug in Khweeteur, a Twitter / identi.ca client: Khweeteur can notify the user when there are new status updates, however, it wasn't overlaying the notification window on the application window, like the email client does. I spent some time investigating the problem: the fix is easy, but non-obvious, so I'm recording it here.

A notification window overlays the window whose WM_CLASS property matches the specified desktop entry (and is correctly configured in /etc/hildon-desktop/notification-groups.conf). Khweeteur was doing the following:

import dbus

bus = dbus.SystemBus()
notify = bus.get_object('org.freedesktop.Notifications',
                        '/org/freedesktop/Notifications')
iface = dbus.Interface(notify, 'org.freedesktop.Notifications')

id = 0
msg = 'New tweets'
count = 1
amount = 1
id = iface.Notify(
    'khweeteur',
    id,
    'khweeteur',
    msg,
    msg,
    ['default', 'call'],
    {
        'category': 'khweeteur-new-tweets',
        'desktop-entry': 'khweeteur',
        'dbus-callback-default'
            : 'net.khertan.khweeteur /net/khertan/khweeteur net.khertan.khweeteur show_now',
        'count': count,
        'amount': count,
        },
    -1,
    )

This means that the notification will overlay the window whose WM_CLASS property is khweeteur. The next step was to figure out whether Khweeteur's WM_CLASS property was indeed set to khweeteur:

$ xwininfo -root -all | grep Khweeteur
        0x3e0000d "Khweeteur: Home": ("__init__.py" "__init__.py")  800x424+0+56  +0+56
        ^ Window id                   ^ WM_CLASS (class, instance)
$ xprop -id 0x3e0000d | grep WM_CLASS
WM_CLASS(STRING) = "__init__.py", "__init__.py"

Ouch! It appears that a program's WM_CLASS is set to the name of its "binary". In this case, /usr/bin/khweeteur was just a dispatcher that executes the right command depending on the arguments. When starting the frontend, it was running a Python interpreter. Adjusting the dispatcher to not exec fixed the problem:

$ xwininfo -root -all | grep Khweeteur
     0x3e00014 "khweeteur": ("khweeteur" "Khweeteur")  400x192+0+0  +0+0
        0x3e0000d "Khweeteur: Home": ("khweeteur" "Khweeteur")  800x424+0+56  +0+56
Posted Sat 05 Nov 2011 11:55:34 AM EDT Tags: ?hacking

While working on the Woodchuck support in gPodder, I decided to profile the code. Reading the Python manual, I thought it would be as easy as:

    import cProfile
    cProfile.run('foo()')

On both Debian and Maemo, this results in an import error:

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib/python2.6/cProfile.py", line 36, in run
      result = prof.print_stats(sort)
    File "/usr/lib/python2.6/cProfile.py", line 80, in print_stats
      import pstats
    ImportError: No module named pstats

To my eyes, this looks like I need to install some package. This is indeed the case: the python-profiler package provides the pstats module. Unfortunately, python-profiler is not free. There's a depressing back story involving ancient code and missing rights holders.

If you're on Debian, you can just install the python-profiler package. Alas, the package does not appear to be compiled for Maemo.

Happily, kernprof works around this and is easy to use:

    # wget http://packages.python.org/line_profiler/kernprof.py
    # python -m kernprof /usr/bin/gpodder

Kernprof saves the statistics in the file program.prof in the current directory (in this case, it saves the data in gpodder.prof).

To analyize the data, you'll need to copy the file to a system that has python-profiler installed. Then run:

    # python -m pstats gpodder.prof
    Welcome to the profile statistics browser.
    % sort time
    % stats 10
    Tue Nov  1 13:09:54 2011    gpodder.prof

             105542 function calls (101494 primitive calls) in 117.449 CPU seconds

       Ordered by: internal time
       List reduced from 1138 to 10 due to restriction <10>

       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            1   57.458   57.458   69.012   69.012 {exec_}
            1   16.052   16.052   26.417   26.417 /usr/lib/python2.5/site-packages/gpodder/qmlui/__init__.py:405(__init__)
            1    8.591    8.591   13.790   13.790 /usr/lib/python2.5/site-packages/gpodder/qmlui/__init__.py:24(<module>)
           60    7.041    0.117    7.041    0.117 {method 'send_message_with_reply_and_block' of '_dbus_bindings.Connection' objects}
            3    6.357    2.119    7.469    2.490 {method 'reset' of 'PySide.QtCore.QAbstractItemModel' objects}
           36    2.636    0.073    2.636    0.073 {method 'execute' of 'sqlite3.Cursor' objects}
            1    2.283    2.283    2.284    2.284 {method 'setSource' of 'PySide.QtDeclarative.QDeclarativeView' objects}
            1    1.848    1.848    1.848    1.848 /usr/lib/python2.5/site-packages/PySide/private.py:1(<module>)
            2    1.789    0.895    1.789    0.895 {posix.listdir}
            1    0.765    0.765    4.234    4.234 /usr/lib/python2.5/site-packages/gpodder/__init__.py:20(<module>)

The statistics browser is relatively easy to use (at least for the simple things I've wanted to see so far). Help is available online using its help command.

Posted Tue 01 Nov 2011 09:01:45 AM EDT Tags: ?hacking ?woodchuck

Khweeteur is a great twitter and identi.ca client for Maemo. One feature I particularly like is its support for queuing of status updates, which is useful when connectivity is poor or non-existent (which, for me, is typically when something tweet-worthy happens). It also supports multiple accounts, e.g., a twitter account and an identi.ca account.

Khwetteur can automatically download updates and notify you when something happens. Enabling this option causes Khwetteur to periodically perform updates whenever there is an internet connection---whether it is a WiFi connection or via cellular. This is unfortunate for those, who like me, have limited data transfer budgets.

Deciding when to transfer updates is exactly what Woodchuck was designed for, and recently, I added Woodchuck support to Khweeteur. Now, if Woodchuck is found, Khweeteur will rely on it to determine when to schedule updates (of course, you can still manually force an update whenever you like!).

While modifying the code, I also made a few bug fixes and some small enhancements. Two improvements that, I think, are noteworthy are: displaying unread messages in a different color from read messages, and indicating when the last update attempt occured.

You can install the Woodchuck-enabled version of Khweeteur on your N900 using this installer. You'll also need to install the Woodchuck server, to profit from the Woodchuck support. Hopefully, the version in Maemo extras will be updated soon!

Other Woodchuck-enabled software for the N900 include:

If you are interested in adding Woodchuck support to your software, let me know either via email or join #woodchuck on irc.freenode.net.

Posted Thu 20 Oct 2011 05:18:57 PM EDT Tags: ?hacking ?woodchuck