GNOME Disks Development News

Juni 1st, 2018 by Kai

The most visible change for GNOME Disks in 3.28 was the new format dialog. Unfortunately the redesign of the partition view has not been tackled in the last development cycle.

A contribution from Andrea Azzarone improves the experience with snap applications because GNOME Disks will now hide all images mounted with the x-gdu.hide option, e.g.
sudo mount special.img /mnt/test/ -o loop,x-gdu.hide

Also several bugs are fixed in 3.28.3, so I recommend distributions to update. A Flatpak release is not yet available and due to some minor issues regarding the integration with the host system I don’t know yet if this really makes much sense.

For 3.30 there is support for opening VeraCrypt images, which was contributed by segfault and the Tails project. This needs a UDisks version which is not yet available and requires the creation of the file /etc/udisks2/tcrypt.conf if not shipped by the distribution.

Finally, the gnome-disk-utility repository has been moved to GNOME’s GitLab. The repository also contains the code for gnome-disk-image-mounter and the gsd-disk-utility-notify service for S.M.A.R.T. events. I hope this will attract some more people to work on GNOME Disks.

Interaction Between the User and Kernel Space in Linux

Januar 21st, 2018 by Kai

Seminar Paper June 2017, Interaction Between the User and Kernel Space in Linux, Kai Lüke


System calls based on context switches from user to kernel space are the established concept for interaction in operating systems. On top of them the Linux kernel offers various paradigms for communication and management of resources and tasks. The principles and basic workings of system calls, interrupts, virtual system calls, special purpose virtual filesystems, process signals, shared memory, pipes, Unix or IP sockets and other IPC methods like the POSIX or System V message queue and Netlink are are explained and related to each other in their differences. Because Linux is not a puristic project but home for many different concepts, only a mere overview is presented here with focus on system calls.

Download paper: Interaction Between the User and Kernel Space in Linux, Kai Lüke

Converted to HTML with pdf2htmlEX: Interaction Between the User and Kernel Space in Linux, Kai Lüke

Last Project Phase and 3.26 Features

August 22nd, 2017 by Kai

Repair and resize is available in the recent 3.25 release and needs at least UDisks 2.7.2. Currently Ext4, XFS and FAT are supported through libblockdev and I hope to extend this list with NTFS soon. There were some race conditions when a resized partition is detected by the kernel again and also the FAT support through libparted is still a bit shaky.
Showing the proportion of used disk space in the slider was the last appearance change for the resize dialog.
I’ve written some retrospective words about the project at the end of its wiki page – thank to all for this learning experience!

The new format dialog did not get merged yet and will come to master after the freeze.
Not yet implemented are the mockups for the whole UI where the partition list is shown. Jimmy Scionti and others in #gnome-design worked on my changes to Allan’s original mockup and the direction seems to stabilize now.

It was nice to visit GUADEC and meet people in person and discuss various things in the days after as well. There are too many areas where I also would like to do something but rightnow the time is short, and the list of plans for Disks is also growing…

Development News

Both 3.25.90 and 3.25.91 have been released and I think that 3.26 will be a good improvement compared to 3.24. Please report issues you experience.

GNOME Disks: Integrate Resize and Repair

Juli 25th, 2017 by Kai

The basic prototype patches let GNOME Disks resize partitions and their filesystems, check filesystems and repair them – at least when built against UDisks master where the actual work is performed. It needs some fixes on how jobs and UI states correspond but here a glimpse on how the resize looks like.

The current pending mockups for the whole UI will be discussed in person at GUADEC. Where to show the result of the filesystem check or repair and where to show the jobs are some of the questions.

In the implementation some problems pop up as usual, this time with some impact on the robustness but hopefully nothing again at the level of dealing with libparted geometry configuration.
The prototype needs some polishing as well, and in the underlaying layers I would like to add NTFS support and improve the FAT resize support. Finally it would be nice if the UDisks jobs indicate the progress which is not supported in libblockdev yet.

Resizing other block devices than partitions is also left to do (e.g. LUKS, backing loop file). Offering a filesystem resize where the block device can’t be fitted is a strange experience because it’s in general impossible to tell up to which size a filesystem occupies space in this block, meaning the result would look the same except for a difference in free space. Thus for the beginning only filesystems in partitions are resizable.

Development News

Recently 3.25.4 was released and the last changes improved the build experience a lot – Iñigo Martínez gave a meson port as surprise gift. Let’s see whether it’s enough time for the ongoing work to land on master for 3.26.

UDisks Repair and Resize API

Juni 27th, 2017 by Kai

Almost all actions in GNOME Disks rely on the UDisks system service. That way the authorization policy is handled by PolKit and Disks does not have to spawn any root processes. Because format and mount jobs are registered there to the device objects it is possible to initiate them from command line via gio mount, udisksctl or another D-Bus client such that Disks can still show these ongoing operations in the UI.

There are cases where D-Bus services are not possible (e.g. in an OS installer) and therefore libblockdev is a new library with similar functionality. The last UDisks release offloads many tasks to it instead of spawning utility binaries itself. In libblockdev the preference is to reuse libraries like libparted if possible and otherwise spawn utility binaries.

To have resize and repair support in Disks I had to introduce querying for installed support of an action and the action itself in UDisks and thus in libblockdev. The advantages are only small changes in Disks and Cockpit will also benefit. Now there are PRs pending for a generic repair and check function, a partition resize function and query functions for repair, resize and consistency check support of filesystem types in libblockdev. Then the PRs pending for UDisks can make use of them to expose the following API methos:

CanFormat(fstype) -> (available, required_utility)
CanResize(fstype) -> (available, mode, required_utility)
CanCheck(fstype) -> (available, required_utility)
CanRepair(fstype) -> (available, required_utility)

Resize(size, options)

Resize(size, options)
Check(options) -> consistent
Repair(options) -> repaired

The resize mode flags indicate if online or offline shrinking/growing is supported. The size argument for partitions should guarantee that content of this size will fit in and alignment might only make it bigger. The progress percentage property of the running job is not yet set because this needs parsing stdout in libblockdev, so just a spinner in Disks for now. It’s possible to test them from d-feet, e.g. to see if the F2FS format is available and what utility is missing if not:

On the command line it would be gdbusmdbus2 or dbus-send or the D-Bus library of your choice (e.g. in Python3 from gi.repository import Gio). Here assuming an almost empty unmounted ext4 partition 1 on a loop device:

$ # repair first (the option dict {} is unused)
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Filesystem.Repair {}

$ # shrink filesystem to 50 MB
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Filesystem.Resize 50000000 {}

$ # shrink partition to 50 MB
$ gdbus call --system --dest org.freedesktop.UDisks2 \
 --object-path /org/freedesktop/UDisks2/block_devices/loop0p1 \
 --method org.freedesktop.UDisks2.Partition.Resize 50000000 {}

GNOME Disks Development News

After Disks 3.25.2 was released there were only small changes in master. The standby menu entry was fixed to be usable again after the action took effect. Mitchell took care of large file support on 32-bit systems, an interesting topic. The biggest current change is the new format dialog. The new UDisks API must be integrated now in the UI. It would be nice to get NTFS (and progress) support in libblockdev and finally the Disks UI mockups done as well.

An Observation in UI Design

Juni 12th, 2017 by Kai

Reading the GNOME Human Interface Guidelines gives a good idea on how to arrange elements and reduce complexity. The HIGs also emphasize on having a clear goal which helps in deciding which elements need to be arranged at all. But I did not grasp the wideness of being purpose-driven for this goal of the application which might then mean to abstract from technical details on the way. So now I try to explain this observation here.

I think there are two opposite paradigms one can follow in UI design, and thus what users can expect from UIs. This is not a thorough research on the topic, so you can leave a comment if you have insights to share on whether this view is mistaken.
Maybe you will find your own examples during reading if you try to think about image, audio and video formats and their parameters, or network connection and encryption stuff in UIs.

The first paradigm is to make features accessible in the UI. This could just mean showing technical choices and configuration values coming from an API or command line tools. The UI serves as big toolbox with few limits, allowing to be creative and meet various use cases. But from the viewpoint of a newbie it is a big frustration to use these kind of applications because the options don’t have a meaning. Reaching the point where all implications of the parameters are understood is not possible in a few minutes. The whole research task for appropriate parameters has been offloaded to the user.

Therefore, the other paradigm comes into play and thinks from the main purpose of the UI and what elements it needs to reach the goal. That’s what I expect in GNOME. The technical details go to the background and may not be found as UI elements anymore. This needs clever ways to find optimal parameters, or studies of the technical and social/psychological conditions to reduce the workload for the user. First this looks like hiding ‚features‘ and configuration options (i.e. technical details) but it might be good to take a step back and look at the state of IT. Is it really necessary that everyone fiddles around with the available technical options? Can the decision for a configuration value be automated or evaluated? And how did we get into the situation of having so many competing ’standards‘ and a rag rug of solutions for the different layers involved and all this strange legacy stuff needed for compatibility.

Maybe we first need to work on a algorithms, libraries and single standards in order to be able to reduce UI complexity. But it’s not always possible to find unique technical solutions for the whole planet and we have to expose some of this strange acronyms and abbreviations in the UI and hope that users have an idea of their meaning. The result is a mixed approach of both paradigms where necessary.

Example: The format dialog in GNOME Disks

In the specific case of the format dialog for pen drives or hard disks you could simply decide to make features of the command line tools available in the UI. This would involve to expose the choice of the filesystem type and if it should be contained in an ecryption layer. Not speaking of overwhelming non-default parameters this toolbox is already hard to understand. For the new design of the format dialog I first proposed this toolbox paradigm approach (even if preference of the tools is indicated and not all are directly visible):

Assessing every option for the given use case and conditions is a lot of work and sometimes a final judgement can’t be made concerning issues with inter-OS compatibility, snapshots, Linux file permissions, encryption and performance (storage type, in-kernel drivers).
With the Opus audio codec joint efforts came finally to a good general solution, maybe too late.
But for filesystems it’s much more complex and the operating systems also have significant differences.

So if we want to use the result-oriented paradigm, we already see the problem that the UI has to guide the user through these OS compatibility issues. But this still means abstracting from selecting a concrete filesystem type and focusing more on use cases which are tied to filesystem choices (here still visible in brackets). The UI does not have to differ too much but important is the base idea. Here we see Allan’s design:

Ext4, NTFS and FAT are the current default filesystems in Disks but maybe exFAT is already more relevant and could e.g. replace both NTFS and FAT or even cover a distinct use case – that needs further research.

But also for Linux-only use cases there are other filesystems than Ext4 which could replace it as default choice or at least cover special demands like snapshots or being suited for flash storage.
Listing all these use cases would almost have the same effect like listing the filesystems with explanation, and that is what should be avoided for a common formating action of an external hard drive to back up some files.

A pragmatic middle path would e.g. as above show swap partitions and XFS for large amounts of data.
Yet I would argue that once people set up swap partitions or e.g. Btrfs volumes they can just handle the toolbox paradigm during that step. So as remembered from install wizards, the format dialog could do a paradigm shift and expose the toolbox if a custom filesystem setup is chosen instead of the defaults:

I think that this flexibility is needed because I don’t see a single filesystem to emerge soon which fits all the demands. But I would highly appreciate if people do not have to think about filesystems anymore because operating systems switched to let’s say Btrfs in collaboration and even your digital camera can handle it.

With this fallback available there is less pressure in finding optimal choices for the major use cases.
It can be observed if they need to be adjusted instead of campaigning for agreement on e.g. a single Linux-only and also a single inter-OS filesystem.
Still it would be nice to find out how user interaction can be reduced and less options presented by e.g. user surveys or applying automatic constraints based on the context.

GNOME Disks spins up for GSoC

Mai 17th, 2017 by Kai

This year’s Google Summer of Code features many valuable open source organizations and GNOME is one of them.
In GNOME’s project list you can find Disks, the GNOME utility for dealing with storage devices.
I’m happy that it worked out, because at first it was not in the list of ideas. The fact that Disks was considered unmaintained bothered me, but also was a big hint to finally get involved instead of mere bug reporting. So the plan emerged to propose working on two key features: Resize and repair filesystems.

Getting in contact with people through open bugs, IRC and the mailing lists went well and I’m grateful for the support (extra thanks to Michael!). Ondrej stepped in as mentor and made this possible, I already learned a lot and just took over maintainership. In the rest of this post I write a bit about the GSoC project outline and recent development in GNOME Disks.

GNOME Disks should support repairing and resizing filesystems through UDisks, thus
enabling non-root users to safely set up their devices from a graphical interface.

GParted does not use UDisks, runs only as root and thus not under Wayland.


  • Work out new D-Bus interface functions in cooperation with the UDisks team so that
    UDisks offers resize/repair through libblockdev (FS plugin) and is also tooling aware by
    probing relevant binaries. The exact solution is to be negotiated, since Cockpit might
    make use of it too.
  • Use that new UDisks functionality in GNOME Disks, design and implement UI (follow
    HIG, good progress indication)
  • Focus on supporting just a few filesystems in the beginning

The first task is to rework the format dialog, which should stick with the principle to offer just a few sane defaults but also not forbid to choose others (list available filesystems and find a good solution for missing tooling).

(Needs Radio Buttons for the selection.)

Development News

Translation work is constantly going on – great to see that!

I’ve set up a GNOME Wiki page as website for Disks.

A bug preventing unmounting and locking is fixed as well as a bug in the format dialog.

Taking care of loop devices:

  • Explain the auto-clear option, i.e. detaching loop devices if they are not mounted anymore
  • Disable auto-clear during operations if it would lead to unwanted detachment
  • Create a new empty disk image from the AppMenu

UI improvements:

  • Show UUID
  • Hints for passphrase missmatch
  • Indicate maximal FS label length
  • Explain the meaning of the mounting and encryption options, i.e. overwriting the session defaults with fstab/crypttab entries
  • Use mimetypes for disk image selection (welcome Mitchell as new contributor!)

There are many ways to contribute to GNOME, please support the Outreachy by spreading the word!

Design of a Python-subset Compiler in Rust targeting ZPAQL

Oktober 19th, 2016 by Kai

B.Sc. Thesis, Computer Science Freie Universität Berlin, Kai Lüke


The compressed data container format ZPAQ embeds decompression algorithms as ZPAQL bytecode in the archive. This work contributes a Python-subset compiler written in Rust for the assembly language ZPAQL, discusses design decisions and improvements. On the way it explains ZPAQ and some theoretical and practical properties of context mixing compression by the example of compressing digits of π. As use cases for the compiler it shows a lossless compression algorithm for PNM image data, a LZ77 variant ported to Python from ZPAQL to measure compiler overhead and as most complex case an implementation of the Brotli algorithm. It aims to make the development of algorithms for ZPAQ more accessible and leverage the discussion whether the current specification limits the suitability of ZPAQ as an universal standard for compressed archives.

Download paper: Design of a Python-subset Compiler in Rust targeting ZPAQL, Kai Lüke

Converted to HTML with pdf2htmlEX: Design of a Python-subset Compiler in Rust targeting ZPAQL, Kai Lüke

Web Connectivity in Ghana – A survey

Juli 11th, 2015 by Kai


Starting from the research “Dissecting web latency in Ghana” of Zaki et al. the field of Internet connectivity in Ghana is explored and the different aspects like latency, DNS resolution times, complexity of web sites, caching and prefetching, TLS/SSL, TCP and cellular network performance, role of ISPs, Internet Exchange Points, are brought in relation. Measurement approaches and tailored solutions for areas with bad connectivity are listed. Possible ways of improvement are discussed.

Download paper: Web Connectivity in Ghana – A survey, Kai Lüke

Converted to HTML with pdf2htmlEX: Web Connectivity in Ghana – A survey, Kai Lüke

Categories and Subject Descriptors: C.2.6 [Internetworking]: Standards; C.4 [Performance of Systems]: Reliability, availability, and serviceability

Keywords: Developing Countries, Internet, Latency, Performance, TCP, Mobile, Cellular

modifying kernel space – uname hack revamped

August 27th, 2013 by Kai

This change of the original uname hack kernel module was laying unreleased and forgotten on the disk, but after some tweaking to support all options, here it comes — ugly and not meant for serious usage:

uname_kernel_hack_module.tar.gz (1,7 kB)

Some months ago I modified it to not change the machine from e.g. amd64 to i386 but the kernel version. Curiously to solve an issue it itself is having in the second line of the Makefile (uname -r); some actions rely on the kernel version to access the actual initramdisk or kernel headers. These might be different versions within guest system containers and led to the ugly hack of overwriting the kernel struct (yes, we are on Linux, not GNU Hurd) where the uname tool gets it’s release version from:

struct new_utsname {
char sysname[__NEW_UTS_LEN + 1];
char nodename[__NEW_UTS_LEN + 1];
char release[__NEW_UTS_LEN + 1];
char version[__NEW_UTS_LEN + 1];
char machine[__NEW_UTS_LEN + 1];
char domainname[__NEW_UTS_LEN + 1];

At initialisation the module saves the original value of e.g. sysname and overwrites the kernel space struct with the value you want. After usage, when the module is unloaded, the original value will be restored.

So when using the provided Makefile you could test how your system looks like when pretending to be something else:

make KRELEASE=2.5.2-0 KSYSNAME=Minix KVERSION="Debian 2.5 (2012-03-04)" KMACHINE=i386 test

The output while running make could look like:

uname -a
Linux hstnm 3.10-2-amd64 #1 SMP Debian 3.10.5-1 (2013-08-07) x86_64 GNU/Linux
sudo /sbin/insmod unamehack.ko
[sudo] password for kai: 
uname -a
Minix hstnm 2.5.2-0 Debian 2.5 (2012-03-04) i386 GNU/Linux
sudo /sbin/rmmod unamehack
uname -a
Linux hstnm 3.10-2-amd64 #1 SMP Debian 3.10.5-1 (2013-08-07) x86_64 GNU/Linux

So a longer usage would require you to enter

make KMACHINE=i386 insertunamemodule

and revert the changes as soon as possible by

make removeunamemodule

So, good luck! You’ll better not use it and try to fix cases where it’s necessary.