Group: Guix/GSoC-2020
This page is for tracking ideas and process for participating in Google Summer of Code 2020.
Contents
Project ideas for GNU Guix
Guile based build-tool
Guile-based build tool for advanced software developers who do not want to deal with the complexity of multi-platform deployment and dependency hell. By default runs inside the GNU Guix build system and effectively replaces the likes of autotools, cmake, etc.
Build tools such as automake and cmake are complex beasts. What they try to do is target multiple build environments by generating make files using a complex list of string expansions. Complexity, in general, means hard to understand, change and maintain. The bad thing is that almost every software project has to deal with this complexity to make the software deployable.
What is wrong with these build tools? In short:
1. Complex - and therefore hard to understand and maintain
2. Simplistic language - simple string expansion systems are not nice and not DRY
3. Race conditions - make's time stamps are not suitable for building across servers
4. Different combinations of options are badly handled.
5. No intermediate representation
Alternatives, such as Java ant, Scala sbt, and Ruby rake, try to resolve similar issues but are specialised for their environments. Here we have a *fresh* start of a build tool which should ultimately be able to run complex workflows on multiple servers and make use of opportunistic execution, i.e. jobs may be distributed multiple times and the first one that comes back wins.
We primarily targets GNU Guix because Guix already solves the complexity of handling dependencies and software deployment. This simplifies things dramatically because the build system does not have to deal with platform or architecture targets. Even so, in time, we can target out of GNU Guix builds.
Mentors: Pjotr Prins and other Guix members
GNU Guix system monitor
Sheepdog is a Guile-based system monitoring tool that will be a feature of GNU Guix. A project description can be found [[1][here]]. The general idea is to introduce a message broker which can can pass messages between machines in two directions. This will help with system monitoring because a server can act on incoming information, for example by reconfiguring a firewall. The message broker will be generic and monitoring tools can be written in any programming language. Eventually the same system may also be used for remote execution of builds and orchestration of containers.
Mentors: Pjotr Prins and other Guix members
Content-addressed protocol for substitutes
The goal of this project would be to reduce the amount of data that needs to be transferred when retrieving substitutes. The rationale and design idea is described in this message.
Mentor: Ludovic
Booting via network
Some administrators have a lot of machines and would like to install the Guix system via the network on an "empty" machine without walking to each one.
A possible way to do that is to make the "empty" machine request an IP address to use (via DHCP), load the files required for booting (via TFTP) and then boot Linux (make the CPU jump there) and have the booted Linux mount the root (via NFS).
The "empty" computer has to support the first part. Fortunately, a lot of computers do support this. Network cards and many laptops support booting in this manner - and many fall back to it if nothing else works. The technology is called "PXE". There's also the U-Boot pxe module (part of U-Boot) which allows you to boot ARMv7 machines from the network in this manner (however, it's not clear whether these machines have a fixed MAC address or other unique ID).
This work would entail:
1. Checking the feasibility by manually configuring dnsmasq, tftpd and NFS to serve files - and testing it for an "empty" computer.
2. Extending the boot mechanism of the Guix system to mount root from NFS in case the system has been booted via PXE.
3. Working out the actual design.
4. Extending the Guix service "dnsmasq-service" to provide a DHCP server and to also provide PXE-specific DHCP options. See also https://www.tecmint.com/install-pxe-network-boot-server-in-centos-7/
5. Creating a new Guix service "tftpd-service" which allows users to use the service extension facility so that other services can provide files - like bootloader files, installation files, other misc files etc. Guix's inetd already contains tftpd (a file server using the TFTP protocol - which is customary to use) - however, tftpd's shared directory cannot managed by Guix yet. Dnsmasq also has its own tftp server, so this would have to be investigated. It might be useful to serve the contents of Guix system disk images, see https://wellsie.net/p/286/ .
6. Providing said files also via NFS. It might be useful to provide loop mounts to be able to serve the contents of Guix system disk images.
7. Creating a new Guix service "guix-installation-service" which extends or requires other services (like tftpd, dhcpd) and allows the user to specify a system to serve to the user.
8. Working out and documenting the security implications.
Future work could be to extend this to supply different systems to different users, however this work wouldn't require it.
Mentors: Danny Milosavljevic
Project ideas for the GNU Shepherd
Syntax and semantics of systemd units in the Shepherd
The GNU Shepherd has a Scheme interface to define services, most importantly their dependencies and actions associated with them. The goal of this project is twofold. The first part consists in matching the semantics of systemd's .service unit files, more precisely a subset thereof (for instance, the DBus-related part may be omitted.) As part of this work, the Shepherd should be extended with cgroup support on systems that support it. The Shepherd should also be extended with a procedure to load a .service file and return a <service> object.
The second part will consist in implementing other types of units, in particular socket activation and .device.
Mentors: Ludovic
Add an extensible event loop
As of version 0.3, the Shepherd uses a trivial event loop where it waits for connections from clients such as the 'herd' command and also, via Guile "system asyncs", waits for SIGCHLD signals corresopnding to the death of child processes. Thus, we assume that clients will not block, and we also expect the 'start' and 'stop' methods of each service (and in fact, each service action) to be non-blocking. Consequently, the Shepherd can only serve one client request at a time, and services are retricted in what they can do. For instance, it's currently impossible to have a REPL server in the Shepherd, unless resorting to multiple threads, which is not an option.
The goal of this project is to make the Shepherd rely on an extensible event loop. Services should be able to add file descriptors to watch or delays to wait for to the event loop. The event loop will avoid the "callback hell" by using either a monad (a monad is essentially "continuation-passing style", CPS, which is essentially "callbacks"), or by using delimited continuations, for instance via Fibers or maybe 8sync.
Mentors: Ludovic + Chris Webber(, +...?)