The Hiatus
Working on a software project can be very tasking and time consuming. For about seven weeks now, I have been working on a project for Libvirt by adding container functionality to lcitool. This would make it easier to do Libvirt specific testing locally and more.
The initial approach was to make use of a library binding called podman-py which is a Python library that provides a lot of podman-related functionality by sending an API request to podman API service. This library would enable us to build container images, run commands/workloads in a container and perform other container operations.
The first month of working on this project was spent researching and building a repertoire of APIs from podman-py to be used to carry out the project. The research led to a couple of "not-so-encouraging" findings about the library which are clearly detailed in this blog post. These findings caused us(my mentor and I) to re-evaluate the approach to solving the project.
It was then decided that directly wrapping the Podman CLI utility would be the new approach. The last two weeks have been spent revamping all the work done with the podman-py library. to use the new approach The new approach would involve using subprocess, a Python inbuilt module to wrap the container CLI commands; this would result to the equivalence of calling these commands on the command line.
That is;
import subprocess
subprocess.run(args=["podman", "version"], shell=True)
which is the equivalence of opening a bash terminal and running podman version.
Working on integrating this new approach has made me very busy in the last couple of weeks causing me to skip two blog weeks.
The last weeks have been spent remapping the {uid,gid}map logic from bash to Python. Podman cannot reuse host namespace when running non-root containers; so, in order to use these host namespace, there is a need to concoct a bit of black magic(just kidding) with --uidmap and --gidmap postional arguments in Podman. These positional arguments helps to solve the non-root container problem.
You might be wondering what namespaces are; Namespaces in Linux are features that helps to partition kernel resources such that sets of running processes use different resources. Container technologies such as Podman and Docker make use of these concepts in their inception especially the user namespace.
User namespace is an isolation feature that allows a set of users to run processes with different IDs in the container and on the the host system. Every container inherits its permission from the user that creates the new user namespace i.e if a user with 100 ID on the host system creates a container; by default, process in the container run with 100 ID.
User namespace can provide a scenario where the user running on the host system(100 in our case) might be root(use with ID 0) in the container without having admin privileges like the actual root in the host system. This sort of mapping is one of the advantages of user namespace.
After successfully remapping the IDs, the next thing was figuring out the initial interface of the run command. The run command was expected to have two workflows which are:
provide a --scratch, --script, --target and --project argument: This workflow will be required to build the image to be run in the container. This is the preferred choice for testing new changes locally.
lcitool container run --scratch /tmp/scratch --script build.sh --target ubuntu-2204 --project libvirt-pythonor provide a --scratch, --script, --image arguments: This workflow will pull an image from an online registry and run a workload on it.
lcitool container run --scratch /tmp/scratch --script build.sh --image registry.gitlab.com/libvirt/libvirt/ci-fedora-36
The --scratch argument would be a required optional argument which is a path to the directory to be mounted as user's home in the container. It should contain another folder which includes the source code to be build, and a script file which contains the bash script for building the folder.
The --script argument would also be a required optional argument which would point to the build script to be ran inside the container. NB: The build script must be in the directory provided by the --scratch option.
The --target argument is an optional argument that indicates the target OS. It is required with the --project option in order to build Dockerfile/Containerfile — which would be used to build the image locally. An examples is ubuntu-2204, fedora-36 et.c
The --project argument is an optional argument that indicates the projects to build. An example could be one of the projects under the Libvirt umbrella e.g libvirt, libvirt-python, libvirt-tck et.c.
The --image argument is an optional argument that is mutually exclusive to the --target and --project argument. An example would be a link to an image hosted on a containe registry. e.g registry.gitlab.com/libvirt/libvirt/ci-ubun...
Finally, I submitted a merge request (MR) containing the initial draft for the whole project.