GSoC - week 4

Last week I had exams at the university and that left me with less time for work. But I made some progress anyway.

Mock performance
Mock builds usually take a considerable amount of time and there is nothing much that can be done about the speed of actual building but the package installation can be improved. Last time I created noverify plugin which provided considerable speed up and my mentor recomended to try removing fsync calls during the package installation. I do that by making a small library in C containg only empty fsync() and fdatasync() functions and copying it into the buidroot. Then using LD_PRELOAD to make it replace actual libc implementation of this calls. My mentor measured the performance differences on his kvm virtual machine and the results are amazing - times installing @buildsys-build and maven-local (look at the wall clock time):

Standard yum:
User time (seconds): 55.66
System time (seconds): 5.78
Percent of CPU this job got: 25%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:02.03

Standard dnf:
User time (seconds): 49.61
System time (seconds): 5.68
Percent of CPU this job got: 23%
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:50.94

With noverify plugin:
User time (seconds): 47.85
System time (seconds): 5.32
Percent of CPU this job got: 36%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:25.25
Maximum resident set size (kbytes): 150248

With noverify plugin, fsync() and fdatasync() disabled:
User time (seconds): 46.38
System time (seconds): 4.97
Percent of CPU this job got: 87%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:58.56
Maximum resident set size (kbytes): 150260

That's more than 4x faster and could be valuable improvement for both packagers and koji builders.

LVM plugin
I started implementing the basis for LVM support. Now it can already use LVM snapshot instead of root cache. To enable it, put the following in your config:
config_opts['plugin_conf']['root_cache_enable'] = False
config_opts['plugin_conf']['lvm_root_enable'] = True
config_opts['plugin_conf']['lvm_root_opts'] = {
    'volume_group': 'mock-vg'
}
where mock-vg is the name (not path) of the volume group you want mock to use for creating new volumes. There are other configuration options possible - filesystem, size, snapshot_size, mkfs_args. Root cache is disabled because it would be redundant. When started, it creates a logical volume, mounts it and after it's initialized, it makes a snapshot and mounts the snapshot instead of the original. Then all following builds alter only the snapshot and when clean command is executed (usually at the beginning of new build) the snapshot is deleted and replaced with new one. I originally tried to implement it the other way around - making a snapshot and still working with the original volume and then merging it when cleaning. But it was very slow - the merging took more than 10s. The current approach is fast enough - cleaning is just deleting a snapshot and creating new one which happens almost instantly (compared to deleting buildroot and unpacking root cache).

The next week I'll try to implement more advanced features of the LVM plugin - snapshot management which would allow having a hierarchy of snapshots with different preinstalled packages facilitating faster workflow for packagers working with more diverse set of packages.