Saturday, August 9, 2014

GSoC - week 10+11

Last week I've been working on improving the LVM plugin thinpool sharing capabilities. I didn't explain LVM thin-provisioning before, so I'll do it now to rationalize what I'm doing.
With classical volumes you have a volume with given size and it always occupies the whole space that was given to it at the time of creation, which means that when you have more volumes, you usually can't use the space very efficiently because some of the volumes aren't used up to the capacity whereas others are full. Resizing them is a costly and risky operation. With LVM thin-provisioning there's one volume called thinpool, which provides space to thin volumes that are created within. The thin volumes take only as much space as they need and they don't have physical size limit (unless you set virtual size limit). That means that if the space is not used by one volume it can be used by another.
Previously there was one thinpool per configuration which corresponded to one buildroot. It could have snapshots, but there was still one buildroot that could be used at the moment. Now you can use mock's unique-ext mechanism to spawn more buildroots from single config. Unique-ext was there even before I started making changes, but now I implemented proper support for it in the LVM plugin. It's a feature that was designed mostly for buildsystems, but I think it can also be very useful for regular users who want to have more builds running at the same time. With LVM plugin the thinpool and all the snapshots are shared between the unique-exts, which means you can have multiple builds sharing the same cache and each one can be based on different snapshot. The naming scheme had to be changed to have multiple working volumes where the builds are executed. Mock implements locking of the initial volume creation, so if you launch two builds from the same config and there wasn't any snapshot before, only one of the processes will create the initial snapshot with base packages. The other process will block for that time, because otherwise you'll end up with two identical snapshots and that would be a waste of resources.
Other sharing mechanism that is now implemented is sharing the thinpool among independent configs. Then the snapshots aren't shared because the configs can be entirely different (for example different versions of Fedora), but you can have only one big logical volume (the thinpool) for all mock related stuff, which can save a lot of space for people that often use many different configs. You can set it with config_opts['pool_name'] = 'my-pool' and then all the configs with pool_name set to the same name will share the same thinpool.
Other than that I was mostly fixing bugs and communicating with upstream.

This week I've been on Flock and it has been amazing. There were some talks that are relevant for mock, most notably State of Copr build service, which will probably use some of the new features of mock in the future and Env&Stacks WG plans, which also mentioned mock improvements as one of their areas of interest.