GSoC - Mock improvements - week 9

This week I've been mostly focusing on minor improvements and documentation (manpages). Almost all my changes were already submitted upstream and if everything goes well, you can expect a new release of mock to be available in rawhide in the near future. I merged changes from the ready branch to master so now they should differ only in minor things. (Sorry for duplicates in git history, I didn't realize that beforehand)

Support for Mikolaj's nosync external library was added and the old implementations that existed as a part of mock were dropped. You can enable it by setting
config_opts['nosync'] = True
and you have to install the nosync library (mock doesn't require it in the specfile, because it's not available everywhere). If the target is multilib, you need both aritectures of the library to be installed in order to have a preload library for both types of executables. If you don't, it will print a warning and won't be activated. If you can't install both versions and still want to use it, set
config_opts['nosync_force'] = True
but expect a lot of (harmless) warnings from ld.so.  The library is available in rawhide (your mirrors might not have picked it yet)

LVM plugin was moved to separate subpackage and conditionaly disabled on RHEL 6, since it requires lvm2-python-libs and newer kernel and glibc (for setns). One of the things that I needed to sacrifice when I was making the LVM plugin was the IPC namespace unsharing, which mock uses for a long time. The problem was that lvcreate and other commands deadlocked on unitialized semaphore in the new namespace, so I temporarily disabled it and hoped I'll find a solution later. And I did, I wrapped all functions that manipulate LVM in function that calls setns to get back to global IPC namespace and after the command is done, it call setns again to get back to mock's IPC namespace.

One of the other problems I encountered is Python's (2.7) handling of SIGPIPE signal. It sets it to ignored and doesn't set it back to default when it executes a new process, so a shell launched from Python 2 (by Popen, or execve) doesn't always behave the same as regular shell.
Example in shell:
$ cat /dev/zero | head -c5
# cat got SIGPIPE and exited without error

$ python -c 'import subprocess as s; s.call(["bash"])'
$ cat /dev/zero | head -c5
cat: write error: Broken pipe
# SIGPIPE was ignored and cat got EPIPE from write()
It can be fixed by calling
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
in Popen's preexec_fn argument.

For cat it's just an example and it didn't make much difference. But if you put tee between the cat and head, it will loop indefinitely instead of exiting after first 5 bytes. And there are lots of scripts out there relying on the standard behavior. It actually bit me in one of my other programs, so I thought it's worth sharing and I also fixed it in mock.

GSoC - Mock improvements - week 8

Good news, we're merging my changes upstream. It's been a lot of changes and the code wasn't always in the best shape, so I didn't want to submit it before all major features are implemented. Mirek Suchy agreed he'll do the code review and merge the changes. Big thanks to him for that :)
I've setup a new branch, rebased it on top of current upstream and tried to revisit all my code and get rid of changes that were reverted/superseded, or are not appropriate for merging yet.  I squashed fixup commits to their originl counterparts to reduce the number of commits and changed lines.
The changes that weren't submitted are:

• C nofsync library, because Mikolaj made a more robust nosync library that is packaged separately, and therefore supersedes the bundled one.
  Link: https://github.com/kjn/nosync/
  I did the review: https://bugzilla.redhat.com/show_bug.cgi?id=1118850
  That way mock can stay noarch, which gets rid of lots of packaging issues. And also saves me a lot problems with autoconf/automake. There is no support for it yet, because I need to figure out how to make it work correctly in multilib environment.
• nofsync DNF plugin - it's an ugly DNF hack and I consider it superseded by aforementioned nosync library
• noverify plugin - it's also a DNF hack, I will make a RFE for optional verification in DNF upstream instead

Everything else was submitted including the LVM plugin.  The merging branch is not pushed on github because I frequently need to make changes by interactive rebasing and force pushing the branch each time kind of defeats the purpose of SCM.
Other than that I was mostly fixing bugs, the only new features are the possibility of specifying additional commandline options to rpmbuild, such as --rpmfcdebug with --rpmbuild-opts option and ability to override command executable paths for rpm, rpmbuild, yum, yum-builddep and dnf, in order to be able to use different version of the tools than the system-wide version.

GSoC 2014 - week 7

Hi again, I'm sorry I didn't post last week, because I've been on a vacation.
Here's what I've done this week:

Passing additional options to underlying tools
rpmbuild has an option --short-circuit that skips stages of build preceding the specified one. It doesn't build a complete RPM package, but it's very handy for debugging builds that fail, especially in the install section. But this option is not accessible from within mock and I already mentioned in my proposal that I want to make it available. The option is also called --short-circuit and it accepts an argument - either build, install, or binary, representing the build phase that will be the first while the preceding phases would be skipped.
Example invocation:
$ mock rnv-1.7.11-6.fc21.src.rpm --short-circuit install

For Yum or DNF some of the options that are often used when user invokes the package manager directly also weren't available in mock. --enablerepo and --disablerepo are very common ones and now they are also supported by mock - they're directly passed to the underlying package manager.
Example invocation:
$ mock --install xmvn maven-local --enablerepo jenkins-xmvn --enablerepo jenkins-javapackages
The repos of course have to be present in the yum.conf in mock config.

Python 3 support
I started working on porting mock to Python 3. This doesn't mean that mock will run on Python 3 only, I'm trying to preserve compatibility with Python 2.6 without the need to have two version of mock for each. I changed the trace_decorator to use regular Python decorators instead of peak.utils.decorate and dropped dependency on the decoratortools package. There are slight changes in traceLog's output, that I don't consider important, but if someone did, it could be solved by using python-decorator package, which is available for both versions. There are some features that are still untested, but the regularly used functionality is already working. Rebuilding RPMs, SRPMs, working in shell, manipulating packages is tested. The plugins, that are enabled by default (yum-cache, root-cache, ccache, selinux) also work. What doesn't work is the LVM plugin, because it uses lvm2-python-libs, which doesn't have a Python 3 version yet. Same applies to mockchain, which uses urlgrabber. To try mock with Python 3, either change your system default Python implementation or manually hardcode python3 as the interpreter to the shebang in /usr/sbin/xmock.