Introduction to Mock (20090308 Classroom)

From FedoraProject

Jump to: navigation, search

Resources

IRC Log

-!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - An introduction to mock, the chroot package builder with your teacher ( nirik ) - See https://fedoraproject.org/wiki/Classroom for todays class schedule and more info. 14:45
@nirik welcome back everyone. 14:45
@nirik Our next talk is an Introduction to mock. 14:45
DiscordianUK Hi 14:45
@nirik with your teacher... me. ;) 14:45
@nirik So, I am gonna go over what mock is, why it's needed and how to use it, then some tricks and tips. 14:46
sdodson Sounds good. 14:46
@nirik feel free to ask questions at anytime. 14:46
@nirik ok, so some intro. 14:47
@nirik mock is a package that makes a chroot and builds other packages in it. 14:47
@nirik it's available in any of the fedora repos. You can install it with a simple 'yum install mock' 14:47
@nirik You can also build packages other releases with it. For example, if you have F10 and want to build a rawhide package you can, or if you have f10 and want to build a f9 package. 14:48
@nirik mock downloads and installs all the needed packages in it's chroot to build your package. 14:48
@nirik It has a bunch of caching ability, so you only usually suffer a large download the first time you build something. I will talk about the cache later. 14:49
@nirik So, why do we want to go to all this trouble? why not just build the package locally and be done with it? 14:49
@nirik The reason is rerproducability. Some packages look for things at compile time. We have to know that when we build something it's reproducable and we can build it the same way again. 14:50
@nirik so, we have a known set of base packages that everything gets and then after that, packages must specify exactly what they need to build. 14:50
@nirik The Fedora build system uses mock to build each package that maintainers build. 14:50
@nirik any questions on what mock is or why we have it? 14:51
@nirik ok, moving on... 14:51
Ritmo2k Yeah, wo its the std base set of packages that make it advantagous over a non root build? 14:51
@nirik ah, excellect question. 14:51
@nirik for fedora, look at http://fedoraproject.org/wiki/Packaging/Guidelines and the buildrequires section... 14:52
zcat what was used before mock? custom rpmbuild scripts? 14:53
@nirik basically it's a group called 'buildsys-build' 14:53
@nirik so if you look at that group in yum you can see exactly whats in it. 14:53
@nirik zcat: yeah, I think it was a simpler version in the buildsys.... 14:54
DiscordianUK so what if yu're 14:54
DiscordianUK so what if what you're building exceeds the buildsys-build specs? 14:54
@nirik despite mock being made for building packages you can also use it for testing or making populated chroots. I will talk about that a bit later as well. 14:54
@nirik DiscordianUK: then you need to add to your package spec file a "BuildRequires: whatever you need" 14:55
DiscordianUK ok 14:55
@nirik it will then install those things as well and their dependencies. 14:55
DiscordianUK cool 14:55
@nirik ok, so moving on to some simple usage. If anyone would like to follow along, you can 'yum install mock' first. 14:55
@nirik the first thing you want to do is add your user (or the user you want to use mock) to the 'mock' group so it can run the command. 14:56
@nirik usermod -a -G mock myusername 14:56
@nirik for example 14:56
@nirik you may need to logout and back on to get the new group or run 'newgrp mock' after that. 14:56
@nirik After that you can run mock. The easiest command that most people will use is 'rebuild' 14:57
@nirik so, you do 'mock rebuild my.src.rpm' 14:57
@nirik when you issue that command, mock creates a chroot of your current fedora version, installs the buildsys-build group into it, then installs the src.rpm's BuildRequirers packages and all their dependencies into it, 14:58
Ritmo2k how do specify where the chroot takes place in the event the deps are huge and need lotsa space? 14:58
@nirik it then calls the rpmbuild in the chroot and rebuilds the package. You should ideally end up with your binary and src.rpm packages at the end. 14:58
@nirik Ritmo2k: good question. By default mock uses /var/lib/mock/ and cache in /var/cache/mock. You can specify other dirs on the command line. 14:59
@nirik or you can edit the mock config file in /etc/mock/ 14:59
@nirik /etc/mock/site-defaults.cfg is the config you want to change there. 15:00
@nirik depending on your package mock could take up a fair bit of space. 15:00
@nirik mock has some other base command line arguments as well... 15:01
@nirik init - init a chroot, but don't do anything with it yet. 15:01
@nirik chroot <cmd> - chroot to the mock chroot you made and run command <cmd> 15:01
@nirik installdeps / install <pkg> - will install a package or a package and all it's deps into the chroot. 15:02
Ritmo2k Can you elaborate? 15:02
@nirik on which item? :) 15:02
Ritmo2k The first, 15:03
@nirik using those commands you can use mock to make a chroot that has whatever packages you like in it. 15:03
AndyCap Hmm, my mock on F10 didn't have an /etc/mock/default.cfg 15:03
@nirik on init? Basically it does the create the chroot step and installs buildsys-build into it. It doesn't build anything after that or do anything until you run other commands on that chroot 15:04
Ritmo2k ok 15:04
@nirik AndyCap: should be site-default.cfg 15:04
@nirik AndyCap: but I am looking at rawhide, so it might be a difference with the f10 version. ;( 15:04
@nirik so, when mock is running on say a rebuild of a src.rpm, it makes some log files with all the things it's doing. 15:05
AndyCap nirik: what I did was symlink a distro-arch.cfg to default.cfg, at least that seemed to be what the manpage meant 15:05
@nirik yeah, I would have to check a f10 machine. Not sure whats going on there. ;( 15:05
@nirik any further questions on the base commands? AndyCap: I can get a f10 machine here after the talk and see whats going on for you... 15:06
AndyCap I'll do fine. don't worry 15:07
@nirik ok, so lets talk a bit about log files. ;) mock makes logs of it's actions, which are often quite helpfull. 15:07
@nirik Since the fedora buildsys uses mock, we can take a look at an example there. 15:07
@nirik lets take a look at: http://kojipkgs.fedoraproject.org/packages/rkhunter/1.3.4/4.fc11/data/logs/noarch/ 15:07
@nirik these are the 3 log files mock makes on a build run. First state.log shows high level commands it's running. 15:08
@nirik then root.log shows the packages it's installing in the chroot... both the buildsys-build and the deps. 15:08
@nirik then build.log is the output of rpmbuild on the src.rpm in the chroot. 15:09
* nirik will pause for a sec and let everyone look over logs. Any questions on those? 15:09
Ritmo2k Wow, so do those build servers grab deps from a cached mirror :) That would amount to alot of dl'ing for every package! 15:10
@nirik ok, lets move on to talking about mock's caching ability. 15:10
Ritmo2k cached 'local' mirror 15:11
@nirik Ritmo2k: they do. They have a caching proxy over a local mirror. ;) 15:11
@nirik but mock also does some nice caching. ;) 15:11
@nirik so mock can cache at 3 points (and does by default). 15:11
@nirik You can see the buildsys-build download there was 113MB. It would be anoying to download that everytime you rebuilt your package. 15:12
@nirik So, mock keeps a cache of the chroot it last used for your package. It can just unpack that again and apply any updates since it was last used, and use it for the build. 15:12
sdodson That's per package? 15:13
AndyCap mock uses a "clean" chroot for each build? 15:13
@nirik sdodson: I think it's per os-arch. So, if you built a package with f9/i386 it would re-use that cache. 15:13
@nirik you can of course tell it not to use the cache. 15:14
@nirik AndyCap: unless it's using a cached one from a previous build. 15:14
@nirik mock also uses yum to install things, so it can use your yum cache if you have that enabled to pull packages from. 15:14
Ritmo2k So the chroot is a minimal installation of your target OS, then it adds what ever deps it needs? 15:15
@nirik finally mock can use ccache for compiled binaries 15:15
@nirik Ritmo2k: yes. 15:15
AndyCap nirik: I guess what I'm asking is it unpacking a pristine cache like rpm does with sources or is it using the same directory if I were to build more than one package without asking mock to clean up? 15:15
@nirik AndyCap: it does unpack a new one. What it does is the first time you compile a given os/arch it will make a root cache file... basically a tar.gz of the root 15:16
@nirik then it you compile another thing using that same os/arch, it will unpackage that 'clean' root into the new build. 15:16
AndyCap good, thank you. 15:17
@nirik so there is always unpacking time/disk io involved. 15:17
@nirik but if you are testing rebuilds of the same package over and over, the caching should really help you not use too much network BW on downloads. 15:18
@nirik Here you can see an example of a mockbuild I did the other day... 15:18
@nirik http://scrye.com/pastebin/196 15:18
@nirik this shows the typical output from a rebuild... 15:19
@nirik you can see there that it used a existing cache for the root 15:19
@nirik it also tells you how long the build took. 15:20
@nirik any further questions on cache or the output? 15:20
@nirik ok, moving along to some talk of the config files... 15:21
@nirik mock has it's config under /etc/mock as I mentioned earier. There are config files for each os/arch... 15:22
@nirik if you need to modify what repository you are using, or add a local repo for some packages not in fedora, thats where you do that. 15:22
@nirik or if you need to get more debugging from yum or mock when running it to track down a problem. 15:22
@nirik in those files you find the yum.conf that mock uses to install with. 15:23
@nirik also, this is a good time to mention smock... it's a wrapper around mock that rwmjones made to allow you to use local repos easily in your builds. 15:24
@nirik http://hg.et.redhat.com/cgi-bin/hg-misc.cgi/fedora-mingw--devel/file/tip/smock 15:24
@nirik any questions on the config files? 15:25
@nirik AndyCap: yeah, it looks like you should ln default.cfg to whatever os/arch config you want to use by default. 15:26
@nirik ok, moving along then... 15:26
AndyCap nirik: ah. read it again and I see you can choose release-arch with -r 15:27
@nirik AndyCap: yes, you can choose which specific config you want to use with -r 15:27
@nirik note that you can also use centos repos and build against centos4/centos5 with those configs. 15:27
@nirik ok, so you can also use mock to generate chroots for testing packages or just whatever reason you might have to use a fedora chroot. 15:28
@nirik you can use the init and installdeps and chroot for this. 15:29
@nirik along with the 'shell' command to run a shell in the chroot. 15:30
@nirik one thing worth noting, if you are running multiple mock commands you will want to use the --uniqueext option, so it makes different chroots for each instance. 15:31
@nirik if you do that it should work fine to run multiple mock instances at a time. 15:31
* nirik checks his outline to see what else he wanted to cover... 15:32
@nirik any other general questions. 15:33
@nirik Another thing to note is that if you have a x86_64 machine, you can use mock to build both i386 and x86_64 packages, simply choose the right config with -r when you run it. 15:33
@nirik There are some more tips and suggestions at the mock tips page on the fedora wiki: https://fedoraproject.org/wiki/Extras/MockTricks 15:34
@nirik sorry, thats an old link. You want: https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds 15:35
@nirik 'man mock' has a good deal of info as well. 15:35
@nirik ok, I think thats about all I was hoping to cover... 15:36
AndyCap nirik: thank you. it was interesting 15:37
@nirik no problem... mock is handy for building and testing things for sure. :) 15:38
AndyCap and killing your laptop. :P 15:38
@nirik depending on what you are building, yeah. ;) 15:39
baconfork nirik: Thank you 15:39
sdodson If you were working in an environment where the buildroot changes a lot, like a koji builder, is squid probably the best caching tool? 15:39
@nirik sdodson: yeah, squid works fine. I think thats what the fedora buildsys uses to cache packages like that. 15:40
@nirik all the ones in buildsys-build then are always hot in the cache. 15:40
sdodson nirik: *nod* that doc you referenced has squid settings in it 15:40
sdodson We have a koji hub that has horrible connectivity between it and the builders. 15:41
@nirik yeah, really you can use whatever you like, anything that caches http and works with yum 15:41
@nirik but squid is pretty quick and easy to setup to do this. 15:41
@nirik ok, if there are no further questions, I guess we can close up this session. ;) 15:42
@nirik next up is the wrap up session... which we can start anytime. 15:42
ndim nirik: Would it be difficult to make mock not just build a chroot, but a KVM/qemu image for different arches? 15:43
@nirik ndim: humm... it would need to make a image of the chroot and also install a boot loader, etc... but it should be possible. 15:44
-!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Wrap up session - See https://fedoraproject.org/wiki/Classroom for todays class schedule and more info. 15:44
@nirik so, I guess we can take a break here, or folks can feel free to chime in now with any feedback. Suggestions for how to better run classes? ideas or comments? all feedback welcome. 15:45
@nirik I'd like to thank everyone for coming. Please suggest new topics on the wiki, or better sign up to teach something you know! 15:46
* nirik notes its officially the start of the wrap up session time. Feel free to bring up any feedback... or catch herlo or I with any suggestions. 16:01
-!- nirik changed the topic of #fedora-classroom to: Fedora IRC Classroom - Next Classes (tenative) 10:00 UTC 2009-04-04/2009-04-05 - See https://fedoraproject.org/wiki/Classroom more information. End user help /join #fedora 16:12

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!