Mah Blog

The information on Jeremy Olexa’s blog really helped me start this out, but it seems that there’s a couple of things that vary between installations, and I’m trying to list all that can “go wrong” with such an upgrade here (if you run into other problems, then post a comment, and I’ll see if I can figure something out!). This particular blog post is based on this VMware image.

Yes, I realize that it would probably have been a shit-ton easier to install a fresh version of Gentoo, but I absolutely loathe the Gentoo installation process because I normally mess up one tiny thing than ends up causing me to have to re-install it again, or at the very least, boot up a repair CD. On the plus side, this is a very good learning experience. :-)

To be clear: IF YOUR INSTALL IS THIS OLD, IT’S PROBABLY JUST AS EASY TO BACK UP YOUR IMPORTANT FILES AND START AFRESH. Unless you have a particularly convincing reason that you need to upgrade this machine and not re-install, I would recommend you take that path. My reason? Purely academic. ;-)

Now, let’s begin!

Intro

First of all, get your Gentoo VM (or real install? I don’t know what kind of setups you people on there on the bloggerchats are running!) started up. If you’re following along with me, and are using the VM, then you’ll want to set a new password and start up SSHD. The password below is the default pass for the VM, yours might be different (hopefully they are if you’ve changed them! “gentoo” would be a horrible password ;-).

login as: root
Using keyboard-interactive authentication.
Password:  gentoo
gentoo-vm ~ # passwd
New UNIX password: *************
Retype new UNIX password: *************
passwd: password updated successfully
gentoo-vm ~ # rc-update add sshd default
gentoo-vm ~ # /etc/init.d/sshd start

Your SSHD will generate some certificates, if this is your first time to start the service. Now we really begin! This is from the original page where I got a lot of my information, so pay heed:

Between the first tar command and the success of ’emerge -uDN system’, the true state of the system, and the state according to portage are different. This is rather unsafe, so be sure you understand what you are doing. Consider this your big red warning to backup important files before attempting an update, and consider a re-install if possible.

Getting Portage Up-to-Date

First thing we need to do is sync emerge. REMEMBER! ONCE YOU DO THIS, YOU CANNOT GO BACK. If you’re running on 2006.0 like this VM, then it will remove that profile, leaving you with a broken /etc/make.profile symlink, and a lot of the packages that you’ll need to update the system (update in a “normal” way) have been removed from the Gentoo mirrors, causing quite the headache. With that out of the way, if you’re still interested, let’s begin!

emerge --sync

Once that’s done (and it’ll take a while, but what did you expect? You practically have to fetch a brand-new portage tree!), then you’ll want to set your new profile. To do this, you’ll do:

unlink /etc/make.profile  # Don't add a trailing slash, if you tab-complete
ln -snf /usr/portage/profiles/default/linux/x86/2008.0 /etc/make.profile

OK, now you may be thinking, “What the heck am I going to do now? ’emerge -uavND portage’ is complaining I don’t have bash and python, python is complaining I don’t have a newer version of portage, my system is bork!” Yes, friend. At this point, your system is pretty much broken! But that’s OK! Luckily for us, Gentoo provides some packages pre-built on the Tinderbox I have mirrored the packages that we will require on my hosting in case they’re taken down — they are incredibly old packages.

However, before you go around trying to install the pre-built versions of python, portage and bash that we need, let me forewarn you: if you system is really old, you won’t have glibc-2.4. Why is this important you may ask? Because the pre-built package of bash that is up on the Tinderbox is built on glibc-2.4, and you will get some unfixable (or if not unfixable, extremely hard to fix) problems if you just replace your current version of bash with that one.

If you’re unsure, always untar the pre-built packages to a directory somewhere not on the root (I.E., /home/user/test). That way, you can always try the package first, and if you get library errors, you’re in trouble, but you won’t have killed your system by replacing the system versions of those packages.

Now, you may be wondering, how do we fix this problem? Portage needs bash, but we can’t install bash because of glibc problems. Well, we cheat! Luckily for us, the python and portage versions on the site (at least the ones I’ve mirrored) don’t require glibc-2.4. So, what do we need to do? First, we fetch the packages from the Interweb:

wget http://files.galador.org/apps/python-2.5.2-r7.tbz2
wget http://files.galador.org/apps/portage-2.1.6.7.tbz2

If you want to test the packages first, then do this:

cd ~
mkdir test
cd test/
tar xfpj ../python-2.5.2-r7.tbz2
tar xfpj ../portage-2.1.6.7.tbz2

If you get any “bzip2: (stdin): trailing garbage after EOF ignored” errors, don’t worry about them. That’s just a “hack” in the bzip2 protocol that Gentoo uses to sign their packages.

usr/bin/emerge --version

Should output something like: “Portage 2.1.6.7 (default/linux/x86/2008.0, gcc-3.4.4, glibc-2.3.5-r2, 2.6.15-gentoo-r1 i686)”

So far, so good! Now to actually get the packages install system-wide:

cd /
tar xfpjv root/python-2.5.2-r7.tbz2
tar xfpjv root/portage-2.1.6.7.tbz2
which emerge # This should be /usr/bin/emerge
emerge --version  # This should match the version you just installed -- 2.1.6.7.

If you’ve made it this far, pat yourself on the back, you haven’t fucked anything up yet! But you’re still far from a stable system. As you know, we need a newer version of bash, but portage still internally thinks we’re running the old version of portage, which is blocking bash. There’s a quick-and-dirty workaround for this, though. We have to edit the bash ebuild and remove the requirement for portage. (I use nano because I’m familiar with it, if you use something else, then substitute nano with your editor of choice.)

cd /usr/portage/app-shells/bash
nano /usr/portage/app-shells/bash/bash-3.2_p39.ebuild

In nano, search for the line that looks like “RDEPEND=”${DEPEND}”. The next line should include a portage reference, just delete that line. (Leave the paludis line). Save your changes and rebuild the ebuild:

repoman manifest
cd /

Now, let’s get bashing!

emerge -uavND app-shells/bash

This’ll pull in some new configuration files that we need to use, so:

dispatch-conf  #  Scroll to the bottom with Page Down, hit "u"

At this point, I find the easiest way to make sure you’re using the new bash is to just log out and log back in, so type “exit” a few times until you’re back to a login prompt, and log back in so we can delve deeper into the madness!

Updating System

As of right now, if you’ve made it this far, you’re really done with most of the hard stuff. Now you just need to update some (OK, who am I kidding, all) of the packages on your system. However, before we do that, I’m going to get distcc and ccache set up and running on the machine. This next Section is completely optional, so if you have no need for either of those, then skip to the next section.

OPTIONAL SECITION

distcc

Distcc is a distributed compiler daemon that spreads compilation tasks across computers running the daemon. To install it, do the following:

emerge distcc
distcc-config --set-hosts "localhost 1.2.3.4 5.6.7.8 9.10.11.12"  # etc.
nano /etc/conf.d/distccd # Be sure to set the --allow and --listen directives
rc-update add distccd default # Add to the default runlevel
/etc/init.d/distccd start  # Start 'er up!

ccache

Ccache is a compiler-caching layer. Believe me, you’re going to be compiling a lot of shit, so you want to eke out every megahert possible from your processor, and this package will help the load. There is a caveat, though. I’m not sure if it’s just me, but there have been some problems I’ve had with compiling programs that go away when ccache is disabled. It’s up to you to decide whether ccache is worth your time or not. Let’s go!

emerge ccache

Now, you need to configure your computer to use ccache instead of the “regular” compiler. You need to know your CHOST, so do:

emerge --info | grep CHOST

If you’re like me, that probably means you have “i686-pc-linux-gnu”, so do:

ccache-config --install-links i686-pc-linux-gnu

Now, ccache has some weird cache directory layouts, so let’s make sure all compiles use the same cache directory:

rm -rf /root/.ccache
ln -s /var/tmp/ccache /root/.ccache

Lastly, we need to configure portage to use both of these, so open up your make.conf:

nano /etc/make.conf

Add these lines. Make sure you put ccache before distcc if you want to take advantage of both:

FEATURES="ccache distcc"  
CCACHE_DIR="/var/tmp/ccache"
CCACHE_SIZE="256M" # You can set this to whatever you like.
MAKEOPTS="-jN"  # Again, "N" depends on your own config, do some tweaking!

END OF OPTIONAL SECTION

Before we get back to updating, a quick note: Unless you plan to use GCC with fortran (probably a small percentage of users), you will want to disabled the “fortran” USE flag with GCC. Even if you do plan on using fortran with your GCC, I’d disable it until you get your system completely updated… unless you want to waste a few hours compiling GCC to find that there’s some weirdness with your old GCC and fortran. ;-)

echo "sys-devel/gcc -fortran" >> /etc/portage/package.use

Then, proceed to updating your world:

emerge -auDNv world

Fixing Portage Update Problems

In this section, I’ll list the problems I faced when updating my Gentoo install. You should probably read ahead in this section to be aware of things that might come up. Obviously, after “fixing” all of these, keep running `emerge -uavND world` until you run into the next problem.

I would also recommend that as soon as you compile the new version of GCC that you stop your updating (Ctrl + C), and switch to it to prevent duplicating code compilation. So, based on the Gentoo GCC upgrade guide, let’s switch that over. Remember to use the correct chost-gcc_version below.

gcc-config <CHOST>-<new-gcc-version>
env-update && source /etc/profile
/usr/share/gcc-data/<CHOST>/<new-gcc-version>/fix_libtool_files.sh <old-gcc-version>
emerge --oneshot -av libtool

Now at this point, the Gentoo GCC upgrade guide recommends that you rebuild your entire toolchain and world with an empty portage tree (basically, re-installing everything) with the new compiler. I’ve seen a few articles saying that this is really not needed (they said it was for those “ricer” types who like to squeeze the last ounce of performance out of their computers). I’m not sure which side to believe, but if Gentoo says you should do it, it’s probably a good idea. :-P So:

emerge -eav world

Remember, this is updating everything using the new compiler and re-building all libraries. This is going to take a VERY long time. I would recommend you do this overnight. ;-) However, this is where you’ll probably start running into problems, so keeping an eye on it in the background might not be a bad idea, either.

Now, we start with the possible problems:

Possible World Update Problems

If you get complaints that “sys-apps/util-linux (is blocking sys-apps/coreutils)”, then make sure that you’re trying to pull in a relatively new version (sometimes portage tries to pull in older versions) by doing:

emerge -uav  "=sys-apps/util-linux-2.13.1.1"

You may run into blocks whilst doing this with debianutils. Just unmerge debianutils. All the important stuff from that package has moved to coreutils.

emerge --unmerge sys-apps/debianutils

The next thing it’ll might complain about is an old version of python-updater. Portage thinks you’re using python2.4, but you installed python2.5 from a pre-built package, so you’ve confused it. Just unmerge python-updater, and re-install it!

rm /usr/sbin/python-updater
emerge -1 python-updater

You may get some blocks with man-pages and man-pages-posix. Just unmerge man-pages as they’re not required for system stability, and will be re-merged when you update world.

emerge --unmerge sys-apps/man-pages

Your next problem might be with sts-libs/timezone-data. In the old versions, timezone data was provided by glibc, but was moved to a new package later on. This, however, causes some collision problems, so for that package turn on collision ignore:

COLLISION_IGNORE="/usr" emerge -av timezone-data

sys-apps/kbd might fail next because it can’t find “libexpat.so.0”. A newer version may have been installed, so the symlink was removed. Just do this to fix:

ln -s /usr/lib/libexpat.so.1.5.2 /usr/lib/libexpat.so.0

Your next problem might be file collisions with perl and perl-core/Test-Harness. Again, it’s usually easiest to just unmerge the packages in question and re-merge. If it still complain after unmerging Test-Harness, then unmerge perl as well. It’s a good thing that Gentoo’s package manager is written in Python, not perl!

emerge --unmerge  perl-core/Test-Harness

Post-Update Cleanup

Once you’ve completed the update of world, there are a few things your probably need to do that were reported to you via update messages which you might have missed, so we’ll take care of those:

hash -c  # Required by coreutils

(I’ve found this command doesn’t actually work with newer versions of hash, yet coreutils continues to tell you to do this. :-S )

The new certificate package might have left some old orphan certificates, so get rid of those:

find -L /etc/ssl/certs/ -type l -exec rm {} +

You need to ensure that all your packages are using consistent library linking. In order to do this, we use the fine “revdep-rebuild” tool which is provided by the “gentoolkit” package, so emerge that:

emerge -uavND app-portage/gentoolkit
revdep-rebuild
rm '/usr/lib/libcrypto.so.0.9.6'
rm '/usr/lib/libcrypto.so.0.9.7'
rm '/usr/lib/libssl.so.0.9.6'
rm '/usr/lib/libssl.so.0.9.7'

Probably got a new version of perl earlier, so run the cleaner:

perl-cleaner

The “slocate” package now uses the group “locate” instead of “slocate”

groupmod -n locate slocate
updatedb # Update your locate DB while you're at it

The last major problem should be with PAM. Versions older that 0.99 had a completely different setup, so we need to make a few changes to that.

# First, we need to create some directory structure:
cd /etc/pam.d/
mkdir auth
mkdir include
emerge pam
dispatch-conf

Now, before you do *ANYTHING* else, make sure that the new PAM install is working by logging in. If PAM’s not working, then you’re not going to be able to login, and if you close your current terminal, then you’re going to be left with a borked system.

At this point, you’re getting pretty close to a fixed system! There should only be a couple of packages left when you do “emerge -uavND world”. Get those compiled, then “dispatch-conf” to get the configurations.

The last thing to do is compile the kernel. I’ll leave that to your own devices. Be sure to re-install grub (you compiled a new version, remember?) with “grub-install –no-floppy /your/bootdrive”

Restart your computer with “shutdown -r now”, and enjoy your now-updated Gentoo install!