Cleaning Up Ports

Despite all the praise the FreeBSD ports system gets, it has limitations. One of these limitations is actually related to one of FreeBSD's other strengths -- the upgrade system. The two interact in a very clumsy way.

The port installation process records every file a port includes and all of the port's dependencies. For example, many ports require the PNG graphics library. The port records the exact version of PNG it used, such as version 1.0.11. This is generally the same version found in the ports tree when you installed your first piece of software. The FreeBSD upgrade process can upgrade the version of the software supported by the ports tree. For example, PNG was recently upgraded to version 1.0.12. The library version number didn't increase, but the new library has minor bug fixes. I'd like to upgrade. If I do that, however, the 30-odd ports that have the exact PNG version hard-coded in them will have incorrect records.

My laptop has 187 packages. I upgrade my ports tree on a weekly basis. This gets messy very quickly, and impossible soon after. Fortunately, Akinori MUSHA has stepped up to the plate and developed a tool to untangle this mess: portupgrade(1) and friends.

The tools included in portupgrade implement some new features in the FreeBSD ports system. First is pkgdb and portdb, which build databases to index /var/db/pkg and the ports tree. This accelerates searching and manipulating this information. These tools also rewrite the various /var/db/pkg files to maintain consistency. Finally, there are wrappers for the various pkg_* commands that handle rewriting both the databases and the plain-text records when you add or remove software. You get all this by installing /usr/ports/sysutils/portupgrade.

Before you start using portupgrade, back up /var/db/pkg! I've never had any trouble with this program, but any tool that directly alters system records has boundless potential to make your life difficult. If you completely destroy your package records, you want to be able to restore them.

# tar -czvf dbpkg.tgz /var/db/pkg

Run pkgdb -F to do your initial setup and do some basic clean up. If you have a lot of ports, and have upgraded your ports tree several times, set aside a solid block of time to do this the first time. Once you have a consistent database, maintenance is quick and easy; it's only this first install that is time-consuming.

#pkgdb -F
Checking the origin of Hermes-1.3.2
Checking the origin of Mesa-3.4.1
Checking the origin of Mesa-3.4.2_1
Checking the origin of ORBit-0.5.10_5
Checking the origin of XFree86-aoutlibs-3.3.6.9
...

When it comes across a package with changed dependencies, pkgdb will ask you if you want to correct them.

Stale dependency: esound-0.2.22 -> libaudiofile-0.2.2:
libaudiofile-0.2.1_1 ? ([y]es/[n]o/[a]ll) [yes]

So, esound has recorded libaudiofile version 0.2.2 as a dependency, but version 0.2.1_1 is installed. I probably did a pkg_add -f at some point in the past, thinking that it would work despite the minor difference in version numbers. (It did.) It's asking if I want to update my entry in /var/db/pkg/esound-0.2.22/+CONTENTS to point to the version of libaudiofile that's actually installed. If I take the default, it will correct the entry. If I type "a", it will not only correct the libaudiofile entry for esound but also for every other package that uses libaudiofile. This is very handy for packages that have many dependent packages, such as PNG.

In the previous example, pkgdb was able to make an intelligent guess about the dependency. Other times, it cannot guess at all. For example, I built XFree86 from xfree86.org's CVS repository. imake is part of XFree86, so it has no record in /var/db/pkg. This is OK; the port-building process checks for the existence of a program, not for its record in /var/db/pkg. When a port's installation record is placed in /var/db/pkg, however, it assumes that the port is there if it found the program. (This is arguably a shortcoming in the ports system, but there's no easy fix for this.)

What this means is that when a port lists imake as a dependency, pkgdb chokes.

Stale dependency: Xaw3d-1.5 -> imake-4.1.0:
New dependency? (? to help): ?

Here I have a choice. I can either assign a new dependency, or I can tell the package that this is no longer a dependency. While Xaw3d does require imake, I know that on this particular system it will never find an imake package installed. I want to delete the dependency. If you hit "?" for help, you'll see:

[Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to complete
New dependency? (? to help):

Control-D it is.
Delete this? ([y]es/[n]o/[a]ll) [yes]
Deleted.

If I choose "yes," it will delete just this dependency. If I choose "all" it will delete this dependency whenever it appears. A lot of ports use imake; I want to delete them all. You'll see this show up later:

Stale dependency: xneko-4.4 -> XFree86-libraries-4.1.0:
Delete this? ([y]es/[n]o/[a]ll) [yes] Deleted.

Other times, there really is a dependency elsewhere that pkgdb can't guess at.

Stale dependency: plugger-3.3_1 -> timidity++-esound-2.10.4:
New dependency? (? to help): ?
 [Enter] to skip, [Ctrl]+[D] to delete, [.][Enter] to abort, [Tab] to complete
New dependency? (? to help): timidity++-2.10.4

The tab completion makes this doable. I know that timidity++-esound is using some version of timidity, but I don't want to go digging around to find out the whole version number. I can type in the first few letters and hit tab, and get the rest of the package name filled in.

Finally, at the end of the whole process pkgdb will warn you about duplicates.

Duplicated origin: graphics/Mesa3 - Mesa-3.4.1 Mesa-3.4.2_1
Remove any of them? [no]

Here I have a problem. It's not uncommon when you install software from packages to have the package require another package. You might have a slightly different version of the package already installed. If you're installing via package, however, it automatically installs dependencies even if a slightly different version is already installed. At some point this happened to my system. My laptop has two different versions of Mesa, but one has overwritten the other. It obviously cannot be too bad, as my system still works. (Eventually, I should entirely remove Mesa and install a known good version.) Still, I need to clean up the package database. I arbitrarily removed the older version.