Maintaining Your Own pkgng Repository

pkgng is next generation package management utility for FreeBSD. Expected to be the default package manager in FreeBSD 9.1-RELEASE, its functionality far surpasses the functionality (or lack thereof) of its predecessor — pkg_install.

One of the great features about pkgng is security — packages installed on the system can be verified against an RSA signature, ensuring the package that is intended to be installed is the package that will be installed.

This "non-blog" entry will cover how I set up my pkgng package building system for my laptop.

I use a FreeBSD Ports Tinderbox regularly for various tasks, such as one-off package builds for a particular piece of software, or testing an update to a FreeBSD port for all supported architectures. Very recently, an update was made to the ports-mgmt/tinderbox-devel FreeBSD port, which allows a user to build packages for use with pkgng.

One of the key differences between pkgng packaging format and the traditional pkg_install packaging format is the metadata stored within the package, and where this metadata is stored. Additionally, the contents within the resulting package differ as well.

To illustrate why additional steps are necessary to build packages in pkgng format, the following is the contents of the resulting packages of the archivers/zip port. This is the traditional pkg_install format:

	% tar tf zip.tbz
	+CONTENTS
	+COMMENT
	+DESC
	+MTREE_DIRS
	bin/zip
	bin/zipcloak
	bin/zipnote
	bin/zipsplit
	man/man1/zip.1.gz
	man/man1/zipcloak.1.gz
	man/man1/zipnote.1.gz
	man/man1/zipsplit.1.gz

This is the contents of the new pkgng format:

	% tar tf zip.txz
	+MANIFEST
	+MTREE_DIRS
	/usr/local/bin/zip
	/usr/local/bin/zipcloak
	/usr/local/bin/zipnote
	/usr/local/bin/zipsplit
	/usr/local/man/man1/zip.1.gz
	/usr/local/man/man1/zipcloak.1.gz
	/usr/local/man/man1/zipnote.1.gz
	/usr/local/man/man1/zipsplit.1.gz
	/usr/local/share/doc/zip/CHANGES
	/usr/local/share/doc/zip/LICENSE
	/usr/local/share/doc/zip/README
	/usr/local/share/doc/zip/README.CR
	/usr/local/share/doc/zip/TODO
	/usr/local/share/doc/zip/WHATSNEW
	/usr/local/share/doc/zip/WHERE
	/usr/local/share/doc/zip/

One noteworthy change is the removal of the +COMMENTS, +DESC, and so on, with a master +MANIFEST file. Another noteworthy change is the xz compression algorithm, which provides greater compression than the traditional bzip2 algorithm.

So, having given the absolute minimum information on what is changed to show why it is necessary to configure a package building system special for pkgng, here is how I chose to configure mine.

There are a few ways to set up a package building system:

  • The ports-mgmt/tinderbox-devel port
  • The ports-mgmt/poudriere port
  • No additional software, and using make package from within a specified port directory

I personally do not have any strong feelings about one solution being better than the other, but since I already had Tinderbox set up, I decided to use that, in addition to the target machine being a laptop I did not want to keep busy with compiling software.

Building the Packages

Setting up Tinderbox to build pkgng-compatible packages was fairly straight forward:

  1. Install the ports-mgmt/tinderbox port
  2. Set the necessary environment variables
  3. Add ports to the Tinderbox build

I will not go into configuration of Tinderbox. Please see the Tinderbox README for that.

For pkgng packages, set the following in the appropriate ${pb}/scripts/etc/env/ file for the pkgng build:

	export WITH_PKGNG=yes
	export PKGSUFFIX=.txz

Set additional environment settings as needed for the build.

Next, add ports to the build. I named my build "10-64-LAPTOP", since my laptop runs FreeBSD 10-CURRENT, amd64.

Rather than specifying each port, one at a time, to be added to the build, I wrote a meta-port for the software I want installed on my laptop. This is an abridged version of the misc/laptop port Makefile:

	#
	# $Id: Makefile 795 2012-06-11 16:26:54Z gjb $
	#
	PORTNAME=	laptop
	PORTVERSION=	0.1
	PORTREVISION=	2
	CATEGORIES=	misc
	MASTER_SITES=	# Empty
	DISTFILES=	# None

	MAINTAINER=	me
	COMMENT=	LOCAL Metaport for default laptop software

	NO_BUILD=	yes
	WITH_NEW_XORG=	yes
	WITH_KMS=	yes
	PERL_VERSION=	5.14

	# Use ccache for building everything
	BUILD_DEPENDS=	ccache:${PORTSDIR}/devel/ccache

	RUN_DEPENDS=	# Empty this so subsequent RUN_DEPENDS need +=

	# Essential software
	RUN_DEPENDS+=	\
			vim:${PORTSDIR}/editors/vim-lite \
			svn:${PORTSDIR}/devel/subversion

	# Web development software
	RUN_DEPENDS+=	\
			httpd:${PORTSDIR}/www/apache22 \
			php:${PORTSDIR}/lang/php5 \
			${PORTSDIR}/lang/php5-extensions \
			nginx:${PORTSDIR}/www/nginx

	# Email software
	RUN_DEPENDS+=	\
			mutt:${PORTSDIR}/mail/mutt-devel \
			procmail:${PORTSDIR}/mail/procmail \
			mairix:${PORTSDIR}/mail/mairix

	# Desktop software
	RUN_DEPENDS+=	\
			${PORTSDIR}/x11/xorg \
			irssi:${PORTSDIR}/irc/irssi-devel \
			fluxbox:${PORTSDIR}/x11-wm/fluxbox \
			firefox:${PORTSDIR}/www/firefox \
			chrome:${PORTSDIR}/www/chromium \
			opera:${PORTSDIR}/www/opera \
			${PORTSDIR}/x11-fonts/webfonts

	# FreeBSD Documentation Project build stuff
	RUN_DEPENDS+=	\
			${PORTSDIR}/textproc/docproj-jadetex \
			igor:${PORTSDIR}/textproc/igor \
			mandoc:${PORTSDIR}/textproc/mdocml

	# Miscellaneous software
	RUN_DEPENDS+=	\
			tmux:${PORTSDIR}/sysutils/tmux \
			ezjail-admin:${PORTSDIR}/sysutils/ezjail \
			portlint:${PORTSDIR}/ports-mgmt/porttools \
			rsnapshot:${PORTSDIR}/sysutils/rsnapshot

	do-install:

	.include <bsd.port.mk>

With the meta-port directory contents added to the Tinderbox build Ports tree, add the port and its dependencies to the build, configuring options for each port (if desired):

    [root]# cd ${pb}
    [root]# ./tc addPort -b 10-64-LAPTOP \
	    -d misc/laptop -O

Once the port and dependencies are added, start the build:

    [root]# ./tc tinderbuild -b 10-64-LAPTOP -nullfs

When the packages have finished building, it is time to create a repository, sign the packages, and distribute the packages to the target machine. Tinderbox package building uses the same layout as the FreeBSD FTP server, so you end up with a hierarchy such as:

	All/
	Latest/
	archivers/
	...

Signing and Distributing the Packages

I use a 4096-bit RSA key for signing the package repository contents. The key is created with the openssl command:

    [root]# openssl genrsa -out pkg.key 4096

Then the certificate for distribution to the machines using the repository is created from the generated key:

    [root]# openssl rsa -in pkg.key -pubout > pkg.cert

Note: The name of the certificate file, in my case pkg.cert, must match the file name set as PUBKEY in /usr/local/etc/pkg.conf (if it does not exist, copy /usr/local/etc/pkg.conf.sample and modify to fit your needs).

Next, the packages resulting from the Tinderbox build are signed with the RSA private key:

[root]# pkg repo ${pb}/packages/10-64-Laptop/All/ \
	/path/to/pkg.key

With updated package builds and the repository signed, update the repository metadata on the client, and perform an upgrade, assuming pkg2ng has already been run on the target machine.

    [root]# pkg update
    [root]# pkg upgrade

Now, as ports are updated within the FreeBSD Ports Collection, you can rebuild the misc/laptop port, resign the resulting packages, and easily upgrade your system using pkgng.