This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.
Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.
This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.
First install the sysutils/sysbuild-user package and trigger a full build of NetBSD so that you get usable release sets for pkg_comp. See sysbuild(1) and pkg_info sysbuild-user for details on how to do so. Alternatively, download release sets from the FTP site and later tell pkg_comp where they are.
Then install the pkgtools/pkg_comp-cron package. The rest of this tutorial assumes you have done so.
To use pkg_comp for periodic builds, you’ll need to do some minimal edits to the default configuration files. The files can be found directly under /var/pkg_comp/, which is pkg_comp-cron’s “home”:
/var/pkg_comp/pkg_comp.conf: This is pkg_comp’s own configuration file and the defaults installed by pkg_comp-cron should be good to go.
The contents here are divided in three major sections: declaration on how to download pkgsrc, definition of the file system layout on the host machine, and definition of the file system layout for the built packages.
You may want to customize the target system paths, such as LOCALBASE or SYSCONFDIR, but you should not have to customize the host system paths.
/var/pkg_comp/list.txt: This determines the set of packages you want to build in your periodic cron job. The builds will fail unless you list at least one package.
WARNING: Make sure to include pkg_comp-cron and pkgin in this list so that your binary kit includes these essential package management tools. Otherwise you’ll have to deal with some minor annoyances after rebootstrapping your system.
Lastly, review root’s crontab to ensure the job specification for pkg_comp is sane. On slow machines, or if you are building many packages, you will probably want to decrease the build frequency from @daily to @weekly.
Here is what the configuration looks like on my NetBSD development machine as dumped by the config subcommand. Use this output to get an idea of what to expect. I’ll be using the values shown here in the rest of the tutorial:
# pkg_comp -c /var/pkg_comp/pkg_comp.conf config AUTO_PACKAGES = autoconf automake bash colordiff dash emacs-nox11 git-base git-docs gmake gnuls lua52 mozilla-rootcerts pdksh pkg_comp-cron pkg_developer pkgin sqlite3 sudo sysbuild sysbuild-user sysupgrade tmux vim zsh CVS_ROOT = :ext:anoncvs@anoncvs.NetBSD.org:/cvsroot CVS_TAG is undefined DISTDIR = /var/pkg_comp/distfiles EXTRA_MKCONF = /var/pkg_comp/extra.mk.conf FETCH_VCS = cvs GIT_BRANCH = trunk GIT_URL = https://github.com/jsonn/pkgsrc.git LOCALBASE = /usr/pkg NJOBS = 2 PACKAGES = /var/pkg_comp/packages PBULK_PACKAGES = /var/pkg_comp/pbulk-packages PKG_DBDIR = /usr/pkg/libdata/pkgdb PKGSRCDIR = /var/pkg_comp/pkgsrc SANDBOX_CONFFILE = /var/pkg_comp/sandbox.conf SYSCONFDIR = /etc UPDATE_SOURCES = true VARBASE = /var NETBSD_NATIVE_RELEASEDIR = /home/sysbuild/release/amd64 NETBSD_RELEASE_RELEASEDIR = /home/sysbuild/release/amd64 NETBSD_RELEASE_SETS is undefined SANDBOX_ROOT = /var/pkg_comp/sandbox SANDBOX_TYPE = netbsd-release
Now that you are fully installed and configured, you’ll build some stuff by hand to ensure the setup works before the cron job comes in.
The simplest usage form, which involves full automation, is something like this:
# pkg_comp -c /var/pkg_comp/pkg_comp.conf auto
This trivially-looking command will:
After a successful invocation, you’ll be left with a collection of packages in the directory you set in PACKAGES, which in the default pkg_comp-cron installation is /var/pkg_comp/packages/.
If you’d like to restrict the set of packages to build during a manually-triggered build, provide those as arguments to auto. This will override the contents of AUTO_PACKAGES (which was derived from your list.txt file).
But what if you wanted to invoke all stages separately, bypassing auto? The command above would be equivalent to:
# pkg_comp -c /var/pkg_comp/pkg_comp.conf fetch # pkg_comp -c /var/pkg_comp/pkg_comp.conf sandbox-create # pkg_comp -c /var/pkg_comp/pkg_comp.conf bootstrap # pkg_comp -c /var/pkg_comp/pkg_comp.conf build <package names here> # pkg_comp -c /var/pkg_comp/pkg_comp.conf sandbox-destroy
Go ahead and play with these. You can also use the sandbox-shell command to interactively enter the sandbox. See pkg_comp(8) for more details.
Lastly note that the root user will receive email messages if the periodic pkg_comp cron job fails, but only if it fails. That said, you can find the full logs for all builds, successful or not, under /var/pkg_comp/log/.
Now that you have built your first set of packages, you will want to install them. On NetBSD, the default pkg_comp-cron configuration produces a set of packages for /usr/pkg so you have to wipe your existing packages first to avoid build mismatches.
WARNING: Yes, you really have to wipe your packages. pkg_comp currently does not recognize the package tools that ship with the NetBSD base system (i.e. it bootstraps pkgsrc unconditionally, including bmake), which means that the newly-built packages won’t be compatible with the ones you already have. Avoid any trouble by starting afresh.
To clean your system, do something like this:
# ... ensure your login shell lives in /bin! ... # pkg_delete -r -R "*" # mv /usr/pkg/etc /root/etc.old # Backup any modified files. # rm -rf /usr/pkg /var/db/pkg*
Now, rebootstrap pkgsrc and reinstall any packages you previously had:
# cd /
# tar xzvpf /var/pkg_comp/packages/bootstrap.tgz
# echo "pkg_admin=/usr/pkg/sbin/pkg_admin" >>/etc/pkgpath.conf
# echo "pkg_info=/usr/pkg/sbin/pkg_info" >>/etc/pkgpath.conf
# export PATH=/usr/pkg/bin:/usr/pkg/sbin:${PATH}
# export PKG_PATH=file:///var/pkg_comp/packages/All
# pkg_add pkgin pkg_comp-cron <other package names>
Finally, reconfigure any packages where you had have previously made custom edits. Use the backup in /root/etc.old to properly update the corresponding files in /etc. I doubt you made a ton of edits so this should be easy.
IMPORTANT: Note that the last command in this example includes pkgin and pkg_comp-cron. You should install these first to ensure you can continue with the next steps in this tutorial.
If you paid attention when you installed the pkg_comp-cron package, you should have noticed that this configured a cron job to run pkg_comp daily. This means that your packages repository under /var/pkg_comp/packages/ will always be up-to-date so you can use that to quickly upgrade your system with minimal downtime.
Assuming you are going to use pkgtools/pkgin (and why not?), configure your local repository:
# echo 'file:///var/pkg_comp/packages/All' >>/etc/pkgin/repositories.conf
And, from now on, all it takes to upgrade your system is:
# pkgin update # pkgin upgrade
Enjoy!