Stefan Schaeckeler. NetBSD Tips and Tricks, 2021

Foreword

These are my notes for NetBSD. Originally written for NetBSD 7, they have been slightly extended and do also apply to NetBSD 8 and 9.

Please note the prompts:

Powermanagement

Current and available CPU frequencies:

$ sysctl -a | grep freq
machdep.est.frequency.current = 1333
machdep.est.frequency.available = 2167 1667 1333 1000

Setting CPU frequency:

# sysctl -w machdep.est.frequency.current=1333

Frequency scaling daemon in pkgsrc sysutils/estd:

/etc/rc.conf:

estd=YES estd_flags="-d -b -M 1667" # upper limit 1667 MHz

TODO: Investigate C-state and P-state support in NetBSD.

Suspend to memory on amd64:

# sysctl -w hw.acpi.sleep.state=3

Suspend to memory on i386:

# zzz

Suspend to disk:

Suspend to disk is not supported by {Net|Open|Free}BSD

Disabling power management to fix Load_Cycle_Count:

# atactl wd0 apm set 254

PS: atactl wd0 apm disable does not do the job.

Setting apm level:

# atactl wd0 apm set <value>

Reading apm level (doesn't seem to be possible with atactl)

# smartctl -g apm /dev/rwd0d 
APM level is:     254 (maximum performance)

On my system power management is disabled in /etc/rc.local via atactl wd0 apm set 254. This setting does not survive suspend to mem, so it must be re-executed, e.g. via:

# zzz; date; sleep 30; date; . /etc/rc.local; smartctl -g apm /dev/rwd0d

Note the sleep: after executing zzz, the machine is still running for a few sec and without the sleep, the execution of /etc/rc.local would get executed before the actual suspension and not after the suspension. 30 sec work for my machine.

Update Intel Microcode:

Package sysutils/intel-microcode-netbsd dynamically loads the latest microcode from Intel.

Manual load

# /usr/pkg/share/examples/rc.d/intel-microcode onestart

Autoload on boot

# cp /usr/pkg/share/examples/rc.d/intel-microcode /etc/rc.d
# echo microcode=YES >> /etc/rc.conf

My original microcode version is 0xc2

$ cpuctl identify 0 | grep microcode
cpu0: microcode version 0xc2, platform ID 1
$ cpuctl identify 1 | grep microcode
cpu1: microcode version 0xc2, platform ID 1

Updated microcode to version 0xe2

$ cpuctl identify 0 | grep microcode
cpu0: microcode version 0xe2, platform ID 1
$ cpuctl identify 1 | grep microcode
cpu1: microcode version 0xe2, platform ID 1

Display sensors (temperature, battery):

$ envstat
                         Current  CritMax  WarnMax  WarnMin  CritMin  Unit
:
cpu0/cpu1 temperature:    69.000   99.000                             degC
:

CPU temperature table for cpu0/cpu1 (on a hot summer evening) (set via sysctl -w machdep.est.frequency.target=1000)

Freq idle make-j2
1000 MHz 47 ℃ 57 ℃
1333 MHz 51 ℃ 68 ℃
1667 MHz 56 ℃ 80 ℃
2167 MHz 65 ℃ 94 ℃

smartctl:

sysutils/smartmontools provides smartctl to display hard drive health:

# smartctl -a /dev/rwd0d
:

e.g.

# smartctl -a /dev/rwd0d | grep Load_Cycle_Count

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
193 Load_Cycle_Count        0x0032   086   086   000    Old_age   Always   144467

Battery status:

Display current battery status (via package graphics/gnuplot)

$ while true; do envstat | grep charge: | cut -b 25-35 >> charge; gnuplot -e "set term xterm; plot [0:][0:] 'charge' using 0:1"; sleep 10; done

Kernel

Kernel Interface

There is no /sysfs filesystem on BSD. The kernel interface is exposed to userland via sysctl, e.g. sysctl -a, sysctl -w xxx.yyy.zzz=3.

Compilation Time

Kernel compilation time NetBSD 7.0-RC2 GENERIC, core duo, system compiler gcc-4.8.4, X11 running:

parallelization 1000 Mhz 1333 Mhz 1667 MHz 2167 MHz
make -j1 depend 4m13.150s 3m11.536s 3m14.164s 2m11.462s
make -j1 25m25.067s 19m15.233s 16m16.665s 12m36.376s
make -j2 depend 2m40.423s 2m10.196s 1m48.069s 1m29.399s
make -j2 14m27.399s 11m20.775s 9m20.556s 8m00.394s

XXX make -j1 depend discrepancy between 1333 and 1667 MHz XXX

Building kernel:

See Userland section. Config file is /usr/src/sys/arch/*/conf/GENERIC.

Userland

Building Userland Steps:

  1. First time checkout (-7 is always latest stable, e.g. 7.0, 7.1, etc):

    # cd /usr
    # export CVS_RSH=ssh 
    # cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot co -r netbsd-7 -P src
    # cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot co -r netbsd-7 -P xsrc
    1. Further source updates of tools, kernel and userland:

      # cd /usr/src
      # export CVS_RSH=ssh 
      # cvs update -dP
    2. Further source updates of x-window:

      # cd /usr/xsrc
      # export CVS_RSH=ssh 
      # cvs update -dP
  2. Building tools, kernel, userland and x-window [timings with 1.667 GHz][-u: don't clean][-x: build and install x-window as well (this is not done from /usr/xsrc)][MKDEBUG: build debug symbols and install into /usr/XXX]:

    # mkdir /usr/obj /usr/tools
    # cd /usr/src
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools tools                                    #        24 min 08 sec
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools kernel=GENERIC                           #        21 min 57 sec
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools modules                                  #        29 min 54 sec
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin MKDEBUG=1 MKDEBUGLIB=1 ./build.sh -x -u -O ../obj -T ../tools distribution   # 5 hour 15 min 59 sec
  3. Installing (tools?,) kernel, userland and x-window:

    # cd /usr/src
    # mv /netbsd /netbsd.old
    # mv /usr/obj/sys/arch/amd64/compile/GENERIC/netbsd /
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh    -O ../obj -T ../tools installmodules=/
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -x -O ../obj -T ../tools install=/ 
    # rm -rf /usr/tools /usr/obj
    # shutdown -r now
  4. Building and installing everything as a one-liner:

    PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools tools && PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools kernel=GENERIC && PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -u -O ../obj -T ../tools modules && PATH=/sbin:/usr/sbin:/bin:/usr/bin MKDEBUG=1 MKDEBUGLIB=1 ./build.sh -x -u -O ../obj -T ../tools distribution && mv /netbsd /netbsd.old && mv /usr/obj/sys/arch/amd64/compile/GENERIC/netbsd / && PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -O ../obj -T ../tools installmodules=/ && PATH=/sbin:/usr/sbin:/bin:/usr/bin ./build.sh -x -O ../obj -T ../tools install=/

Debugging NetBSD Userland Steps:

My steps to fix bug lib/50367:

  1. cscope /usr/src (indexes also #includes from /usr/include):

    # cd /usr/src
    # cscope-indexer -r -v
  2. Build optimized bins/libs and debug symbol files:

    # cd /usr/src    [-u doesn't clean - omit the first time]
    # PATH=/sbin:/usr/sbin:/bin:/usr/bin MKDEBUG=1 MKDEBUGLIB=1 ./build.sh -u -O ../obj -T ../tools distribution

    XXX debug versions don't seem to be installed - perhaps set MKDEBUG environment, variables for installation, as well

    # ls /usr/obj/usr.bin/getaddrinfo/
    getaddrinfo.c getaddrinfo.o getaddrinfo getaddrinfo.debug
  3. Debug new Binary:

    # LD_LIBRARY_PATH=/usr/obj/lib/libc gdb getaddrinfo
    (gdb) set solib-search-path /usr/obj/lib/libc
    Reading symbols from /usr/obj/lib/libc/libc.so.12.193.1...
    Reading symbols from /usr/obj/lib/libc/libc.so.12.193.1.debug...
  4. Run Program against newly build Shared Library:

    # LD_LIBRARY_PATH=/usr/obj/lib/libc getaddrinfo heise.de
  5. Save Changes:

    # cd /usr/src
    # cvs diff > changes.diff   # diff against cvs server version
    # patch -R < changes.diff   # undo changes

Pkgsrc

Config files:

/etc/mk.conf, /root/.profile (partial) and /root/.cvsrc. Please run all commands in /usr/pkgsrc:

Check package consistency:

# pkg_admin check

Update pkgsrc files:

# cvs update

Update installed packages:

  1. check packages that need updates (cmp pkgsrc vs installed packages):

    # pkg_chk -u -q
  2. update these packages (dependent packages are recompiled as well):

    # pkg_chk -u -s

Switch to another branch:

# cvs update -dP -rpkgsrc-2013Q2

Switching to current:

# cvs update -dPA

Purge from time to time:

# rm -rf /var/tmp/pkgsrc/ # this is my /etc/mk.conf WRKOBJDIR PATH
# rm /usr/pkgsrc/distfiles/*
# rm /usr/pkgsrc/packages/All/*

Wifi

TODO. For now, try ifconfig wpi0 list scan and wap_gui.

Virtualization

Since NetBSD 9.0 and pkgsrc 2020Q1, NetBSD provides qemu acceleration via nvmm [details]. Speed improvement over plain qemu is around factor 20 for compiling the Linux kernel in a Debian 10 guest.

  1. Install emulators/qemu/ from pkgsrc-2020Q1 onward
  2. Make nvmm accessible to a user here sschaeck

    # echo "nvmm:*:34:sschaeck" >> /etc/group
    # chown root:nvmm /usr/pkg/bin/qemu-system-x86_64
    # chmod g+s /usr/pkg/bin/qemu-system-x86_64
  3. Boot-time setup (or add module name to /etc/modules.conf)

    # modload nvmm
  4. Optional: create this alias

    $ alias qemu="qemu-system-x86_64 -accel nvmm"
  5. Qemu 101 (assumes above alias)

    1. Boot an iso

      $ qemu -m 512M -cdrom /opt/isos/KNOPPIX*.iso
    2. Create an empty 10GB disk

      $ qemu-img create -f qcow2 /opt/qemu/debian.img 10G
    3. Install an operating system from a bootable iso onto the just created empty disk (the booted iso will see the empty disk)

      $ qemu -hda /opt/qemu/debian.img -cdrom /opt/isos/firmware-10.3.0-amd64-netinst.iso -boot d -m 512M -netdev user,id=mynet0,host=10.0.2.10 -device e1000,netdev=mynet0
    4. Boot an operating system from a disk (guest ip 10.0.2.15 / ssh from host to guest ssh sschaeck@127.0.0.1 -p 7001)

      $ qemu -hda /opt/qemu/debian.img -m 512M -netdev user,id=mynet0,host=10.0.2.10,hostfwd=tcp:127.0.0.1:7001-:22 -device e1000,netdev=mynet0
    5. Mount guest from host (hack hack hack)

      # mount_psshfs -O Port=7001 sschaeck@127.0.0.1:/ /mnt/tmp 
        ...
      # umount /mnt/tmp

Misc

Mixer:

Volume control via mixerctl, e.g. mixerctl -a; mixerctl -w monitor.output=200.

Sshfs:

X:

Starting X applications as root or any another user (here named otheruser).

$ echo $DISPLAY
$ xhost local:otheruser
$ su - otheruser
$ export DISPLAY=:0 # the DISPLAY is a moving target increasing on each start. Take it from command #1

East-Asian Language Fonts:

Provided by packages fonts/cyberbit-ttf and fonts/kochi-ttf (both are required).

Burning CDs/DVDs:

TODO

Linux Emulation:

NetBSD provides a seamless Linux emulation. Package meta-pkgs/suse* provides a whole Linux userland in /usr/pkg/emul/linux[32|64]/.

A Linux binary, say downloaded from the Internet with dependencies on libraries from /usr/pkg/emul/linux[32|64]/, can be executed from any directory such as /opt/bin/. One can also chroot into /usr/pkg/emul/linux[32|64]/:

# chroot /usr/pkg/emul/linux32/ /bin/bash
bash-3.00# uname -a
Linux hostname 3.11.6 #1 SMP PREEMPT Thu Oct 24 16:23:02 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

TODO: Try building the Linux kernel in a chrooted environment.

Disk Encryption:

Opposed to OpenBSD and FreeBSD, NetBSD does not support whole disk encryption. NetBSD's crypto framework cgd encrypts whole slices with one 1 passphrase for all partitions within it. I encrypted a slice holding exactly one partition, the /home partition. Also, swap on yet another slice can be encrypted with an always randomly generated passphrase. Here I succeeded to a certain degree, only.

Update: I learned of cgdroot for encrypting the root filesystem. I have not used it.

Limitations

Contact

You are welcome to send me comments. Contact information is available on the main page.