Packaging a daemon for macOS
Vincent Bernat
There are three main ways to distribute a command-line daemon for macOS:
- Distributing source code and instructions on how to compile it.
- Using a third-party package manager, like Homebrew.
- Providing an installer package.
Homebrew#
Homebrew is a popular package management system. It works like the BSDs’ ports collections by downloading, compiling and installing the requested software, while also installing any required dependencies automatically. Creating a new package is quite easy and there are a lot of examples available.
However, there are some limitations:
- You don’t really build a package but execute a recipe to locally install the software.
- You need to install development tools, either a whole Xcode installation1 or the command line version.
- If you need to execute some steps as root, you will have to
explain them to the user for them to execute by hand. This includes
the creation of a system user or the installation of a daemon
through
launchd
.
If you can’t live with these limitations, you may need to build an installer package.
Building an installer package#
macOS comes with a graphical and a command-line installer. The graphical one is run by opening a package from the Finder. Then, the user experiences some familiar wizard allowing them to install the software in a matter of a few seconds.
The documentation around how to build such a package is somewhat clumsy. You may find outdated information or pieces of information that only apply to projects using Xcode. I will try to provide here accurate bits in the following context:
- You are packaging a pure command-line tool.
- You are using Autoconf and Automake as a build system.
- You want to support several architectures.
- You want to support older versions of macOS.
Creating a package#
Building such a package was previously done with some graphical tool called PackageMaker. This tool is not available any more and developers are asked to switch to pkgbuild and productbuild. There is a very neat Stackoverflow article on how these tools work.
A package is built in two steps:
- Build component packages.
- Combine them into a product archive.
A component package contains a set of files and a set of scripts to
execute at various steps of the installation. You can have several
component packages, for example a package for the daemon and a package
for the client. They are built with pkgbuild
.
A product archive contains the previously created component packages as well as a file describing various options of the installer (required and optional components, license, welcome text, …).
To create a component package, we need to install the needed files in some directory:
$ ./configure --prefix=/usr/local --sysconfdir=/private/etc $ make $ make install DESTDIR=$PWD/osx-pkg
Update (2022-08)
Since macOS 10.11, it is not possible to
install in /usr
. You need to switch to /usr/local
. This document
has been updated to reflect this change.
We will now put the content of osx-pkg
into a component package with
pkgbuild
:
$ mkdir pkg1 $ pkgbuild --root osx-pkg \ > --identifier org.someid.daemon \ > --version 0.47 \ > --ownership recommended \ > pkg1/output.pkg pkgbuild: Inferring bundle components from contents of osx-pkg pkgbuild: Wrote package to output.pkg
You need to be careful with the identifier. It needs to be unique and
identify your software as well as the specific component. Then, you
need to create an XML file describing the installer. Let’s call it
distribution.xml
:
<?xml version="1.0" encoding="utf-8" standalone="no"?> <installer-gui-script minSpecVersion="1"> <title>Some daemon</title> <organization>org.someid</organization> <domains enable_localSystem="true"/> <options customize="never" require-scripts="true" rootVolumeOnly="true" /> <!-- Define documents displayed at various steps --> <welcome file="welcome.html" mime-type="text/html" /> <license file="license.html" mime-type="text/html" /> <conclusion file="conclusion.html" mime-type="text/html" /> <!-- List all component packages --> <pkg-ref id="org.someid.daemon" version="0" auth="root">output.pkg</pkg-ref> <!-- List them again here. They can now be organized as a hierarchy if you want. --> <choices-outline> <line choice="org.someid.daemon"/> </choices-outline> <!-- Define each choice above --> <choice id="org.someid.daemon" visible="false" title="some daemon" description="The daemon" start_selected="true"> <pkg-ref id="org.someid.daemon"/> </choice> </installer-gui-script>
Fortunately, this file is
documented on Apple Developer Library. Since we only describe one
package, we pass customize="never"
as an option to skip the choice
part. You can however remove this attribute when you have several
component packages. The attribute rootVolumeOnly="true"
explains
that this daemon can only be installed system-wide. It is marked as
deprecated but still works. Its replacement (domains
tag) displays
an unusual and buggy pane which Apple doesn’t use in any of its
packages.
You need to put the HTML2 documents into a resources
directory. It is also possible to choose the background by specifying
a <background/>
tag.
The following command will generate the product archive:
$ productbuild --distribution distribution.xml \ > --resources resources \ > --package-path pkg1 \ > --version 0.47 \ > ../final.pkg productbuild: Wrote product to ../final.pkg
Scripts#
The installer can execute some scripts during installation. For
example, suppose we would like to register our daemon with launchd
for it to be run at system start. You first need to write some
org.someid.plist
file and ensure it will get installed in
/Library/LaunchDaemons
:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>org.someid</string> <key>ProgramArguments</key> <array> <string>/usr/local/sbin/mydaemon</string> <string>-d</string> </array> <key>RunAtLoad</key><true/> <key>KeepAlive</key><true/> </dict> </plist>
Then create a scripts/postinstall
file with the following content:
#!/bin/sh set -e /bin/launchctl load "/Library/LaunchDaemons/org.someid.plist"
You also need a scripts/preinstall
file which will allow your
program to be smoothly upgraded:
#!/bin/bash set -e if /bin/launchctl list "org.someid" &> /dev/null; then /bin/launchctl unload "/Library/LaunchDaemons/org.someid.plist" fi
Ensure that these scripts are executable and add --scripts scripts
to pkgbuild
invocation.
If you need to add a system user, use dscl.
Dependencies#
There is no dependency management built into macOS
installer. Therefore, you should ensure that everything is present in
the package or in the base system. For example, lldpd
relies on
libevent, an event notification library. This library is not
provided with macOS. When it is not found, the build system will use an
embedded copy. However, if you have installed libevent with
Homebrew, you will get binaries linked to this local libevent
installation. Your package won’t work on another host.
The output of otool -L
can detect unwanted dependencies:
$ otool -L build/usr/local/sbin/lldpd build/usr/local/sbin/lldpd: /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 744.19.0) /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 945.18.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
Compilation for older versions of macOS#
The package as built above may only work on the same version of macOS that you use to compile. If you want a package that will work on Mac OS X 10.6, you will need to download the appropriate SDK.3
Then, you need to tell the compiler the target version of macOS and the
SDK location. You can do this by setting CFLAGS
and LDFLAGS
(and
CXXFLAGS
if you use C++):
$ SDK=/Developer/SDKs/MacOSX10.6.sdk $ ./configure --prefix=/usr/local --sysconfdir=/private/etc \ > CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \ > LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK"
The -mmacosx-version-min
flag will be used by various macros in the
headers to mark some functions as available or not, depending on the
version of macOS you target. It will also be used by the linker.
The -isysroot
will tell the location of the SDK. Headers and
libraries will be looked up in this location first.
Update (2022-08)
Starting with SDK 10.9, you don’t have to
specify -isysroot
anymore.
Universal binaries#
Mac OS X 10.6 was available for both IA-32 and x86-64 architectures. If you want to support both installations with a single package, you need to build universal binaries. The Mach object file format, used by macOS, allows several versions of the executable in the same file. The operating system will select the most appropriate one.
An easy way to generate such files is to pass -arch i386 -arch
x86_64
to the compiler:
$ ./configure --prefix=/usr/local --sysconfdir=/private/etc \ > CC="gcc -arch i386 -arch x86_64" \ > CPP="gcc -E"
However, this is a dangerous option. Suppose that your ./configure
script tries to determine some architecture-dependent parameter, like
the size of an integer (with AC_CHECK_SIZEOF
); it will compute it
for the host architecture. The binary generated for the other
architecture will therefore use a wrong value and may crash at some
point.
The correct way to generate a universal binary is to execute two separate compilations and to build the universal binary with lipo:
$ for arch in i386 x86_64; do > mkdir $arch ; cd $arch > ../configure --prefix=/usr/local --sysconfdir=/private/etc \ > CC="gcc -arch $ARCH" \ > CPP="gcc -E" > make > make install DESTDIR=$PWD/../target-$ARCH > cd .. > done […] $ lipo -create -output daemon i386/usr/local/sbin/daemon x86_64/usr/local/sbin/daemon
Since lipo
only works on file, I have written a Python script
applying lipo
recursively to several directories.
Update (2022-08)
Nowadays, you should replace i386
by arm64
.
Apple recommends using -target
to combine both the architecture
and the SDK to target—for example, -target arm64-apple-macos11
.
However, the -arch
flag is still working and you can combine it with
-mmacosx-version-min
if needed. Note that it requires you to compile
on a platform supporting running both architectures (currently, an
Apple M1 or M2).
Putting everything together#
Now, we need to automate a bit. You could provide some nifty script. I
propose an appropriate Makefile.am
instead. This way, we can use the
output variables, like @VERSION@
, from ./configure
to generate
some of the files, like distribution.xml
. You need to add that in
your configure.ac
:
AC_CONFIG_FILES([osx/Makefile osx/distribution.xml osx/im.bernat.lldpd.plist]) AC_CONFIG_FILES([osx/scripts/preinstall], [chmod +x osx/scripts/preinstall]) AC_CONFIG_FILES([osx/scripts/postinstall], [chmod +x osx/scripts/postinstall]) AC_SUBST([CONFIGURE_ARGS], [$ac_configure_args])
Let’s have a look at osx/Makefile.am
. First, we
define some variables:
PKG_NAME=@PACKAGE@-@VERSION@.pkg PKG_TITLE=@PACKAGE@ @VERSION@ PKG_DIR=@PACKAGE@-@VERSION@ ARCHS=@host_cpu@
If we want to build for several architectures, we will type make
ARCHS="x86_64 i386"
. Otherwise it defaults to the current host’s
architecture.
We use install-data-local
target to install files specific to macOS:
install-data-local: install -m 0755 -d $(DESTDIR)/Library/LaunchDaemons install -m 0644 im.bernat.@PACKAGE@.plist $(DESTDIR)/Library/LaunchDaemons uninstall-local: rm -f $(DESTDIR)/Library/LaunchDaemons/im.bernat.@PACKAGE@.plist
The main target is the product archive, built with productbuild
:
../$(PKG_NAME): pkg.1/$(PKG_NAME) distribution.xml resources $(PRODUCTBUILD) \ --distribution distribution.xml \ --resources resources \ --package-path pkg.1 \ --version @VERSION@ \ $@
Its main dependency is the component package:
pkg.1/$(PKG_NAME): $(PKG_DIR) scripts [ -d pkg.1 ] || mkdir pkg.1 $(PKGBUILD) \ --root $(PKG_DIR) \ --identifier im.bernat.@PACKAGE@.daemon \ --version @VERSION@ \ --ownership recommended \ --scripts scripts \ $@
Now, we need to build $(PKG_DIR)
:
$(PKG_DIR): stamp-$(PKG_DIR) stamp-$(PKG_DIR): $(ARCHS:%=%/$(PKG_DIR)) $(srcdir)/lipo $(PKG_DIR) $^ touch $@
The $(ARCHS:%=%/$(PKG_DIR))
will expand to x86_64/$(PKG_DIR)
i386/$(PKG_DIR)
. This is a substitution reference.4
Before applying our lipo
script, we need to be able to build the
architecture-dependent package directories:
pkg_curarch = $(@:stamp-%=%) $(ARCHS:%=%/$(PKG_DIR)): %/$(PKG_DIR): stamp-% $(ARCHS:%=stamp-%): stamp-%: im.bernat.lldpd.plist [ -d $(pkg_curarch) ] || mkdir -p $(pkg_curarch) (cd $(pkg_curarch) && \ $(abs_top_srcdir)/configure @CONFIGURE_ARGS@ \ CC="@CC@ -arch $(pkg_curarch)" \ CPP="@CPP@") (cd $(pkg_curarch) && \ $(MAKE) install DESTDIR=$(abs_builddir)/$(pkg_curarch)/$(PKG_DIR)) touch $@
This may seem a bit obscure, but this is really like what we have
described previously. Note that we use @CONFIGURE_ARGS@
which is a
variable we have defined in configure.ac
.
Here is how a user would create its own macOS package:
$ SDK=/Developer/SDKs/MacOSX10.6.sdk $ mkdir build && cd build $ ../configure --prefix=/usr/local --sysconfdir=/private/etc --with-embedded-libevent \ > CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \ > LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" […] $ make -C osx pkg ARCHS="i386 x86_64" […] productbuild: Wrote product to ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg The package has been built in ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg.
-
Even if Xcode is free, you will need to register to Apple and show a valid credit card number. That’s quite an interesting requirement. ↩︎
-
It is possible to use other formats (like RTF or just plain text) for documents but HTML seems the best choice if you want some formatting without fiddling with RTF. ↩︎
-
Unfortunately, Apple makes it difficult to find them. The easiest way to get the SDK for Mac OS X 10.6 is to download Xcode 4.3.3, mount the image and copy the SDK to
/Developer/SDKs
. ↩︎ -
This is equivalent to the use of
patsubst
function. However, this function is specific to GNU make. The substitution reference pattern was accepted recently in POSIX and is coined as pattern macro expansion. ↩︎