第10章 上級パッケージング

# 第10章 上級パッケージング

Let’s describe advanced topics on Debian packaging.

## 10.1. Historical perspective

Let me oversimplify historical perspective of Debian packaging practices focused on the non-native packaging.

Debian was started in 1990s when upstream packages were available from public FTP sites such as Sunsite. In those early days, Debian packaging used Debian source format currently known as the Debian source format 「1.0」:

- The Debian source package ships a set of files for the Debian source package.

  - package_version.orig.tar.xz : symlink to or copy of the upstream released file.
  - package_version-revision.diff.gz : 「One big patch」 for Debian modifications.
  - package_version-revision.dsc : package description.
- Several workaround approaches such as dpatch, dbs, or cdbs were deployed to manage multiple topic patches.

The modern Debian source format 「3.0 (quilt)」 was invented around 2008 (see 「ProjectsDebSrc3.0」):

- The Debian source package ships a set of files for the Debian source package.

  - package_version.orig.tar.?z : symlink to or copy of the upstream released file.

  - package_version-revision.debian.tar.?z : tarball of debian/ for Debian modifications.

    - The debian/source/format file contains 「3.0 (quilt)」.
    - Optional multiple topic patches are stored in the debian/patches/ directory.
  - package_version-revision.dsc : package description.
- The standardized approach to manage multiple topic patches using quilt(1) is deployed for the Debian source format 「3.0 (quilt)」.

Most Debian packages adopted the Debian source formats 「3.0 (quilt)」 and 「3.0 (native)」.

Now, the git(1) is popular with upstream and Debian developers. The git and its associated tools are important part of the modern Debian packaging workflow. This modern workflow involving git will be mentioned later in 「11章Packaging with git」.

## 10.2. Current trends

Current Debian packaging practices and their trends are moving target. See:

- 「Debian Trends」— Hints for 「De facto standard」 of Debian practices

  - Build systems: dh
  - Debian source format: 「3.0 (quilt)」
  - VCS: git
  - VCS Hosting: salsa
  - Rules-Requires-Root: adopted, fakeroot
  - Copyright format: DEP-5
- 「debhelper-compat-upgrade-checklist(7) manpage」— Upgrade checklist for debhelper
- 「DEP - Debian Enhancement Proposals」— Formal proposals to enhance Debian

You can also search entire Debian source code data by yourself, too.

- 「Debian Sources」— code search tool

  - 「Debian Code Search」— wiki page describing its usage
- 「Debian Code Search」— another code search tool

## 10.3. Note on build system

Auto-generated files of the build system may be found in the released upstream tarball. These should be regenerated when Debian package is build. E.g.:

- 「dh $@ --with autoreconf」 should be used in the debian/rules if Autotools (autoconf + automake) are used.

Some modern build system may be able to download required source codes and binary files from arbitrary remote hosts to satisfy build requirements. Don’t use this download feature. The official Debian package is required to be build only with packages listed in Build-Depends: of the debian/control file.

## 10.4. Continuous integration

The dh_auto_test(1) command is a debhelper command that tries to automatically run the test suite provided by the upstream developer during the Debian package building process.

The autopkgtest(1) command can be used after the Debian package building process. It tests generated Debian binary packages in the virtual environment using the debian/tests/control RFC822-style metadata file as continuous integration (CI). See:

- Documents in the /usr/share/doc/autopkgtest/ directory
- 「4. autopkgtest: Automatic testing for packages」 of the 「Ubuntu Packaging Guide」

There are several other CI tools on Debian for you to explore.

- The Salsa offers 「「Salsa CI service」」.
- The debci package: CI platform on top of the autopkgtest package
- The jenkins package: generic CI platform

## 10.5. Bootstrapping

Debian cares about supporting new ports or flavours. The new ports or flavours require bootstrapping operation for the cross-build of the initial minimal native-building system. In order to avoid build-dependency loops during bootstrapping, the build-dependency needs to be reduced using the DEB_BUILD_PROFILES environment variable.

See Debian wiki: 「BuildProfileSpec」.

| ヒント
If a core package foo build depends on a package bar with deep build dependency chains but bar is only used in the test target in foo, you can safely mark the bar with <!nocheck> in the Build-depends of foo to avoid build loops.

## 10.6. Compiler hardening

The compiler hardening support spreading for Debian jessie (8.0) demands that we pay extra attention to the packaging.

You should read the following references in detail.

- Debian wiki: 「Hardening」
- Debian wiki: 「Hardening Walkthrough」

The debmake command adds template comments to the debian/rules file as needed for DEB_BUILD_MAINT_OPTIONS, DEB_CFLAGS_MAINT_APPEND, and DEB_LDFLAGS_MAINT_APPEND (see 「5章Simple packaging」 and dpkg-buildflags(1)).

## 10.7. Reproducible build

Here are some recommendations to attain a reproducible build result.

- Don’t embed the timestamp based on the system time.
- Don’t embed the file path of the build environment.
- Use 「dh $@」 in the debian/rules to access the latest debhelper features.
- Export the build environment as 「LC_ALL=C.UTF-8」 (see 「「Build under UTF-8」」).
- Set the timestamp used in the upstream source from the value of the debhelper-provided environment variable $SOURCE_DATE_EPOCH.

- Read more at 「ReproducibleBuilds」.

  - 「ReproducibleBuilds Howto」.
  - 「ReproducibleBuilds TimestampsProposal」.

Reproducible builds are important for security and quality assurance. They allow independent verification that no vulnerabilities or backdoors have been introduced during the build process.

The control file source-name_source-version_arch.buildinfo generated by dpkg-genbuildinfo(1) records the build environment. See deb-buildinfo(5)

## 10.8. Substvar

The debian/control file also defines the package dependency in which the 「variable substitutions mechanism」 (substvar) may be used to free package maintainers from chores of tracking most of the simple package dependency cases. See deb-substvars(5).

The debmake command supports the following substvars:

- ${misc:Depends} for all binary packages
- ${misc:Pre-Depends} for all multiarch packages
- ${shlibs:Depends} for all binary executable and library packages
- ${python:Depends} for all Python packages
- ${python3:Depends} for all Python3 packages
- ${perl:Depends} for all Perl packages
- ${ruby:Depends} for all Ruby packages

For the shared library, required libraries found simply by 「objdump -p /path/to/program | grep NEEDED」 are covered by the shlib substvar.

For Python and other interpreters, required modules found simply looking for lines with 「import」, 「use」, 「require」, etc., are covered by the corresponding substvars.

For other programs which do not deploy their own substvars, the misc substvar covers their dependency.

For POSIX shell programs, there is no easy way to identify the dependency and no substvar covers their dependency.

For libraries and modules required via the dynamic loading mechanism including the 「GObject introspection」 mechanism, there is no easy way to identify the dependency and no substvar covers their dependency.

## 10.9. Library package

Packaging library software requires you to perform much more work than usual. Here are some reminders for packaging library software:

- The library binary package must be named as in 「「Library package name」」.
- Debian ships shared libraries such as /usr/lib/<triplet>/lib foo-0.1.so.1.0.0 (see 「「マルチアーチ」」).
- Debian encourages using versioned symbols in the shared library (see 「「Library symbols」」).
- Debian doesn’t ship *.la libtool library archive files.
- Debian discourages using and shipping *.a static library files.

Before packaging shared library software, see:

- 「Chapter 8 - Shared libraries」 of the 「Debian Policy Manual」
- 「10.2 Libraries」 of the 「Debian Policy Manual」
- 「6.7.2. Libraries」 of the 「Debian Developer’s Reference」

For the historic background study, see:

- 「Escaping the Dependency Hell」 [18]

  - This encourages having versioned symbols in the shared library.

- 「Debian Library Packaging guide」 [19]

  - Please read the discussion thread following its announcement, too.

## 10.10. マルチアーチ

Multiarch support for cross-architecture installation of binary packages (particularly i386 and amd64, but also other combinations) in the dpkg and apt packages introduced in Debian wheezy (7.0, May 2013), demands that we pay extra attention to packaging.

You should read the following references in detail.

- Ubuntu wiki (upstream)

  - 「MultiarchSpec」

- Debian wiki (Debian situation)

  - 「Debian multiarch support」
  - 「Multiarch/Implementation」

The multiarch is enabled by using the <triplet> value such as i386-linux-gnu and x86_64-linux-gnu in the install path of shared libraries as /usr/lib/<triplet>/, etc..

- The <triplet> value required internally by debhelper scripts is implicitly set in themselves. The maintainer doesn’t need to worry.

- The <triplet> value used in override_dh_* target scripts must be explicitly set in the debian/rules file by the maintainer. The <triplet> value is stored in the $(DEB_HOST_MULTIARCH) variable in the following debian/rules snippet example:

      DEB_HOST_MULTIARCH = $(shell dpkg-architecture -qDEB_HOST_MULTIARCH)
      ...
      override_dh_install:
      	mkdir -p package1/lib/$(DEB_HOST_MULTIARCH)
      	cp -dR tmp/lib/. package1/lib/$(DEB_HOST_MULTIARCH)

See:

- 「「Variables for debian/rules」」
- 「「debmake -b」」
- 「「Package split scenario and examples」」
- 「dpkg-architecture(1) manpage」

## 10.11. Split of a Debian binary package

For well behaving build systems, the split of a Debian binary package into small ones can be realized as follows.

- Create binary package entries for all binary packages in the debian/control file.
- List all file paths (relative to debian/tmp) in the corresponding debian/binarypackage.install files.

Please check examples in this guide:

- 「「Autotools (multi-binary package)」」 (Autotools based)
- 「「CMake (multi-binary package)」」 (CMake based)

An intuitive and flexible method to create the initial template debian/control file defining the split of the Debian binary packages is accommodated with the -b option. See 「「debmake -b」」.

## 10.12. Package split scenario and examples

Here are some typical multiarch package split scenarios for the following upstream source examples using the debmake command:

- a library source libfoo-1.0.tar.xz
- a tool source bar-1.0.tar.xz written in a compiled language
- a tool source baz-1.0.tar.xz written in an interpreted language

binarypackage | type | Architecture: | Multi-Arch: | パッケージ内容
--- | --- | --- | --- | ---
lib foo1 | lib* | any | same | 共有ライブラリー、同時インストール可能
lib foo-dev | dev* | any | same | 共有ライブラリーヘッダーファイル他、同時インストール可能
lib foo-tools | bin* | any | foreign | 実行時サポートプログラム、同時インストール不可
lib foo-doc | doc* | all | foreign | 共有ライブラリーのドキュメンテーションファイル
bar | bin* | any | foreign | コンパイルされ実行されるプログラムファイル、同時インストール不可
bar-doc | doc* | all | foreign | プログラムのドキュメンテーションファイル
baz | script | all | foreign | インタープリターで実行されるプログラムファイル

## 10.13. Multiarch library path

Debian policy requires to comply with the 「Filesystem Hierarchy Standard (FHS), version 3.0」, with the exceptions noted in 「File System Structure」.

The most notable exception is the use of /usr/lib/<triplet>/ instead of /usr/lib<qual>/ (e.g., /lib32/ and /lib64/) to support a multiarch library.

表10.1 The multiarch library path options

Classic path | i386 マルチアーチパス | amd64 マルチアーチパス
--- | --- | ---
/lib/ | /lib/i386-linux-gnu/ | /lib/x86_64-linux-gnu/
/usr/lib/ | /usr/lib/i386-linux-gnu/ | /usr/lib/x86_64-linux-gnu/

For Autotools based packages under the debhelper package (compat>=9), this path setting is automatically taken care by the dh_auto_configure command.

For other packages with non-supported build systems, you need to manually adjust the install path as follows.

- If 「./configure」 is used in the override_dh_auto_configure target in debian/rules, make sure to replace it with 「dh_auto_configure --」 while re-targeting the install path from /usr/lib/ to /usr/lib/$(DEB_HOST_MULTIARCH)/.
- Replace all occurrences of /usr/lib/ with /usr/lib/*/ in debian/foo.install files.

All files installed simultaneously as the multiarch package to the same file path should have exactly the same file content. You must be careful with differences generated by the data byte order and by the compression algorithm.

The shared library files in the default path /usr/lib/ and /usr/lib/<triplet>/ are loaded automatically.

For shared library files in another path, the GCC option -l must be set by the pkg-config command to make them load properly.

## 10.14. Multiarch header file path

GCC includes both /usr/include/ and /usr/include/<triplet>/ by default on the multiarch Debian system.

If the header file is not in those paths, the GCC option -I must be set by the pkg-config command to make "#include <foo.h>" work properly.

表10.2 The multiarch header file path options

Classic path | i386 マルチアーチパス | amd64 マルチアーチパス
--- | --- | ---
/usr/include/ | /usr/include/i386-linux-gnu/ | /usr/include/x86_64-linux-gnu/
/usr/include/packagename/ | /usr/include/i386-linux-gnu/packagename/ | /usr/include/x86_64-linux-gnu/packagename/
| /usr/lib/i386-linux-gnu/packagename/ | /usr/lib/x86_64-linux-gnu/packagename/

The use of the /usr/lib/<triplet>/packagename/ path for the library files allows the upstream maintainer to use the same install script for the multiatch system with /usr/lib/<triplet> and the biarch system with /usr/lib<qual>/. [20]

The use of the file path containing packagename enables having more than 2 development libraries simultaneously installed on a system.

## 10.15. Multiarch *.pc file path

The pkg-config program is used to retrieve information about installed libraries in the system. It stores its configuration parameters in the *.pc file and is used for setting the -I and -l options for GCC.

表10.3 The *.pc file path options

Classic path | i386 マルチアーチパス | amd64 マルチアーチパス
--- | --- | ---
/usr/lib/pkgconfig/ | /usr/lib/i386-linux-gnu/pkgconfig/ | /usr/lib/x86_64-linux-gnu/pkgconfig/

## 10.16. Library symbols

The symbols support in dpkg introduced in Debian lenny (5.0, May 2009) helps us to manage the backward ABI compatibility of the library package with the same package name. The DEBIAN/symbols file in the binary package provides the minimal version associated with each symbol.

An oversimplified method for the library packaging is as follows.

- Extract the old DEBIAN/symbols file of the immediate previous binary package with the 「dpkg-deb -e」 command.

  - Alternatively, the mc command may be used to extract the DEBIAN/symbols file.

- Copy it to the debian/binarypackage.symbols file.

  - If this is the first package, use an empty content file instead.

- Build the binary package.

  - If the dpkg-gensymbols command warns about some new symbols:

    - Extract the updated DEBIAN/symbols file with the 「dpkg-deb -e」 command.
    - Trim the Debian revision such as -1 in it.
    - Copy it to the debian/binarypackage.symbols file.
    - Re-build the binary package.

  - If the dpkg-gensymbols command does not warn about new symbols:

    - You are done with the library packaging.

For the details, you should read the following primary references.

- 「8.6.3 The symbols system」 of the 「Debian Policy Manual」
- 「dh_makeshlibs(1) manapage」
- 「dpkg-gensymbols(1) manapage」
- 「dpkg-shlibdeps(1) manapage」
- 「deb-symbols(5) manapage」

You should also check:

- Debian wiki: 「UsingSymbolsFiles」
- Debian wiki: 「Projects/ImprovedDpkgShlibdeps」
- Debian kde team: 「Working with symbols files」
- 「「Autotools (multi-binary package)」」
- 「「CMake (multi-binary package)」」

| ヒント
For C++ libraries and other cases where the tracking of symbols is problematic, follow 「8.6.4 The shlibs system」 of the 「Debian Policy Manual」, instead. Please make sure to erase the empty debian/binarypackage.symbols file generated by the debmake command. For this case, the DEBIAN/shlibs file is used.

## 10.17. Library package name

Let’s consider that the upstream source tarball of the lib foo library is updated from lib foo-7.0.tar.xz to lib foo-8.0.tar.xz with a new SONAME major version which affects other packages.

The binary library package must be renamed from lib foo7 to lib foo8 to keep the unstable suite system working for all dependent packages after the upload of the package based on the lib foo-8.0.tar.xz.

| 警告
If the binary library package isn’t renamed, many dependent packages in the unstable suite become broken just after the library upload even if a binNMU upload is requested. The binNMU may not happen immediately after the upload due to several reasons.

The -dev package must follow one of the following naming rules:

- Use the unversioned -dev package name: lib foo-dev

  - This is the typical one for leaf library packages.

  - Only one version of the library source package is allowed in the archive.

    - The associated library package needs to be renamed from lib foo7 to lib foo8 to prevent dependency breakage in the unstable suite during the library transition.
  - This approach should be used if the simple binNMU resolves the library dependency quickly for all affected packages. (ABI change by dropping the deprecated API while keeping the active API unchanged.)
  - This approach may still be a good idea if manual code updates, etc. can be coordinated and manageable within limited packages. (API change)

- Use the versioned -dev package names: lib foo7-dev and lib foo8-dev

  - This is typical for many major library packages.

  - Two versions of the library source packages are allowed simultaneously in the archive.

    - Make all dependent packages depend on lib foo-dev.
    - Make both lib foo7-dev and lib foo8-dev provide lib foo-dev.
    - The source package needs to be renamed as lib foo7-7.0.tar.xz and lib foo8-8.0.tar.xz respectively from lib foo-?.0.tar.xz.
    - The package specific install file path including lib foo7 and lib foo8 respectively for header files etc. needs to be chosen to make them co-installable.
  - Do not use this heavy handed approach, if possible.
  - This approach should be used if the update of multiple dependent packages require manual code updates, etc. (API change) Otherwise, the affected packages become RC buggy with FTBFS (Fails To Build From Source).

| ヒント
If the data encoding scheme changes (e.g., latin1 to utf-8), the same care as the API change needs to be taken.

See 「「Library package」」.

## 10.18. Library transition

When you package a new library package version which affects other packages, you must file a transition bug report against the release.debian.org pseudo package using the reportbug command with the ben file and wait for the approval for its upload from the Release Team.

Release team has the 「transition tracker」. See 「Transitions」.

| 注意
Please make sure to rename binary packages as in 「「Library package name」」.

## 10.19. binNMU safe

A 「binNMU」 is a binary-only non-maintainer upload performed for library transitions etc. In a binNMU upload, only the 「Architecture: any」 packages are rebuilt with a suffixed version number (e.g. version 2.3.4-3 will become 2.3.4-3+b1). The 「Architecture: all」 packages are not built.

The dependency defined in the debian/control file among binary packages from the same source package should be safe for the binNMU. This needs attention if there are both 「Architecture: any」 and 「Architecture: all」 packages involved in it.

- 「Architecture: any」 package: depends on 「Architecture: any」 foo package

  - Depends: foo (= ${binary:Version})

- 「Architecture: any」 package: depends on 「Architecture: all」 bar package

  - Depends: bar (= ${source:Version})

- 「Architecture: all」 package: depends on 「Architecture: any」 baz package

  - Depends: baz (>= ${source:Version}), baz (<< ${source:Version}.0~)

## 10.20. Debugging information

The Debian package is built with the debugging information but packaged into the binary package after stripping the debugging information as required by 「Chapter 10 - Files」 of the 「Debian Policy Manual」.

See

- 「6.7.9. Best practices for debug packages」 of the 「Debian Developer’s Reference」.
- 「18.2 Debugging Information in Separate Files」 of the 「Debugging with gdb」
- 「dh_strip(1) manapage」
- 「strip(1) manapage」
- 「readelf(1) manapage」
- 「objcopy(1) manapage」
- Debian wiki: 「DebugPackage」
- Debian wiki: 「AutomaticDebugPackages」
- Debian debian-devel post: 「Status on automatic debug packages」 (2015-08-15)

## 10.21. -dbgsym package

The debugging information is automatically packaged separately as the debug package using the dh_strip command with its default behavior. The name of such a debug package normally has the -dbgsym suffix.

- The debian/rules file shouldn’t explicitly contain dh_strip.
- Set the Build-Depends to debhelper-compat (>=14) while removing Build-Depends to debhelper in debian/control.

## 10.22. debconf

The debconf package enables us to configure packages during their installation in 2 main ways:

- non-interactively from the debian-installer pre-seeding.

- interactively from the menu interface (dialog, gnome, kde, …​)

  - the package installation: invoked by the dpkg command
  - the installed package: invoked by the dpkg-reconfigure command

All user interactions for the package installation must be handled by this debconf system using the following files.

- debian/binarypackage.config

  - This is the debconf config script used for asking any questions necessary to configure the package.

- debian/binarypackage.template

  - This is the debconf templates file used for asking any questions necessary to configure the package.

These debconf files are called by package configuration scripts in the binary Debian package

- DEBIAN/binarypackage.preinst
- DEBIAN/binarypackage.prerm
- DEBIAN/binarypackage.postinst
- DEBIAN/binarypackage.postrm

See dh_installdebconf(1), debconf(7), debconf-devel(7) and 「3.9.1 Prompting in maintainer scripts」 in the 「Debian Policy Manual」.

[18] This document was written before the introduction of the symbols file.

[19] The strong preference is to use the SONAME versioned -dev package names over the single -dev package name in 「Chapter 6. Development (-DEV) packages」, which does not seem to be shared by the former ftp-master (Steve Langasek). This document was written before the introduction of the multiarch system and the symbols file.

[20] This path is compliant with the FHS. 「Filesystem Hierarchy Standard: /usr/lib : Libraries for programming and packages」 states 「Applications may use a single subdirectory under /usr/lib. If an application uses a subdirectory, all architecture-dependent data exclusively used by the application must be placed within that subdirectory.」

第9章 More on packaging |  | 第11章 Packaging with git