# mini_buildd.sbuild module

Attention

sbuild (retry-workaround in `gnupg.py`): Spurious build failures `can't connect to the agent: IPC connect call failed` (Debian Bug #849551)

Attention

sbuild: Fails to build “all” packages with “build archall” flag set to arch “x” in case DSP has >= 1 arch “all” and >=1 arch “y” binary package (Debian Bug #706086)

This is due to sbuild and in in more detail explained in the Debian bug report.

A bad one-package workaround would be to change the “build archall” flag to arch “y”.

Attention

sbuild (workaround in `apt-update` snippet): `apt-get update` “File has unexpected size … Mirror sync in progress?” failures

This may happen if by chance there are reprepro updates around the same time `apt-get update` runs in the build.

You may reproduce this quite reliably with the testsuite (it’s due to the immediate parent package remove, see `./devel`):

1. testsuite: Change `installed-port` test to (also) port for running codename (`changelog` + `./devel`)

2. ./devel updatetestall

3. ./devel testsuite-packages installed-port

Internal sbuild snippet works around this problem by retrying `apt-get update` – but you still see the error lines in the buildlog if that happens.

Solution could be to lock HTTP delivery on any reprepro change, but for now this seems overkill to me.

Attention

sbuild+apt-cacher-ng (workaround in `apt-update` snippet): `apt-get update` stalls on parallel access

Phenomena: A build just always remains ongoing && the buildlog just stalls during `apt-get update`.

Repeat bug:

/usr/share/doc/mini-buildd/examples/apt-parallel-bug

    #!/bin/bash -e
    # Precondition: Have ``apt-cacher-ng`` installed, default config

    # Phaenomena description: Running this (on a fairly fast and parallel host) yields to ``apt-get update`` to stall
    #
    # If that happens, ``ps aux | grep method`` will then show existing/standing ``/usr/lib/apt/methods/http``, ``/usr/lib/apt/methods/gpgv`` processes
    # Killing any of these will make apt continue (with an error)

    # Note: Acquire::http::Pipeline-Depth and Acquire::QueueMode don't seem to change anything
    # Note: Using ``parallel --jobs=1`` remedies the bug

    read -p "Will create chroots in $(pwd) and do other dirty sudo thing (Ctl-C to abort)"

    # At least two working apt lines are needed to trigger bug
    APT_SOURCES_LIST="\
    deb [] http://localhost:3142/debian sid main contrib non-free non-free-firmware
    deb [] http://localhost:3142/debian experimental main contrib non-free non-free-firmware
    "

    CHROOTS="sid0 sid1"

    for C in ${CHROOTS}; do
    	[ -e "${C}" ] || sudo debootstrap sid "${C}" http://localhost:3142/debian/

    	# Configure sources.list for local apt-cacher-ng with at least two workng lines
    	printf "%s" "${APT_SOURCES_LIST}" | sudo tee ./${C}/etc/apt/sources.list

    	# Remove any already download source.list files, so proper HTTP download is triggered
    	sudo rm -rf "./${C}/var/lib/apt/lists/"
    done

    # Run apt-get in parallel
    seq 0 1 | parallel sudo chroot ./sid{}/ timeout --verbose 10s apt-get update

Remedy situation at hand:

1. Kill corresponding process `/usr/lib/apt/methods/gpgv` or `/usr/lib/apt/methods/http` (apt-get update may be retried and build continues)

2. Cancel resp. build(s) (then retry)

Related 3rd party findings:

> - https://bugs.launchpad.net/ubuntu/+source/apt-cacher-ng/+bug/1983856

> - https://bugs.launchpad.net/ubuntu/+source/apt/+bug/2003851

mini_buildd.sbuild.CONFIG_DIR = '.config'

  Build dir constants.

mini_buildd.sbuild.APT_GET = 'apt-get --quiet --yes --option=APT::Install-Recommends=false --option=Acquire::Languages=none --option=Dpkg::Use-Pty=false --option=Dpkg::Options::=--force-confdef --option=Dpkg::Options::=--force-confnew'

  Quiet, non-interactive, least invasive and loggable apt-get call (Dpkg::Use-Pty=false is to avoid https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539617).

mini_buildd.sbuild.BASH_VARS = 'MBD_SCRIPT="$(realpath "${0}")" # Note: Older versions of realpath (<=wheezy) did not support options, just avoid them\nMBD_BASE="${MBD_SCRIPT%/*/*}"'

  Safe canonized paths

class mini_buildd.sbuild.Blocks(type_)

  Bases: `Dir`

  usage()
  validate(value)

    Validate value from user space (string, space separated), return value as list

  line(value)

class mini_buildd.sbuild.ConfigBlocks

  Bases: `Blocks`

  configure(file_, blocks)

    Add configured blocks to provided file

class mini_buildd.sbuild.SetupBlocks

  Bases: `Blocks`

  configure(dir_, blocks)

    Add configured blocks to provided dir

class mini_buildd.sbuild.SBuild(breq, extra_args)

  Bases: `object`

  BUILDLOG_STATUS_REGEX = re.compile('^(Status|Lintian|Piuparts|Autopkgtest|Build-Time|Package-Time|Space|Build-Space): [^ ]+.*$')
  run()

    Run sbuild && update buildresult from build log

    Note

    This will iterate all lines of the build log, and parse out the selection of sbuild’s summary status we need. In case the build log above does write the same output like ‘Status: xyz’, sbuild’s correct status at the bottom will override this later. Best thing, though, would be if sbuild would eventually provide a better way to get these values.

  cancel(who)