USCAN(1)

NAME
uscan - scan/watch upstream sources for new releases of software

SYNOPSIS
uscan [options] [path]

DESCRIPTION
For basic usage, uscan is executed without any arguments from the root of the Debianized source tree where you see the debian/ directory, or a directory containing multiple source trees.

Unless --watchfile is given, uscan looks recursively for valid source trees starting from the current directory (see the below section "Directory name checking" for details).

For each valid source tree found, typically the following happens:

•   uscan reads the first entry in debian/changelog to determine the source package name <spkg> and the last upstream version.

•   uscan process the watch lines debian/watch from the top to the bottom in a single pass.

    •   uscan downloads a web page from the specified URL in debian/watch.

    •   uscan extracts hrefs pointing to the upstream tarball(s) from the web page using the specified matching-pattern in debian/watch.

    •   uscan downloads the upstream tarball with the highest version newer than the last upstream version.

    •   uscan saves the downloaded tarball to the parent ../ directory: ../<upkg>-<uversion>.tar.gz

    •   uscan invokes mk-origtargz to create the source tarball: ../<spkg>_<oversion>.orig.tar.gz

        •   For a multiple upstream tarball (MUT) package, the secondary upstream tarball will instead be named ../<spkg>_<oversion>.orig-<component>.tar.gz.

    •   Repeat until all lines in debian/watch are processed.

•   If requested, uscan invokes an update script. This may be uupdate to create the Debianized source tree ../<spkg>-<oversion>/* or a custom script.

Please note the following.

•   For simplicity, the compression method used in examples is gzip with .gz suffix. Other methods such as xz, bzip2, and lzma with corresponding xz, bz2 and lzma suffixes may also be used.

•   Since version 4 of debian/watch, uscan enables handling of multiple upstream tarball (MUT) packages but this is a rare case for Debian packaging. For a single upstream tarball package, there is only one watch line and no ../<spkg>_<oversion>.orig-<component>.tar.gz.

•   uscan with the --verbose option produces a human readable report of uscan's execution.

•   uscan with the --debug option produces a human readable report of uscan's execution including internal variable states.

•   uscan with the --extra-debug option produces a human readable report of uscan's execution including internal variable states and remote content during "search" step.

•   uscan with the --dehs option produces an upstream package status report in XML format for other programs such as the Debian External Health System.

•   The primary objective of uscan is to help identify if the latest version upstream tarball is used or not; and to download the latest upstream tarball. The ordering of versions is decided by dpkg --compare-versions.

•   uscan with the --safe option limits the functionality of uscan to its primary objective. Both the repacking of downloaded files and updating of the source tree are skipped to avoid running unsafe scripts. This also changes the default to --no-download and --skip-signature.

FORMAT OF THE WATCH FILE
The current debian/watch format is described in debian-watch(5) manpage. Old formats (version 1 to 4) are described in debian-watch-4(5) manpage.

To convert a version <= 4 file into the last version, simply use:

  uscan --update-watchfile

and then check the result.

COPYRIGHT FILE EXAMPLES
Here is an example for the debian/copyright file which initiates automatic repackaging of the upstream tarball into <spkg>_<oversion>.orig.tar.gz (In debian/copyright, the Files-Excluded and Files-Excluded-component stanzas are a part of the first paragraph and there is a blank line before the following paragraphs which contain Files and other stanzas.):

  Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
  Files-Excluded: exclude-this
   exclude-dir
   */exclude-dir
   .*
   */js/jquery.js

  Files: *
  Copyright: ...
  ...

Here is another example for the debian/copyright file which initiates automatic repackaging of the multiple upstream tarballs into <spkg>_<oversion>.orig.tar.gz and <spkg>_<oversion>.orig-bar.tar.gz:

  Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
  Files-Excluded: exclude-this
   exclude-dir
   */exclude-dir
   .*
   */js/jquery.js
  Files-Excluded-bar: exclude-this
   exclude-dir
   */exclude-dir
   .*
   */js/jquery.js

  Files: *
  Copyright: ...
  ...

The debian/copyright file may also contain Files-Included and Files-Included-component stanzas which include files that were previously excluded. This is useful to exclude most but not all files in a directory:

  Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
  Files-Excluded: vendor-dir
  Files-Included:
   vendor-dir/directory/to/keep
   vendor-dir/*/file-to-keep

  Files: *
  Copyright: ...
  ...

See mk-origtargz(1).

KEYRING FILE EXAMPLES
Let's assume that the upstream "uscan test key (no secret) <none@debian.org>" signs its package with a secret OpenPGP key and publishes the corresponding public OpenPGP key. This public OpenPGP key can be identified in 3 ways using the hexadecimal form.

•   The fingerprint as the 20 byte data calculated from the public OpenPGP key. E. g., 'CF21 8F0E 7EAB F584 B7E2 0402 C77E 2D68 7254 3FAF'

•   The long keyid as the last 8 byte data of the fingerprint. E. g., 'C77E2D6872543FAF'

•   The short keyid is the last 4 byte data of the fingerprint. E. g., '72543FAF'

Considering the existence of the collision attack on the short keyid, the use of the long keyid is recommended for receiving keys from the public key servers. You must verify the downloaded OpenPGP key using its full fingerprint value which you know is the trusted one.

The armored keyring file debian/upstream/signing-key.asc can be created by using the gpg command as follows.

  $ gpg --recv-keys "C77E2D6872543FAF"
  ...
  $ gpg --finger "C77E2D6872543FAF"
  pub   4096R/72543FAF 2015-09-02
        Key fingerprint = CF21 8F0E 7EAB F584 B7E2  0402 C77E 2D68 7254 3FAF
  uid                  uscan test key (no secret) <none@debian.org>
  sub   4096R/52C6ED39 2015-09-02
  $ cd path/to/<upkg>-<uversion>
  $ mkdir -p debian/upstream
  $ gpg --export --export-options export-minimal --armor \
        'CF21 8F0E 7EAB F584 B7E2  0402 C77E 2D68 7254 3FAF' \
        >debian/upstream/signing-key.asc

The binary keyring files, debian/upstream/signing-key.pgp and debian/upstream-signing-key.pgp, are still supported but deprecated.

If a group of developers sign the package, you need to list fingerprints of all of them in the argument for gpg --export ... to make the keyring to contain all OpenPGP keys of them.

Sometimes you may wonder who made a signature file. You can get the public keyid used to create the detached signature file foo-2.0.tar.gz.asc by running gpg as:

  $ gpg -vv foo-2.0.tar.gz.asc
  gpg: armor: BEGIN PGP SIGNATURE
  gpg: armor header: Version: GnuPG v1
  :signature packet: algo 1, keyid C77E2D6872543FAF
        version 4, created 1445177469, md5len 0, sigclass 0x00
        digest algo 2, begin of digest 7a c7
        hashed subpkt 2 len 4 (sig created 2015-10-18)
        subpkt 16 len 8 (issuer key ID C77E2D6872543FAF)
        data: [4091 bits]
  gpg: assuming signed data in `foo-2.0.tar.gz'
  gpg: Signature made Sun 18 Oct 2015 11:11:09 PM JST using RSA key ID 72543FAF
  ...

COMMANDLINE OPTIONS
For the basic usage, uscan does not require to set these options.

--conffile, --conf-file
    Add or replace default configuration files ("/etc/devscripts.conf" and "~/.devscripts"). This can only be used as the first option given on the command-line.

    replace:
          uscan --conf-file test.conf --verbose

    add:
          uscan --conf-file +test.conf --verbose

        If one --conf-file has no "+", default configuration files are ignored.

--no-conf, --noconf
    Don't read any configuration files. This can only be used as the first option given on the command-line.

--no-verbose
    Don't report verbose information. (default)

--verbose, -v
    Report verbose information.

--debug, -vv
    Report verbose information and some internal state values.

--extra-debug, -vvv
    Report verbose information including the downloaded web pages as processed to STDERR for debugging.

--dehs
    Send DEHS style output (XML-type) to STDOUT, while send all other uscan output to STDERR.

--no-dehs
    Use only traditional uscan output format. (default)

--download, -d
    Download the new upstream release. (default)

--force-download, -dd
    Download the new upstream release even if up-to-date. (may not overwrite the local file)

--overwrite-download, -ddd
    Download the new upstream release even if up-to-date. (may overwrite the local file)

--no-download, --nodownload
    Don't download and report information.

    Previously downloaded tarballs may be used.

    Change default to --skip-signature.

--signature
    Download signature. (default)

--no-signature
    Don't download signature but verify if already downloaded.

--skip-signature
    Don't bother download signature nor verifying signature.

--safe, --report
    Avoid running unsafe scripts by skipping both the repacking of the downloaded package and the updating of the new source tree.

    Change default to --no-download and --skip-signature.

    When the objective of running uscan is to gather the upstream package status under the security conscious environment, please make sure to use this option.

--report-status
    This is equivalent of setting "--verbose --safe".

--download-version version
    Specify the version which the upstream release must match in order to be considered, rather than using the release with the highest version. (a best effort feature)

--download-debversion version
    Specify the Debian package version to download the corresponding upstream release version. The dversionmangle and uversionmangle rules are considered. (a best effort feature)

--download-current-version
    Download the currently packaged version. (a best effort feature)

--check-dirname-level N
    See the below section "Directory name checking" for an explanation of this option.

--check-dirname-regex regex
    See the below section "Directory name checking" for an explanation of this option.

--destdir path Normally, uscan changes its internal current directory to the package's source directory where the debian/ is located.  Then the destination directory for the downloaded tarball and other files is set to the parent directory ../ from this internal current directory.
    This default destination directory can be overridden by setting --destdir option to a particular path. If this path is a relative path, the destination directory is determined in relative to the internal current directory of uscan execution. If this path is a absolute path, the destination directory is set to path irrespective of the internal current directory of uscan execution.

    The above is true not only for the simple uscan run in the single source tree but also for the advanced scanning uscan run with subdirectories holding multiple source trees.

    One exception is when --watchfile and --package are used together. For this case, the internal current directory of uscan execution and the default destination directory are set to the current directory . where uscan is started. The default destination directory can be overridden by setting --destdir option as well.

--package package
    Specify the name of the package to check for rather than examining debian/changelog; this requires the --upstream-version (unless a version is specified in the watch file) and --watchfile options as well. Furthermore, no directory scanning will be done and nothing will be downloaded. This option automatically sets --no-download and --skip-signature; and probably most useful in conjunction with the DEHS system (and --dehs).

--upstream-version upstream-version
    Specify the current upstream version rather than examine debian/watch or debian/changelog to determine it. This is ignored if a directory scan is being performed and more than one debian/watch file is found.

--vcs-export-uncompressed
    Disable compression of tarballs exported from a version control system (Git or Subversion). This takes more space, but saves time if mk-origtargz must repack the tarball to exclude files. It forces repacking of all exported tarballs.

--watchfile watchfile
    Specify the watchfile rather than perform a directory scan to determine it. If this option is used without --package, then uscan must be called from within the Debian package source tree (so that debian/changelog can be found simply by stepping up through the tree).

    One exception is when --watchfile and --package are used together. uscan can be called from anywhare and the internal current directory of uscan execution and the default destination directory are set to the current directory . where uscan is started.

    See more in the --destdir explanation.

--bare
    Disable all site specific special case codes to perform URL redirections and page content alterations.

--http-header
    Add specified header in HTTP requests for matching url. This option can be used more than one time, values must be in the form "baseUrl@Name=value. Example:

      uscan --http-header https://example.org@My-Token=qwertyuiop

    Security:

    The given baseUrl must exactly match the base url before '/'. Examples:
          |        --http-header value         |           Good for          | Never used |
          +------------------------------------+-----------------------------+------------+
          | https://example.org.com@Hdr=Value  | https://example.org.com/... |            |
          | https://example.org.com/@Hdr=Value |                             |     X      |
          | https://e.com:1879@Hdr=Value       | https://e.com:1879/...      |            |
          | https://e.com:1879/dir@Hdr=Value   | https://e.com:1879/dir/...  |            |
          | https://e.com:1879/dir/@Hdr=Value  |                             |     X      |

    It is strongly recommended to not use this feature to pass a secret token over unciphered connection (http://)
    You can use "USCAN_HTTP_HEADER" variable (in "~/.devscripts") to hide secret token from scripts
--no-exclusion
    Don't automatically exclude files mentioned in debian/copyright field Files-Excluded.

--no-symlink
    Don't rename nor repack upstream tarball.

--timeout N
    Set timeout to N seconds (default 20 seconds).

--user-agent, --useragent
    Override the default user agent header.

--help
    Give brief usage information.

--version
    Display version information.

uscan also accepts following options and passes them to mk-origtargz:

--symlink
    Make orig.tar.gz (with the appropriate extension) symlink to the downloaded files. (This is the default behavior.)

--copy
    Instead of symlinking as described above, copy the downloaded files.

--rename
    Instead of symlinking as described above, rename the downloaded files.

--repack
    After having downloaded an lzma tar, xz tar, bzip tar, gz tar, lz tar, zip, jar, xpi, zstd archive, repack it to the specified compression (see --compression).

    The unzip package must be installed in order to repack zip, jar, and xpi archives, the xz-utils package must be installed to repack lzma or xz tar archives, zstd must be installed to repack zstd archives, and lzip must be installed to repack lz tar archives.

--compression [ gzip | bzip2 | lzma | xz ]
    In the case where the upstream sources are repacked (either because --repack option is given or debian/copyright contains the field Files-Excluded), it is possible to control the compression method via the parameter. The default is gzip for normal tarballs, and xz for tarballs generated directly from the git repository.

--copyright-file copyright-file
    Exclude files mentioned in Files-Excluded in the given copyright-file. This is useful when running uscan not within a source package directory.

DEVSCRIPT CONFIGURATION VARIABLES
For the basic usage, uscan does not require to set these configuration variables.

The two configuration files /etc/devscripts.conf and ~/.devscripts are sourced by a shell in that order to set configuration variables. These may be overridden by command line options. Environment variable settings are ignored for this purpose. If the first command line option given is --noconf, then these files will not be read. The currently recognized variables are:

USCAN_DOWNLOAD
    Download or report only:

    no: equivalent to --no-download, newer upstream files will not be downloaded.
    yes: equivalent to --download, newer upstream files will be downloaded. This is the default behavior.
        See also --force-download and --overwrite-download.

USCAN_SAFE
    If this is set to yes, then uscan avoids running unsafe scripts by skipping both the repacking of the downloaded package and the updating of the new source tree; this is equivalent to the --safe options; this also sets the default to --no-download and --skip-signature.

USCAN_TIMEOUT
    If set to a number N, then set the timeout to N seconds. This is equivalent to the --timeout option.

USCAN_SYMLINK
    If this is set to no, then a pkg_version.orig.tar.{gz|bz2|lzma|xz} symlink will not be made (equivalent to the --no-symlink option). If it is set to yes or symlink, then the symlinks will be made. If it is set to rename, then the files are renamed (equivalent to the --rename option).

USCAN_DEHS_OUTPUT
    If this is set to yes, then DEHS-style output will be used. This is equivalent to the --dehs option.

USCAN_VERBOSE
    If this is set to yes, then verbose output will be given. This is equivalent to the --verbose option.

USCAN_USER_AGENT
    If set, the specified user agent string will be used in place of the default. This is equivalent to the --user-agent option.

USCAN_DESTDIR
    If set, the downloaded files will be placed in this directory. This is equivalent to the --destdir option.

USCAN_REPACK
    If this is set to yes, then after having downloaded a bzip tar, lzma tar, xz tar, zip or zstd archive, uscan will repack it to the specified compression (see --compression). This is equivalent to the --repack option.

USCAN_EXCLUSION
    If this is set to no, files mentioned in the field Files-Excluded of debian/copyright will be ignored and no exclusion of files will be tried. This is equivalent to the --no-exclusion option.

USCAN_HTTP_HEADER
    If set, the specified http header will be used if URL match. This is equivalent to --http-header option.

USCAN_VCS_EXPORT_UNCOMPRESSED
    If this is set to yes, tarballs exported from a version control system will not be compressed. This is equivalent to the --vcs-export-uncompressed option.

EXIT STATUS
The exit status gives some indication of whether a newer version was found or not; one is advised to read the output to determine exactly what happened and whether there were any warnings to be noted.

0   Either --help or --version was used, or for some watch file which was examined, a newer upstream version was located.

1   No newer upstream versions were located for any of the watch files examined.

ADVANCED FEATURES
uscan has many other enhanced features which are skipped in the above section for the simplicity. Let's check their highlights.

uscan can be executed with path as its argument to change the starting directory of search from the current directory to path .

If you are not sure what exactly is happening behind the scene, please enable the --verbose option. If this is not enough, enable the --debug option too see all the internal activities.

See "COMMANDLINE OPTIONS" and "DEVSCRIPT CONFIGURATION VARIABLES" for other variations.

  Custom script
After processing other stuff uscan will run script if requested in debian/watch. Otherwise there will be no extra action.

A typical script is uupdate for non-native packages, but a custom script may be used. For compatibility with other tools such as git-buildpackage, it may not be wise to create custom scripts with random behavior. Custom scripts, if created, should behave as if uupdate. For possible use case, see <http://bugs.debian.org/748474> as an example.

For info on how to request this script, please see debian-watch(5) for recommended version 5 format or debian-watch-4(5) for old version 4 format.

See "HISTORY AND UPGRADING" for how uscan invokes a custom script (or uupdate).

  URL diversion
Some popular web sites changed their web page structure causing maintenance problems to the watch file. There are some redirection services created to ease maintenance of the watch file. Currently, uscan makes automatic diversion of URL requests to the following URLs to cope with this situation.

•   <http://sf.net>

•   <http://pypi.python.org>

  Directory name checking
Similarly to several other scripts in the devscripts package, uscan explores the requested directory trees looking for debian/changelog and debian/watch files. As a safeguard against stray files causing potential problems, and in order to promote efficiency, it will examine the name of the parent directory once it finds the debian/changelog file, and check that the directory name corresponds to the package name. It will only attempt to download newer versions of the package and then perform any requested action if the directory name matches the package name. Precisely how it does this is controlled by two configuration file variables DEVSCRIPTS_CHECK_DIRNAME_LEVEL and DEVSCRIPTS_CHECK_DIRNAME_REGEX, and their corresponding command-line options --check-dirname-level and --check-dirname-regex.

DEVSCRIPTS_CHECK_DIRNAME_LEVEL can take the following values:

0   Never check the directory name.

1   Only check the directory name if we have had to change directory in our search for debian/changelog, that is, the directory containing debian/changelog is not the directory from which uscan was invoked. This is the default behavior.

2   Always check the directory name.

The directory name is checked by testing whether the current directory name (as determined by pwd(1)) matches the regex given by the configuration file option DEVSCRIPTS_CHECK_DIRNAME_REGEX or by the command line option --check-dirname-regex regex. Here regex is a Perl regex (see perlre(3perl)), which will be anchored at the beginning and the end. If regex contains a /, then it must match the full directory path. If not, then it must match the full directory name. If regex contains the string package, this will be replaced by the source package name, as determined from the debian/changelog. The default value for the regex is: package(-.+)?, thus matching directory names such as package and package-version.

HISTORY AND UPGRADING
This section briefly describes the backwards-incompatible watch file features which have been added in each watch file version, and the first version of the devscripts package which understood them.

Pre-version 2
    The watch file syntax was significantly different in those days. Don't use it. If you are upgrading from a pre-version 2 watch file, you are advised to read this manpage and to start from scratch.

Version 2
    devscripts version 2.6.90: The first incarnation of the current style of watch files. This version is also deprecated and will be rejected after the Debian 11 release.

Version 3
    devscripts version 2.8.12: Introduced the following: correct handling of regex special characters in the path part, directory/path pattern matching, version number in several parts, version number mangling. Later versions have also introduced URL mangling.

    If you are upgrading from version 2, the key incompatibility is if you have multiple groups in the pattern part; whereas only the first one would be used in version 2, they will all be used in version 3. To avoid this behavior, change the non-version-number groups to be (?: ... ) instead of a plain ( ... ) group.

    •   uscan invokes the custom script as "script --upstream-version version ../spkg_version.orig.tar.gz".

    •   uscan invokes the standard uupdate as "uupdate --no-symlink --upstream-version version ../spkg_version.orig.tar.gz".

Version 4
    devscripts version 2.15.10: The first incarnation of watch files supporting multiple upstream tarballs.

    The syntax of the watch file is relaxed to allow more spaces for readability.

    If you have a custom script in place of uupdate, you may also encounter problems updating from Version 3.

    •   uscan invokes the custom script as "script --upstream-version version".

    •   uscan invokes the standard uupdate as "uupdate --find --upstream-version version".

    Restriction for --dehs is lifted by redirecting other output to STDERR when it is activated.

SEE ALSO
dpkg(1), mk-origtargz(1), perlre(1), uupdate(1), devscripts.conf(5), debian-watch(5), debian-watch-4(5)

AUTHOR
The original version of uscan was written by Christoph Lameter <clameter@debian.org>. Significant improvements, changes and bugfixes were made by Julian Gilbey <jdg@debian.org>. HTTP support was added by Piotr Roszatycki <dexter@debian.org>. The program was rewritten in Perl by Julian Gilbey. Xavier Guimard converted it in object-oriented Perl using Moo.