package_publish

# Workflow `package_publish`

This workflow publishes source and/or binary packages to a given target suite. It is normally expected to be used as a sub-workflow.

## Permission considerations

Copying artifacts requires both the ability to read from the source and the ability to write to the destination (either directly or via a workflow).

After artifacts have been made public, it’s helpful to be able to see the work request that created them, without having to somehow also copy the work request around. To achieve this, the permission predicate that checks whether a user can see a work request may check whether any of the artifacts produced by the work request are visible to that user, and return True in that case even if the work request itself would not ordinarily be visible.

Note

It may be surprising that this rule is “any of the artifacts produced by the work request” rather than “all of the artifacts produced by the work request”; but there isn’t usually anywhere useful to copy debusine:work-request-debug-logs artifacts to, and making only some of the artifacts produced by a work request public seems unlikely to be a realistic unembargoing use case.

While build logs may expose additional information not in the output artifacts (such as build-dependencies where security updates are also being prepared), similar information might easily be exposed by the output artifacts themselves anyway, so the onus is on people who make artifacts public to check that it is safe to do so.

Todo

Permission checks for this workflow are not yet implemented.

## Resource accounting considerations

We want to be able to track the resource usage of workspaces and scopes. If artifacts are copied between workspaces (and hence perhaps between scopes), then the same files may exist in multiple workspaces, complicating this kind of analysis. The question is likely to be something along the lines of “how much data does Debusine need to store on behalf of this workspace or scope that it would not otherwise need to store?”.

A reasonable first cut would be to track the origin of copies, and to account an artifact’s files to a workspace (and its containing scope) if the artifact is in that workspace and is no longer in its origin workspace. We therefore add a nullable `Artifact.original_artifact` foreign key, with `on_delete=SET_NULL`.

Some other variations are possible, and are not made more difficult by this design. For example, we may wish to account for each workspace’s usage without considering whether files have been copied from or to other workspaces (in which case the total file store size may be less than the sum of the sizes of all workspaces); or to calculate the “unique” size of a workspace as the total size of all files that appear only in that workspace.

## Workflow definition

- `task_data`:

  - `source_artifact` (Single lookup, optional): a debian:source-package or debian:upload artifact representing the source package (the former is used when the workflow is started based on a `.dsc` rather than a `.changes`)

  - `binary_artifacts` (Multiple lookup, optional): a list of debian:upload artifacts representing the binary packages

  - `target_suite` (Single lookup, required): the debian:suite collection to publish packages to; this must be in the same workspace as the workflow

  - `unembargo` (boolean, defaults to False): if True, allow publishing artifacts from private workspaces to public suites

  - `replace` (boolean, defaults to False): if True, replace existing similar items

  - `suite_variables` (dictionary, optional): pass these variables when adding items to the target suite collection; see debian:suite for the available variable names

  - `update_indexes` (boolean, defaults to True): if True, update indexes in the target suite after copying items

At least one of `source_artifact` and `binary_artifacts` must be set.

The workflow computes dynamic metadata as:

- subject: source package names of `binary_artifacts` separated by spaces

- parameter_summary: source package names of `binary_artifacts` separated by spaces → suite

- source_artifact_id: id of the source artifact

- binary_artifacts_ids: ids of the binary artifacts

- target_suite_id: collection id of the target_suite

The `component`, `section`, and (for binary packages) `priority` variables need to be set when adding packages to the target suite. They are determined as follows:

- default to the values from the artifact:

  - for a debian:upload source artifact, use the component and section listed for the `.dsc` file in the `.changes` file

  - for a debian:source-package source artifact, then its component and section defaults are unavailable without unpacking the source package, so set the `component` to `main` and the `section` to `misc`

  - for a binary artifact, use the Section field in the `.deb` to set the component and section and the Priority field to set the priority, falling back to `main`, `misc`, and `optional` respectively

- merge in the values from `suite_variables`, which take priority over the defaults above

The workflow creates a CopyCollectionItems task. The `copies` field in its task data is as follows:

- `source_items`: the union of whichever of `{source_artifact}` and `{binary_artifacts}` are set

- `target_collection`: `{target_suite}`

- `unembargo`: `{unembargo}`

- `replace`: `{replace}`

- `variables`: `{suite_variables}`

Any of the lookups in `source_items` may result in promises, and in that case the workflow adds corresponding dependencies.

If `binary_artifacts` is set and the source and target workspaces have different instances of the debian:package-build-logs collection, then the workflow also adds an entry to `copies` as follows:

- `source_items`:

      collection: {source build logs collection}
      lookup__same_work_request: {binary_artifacts}

- `target_collection`: target build logs collection

- `unembargo`: `{unembargo}`

- `replace`: `{replace}`

If `binary_artifacts` is set and the source and target workspaces have different instances of the debusine:task-history collection, then the workflow also adds an entry to `copies` as follows:

- `source_items`:

      collection: {source task history collection}
      lookup__same_workflow: {binary_artifacts}

- `target_collection`: target task history collection

- `unembargo`: `{unembargo}`

- `replace`: `{replace}`

If `update_indexes` is True, then once the CopyCollectionItems task has completed, the workflow triggers an update_suites workflow in the target workspace, with `only_suites` set to a list containing the name of the target suite.