GLEP 65: Post-install QA checks

QA check types

There are two kinds of QA checks defined in this specification:

1. Post-install QA checks (install-qa-check.d),
2. Post-merge QA checks (postinst-qa-check.d).

The post-install QA checks are are executed after the src_install ebuild phase finishes but before the binary package is built or the pkg_preinst phase is executed. They can use the same commands as are permitted in src_install, and access the installation image ${D} and the temporary directory ${T}.

In case of severe QA issues, the checks are allowed to alter the contents of the installation image in order to sanitize them, or call the die function to abort the build.

The post-merge QA checks are executed after the pkg_postinst ebuild phase finishes. They can use the same commands as are permitted in pkg_postinst, and access the installed system location ${ROOT} and the temporary directory ${T}.

The checks are allowed to alter the contents of the filesystem to the same degree as pkg_postinst phase is. They must not call die.

QA check file format & locations

QA checks are stored as bash scripts. The checks are identified and ordered by file name. If files with same names are present in multiple locations, the file in location with the highest priority is used.

The specification defines four sources of QA checks, listed in the order of increasing priority:

1. internal checks included in the Package Manager,
2. repository-specific QA checks,
3. package-installed QA checks,
4. sysadmin-defined QA checks.

The internal checks are stored in Package Manager-specific location and should be installed along with the Package Manager. It is recommended that only generic checks are included in the Package Manager and not checks specific to Gentoo policies, packages or eclasses included in Gentoo.

Repository-specific QA checks are included in metadata/install-qa-check.d and metadata/postinst-qa-check.d directories of a repository. For an ebuild in question, the repository containing it and its masters are traversed for QA checks, with priority decreasing with each inheritance level.

The package-installed QA checks are located in /usr/lib/install-qa-check.d and /usr/lib/postinst-qa-check.d, and are intended to be installed by packages. The sysadmin-defined QA checks are located in /usr/local/lib/install-qa-check.d and /usr/local/lib/postinst-qa-check.d.

QA check script format

QA checks are sourced as bash scripts by the Package Manager. QA scripts are run in an isolated subshell, and therefore can safely alter the environment and change the working directory. QA scripts must always end with a command terminating with a successful exit code.

Aside from the standard PMS functions, two additional commands are provided:

1. eqawarn to output QA warnings to user,
2. eqatag to store machine-readable information about QA issues.

Repository-defined QA checks are allowed to inherit eclasses from the repository providing the check or any of its masters. The same inheritance rules apply as to ebuilds in the particular repository. Sourced eclasses do not affect the final ebuild metadata or phase functions.

Function specification

QA check types

The two types of QA checks were created to account for different kinds of common mistakes in ebuilds.

Post-install QA checks can be used to verify the installation image before it is merged to a live system or published as a binary package. They can account for various problems caused by the ebuild code up to and including src_install, the upstream code executed as part of any of those phases and the supplied files.

Post-merge QA checks can be used to verify the state of system after the package is merged and its pkg_postinst phase is executed. They mostly aim to detect missing postinst actions but can do other live system integrity checks.

QA check file format & locations

The multiple locations for QA checks aim to get the best coverage for various requirements.

The checks installed along with the Package Manager are meant to cover the generic cases and other checks that rely on Package Manager internals. Unlike other categories of QA checks, those checks apply to a single Package Manager only and can therefore use internal API. However, it is recommended that this category is used scarcely.

Storing checks in the repository allows developers to strictly bind them to a specific version of the distribution and update them along with the relevant policies and/or eclasses. In particular, rules enforced by Gentoo policies and eclasses don't have to apply to other distributions using Portage.

The QA checks are applied to sub-repositories (via masters attribute) likewise eclasses. This makes sure that the majority of repositories don't lose QA checks. The QA checks related to eclasses are inherited the same way as eclasses are. Similarly to eclasses, sub-repositories can override (or disable) QA checks.

System-wide QA checks present the opportunity of installing QA checks along with packages. In the past, some QA checks were run only conditionally depending on existence of external checker software. Instead, the software packages can install their own QA checks directly.

The administrative override via /usr/local is a natural extension of system-wide QA checks. Additionally, it can be used by the sysadmin to override or disable practically any other QA check, either internal Portage or repository-wide.

Sharing the QA checks has the additional advantage of having unified QA tools for all Package Managers.

QA check script format

Use of bash is aimed to match the ebuild format. The choice of functions aims at portability between Package Managers.

The scripts are run in isolated subshell to simplify the checks and reduce the risk of accidental cross-script issues.

The script need to end with a successful command as a result of bash limitation:

source foo || die "source failed"

The source call either returns the exit code of last command in the script or unsuccessful exit code in case of sourcing error. In order to distinguish between the two, we need to guarantee that the script always returns successfully.

The extra eqawarn log function aims to provide the user with distinction between important user-directed warnings and developer-oriented QA issues. The eqatag function aims to store check results in a machine-readable format for further processing.

Inheriting eclasses makes it possible to reuse code and improve maintainability. The possibility is mostly intended for eclass-specific checks that may want to e.g. obtain search paths from the eclass.

Inheriting is allowed only in repository-specific since it is the only location where availability of eclasses can be assumed. For system-wide checks, we can't assume that the source repository will be available when ebuild in question is processed.

Function specification

eqawarn "The following files are frobnicated incorrectly:"
eqawarn
eqatag -v frobnicate "${files[@]}"
eqawarn
eqawarn "Please consult http://example.com/frobnicate for more details."

* The following files are frobnicated incorrectly:
*
*   /usr/bin/frobnicatee
*   /usr/bin/other-frobnicatee
*
* Please consult http://example.com/frobnicate for more details.