bob-archive - Manage binary artifacts archive
Generic command format:
bob archive [-h] subcommand ...
bob archive clean [-h] [--dry-run] [-n] [-v] [-f] expression [expression ...] bob archive find [-h] [-n] [-v] [-f] expression [expression ...] bob archive scan [-h] [-v] [-f]
The bob archive command can be used to manage local binary artifact archives. The command must be executed in the root of the archive and needs write access to create an index cache.
Artifacts are managed by the information included in their Audit Trail. See the Audit Trail documentation for a detailed description of
the included data. Currently the
bob archive command has access to the
metaEnv sections of the audit trail:
build.date: The date and time of the build (UTC, ISO 8601), e.g. “2019-12-02T13:19:34.193136+00:00”.
build.machine: The hardware identifier as returned by the uname system call.
build.nodename: The host name.
build.os-release: Content of
/etc/os-release, if existing.
build.release: The operating system release.
build.sysname: The operating system name (e.g. “Linux”).
build.version: The operating system version.
meta.bob: Bob version string.
$BUILD_TAG(only present on Jenkins builds).
$BUILD_URL(only present on Jenkins builds).
$NODE_NAME(only present on Jenkins builds).
meta.language: “bash” / “PowerShell”
meta.package: Package path of the artifact that was built.
meta.recipe: Name of the recipe that declared the package.
meta.step: “src” / “build” / “dist”
metaEnv.<VAR>: Value of metaEnvironment variable
Be careful when matching by
meta.package. The retention expression (see
clean command below) has to match an actually present artifact. There
may be more than one possible path trough the dependency tree to the same
package. It is also possible that multiple packages produce the identical
result. Only one such package will usually be built by Bob. None of these
alternate possible package paths are recorded so you should double check if
you query actually maches.
- Do not actually delete any artifacts but show what would get removed.
- Don’t rescan the archive for new artifacts. The command will work on the last scanned data. Useful if the scan takes a long time (e.g. big archive on network mount) and was already run recently.
- Be a bit more chatty on what is done.
- Return a non-zero exit code in case of errors
Remove unneeded artifacts from the archive.
The command takes one or more retention expressions. Any artifact that is matched by at least one of the expressions or referenced transitively by a matched artifact is kept. If an artifact is neither matched by any expression nor referenced by a retained artifact it is deleted.
The expression language has the following general syntax:Predicate [
ORDER BYField [
The Predicate supports the following constructs:
- Strings are written with double quotes, e.g.
"foo". To embed double quotes in the string itself escape them with
- Certain fields from the audit trail can be accessed by their name.
Sub-fields are specified with a dot operator, e.g.
meta.package. All fields are case sensitive and of string type. Referencing a non-existing field is supported but will yield a distinct “undefined” value. This special value can only be compared with
!=with other values.
- Strings and fields can be compared by the following operators (in
!=. They are compared character by character by their unicode code point. If the end of a operand is reached before finding a difference the string lengths are compared instead.
- String comparisons can be logically combined with
||(or). There is also a
!(not) logical operator.
- Parenthesis can be used to override precedence.
The optional Limit field must be an integer number greater than zero. It limits the number of artifacts that are retained by Predicate. If no Limit is specified all matching artifacts are retained. By default the artifacts are sorted by the
build.datefield in descending order so that only the most recent Limit artifacts are retained. If Field is not populated the artifact is always put at the end of the list. Specify
ASCto sort the artifacts in ascending order by Field.
A typical usage of the
cleancommand is to remove old artifacts from a continuous build artifact archive. Suppose the root package that is built is called
platform/appand we want to retain only artifacts that are referenced by builds that are at most seven days old:
bob archive clean "meta.package == \"platform/app\" && \ build.date >= \"$(date -u -Idate -d-7days)\""
The following example retains only the last three builds from a recipe:
bob archive clean 'meta.recipe == "root" LIMIT 3'
Both examples above can be combined, e.g. to keep all builds of the last week while making sure that at least the last build is kept, even if that build is older.
bob archive clean "meta.package == \"platform/app\" && \ build.date >= \"$(date -u -Idate -d-7days)\"" \ 'meta.package == \"platform/app\" LIMIT 1'
- Strings are written with double quotes, e.g.
Find artifacts matching a retention expression.
This expressions that can be given to this command are the same as for the
cleancommand above. All artifacts that match at least one of the expressions are printed on stdout. Use this command to search for particular artifacts or to check that you retention expressions actually match the intended artifacts.
Scan for added artifacts.
archivecommand keeps a cache of all indexed artifacts. To freshen this cache use this command. Even though other sub-commands will do a scan too (unless suppressed by
-n) it might be helpful to do the scan on a more convenient time. If the archive is located e.g. on a slow network drive it could be advantageous to scan the archive with a cron job over night.
bob archive only works for local binary artifact archives. If you’re using a
remote archive, you need shell access and a working Bob installation on the
machine providing your archive in order to be able to use