bob-jenkins
Name
bob-jenkins - Configure Jenkins server
Synopsis
Generic command format:
bob jenkins [-h] [-c NAME] subcommand ...
Available sub-commands:
bob jenkins add [-h] [-n NODES] [-o OPTIONS]
[--host-platform {linux,msys,win32}] [-w] [-p PREFIX]
[-r ROOT] [-D DEFINES] [--keep] [--download] [--upload]
[--sandbox | --slim-sandbox | --dev-sandbox | --strict-sandbox | --no-sandbox]
[--credentials CREDENTIALS] [--clean | --incremental]
[--shortdescription | --longdescription]
name url
bob jenkins export [-h] name dir
bob jenkins graph [-h] name
bob jenkins ls [-h] [-v]
bob jenkins prune [-h] [--obsolete | --intermediate] [--no-ssl-verify]
[--user USER] [--password PASSWORD] [-q] [-v]
name
bob jenkins push [-h] [-f] [--no-ssl-verify] [--no-trigger]
[--user USER] [--password PASSWORD] [-q] [-v]
name
bob jenkins rm [-h] [-f] name
bob jenkins set-options [-h] [--reset] [-n NODES] [-o OPTIONS]
[--host-platform {linux,msys,win32}]
[-p PREFIX] [--add-root ADD_ROOT]
[--del-root DEL_ROOT] [-D DEFINES]
[-U UNDEFINES] [--credentials CREDENTIALS]
[--authtoken AUTHTOKEN]
[--shortdescription | --longdescription]
[--keep | --no-keep]
[--download | --no-download]
[--upload | --no-upload]
[--sandbox | --slim-sandbox | --dev-sandbox | --strict-sandbox | --no-sandbox]
[--clean | --incremental]
name
bob jenkins set-url [-h] name url
Description
The bob jenkins
command can be used to build a project on a Jenkins server
remotely. Bob will create the necessary Jenkins jobs on the server, configure
them and queue their build. If recipes or configuration options are updated,
the Jenkins jobs will be updated incrementally as needed.
For the most common usage two steps are necessary: add a Jenkins configuration
under an alias name and push this configuration to the server. A Jenkins
configuration is created by bob jenkins add
which can later be updated by
bob jenkins set-options
or set-url
. To push the current configuration
to the server use bob jenkins push
. As a rule of thumb Bob will create one
job for each built recipe.
For more details and an overview about the required Jenkins configuration, please read the Jenkins Tutorial.
Options
--add-root ADD_ROOT
Add new root package. See the
--root
option below for its detailed description.--authtoken AUTHTOKEN
Add an authentication token to restrict remote build triggers.
A job build can be scheduled by sending a HTTP POST request to
$JENKINS_URL/job/$JOB/build
. To allow unauthenticated scripts to be able to trigger builds too, a secret authentication token can be added to all jobs and the build can be started by POSTing to$JENKINS_URL/job/$JOB/build?token=AUTHTOKEN
. Note that you have to keep the token secret because anyone who has the right to view the job configuration will be able to trigger a build.--clean
Do clean builds. This is the default.
Whenever a Jenkins job is triggered, the workspace is erased first. This will cause a fresh checkout of the sources and a clean (re)build of all packages in the job.
Use this option to ensure reproducibility at the expense of speed. Enabled by default. See
--incremental
for the opposite switch.--credentials CREDENTIALS
Credentials ID for git/svn SCM checkouts.
By default SCM checkouts of Git and Subversion modules are done anonymously. This might be insufficient if the Jenkins build nodes lack the proper credentials to access the repositories. Jenkins has a built-in credentials store (“Manage Jenkins » Credentials”) where passwords and SSH keys can be managed. Add the required credentials there and use the “ID” field of the required entry here as
CREDENTIALS
parameter. Usually the ID will be a UUID but Jenkins allows any unique identifier to be used.All jobs of an alias will be configured with the same credentials. If a more fine grained credentials configurations is required, a plugin must be used. See the “Jenkins job mangling” section in the manual for more information.
-D DEFINES
Override default environment variable.
--del-root DEL_ROOT
Remove existing root package.
--dev-sandbox
Enable development sandboxing.
Always build packages in an isolated environment where only declared dependencies are visible. If a sandbox image is available, it is used. Otherwise the host paths are made read-only.
--download
Enable downloads from binary archive. Disabled by default. There must be at least one binary archive in the user configuration archive section that is enabled for Jenkins builds.
-f, --force
Force the operation, potentially with loss of information. The exact semantics depend on the sub-command where the switch is used:
push
Overwrite existing jobs.
By default, Bob will refuse to overwrite jobs that were not created by himself. If you are sure that the existing jobs are safe to be overwritten, you can use this switch. Otherwise the jobs must be either deleted manually or by the
prune
command of the project that created the them in the first place. Additionally all job configurations are written, even if they have not changed. This overwrites any possible manual changes made to the jobs.rm
Remove the Jenkins alias, even if there are active jobs. You will have to delete the jobs manually.
--host-platform
Jenkins host platform type. May be any of
linux
,msys
orwin32
.This specifies the host operating system where the Jenkins master and the build slaves are running. By default this is the type of the current operating system.
--incremental
Reuse workspace for incremental builds.
Bob will still apply the internal heuristics to make clean builds where recipes or any of the dependencies were changed. Use
--clean
to always force clean builds of packages.--intermediate
Delete everything except root jobs.
Use this switch if you want to delete a project from the Jenkins server but want to keep the jobs with the final artifacts. The root jobs will be disabled because their dependencies are deleted. You can push an alias again to re-create all jobs and re-enable the root jobs.
--keep
Keep obsolete jobs by disabling them instead of deleting.
If the recipes or configuration of a project is changed, some of the previously required packages could become unnecessary. By default Bob will delete the corresponding jobs. By using the
--keep
switch these jobs will merely be disabled. This retains the build logs and artifacts.You can use
bob jenkins prune --obsolete
to delete disabled jobs manually. See--no-keep
for the inverse option.--longdescription
Display all paths of all packages in the job description.
Note that the number of displayed package paths of (content wise) identical packages is still limited. Nonetheless it is computationally expensive to calculate every possible package path in the first place. Except for trivial projects this might cause a noticeable delays in the Jenkins configuration. See
--shortdescription
on how to disable this behaviour.-n NODES, --nodes NODES
Label expression for Jenkins slave. If empty, the jobs can be scheduled on any slave.
In the Jenkins configuration every build node can be assigned one or more label. The expression given in
NODES
restricts on which build nodes the jobs can be scheduled. It can either be a single label or a boolean expression of labels. The “built-in” label is pre-defined and identifies the Jenkins master. Expressions can use parentheses “(expression)”, negation “!”, logical AND “&&” and logical OR “||”.Examples:
win32
linux && 64bit
!win32 || (vm && mysql)
--no-download
Disable binary archive downloads. This is the default. See
--download
for the enabling counterpart.--no-keep
Delete obsolete jobs. This is the default.
Jobs that are not required any more will be deleted. Use
--keep
if you instead want to just disable such jobs.--no-sandbox
Disable sandboxing during builds.
Unless required by the project, it is discouraged to disable the sandbox feature. See
--sandbox
,--slim-sandbox
,--dev-sandbox
or--strict-sandbox
for the opposite switches.--no-ssl-verify
Disable HTTPS certificate checking.
By default only secure connections are allowed to HTTPS Jenkins servers. If this option is given then any certificate error is ignored. This was the default before Bob 0.15.
--no-trigger
Do not trigger build for updated jobs.
You have to manually schedule the build of all changed jobs. Triggering only a subset of the affected jobs can lead to build errors because of unbuilt dependencies. Use with caution.
--no-upload
Disable binary archive uploads. This is the default. See
--upload
for the enabling counterpart.-o OPTIONS
Set extended Jenkins options. This option expects a
key=value
pair to set one particular extended configuration parameter. May be specified multiple times. See Extended Options for the list of available options. Setting an empty value deletes the option.--obsolete
Delete obsolete jobs that are currently not needed according to the recipes. Use this switch with the
prune
command to delete jobs that are left disabled due to--keep
being active.-p PREFIX, --prefix PREFIX
Prefix for job names.
By default the job names are derived from the recipe and package names. If you want to build the same project with different configurations on the same server you will have to use unique prefixes for each. Otherwise the jobs names will collide and configuration will fail.
--password
Set password for Jenkins authentication.
You can also set the user name and password persistently by encoding it into the Jenkins url directly, e.g. https:://user:password@host/.
Attention
On Linux users can usually see the program arguments of processes from other users. By using the
--password
you could inadvertently reveal the password to untrusted other users that have access to the same machine. It is safer to either enter the password manually or to pipe it through stdin.-q, --quiet
Decrease verbosity (may be specified multiple times).
-r ROOT, --root ROOT
Root package to build (may be specified multiple times).
Specify the root packages that are built. All dependencies are added implicitly. Jobs building the root packages are treated a bit differently in that their logs and artifacts will be retained indefinitely by default. See the
jobs.gc.*
extended options on how to tweak this behavior.--reset
Reset all options to their default.
Use this option to revert all configuration options back to their default state. This option is applied before all other options of the
set-options
sub-command. Use it to configure an alias without relying on the previous state.--sandbox
Enable partial sandboxing. This is the default.
Build packages in an ephemeral container if a sandbox image is available for the package. Inside the sandbox, stable execution paths are used. In absence of a sandbox image, no isolation is performed.
--shortdescription
Do not calculate every possible path of each package in a job for the description. This leads to shorter job descriptions and, depending on the project complexity, might reduce the configuration time considerably. The drawback is that not all packages are then listed in the job description. For each unique package only one example path will be shown.
--slim-sandbox
Enable slim sandboxing.
Build packages in an isolated mount namespace. Most of the host paths are available read-only. Other workspaces are hidden when building a package unless they are a declared dependency. An optionally available sandbox image is not used.
--strict-sandbox
Enable strict sandboxing.
Always build packages in an isolated environment where only declared dependencies are visible. If a sandbox image is available, it is used. Otherwise the host paths are made read-only. The build path is always a reproducible, stable path.
-U UNDEFINES
Undefine environment variable override. This removes a variable previously defined with
-D
.--upload
Upload to binary archives. Disabled by default. There must be at least one binary archive in the user configuration archive section that is enabled for Jenkins builds.
If the upload fails the respective job will fail too, unless the
nofail
option was set on the archive entry in the configuration.--user
Set user name for Jenkins authentication.
You can also set the user name persistently by encoding it into the Jenkins url directly, e.g. https:://user@host/.
-v, --verbose
Show additional information. Can be given multiple times to further increase the output verbosity.
-w, --windows
Jenkins is running on Windows with an MSYS2 environment. This option has been deprecated in favour of
--host-platform msys
switch.
Commands
- add
Add an alias for a Jenkins configuration.
The alias will hold the URL of the Jenkins, the desired configuration (e.g. what packages should be built) and the state of the last uploaded configuration. The state will be stored in the current project workspace. Any number of aliases can be added.
Adding an alias is the first step required to build a project on a Jenkins server. The configuration for this alias can be later updated by the
set-options
andset-url
commands. To remove an alias use therm
command.- export
Write the Jenkins configuration of an alias to a directory.
For each job, the generated config.xml file will be created in the output directory. This is mainly a debugging aid and can be used to inspect the generated configuration. It is not intended to upload these configuration files to a Jenkins server. Use
push
for that.- graph
Generate a Graphviz dot graph.
Feed the generated graph through the
dot
tool to get a visualization about the jobs and their dependencies.- ls
List all configured Jenkins aliases and their configuration.
Without any further options, only the list of Jenkins aliases is shown. By adding the
-v
option the configuration of each alias is displayed too. A 2nd-v
will additionally show all currently configured jobs.- prune
Prune jobs from Jenkins server.
By default all jobs managed by the Jenkins alias will be deleted. If the
--keep
option is enabled for this alias, you may use the--obsolete
option to delete only currently disabled (obsolete) jobs. Alternatively you may delete all intermediate jobs and keep only the root jobs by using--intermediate
. This will disable the root jobs because they cannot run anyway without failing.- push
Push current configuration of an alias to the Jenkins server.
This will create or update all necessary jobs and schedule their build. By default obsolete jobs will be deleted unless the
--keep
option has been enabled. If you just want to create or update the jobs without scheduling their build, use the--no-trigger
option. Bob won’t overwrite jobs that were not created by Bob for the Jenkins alias itself unless the-f
option is given.Existing jobs will be updated as necessary. In the default configuration this happens always because the job description displays the state of the recipes and the time of the
bob jenkins push
operation. Use one of the other modes of thejobs.update
extended option to speed up the push operation at the expense of slightly outdated job descriptions.- rm
Remove Jenkins alias.
The alias will not be removed if jobs are still existing. It is thus usually required to run the
prune
command before to delete all jobs of an alias. Alternatively the-f
switch may be used to remove the alias even though the state indicates that there are still existing jobs. This is useful e.g. if the Jenkins server is not running any more or the jobs have already been deleted externally.- set-options
Change configuration of an alias.
Can update all options of an alias except the server URL. The new configuration can then be synchronized to the Jenkins server by a subsequent
push
command. To revert the whole configuration to its default state use--reset
. This is done as the first step so that you can combine--reset
with all other options to fully control all options.- set-url
Update server URL of an alias.
Extended Options
The following extended Jenkins options are available. Any unrecognized options will be rejected.
- artifacts.copy
This options selects the way of sharing archives between workspaces. Possible values are:
- jenkins
Store the result and Build-Id of the job on the Jenkins master. Subsequently the downstream job will be configured to use the copy artifact plugin to copy the artifact into it’s workspace. This is the default.
- archive
Only store the Build-Id on the Jenkins master and use a separate binary archive for sharing artifacts. Must be used together with
--upload
and--download
.
- audit.meta.<var>
Assign the meta variable
<var>
to the given value in the audit trail. The variable can later be matched by bob archive asmeta.<var>
to select artifacts built by this project. Variables that are defined by Bob itself (e.g.meta.jenkins-node
) cannot be redefined!- jobs.gc.deps.artifacts
The number of build artifacts that are retained of intermediate or leaf jobs. Only useful for
artifacts.copy=jenkins
. Protocols and build logs are not affected and will still be kept. Defaults to1
. If set to 0 all artifacts will be retained.- jobs.gc.deps.builds
Configure the number of builds that are retained of intermediate and leaf jobs. Logs and artifacts of old builds exceeding this threshold are deleted automatically by Jenkins. A separate binary archive (
artifacts.copy=archive
) is not affected and must be separately managed with bob archive. If not set, all Jenkins builds will be kept.- jobs.gc.root.artifacts
The number of build artifacts that are retained of root-jobs. Only useful for
artifacts.copy=jenkins
. Protocols and build logs are not affected and will still be kept. By default everything will be retained.- jobs.gc.root.builds
Configure the number of builds that are retained of root-jobs. These are jobs that build packages that were given by the
-r
option. Logs and artifacts of old builds exceeding this threshold are deleted automatically by Jenkins. A separate binary archive (artifacts.copy=archive
) is not affected but must be separately managed with bob archive. If not set, all Jenkins builds will be kept.- jobs.isolate
Regular expression that is matching package names. Any package that is matched is put into a separate job. Multiple variants of the same package are still kept in the same job, though.
This option might be used to single out specific packages into dedicated Jenkins jobs that are unrelated to other jobs in the recipe. Typical use cases are documentation and testing
multiPackage
that should not prevent other packages from building if they fail. The obvious draw back is that common checkout and build steps might be duplicated to multiple jobs, though.- jobs.policy
Controls how downstream jobs are triggered and which artifacts of the upstream jobs are used. By default only stable jobs trigger further downstream builds. The following settings are available:
- stable
Downstream jobs are triggered only if the build was stable. Likewise, only the artifacts of stable upstream builds are used. This is the default.
- unstable
Downstream jobs are triggered on successful builds, that is stable and unstable builds. The downstream jobs will also use the last build that succeeded, even if that build was unstable.
- always
Downstream jobs are triggered regardless of the build result, even on failed builds. The artifacts are taken from the last completed build of the upstream job which might not necessarily have published one because it failed before archiving them.
- jobs.update
Whenever the recipes are changed Bob has to update the individual Jenkins jobs that are affected by the change. This switch controls how the description and audit trail information is updated if only these are affected by the change. Their update may be deferred unless strictly necessary and still generate a correct build result at the expense of the freshness of this information.
- always
Always update the description and audit trail information if they change. This is the default. Note that
bob jenkins push
will always update the description because the date and time of the update is part of the job description.- description
Keep the description up-to-date but defer audit trail updates unless strictly necessary. This may provide marginal speed gains but will still update all jobs because the description contains the recipe version and update time.
- lazy
Only update a job if it will build a different artifact than before. The description and audit trail information will be left unchanged otherwise. This will provide considerable speed improvements at the expense of an outdated description of the unchanged jobs.
- scm.always-checkout
Boolean option (possible values: ‘0’ or ‘false’ resp. ‘1’ or ‘true’) that forces the execution of checkout steps. This option is enabled by default. If disabled, the checkout might be skipped if a matching binary artifact can be found.
Disabling this option can increase the build speed. On the other hand, it might hide problems in recipes where the checkout step is not deterministic. Note that git and svn SCMs are checked out regardless of this option. For release builds it is best to keep the option enabled.
- scm.git.shallow
Instruct the Jenkins git plugin to create shallow clones with a history truncated to the specified number of commits. If the parameter is unset or “0” the full history will be cloned.
Warning
Setting this parameter too small may prevent the creation of a proper change log. Jenkins will not be able to find the reference commit of the last run if the branch advanced by more commits than were cloned.
- scm.git.timeout
Instruct the Jenkins git plugin to use the given timeout (minutes) for clone and fetch operations.
- scm.ignore-hooks
Boolean option (possible values: ‘0’ or ‘false’ resp. ‘1’ or ‘true’) to set the “Ignore post-commit hooks” option on all jobs. This instructs Jenkins to ignore changes notified by SCM post-commit hooks if enabled. You should probably set a sensible polling interval with the
scm.poll
option unless you want to trigger the generated jobs manually.- scm.poll
Without this option the Jenkins server is dependent on external commit hooks to be notified of changes in the source code repositories. While this is the preferred solution it might be necessary to fall back to polling in some setups. Set this option to a Jenkins flavoured cron line, e.g.
H/15 * * * *
.- shared.dir
Any packages that are marked as shared (
shared: True
) are installed upon usage on a Jenkins slave in a shared location. By default this is${JENKINS_HOME}/bob
. To use another directory set this option to an absolute path.Attention
The string is subject to string substitution. It is possbile to substitute envirionment variables that are set in the Jenkins execution environment. Make sure that any meta characters are properly escaped. Because backslash is such a character, special care must be taken on Windows. It is best to always use forward slashes, even on Windows, to evade any escaping issues.
- shared.quota
Set a limit to the amount of disk space that is used for the shared location on each build node. By default there is no limit. The size is given in bytes with optional magnitude suffix. The standard IEC units are supported (
KiB
,MiB
,GiB
andTiB
) which can optionally be abbreviated by leaving out theiB
suffix (e.g.G
forGiB
). SI units (base 1000) are supported too (KB
,MB
,GB
, andTB
).Note
Only unused packages will be deleted when the quota is reached. If there are no unused shared packages, e.g. because the workspaces of obsolte jobs were not deleted, it is still possible that the disk usage is above the quota.