Optional Components on Prow

Branchprotector

Mon, 01 Jan 0001 00:00:00 +0000

branchprotector configures github branch protection according to a specified policy.

Policy configuration

Extend the primary prow config.yaml document to include a top-level branch-protection key that looks like the following:


branch-protection:
  orgs:
    kubernetes:
      repos:
        test-infra:
          # Protect all branches in kubernetes/test-infra
          protect: true
          # Always allow the org's oncall-team to push
          restrictions:
            teams: ["oncall-team"]
          # Ensure that the extra-process-followed github status context passes.
          # In addition, adds any required prow jobs (aka always_run: true)
          required_status_checks:
            contexts: ["extra-process-followed"]

presubmits:
  kubernetes/test-infra:
  - name: fancy-job-name
    context: fancy-job-name
    always_run: true
    spec:  # podspec that runs job

This config will:

Exporter

Mon, 01 Jan 0001 00:00:00 +0000

The prow-exporter exposes metrics about prow jobs while the metrics are not directly related to a specific prow-component.

Metrics

Metric name	Metric type	Labels/tags
prow_job_labels	Gauge	`job_name`=<prow_job-name> `job_namespace`=<prow_job-namespace> `job_agent`=<prow_job-agent> `label_PROW_JOB_LABEL_KEY`=<PROW_JOB_LABEL_VALUE>
prow_job_annotations	Gauge	`job_name`=<prow_job-name> `job_namespace`=<prow_job-namespace> `job_agent`=<prow_job-agent> `annotation_PROW_JOB_ANNOTATION_KEY`=<PROW_JOB_ANNOTATION_VALUE>
prow_job_runtime_seconds	Histogram	`job_name`=<prow_job-name> `job_namespace`=<prow_job-namespace> `type`=<prow_job-type> `last_state`=<last-state> `state`=<state> `org`=<org> `repo`=<repo> `base_ref`=<base_ref>

For example, the metric prow_job_labels is similar to kube_pod_labels defined in kubernetes/kube-state-metrics. A typical usage of prow_job_labels is to join it with other metrics using a Prometheus matching operator.

gcsupload

Mon, 01 Jan 0001 00:00:00 +0000

gcsupload uploads artifacts to cloud storage at a path resolved from the job configuration.

gcsupload can be configured by either passing in flags or by specifying a full set of options as JSON in the $GCSUPLOAD_OPTIONS environment variable, which has the following form:

{
    "bucket": "kubernetes-jenkins",
    "sub_dir": "",
    "items": [
        "/logs/artifacts/"
    ],
    "path_strategy": "legacy",
    "default_org": "kubernetes",
    "default_repo": "kubernetes",
    "gcs_credentials_file": "/secrets/gcs/service-account.json",
    "dry_run": "false"
}

In addition to this configuration for the tool, the $JOB_SPEC environment variable should be present to provide the contents of the Prow downward API for jobs. This data is used to resolve the exact location in GCS to which artifacts and logs will be pushed.

Gerrit

Mon, 01 Jan 0001 00:00:00 +0000

Gerrit is a Prow-gerrit adapter for handling CI on gerrit workflows. It can poll gerrit changes from multiple gerrit instances, and trigger presubmits on Prow upon new patchsets on Gerrit changes, and postsubmits when Gerrit changes are merged.

Deployment Usage

When deploy the gerrit component, you need to specify --config-path to your prow config, and optionally --job-config-path to your prowjob config if you have split them up.

Set --gerrit-projects to the gerrit projects you want to poll against.

HMAC

Mon, 01 Jan 0001 00:00:00 +0000

hmac is a tool to update the HMAC token, GitHub webhooks and HMAC secret for the orgs/repos as per the managed_webhooks configuration changes in the Prow config file.

Prerequisites

To run this tool, you’ll need:

A github account that has admin permission to the orgs/repos.
A personal access token for the github account. Note the token must be granted admin:repo_hook and admin:org_hook scopes.
Permissions to read&write the hmac secret in the Prow cluster.

jenkins-operator

Mon, 01 Jan 0001 00:00:00 +0000

jenkins-operator is a controller that enables Prow to use Jenkins as a backend for running jobs.

Jenkins configuration

A Jenkins master needs to be provided via --jenkins-url in order for the operator to make requests to. By default, --dry-run is set to true so the operator will not make any mutating requests to Jenkins, GitHub, and Kubernetes, but you most probably want to set it to false. The Jenkins operator expects to read the Prow configuration by default in /etc/config/config.yaml which can be configured with --config-path.

status-reconciler

Mon, 01 Jan 0001 00:00:00 +0000

status-reconciler ensures that changes to blocking presubmits in Prow configuration while PRs are in flight do not cause those PRs to get stuck.

When the set of blocking presubmits changes for a repository, one of three cases occurs:

a new blocking presubmit exists and should be triggered for every trusted pull request in flight
an existing blocking presubmit is removed and should have its’ status retired
an existing blocking presubmit is renamed and should have its’ status migrated

The status-reconciler watches the job configuration for Prow and ensures that the above actions are taken as necessary.

Gangway (Prow API)

Mon, 01 Jan 0001 00:00:00 +0000

Architecture

See the design doc.

Gangway uses gRPC to serve several endpoints. These can be seen in the gangway.proto file, which describes the gRPC endpoints. The proto describes the interface at a high level, and is converted into low-level Golang types into gangway.pb.go and gangway_grpc.pb.go. These low-level Golang types are then used in the gangway.go file to implement the high-level intent of the proto file.

As Gangway only understands gRPC natively, if you want to use a REST client against it you must deploy Gangway. For example, on GKE you can use Cloud Endpoints and deploy Gangway behind a reverse proxy called “ESPv2”. This ESPv2 container will forward HTTP requests made to it to the equivalent gRPC endpoint in Gangway and back again.

Sub

Mon, 01 Jan 0001 00:00:00 +0000

Sub is a Prow component that can trigger new Prow jobs (PJs) using Pub/Sub messages. The message does not need to have the full PJ defined; instead you just need to have the job name and some other key pieces of information (more on this below). The rest of the data needed to create a full-blown PJ is derived from the main Prow configuration (or inrepoconfig).

Deployment Usage

Sub can listen to Pub/Sub subscriptions (known as “pull subscriptions”).