Builds

A build is the process of transforming input parameters into a resulting object. Most often, the process is used to transform input parameters or source code into a runnable image. A BuildConfig object is the definition of the entire build process.

Digital Garage leverages Kubernetes by creating Docker-formatted containers from build images and pushing them to a container registry.

Build objects share common characteristics: inputs for a build, the need to complete a build process, logging the build process, publishing resources from successful builds, and publishing the final status of the build. Builds take advantage of resource restrictions, specifying limitations on resources such as CPU usage, memory usage, and build or pod execution time.

The resulting object of a build depends on the builder used to create it. For Docker and S2I builds, the resulting objects are runnable images. For Custom builds, the resulting objects are whatever the builder image author has specified.

For a list of build commands, see the Developer’s Guide.

For more information on how Digital Garage leverages Docker for builds, see the upstream documentation.

Source-to-Image (S2I) Build

Source-to-Image (S2I) is a tool for building reproducible, Docker-formatted container images. It produces ready-to-run images by injecting application source into a container image and assembling a new image. The new image incorporates the base image (the builder) and built source and is ready to use with the docker run command. S2I supports incremental builds, which re-use previously downloaded dependencies, previously built artifacts, etc.

The advantages of S2I include the following:

Image flexibility

S2I scripts can be written to inject application code into almost any existing Docker-formatted container image, taking advantage of the existing ecosystem. Note that, currently, S2I relies on tar to inject application source, so the image needs to be able to process tarred content.

Speed

With S2I, the assemble process can perform a large number of complex operations without creating a new layer at each step, resulting in a fast process. In addition, S2I scripts can be written to re-use artifacts stored in a previous version of the application image, rather than having to download or build them each time the build is run.

Patchability

S2I allows you to rebuild the application consistently if an underlying image needs a patch due to a security issue.

Operational efficiency

By restricting build operations instead of allowing arbitrary actions, as a Dockerfile would allow, the PaaS operator can avoid accidental or intentional abuses of the build system.

Operational security

Building an arbitrary Dockerfile exposes the host system to root privilege escalation. This can be exploited by a malicious user because the entire Docker build process is run as a user with Docker privileges. S2I restricts the operations performed as a root user and can run the scripts as a non-root user.

User efficiency

S2I prevents developers from performing arbitrary yum install type operations, which could slow down development iteration, during their application build.

Ecosystem

S2I encourages a shared ecosystem of images where you can leverage best practices for your applications.

Reproducibility

Produced images can include all inputs including specific versions of build tools and dependencies. This ensures that the image can be reproduced precisely.

Pipeline Build

The Pipeline build strategy allows developers to define a Jenkins pipeline for execution by the Jenkins pipeline plugin. The build can be started, monitored, and managed by Digital Garage in the same way as any other build type.

Pipeline workflows are defined in a Jenkinsfile, either embedded directly in the build configuration, or supplied in a Git repository and referenced by the build configuration.

The first time a project defines a build configuration using a Pipeline strategy, Digital Garage instantiates a Jenkins server to execute the pipeline. Subsequent Pipeline build configurations in the project share this Jenkins server.

The Jenkins server is not automatically removed, even if all Pipeline build configurations are deleted. It must be manually deleted by the user.

For more information about Jenkins Pipelines, please see the Jenkins documentation.

Image Streams

An image stream and its associated tags provide an abstraction for referencing Docker images from within Digital Garage. The image stream and its tags allow you to see what images are available and ensure that you are using the specific image you need even if the image in the repository changes.

Image streams do not contain actual image data, but present a single virtual view of related images, similar to an image repository.

You can configure Builds and Deployments to watch an image stream for notifications when new images are added and react by performing a Build or Deployment, respectively.

For example, if a Deployment is using a certain image and a new version of that image is created, a Deployment could be automatically performed to pick up the new version of the image.

However, if the image stream tag used by the Deployment or Build is not updated, then even if the Docker image in the Docker registry is updated, the Build or Deployment will continue using the previous (presumably known good) image.

The source images can be stored in any of the following:

  • Digital Garage’s integrated registry

  • An external registry, for example registry.access.redhat.com or hub.docker.com

  • Other image streams in the Digital Garage cluster

When you define an object that references an image stream tag (such as a or Build configuration or Deployment configuration), you point to an image stream tag, not the Docker repository. When you Build or Deploy your application, Digital Garage queries the Docker repository using the image stream tag to locate the associated ID of the image and uses that exact image.

The image stream metadata is stored in the etcd instance along with other cluster information.

The following image stream contains two tags: 34 which points to a Python v3.4 image and 35 which points to a Python v3.5 image:

oc describe is python
Name:			python
Namespace:		imagestream
Created:		25 hours ago
Labels:			app=python
Annotations:		openshift.io/generated-by=OpenShiftWebConsole
			openshift.io/image.dockerRepositoryCheck=2017-10-03T19:48:00Z
Docker Pull Spec:	docker-registry.default.svc:5000/imagestream/python
Image Lookup:		local=false
Unique Images:		2
Tags:			2

34
  tagged from centos/python-34-centos7

  * centos/python-34-centos7@sha256:28178e2352d31f240de1af1370be855db33ae9782de737bb005247d8791a54d0
      14 seconds ago

35
  tagged from centos/python-35-centos7

  * centos/python-35-centos7@sha256:2efb79ca3ac9c9145a63675fb0c09220ab3b8d4005d35e0644417ee552548b10
      7 seconds ago

Using image streams has several significant benefits:

  • You can tag, rollback a tag, and quickly deal with images, without having to re-push using the command line.

  • You can trigger Builds and Deployments when a new image is pushed to the registry. Also, Digital Garage also has generic triggers for other resources (such as Kubernetes objects).

  • You can mark a tag for periodic re-import. If the source image has changed, that change is picked up and reflected in the image stream, which triggers the Build and/or Deployment flow, depending upon the Build or Deployment configuration.

  • You can share images using fine-grained access control and quickly distribute images across your teams.

  • If the source image changes, the image stream tag will still point to a known-good version of the image, ensuring that your application will not break unexpectedly.

  • You can configure security around who can view and use the images through permissions on the image stream objects.

  • Users that lack permission to read or list images on the cluster level can still retrieve the images tagged in a project using image streams.

For a curated set of image streams, see the OpenShift Image Streams and Templates library.

When using image streams, it is important to understand what the image stream tag is pointing to and how changes to tags and images can affect you. For example:

  • If your image stream tag points to a Docker image tag, you need to understand how that Docker image tag is updated. For example, a Docker image tag docker.io/ruby:2.4 will probably always point to a v2.4 ruby image. But, a Docker image tag docker.io/ruby:latest will probably change with major versions. So, the Docker image tag that a image stream tag points to can tell you how stable the image stream tag will be, if you choose to reference it.

  • If your image stream tag follows another image stream tag (it does not point directly to a docker image tag), it is possible that the image stream tag will be updated to follow a different image stream tag in the future. Again, this could result in picking up an incompatible version change.

Important terms

Docker repository

A collection of related docker images and tags identifying them. For example, the OpenShift Jenkins images are in a Docker repository:

docker.io/openshift/jenkins-2-centos7
Docker registry

A content server that can store and service images from Docker repositories. For example:

registry.access.redhat.com
Docker image

A specific set of content that can be run as a container. Usually associated with a particular tag within a Docker repository.

Docker image tag

A label applied to a Docker image in a repository that distinguishes a specific image. For example, here 3.6.0 is a tag:

docker.io/openshift/jenkins-2-centos7:3.6.0

A Docker image tag can be updated to point to new Docker image content at any time.

Docker image ID

A SHA (Secure Hash Algorithm) code that can be used to pull an image. For example:

docker.io/openshift/jenkins-2-centos7@sha256:ab312bda324

A SHA image ID cannot change. A specific SHA identifier always references the exact same docker image content.

Image stream

An Digital Garage object that contains pointers to any number of Docker-formatted container images identified by tags. You can think of an image stream as equivalent to a Docker repository.

Image stream tag

A named pointer to an image in an image stream. An image stream tag is similar to a Docker image tag. See Image Stream Tag below.

Image stream image

An image that allows you to retrieve a specific Docker image from a particular image stream where it is tagged. An image stream image is an API resource object that pulls together some metadata about a particular image SHA identifier. See Image Stream Images below.

Image stream trigger

A trigger that causes a specific action when an image stream tag changes. For example, importing can cause the value of the tag to change, which causes a trigger to fire when there are Deployments, Builds, or other resources listening for those. See Image Stream Triggers below.

Configuring Image Streams