README.md
1![containerd banner](https://raw.githubusercontent.com/cncf/artwork/master/projects/containerd/horizontal/color/containerd-horizontal-color.png)
2
3[![PkgGoDev](https://pkg.go.dev/badge/github.com/containerd/containerd)](https://pkg.go.dev/github.com/containerd/containerd)
4[![Build Status](https://github.com/containerd/containerd/workflows/CI/badge.svg)](https://github.com/containerd/containerd/actions?query=workflow%3ACI)
5[![Nightlies](https://github.com/containerd/containerd/workflows/Nightly/badge.svg)](https://github.com/containerd/containerd/actions?query=workflow%3ANightly)
6[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fcontainerd%2Fcontainerd.svg?type=shield)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2Fcontainerd%2Fcontainerd?ref=badge_shield)
7[![Go Report Card](https://goreportcard.com/badge/github.com/containerd/containerd)](https://goreportcard.com/report/github.com/containerd/containerd)
8[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/1271/badge)](https://bestpractices.coreinfrastructure.org/projects/1271)
9
10containerd is an industry-standard container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc.
11
12containerd is a member of CNCF with ['graduated'](https://landscape.cncf.io/selected=containerd) status.
13
14containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users.
15
16![architecture](design/architecture.png)
17
18## Now Recruiting
19
20We are a large inclusive OSS project that is welcoming help of any kind shape or form:
21* Documentation help is needed to make the product easier to consume and extend.
22* We need OSS community outreach / organizing help to get the word out; manage
23and create messaging and educational content; and to help with social media, community forums/groups, and google groups.
24* We are actively inviting new [security advisors](https://github.com/containerd/project/blob/master/GOVERNANCE.md#security-advisors) to join the team.
25* New sub-projects are being created, core and non-core that could use additional development help.
26* Each of the [containerd projects](https://github.com/containerd) has a list of issues currently being worked on or that need help resolving.
27 - If the issue has not already been assigned to someone, or has not made recent progress and you are interested, please inquire.
28 - If you are interested in starting with a smaller / beginner level issue, look for issues with an `exp/beginner` tag, for example [containerd/containerd beginner issues.](https://github.com/containerd/containerd/issues?q=is%3Aissue+is%3Aopen+label%3Aexp%2Fbeginner)
29
30## Getting Started
31
32See our documentation on [containerd.io](https://containerd.io):
33* [for ops and admins](docs/ops.md)
34* [namespaces](docs/namespaces.md)
35* [client options](docs/client-opts.md)
36
37See how to build containerd from source at [BUILDING](BUILDING.md).
38
39If you are interested in trying out containerd see our example at [Getting Started](docs/getting-started.md).
40
41## Nightly builds
42
43There are nightly builds available for download [here](https://github.com/containerd/containerd/actions?query=workflow%3ANightly).
44Binaries are generated from `master` branch every night for `Linux` and `Windows`.
45
46Please be aware: nightly builds might have critical bugs, it's not recommended for use in production and no support provided.
47
48## Runtime Requirements
49
50Runtime requirements for containerd are very minimal. Most interactions with
51the Linux and Windows container feature sets are handled via [runc](https://github.com/opencontainers/runc) and/or
52OS-specific libraries (e.g. [hcsshim](https://github.com/Microsoft/hcsshim) for Microsoft). The current required version of `runc` is always listed in [RUNC.md](/docs/RUNC.md).
53
54There are specific features
55used by containerd core code and snapshotters that will require a minimum kernel
56version on Linux. With the understood caveat of distro kernel versioning, a
57reasonable starting point for Linux is a minimum 4.x kernel version.
58
59The overlay filesystem snapshotter, used by default, uses features that were
60finalized in the 4.x kernel series. If you choose to use btrfs, there may
61be more flexibility in kernel version (minimum recommended is 3.18), but will
62require the btrfs kernel module and btrfs tools to be installed on your Linux
63distribution.
64
65To use Linux checkpoint and restore features, you will need `criu` installed on
66your system. See more details in [Checkpoint and Restore](#checkpoint-and-restore).
67
68Build requirements for developers are listed in [BUILDING](BUILDING.md).
69
70## Features
71
72### Client
73
74containerd offers a full client package to help you integrate containerd into your platform.
75
76```go
77
78import (
79 "github.com/containerd/containerd"
80 "github.com/containerd/containerd/cio"
81)
82
83
84func main() {
85 client, err := containerd.New("/run/containerd/containerd.sock")
86 defer client.Close()
87}
88
89```
90
91### Namespaces
92
93Namespaces allow multiple consumers to use the same containerd without conflicting with each other. It has the benefit of sharing content but still having separation with containers and images.
94
95To set a namespace for requests to the API:
96
97```go
98context = context.Background()
99// create a context for docker
100docker = namespaces.WithNamespace(context, "docker")
101
102containerd, err := client.NewContainer(docker, "id")
103```
104
105To set a default namespace on the client:
106
107```go
108client, err := containerd.New(address, containerd.WithDefaultNamespace("docker"))
109```
110
111### Distribution
112
113```go
114// pull an image
115image, err := client.Pull(context, "docker.io/library/redis:latest")
116
117// push an image
118err := client.Push(context, "docker.io/library/redis:latest", image.Target())
119```
120
121### Containers
122
123In containerd, a container is a metadata object. Resources such as an OCI runtime specification, image, root filesystem, and other metadata can be attached to a container.
124
125```go
126redis, err := client.NewContainer(context, "redis-master")
127defer redis.Delete(context)
128```
129
130### OCI Runtime Specification
131
132containerd fully supports the OCI runtime specification for running containers. We have built in functions to help you generate runtime specifications based on images as well as custom parameters.
133
134You can specify options when creating a container about how to modify the specification.
135
136```go
137redis, err := client.NewContainer(context, "redis-master", containerd.WithNewSpec(oci.WithImageConfig(image)))
138```
139
140### Root Filesystems
141
142containerd allows you to use overlay or snapshot filesystems with your containers. It comes with built in support for overlayfs and btrfs.
143
144```go
145// pull an image and unpack it into the configured snapshotter
146image, err := client.Pull(context, "docker.io/library/redis:latest", containerd.WithPullUnpack)
147
148// allocate a new RW root filesystem for a container based on the image
149redis, err := client.NewContainer(context, "redis-master",
150 containerd.WithNewSnapshot("redis-rootfs", image),
151 containerd.WithNewSpec(oci.WithImageConfig(image)),
152)
153
154// use a readonly filesystem with multiple containers
155for i := 0; i < 10; i++ {
156 id := fmt.Sprintf("id-%s", i)
157 container, err := client.NewContainer(ctx, id,
158 containerd.WithNewSnapshotView(id, image),
159 containerd.WithNewSpec(oci.WithImageConfig(image)),
160 )
161}
162```
163
164### Tasks
165
166Taking a container object and turning it into a runnable process on a system is done by creating a new `Task` from the container. A task represents the runnable object within containerd.
167
168```go
169// create a new task
170task, err := redis.NewTask(context, cio.NewCreator(cio.WithStdio))
171defer task.Delete(context)
172
173// the task is now running and has a pid that can be used to setup networking
174// or other runtime settings outside of containerd
175pid := task.Pid()
176
177// start the redis-server process inside the container
178err := task.Start(context)
179
180// wait for the task to exit and get the exit status
181status, err := task.Wait(context)
182```
183
184### Checkpoint and Restore
185
186If you have [criu](https://criu.org/Main_Page) installed on your machine you can checkpoint and restore containers and their tasks. This allows you to clone and/or live migrate containers to other machines.
187
188```go
189// checkpoint the task then push it to a registry
190checkpoint, err := task.Checkpoint(context)
191
192err := client.Push(context, "myregistry/checkpoints/redis:master", checkpoint)
193
194// on a new machine pull the checkpoint and restore the redis container
195checkpoint, err := client.Pull(context, "myregistry/checkpoints/redis:master")
196
197redis, err = client.NewContainer(context, "redis-master", containerd.WithNewSnapshot("redis-rootfs", checkpoint))
198defer container.Delete(context)
199
200task, err = redis.NewTask(context, cio.NewCreator(cio.WithStdio), containerd.WithTaskCheckpoint(checkpoint))
201defer task.Delete(context)
202
203err := task.Start(context)
204```
205
206### Snapshot Plugins
207
208In addition to the built-in Snapshot plugins in containerd, additional external
209plugins can be configured using GRPC. An external plugin is made available using
210the configured name and appears as a plugin alongside the built-in ones.
211
212To add an external snapshot plugin, add the plugin to containerd's config file
213(by default at `/etc/containerd/config.toml`). The string following
214`proxy_plugin.` will be used as the name of the snapshotter and the address
215should refer to a socket with a GRPC listener serving containerd's Snapshot
216GRPC API. Remember to restart containerd for any configuration changes to take
217effect.
218
219```
220[proxy_plugins]
221 [proxy_plugins.customsnapshot]
222 type = "snapshot"
223 address = "/var/run/mysnapshotter.sock"
224```
225
226See [PLUGINS.md](/docs/PLUGINS.md) for how to create plugins
227
228### Releases and API Stability
229
230Please see [RELEASES.md](RELEASES.md) for details on versioning and stability
231of containerd components.
232
233Downloadable 64-bit Intel/AMD binaries of all official releases are available on
234our [releases page](https://github.com/containerd/containerd/releases).
235
236For other architectures and distribution support, you will find that many
237Linux distributions package their own containerd and provide it across several
238architectures, such as [Canonical's Ubuntu packaging](https://launchpad.net/ubuntu/bionic/+package/containerd).
239
240#### Enabling command auto-completion
241
242Starting with containerd 1.4, the urfave client feature for auto-creation of bash and zsh
243autocompletion data is enabled. To use the autocomplete feature in a bash shell for example, source
244the autocomplete/ctr file in your `.bashrc`, or manually like:
245
246```
247$ source ./contrib/autocomplete/ctr
248```
249
250#### Distribution of `ctr` autocomplete for bash and zsh
251
252For bash, copy the `contrib/autocomplete/ctr` script into
253`/etc/bash_completion.d/` and rename it to `ctr`. The `zsh_autocomplete`
254file is also available and can be used similarly for zsh users.
255
256Provide documentation to users to `source` this file into their shell if
257you don't place the autocomplete file in a location where it is automatically
258loaded for the user's shell environment.
259
260### CRI
261
262`cri` is a [containerd](https://containerd.io/) plugin implementation of the Kubernetes [container runtime interface (CRI)](https://github.com/kubernetes/cri-api/blob/master/pkg/apis/runtime/v1alpha2/api.proto). With it, you are able to use containerd as the container runtime for a Kubernetes cluster.
263
264![cri](./docs/cri/cri.png)
265
266#### CRI Status
267
268`cri` is a native plugin of containerd. Since containerd 1.1, the cri plugin is built into the release binaries and enabled by default.
269
270> **Note:** As of containerd 1.5, the `cri` plugin is merged into the containerd/containerd repo. For example, the source code previously stored under [`containerd/cri/pkg`](https://github.com/containerd/cri/tree/release/1.4/pkg)
271was moved to [`containerd/containerd/pkg/cri` package](https://github.com/containerd/containerd/tree/master/pkg/cri).
272
273The `cri` plugin has reached GA status, representing that it is:
274* Feature complete
275* Works with Kubernetes 1.10 and above
276* Passes all [CRI validation tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/cri-validation.md).
277* Passes all [node e2e tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/e2e-node-tests.md).
278* Passes all [e2e tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md).
279
280See results on the containerd k8s [test dashboard](https://k8s-testgrid.appspot.com/sig-node-containerd)
281
282#### Validating Your `cri` Setup
283A Kubernetes incubator project, [cri-tools](https://github.com/kubernetes-sigs/cri-tools), includes programs for exercising CRI implementations. More importantly, cri-tools includes the program `critest` which is used for running [CRI Validation Testing](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/cri-validation.md).
284
285#### CRI Guides
286* [Installing with Ansible and Kubeadm](contrib/ansible/README.md)
287* [For Non-Ansible Users, Preforming a Custom Installation Using the Release Tarball and Kubeadm](docs/cri/installation.md)
288* [CRI Plugin Testing Guide](./docs/cri/testing.md)
289* [Debugging Pods, Containers, and Images with `crictl`](./docs/cri/crictl.md)
290* [Configuring `cri` Plugins](./docs/cri/config.md)
291* [Configuring containerd](https://github.com/containerd/containerd/blob/master/docs/man/containerd-config.8.md)
292
293### Communication
294
295For async communication and long running discussions please use issues and pull requests on the github repo.
296This will be the best place to discuss design and implementation.
297
298For sync communication catch us in the `#containerd` and `#containerd-dev` slack channels on Cloud Native Computing Foundation's (CNCF) slack - `cloud-native.slack.com`. Everyone is welcome to join and chat. [Get Invite to CNCF slack.](https://slack.cncf.io)
299
300### Security audit
301
302A third party security audit was performed by Cure53 in 4Q2018; the [full report](docs/SECURITY_AUDIT.pdf) is available in our docs/ directory.
303
304### Reporting security issues
305
306__If you are reporting a security issue, please reach out discreetly at security@containerd.io__.
307
308## Licenses
309
310The containerd codebase is released under the [Apache 2.0 license](LICENSE).
311The README.md file, and files in the "docs" folder are licensed under the
312Creative Commons Attribution 4.0 International License. You may obtain a
313copy of the license, titled CC-BY-4.0, at http://creativecommons.org/licenses/by/4.0/.
314
315## Project details
316
317**containerd** is the primary open source project within the broader containerd GitHub repository.
318However, all projects within the repo have common maintainership, governance, and contributing
319guidelines which are stored in a `project` repository commonly for all containerd projects.
320
321Please find all these core project documents, including the:
322 * [Project governance](https://github.com/containerd/project/blob/master/GOVERNANCE.md),
323 * [Maintainers](https://github.com/containerd/project/blob/master/MAINTAINERS),
324 * and [Contributing guidelines](https://github.com/containerd/project/blob/master/CONTRIBUTING.md)
325
326information in our [`containerd/project`](https://github.com/containerd/project) repository.
327
328## Adoption
329
330Interested to see who is using containerd? Are you using containerd in a project?
331Please add yourself via pull request to our [ADOPTERS.md](./ADOPTERS.md) file.
332