1# sample-apiserver
2
3Demonstration of how to use the k8s.io/apiserver library to build a functional API server.
4
5**Note:** go-get or vendor this package as `k8s.io/sample-apiserver`.
6
7## Purpose
8
9You may use this code if you want to build an Extension API Server to use with API Aggregation, or to build a stand-alone Kubernetes-style API server.
10
11However, consider two other options:
12  * **CRDs**:  if you just want to add a resource to your kubernetes cluster, then consider using Custom Resource Definition a.k.a CRDs.  They require less coding and rebasing.  Read about the differences between Custom Resource Definitions vs Extension API Servers [here](https://kubernetes.io/docs/concepts/api-extension/custom-resources).
13  * **Apiserver-builder**: If you want to build an Extension API server, consider using [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder) instead of this repo.  The Apiserver-builder is a complete framework for generating the apiserver, client libraries, and the installation program.
14
15If you do decide to use this repository, then the recommended pattern is to fork this repository, modify it to add your types, and then periodically rebase your changes on top of this repo, to pick up improvements and bug fixes to the apiserver.
16
17
18## Compatibility
19
20HEAD of this repo will match HEAD of k8s.io/apiserver, k8s.io/apimachinery, and k8s.io/client-go.
21
22## Where does it come from?
23
24`sample-apiserver` is synced from https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/sample-apiserver.
25Code changes are made in that location, merged into `k8s.io/kubernetes` and later synced here.
26
27## Fetch sample-apiserver and its dependencies
28
29Like the rest of Kubernetes, sample-apiserver has used
30[godep](https://github.com/tools/godep) and `$GOPATH` for years and is
31now adopting go 1.11 modules.  There are thus two alternative ways to
32go about fetching this demo and its dependencies.
33
34### Fetch with godep
35
36When NOT using go 1.11 modules, you can use the following commands.
37
38```sh
39go get -d k8s.io/sample-apiserver
40cd $GOPATH/src/k8s.io/sample-apiserver  # assuming your GOPATH has just one entry
41godep restore
42```
43
44### When using go 1.11 modules
45
46When using go 1.11 modules (`GO111MODULE=on`), issue the following
47commands --- starting in whatever working directory you like.
48
49```sh
50git clone https://github.com/kubernetes/sample-apiserver.git
51cd sample-apiserver
52```
53
54Note, however, that if you intend to
55[generate code](#changes-to-the-types) then you will also need the
56code-generator repo to exist in an old-style location.  One easy way
57to do this is to use the command `go mod vendor` to create and
58populate the `vendor` directory.
59
60### A Note on kubernetes/kubernetes
61
62If you are developing Kubernetes according to
63https://github.com/kubernetes/community/blob/master/contributors/guide/github-workflow.md
64then you already have a copy of this demo in
65`kubernetes/staging/src/k8s.io/sample-apiserver` and its dependencies
66--- including the code generator --- are in usable locations.
67
68
69## Normal Build and Deploy
70
71### Changes to the Types
72
73If you change the API object type definitions in any of the
74`pkg/apis/.../types.go` files then you will need to update the files
75generated from the type definitions.  To do this, first
76[create the vendor directory if necessary](#when-using-go-111-modules)
77and then invoke `hack/update-codegen.sh` with `sample-apiserver` as
78your current working directory; the script takes no arguments.
79
80### Authentication plugins
81
82The normal build supports only a very spare selection of
83authentication methods.  There is a much larger set available in
84https://github.com/kubernetes/client-go/tree/master/plugin/pkg/client/auth
85.  If you want your server to support one of those, such as `oidc`,
86then add an import of the appropriate package to
87`sample-apiserver/main.go`.  Here is an example:
88
89``` go
90import _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
91```
92
93Alternatively you could add support for all of them, with an import
94like this:
95
96``` go
97import _ "k8s.io/client-go/plugin/pkg/client/auth"
98```
99
100### Build the Binary
101
102With `sample-apiserver` as your current working directory, issue the
103following command:
104
105```
106CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o artifacts/simple-image/kube-sample-apiserver
107```
108
109### Build the Container Image
110
111With `sample-apiserver` as your current working directory, issue the
112following commands with `MYPREFIX` and `MYTAG` replaced by something
113suitable.
114
115```
116docker build -t MYPREFIX/kube-sample-apiserver:MYTAG ./artifacts/simple-image
117docker push MYPREFIX/kube-sample-apiserver:MYTAG
118```
119
120### Deploy into a Kubernetes Cluster
121
122Edit `artifacts/example/deployment.yaml`, updating the pod template's image
123reference to match what you pushed and setting the `imagePullPolicy`
124to something suitable.  Then call:
125
126```
127kubectl apply -f artifacts/example
128```
129
130## Running it stand-alone
131
132During development it is helpful to run sample-apiserver stand-alone, i.e. without
133a Kubernetes API server for authn/authz and without aggregation. This is possible, but needs
134a couple of flags, keys and certs as described below. You will still need some kubeconfig,
135e.g. `~/.kube/config`, but the Kubernetes cluster is not used for authn/z. A minikube or
136hack/local-up-cluster.sh cluster will work.
137
138Instead of trusting the aggregator inside kube-apiserver, the described setup uses local
139client certificate based X.509 authentication and authorization. This means that the client
140certificate is trusted by a CA and the passed certificate contains the group membership
141to the `system:masters` group. As we disable delegated authorization with `--authorization-skip-lookup`,
142only this superuser group is authorized.
143
1441. First we need a CA to later sign the client certificate:
145
146   ``` shell
147   openssl req -nodes -new -x509 -keyout ca.key -out ca.crt
148   ```
149
1502. Then we create a client cert signed by this CA for the user `development` in the superuser group
151   `system:masters`:
152
153   ``` shell
154   openssl req -out client.csr -new -newkey rsa:4096 -nodes -keyout client.key -subj "/CN=development/O=system:masters"
155   openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt
156   ```
157
1583. As curl requires client certificates in p12 format with password, do the conversion:
159
160   ``` shell
161   openssl pkcs12 -export -in ./client.crt -inkey ./client.key -out client.p12 -passout pass:password
162   ```
163
1644. With these keys and certs in-place, we start the server:
165
166   ``` shell
167   etcd &
168   sample-apiserver --secure-port 8443 --etcd-servers http://127.0.0.1:2379 --v=7 \
169      --client-ca-file ca.crt \
170      --kubeconfig ~/.kube/config \
171      --authentication-kubeconfig ~/.kube/config \
172      --authorization-kubeconfig ~/.kube/config
173   ```
174
175   The first kubeconfig is used for the shared informers to access
176   Kubernetes resources. The second kubeconfig passed to
177   `--authentication-kubeconfig` is used to satisfy the delegated
178   authenticator. The third kubeconfig passed to
179   `--authorized-kubeconfig` is used to satisfy the delegated
180   authorizer. Neither the authenticator, nor the authorizer will
181   actually be used: due to `--client-ca-file`, our development X.509
182   certificate is accepted and authenticates us as `system:masters`
183   member. `system:masters` is the superuser group such that delegated
184   authorization is skipped.
185
1865. Use curl to access the server using the client certificate in p12 format for authentication:
187
188   ``` shell
189   curl -fv -k --cert-type P12 --cert client.p12:password \
190      https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders
191   ```
192
193   Or use wget:
194   ``` shell
195   wget -O- --no-check-certificate \
196      --certificate client.crt --private-key client.key \
197      https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders
198   ```
199
200   Note: Recent OSX versions broke client certs with curl. On Mac try `brew install httpie` and then:
201
202   ``` shell
203   http --verify=no --cert client.crt --cert-key client.key \
204      https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders
205   ```
206