1# sample-apiserver 2 3Demonstration of how to use the k8s.io/apiserver library to build a functional API server. 4 5**Note:** go-get or vendor this package as `k8s.io/sample-apiserver`. 6 7## Purpose 8 9You may use this code if you want to build an Extension API Server to use with API Aggregation, or to build a stand-alone Kubernetes-style API server. 10 11However, consider two other options: 12 * **CRDs**: if you just want to add a resource to your kubernetes cluster, then consider using Custom Resource Definition a.k.a CRDs. They require less coding and rebasing. Read about the differences between Custom Resource Definitions vs Extension API Servers [here](https://kubernetes.io/docs/concepts/api-extension/custom-resources). 13 * **Apiserver-builder**: If you want to build an Extension API server, consider using [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder) instead of this repo. The Apiserver-builder is a complete framework for generating the apiserver, client libraries, and the installation program. 14 15If you do decide to use this repository, then the recommended pattern is to fork this repository, modify it to add your types, and then periodically rebase your changes on top of this repo, to pick up improvements and bug fixes to the apiserver. 16 17 18## Compatibility 19 20HEAD of this repo will match HEAD of k8s.io/apiserver, k8s.io/apimachinery, and k8s.io/client-go. 21 22## Where does it come from? 23 24`sample-apiserver` is synced from https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/sample-apiserver. 25Code changes are made in that location, merged into `k8s.io/kubernetes` and later synced here. 26 27## Fetch sample-apiserver and its dependencies 28 29Like the rest of Kubernetes, sample-apiserver has used 30[godep](https://github.com/tools/godep) and `$GOPATH` for years and is 31now adopting go 1.11 modules. There are thus two alternative ways to 32go about fetching this demo and its dependencies. 33 34### Fetch with godep 35 36When NOT using go 1.11 modules, you can use the following commands. 37 38```sh 39go get -d k8s.io/sample-apiserver 40cd $GOPATH/src/k8s.io/sample-apiserver # assuming your GOPATH has just one entry 41godep restore 42``` 43 44### When using go 1.11 modules 45 46When using go 1.11 modules (`GO111MODULE=on`), issue the following 47commands --- starting in whatever working directory you like. 48 49```sh 50git clone https://github.com/kubernetes/sample-apiserver.git 51cd sample-apiserver 52``` 53 54Note, however, that if you intend to 55[generate code](#changes-to-the-types) then you will also need the 56code-generator repo to exist in an old-style location. One easy way 57to do this is to use the command `go mod vendor` to create and 58populate the `vendor` directory. 59 60### A Note on kubernetes/kubernetes 61 62If you are developing Kubernetes according to 63https://github.com/kubernetes/community/blob/master/contributors/guide/github-workflow.md 64then you already have a copy of this demo in 65`kubernetes/staging/src/k8s.io/sample-apiserver` and its dependencies 66--- including the code generator --- are in usable locations. 67 68 69## Normal Build and Deploy 70 71### Changes to the Types 72 73If you change the API object type definitions in any of the 74`pkg/apis/.../types.go` files then you will need to update the files 75generated from the type definitions. To do this, first 76[create the vendor directory if necessary](#when-using-go-111-modules) 77and then invoke `hack/update-codegen.sh` with `sample-apiserver` as 78your current working directory; the script takes no arguments. 79 80### Authentication plugins 81 82The normal build supports only a very spare selection of 83authentication methods. There is a much larger set available in 84https://github.com/kubernetes/client-go/tree/master/plugin/pkg/client/auth 85. If you want your server to support one of those, such as `oidc`, 86then add an import of the appropriate package to 87`sample-apiserver/main.go`. Here is an example: 88 89``` go 90import _ "k8s.io/client-go/plugin/pkg/client/auth/oidc" 91``` 92 93Alternatively you could add support for all of them, with an import 94like this: 95 96``` go 97import _ "k8s.io/client-go/plugin/pkg/client/auth" 98``` 99 100### Build the Binary 101 102With `sample-apiserver` as your current working directory, issue the 103following command: 104 105``` 106CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o artifacts/simple-image/kube-sample-apiserver 107``` 108 109### Build the Container Image 110 111With `sample-apiserver` as your current working directory, issue the 112following commands with `MYPREFIX` and `MYTAG` replaced by something 113suitable. 114 115``` 116docker build -t MYPREFIX/kube-sample-apiserver:MYTAG ./artifacts/simple-image 117docker push MYPREFIX/kube-sample-apiserver:MYTAG 118``` 119 120### Deploy into a Kubernetes Cluster 121 122Edit `artifacts/example/deployment.yaml`, updating the pod template's image 123reference to match what you pushed and setting the `imagePullPolicy` 124to something suitable. Then call: 125 126``` 127kubectl apply -f artifacts/example 128``` 129 130## Running it stand-alone 131 132During development it is helpful to run sample-apiserver stand-alone, i.e. without 133a Kubernetes API server for authn/authz and without aggregation. This is possible, but needs 134a couple of flags, keys and certs as described below. You will still need some kubeconfig, 135e.g. `~/.kube/config`, but the Kubernetes cluster is not used for authn/z. A minikube or 136hack/local-up-cluster.sh cluster will work. 137 138Instead of trusting the aggregator inside kube-apiserver, the described setup uses local 139client certificate based X.509 authentication and authorization. This means that the client 140certificate is trusted by a CA and the passed certificate contains the group membership 141to the `system:masters` group. As we disable delegated authorization with `--authorization-skip-lookup`, 142only this superuser group is authorized. 143 1441. First we need a CA to later sign the client certificate: 145 146 ``` shell 147 openssl req -nodes -new -x509 -keyout ca.key -out ca.crt 148 ``` 149 1502. Then we create a client cert signed by this CA for the user `development` in the superuser group 151 `system:masters`: 152 153 ``` shell 154 openssl req -out client.csr -new -newkey rsa:4096 -nodes -keyout client.key -subj "/CN=development/O=system:masters" 155 openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt 156 ``` 157 1583. As curl requires client certificates in p12 format with password, do the conversion: 159 160 ``` shell 161 openssl pkcs12 -export -in ./client.crt -inkey ./client.key -out client.p12 -passout pass:password 162 ``` 163 1644. With these keys and certs in-place, we start the server: 165 166 ``` shell 167 etcd & 168 sample-apiserver --secure-port 8443 --etcd-servers http://127.0.0.1:2379 --v=7 \ 169 --client-ca-file ca.crt \ 170 --kubeconfig ~/.kube/config \ 171 --authentication-kubeconfig ~/.kube/config \ 172 --authorization-kubeconfig ~/.kube/config 173 ``` 174 175 The first kubeconfig is used for the shared informers to access 176 Kubernetes resources. The second kubeconfig passed to 177 `--authentication-kubeconfig` is used to satisfy the delegated 178 authenticator. The third kubeconfig passed to 179 `--authorized-kubeconfig` is used to satisfy the delegated 180 authorizer. Neither the authenticator, nor the authorizer will 181 actually be used: due to `--client-ca-file`, our development X.509 182 certificate is accepted and authenticates us as `system:masters` 183 member. `system:masters` is the superuser group such that delegated 184 authorization is skipped. 185 1865. Use curl to access the server using the client certificate in p12 format for authentication: 187 188 ``` shell 189 curl -fv -k --cert-type P12 --cert client.p12:password \ 190 https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders 191 ``` 192 193 Or use wget: 194 ``` shell 195 wget -O- --no-check-certificate \ 196 --certificate client.crt --private-key client.key \ 197 https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders 198 ``` 199 200 Note: Recent OSX versions broke client certs with curl. On Mac try `brew install httpie` and then: 201 202 ``` shell 203 http --verify=no --cert client.crt --cert-key client.key \ 204 https://localhost:8443/apis/wardle.example.com/v1alpha1/namespaces/default/flunders 205 ``` 206