# Kubernetes AWS approver

A custom node [CertificateSigningRequest](https://godoc.org/k8s.io/api/certificates/v1beta1#CertificateSigningRequest) approver for kubernetes clusters running on AWS.

## Why do I want this?

[kubelet TLS bootstrapping][tls-bootstrapping] allows kubelet to create CertificateSigningRequest for its client certs. These CertificateSigningRequest need to be approved so as to be signed by kubernetes internal certificate signer controller.

Currently you can either manually approve each node that tries to join the cluster. Or, instruct kubernetes internal certificate approver to approve all certificates from `system:bootstrappers` group. The human intervention required for manual approval is not ideal in auto scaling environments. And the default auto approver performs very basic checks that might not be very secure.

`kube-aws-approver` goes a step further from default auto approver and uses the CertificateSigningRequest generated by node to extract information about the node requesting the certificate, and tries to validate it against AWS API before approving the CertificateSigningRequest.

## How does it work?

kube-aws-approver matches incoming CertificateSigningRequest against two critera:

### Criteria 1: Node already belonging to the cluster requests for client certificate renewal.

- The CertificateRequest `Organization` is equal to `system:nodes`
- The CertificateRequest `DNSNames` list is empty
- The CertificateRequest `EmailAddresses` list is empty
- The CertificateRequest `IPAddresses` list is empty
- The CertificateRequest `CommonName` has `system:node:` prefix
- The CertificateSigningRequest `Username` matches the CertificateRequest `CommonName`, i.e. the node provided its client certificate to create this CertificateSigningRequest.
- The CertificateSigningRequest `Groups` has `system:nodes`
- There exists an `instance` in `running state` corresponding to the `nodename` from the CertificateRequest `CommonName`
- A `node` with the `nodename` from CertificateRequest CommonName is `part` of the cluster and in `Ready` state.
- The `instance` should belong to an AutoScalingGroup.
- The AutoSaclingGroup belongs to the list of whitelisted AutoScalingGroups provided to `kube-aws-approver` at startup.

Only when a CertificateSigningRequest from a node looking to renew its client certificate meets `all` the above requirements, the CertificateSigningRequest is `Approved` [more](#certificateSigningRequest-approval-process)

### Criteria 2: A new node trying to join the cluster requests for a client certificate.

- The CertificateRequest `Organization` is equal to `system:nodes`
- The CertificateRequest `DNSNames` list is empty
- The CertificateRequest `EmailAddresses` list is empty
- The CertificateRequest `IPAddresses` list is empty
- The CertificateRequest `CommonName` has `system:node:` prefix
- The CertificateSigningRequest `Groups` has `system:bootstrappers`
- There exists an instance in `running state` corresponding to the `nodename` from the  CertificateRequest `CommonName`
- The CertificateSigningRequest `Username` shoudd be of the form `system:bootstrappers:<aws-instance-id>` [How you might achieve this?](aws-authenticator)
- The `instance-id` extracted from CertificateSigningRequest `Username` matches the `instance-id` extracted from AWS API for the `nodename` in CertificateRequest `CommonName`
- The node with `nodename` from CertificateRequest CommonName is `not part` of the cluster.
- The `instance` should belong to an AutoScalingGroup.
- The AutoSaclingGroup belongs to the list of whitelisted AutoScalingGroups provided to `kube-aws-approver` at startup.

Only when a CertificateSigningRequest from a new node looking for its client certificate meets `all` the above requirements, the CertificateSigningRequest is `Approved` [more](#certificateSigningRequest-approval-process)

`kube-aws-approver` only approves a CertificateSigningRequest that meets one of these two criteria. It never explicitly declines any CertificateSigningRequest.
The approved CertificateSigningRequest is then signed by the kube-controller-manager and can be used by the node as client certificate.

### CertificateSigningRequest Approval process

The following example go code shows how the `kube-aws-approver` marks a CertificateSigningRequest approved.
```go
...
...
// csr is the CertificateSigningRequest(https://godoc.org/k8s.io/api/certificates/v1beta1#CertificateSigningRequest) object received by the approver.
csr.Status.Conditions = append(csr.Status.Conditions, CertificateSigningRequestCondition{
    Type:    CertificateApproved,
    Reason:  "AutoApproved",
    Message: successMessage,
})
_, err = client.CertificatesV1beta1().CertificateSigningRequests().UpdateApproval(csr)

...
...
```

## How to run?

### Running `kube-aws-approver` outside the target cluster

#### 1. Contacting the cluster

`--kubeconfig` should point to a valid kubeconfig file that has access CertificateSigningRequest.

#### 2. Contacting the AWS API

`--region-name` should be set to the AWS region where the instances and autoscalinggroups are running.

The `kube-aws-approver` uses the following priority for the source of credentials to contact AWS API. `ENV` variables > `~/.aws/` config file > `EC2Role` provided credentials.

#### 3. Setting the ASG whitelist

`--allowed-asgs` should contain a comma separated list of the names of AutoScalingGroups that instances requesting to join the cluster can belong to.


#### 4. Running the approver

TODO(abhinav): run as a container.

### Running `kube-aws-approver` inside the target cluster

#### 1. Contacting the cluster

Either `--kubeconfig` should point to a valid kubeconfig file that has access CertificateSigningRequest. Or
If the `--kubeconfig` is empty, `kube-aws-approver` will use InCluster Config to contact the apiserver.

#### 2. Contacting the AWS API

Either `--region-name` should be set to the AWS region where the instances and autoscalinggroups are running. Or
If the `--region-name` is empty, `kube-aws-approver` will fetch the AWS region from instance metadata service.

The `kube-aws-approver` uses the following priority for the source of credentials to contact AWS API. `ENV` variables > `~/.aws/` config file > `EC2Role` provided credentials. Recommended approach is to use EC2Role as credentials.

#### 3. Setting the ASG whitelist

`--allowed-asgs` should contain a comma separated list of the names of AutoScalingGroups that instances requesting to join the cluster can belong to.

#### 4. Running the approver

TODO(abhinav): run as a deployment, with service account, cluster role bindings and config map for allowed ASGs.


[tls-bootstrapping]: https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/
[aws-authenticator]: https://github.com/heptio/authenticator