K8s-pki
Заметка о PKI/CA в kubernetes
Эта заметка появилась для того что бы раз и навсегда прояснить какие CA и сертификаты используются в control-plane, для чего они используются и почему именно такие.
Предполагается что у читающего есть базовое понимание TLS/SSL/CA/PKI/OpenSSL
Julia Evans - мой герой! https://jvns.ca/blog/2017/08/05/how-kubernetes-certificates-work/
Далее - частичный перевод а частично мои собственные мысли.
Почему возникла эта заметка
Официальная документация гласит:
Kubernetes requires PKI for the following operations:
- Client certificates for the kubelet to authenticate to the API server
- Server certificate for the API server endpoint
- Client certificates for administrators of the cluster to authenticate to the API server
- Client certificates for the API server to talk to the kubelets
- Client certificate for the API server to talk to etcd
- Client certificate/kubeconfig for the controller manager to talk to the API server
- Client certificate/kubeconfig for the scheduler to talk to the API server.
- Client and server certificates for the front-proxy
Note: front-proxy certificates are required only if you run kube-proxy to support an extension API server.
etcd also implements mutual TLS to authenticate clients and peers.
При этом я не нашел никакой информации о том каким свойствами должны обладать сертификаты, должны ли они быть подписаны одним CA?
Вот что пишет по этому поводу Джулия (перевод мой):
В различных компонентах Kubernetes есть ТЫЩИ разных мест, где вы можете разместить a certificate/certificate authority. Когда мы настраивали кластер, я чувствовал, что существует около 10 миллиардов различных аргументов командной строки для сертификатов, ключей и центров сертификации, и я не понимал, как все они сочетаются друг с другом. На самом деле нет 10 миллиардов аргументов командной строки, но их довольно много. Например! Давайте просто посмотрим на аргументы командной строки для сервера API.
У API сервера 16 (ШЕСТНАДЦАТЬ, КАРЛ) аргументов которые так или иначе относятся к шифрованию:
--cert-dir string
The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")--client-ca-file string
If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.--etcd-certfile string
SSL certification file used to secure etcd communication.--etcd-keyfile string
SSL key file used to secure etcd communication.--etcd-cafile string
SSL Certificate Authority file used to secure etcd communication.--kubelet-certificate-authority string
Path to a cert file for the certificate authority.--kubelet-client-certificate string
Path to a client cert file for TLS.--kubelet-client-key string
Path to a client key file for TLS.--proxy-client-cert-file string
Client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins. It is expected that this cert includes a signature from the CA in the --requestheader-client-ca-file flag. That CA is published in the 'extension-apiserver-authentication' configmap in the kube-system namespace. Components recieving calls from kube-aggregator should use that CA to perform their half of the mutual TLS verification.--proxy-client-key-file string
Private key for the client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins.--requestheader-allowed-names stringSlice
List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.--requestheader-client-ca-file string
Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers--service-account-key-file stringArray
File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used. The specified file can contain multiple keys, and the flag can be specified multiple times with different files.--ssh-keyfile string
If non-empty, use secure SSH proxy to the nodes, using this user keyfile--tls-ca-file string
If set, this certificate authority will used for secure access from Admission Controllers. This must be a valid PEM-encoded CA bundle. Alternatively, the certificate authority can be appended to the certificate provided by --tls-cert-file.--tls-cert-file string
File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.--tls-private-key-file string
File containing the default x509 private key matching --tls-cert-file.--tls-sni-cert-key namedCertKey
A pair of x509 certificate
Отдельно радует именование, блять, параметров: ca-file или cafile???
А может лучше certificate-authority.
Не можем решить и оставим все три варианта!
Четыреждыблядская ярость. На этом месте я подумал что может ну его нахуй и не так уж сильно я хочу изучить Kubernetes.
Еще информация к размышлению (да я тоже был очень удивлен!):
One thing I found surprising about this is – almost everything else in the universe that uses TLS will look in /etc/ssl to find a list of certificate authorities the computer trusts by default. But Kubernetes doesn’t do that, instead it mandates “no, you have to say exactly which CA issued the API server’s TLS cert”.
Другими словами использовать добавленные в систему доверенные CA не получится.
Нужен ли единый CA для кластера (нет)
Такой вопрос возник не только у меня:
Does a Kubernetes cluster have to have a single root certificate authority? (no)
If you read a lot of guides to how to set up Kubernetes, you’ll see a step like “set up a certificate authority”. Kelsey Hightower’s great “kubernetes the hard way” doc has it as Step 2, and the Trusting TLS in a cluster guide says:
Every Kubernetes cluster has a cluster root Certificate Authority (CA). The CA is generally used by cluster components to validate the API server’s certificate, by the API server to validate kubelet client certificates, etc.
The way this basically works is:
Set up a certificate authority Use that certificate authority to generate a bunch of different certificates that you’ll give to different parts of the Kubernetes infrastructure. But what if you don’t want to set up a new certificate authority for each kubernetes cluster? We didn’t want to do this for various reasons that I won’t go into, and at first I was worried that you had to set up a single root cluster CA.
It turns out that this sentence (“every Kubernetes cluster has a [single] cluster root Certificate Authority”) is not true – you can actually use certificates issued by several completely different CAs to manage your Kubernetes cluster and it’s fine.
To be clear – I’m not saying you shouldn’t have a single root certificate for your Kubernetes cluster, I’m just saying you don’t have to if for whatever reason that way doesn’t meet your requirements.
Let’s break down some of these command line arguments about certificates and how they relate to each other. Each of these sections is about one certificate authority you can define. They’re all independent – none of them need to be the same as any other one. (though in practice you will probably want to make some of them the same, you don’t want to be managing like 6 CAs probably).
The API server’s TLS certificate (and certificate authority)
- --tls-cert-file string
File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
- --tls-private-key-filestring
File containing the default x509 private key matching --tls-cert-file.
