K8s Notes: различия между версиями
Материал из noname.com.ua
Перейти к навигацииПерейти к поискуSirmax (обсуждение | вклад) |
Sirmax (обсуждение | вклад) |
||
| Строка 1: | Строка 1: | ||
[[Категория:k8s]] |
[[Категория:k8s]] |
||
| + | [[Категория:MCC]] |
||
| + | [[Категория:MOSK]] |
||
| + | [[Категория:MIRANTIS]] |
||
=Заметки= |
=Заметки= |
||
Это заметки о том что могу забыть что бы не искать долго |
Это заметки о том что могу забыть что бы не искать долго |
||
| + | <BR>Касаются в основном продуктов Mirantis (MCC/MOS/MOSK) |
||
=PDB= |
=PDB= |
||
| − | Проблема удаления ноды (drain): <BR> |
+ | Проблема удаления ноды (drain): <BR> |
| + | Иногда upgrade может зависнутьи висеть бесконечно долго. Эту проблему вызывает <code>PolicyDesruptionBudget</code><BR> |
||
| ⚫ | |||
| + | |||
| ⚫ | |||
| + | В логах это выглядит так |
||
| + | <code>kubectl -n kaas logs lcm-lcm-controller-67b54d6855-68mcl --since=50m | grep -iE "evict|pdb|budget|drain" --color</code> |
||
| + | <PRE> |
||
| + | {"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"workload-xzk4w","needCordonDrain":true} |
||
| + | {"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kng-kubecondemo-auto-upg-node-bfcbf","needCordonDrain":true} |
||
| + | {"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kubeflow-nodes-tq2q7","needCordonDrain":true} |
||
| + | evicting pod knative-serving/webhook-6975ccdbf4-wd6fm |
||
| + | error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget. |
||
| + | evicting pod knative-serving/webhook-6975ccdbf4-wd6fm |
||
| + | error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget. |
||
| + | evicting pod knative-serving/webhook-6975ccdbf4-wd6fm |
||
| + | error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget. |
||
| + | {"level":"info","ts":"2023-03-24T13:25:31Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"workload-xzk4w","needCordonDrain":true} |
||
| + | {"level":"info","ts":"2023-03-24T13:25:31Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kng-kubecondemo-auto-upg-node-bfcbf","needCordonDrain":true} |
||
| + | </PRE> |
||
Версия 15:31, 24 марта 2023
Заметки
Это заметки о том что могу забыть что бы не искать долго
Касаются в основном продуктов Mirantis (MCC/MOS/MOSK)
PDB
Проблема удаления ноды (drain):
Иногда upgrade может зависнутьи висеть бесконечно долго. Эту проблему вызывает PolicyDesruptionBudget
- PolicyDesruptionBudget: https://cloud.google.com/anthos/clusters/docs/bare-metal/latest/troubleshooting/remove-node-issues
В логах это выглядит так
kubectl -n kaas logs lcm-lcm-controller-67b54d6855-68mcl --since=50m | grep -iE "evict|pdb|budget|drain" --color
{"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"workload-xzk4w","needCordonDrain":true}
{"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kng-kubecondemo-auto-upg-node-bfcbf","needCordonDrain":true}
{"level":"info","ts":"2023-03-24T13:25:18Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kubeflow-nodes-tq2q7","needCordonDrain":true}
evicting pod knative-serving/webhook-6975ccdbf4-wd6fm
error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
evicting pod knative-serving/webhook-6975ccdbf4-wd6fm
error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
evicting pod knative-serving/webhook-6975ccdbf4-wd6fm
error when evicting pods/"webhook-6975ccdbf4-wd6fm" -n "knative-serving" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
{"level":"info","ts":"2023-03-24T13:25:31Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"workload-xzk4w","needCordonDrain":true}
{"level":"info","ts":"2023-03-24T13:25:31Z","logger":"lcmcluster-controller","msg":"holding off machine update","machine":"kng-kubecondemo-auto-upg-node-bfcbf","needCordonDrain":true}
