MicroK8s : Observability を有効にする2023/06/22

Observability アドオンを有効にして、Prometheus による MicroK8s クラスターの各種メトリクスの収集を行います。

[1]	プライマリーノードで Observability アドオンを有効にします。

root@dlp:~#

microk8s enable observability

Infer repository core for addon observability
Addon core/dns is already enabled
Addon core/helm3 is already enabled
Addon core/hostpath-storage is already enabled
Enabling observability
Release "kube-prom-stack" does not exist. Installing it now.
NAME: kube-prom-stack
LAST DEPLOYED: Wed Jun 21 23:28:56 2023
NAMESPACE: observability
STATUS: deployed
REVISION: 1
NOTES:
kube-prometheus-stack has been installed. Check its status by running:
kubectl --namespace observability get pods -l "release=kube-prom-stack"

Visit https://github.com/prometheus-operator/kube-prometheus for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator.
Release "loki" does not exist. Installing it now.
NAME: loki
LAST DEPLOYED: Wed Jun 21 23:29:24 2023
NAMESPACE: observability
STATUS: deployed
REVISION: 1
NOTES:
The Loki stack has been deployed to your cluster. Loki can now be added as a datasource in Grafana.

See http://docs.grafana.org/features/datasources/loki/ for more detail.
Release "tempo" does not exist. Installing it now.
NAME: tempo
LAST DEPLOYED: Wed Jun 21 23:29:26 2023
NAMESPACE: observability
STATUS: deployed
REVISION: 1
TEST SUITE: None
Adding argument --authentication-kubeconfig to nodes.
Adding argument --authorization-kubeconfig to nodes.
Restarting nodes.
Adding argument --authentication-kubeconfig to nodes.
Adding argument --authorization-kubeconfig to nodes.
Restarting nodes.
Adding argument --metrics-bind-address to nodes.
Restarting nodes.

Note: the observability stack is setup to monitor only the current nodes of the MicroK8s cluster.
For any nodes joining the cluster at a later stage this addon will need to be set up again.

Observability has been enabled (user/pass: admin/prom-operator)

root@dlp:~#

microk8s kubectl get services -n observability

NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                                                                       AGE
kube-prom-stack-kube-state-metrics         ClusterIP   10.152.183.71    <none>        8080/TCP                                                                                                                      118s
kube-prom-stack-kube-prome-operator        ClusterIP   10.152.183.183   <none>        443/TCP                                                                                                                       118s
kube-prom-stack-grafana                    ClusterIP   10.152.183.189   <none>        80/TCP                                                                                                                        118s
kube-prom-stack-kube-prome-alertmanager    ClusterIP   10.152.183.197   <none>        9093/TCP                                                                                                                      118s
kube-prom-stack-prometheus-node-exporter   ClusterIP   10.152.183.248   <none>        9100/TCP                                                                                                                      118s
kube-prom-stack-kube-prome-prometheus      ClusterIP   10.152.183.59    <none>        9090/TCP                                                                                                                      118s
alertmanager-operated                      ClusterIP   None             <none>        9093/TCP,9094/TCP,9094/UDP                                                                                                    113s
prometheus-operated                        ClusterIP   None             <none>        9090/TCP                                                                                                                      112s
loki-memberlist                            ClusterIP   None             <none>        7946/TCP                                                                                                                      102s
loki-headless                              ClusterIP   None             <none>        3100/TCP                                                                                                                      102s
loki                                       ClusterIP   10.152.183.134   <none>        3100/TCP                                                                                                                      102s
tempo                                      ClusterIP   10.152.183.91    <none>        3100/TCP,16687/TCP,16686/TCP,6831/UDP,6832/UDP,14268/TCP,14250/TCP,9411/TCP,55680/TCP,55681/TCP,4317/TCP,4318/TCP,55678/TCP   100s

root@dlp:~#

microk8s kubectl get pods -n observability

NAME                                                     READY   STATUS    RESTARTS        AGE
kube-prom-stack-kube-prome-operator-64ffd55b77-59922     1/1     Running   0               3m30s
tempo-0                                                  2/2     Running   0               3m12s
kube-prom-stack-prometheus-node-exporter-lw2km           1/1     Running   0               3m30s
kube-prom-stack-prometheus-node-exporter-fz9d2           1/1     Running   0               3m30s
alertmanager-kube-prom-stack-kube-prome-alertmanager-0   2/2     Running   1 (3m15s ago)   3m25s
kube-prom-stack-kube-state-metrics-6c586bf4c8-k4wzv      1/1     Running   0               3m30s
loki-promtail-784k8                                      1/1     Running   0               3m14s
kube-prom-stack-grafana-6c47f548d6-hb9p8                 3/3     Running   0               3m30s
prometheus-kube-prom-stack-kube-prome-prometheus-0       2/2     Running   0               3m24s
loki-promtail-44lvf                                      1/1     Running   0               3m14s
loki-0                                                   1/1     Running   0               3m14s

# 外部からもアクセスできるようにする場合は要ポートフォワードする
# Prometheus UI

root@dlp:~#

microk8s kubectl port-forward -n observability service/prometheus-operated --address 0.0.0.0 9090:9090

Forwarding from 0.0.0.0:9090 -> 9090

# Grafana UI

root@dlp:~#

microk8s kubectl port-forward -n observability service/kube-prom-stack-grafana --address 0.0.0.0 3000:80

Forwarding from 0.0.0.0:3000 -> 3000

[2]	ローカルネットワーク内の任意のクライアントコンピューターで Web ブラウザーを起動し、 [http://(MicroK8s プライマリーノードのホスト名または IP アドレス):(設定したポート)/] にアクセスして、Prometheus や Grafana の UI が表示されれば OK です。なお、Grafana の [デフォルトユーザー/パスワード] は [admin/prom-operator] です。