Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

Kubernetes-operator

  • https://kubernetes.io/ko/docs/concepts/extend-kubernetes/operator/
  • Info

    Operator는 특정 애플리케이션 컨트롤러 인 Kubernetes API를 확장하기 위해 회사에서 개발   CoreOS에 의해 생성됩니다 . 

    데이터베이스, 캐싱 및 모니터링 시스템과 같은 복잡한 상태 저장 애플리케이션을 생성, 구성 및 관리하는 데 사용됩니다. 

    Operator는 Kubernetes의 리소스 및 컨트롤러 개념을 기반으로하지만 일부 애플리케이션 별 전문 지식도 포함합니다.

    예를 들어 데이터베이스 운영자를 생성하려면 생성 된 데이터베이스의 다양한 운영 및 유지 관리 방법에 매우 익숙해야합니다. 

    Operator를 만드는 열쇠는 CRD (custom resources)의 디자인입니다.

    Image Added

ServiceMonitor

Summary

  • kube-prometheus-stack collects Kubernetes manifests, Grafana dashboards, and Prometheus rules combined with documentation and scripts to provide easy to operate end-to-end Kubernetes cluster monitoring with Prometheus using the Prometheus Operator.
  • Installs the kube-prometheus stack, a collection of Kubernetes manifests, Grafana dashboards, and Prometheus rules combined with documentation and scripts to provide easy to operate end-to-end Kubernetes cluster monitoring with Prometheus using the Prometheus Operator.

Dependencies

By default this chart installs additional, dependent charts:

Guide for kube-prometheus-stack with Helm3.x

...

  • Prometheus Stacks에서 사용가능한 StorageClass는 AzureDisk이므로 'default 또는 managed-premium'을 사용해야 합니다.

...

Code Block
sansae@win10pro-worksp:/workspaces$ kubectl get alertmanager -n monitor-po
NAME                                 VERSION   REPLICAS   AGE
kube-prometheus-stack-alertmanager   v0.21.0   1          6d3h


sansae@win10pro-worksp:/workspaces$ kubectl edit alertmanager kube-prometheus-stack-alertmanager -n monitor-po
=============================================================
  storage:
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 2Gi
        storageClassName: managed-premium
=============================================================
alertmanager.monitoring.coreos.com/kube-prometheus-stack-alertmanager edited


sansae@win10pro-worksp:/workspaces$ kubectl get pvc -n monitor-po
NAME                                                                                                   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
alertmanager-kube-prometheus-stack-alertmanager-db-alertmanager-kube-prometheus-stack-alertmanager-0   Bound    pvc-5e7b02eb-7109-417d-9c14-dec0d335f50cxxxxxxxx   2Gi        RWO            managed-premium   6d2h

...

Code Block
sansae@win10pro-worksp:/workspaces$ vim grafana-pvc.yaml
=============================================================
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: grafana-pv-claim
  labels:
    app: grafana
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: managed-premium
=============================================================
sansae@win10pro-worksp:/workspaces$ kubectl create -f grafana-pvc.yaml
create pvc!!!



sansae@win10pro-worksp:/workspaces$ kubectl editget deploy kube-prometheus-stack-grafana -n monitor-po
=============================================================
400pga
NAME                                  - mountPath: /var/lib/grafana
401   READY   UP-TO-DATE   AVAILABLE  name: grafana-persistent-storage
-------------------------------------------------------
453   AGE
kube-prometheus-stack-grafana              1/1     -1 name: grafana-persistent-storage
454         persistentVolumeClaim:
455 1          claimName: grafana-pv-claim
== 6d3h


sansae@win10pro-worksp:/workspaces$ kubectl edit deploy kube-prometheus-stack-grafana -n monitor-po
===========================================================
deployment.apps/kube-prometheus-stack-grafana edited


sansae@win10pro-worksp:/workspaces$ kubectl get pvc -n monitor-po
NAME==
       volumeMounts:
       - mountPath: /var/lib/grafana
                  name: grafana-persistent-storage
-------------------------------------------------------
       volumes:
       - name: grafana-persistent-storage
         persistentVolumeClaim:
                                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
grafana-pv-claim              claimName: grafana-pv-claim
=============================================================
deployment.apps/kube-prometheus-stack-grafana edited


sansae@win10pro-worksp:/workspaces$ kubectl get pvc -n monitor-po
NAME                  STATUS   VOLUME                                 CAPACITY   ACCESS MODES   STORAGECLASS      AGE
grafana-pv-claim      Bound    pvc-a13463fa-ebda-4632-a9c0-60fd56570ae3xxxxxxxx   1Gi        RWO            managed-premium   6d2h

...

Code Block
sansae@win10pro-worksp:$ kubectl port-forward service/kube-prometheus-stack-alertmanager 9093:9093 -n monitor-po
Forwarding from 127.0.0.1:9093 -> 9093
Forwarding from [::1]:9093 -> 9093 -> 9093

Image Removed

참고: 삭제할 경우 namespace이외에 삭제 해야할 리소스

Image Added


참고: 삭제할 경우 namespace이외에 삭제 해야할 리소스

Code Block
for n in $(kubectl get namespaces -o jsonpath={..metadata.name}); do
  kubectl delete --all --namespace=$n prometheus,servicemonitor,podmonitor,alertmanager
done

Code Block
kubectl delete crd alertmanagerconfigs.monitoring.coreos.com
kubectl delete crd alertmanagers.monitoring.coreos.com
kubectl delete crd podmonitors.monitoring.coreos.com
kubectl delete crd probes.monitoring.coreos.com
kubectl delete crd prometheuses.monitoring.coreos.com
kubectl delete crd prometheusrules.monitoring.coreos.com
kubectl delete crd servicemonitors.monitoring.coreos.com
kubectl delete crd thanosrulers.monitoring.coreos.com

kubectl delete clusterrole kube-prometheus-stack-grafana-clusterrole
kubectl delete clusterrole kube-prometheus-stack-kube-state-metrics
kubectl delete clusterrole kube-prometheus-stack-operator
kubectl delete clusterrole kube-prometheus-stack-operator-psp
kubectl delete clusterrole kube-prometheus-stack-prometheus
kubectl delete clusterrole kube-prometheus-stack-prometheus-psp
kubectl delete clusterrole psp-kube-prometheus-stack-kube-state-metrics
kubectl delete clusterrole psp-kube-prometheus-stack-prometheus-node-exporter

kubectl delete clusterrolebinding kube-prometheus-stack-grafana-clusterrolebinding
kubectl delete clusterrolebinding kube-prometheus-stack-kube-state-metrics
kubectl delete clusterrolebinding kube-prometheus-stack-operator
kubectl delete clusterrolebinding kube-prometheus-stack-operator-psp
kubectl delete clusterrolebinding kube-prometheus-stack-prometheus
kubectl delete clusterrolebinding kube-prometheus-stack-prometheus-psp
kubectl delete clusterrolebinding psp-kube-prometheus-stack-kube-state-metrics
kubectl delete clusterrolebinding psp-kube-prometheus-stack-prometheus-node-exporter

kubectl delete svc kube-prometheus-stack-coredns -n kube-system
kubectl delete svc kube-prometheus-stack-kube-controller-manager -n kube-system
kubectl delete svc kube-prometheus-stack-kube-etcd -n kube-system
kubectl delete svc kube-prometheus-stack-kube-proxy -n kube-system
kubectl delete svc kube-prometheus-stack-kube-scheduler -n kube-system
kubectl delete svc kube-prometheus-stack-kubelet -n kube-system
kubectl delete svc prometheus-kube-prometheus-kubelet -n kube-system

kubectl delete MutatingWebhookConfiguration kube-prometheus-stack-admission
kubectl delete ValidatingWebhookConfiguration kube-prometheus-stack-admission

...

Info

https://containerjournal.com/topics/container-management/cluster-monitoring-with-prometheus-operator/

https://kangwoo.kr/2020/06/20/kubernetes-monitoring-3-kube-prometheus-%ec%84%a4%ec%b9%98-%ed%9b%84-%ed%95%b4%ec%95%bc-%ed%95%a0%ec%9d%bc/

https://waspro.tistory.com/588

https://strange-developer.tistory.com/53

https://docs.syseleven.de/metakube/de/metakube-accelerator/building-blocks/observability-monitoring/kube-prometheus-stack

https://kangwoo.kr/2020/06/13/kubernetes-monitoring-1-kube-prometheus-%EC%84%A4%EC%B9%98%ED%95%98%EA%B8%B0/