kubesphere 설치하기 (Minimal KubeSphere on Kubernetes) + kubesphere Loadbalaner 서비스 + 프로메테우스, 그라파나

Local에 storageclass 생성

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

 

생성한 sc를 default로 설정

kubectl patch storageclass local-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

 

그 다음은 공식문서 참조

https://kubesphere.io/docs/quick-start/minimal-kubesphere-on-k8s/

Minimal KubeSphere on Kubernetes

 

설치 명령어

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yaml
   
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/cluster-configuration.yaml

 

설치 로그 확인

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

 

kubesphere의 ks-console을 edit이나 patch명령어로 loadbalancer로 바꾸려고 하면 시스템 재시작할 때마다 다시 nodeport로 바뀌는 것을 알 수 있다. 이를 위해서는 ks-installer라는 clusterconfiguration을 바꿔줘야 한다.

  common:
    core:
      console:
        enableMultiLogin: true  # Enable or disable simultaneous logins. It allows different users to log in with the same account at the same time.
        port: 30880
        type: LoadBalancer

참고: https://github.com/kubesphere/kubesphere/issues/4423

 

프로메테우스

kubesphere로 설치했더니 자동으로 셋팅되었다. 그렇지만 만약 이미 서버에 프로메테우스 서버가 구축되어있다면 아래의 링크에서 customize할 수 있다.

https://kubesphere.io/docs/faq/observability/byop/

Bring Your Own Prometheus

kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/alertmanager/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/devops/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/etcd/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/grafana/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/kube-state-metrics/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/node-exporter/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/upgrade/ 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/prometheus-rules-v1.16\+.yaml 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/prometheus-rules.yaml 2>/dev/null
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/prometheus 2>/dev/null
# 나의 경우엔 아래를 주석해제하고 명령어 실행
kubectl -n kubesphere-system exec $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -- kubectl delete -f /kubesphere/kubesphere/prometheus/init/ 2>/dev/null

kubectl -n kubesphere-monitoring-system delete pvc `kubectl -n kubesphere-monitoring-system get pvc | grep -v VOLUME | awk '{print $1}' |  tr '\n' ' '`

 

이후 프로메테우스를 설치한다

참고: https://jmholly.tistory.com/entry/Helm-%EC%9C%BC%EB%A1%9C-Prometheus-%EC%84%A4%EC%B9%98?category=996026 

 

그라파나

제 블로그의 helm으로 프로메테우스 설치하는 방법인 위 포스트를 통해 kube prometheus stack을 설치하시면 grafana까지 설치할 수 있습니다. 

 

https://jmholly.tistory.com/entry/Grafana-%EC%84%A4%EC%B9%98-%EB%B0%8F-%EC%85%8B%ED%8C%85%ED%95%98%EA%B8%B0-%ED%94%84%EB%A1%9C%EB%A9%94%ED%85%8C%EC%9A%B0%EC%8A%A4-%EC%97%B0%EB%8F%99