KubeEye is an inspection software for Kubernetes to find Kubernetes sources (by OPA ), cluster elements, cluster nodes (by Node-Downside-Detector) and different configurations are assembly with greatest practices, and giving options for modification.
KubeEye helps customized inspection guidelines and plugins set up. By means of KubeEye Operator, you’ll be able to view the inspection outcomes and modification options by the graphical show on the net web page.
Structure
KubeEye get cluster useful resource particulars by the Kubernetes API, examine the useful resource configurations by inspection guidelines and plugins, and generate inspection outcomes. See Structure for particulars.
How you can use
-
Set up KubeEye in your machine
-
Obtain pre constructed executables from Releases.
-
Or you’ll be able to construct from supply code
Word: make set up will create kubeeye in /usr/native/bin/ in your machine.
git clone https://github.com/kubesphere/kubeeye.git
cd kubeeye
make installke -
-
[Optional] Set up Node-problem-Detector
Word: This can set up npd in your cluster, solely required in order for you detailed report.
Word: The outcomes of kubeeye type by useful resource sort.
kubeeye audit
KIND NAMESPACE NAME REASON LEVEL MESSAGE
Node docker-desktop kubelet has no enough reminiscence accessible warning KubeletHasNoSufficientMemory
Node docker-desktop kubelet has no enough PID accessible warning KubeletHasNoSufficientPID
Node docker-desktop kubelet has disk strain warning KubeletHasDiskPressure
Deployment default testkubeeye NoCPULimits
Deployment default testkubeeye NoReadinessProbe
Deployment default testkubeeye NotRunAsNonRoot
Deployment kube-system coredns NoCPULimits
Deployment kube-system coredns ImagePullPolicyNotAlways
Deployment kube-system coredns NotRunAsNonRoot
Deployment kubeeye-system kubeeye-controller-manager ImagePullPolicyNotAlways
Deployment kubeeye-system kubeeye-controller-manager NotRunAsNonRoot
DaemonSet kube-system kube-proxy NoCPULimits
DaemonSet okay ube-system kube-proxy NotRunAsNonRoot
Occasion kube-system coredns-558bd4d5db-c26j8.16d5fa3ddf56675f Unhealthy warning Readiness probe failed: Get "http://10.1.0.87:8181/prepared": dial tcp 10.1.0.87:8181: join: connection refused
Occasion kube-system coredns-558bd4d5db-c26j8.16d5fa3fbdc834c9 Unhealthy warning Readiness probe failed: HTTP probe failed with statuscode: 503
Occasion kube-system vpnkit-controller.16d5ac2b2b4fa1eb BackOff warning Again-off restarting failed container
Occasion kube-system vpnkit-controller.16d5fa44d0502641 BackOff warning Again-off restarting failed container
Occasion kubeeye-system kubeeye-controller-manager-7f79c4ccc8-f2njw.16d5fa3f5fc3229c Failed warning Failed to drag picture "controller:newest": rpc error: code = Unknown desc = Error response from daemon: pull entry denied for controller, repository doesn't exist or might require 'docker login': denied: requested entry to the useful resource is denied
Occasion kubeeye-system kubeeye-controller-manager-7f79c4ccc8-f2njw.16d5fa3f61b28527 Failed warning Error: ImagePullBackOff
Position kubeeye-system kubeeye-leader-election-role CanDeleteResources
ClusterRole kubeeye-manager-role CanDeleteResources
ClusterRole kubeeye-manager-role CanModifyWorkloads
ClusterRole vpnkit-controller CanImpersonateUser
ClusterRole vpnkit-controller CanDeleteResourcesWhat KubeEye can do
- KubeEye inspects cluster sources in accordance with Kubernetes greatest practices, to make cluster steady.
- KubeEye can discover issues of your cluster management airplane, together with kube-apiserver/kube-controller-manager/etcd, and many others.
- KubeEye helps you detect every kind of cluster nodes issues, together with reminiscence/cpu/disk strain, surprising kernel error logs, and many others.
Guidelines
| YES/NO | CHECK ITEM | Description | Degree |
|---|---|---|---|
|
✅ |
PrivilegeEscalationAllowed | Privilege escalation is allowed | hazard |
|
✅ |
CanImpersonateUser | The position/clusterrole can impersonate different consumer | warning |
|
✅ |
CanModifyResources | The position/clusterrole can delete kubernetes sources | warning |
|
✅ |
CanModifyWorkloads | The position/clusterrole can modify kubernetes workloads | warning |
|
✅ |
NoCPULimits | The useful resource doesn’t set limits of CPU in containers.sources | hazard |
|
✅ |
NoCPURequests | The useful resource doesn’t set requests of CPU in containers.sources | hazard |
|
✅ |
HighRiskCapabilities | Have high-Threat choices in capabilities akin to ALL/SYS_ADMIN/NET_ADMIN | hazard |
|
✅ |
HostIPCAllowed | HostIPC Set to true | hazard |
|
✅ |
HostNetworkAllowed | HostNetwork Set to true | hazard |
|
✅ |
HostPIDAllowed | HostPID Set to true | hazard |
|
✅ |
HostPortAllowed | HostPort Set to true | hazard |
|
✅ |
ImagePullPolicyNotAlways | Picture pull coverage not all the time | warning |
|
✅ |
ImageTagIsLatest | The picture tag is newest | warning |
|
✅ |
ImageTagMiss | The picture tag don’t declare | hazard |
|
✅ |
InsecureCapabilities | Have insecure choices in capabilities akin to KILL/SYS_CHROOT/CHOWN | hazard |
|
✅ |
NoLivenessProbe | The useful resource doesn’t set livenessProbe | warning |
|
✅ |
NoMemoryLimits | The useful resource doesn’t set limits of reminiscence in containers.sources | hazard |
|
✅ |
NoMemoryRequests | The useful resource doesn’t set requests of reminiscence in containers.sources | hazard |
|
✅ |
NoPriorityClassName | The useful resource doesn’t set priorityClassName | ignore |
|
✅ |
PrivilegedAllowed | Operating a pod in a privileged mode signifies that the pod can entry the host’s sources and kernel capabilities | hazard |
|
✅ |
NoReadinessProbe | The useful resource doesn’t set readinessProbe | warning |
|
✅ |
NotReadOnlyRootFilesystem | The useful resource doesn’t set readOnlyRootFilesystem to true | warning |
|
✅ |
NotRunAsNonRoot | The useful resource doesn’t set runAsNonRoot to true, perhaps executed run as a root account | warning |
|
✅ |
CertificateExpiredPeriod | Certificates expiration date lower than 30 days | hazard |
|
✅ |
EventAudit | Occasion audit | warning |
|
✅ |
NodeStatus | node standing audit | warning |
|
✅ |
DockerStatus | docker standing audit | warning |
|
✅ |
KubeletStatus | kubelet standing audit | warning |
Add your personal inspection guidelines
Add customized OPA guidelines
- Add customized OPA guidelines recordsdata
Word: the OPA rule for workloads, bundle title should be kubeeye_workloads_rego
for RBAC, bundle title should be kubeeye_RBAC_rego
for nodes, bundle title should be kubeeye_nodes_rego
- Save the next guidelines to rule file akin to imageRegistryRule.rego to examine the picture registry tackle complies with guidelines.
bundle kubeeye_workloads_regodeny[msg] {
useful resource := enter
kind := useful resource.Object.sort
resourcename := useful resource.Object.metadata.title
resourcenamespace := useful resource.Object.metadata.namespace
workloadsType := {"Deployment","ReplicaSet","DaemonSet","StatefulSet","Job"}
workloadsType[type]
not workloadsImageRegistryRule(useful resource)
msg := {
"Identify": sprintf("%v", [resourcename]),
"Namespace": sprintf("%v", [resourcenamespace]),
"Kind": sprintf("%v", [type]),
"Message": "ImageRegistryNotmyregistry"
}
}
workloadsImageRegistryRule(useful resource) {
regex.match("^myregistry.public.kubesphere/fundamental/.+", useful resource.Object.spec.template.spec.containers[_].picture)
}
- Run KubeEye with customized guidelines
Word: Specify the trail then Kubeeye will learn all recordsdata within the listing that finish with .rego.
root:# kubeeye audit -p ./opa
NAMESPACE NAME KIND MESSAGE
default nginx1 Deployment [ImageRegistryNotmyregistry NotReadOnlyRootFilesystem NotRunAsNonRoot]
default nginx11 Deployment [ImageRegistryNotmyregistry PrivilegeEscalationAllowed HighRiskCapabilities HostIPCAllowed HostPortAllowed ImagePullPolicyNotAlways ImageTagIsLatest InsecureCapabilities NoPriorityClassName PrivilegedAllowed NotReadOnlyRootFilesystem NotRunAsNonRoot]
default nginx111 Deployment [ImageRegistryNotmyregistry NoCPULimits NoCPURequests ImageTagMiss NoLivenessProbe NoMemoryLimits NoMemoryRequests NoPriorityClassName NotReadOnlyRootFilesystem NoReadinessProbe NotRunAsNonRoot]Add customized NPD guidelines
kubectl edit ConfigMap node-problem-detector-config -n kube-system kubectl rollout restart DaemonSet node-problem-detector -n kube-systemKubeEye Operator
What’s KubeEye Operator
KubeEye Operator is an inspection platform for Kubernetes, handle KubeEye by operator and generate inspection consequence.
What KubeEye Operator can do
- KubeEye Operator gives administration features by way of net web page.
- KubeEye Operator recode inspection outcomes by CR, can view and evaluate cluster inspection outcomes by net web page.
- KubeEye Operator gives extra plugins.
- KubeEye Operator gives extra detailed modification options.
deploy Kubeeye
kubectl apply -f https://uncooked.githubusercontent.com/kubesphere/kubeeye/fundamental/deploy/kubeeye.yaml
kubectl apply -f https://uncooked.githubusercontent.com/kubesphere/kubeeye/fundamental/deploy/kubeeye_insights.yamlget the inspection outcomes
kubectl get clusterinsight -o yamlapiVersion: v1
gadgets:
- apiVersion: kubeeye.kubesphere.io/v1alpha1
sort: ClusterInsight
metadata:
title: clusterinsight-sample
namespace: default
spec:
auditPeriod: 24h
standing:
auditResults:
auditResults:
- resourcesType: Node
resultInfos:
- namespace: ""
resourceInfos:
- gadgets:
- stage: warning
message: KubeletHasNoSufficientMemory
purpose: kubelet has no enough reminiscence accessible
- stage: warning
message: KubeletHasNoSufficientPID
purpose: kubelet has no enough PID accessible
- stage: warning
message: KubeletHasDiskPressure
purpose: kubelet has disk strain
title: kubeeyeNodePaperwork
