Kubernetes Monitoring and Observability on AWS EKS
This project focused on building an end-to-end Kubernetes monitoring and observability workflow for a containerized application deployed on AWS EKS. I installed the monitoring agent, collected Kubernetes and container telemetry, analyzed workload usage, queried application metrics with PromQL, and created custom dashboards to understand performance, resource usage, pod readiness, and network activity.
The goal was to demonstrate how observability helps teams monitor application health, understand workload behavior, identify resource optimization opportunities, and maintain operational visibility across a Kubernetes environment.
What I Built
AWS EKS Cluster → Monitoring Agent → Kubernetes Telemetry → PromQL Metrics → Custom Dashboard → Optimization Insights
1. Monitoring Agent Installation
Installed the monitoring agent on the EKS cluster using Helm to collect Kubernetes, container, workload, infrastructure, and application telemetry.
2. Kubernetes Telemetry Collection
Collected cluster-level and workload-level telemetry, including CPU usage, memory usage, pod readiness, namespace activity, and running workload status.
3. Resource Usage Analysis
Analyzed actual workload usage against resource requests, limits, and available cluster capacity to identify right-sizing and optimization opportunities.
4. Application Metrics with PromQL
Queried application-level metrics using PromQL
5. Custom Observability Dashboard
Created a custom dashboard to monitor application performance, resource usage, pod readiness, database connection usage, and network activity in one place.
6. Operational Visibility
Used the collected metrics and dashboard views to understand application health, cluster capacity, workload behavior, and infrastructure-level performance.
Monitoring Workflows Implemented
1. Cluster Resource Monitoring
Monitored CPU, memory, pod capacity, resource requests, and resource limits across the Kubernetes cluster.
What I learned:
I learned how resource requests and limits help Kubernetes schedule workloads, and how comparing actual usage against reserved resources can reveal over-provisioning or right-sizing opportunities.
Impact:
Helped identify whether the cluster had enough capacity and whether workloads were using resources efficiently.
2. Workload and Namespace Analysis
Reviewed resource usage across workloads and namespaces to understand which components were consuming the most CPU, memory, and pod capacity.
What I learned:
I learned that observability is not only about checking if an application is running, but also understanding how different workloads and namespaces contribute to overall cluster usage.
Impact:
Provided visibility into workload behavior and helped identify where resource usage was concentrated.
3. PromQL-Based Application Metrics
Used PromQL queries to analyze application metrics such as Redis throughput and active clients.
What I learned:
I learned how PromQL helps query application behavior and correlate user activity with backend system performance.
Impact:
Helped validate traffic patterns and understand how application components were behaving over time.
4. Pod Readiness and Health Monitoring
Monitored pod readiness status to confirm whether application pods were available and ready to serve traffic.
What I learned:
I learned how pod readiness is important for Kubernetes application reliability because running pods are not always ready to handle traffic.
Impact:
Helped track application availability and workload health.
5. Host-Level Network Telemetry
Added a host-level network connection metric to observe connection rates from the infrastructure supporting the Kubernetes cluster.
What I learned:
I learned how host-level metrics provide additional infrastructure context beyond pod and container metrics.
Impact:
Expanded the dashboard from workload monitoring to broader Kubernetes infrastructure visibility.
6. Dashboard-Based Observability
Created a custom observability dashboard that brought together application metrics, workload resource usage, pod readiness, database connection usage, and network activity.
What I learned:
I learned how dashboards help convert raw telemetry into clear operational views that engineering and infrastructure teams can use for monitoring and troubleshooting.
Impact:
Created a single view to monitor application performance, health, capacity, and workload behavior.
Key Skills
🌐 Cloud & Infrastructure
AWS, AWS EKS, Load Balancer, Kubernetes, Containers, Helm
📊 Monitoring & Observability
Kubernetes Monitoring, Observability, Telemetry, Metrics, Dashboarding, Resource Usage Analysis, Pod Readiness, Network Activity Monitoring
🔎 Metrics Querying
PromQL
Key Learnings
Kubernetes Observability
Gained hands-on experience using Kubernetes observability to track workload health, resource usage, pod status, and application behavior.
Resource Optimization
Analyzed actual usage against requested resources, limits, and available cluster capacity to identify right-sizing opportunities in Kubernetes environments.
Metrics-Driven Troubleshooting
Applied PromQL-based metric analysis to understand application behavior, validate traffic patterns, and identify potential performance or reliability issues.
Dashboard Design
Built dashboards that combined performance, health, and infrastructure metrics in one centralized view to monitor system behavior, track workload health, and identify potential issues quickly.