Autoscaling with HPA, VPA & KEDA

Module 7 takes the agent you built in Module 6 and turns it into a production cloud service. You'll containerize the stack, orchestrate it on Kubernetes, automate delivery, and operate it with observability, security, and cost controls. The goal: a reliable Digital FTE that runs 24/7 for real users.

Prerequisites: Modules 4-6. You need a working agent service to deploy.

You deployed your Task API with 3 replicas. At 2 AM, all three pods sit idle, consuming resources and costing money. At noon, traffic spikes and 3 replicas cannot keep up—users see latency, requests queue, and eventually fail. Fixed replica counts waste money during quiet periods and fail during busy ones.

Autoscaling matches capacity to demand automatically. Kubernetes provides Horizontal Pod Autoscaler (HPA) for scaling replica counts based on metrics. Vertical Pod Autoscaler (VPA) right-sizes individual pods. For event-driven workloads—like AI agents processing queue messages—KEDA enables scaling based on queue depth, Prometheus metrics, or even scaling to zero when idle.

This lesson teaches you to configure HPA for CPU-based scaling, understand VPA for resource optimization, install KEDA for event-driven autoscaling, and implement scale-to-zero for cost efficiency. By the end, your services will scale up when needed and scale down (or to zero) when idle.

---

How Autoscaling Works in Kubernetes

Kubernetes autoscaling operates through a control loop. A controller periodically checks metrics, compares them to targets, and adjusts replicas or resources accordingly.

The Three Autoscaling Approaches

When Each Approach Applies

---

Horizontal Pod Autoscaler (HPA)

HPA scales the number of pod replicas based on observed metrics. When CPU utilization exceeds 80%, HPA adds more pods. When utilization drops, HPA removes pods.

Prerequisites: Metrics Server

HPA requires metrics-server to provide CPU and memory metrics:

Output (if installed):

If not installed:

Output:

Creating an HPA

Define HPA for the Task API:

Apply and verify:

Output:

The TARGETS column shows current utilization (23%) versus target (70%).

Understanding HPA Fields

Scaling Based on Multiple Metrics

Scale on both CPU and memory:

HPA scales based on whichever metric requires more replicas.

Observing HPA Behavior

Generate load and watch scaling:

Output (terminal 1):

HPA detected CPU exceeding 70% and scaled from 2 to 4 replicas.

HPA Scaling Behavior Configuration

Control how aggressively HPA scales:

Behavior settings:

---

Vertical Pod Autoscaler (VPA)

VPA adjusts CPU and memory requests for pods based on historical usage. Instead of adding more pods, VPA makes existing pods bigger (or smaller).

When VPA Helps

Installing VPA

Output:

Creating a VPA

Update modes:

Viewing VPA Recommendations

Output (recommendations section):

This tells you the pod currently requests too much (or too little) resources. The target is VPA's recommended setting.

VPA Limitations

VPA cannot coexist with HPA on CPU/memory. Both try to control the same resources.

Solutions:

1. Use VPA for recommendations only (updateMode: Off)
2. Use HPA for replica scaling + VPA for right-sizing during deployments
3. Use KEDA (which can work alongside VPA)

---

KEDA: Event-Driven Autoscaling

KEDA (Kubernetes Event-Driven Autoscaling) extends HPA with support for any metric source and scale-to-zero capability. KEDA is essential for:

• Queue-based workers (Kafka, RabbitMQ, SQS)
• Cron-based scaling (scale up at 9 AM, down at 6 PM)
• Prometheus metrics (custom application metrics)
• Cost optimization (scale to zero when idle)

How KEDA Works

KEDA creates and manages HPAs automatically based on ScaledObject definitions.

Installing KEDA

Output:

Verify installation:

Output:

ScaledObject: The Core KEDA Resource

ScaledObject tells KEDA what to scale and based on which metrics:

Apply and verify:

Output:

Understanding ScaledObject Fields

---

Prometheus Scaler

The Prometheus scaler queries your Prometheus server for custom metrics—request rate, queue depth, latency percentiles, or any metric your application exposes.

Scaling Based on Request Rate

Scale based on requests per second:

How it works:

• Query calculates requests per second over the last minute
• When requests exceed 50/second, KEDA adds pods
• Each additional pod handles ~50 requests/second

Scaling Based on Latency

Scale when response times degrade:

When p95 latency exceeds 500ms, KEDA adds pods to reduce load per instance.

Testing Prometheus Scaler

Generate load and observe scaling:

Output:

---

Kafka Scaler for Event-Driven Workloads

For AI agents processing messages from Kafka, scale based on consumer lag—how many unprocessed messages are waiting.

Kafka Consumer Lag Scaling

How it works:

• KEDA queries Kafka for consumer group lag
• When 10+ messages are waiting per partition, scale up
• When queue is empty, scale to zero

Kafka Scaler with Authentication

For production Kafka clusters requiring authentication:

---

Scale-to-Zero Pattern

Scale-to-zero is KEDA's defining feature. When no work exists, why pay for idle pods?

Configuring Scale-to-Zero

Observing Scale-to-Zero

Output:

The pod terminates when there's no work. When new tasks arrive, KEDA scales back up.

Cold Start Considerations

Scale-to-zero introduces cold start latency. The first request waits for:

1. KEDA to detect the metric change
2. Pod scheduling and startup
3. Container initialization
4. Application readiness

Mitigation strategies:

---

Choosing the Right Autoscaler

---

Exercises

Exercise 1: Configure HPA for CPU Scaling

Create HPA for your Task API:

Verify:

Expected Output:

Exercise 2: Install KEDA

Install KEDA in your cluster:

Verify:

Expected Output:

Exercise 3: Create ScaledObject with Prometheus

Configure KEDA to scale based on request rate:

Verify:

Expected Output:

Exercise 4: Observe Scale-to-Zero

With no traffic, watch the deployment scale to zero:

Expected Output (after cooldown):

Generate traffic and watch pods scale up:

Expected Output:

---

Reflect on Your Skill

You built a traffic-engineer skill in Lesson 0. Based on what you learned about autoscaling:

Add Autoscaling Decision Logic

Your skill should ask:

Add ScaledObject Templates

Prometheus-based ScaledObject:

HPA template:

Update Troubleshooting Guidance

---

Try With AI

Generate HPA Configuration

Ask your traffic-engineer skill:

What you're learning: AI generates HPA with behavior configuration. Review the output—did AI include the behavior section with stabilization windows? Are the scaling policies correct for your requirements?

Evaluate and Refine

Check AI's output:

• Does it use autoscaling/v2 (not v1)?
• Is behavior.scaleUp configured for fast scaling?
• Is behavior.scaleDown configured for gradual reduction?
• Are stabilization windows appropriate?

If the scale-down is too aggressive, provide feedback:

Add KEDA ScaledObject

Extend to event-driven scaling:

What you're learning: AI generates KEDA configuration. Verify the Prometheus query is correct and the ScaledObject references the right deployment.

Validate Configuration

Before applying:

This iteration—requirements, generation, validation, refinement—produces production-ready autoscaling configurations.

Safety Note

Autoscaling affects resource consumption and costs. Start with conservative settings (higher minReplicaCount, longer cooldownPeriod) and tune based on observed behavior. Monitor your cluster's node autoscaler to ensure it can provision nodes for scaled-up workloads. Test scale-to-zero behavior in staging before production—cold starts may impact user experience.