Helm Hooks and Lifecycle Management

When you deploy a Helm chart with helm install, Kubernetes doesn't just create resources and declare victory. Before the deployment runs, you might need to initialize a database schema. After the deployment succeeds, you might need to invalidate cached data. Before rolling back, you might need to run data migration reversals. If something fails catastrophically, you might want to trigger an alert or notification.

This is where Helm hooks come in. Hooks are Kubernetes resources (usually Jobs) that execute at specific points in the release lifecycle—before installation, after upgrade, before rollback—giving you fine-grained control over what happens when.

Think of hooks as lifecycle-aware automation. Instead of manually running kubectl exec to initialize databases, you declare what should happen using Helm hooks, and your chart orchestrates the entire sequence automatically.

---

What You'll Learn

This chapter covers 10 concepts in 3 groups:

Prerequisites: You should be comfortable with:

• Kubernetes Jobs and Pods
• kubectl logs and kubectl describe
• Basic Helm install/upgrade/rollback cycle

Why this matters: Without hooks, you'd manually run kubectl exec for database migrations before every deployment. Hooks automate this—your chart becomes self-orchestrating.

Time estimate: 50-60 minutes

---

Concept 1: What Are Helm Hooks?

A Helm hook is a Kubernetes resource (Pod, Job, Deployment, etc.) that Helm executes at a specific point in the release lifecycle. Hooks are declared using annotations in your chart templates.

The Hook Annotation

Every hook requires this annotation:

Output: When you run helm install my-app ./my-chart, Helm detects this annotation and executes the Job BEFORE creating other resources.

The key insight: Hooks are regular Kubernetes resources with special timing. You write them like any other resource, then add the annotation to control WHEN they execute.

---

Concept 2: The 9 Hook Types

Helm defines 9 hook types that correspond to points in the release lifecycle. Understanding when each executes is critical to using hooks correctly.

Hook Execution Timeline

Here's a visual representation of when each hook runs:

The 9 Hook Types Explained

1. pre-install

Runs BEFORE any resources are created during helm install. Use this for initialization work that must complete before the main deployment:

Output: When you run helm install my-app ./chart, this Job runs first, initializing the database schema. Only after it succeeds do other resources deploy.

---

2. post-install

Runs AFTER all resources are created during helm install. Use this for post-deployment configuration, registrations, or notifications:

Output: This Job runs after the Deployment and Service are ready, registering the service in a service registry. The 10-second delay ensures the service has time to start.

---

3. pre-upgrade

Runs BEFORE resources are updated during helm upgrade. Use this for migrations that must happen before the new code runs:

Output: When you run helm upgrade my-app ./chart, this migration Job runs first. The new code version expects the status column, which this hook creates. If migration fails, the upgrade is blocked (preventing incompatible code from running against old schema).

---

4. post-upgrade

Runs AFTER resources are updated during helm upgrade. Use this for cache invalidation, notifications, or health checks:

Output: After the new Deployment is running, this hook connects to Redis and flushes all cached data. The new code version might have different caching logic, so stale cache entries would cause bugs. This ensures a clean cache.

---

5. pre-delete

Runs BEFORE resources are deleted during helm uninstall. Use this for data export, backup, or graceful shutdown sequences:

Output: When you run helm uninstall my-app, this Job backs up the database to a PVC before any resources are removed.

---

6. post-delete

Runs AFTER resources are deleted during helm uninstall. Use this for cleanup tasks or notifications:

Output: After the entire release is deleted, this Job deregisters the service from the service registry.

---

7. pre-rollback

Runs BEFORE rolling back to a previous release. Use this for data state checks or backup creation:

Output: If a deployment fails and you need to helm rollback, this hook runs first, preserving the current database state as a backup.

---

8. post-rollback

Runs AFTER rolling back to a previous release. Use this for recovery validation or notifications:

Output: After rollback completes, this hook verifies the previous version is actually running and healthy before declaring the rollback successful.

---

9. test

A special hook that runs during helm test (not during install/upgrade/delete). Use this for post-deployment testing:

Output: When you run helm test my-app, Helm executes this Job and reports the result. If the test succeeds (exit 0), Helm shows "TEST SUITE: PASSED". If it fails (non-zero exit), Helm shows "TEST SUITE: FAILED" and you know there's a problem.

---

Checkpoint: Hook Types Complete

You've now learned the 9 hook types. Here's the quick reference:

The most common pair: pre-upgrade for database migrations + post-upgrade for cache clearing.

Quick self-check: Can you identify which hook you'd use for...

• Initializing a database before first deploy? (pre-install)
• Running migrations before new code deploys? (pre-upgrade)
• Verifying your app works after deploy? (test)

If yes, continue to learn how to control hook execution.

---

Concept 3: Hook Resources (Usually Jobs)

While Hooks can be any Kubernetes resource, the most common choice is Jobs because:

1. Jobs have explicit completion semantics: A Job succeeds when it completes (exit 0) and fails when it doesn't.
2. Jobs are idempotent: Running a Job multiple times should be safe.
3. Jobs timeout naturally: If a Job hangs, Kubernetes will eventually kill it.

You CAN use other resources as hooks:

But Jobs are strongly preferred. Use Jobs for all lifecycle hooks.

---

Concept 4: Hook Weights (Execution Order)

When multiple hooks run at the same phase (e.g., two pre-install hooks), Helm needs to know which runs first. You control this with hook weights:

Output: When you run helm install my-app ./chart, Helm executes:

1. schema-init (weight -5)
2. seed-data (weight 0)
3. verify-init (weight 5)

Rule: Lower weight = earlier execution. Think of weights as "sort order":

• Negative weights run first
• Zero weights run in the middle
• Positive weights run last

Use gaps in weights (e.g., -5, 0, 5) to allow future hooks to insert themselves without renumbering.

---

Concept 5: Hook Delete Policies (Resource Cleanup)

After a hook completes, you probably want to clean up its Pod/Job. You control this with the hook-delete-policy annotation:

The 3 Delete Policies

1. hook-succeeded

Delete the Job ONLY if it succeeds (exit code 0). If it fails, leave it for debugging:

Output: If the Job succeeds, Helm deletes it after completion. If it fails, the Job remains in the cluster so you can inspect its logs with kubectl logs job/db-init.

---

2. hook-failed

Delete the Job ONLY if it fails. If it succeeds, keep it:

Output: Useful if you want to keep evidence of successful migrations for audit purposes. If the Job fails, delete it to avoid cluster clutter.

---

3. before-hook-creation

Delete ANY previous hook with the same name BEFORE creating a new one. Useful for upgrades where the same hook runs multiple times:

Output: On first helm upgrade, this creates the Job. On second helm upgrade, Helm deletes the old Job, then creates a new one. This prevents accumulation of old Jobs in the cluster.

---

Combining Policies

You can combine multiple policies:

Output: This means:

• Delete the Job if it succeeds
• AND delete any previous Job with the same name before creating this one

This is the most common combination for upgrade hooks—clean up old jobs and remove successful ones to keep the cluster tidy.

---

Concept 6: Hook Success/Failure Semantics

What happens when a hook fails? Understanding this is critical to using hooks reliably.

When a Hook Fails

If a hook Job exits with non-zero code (indicating failure), Helm's behavior depends on the hook type:

Pre-Install/Pre-Upgrade Hooks: Release BLOCKED

Example: Your migration hook fails. Helm immediately stops the upgrade and rolls back. The new code never runs. This prevents deploying incompatible code to incompatible database schema.

Output: helm upgrade returns error, release stays at previous version:

---

Post-Install/Post-Upgrade Hooks: Release COMPLETES (but marked degraded)

Example: Your cache-bust hook fails. The upgrade already succeeded (your new code is running). The hook failure is reported but doesn't block the release.

Output: helm upgrade completes, but with warning:

---

Delete Hooks: Release BLOCKING

If a pre-delete hook fails, the release is NOT uninstalled:

Output: helm uninstall returns error, resources remain in cluster:

---

Concept 7: Hook Annotations (The Complete Set)

When declaring a hook, you use these annotations. Here's the full set you need to know:

Common Combinations:

---

Concept 8: Testing Hooks with helm test

The test hook is special—it doesn't run during install/upgrade/delete. Instead, it runs when you explicitly call:

Example test hook:

Output: Running helm test my-app executes this Job and reports the result:

If the test succeeds (exit 0):

If the test fails (non-zero exit):

---

Concept 9: Debugging Hooks with kubectl

When a hook fails, you need to see what went wrong. Hooks are just Kubernetes Jobs, so you debug them like any Job:

List all hook Jobs in the release

Output:

View hook Job logs

Output:

Describe a failed Job to see why it failed

Output:

This tells you the Job ran, the Pod failed, and Helm gave up retrying. Next, check the Pod logs:

Output:

Now you see the problem: The database isn't running or the hostname is wrong.

---

Concept 10: Rollback and Re-execution

One critical thing to understand: When you upgrade a chart with hooks, and the pre-upgrade hook fails, what happens?

The release rolls back to the previous version. The Pods are reverted, and the cluster state is restored.

But hooks themselves are NOT rolled back. If your pre-upgrade migration hook ran and PARTIALLY modified the database (created a column, then failed), that partial state is NOT automatically reverted.

This is why idempotency is critical:

By using idempotent migrations (IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, etc.), you ensure that re-running a failed hook is safe.

---

Exercises

Exercise 1: Pre-Install Hook for Database Schema

Create a pre-install hook that initializes a PostgreSQL database schema:

Create task-api-chart/templates/hooks/db-init.yaml:

Output: Running helm install my-app ./task-api-chart executes this Job first. The database schema is initialized idempotently (IF NOT EXISTS prevents errors if run multiple times). Only after successful schema creation do other resources deploy.

---

Exercise 2: Pre-Upgrade Hook with Weight Ordering

Create two pre-upgrade hooks where the migration runs before validation:

Create task-api-chart/templates/hooks/pre-upgrade.yaml:

Output: When you run helm upgrade my-app ./chart:

1. Migrate hook runs (weight -5)
2. Validate hook runs (weight 0)

If validation fails, the upgrade is blocked and rolled back. The deployment doesn't update.

---

Exercise 3: Post-Upgrade Hook for Cache Invalidation

Create a hook that clears Redis cache after a successful upgrade:

Create task-api-chart/templates/hooks/post-upgrade.yaml:

Output: After helm upgrade my-app ./chart completes, this Job runs. It connects to Redis and clears all cached data. Clients that request data will miss cache, refill from source, and have fresh data for the new deployment version.

---

Exercise 4: Intentional Hook Failure

Create a pre-upgrade hook that intentionally fails to see how Helm handles it:

Create task-api-chart/templates/hooks/debug-failure.yaml:

Steps:

1. Run helm upgrade my-app ./chart
2. Watch the error: Error: release failed: pre-upgrade hook failed
3. Check hook status: kubectl get jobs | grep debug-fail
4. View logs: kubectl logs job/my-app-debug-fail
5. Check release status: helm status my-app

Output: The upgrade is blocked. The release remains at its previous version. The Job shows 0 completions because it failed. When you remove this hook and upgrade again, the release proceeds normally.

---

Exercise 5: Hook Delete Policy (hook-succeeded)

Deploy a hook, let it succeed, and verify it's cleaned up:

Create task-api-chart/templates/hooks/cleanup-test.yaml:

Steps:

1. Run helm install my-app ./chart
2. Wait 10 seconds
3. Check Jobs: kubectl get jobs
4. Notice the Job is GONE (deleted because it succeeded)

Output: The Job appears briefly, runs to completion, and Helm automatically deletes it. Your cluster stays clean.

---

Exercise 6: Hook Weights with Three Jobs

Create three hooks that run in specific order using different weights:

Create task-api-chart/templates/hooks/sequence.yaml:

Output: When you run helm upgrade, the hooks execute in order:

Use kubectl get jobs to see all three Jobs created in sequence.

---

Exercise 7: Checking Hook Execution with kubectl

Examine the hooks deployed in your chart:

Steps:

1. Deploy the chart: helm install my-app ./task-api-chart
2. List all Jobs: kubectl get jobs
3. Get detailed Job info: kubectl get jobs -o wide
4. View a Job's completion: kubectl describe job my-app-db-init
5. Check logs: kubectl logs job/my-app-db-init
6. List hook Events: kubectl get events --sort-by=.metadata.creationTimestamp

Output: You'll see:

• Jobs created for each hook
• Completion status (1/1 means succeeded, 0/1 means failed)
• Logs showing what each hook did
• Events showing the timeline of hook execution

---

Try With AI

Step 1: Exploring Hook Patterns

Ask AI to explain a specific hook use case:

Prompt: "I'm deploying an AI agent to Kubernetes using Helm. Before each upgrade, I need to:

1. Back up the current database
2. Run schema migrations
3. Validate the migration succeeded

Show me a complete pre-upgrade hook sequence using weights and delete policies."

What to evaluate:

• Does the response use three separate Jobs with weights (-10, 0, 10)?
• Does it use idempotent SQL (IF NOT EXISTS, IF NOT PRESENT)?
• Does it set appropriate delete-policy (hook-succeeded,before-hook-creation)?
• Are the Job definitions complete and runnable?

Step 2: Refining for Your Use Case

Based on AI's response, customize it:

Prompt: "I'm using PostgreSQL version 13 and the database name is 'agent_db'. The credentials are in a Secret called 'postgres-creds'. Update the migration hook to match my environment."

What to evaluate:

• Does it reference the correct Secret name?
• Are environment variables set from Secret values?
• Is the SQL appropriate for PostgreSQL 13?

Step 3: Debugging Hook Failures

Ask AI to help debug a failing hook:

Prompt: "My pre-upgrade hook is failing with 'connection refused'. The hook is trying to connect to 'postgres' but the actual service hostname is 'postgres-svc.data-layer'. Show me:

1. What the problem is
2. How to fix the hook
3. How to verify it's fixed"

What to evaluate:

• Does AI identify the hostname mismatch?
• Does it suggest updating Values.yaml for environment-specific hostnames?
• Does it recommend debugging commands (kubectl logs, kubectl describe)?

Step 4: Validating Hook Safety

Ask AI to review a hook you've written:

Prompt: "Review this hook for production safety: [paste your hook YAML]

Check for:

• Idempotency issues
• Missing error handling
• Delete policy appropriateness
• Timeout concerns"

What to evaluate:

• Does AI catch non-idempotent operations (operations that fail if run twice)?
• Does it suggest adding set -e (fail on first error)?
• Does it recommend reasonable timeouts?

Step 5: Integration Testing

Ask AI for a test strategy:

Prompt: "How do I test that my pre-upgrade hook works correctly without actually running helm upgrade in production? Show me a test plan using kind or Minikube."

What to evaluate:

• Does it suggest spinning up a test cluster?
• Does it recommend testing both success and failure cases?
• Does it mention checking hook order with weights?

---

Reflect on Your Skill

You built a helm-chart skill in Chapter 0. Test and improve it based on what you learned.

Test Your Skill

Improve Your Skill

If you found gaps: