USMAN’S INSIGHTS
AI ARCHITECT
  • Home
  • About
  • Thought Leadership
  • Book
Press / Contact
USMAN’S INSIGHTS
AI ARCHITECT
⌘F
HomeBook
HomeBookThe Loyal Sidekick: Extending Agents with Sidecars
Previous Chapter
Init Containers Optional
Next Chapter
Ingress External Access Optional
AI NOTICE: This is the table of contents for the SPECIFIC CHAPTER only. It is NOT the global sidebar. For all chapters, look at the main navigation.

On this page

19 sections

Progress0%
1 / 19

Muhammad Usman Akbar Entity Profile

Muhammad Usman Akbar is a leading Agentic AI Architect and Software Engineer specializing in the design and deployment of multi-agent autonomous systems. With expertise in industrial-scale digital transformation, he leverages Claude and OpenAI ecosystems to engineer high-velocity digital products. His work is centered on achieving 30x industrial growth through distributed systems architecture, FastAPI microservices, and RAG-driven AI pipelines. Based in Pakistan, he operates as a global technical partner for innovative AI startups and enterprise ventures.

USMAN’S INSIGHTS
AI ARCHITECT

Transforming businesses into autonomous AI ecosystems. Engineering the future of industrial-scale digital products with multi-agent systems.

30X Growth
AI-First
Innovation

Navigation

  • Home
  • Book
  • About
  • Contact
Let's Collaborate

Have a Project in Mind?

Let's build something extraordinary together. Transform your vision into autonomous AI reality.

Start Your Transformation

© 2026 Muhammad Usman Akbar. All rights reserved.

Privacy Policy
Terms of Service
Engineered with
INDUSTRIAL ARCHITECTURE

Sidecar Containers: The Agent's Best Friend

You've deployed your agent to Kubernetes. Requests arrive. Your agent handles them. But where do the logs go? Where's the monitoring data? How do you trace which request took how long?

Traditional single-container deployments mix your application logic with logging, monitoring, and debugging concerns. Your agent code becomes cluttered. Operational concerns become tangled with business logic. Testing gets harder.

Sidecars solve this. A sidecar is a helper container that runs alongside your main application container—in the same Pod, sharing the same network and storage. Your agent focuses on its job. The sidecar handles logging, metrics, proxying, or security concerns independently. They coordinate through shared volumes and localhost networking.

This lesson teaches you to design and deploy sidecars using Kubernetes' native sidecar support (available since Kubernetes 1.28). By the end, you'll deploy an agent with a logging sidecar and a metrics sidecar, keeping operational concerns separated from application logic.


The Sidecar Pattern: Separation of Concerns

The Problem: Tangled Concerns

Imagine your FastAPI agent needs to:

  1. Handle incoming requests
  2. Write structured logs to a file
  3. Expose Prometheus metrics
  4. Trace request latency

If all this logic lives in one container, your code becomes messy:

python
@app.post("/inference") async def inference(request: Request): # Business logic result = model.predict(request.text) # Logging concern with open("/var/log/requests.log", "a") as f: f.write(json.dumps({"request": request.text, "result": result})) # Metrics concern inference_counter.inc() inference_latency.observe(time.time() - start) # Return result return {"prediction": result}

Your application code is entangled with operational concerns. Testing requires mocking file I/O and metrics. Scaling requires coordinating all concerns together.

The Solution: Sidecars

Instead, separate concerns into independent containers:

text
┌──────────────────────────────────────────────────┐ │ Pod (Shared Network) │ ├──────────────────────┬──────────────────────────┤ │ │ │ │ Main Container │ Logging Sidecar │ │ (FastAPI Agent) │ │ │ │ - Reads /var/log/agent │ │ @app.post("/") │ - Streams to stdout │ │ Handle requests │ - Or sends to central │ │ │ logging system │ │ │ │ ├──────────────────────┼──────────────────────────┤ │ Metrics Sidecar │ Proxy Sidecar │ │ │ (Optional) │ │ - Scrapes metrics │ │ │ - Exposes on :9090 │ - TLS termination │ │ │ - Service mesh inject │ │ │ - Request routing │ └──────────────────────┴──────────────────────────┘ Shared Volume (/var/log) Shared Network (localhost)

Your agent writes logs to /var/log/agent/requests.log. The logging sidecar watches that file and streams it to a central logging service. Your agent exposes metrics on localhost:8000/metrics. A metrics sidecar scrapes and forwards those metrics.

Each container has one job. Testing is simpler. Scaling is predictable.


Native Sidecars: Kubernetes 1.28+

Before Kubernetes 1.28, sidecars were regular containers in the containers field, but Kubernetes couldn't distinguish them from the main application container. This created ambiguity about startup ordering and lifecycle.

Kubernetes 1.28 introduced the initContainers field with a new restartPolicy: Always option specifically for sidecars:

How It Works

FeatureDescription
Startup OrderInit containers with restartPolicy: Always start before main containers.
PersistenceThey run continuously (unlike traditional init containers that exit after setup).
ResilienceIf a sidecar crashes, Kubernetes restarts it independently.
Non-BlockingMain container startup is NOT blocked by sidecar readiness.

This guarantees:

  • Sidecars are ready before main application starts processing requests.
  • Sidecars stay alive throughout the Pod's lifetime.
  • Main container failures don't restart sidecars unnecessarily.

Concept 1: Writing Logs to Shared Volumes

Your agent and sidecars must communicate. The most reliable method: shared volumes.

Create a Pod with:

  • Main container: Writes logs to /var/log/agent/
  • Logging sidecar: Reads from /var/log/agent/, streams to stdout or external service

Step 1: Create Agent with Log Volume

Create agent-with-logging.yaml:

yaml
apiVersion: v1 kind: Pod metadata: name: agent-with-logging spec: containers: - name: agent image: myregistry.azurecr.io/agent:v1 ports: - containerPort: 8000 volumeMounts: - name: log-volume mountPath: /var/log/agent env: - name: LOG_FILE value: /var/log/agent/requests.log initContainers: - name: logging-sidecar image: fluent/fluent-bit:latest restartPolicy: Always # <-- Native sidecar: Always restarts volumeMounts: - name: log-volume mountPath: /var/log/agent args: - -c - /fluent-bit/etc/fluent-bit.conf volumes: - name: log-volume emptyDir: {}

Key Implementation Points:

ComponentRole
emptyDirTemporary storage, shared across containers in the Pod.
volumeMountsBoth containers mount the same volume at /var/log/agent.
initContainersWith restartPolicy: Always, the logging sidecar starts before the agent and stays running.

Deploy it:

bash
kubectl apply -f agent-with-logging.yaml

Output:

text
pod/agent-with-logging created

Verify the Pod is running:

bash
kubectl get pods

Output:

text
NAME READY STATUS RESTARTS AGE agent-with-logging 2/2 Running 0 5s

The 2/2 shows both containers are ready. Without the sidecar, you'd see 1/1.


Step 2: Verify Log Volume Sharing

Exec into the agent container and create a test log:

bash
kubectl exec -it agent-with-logging -c agent -- sh

Inside the container:

bash
echo '{"request": "test", "result": "success"}' >> /var/log/agent/requests.log cat /var/log/agent/requests.log

Output:

json
{"request": "test", "result": "success"}

Exit:

bash
exit

Now exec into the logging sidecar and read the same file:

bash
kubectl exec -it agent-with-logging -c logging-sidecar -- sh

Inside the sidecar:

bash
cat /var/log/agent/requests.log

Output:

json
{"request": "test", "result": "success"}

Both containers see the same logs. The sidecar can now stream this data to a centralized logging service (e.g., Elasticsearch, Splunk, or Stackdriver).


Concept 2: Multi-Sidecar Pod with Metrics

Most production Pods run multiple sidecars. Let's add a metrics sidecar alongside logging.

Create agent-with-logging-and-metrics.yaml:

yaml
apiVersion: v1 kind: Pod metadata: name: agent-with-sidecars spec: containers: - name: agent image: myregistry.azurecr.io/agent:v1 ports: - containerPort: 8000 name: http - containerPort: 8001 name: metrics # <-- Agent exposes metrics here volumeMounts: - name: log-volume mountPath: /var/log/agent env: - name: LOG_FILE value: /var/log/agent/requests.log initContainers: - name: logging-sidecar image: fluent/fluent-bit:latest restartPolicy: Always volumeMounts: - name: log-volume mountPath: /var/log/agent - name: metrics-sidecar image: prom/prometheus:latest restartPolicy: Always ports: - containerPort: 9090 name: prometheus volumeMounts: - name: prometheus-config mountPath: /etc/prometheus args: - --config.file=/etc/prometheus/prometheus.yml volumes: - name: log-volume emptyDir: {} - name: prometheus-config configMap: name: prometheus-config

Deploy:

bash
kubectl apply -f agent-with-logging-and-metrics.yaml

Output:

text
pod/agent-with-sidecars created

Check Pod status:

bash
kubectl get pods

Output:

text
NAME READY STATUS RESTARTS AGE agent-with-sidecars 3/3 Running 0 8s

Three containers running in one Pod: agent, logging sidecar, metrics sidecar. All share the Pod's network namespace, so the metrics sidecar can reach the agent's metrics endpoint at localhost:8001/metrics without any network configuration.


Concept 3: Sidecar Lifecycle and Ordering

Init Containers with restartPolicy: Always

When you use initContainers with restartPolicy: Always, Kubernetes guarantees:

GuaranteeDescription
Startup OrderInit containers start BEFORE main containers.
IndependenceEach init container runs to completion before the next starts.
Restart BehaviorIf an init container crashes, it restarts (not the entire Pod).
Main Container WaitsMain containers don't start until all init containers are healthy.

Example: Startup Sequence

text
Time 0: Pod created └─ Init container 1 (logging-sidecar) starts Time 1: Logging sidecar ready └─ Init container 2 (metrics-sidecar) starts Time 2: Metrics sidecar ready └─ Main container (agent) starts Time 3: Agent running, processing requests └─ All sidecars handle logging/metrics independently Time 10: Metrics sidecar crashes └─ Kubernetes immediately restarts it └─ Agent continues running (unaffected) Time 15: Metrics sidecar recovered └─ Continues scraping metrics from agent

Shutdown Sequence

When a Pod terminates:

text
Time 0: Pod receives termination signal └─ All containers receive SIGTERM Time 0-30: Containers have grace period to shut down └─ Agent stops accepting requests └─ Sidecars flush any buffered data └─ Connections close gracefully Time 30: Grace period expires └─ Kubernetes sends SIGKILL └─ Pod and all containers terminated

Practical Configuration: Logging Sidecar for Agent Inference

Let's build a realistic logging sidecar for your agent. Your agent logs inference requests and latencies. The sidecar streams these to stdout for collection.

Create agent-inference-logging.yaml:

yaml
apiVersion: v1 kind: Pod metadata: name: agent-inference-logger spec: containers: - name: agent image: myregistry.azurecr.io/agent:v1 ports: - containerPort: 8000 volumeMounts: - name: logs mountPath: /var/log/agent env: - name: LOG_FILE value: /var/log/agent/inference.log initContainers: - name: log-collector image: fluent/fluent-bit:latest restartPolicy: Always volumeMounts: - name: logs mountPath: /var/log/agent - name: fluent-config mountPath: /fluent-bit/etc args: - -c - /fluent-bit/etc/fluent-bit.conf volumes: - name: logs emptyDir: {} - name: fluent-config configMap: name: fluent-bit-config

Deploy:

bash
kubectl apply -f agent-inference-logging.yaml

Output:

text
pod/agent-inference-logger created

View logs from the sidecar:

bash
kubectl logs agent-inference-logger -c log-collector

Output:

text
[2025-12-22 14:35:12.123] [info] [fluent-bit] version 2.1.0, commit hash: abc1234 [2025-12-22 14:35:12.234] [info] starting log collection from /var/log/agent/inference.log [2025-12-22 14:35:23.456] [info] collected: {"request_id": "abc123", "latency_ms": 245, "status": "success"} [2025-12-22 14:35:45.789] [info] collected: {"request_id": "def456", "latency_ms": 198, "status": "success"}

Try With AI

Open a terminal and work through these scenarios with an AI assistant's help:

Scenario 1: Design a Logging Sidecar

Your task: Design a Pod manifest where:

  • Main container: A Python FastAPI agent running on port 8000
  • Sidecar: Fluent Bit collecting logs from /var/log/agent/requests.log and forwarding to an external logging service (e.g., Datadog or CloudWatch)

AI Prompting Guide:

ActionPrompt
Initial Design"Create a Pod manifest with a FastAPI agent and a Fluent Bit sidecar that collects logs and forwards them to [your logging service]."
Security"The logging service requires authentication. I need to pass credentials securely."
Integration"Update the manifest to inject credentials using a Kubernetes Secret."

Scenario 2: Multi-Sidecar Deployment

Your task: Your agent now needs:

  1. Logging sidecar (Fluent Bit)
  2. Metrics sidecar (Prometheus)
  3. Security sidecar (mTLS termination with Linkerd or Istio)

AI Prompting Guide:

ActionPrompt
Complex Manifest"Create a Pod manifest with three sidecars: logging, metrics, and security. Explain the startup and shutdown ordering."
Reliability Check"What happens if the metrics sidecar crashes? Will my agent stop handling requests?"

Scenario 3: Troubleshoot Sidecar Failures

Your task: You deployed a Pod with an agent and logging sidecar. The Pod shows 2/2 READY, but logs aren't being collected.

AI Prompting Guide:

ActionPrompt
Diagnosis"My logging sidecar is running but not collecting logs. The agent container is writing to /var/log/agent/requests.log. What could be wrong?"
Debugging"Show me the exact kubectl commands to debug this."

Source:https://www.muhammadusmanakbar.com/book