Your Task API creates tasks, but that's where it stops. When a task is created, nothing else happens automatically. You want to send email notifications, post to Slack, update a mobile app, log for compliance, and trigger reminder scheduling. But if you add all that logic to the Task API, you've created a monolith that knows about email services, Slack webhooks, mobile push notifications, and audit databases.
This is exactly where event-driven architecture shines. The Task API publishes a single event: "A task was created." It doesn't know or care who's listening. Email service, Slack integration, mobile push, audit logging, and reminder scheduling all subscribe independently. Each service evolves separately. The Task API stays focused on task management.
In this chapter, you'll design event schemas that enable this decoupling and implement the notification fanout pattern that makes it work. The patterns here connect directly to your Module 6 Task API agents, turning isolated API calls into a coordinated system of collaborating services.
Consistent event naming is surprisingly important. When you have dozens of event types flowing through your system, predictable names make debugging, monitoring, and documentation dramatically easier.
The standard pattern is domain.action, using past tense to indicate something that happened:
Why past tense? Events represent facts about things that already happened. They're immutable historical records, not requests to do something. Compare:
This distinction matters when you're designing systems. Events are facts you publish and forget. Commands require acknowledgment and error handling.
Every event in your system needs consistent metadata for tracing, debugging, and compliance. Here's the schema pattern that works at scale:
Output (example event):
These two IDs serve different purposes in distributed tracing:
Correlation ID: Traces a single user request across all services it touches. When a user creates a task through your API, that request gets a correlation ID. The task event, the notification, the Slack message, the audit log entry—all share the same correlation ID so you can reconstruct the complete picture.
Causation ID: Links each event to what directly caused it. This creates an event chain that shows causal relationships, not just temporal relationships.
When something goes wrong, correlation ID shows you everything that happened for a user request. Causation ID shows you why each thing happened.
Here's how your Task API publishes events when tasks are created. The producer integrates with FastAPI's lifespan to ensure proper initialization and cleanup:
Output (console when task created):
The key insight here: the Task API publishes an event and returns immediately. It doesn't wait for email, Slack, or audit services. Those happen asynchronously, triggered by the event.
The power of event-driven architecture becomes clear in the fanout pattern. Multiple services subscribe to the same topic, each processing events for their specific purpose. They operate in separate consumer groups, so each service receives every event independently.
Each service uses a different group.id, ensuring complete independence:
Output:
Output:
For compliance, debugging, and analytics, you often need a complete record of everything that happened. The audit log consumer implements this with a strict append-only pattern:
Output:
Audit log file (/var/log/audit/audit-2025-01-15.jsonl):
Key audit log principles:
You have a choice: put all task events in one topic (task-events) or create separate topics per event type (task-created, task-updated, task-completed).
Recommendation: Start with a single topic per domain (task-events, user-events, order-events). Only split when you have clear performance or isolation requirements.
Your Module 6 Task API agents can now communicate through events. When an orchestrating agent creates a task, it publishes an event that notification agents, scheduling agents, and analytics agents all consume independently.
This architecture means agents evolve independently. Adding a new notification channel (mobile push, SMS) requires only deploying a new consumer—no changes to the Task API or other agents.
You built a kafka-events skill in Chapter 1. Test and improve it based on what you learned.
Ask yourself:
If you found gaps:
Setup: You're designing event schemas for an AI agent system that manages tasks, sends notifications, and maintains audit trails.
Prompt 1: Design a complete event schema
What you're learning: AI will suggest additions like occurred_at for analytics, correlation_id for tracing, causation_id for event chains, and metadata.source for debugging. You'll see how requirements translate to schema decisions.
Prompt 2: Troubleshoot a fanout issue
What you're learning: AI helps you understand how auto.offset.reset: latest means you miss events that occurred before the consumer started. For audit logging, you likely want earliest to never miss an event.
Prompt 3: Extend for your domain
What you're learning: AI helps translate generic patterns to your specific domain, suggesting notification channels appropriate for healthcare (pager, EHR integration) vs e-commerce (email, SMS) and compliance requirements (HIPAA vs PCI-DSS).
Safety note: When publishing events containing user data, consider GDPR and privacy requirements. Events in Kafka are retained based on topic configuration and may need data masking or separate topics for sensitive information.