Production Capstone: Multi-Tenant SaaS Backend

Evolve the ticket API into a production-shaped multi-tenant SaaS backend. Add standards-based identity, resource authorization, object-storage attachments, durable background jobs, optional Redis capacity controls, telemetry, secure delivery, and recovery. The system must prove isolation and failure behavior under concurrency, dependency loss, duplicate messages, and rollback.

What will you be able to demonstrate?

• authenticate users and authorize tenant resources;
• process safe file uploads and background jobs;
• use Redis only for justified, reconstructable ephemeral state;
• observe, deploy, migrate, back up, restore, and recover;
• protect fair capacity and cost across tenants.

What are the required capabilities?

1. Organizations, users, memberships, roles, and resource policies.
2. OIDC/OAuth-based identity or secure server sessions using a real development identity provider.
3. Ticket comments, assignment, audit history, optimistic concurrency, and idempotent creation.
4. Direct or streamed attachment upload through quarantine, scan stub, parse stub, and publish states.
5. Durable jobs with leases, retries, dead letters, cancellation, and SSE progress.
6. Redis-backed rate/concurrency limit and one measured cache; PostgreSQL remains truth.
7. Structured logs, metrics, traces, SLOs, dashboards, and symptom-based alerts.
8. Secure container, CI gates, migration job, managed-container deployment plan, canary, and rollback.
9. PostgreSQL/object backup plus an isolated restore drill; Redis reconstruction plan.

Which threats must the capstone test?

• horizontal tenant access through guessed ticket/document/job IDs;
• vertical privilege escalation from member to tenant admin;
• cache key missing tenant or role variation;
• forged tenant in job payload or upload object key;
• duplicate idempotency key with different request;
• attachment path/content spoofing and oversize input;
• spoofed proxy/request headers;
• Redis unavailable during cache and quota checks;
• database pool exhaustion and provider-independent queue backlog;
• leaked fake secret in logs, image, error body, or telemetry.

What failures must be rehearsed?

What evidence is required?

• data-flow threat model and privilege matrix;
• tenant-isolation matrix across SQL, Redis, queue, storage, stream, logs, and exports;
• architecture and three ADRs: Redis adoption, queue choice, managed platform choice;
• contract/migration compatibility reports;
• load test with noisy tenant and saturation evidence;
• trace of one attachment job across API, storage, queue, and worker;
• CI provenance/scanning output and deployment/rollback record;
• restore report with measured data loss and time.

AI Pair-Programmer Prompt

Rubric

Pass: at least 85%, with zero critical isolation, authorization, durable-data, or secret failures.

Knowledge check

1. Why is Redis allowed here but not required in Capstone 1?
2. What makes a background job durable?
3. Which evidence proves disaster recovery?

---

Answers

1. This system has measured shared capacity/cache/queue requirements and explicit outage behavior.
2. Persistent intent/state, idempotent processing, bounded retries, and observable recovery—not merely a task in memory.
3. A successful isolated restore and service verification measured against RPO/RTO.

Completion checklist

• Isolation matrix passes across all adapters.
• Failure and rollback drills are recorded.
• Redis loss cannot corrupt business truth.
• Production claims link to evidence.

Primary references

• OWASP ASVS
• OpenTelemetry
• Google SRE Workbook