COCONUT LABS  ·  APPLIED AI SYSTEMS  ·  PUBLIC EDITION

Applied AI Systems

A build-first curriculum for engineers who already ship software and want to ship AI systems with the same seriousness. Eight modules and three projects; every concept ends in something that runs, and every claim about quality ends in a measurement.

0 / 0 · 0%
HOW TO RUN THIS

Self-paced, or alongside any live applied-AI cohort you happen to be taking. The rhythm per module: read the objective, build the lab, tick the capability checks only when they are true of you, and write a short note in your own words, because writing is where the learning compounds. Progress lives in your browser via localStorage; nothing leaves your machine. Prerequisites: working Python, an API key for any major LLM provider, and comfort reading systems documentation.

The modules

M0Ground check0 / 5

Objective: arrive at module 1 with zero yak-shaving left: environment, keys, budget, and a written inventory of what you already know that this field renames.

CAPABILITY CHECKS

M1Prompts as contracts0 / 8

Objective: stop treating prompts as incantations and start treating them as interface contracts: with failure modes, versions, and tests, the way you treat a schema.

CAPABILITY CHECKS

LAB M1 · SUPPORT-TICKET CLASSIFIER WITH A HARD CONTRACT

Invent a small SaaS and build a classifier that reads raw support tickets and must return strict JSON: {category, severity, affected_area, suggested_first_action}. Enforce the contract with schema-constrained output plus a parse-and-repair fallback; log every repair. Write a 30-case golden set including five adversarial cases (injection attempts inside ticket text, absurd inputs, empty input). Measure contract-violation rate before and after the repair loop; both numbers go in the README.

M2Retrieval & tools that don’t lie0 / 9

Objective: own the retrieval stack mechanically: not “RAG as a vibe” but chunking → hybrid recall → reranked precision → filtered, cached, measured context.

CAPABILITY CHECKS

LAB M2 · DOCS RAG WITH A LADDER YOU CAN MEASURE

Corpus: the documentation of one substantial open-source project (100 to 300 pages). Build retrieval four ways, in order: dense-only → +BM25 hybrid with RRF → +cross-encoder rerank → +metadata filters (section, version). After each rung, run the same 25-question set and record context precision against a rubric you wrote first. Deliverable: a 4-rung ladder chart with numbers you own, and one paragraph on which rung surprised you.

M3One agent, full loop0 / 8

Objective: internalize the agent loop as a control system (observe, think, act, checkpoint) so “agent” stops meaning magic and starts meaning a while-loop with state you can kill, resume, and audit.

CAPABILITY CHECKS

LAB M3 · BLOG-POST-TO-CODE AGENT WITH A KILL TEST

The agent takes a short technical blog post and produces a runnable skeleton implementing its core idea. Loop: plan → per-step execute with tool calls (file write, lint, dry-run) → checkpoint JSON after every step. The graded part: kill -9 it mid-run, restart, and watch it resume from the checkpoint without redoing or corrupting prior steps. Ship the killed-and-resumed transcript.

M4Crews & memory0 / 9

Objective: treat multi-agent systems as distributed systems (shared state, coordination, partial failure) and memory as an engineered tier structure with budgets, not a growing string.

CAPABILITY CHECKS

LAB M4 · PAYMENT-ALERT TRIAGE CREW WITH AN INJECTED DEADLOCK

Invent an e-commerce payment stream and generate synthetic anomaly alerts. Three agents: a triage orchestrator, an evidence-gatherer with lookup tools over the synthetic data, and a critic that must approve every disposition. Shared memory: one structured case file per alert. Make it work; then inject the classic failure, where the critic demands evidence only producible after disposition, watch it hang, and break it with a timeout plus escalation tie-breaker. Ship the case-file schema and both transcripts.

M5Evals or it didn’t happen0 / 9

Objective: build the reflex that no claim about AI behavior exists without a measurement, then own the machinery: golden sets, judges with named biases, gates in CI, adversarial suites.

CAPABILITY CHECKS

LAB M5 · A REGRESSION HARNESS THAT SAYS NO

Wrap the M1 classifier and M2 RAG lab in one harness: golden sets in the repo, one command runs everything, output is a metrics-versus-thresholds table, exit code nonzero on breach. Add an LLM-judge lane with the three bias mitigations. Prove it works: deliberately degrade a prompt, watch the harness catch it, and keep the failing run’s output as the trophy.

M6Production spine0 / 9

Objective: give AI features the operational spine you give any production service (budgets, caches, traces, fallbacks, cost attribution) so they survive traffic, finance, and 3 a.m.

CAPABILITY CHECKS

LAB M6 · PUT A SPINE ON IT

Take the M3 agent or M2 RAG. Add: prefix-cache-friendly prompt layout, a two-tier model cascade with logged escalation reasons, a per-run cost line written to CSV, spans per step, a hard deadline with a degraded fallback answer, and one chaos test: revoke the primary model’s key mid-run and show the system degrade politely instead of dying. Ship the cost CSV after 50 runs and the chaos transcript.

M7The model layer → capstone0 / 8

Objective: close the gap most curricula skip, choosing and operating the models themselves, then convert the course into evidence: a capstone system and a portfolio artifact.

CAPABILITY CHECKS

CAPSTONE · ONE REAL SYSTEM, END TO END

Pick one genuine problem from your own work or a public dataset. Deliver the full arc: design doc (data flow, memory, failure handling, eval plan) → working prototype wearing the M6 spine → eval report with golden-set numbers → 10-minute demo recording. The bar: a skeptical staff engineer leaves convinced by the measurements, not the demo.

Build windows: the projects

Modules teach capabilities; projects force integration. Three build windows, spaced through the course. Definition-of-done boxes count toward progress.

after M1 to M2 · ~6 hrs PROJECT A · GROUNDED ANSWERS

The M2 docs-RAG with the M1 contract on top: answers cite their retrieved chunks, refuse when retrieval is thin, never break schema.

after M3 to M4 · ~8 hrs PROJECT B · THE RESUMABLE CREW

The M4 triage crew, hardened: checkpoint every step, survive kill -9, break the injected deadlock, keep case files an auditor could read cold.

after M5 to M7 · course end PROJECT C · THE CAPSTONE

The M7 capstone at the full bar: spine on, eval-gated in CI, demoed in 10 minutes. Projects A and B are its rehearsals.