Practice · 2023–now · AI Present
Agent ops / LLM observability
Tracing, cost caps, and prompt versioning for production LLM features — mostly constrained tool loops, not autonomous agents. Datadog for tokens.
Agent ops stuck because token bills and silent regressions appear the moment a demo hits real users. The autonomous-agent buzzword faded; the need for traces, eval hooks, and spend alerts did not. It failed as theater when teams bought tools without defining success metrics first.
Context
Demos consolidate; evals remain
AI pair programming changed how code is typed faster than how it is reviewed. Agent frameworks and standalone vector stores sorted into demos versus durable plumbing; mid-market RAG folded back into Postgres. The permanent layer is familiar: evals that gate deploys, model routing for cost, tool protocols instead of plugin snowflakes, and humans who own production. Autopilot rewrites and vibe-shipped auth middleware are still big-bang migrations with better slides — and a longer on-call.
Compare with
Related
Practice · 2023–now
LLM eval pipelines
Regression tests for nondeterministic models that actually fail the build. The unglamorous CI that separates demos from products.
Framework · 2023–2025
Multi-agent orchestration hype
LangGraph, CrewAI, AutoGPT cosplay — agents delegating to agents for tasks one function call could do. Hype peaked; backlash stuck.
$ Autonomous agent swarms burned API budgets on coordination loops that a single prompt and a cron job would have handled. Debugging "which agent lied" became the new on-call sport.
Practice · 2021–now
AI pair programming
Autocomplete that understands the file — now table stakes. Generation outran review capacity; ownership is the bottleneck.
Architecture · 2024–now
Model routing / tiered inference
Cheap model drafts; frontier model escalates. The boring cost-control layer that survived the agent hype.