Practice · 2023–now · AI Present
Agent ops / LLM observability
Tracing, cost caps, and prompt versioning for production LLM features. Datadog for tokens.
Agent ops stuck because token bills and silent regressions appear the moment a demo hits real users. The framework churn faded; the need for traces, eval hooks, and spend alerts did not. It failed as a buzzword when teams bought tools without defining success metrics first.
Context
The stack gets a co-pilot
AI pair programming is already changing how code is written. Agent frameworks and vector stores are still sorting winners from demos. The durable layer will look familiar: evals, retrieval quality, product UX, and ownership. Autopilot rewrites without tests are just big-bang migrations with better slides.
Compare with
Related
Practice · 2023–now
LLM eval pipelines
Regression tests for nondeterministic models. The unglamorous infrastructure that separates demos from products.
Framework · 2023–now
Multi-agent orchestration hype
LangGraph, CrewAI, AutoGPT cosplay — agents delegating to agents for tasks one function call could do.
$ Autonomous agent swarms burned API budgets on coordination loops that a single prompt and a cron job would have handled. Debugging "which agent lied" became the new on-call sport.
Practice · 2021–now
AI pair programming
Autocomplete that actually understands the file. Changing how code is written faster than how it's reviewed.