Practice · 2023–2025 · AI Present

Fine-tuning as default

When in doubt, fine-tune. Often the expensive answer to a retrieval or eval problem — mostly shelved as a default by 2025.

Fine-tuning-as-default mutated because domain adaptation is real for narrow tasks and wasteful for FAQ bots. Base-model jumps and better RAG killed fine-tune-first for most teams. What sticks is selective fine-tunes with measurement; what fades is fine-tune theater on every backlog item.

ai training cost

Cost of the fad

Fine-tune jobs and GPU bills for problems that prompt + retrieval would have solved. Model zoos became the new snowflake servers.

Context

Demos consolidate; evals remain

AI pair programming changed how code is typed faster than how it is reviewed. Agent frameworks and standalone vector stores sorted into demos versus durable plumbing; mid-market RAG folded back into Postgres. The permanent layer is familiar: evals that gate deploys, model routing for cost, tool protocols instead of plugin snowflakes, and humans who own production. Autopilot rewrites and vibe-shipped auth middleware are still big-bang migrations with better slides — and a longer on-call.

Compare with

vs RAG as default architecture vs LLM eval pipelines vs LLM app frameworks vs XML-as-architecture

Practice · 2023–now

RAG as default architecture

Mutated

Retrieval-augmented generation as the answer to every knowledge problem — then "just stuff the window" as the counter-fad. Often right; often Postgres with pgvector would suffice.

$ Teams stood up vector pipelines, chunking strategies, and rerankers before asking if fine-tuning or a SQL query would answer the question. Retrieval infra became the product.

Practice · 2023–now

LLM eval pipelines

Stuck

Regression tests for nondeterministic models that actually fail the build. The unglamorous CI that separates demos from products.

Framework · 2023–now

LLM app frameworks