Practice · 2023–now · AI Present

Fine-tuning as default

When in doubt, fine-tune. Often the expensive answer to a retrieval or eval problem.

Fine-tuning-as-default mutated because domain adaptation is real for narrow tasks and wasteful for FAQ bots. It failed when teams skipped data quality and evals. What sticks is selective fine-tunes with measurement; what fades is fine-tune theater on every backlog item.

ai training cost

Cost of the fad

Fine-tune jobs and GPU bills for problems that prompt + retrieval would have solved. Model zoos became the new snowflake servers.

Context

The stack gets a co-pilot

AI pair programming is already changing how code is written. Agent frameworks and vector stores are still sorting winners from demos. The durable layer will look familiar: evals, retrieval quality, product UX, and ownership. Autopilot rewrites without tests are just big-bang migrations with better slides.

Compare with

vs RAG as default architecture vs LLM eval pipelines vs LLM app frameworks vs XML-as-architecture

Practice · 2023–now

RAG as default architecture

Mutated

Retrieval-augmented generation as the answer to every knowledge problem. Often right; often Postgres with pgvector would suffice.

$ Teams stood up vector pipelines, chunking strategies, and rerankers before asking if fine-tuning or a SQL query would answer the question. Retrieval infra became the product.

Practice · 2023–now

LLM eval pipelines

Stuck

Regression tests for nondeterministic models. The unglamorous infrastructure that separates demos from products.

Framework · 2023–now

LLM app frameworks