Practice · 2023–2025 · AI Present
Fine-tuning as default
When in doubt, fine-tune. Often the expensive answer to a retrieval or eval problem — mostly shelved as a default by 2025.
How Fine-tuning as default mutated
Fine-tuning-as-default mutated because domain adaptation is real for narrow tasks and wasteful for FAQ bots. Base-model jumps and better RAG killed fine-tune-first for most teams. What sticks is selective fine-tunes with measurement; what fades is fine-tune theater on every backlog item.
Cost of the fad
What Fine-tuning as default cost
Fine-tune jobs and GPU bills for problems that prompt + retrieval would have solved. Model zoos became the new snowflake servers.
Compare with
Related
Practice · 2023–now
RAG as default architecture
Retrieval-augmented generation as the answer to every knowledge problem — then "just stuff the window" as the counter-fad. Often right; often Postgres with pgvector would suffice.
$ Teams stood up vector pipelines, chunking strategies, and rerankers before asking if fine-tuning or a SQL query would answer the question. Retrieval infra became the product.
Practice · 2023–now
LLM eval pipelines
Regression tests for nondeterministic models that actually fail the build. The unglamorous CI that separates demos from products.
Framework · 2023–now
LLM app frameworks
LangChain-class glue mutated into MCP clients and thin wrappers around vendor APIs. The durable pieces are still boring: evals, retrieval, and product UX.