Small language models

When we started building Aina, the obvious choice was to reach for the best frontier model available. GPT-4, Claude, Gemini — they’re impressive. Ask them anything about law, medicine, quantum mechanics, and they’ll give you a coherent answer.

But Aina is a beauty recommendation engine. It doesn’t need to know about quantum mechanics. It needs to know about skin care, ingredients, routines, and how to have a helpful conversation about something personal. A model that knows everything is not automatically the right tool for a job that requires something specific.

Here are five lessons we learned the hard way.

Cost

Frontier models are priced for their breadth. Every token you send to GPT-4 or Claude pays for the ability to answer questions about international tax law — whether your app needs that or not.

A small model trained or fine-tuned on a narrow domain costs a fraction of that per inference. At scale, the difference is significant. An application handling thousands of conversations per day can see cost reductions of an order of magnitude by moving to a purpose-built model. For a startup watching its cloud bill, that’s not a minor optimization — it’s what makes the unit economics work.

Latency

Size is physics. A smaller model has fewer parameters to run through, which means faster inference — often by a wide margin.

In a conversational application, latency is felt directly by the user. A two-second pause in a chat feels long. A five-second pause feels broken. Frontier models running on shared infrastructure can be unpredictable under load. A small model you run yourself or deploy on dedicated hardware gives you control over that response time.

For Aina, where the experience is a personal conversation, a fast reply isn’t a nice-to-have. It’s part of what makes the interaction feel natural.

Fine-tuning

Small models can be adapted to a specific domain in ways frontier models can’t. Two techniques are worth knowing.

RAG (Retrieval-Augmented Generation) retrieves relevant documents at query time and includes them in the prompt. It’s fast to set up and keeps your knowledge up to date — you update the document store, not the model. Good for factual lookups, product catalogues, support knowledge bases.

Fine-tuning trains the model on examples specific to your domain. The model learns your vocabulary, your tone, your edge cases. It’s more work upfront, but the result is a model that behaves consistently in your context — not one that improvises based on whatever it learned about skincare from the general internet.

For Aina, fine-tuning on curated skincare knowledge gives us responses that are accurate, on-brand, and appropriate for the sensitivity of the subject — something a general model can only approximate.

Availability

Frontier models get replaced. GPT-3 is gone. GPT-4 will follow. When that happens, the provider gives you a window — sometimes months, sometimes weeks — to migrate to the new version.

In theory, your application should behave the same on the new model. In practice, it won’t, not exactly. Prompts that worked reliably on one version drift on the next. Behaviours change. You have to test everything, under time pressure, before the old version is shut down.

With a small model you control, that clock doesn’t exist. You upgrade when you choose to, on your own timeline, after proper testing. For a production application with real users, that stability is worth a great deal.

Privacy

When you send a user’s message to a frontier model, that message leaves your infrastructure. For many applications that’s fine. For some, it isn’t.

Aina’s users share personal details — skin conditions, sensitivities, things they might not tell their doctor. That conversation should stay private. Running inference on a model you host means the data never leaves your environment. No third-party provider processes it. No terms of service you don’t control govern what happens to it.

Privacy is increasingly a competitive advantage. Users notice when an application takes it seriously. A small model running on your own infrastructure is one of the clearest ways to make that commitment real.