Ask most people what AI in an app looks like and they will describe a chat box wired to a hosted model. That is one pattern, and a useful one, but it is a small corner of what modern AI can do inside a product. Treating it as the whole toolbox leads teams to bolt a chatbot onto things that never needed one, and to miss the techniques that would actually have moved the needle.
The models you never see
A lot of the most useful AI in a product is invisible. Classification models that route a support message to the right team. Vision models that read a document so a person does not have to. Small reasoning models that understand the structure of a screen and adapt when it changes. None of these present as a chat interface, and several can run entirely on the user's device.
Sometimes the right model is two gigabytes on the device, not an API call to the cloud.
On-device models are the part teams most often overlook. They cost nothing per call, they work offline, and, crucially, the user's data never leaves their device. For anything touching sensitive information, that last point is not a nice-to-have; it is often the difference between a feature you can ship and one you cannot.
A worked example
We tested this ourselves with an internal project, WannaPass. Password managers usually find login forms with hand-written rules and regular expressions, which break the moment a website changes its structure. We tried compact reasoning models running fully on-device to understand forms the way a person does: find the fields, capture the credentials and fill them back in. It classified complex flows, including banking pages, and kept working when the pages changed underneath it. No data left the machine.
The discipline that makes it real
Whatever the model, the difference between a demo and a product is the engineering around it. That means evaluation suites that measure quality on your real inputs before launch, guardrails and fallbacks for when a model is wrong, cost ceilings so a feature cannot run up a surprise bill, and privacy handling that stands up to scrutiny.
The exciting part of AI is rarely the model itself now; capable models are increasingly a commodity. The value is in choosing the right one for the job, wiring it into a real product, and shipping it with the discipline to keep it trustworthy. That is the work.