The problem
The client runs a direct-to-consumer retail app selling home goods. Its catalog was large enough that generic "best sellers" recommendations were leaving real revenue on the table, but, according to the client's own churn surveys, its customers skewed heavily toward people who distrust apps that track browsing server-side. A conventional cloud-based recommendation system, logging every product view and scroll to a server for training, was a product-trust risk and, in several international markets they were expanding into, a genuine data-residency complication.
The ask was specific: recommendations that reflected what a user browsed and bought, without that history ever leaving the device in a form that could be tied back to the user. That ruled out standard centralized collaborative filtering almost immediately and meant the personalization model had to run, and in most cases update, on-device.
What we actually did
We built a lightweight on-device recommendation model that trains incrementally on a user's own in-app behavior, using only signals available locally (views, dwell time, adds-to-cart, purchases), with no raw behavioral event ever transmitted to the client's servers.
Weeks 1-3: picking a model architecture that could actually run on a mid-range phone
The catalog had roughly 14,000 SKUs, too many for a naive on-device embedding table on a mid-range Android device without unacceptable storage and battery cost. We settled on a compact two-tower embedding model. Product embeddings are precomputed server-side (from catalog metadata only, never user behavior) and shipped in the app bundle, while the user-preference vector is built and updated entirely on-device from local interaction history. The split keeps the heavy computation off the phone and everything derived from a user's behavior local.
Weeks 4-6: on-device inference and incremental updates
We used a quantized model running through Core ML on iOS and TensorFlow Lite (now LiteRT) on Android, re-ranking the catalog against the user's local preference vector on every app open and after significant interactions like a purchase. Getting median recommendation latency down to 41 milliseconds on a three-year-old mid-range Android device took real tuning of the quantization level; our first full-precision pass was closer to 300 milliseconds, noticeably laggy on the product feed.
Weeks 7-9: cold-start handling and testing
A brand-new user has no on-device history, so recommendations fall back to catalog-level popularity signals (not user-specific, and safe to ship as static data) until the user has roughly ten meaningful interactions; then the on-device model starts blending in. We ran two rounds of internal dogfooding to check that recommendations felt personalized within a normal first session, because personalization that only becomes noticeable after a week doesn't build the trust it's meant to.
Challenges and tradeoffs
- On-device models are meaningfully less accurate than a server-side model trained on the full user base, and we had to be honest about that upfront. A cloud-based collaborative-filtering system benefits from cross-user signal that an on-device, single-user model structurally can't access. We were explicit from week one that this would cap personalization quality relative to a cloud approach, in exchange for the privacy guarantee the client wanted.
- Battery and storage impact needed real device testing, not just simulator numbers. Incremental on-device updates, even lightweight ones, showed a measurable battery draw on older devices during internal testing. We capped them at once per session rather than after every interaction, trading a little recommendation freshness for a battery impact the client's QA team judged acceptable.
- Explaining "your data never leaves your phone" credibly required a technical audit, not just a claim in the app-store listing. We had the client commission an independent review of the app's network traffic during active use to verify that no behavioral data was transmitted. We expected (correctly) that privacy-focused users would ask, and a marketing claim with no audit behind it would have been worse than making no claim at all.
Results
Over a twelve-week window, the repeat purchase rate among users with the personalized feed rose 22% compared with a control cohort still seeing the old "best sellers" feed. This was measured as a randomized in-app experiment rather than a before-and-after comparison, to control for seasonal patterns. Median on-device recommendation latency held at 41 milliseconds across the device tiers tested, with no user-reported lag complaints in the two months after launch.
The privacy framing became a genuine differentiator in the client's app-store listing and marketing, not just an engineering constraint. The independent traffic audit is linked from their privacy page, which their marketing team said they wouldn't have had the confidence to do without third-party verification.
What we'd do differently
We'd build the battery-impact testing harness in week one instead of discovering the issue during week seven's device testing. We caught it in time, but earlier battery profiling would have let us make the update-frequency tradeoff with real data rather than reacting late.