Demo on your phone · Sample screens only. The VPN button is simulated; your real connection is unchanged. The native iPhone app is still in development, not yet on the App Store.
Live simulation · nothing leaves this page
Pipeline
This is the proposed native-lens pipeline, not a description of the online translator. The separate browser translator sends text and OCR images to Cloudflare when you choose to use it.
Android: a MediaProjection frame every 700 ms (or the accessibility tree, which is cheaper). iOS: a screenshot or the Safari web extension.
ML Kit / Vision returns text, confidence and bounding boxes. LingoLens clusters lines into UI blocks and samples the surrounding pixels for background colour.
On-device NMT for the 12 offline packs, cloud engines for the long tail. A term memory keeps names, prices and units untouched.
Overlay views are placed on the exact bounding boxes, using a font-metric fit so line height, weight and colour match the original. Tap any bubble to hear it.
Reality check
You asked for a lens that rewrites other apps' text and a VPN that works anywhere. Here is the honest engineering picture — it drives the architecture in native/.
| Capability | Android 10+ | iOS 17+ |
|---|---|---|
| Overlay other appsDraw above a third-party app | Yes TYPE_APPLICATION_OVERLAY, needs “Display over other apps”. |
Partial Only a Picture-in-Picture window is allowed above other apps. It's a floating panel, not a full-screen overlay. |
| Replace text in placeSame font, size, colour, layout | Yes OCR boxes + styled overlay. Accessibility tree can also be read directly for text-only apps. | In Safari yes A Safari web extension can rewrite DOM text and inherit the page CSS perfectly. Native apps: no. |
| Background serviceKeep the lens alive while you use other apps | Yes Foreground service + persistent notification. | Limited Extensions run only while invoked; a PiP session can persist while the overlay is open. |
| Own VPN tunnelDevice-wide encrypted egress | Yes VpnService, no special approval. WireGuard is a drop-in. |
With entitlement NEPacketTunnelProvider needs Apple's “Personal VPN” entitlement, granted after review. |
| Works offlineNo network at all | Yes Bundled OCR + translation models. | Yes Apple on-device translation + Vision models. |
| Works “in any country”Regardless of local restrictions | Depends on your servers and local law. A tunnel is only as reachable as its endpoints. No app can promise access everywhere; app stores also require VPN disclosure and prohibit marketing that promises circumvention. Plan for server diversity, obfuscation, and a compliance review. | |
Foreground lens service, ML Kit OCR, overlay redraw, term memory, and a real WireGuard tunnel. This is the build that matches the demo one-to-one.
Reference implementation innative/android
Safari extension does true in-place, style-preserving replacement on the web. Beyond Safari: PiP overlay with a translated image, keyboard extension, share sheet and screenshot translation.
Scaffold innative/ios · entitlement needed for VPN
The VPN interface is a simulation. Native client reference code is in the repo, but no server fleet is deployed. A production VPN needs real endpoints, testing and legal review.
Read docs/VPN.md before shipping