Offline Speech in Real Apps: RecalAI and Lunest Architecture, Privacy, and Failure Boundaries
Can an iPhone or Android app really transcribe and speak offline? This source-backed review compares RecalAI's existing native device-only speech and Lunest's audio recovery with Google's new ML Kit GenAI ASR/TTS, including Mandarin, Basic/Advanced compatibility, resources, permissions and failure states—without claiming a shipped integration.
Evidence 6 cited sources
On this page
“Fully offline voice” sounds straightforward until the app must work on a midrange Android device, handle Mandarin correctly, preserve a user’s private records, and recover after an incoming call. Is a model already installed? Will the feature silently need the network on the first use? Does speech transcription mean the same thing as reliable voice-command execution? Those are more consequential questions than a model’s headline tokens-per-second or speech benchmark.
On November 10, 2026, Google announced on-device automatic speech recognition (ASR) and text-to-speech (TTS) in AI Edge Gallery. The announcement describes integrated Android ML Kit functionality and a separate open-weight route via MediaPipe Tasks and LiteRT. Its impressive speed and power examples are vendor results, not measurements on RecalAI, Lunest, a 4 GB iPhone, or an average Android device. Google’s announcement.
The relevant engineering question is whether speech can enter an existing product without breaking the very promise users already rely on. RecalAI, a local-first personal knowledge application, and Lunest, a sleep/audio application, illustrate why one shared speech architecture would be a bad assumption. This is a source-based product decision study, not a claim that either application currently ships Google’s new SDKs.
The same speech model addresses two fundamentally different user jobs
A RecalAI user might ask, “Find last year’s decision on encrypted backups.” The app’s purpose is not to transcribe a sentence; it is to retrieve the correct personal documents and return a traceable answer. The natural proposed flow is: an explicitly initiated microphone session, on-device transcription, an editable confirmed query, the existing local FTS/vector retrieval and evidence-selection pipeline, citation-validated generation, and optional spoken playback. RecalAI’s existing documented hybrid retrieval design deliberately distinguishes exact matches and semantically related results and validates evidence references: RecalAI hybrid retrieval.
Speech introduces a new risk before retrieval even starts. A recognizer may revise “unencrypted” to “encrypted,” mishear an account of what not to do, or drop a number. If every partial transcription launches a search, intermediate hypotheses can trigger contradictory work and create an apparently grounded answer to the wrong question. The design implication is that a partial transcript is a display state, not a committed instruction. A finalized segment should remain editable; sensitive terms and negation may warrant confirmation. This is an architectural proposal, not a verified existing feature.
Lunest has a different obligation: keep sleep audio playback, monitoring and recovery stable throughout a long night. A general-purpose speech recognizer does not provide validated snoring or sleep-stage classification. Leaving an always-listening ASR pipeline running during sleep can create unnecessary privacy, power, and microphone-contention concerns. An explicit short command while the user is awake may be useful, but it should be separated from the sleep-monitoring engine, and its failure must never reset the playback session or change a wake-up decision.
The design principle is simple: speech is a new input/output surface; it is not permission to change the application’s source of truth or its privacy contract.
What we have actually implemented before evaluating the new Google APIs
This is not a hypothetical product roadmap assembled after a Google announcement. The RecalAI and Lunest repositories already contain on-device speech, evidence retrieval and overnight audio lifecycle code. The distinction is important: shipping-oriented platform code is not proof of integration with the newly announced ML Kit GenAI Speech Recognition or TTS Alpha APIs, nor proof of device acceptance.
| App/platform | Existing implementation found in the project | Claim we cannot yet make |
|---|---|---|
| RecalAI iOS/iPadOS | PMSpeechInputController checks microphone and Speech authorization, requires supportsOnDeviceRecognition, and sets requiresOnDeviceRecognition = true; unsupported locales fail closed | No complete measured Mandarin accuracy, power or offline device matrix |
| RecalAI iOS media import | PMOnDeviceMediaTranscriber uses on-device SFSpeechURLRecognitionRequest in approximately 50-second audio/video segments, with fingerprinted protected transcription checkpoints | Segment timestamps do not establish universal word-level timing |
| RecalAI Android | OnDeviceVoiceInput uses createOnDeviceSpeechRecognizer rather than a generic recognizer; media transcription checks Android 13+ and on-device availability before segmenting audio | This is platform SpeechRecognizer, not the new ML Kit GenAI ASR Basic/Advanced integration |
| RecalAI read-aloud | Android OfflineAnswerReader selects installed non-network TextToSpeech voices; an iOS study-card screen uses AVSpeechSynthesizer | Not a claim that every language or screen is implemented with GenAI TTS |
| RecalAI macOS | PMMacSpeechInputController also refuses locales lacking device recognition and sets requiresOnDeviceRecognition | Not an Android ML Kit implementation |
| Lunest iOS/Android | iOS responds to audio interruptions, route changes and media-service resets; Android SoundPlayer governs audio focus and recovery; acoustic events and HealthKit/Health Connect remain local | No verified general-purpose voice-command ASR/TTS feature: detecting possible sleep speech is not transcribing it |
RecalAI also already has local FTS, embeddings, RRF retrieval and citation-validation work. Speech needs to enter this existing evidence path, not bypass it. For the actual zero-metered-API stack and vendor alternatives, follow our free local speech and agent implementation analysis; for an explicitly authorized remote reasoning step, see the Gemini 4 routing and task-cost study.
What the existing speech experience establishes

Figure: Hands-on feedback is qualitative; no standardized per-stage latency benchmark has been completed.
In day-to-day use, the RecalAI developer finds on-device speech recognition promising (an informal estimate around 99%), search reasonably useful and the overall iPhone 13 interaction somewhat slow. This is neither a labeled-corpus CER result nor a benchmark of Google’s newly released ML Kit speech API. The detailed diagnosis of cold/warm startup, retrieval and model-generation latency belongs in the no-paid-API local voice and agent analysis.
Three Google routes are not a single interchangeable SDK

Figure: Offline speech support varies by platform, locale, device and downloaded model assets.
| Option | Officially documented capability, Nov 17 2026 | Product-level trade-off |
|---|---|---|
| Android ML Kit GenAI ASR Basic | genai-speech-recognition:1.0.0-alpha2; minSdk 26 for the SDK, but Android API 31+ for Basic runtime support; Chinese locale is marked beta | Promising baseline for eligible Android devices; not universal support or proven Mandarin accuracy |
| Android ML Kit GenAI ASR Advanced | Gemini Nano/AICore-based recognition, with a narrower officially listed Pixel/chipset device set | Runtime feature detection is mandatory; never assume all Android devices can use Advanced |
| Android ML Kit GenAI TTS | genai-text-to-speech:1.0.0-alpha1; a limited Pixel/Samsung device list and Google speech component | The published locale list does not include Mandarin Chinese; voice asset availability and checkStatus() must govern UX |
| MediaPipe Tasks / LiteRT models | Google describes cross-platform open-weight speech models and .litertlm bundles | A running demo is not proof of drop-in compatibility on an iPhone 13; pin SDK, model, operator, ABI and license versions |
| Apple Speech and AVSpeechSynthesizer | Native iOS recognition and speech output interfaces, including an on-device recognition capability check | Offline mode depends on locale/device support; microphone permissions, voice availability and interruptions remain application concerns |
The Android ASR documentation explicitly distinguishes microphone, decoded audio file and a custom ParcelFileDescriptor stream. Raw PCM input requires 16 kHz, 16-bit mono. Word timing is documented for streaming input, not as a blanket guarantee for pre-recorded file processing. Streaming recognition emits provisional partial results as well as final ones. Confusing those two leads to duplicate commands, incorrect UI state, or improperly persisted transcripts. ML Kit ASR API.
The GenAI TTS Alpha API lists a restricted set of locales, including English, Japanese and Korean but not Chinese. Therefore a Chinese-language RecalAI or Lunest release cannot honestly label this specific Android API “offline Chinese TTS.” A platform TextToSpeech fallback is an engineering candidate, subject to checking installed voices and actual offline operation. An unsupported device, an undownloaded voice, and low-quality synthesized audio are three different outcomes that should not be conflated. ML Kit TTS API.
On iOS, Apple explicitly states that requiresOnDeviceRecognition is only honored when the recognizer reports supportsOnDeviceRecognition == true. If this test fails, a privacy-first app should refuse the local speech task or explicitly offer a separately consented alternative. It must not silently switch to cloud recognition. AVSpeechSynthesizer supports speech, pausing, stopping and route management, but voice availability and the target locale still need validation. Apple Speech documentation and on-device request semantics.
RecalAI: the existing voice entry must preserve document provenance

Figure: Existing native recognition is not evidence that the new Google ML Kit speech SDK is integrated.
The most dangerous failure mode for a knowledge app is not a visibly failed transcription. It is a misrecognized question followed by a fluent answer with citations that appear authoritative. The voice adapter should therefore have explicit contracts with the existing evidence retrieval subsystem:
| Boundary | Proposed contract | Must be prohibited |
|---|---|---|
| Capture | User action starts microphone session; permissions are requested at point of use | Passive background recording |
| Capability resolution | Locale/device/model state determines READY / DOWNLOADABLE / UNAVAILABLE | Undisclosed network fallback |
| Transcript buffer | Partial results only update an ephemeral view; final results form candidate user text | Indexing or querying every transient revision |
| User confirmation | Query can be inspected or edited; numbers, negation and sensitive entities are handled cautiously | Treating uncertain speech as approval for an action |
| Retrieval | Existing FTS, vector, evidence selection and citation validation continue unchanged | Accepting ASR output as document evidence |
| Spoken response | Only finalized, appropriately disclosed answer content is read aloud | Speaking unverified claims before citation checking |
On a 4 GB iPhone 13, adding independent ASR, TTS and a local language model simultaneously may be more problematic than any model’s claimed size suggests. Peak resident set size, model warm-up time, thermal throttling, interruption and foreground/background contention are the meaningful selection variables. A sensible test order is to validate eligible native speech support first, preserve the original local retrieval behavior, and only then decide whether the quality gain of bundling an independent acoustic model justifies additional memory pressure. No performance or memory values are claimed here.
A local-first boundary also needs lifecycle clarity. Raw audio should remain ephemeral in memory by default, with predictable release on completion or cancellation. Only a separately authorized “save this recording/transcript” action should make material persistent. Telemetry may include device, locale, version, duration, non-sensitive failure codes and memory counters; it must not contain spoken sentences, document excerpts, private filenames or API keys. User-selected third-party cloud models belong to a separate explicit BYOK/consent path, not an invisible emergency fallback.
Lunest integration: audio focus and recoverability outrank a clever command demo

Figure: Local acoustic sleep events are not equivalent to a speech transcription system.
Consider a sleep audio session playing soundscapes when a user says “make it quieter.” Starting speech recognition may change the audio session’s mode, route or focus. A call, alarm, Bluetooth disconnect, headphones removal or a background resource constraint can then interrupt playback at precisely the wrong moment. A recognizer may even misinterpret voices inside the media stream as an instruction.
The proposed architecture separates sleep session, playback session and short voice interaction as different lifecycle owners. A voice session can request a constrained, reversible command; the existing playback controller remains responsible for applying that command. An ASR failure never clears sleep data, destroys the playback queue, restarts the monitoring timeline or silently modifies a wake policy.
An open-ended language model should not directly mutate nighttime state from a noisy transcript. Safety decisions can be deterministic: limited command vocabulary, explicit session activation, confirmation for consequential changes, validated playback controller operations and a refusal path for uncertain speech. General speech recognition is not a replacement for scientifically justified sleep event models or an overnight reliability test.
These are design decisions to validate, not a statement that Lunest currently contains a Google ASR command module.
A useful evaluation produces failure evidence, not just one success clip

Figure: Manual text input remains available when a strictly on-device recognizer cannot run.
Google’s speech model examples and Android integration reduce experimentation costs, but they do not establish real-world results on particular phones. The acceptance matrix should be reproducible:
| Dimension | RecalAI measurement | Lunest measurement | Failure rule |
|---|---|---|---|
| Eligibility | Device/OS/locale, installed or downloadable asset state | Same plus active audio route and headphones | Unsupported mode cannot be called “available” |
| Accuracy | Chinese character error rate (CER), negation, numbers, code-switching, proper nouns | False command activation and safe command rejection | Wrong sensitive action is a hard failure |
| User latency | First partial/final delay, edit/confirm round trip | Interruption introduced to ongoing playback | ASR start must not compromise core audio |
| Resources | Peak memory, warm-up, temperature, competing local LLM | Audio focus, background recovery, resource reclamation | Speech failure must not destroy session |
| Privacy | Network-disabled success, audio retention and logging audit | No ambient audio upload/retention without authorization | Silent remote processing is disallowed |
| Recovery | Cancellation, permission revocation, interruption, OOM | Call/alarm/Bluetooth, media continuation and correct progress | Lost state or duplicated action fails |
For Chinese transcription, CER = (substitutions + deletions + insertions) / reference characters. Reference transcripts should be manually verified, and mixed-language data, noise, speed, numbers and locale should be reported separately. Avoid cherry-picking short clips. TTS needs independent metrics: time to first audio, intelligibility, playback completeness, voice coverage and human listening quality. No quantitative CER, latency, memory or battery result is presented because physical-device testing has not been completed.
A minimum failure-oriented test set includes: airplane mode with all assets present; missing model/voice asset; denied microphone permission; incoming phone call; Bluetooth route change; long or code-switched utterance; memory pressure; cancellation and recovery. Pin audio hashes, SDK/model versions, hardware and OS, then record errors and outcomes without uploading user recordings.

Figure: This is an evaluation plan, not a measured performance result.
Paid cloud speech and Gemini reasoning represent different upgrades
When Android does not support the requested ASR mode or an iPhone cannot enforce on-device recognition for a locale, the first fallback should be a clear unsupported state and a usable text input—not a hidden network call. If the product separately offers an opt-in cloud-speech mode, it needs affirmative consent to transmit the recording. A metered path might use Google Cloud Speech-to-Text, billed on processed audio, and Google Cloud Text-to-Speech, whose character-based or generative-audio token rates depend on the selected voice. These are separate APIs and invoices from Gemini 4 reasoning; a limited Hydrogen free quota is not free Cloud Speech-to-Text or TTS access.
For RecalAI, the optional paid upgrade might be user-authorized complex synthesis over selected evidence rather than sending every microphone recording to a cloud ASR. For Lunest, remote speech processing cannot remove audio interruptions, privacy obligations or the problems of continuous microphone use. The free/local speech and agent decision article matches these layers against no-metered-API candidates. The Gemini 4 Hydrogen/Argon/Carbon routing article is intentionally about paid reasoning and engineering agents rather than speech synthesis.
Is it free? SDK licensing is only one part of the cost
Google describes ML Kit as a free mobile SDK, and its GenAI on-device processing incurs no additional per-request server cost. MediaPipe/LiteRT and eligible open-weight models can similarly avoid cloud speech API fees, but shipping still incurs licensing diligence, asset downloads, binary size, device energy, QA and maintenance costs. A device-restricted Android GenAI TTS alpha feature is not an unlimited, universal Chinese-language speech service. See Google’s ML Kit overview and GenAI on-device cost explanation.
Engineering decision
For RecalAI, speech succeeds when it makes locally held knowledge easier to ask about without damaging traceable answers. For Lunest, it succeeds only if it makes a constrained interaction easier without compromising overnight audio stability or user privacy. ML Kit, MediaPipe/LiteRT and Apple Speech are viable candidates for different subsets of devices; they are not evidence that a universal, all-language, all-device, low-power offline solution already exists.
The next decision is capability probing and fixed-input device tests, not choosing the model with the largest advertised benchmark. A voice product deserves production confidence only after unavailable modes, incomplete transcripts, permission denial, interrupted playback and resource exhaustion have honest, tested outcomes.
About the still-in-development products
These engineering examples come from two XBSTACK apps that are not yet publicly available in app stores: the RecalAI local knowledge project (whose current public product page is labeled Recalia) and the Lunest sleep and soundscape project. Existing code paths do not establish an app-store release, feature availability on every device or standardized performance acceptance.
Source and evidence boundaries
- Google AI Edge Team (Nov 10, 2026), On-Device Speech Recognition and TTS in AI Edge Gallery.
- Google ML Kit, GenAI Speech Recognition API and GenAI TTS API, both in alpha.
- Apple Developer, supportsOnDeviceRecognition, requiresOnDeviceRecognition, AVSpeechSynthesizer.
- RecalAI’s existing hybrid retrieval case study describes its knowledge retrieval context. Proposed ASR/TTS integration here is not a completed test or shipping feature claim.
Further reading: a local voice agent with no paid API bill
The costs that matter after removing cloud fees include Mandarin voice availability, iPhone 13 memory, model redistribution rights and long-running audio reliability. Those choices are examined separately in Can offline speech and a local AI agent run without paid APIs?, so this article can remain focused on Google AI Edge SDK and device boundaries. If a product instead needs a paid remote model for more complex engineering tasks, see Gemini 4 model routing and accepted task costs.
Next step: continue the Agent system
Full decision path →Choose the next architecture, tool, memory, evaluation, security, or production decision instead of starting over.
DISCUSSION
Questions, verification and corrections
Sign in to comment. Every new comment is reviewed before publication; while pending, it is visible only to you and the administrator.