blurr

Blurr implements a multi-layer voice activation system combining manual tap-based triggering via DeltaSymbolView, persistent wake-word detection using Picovoice engine in EnhancedWakeWordService, and Android RoleManager integration for default assistant role. Voice input is captured, transcribed via speech-to-text, and routed to the conversational agent service which interprets natural language intent and triggers the AI agent execution framework. The system maintains always-on listening capability without requiring explicit app focus.

Blurr's perception layer leverages Android's AccessibilityService to read the complete UI hierarchy (AccessibilityNodeInfo tree) from the currently visible screen, extracting semantic information about interactive elements, text content, and layout structure. This accessibility tree is serialized into a structured representation that the LLM can reason about, enabling the agent to understand which buttons, text fields, and interactive components are available without relying on image recognition or OCR. The system captures both the visual state and the semantic meaning of UI elements.

Blurr integrates Firebase Analytics to track user behavior, task execution patterns, and feature usage. Firebase Crashlytics captures runtime errors and exceptions, providing crash reports and stack traces for debugging. The system logs key events (task execution, permission grants, subscription changes) to Firebase for analytics. This data enables the developers to understand user behavior, identify bugs, and optimize the product. Firebase also provides real-time dashboards for monitoring app health and user engagement.

Blurr stores user data locally using Android's persistence mechanisms (likely SharedPreferences for simple data, Room database for complex data structures). Sensitive information (API keys, authentication tokens, user preferences) is encrypted using Android's EncryptedSharedPreferences or similar encryption libraries. The system manages data lifecycle (creation, update, deletion) and handles data migration across app versions. Local storage enables offline operation for certain features and reduces dependency on cloud services.

Blurr enables automation workflows that span multiple applications, maintaining context and state as the agent navigates between apps. The system detects app transitions (via AccessibilityService), preserves task context across app boundaries, and adapts the UI perception and action execution to each app's specific interface. This allows complex workflows like 'open email, find message from John, extract phone number, open contacts, add new contact with that number' where the agent must understand context across three different apps. The agent maintains a unified task model that abstracts away app-specific details.

Blurr implements robust error handling that detects when actions fail (element not found, action timed out, unexpected UI state) and attempts recovery. The system includes fallback strategies: retry with adjusted timing, alternative action paths (e.g., using menu instead of direct button), and user escalation (asking user for help or manual intervention). Error detection works by comparing expected UI state (from LLM reasoning) with actual UI state (from accessibility tree) after each action. The system logs errors for debugging and learns from failures to improve future action selection.

Blurr integrates Google Gemini API as the reasoning engine that receives the current screen state (accessibility tree), user intent (voice command), and task history, then generates the next action to execute. The LLM operates in an agentic loop: it analyzes the current UI state, reasons about the user's goal, selects the most appropriate action (tap, scroll, type, etc.), and provides structured action output that the execution layer interprets. The system maintains conversation context across multiple turns, allowing the agent to handle multi-step workflows that require understanding previous actions and adapting to screen changes.

Blurr's action execution layer translates LLM-generated action specifications into native Android UI automation commands via the AccessibilityService API. The system supports multiple interaction primitives: single/multi-touch taps at specific coordinates, swipe/scroll gestures with configurable velocity and direction, text input via keyboard simulation, and long-press interactions. Actions are queued and executed sequentially with timing controls to allow UI animations to complete between actions. The execution layer includes error detection (checking if expected UI changes occurred after an action) and fallback mechanisms for failed interactions.

Blurr provides a trigger system that decouples task definition from execution, allowing users to define automation rules that execute based on time schedules, app launch events, or system state changes. Triggers are stored in local persistence (likely SharedPreferences or Room database based on Android patterns), with a TriggerMonitoring service that continuously checks trigger conditions and invokes the agent execution framework when conditions are met. The system supports multiple trigger types: cron-like scheduling, app-specific triggers (when app X launches), and state-based triggers (when device reaches certain conditions). This enables background automation without requiring constant user interaction.

Blurr maintains conversation state across multiple voice commands and agent actions, storing interaction history (user intents, agent actions, screen states, outcomes) that the LLM can reference for context-aware decision making. The ConversationalAgentService manages this context, passing relevant history to the Gemini API with each new request. The system implements context window management to stay within token limits while preserving recent and important interactions. This enables the agent to understand references like 'reply to the last message' or 'undo that action' by maintaining awareness of previous interactions.

Blurr captures audio from the device microphone and converts it to text using Android's built-in speech recognition APIs (likely SpeechRecognizer or Google Speech-to-Text). The system handles audio streaming, noise filtering, and transcription with configurable language support. Real-time processing allows the agent to begin reasoning about user intent as soon as speech is recognized, without waiting for the user to finish speaking. The transcribed text is then passed to the conversational agent service for intent interpretation.

Blurr generates spoken responses to user commands using Android's TextToSpeech API, converting agent reasoning and task status into natural language audio output. The system supports multiple voices, speech rates, and languages. Responses are synthesized in real-time and played through device speakers, providing immediate audio feedback about task progress and completion. The TTS system integrates with the conversational agent to provide contextual responses (e.g., 'I found 3 emails from John' instead of just 'task complete').

Blurr implements a comprehensive permission system that detects required capabilities (microphone, accessibility service, location, contacts, etc.) and guides users through permission granting via OnboardingPermissionsActivity. The system checks permission status at runtime and provides clear explanations of why each permission is needed. The permission management system is integrated with the freemium model, potentially restricting certain capabilities based on subscription level. The system respects Android's runtime permission model (API 23+) and handles permission denial gracefully.

Blurr implements a freemium business model with task execution limits for free users and unlimited access for paid subscribers. The system tracks task execution count, enforces quota limits, and prevents execution when limits are exceeded. Billing integration (likely via Google Play Billing Library) handles subscription management, payment processing, and subscription status verification. The quota system is enforced at the agent execution layer, checking remaining quota before executing tasks. Firebase integration likely tracks usage metrics for analytics and quota enforcement.