A next-generation, multimodal browser AI agent built on Chrome Manifest V3 that enables 100% hands-free web automation through natural voice commands, multimodal visual perception, and zero-regex dynamic reasoning across any web application.
About This Project
Whoop is an enterprise-grade autonomous browser agent engineered to execute complex, multi-step web workflows entirely via voice and visual reasoning. Unlike traditional browser bots that rely on fragile DOM selectors, XPath queries, or hardcoded regex templates, Whoop operates on an end-to-end 4-Stage Cognitive Loop (Ear → Brain → Eyes → Voice) backed by a zero-downtime, multi-tier failover matrix.
Whether ordering groceries, scraping structured data, filling out dynamic forms, or navigating legacy intranet portals, Whoop sees the webpage like a human, reasons through intent dynamically, calculates pixel-accurate interactions, and executes deterministic actions under strict security validation.
Core Features
1. Hands-Free Voice-to-Action Execution
Users interact with their browser using natural spoken language. Whoop continuously listens, processes intent in real time, plans the task, and executes complex sequences of clicks, inputs, tab switches, and scrolls without manual intervention.
2. 4-Stage Cognitive Agent Architecture
Structured as a modular human-like cognition loop:
- Ear (Perception): High-speed acoustic transcription (<350ms latency)
- Brain (Reasoning): Contextual intent extraction, multi-step action planning, and self-correction
- Eyes (Spatial Vision): Coordinate grounding and visual element identification
- Voice (Expression): Natural vocal feedback and audible execution confirmation
3. Zero-Regex & 100% Dynamic Reasoning
Replaces fragile hardcoded commands and brittle CSS/XPath heuristics with deep cognitive reasoning powered by Groq LPU models (GPT-OSS 120B / Llama 3.3 70B). The agent adapts instantly to site redesigns, dynamic SPAs, and responsive layout changes.
4. Multimodal Vision & Pixel-Level Spatial Grounding
Leverages vision-language models (Qwen3-VL-8B and Gemini 3.5 Flash) alongside an in-DOM interactive node extractor. It maps visual screenshots directly to exact (x, y) coordinate click targets and bounding boxes even on complex Canvas, SVG, or shadow DOM elements.
5. Multi-Tiered Self-Healing Failover Matrix
Guarantees uninterrupted task execution through automated cascading failovers:
- Ear Failover: Groq Whisper Large V3 → Self-Hosted Colab Whisper → Chrome Native Web Speech
- Brain Failover: Groq GPT-OSS 120B → Groq Llama 3.3 70B → Google Gemini 3.5 Flash (1M Context)
- Eyes Failover: Qwen3-VL (Kaggle/Colab) → Gemini 3.5 Flash Vision → DOM Interactive Node Tree
- Voice Failover: OmniVoice Urdu → David Executive English (Windows Speech) → Chrome Native TTS
6. Dual Neural Voice Profiles
Bilingual persona engine supporting:
- Urdu Neural Clone: Custom trained/cloned OmniVoice model hosted on high-performance FastAPI backends
- David Executive Assistant: Crisp English business persona using high-definition local and cloud speech synthesis
7. Multi-Tab Navigation & Lifecycle Control
Direct control over Chrome tab lifecycles: open new tabs, switch active contexts, navigate URLs, reload pages, execute deterministic back/forward steps, and coordinate cross-tab workflows from a single unified orchestrator.
8. Deterministic DOM Observation & Action Dispatch
The content engine dynamically numbers, annotates, and calculates visibility constraints for interactable nodes. Actions are dispatched using trusted synthetic user events to bypass anti-automation guards and ensure natural input behavior.
9. Secure Offscreen Audio Streaming
Utilizes Chrome's Manifest V3 offscreen document API to handle microphone capture, audio chunking, and base64 audio transport securely without interrupting background service worker states or browser lifecycles.
10. Cloudflare Worker Edge Router
An ultra-fast edge routing layer built on Cloudflare Workers that handles API authentication, payload transformation, model load balancing, and secure proxying to GPU tunnels (Kaggle / Google Colab / ngrok / Cloudflare Tunnels).
11. Strict Observe-Reason-Validate-Authorize-Act-Verify Security
Operates under zero-trust autonomy. Every planned action must pass strict parameter validation and security policies before touching the DOM. Arbitrary code injection and untrusted eval executions are strictly blocked.
12. Edge API Security & Constant-Time Verification
Features constant-time Bearer token validation to eliminate timing attack vulnerabilities, CORS origin allowlist isolation (restricting access exclusively to approved extension IDs and local debug ports), and automated 2.2s action debouncing to prevent command flooding.
13. Interactive Side Panel HUD & Diagnostics
A sleek, modern side panel UI built with glassmorphism, responsive waveforms, real-time token/latency telemetry, live model health badges, and an audible task transcript feed.
14. Persistent Session Hydration & Service Worker Resilience
Engineered to survive Chrome MV3 Service Worker terminations. Conversation histories, execution states, and active tab pointers are continuously hydrated via chrome.storage.session and long-lived bidirectional port messaging.
Platform Summary
| Area | Capability |
|---|---|
| Acoustic Perception (Ear) | Groq Whisper Large V3 (<350ms) + Colab Tunnel + Chrome Web Speech fallback |
| Cognitive Engine (Brain) | Groq GPT-OSS 120B + Llama 3.3 70B + Gemini 3.5 Flash (1M context) |
| Spatial Vision (Eyes) | Qwen3-VL 8B (Kaggle GPU) + Gemini 3.5 Vision + DOM Semantic Tree |
| Voice Synthesis (Voice) | Cloned Urdu (OmniVoice) + David Executive English + Chrome Native TTS |
| Browser Integration | Chrome Manifest V3, Side Panel API, Offscreen Audio Capture, Scripting API |
| Routing & Edge Proxy | Cloudflare Workers API router with dynamic CORS and failover retries |
| Security & Auditing | Constant-time auth checks, strict origin allowlists, debounce guards, zero code eval |
| Interface | Responsive dark HUD side panel, real-time waveform visualizer, diagnostic cards |