Source: arXiv — 2026-07-14
Summary
PalmClaw is an open-source agent framework that runs the entire agent loop — session management, memory, skills, and tool use — natively on-device on a mobile phone, rather than driving the phone through GUI tap-and-swipe automation the way most mobile agents currently work. The paper positions device-native tool-calling as a fundamentally different, more capable paradigm than screen-based automation.
Key Takeaways
- Most existing mobile agents work by taking screenshots and simulating taps/swipes to operate apps as if a human finger were doing it — a fragile, screen-reading-dependent approach that breaks whenever a UI changes.
- PalmClaw instead runs the agent's core loop (memory, skill selection, tool invocation) directly on the device, calling native tools and APIs rather than visually navigating a UI, which sidesteps the fragility of screen automation entirely.
- Running natively on-device also has direct latency and privacy implications: no round trip to a cloud agent-orchestration service is required for the core loop to function, and phone-resident data doesn't need to leave the device to be acted on.
- The framework is open-source with a working demo, making its architecture directly inspectable and reproducible rather than a description-only paper.
Reel Script
Hook (16s / 36 words) Most "AI agents that use your phone" are really just bots faking human finger taps on a screenshot. A new open-source framework skips that entirely — the agent runs natively on the phone itself.
Core Concept (60s / 130 words) Here's how mobile AI agents typically work today: the agent takes a screenshot of your phone, an AI figures out where the buttons are, and it simulates a tap or swipe at those coordinates — essentially pretending to be a human finger. That's fragile, because if an app's layout changes even slightly, the agent's whole understanding of "where things are" breaks. PalmClaw takes a completely different approach: instead of visually operating the screen from the outside, the agent's actual reasoning loop — its memory, its skill selection, its tool calls — runs natively on the device and calls the device's real APIs and tools directly. It's the difference between someone watching your screen over your shoulder and clicking for you, versus someone who actually has the keys to the system and can just act.
Hands-On (55s / 120 words) The architecture worth sketching out: a native on-device agent loop with three core pieces — a memory store that persists across the phone's own sessions, a skills layer (discrete capabilities the agent can invoke), and a tools layer that calls real device APIs instead of simulating UI interaction. Because everything runs locally, there's no dependency on a round-trip to a cloud orchestration service just to decide the next action, and phone-resident data can be acted on without leaving the device. The paper backs this with open-source code and a working demo, so the architecture isn't just described on paper — it's something you can actually run and inspect.
Takeaway (22s / 48 words) If you're evaluating mobile agent frameworks and defaulting to screenshot-and-tap automation because "that's how it's done," this is a working counter-architecture worth a look — especially anywhere latency or on-device privacy actually matters. Go check out the open-source repo.