AI in Your Pocket: A New Era of Possibility on the Horizon
There is a quiet revolution happening in local AI
It's 2035, and a pre-teen in Lagos is arguing with a friend about how to combine proteins in Chemistry class. To settle the argument, he pulls out his glass assistant. The device details the exact protein recipe for the result, the two different ways to get there, and which is the faster method. The device uses its on-device model to accomplish this; no need to check with Open AI’s new GPT X1 model running in a subterranean data center in the Antarctic.
At some point in the medium future, every kid will have an AI assistant with a thousand PhDs in every topic they will ever be interested in. And it will work seamlessly, even with limited connectivity.We're witnessing a remarkable convergence: the simultaneous arrival of desktop/server-class portable computing and transformer-based AI models. This intersection isn't just another incremental step in technology—it's a fundamental shift in how we might interact with computing power in our daily lives.
The Rise of Pocket-Sized Power
The latest processor architectures—Apple's A17 Pro for iPhones, M4 for Macs and iPads, and their Intel equivalents—have essentially erased the traditional boundaries between mobile and desktop computing. These aren't just the usual marketing claims; they're running AAA games at 1440p, managing AI workloads through optimized neural processing units, and replacing workstation-class computers for many professional workflows, from video editing to software development.
And with WiFi 7 offering theoretical speeds up to 46 Gbps and WiFi 8 promising even more, these portable powerhouses are becoming indistinguishable from traditional servers in terms of data throughput as well. A modern iPhone or mini PC can match the performance of a data center server from a decade ago, at least in terms of raw processing capability and network speed.
The implications are profound: your pocket device could potentially serve as a personal server, running applications that others can access, storing and processing data locally, and managing complex workloads that were once the exclusive domain of ‘big iron’.
The AI Revolution for Edge Computing
The timing of this portable computing renaissance couldn't be more significant, coinciding with the emergence of more efficient large language models and AI systems, whose performance challenges that of the first generation of large language models, which once required substantial data center resources.
The implications of running sophisticated AI models on ultra-powerful portable devices extend far beyond simple voice assistants or image recognition. We're entering an era where these devices can run substantial portions of large language models locally, enabling new types of applications that weren't previously possible:
Real-time language translation without internet connectivity
Wherever you are, you will have access to all the languages on the planet, or at least the most popular hundred. We will approach the Tricorder shown in the Star Trek series. This will be the real deal, not the clunky attempts we have today.
Local processing of sensitive business data through AI, without cloud transmission and exposure
Companies will be able to control their privacy and secrecy exposure by taking advantage of local AI for certain workloads, only using public and private cloud-bound models as needed.
AI-enhanced creative tools that can run at full fidelity without server round-trips
Whatever you find magical about LLMs today will be simply run on local devices in a few years because of Moore’s law.
Today’s popular small models like Llama 3.3, Deepseek-r1, and Mistral Small 3 can already do serious damage to fairly complicated inference tasks. Their successors will be even more capable while staying small enough to fit on a small device, a future that Apple is banking on getting here very quickly.
Contextual computing that combines local sensor data with AI processing
AI is coming for day-to-day automation. You will see this technology in industrial settings, but it will also propel the home automation market to new heights.
Today, you can buy network-attached storage (NAS) with Neural Processing Units (NPUs) that hobbyists can run homelab AI workloads with. The neural engines in these devices are specifically designed for AI inference workloads. This means they can run complex transformer models with surprisingly low power consumption, making AI processing practical in mobile and compact scenarios.
Hobbyists are blazing the trail of edge AI
The communities that use HomeAssistant, Frigate, and ProxMox are cobbling together some incredible edge AI use cases and applications.
For years these hobbyists generally have an ethos of running open-source applications that do not depend on the major cloud vendors. Their attention is turning towards self-hosted AI, and they are busy replacing Alexas, Google Homes, Chromecast, Spotify and more; using open source software and cheap hardware you can get from Amazon and Alibaba.
In some cases, they are creating custom hardware and sensors using a framework called ESPHome that is constantly being updated with thousands of sensors - presence, temp, humidity, ambient sound, etc. These platforms make complex hardware and sensor programming accessible to even motivated teenagers.
We’ve verified that it's possible to build a self-hosted home automation system that can be directed by voice and supports open conversations just like Iron Man’s Jarvis!
These efforts are niche but they portend big advances in edge AI to come.What's particularly exciting is how this local AI processing could interact with vehicle systems through platforms much like CarPlay Ultra.
The next version of CarPlay is designed to take over every screen in your vehicle—from your instrument cluster to your climate controls. Its evolution will be a paradigm shift in how we think about vehicle computing.Imagine real-time processing of sensor data combined with AI models for enhanced safety features, all running from your phone, but seamlessly integrated with your vehicle's neural processing systems.
Edge AI is coming whether we like it or not
The democratization of AI processing power in portable form factors means we're likely to see an explosion of innovative applications that combine local processing and edge computing capabilities with the larger cloud-bound models. This isn't just about running existing AI models more efficiently—it's about enabling entirely new categories of applications that weren't practical when AI required constant cloud connectivity and server processing.
We predict that while companies like OpenAI have strong designs on building the next emergent proprietary consumer and work operating system, there will be a resurgence in systems like Linux and other open platforms, to truly unlock the next wave of AI silicon that will transform the world as we know it.
The future of computing is not just portable and powerful—it's intelligent in ways we're only beginning to understand. If you can even imagine it, expect daily life to be transformed at an even faster clip than the last 5 years.
Check out our latest convo about frothy AI acquisition deals….and our podcast on YouTube.
Want to dive deeper?
Check out our book BUILDING ROCKETSHIPS 🚀 and continue this and other conversations in our ProductMind Slack community and our LinkedIn community.