Real-time embedded intelligence: what the experts actually think

If you missed the roundtable on Real-Time Embedded Intelligence, you missed a discussion featuring 4 experts with more than 100 years of combined embedded systems experience. They still get excited talking about signal grammar, Kalman filters, and Tesla’s decision to disable radar-based sensing. This blog captures the key insights, technical debates, practical advice, and candid observations that are often absent from keynote presentations.

The discussion marked the launch of Real-Time Edge Intelligence Solutions Handbook, a new technical handbook by Arm Distinguished Ambassador Sanjeev Sarpal of Advanced Solutions Nederland. The handbook provides practical guidance for engineers building intelligent systems at the edge, and the panel brought together experts who could examine its ideas and recommendations.

The speakers

Sanjeev Sarpal is a Distinguished Arm Ambassador and the founder of Advanced Solutions Nederland BV. He has more than 30 years of product development experience and has delivered more than 26 commercial DSP-based products. Sanjeev wrote the Real-Time Edge Intelligence Solutions Handbook to address a gap he observed repeatedly. Many academic resources that focus on theory and assessment but provide limited guidance on building intelligent systems suitable for commercial deployment.

Jayakumar Singaram (JK) is an Arm Ambassador and edge AI practitioner based in Bangalore, whose work bridges DSP fundamentals with modern deep learning deployments. You can find more of his work and insights at jkuse.com.

Reinhard Keil is a long-standing contributor to the embedded systems industry. He founded Keil Elektronik GmbH and co-created the Keil C51 compiler which has been the de-facto industry standard for 8051 development since 1988.  Reinhard joined Arm when Keil Software was acquired in 2005 and now leads embedded software strategy and the Edge AI business unit, including the new MDK version 6 toolchain built on VS Code.

Joseph Yiu has spent more than 20 years at Arm, working across CPU engineering, IoT, and system architecture. He is the author of several Arm reference books and served as a technical reviewer for Sanjeev's handbook.

Senior Program Specialist at Arm Jayant Roy moderated the discussion. He brought together 40 years of combined expertise across the panel in a focused 60-minute session.

Why this book? Why now? 

The session opened with a simple question: how does a practitioner with three decades of experience end up writing a textbook?

Sanjeev's answer focused on a gap in existing learning resources. The books that exist, even excellent ones like the classic Oppenheimer and Schafer DSP text, are designed to get you through a university module. "At the end of it you are left with the feeling: OK, I will pass the exam. But how do I actually use this for the project I am working on?"

The Real-Time Edge Intelligence Solutions Handbook is Sanjeev's answer to that question. It consolidates the knowledge base he and his team at ASN have built across 26 commercial products. The handbook covers system architecture, algorithm selection, embedded constraints, sensor integration, and AI implementation. "In many ways, where university textbooks finish, this book takes off."

Joseph, who reviewed the manuscript and wrote the foreword, summarized its value: "There is a real knowledge gap. This area is still very new to most developers. It is already very comprehensive and covers a lot of information that is genuinely useful."

If you are building edge AI systems and feeling the gap between theory and production, the handbook is available here.

"Will Edge AI be affordable for everyone?"

An audience member raised a key question early in the session: is edge AI going to remain the preserve of well-funded programmes, or will it reach everyone?

Sanjeev’s answer was optimistic, and focused on practical developments in hardware. Modern system-on-chip devices now provide significant processing capability at prices below $20. He highlighted Texas Instruments millimetre-wave radar chips as an example; a DSP, an Arm Cortex-R4, and a Cortex-M4 integrated on a single chip at commodity pricing. "Years ago with DSP, it was primarily aimed at military applications; overpriced and very specialized. Thanks to Arm, not only the processor designs, but the ecosystem support and the libraries, everything has lowered the threshold for developers."

Joseph highlighted another important consideration: software costs. "One of the biggest factors in cost is software and debugging. That is why there is a strong need for reusable software frameworks across multiple platforms." This is the role that CMSIS, Arm's Common Microcontroller Software Interface Standard, plays, and why the libraries and tooling that Reinhard's team maintains matter as much as the silicon itself.

The DSP-AI tension: Where it really shows up

The most substantive part of the session tackled a question that many embedded engineers working with machine learning eventually run into: the industry talks about AI as if it replaces classical signal processing. In practice, it does not. So where does that tension live?

Sanjeev was characteristically direct: "Mathematical modelling has always been tricky for engineers. AI gives them a shortcut: a way to produce a model without basing it on hard science. The biggest caveat is that we don't actually understand how the model works."

For mission-critical systems, that is not a philosophical concern; it is a certification blocker. You cannot put a black-box model in a safety-critical control loop and tell the regulator you trained it on a lot of examples. "AI at the moment cannot explain itself. That is why DSP is still the foundation, the quality of features fed to the machine learning model must be based on science, on human understanding of the physical world, designed following physics and mathematics."

The book addresses this directly, and it is one of the reasons the Real-Time Edge Intelligence Solutions Handbook is worth reading, even if you consider yourself primarily an AI practitioner rather than an embedded systems engineer.

JK offered a practical engineering example. Consider a communications modem, a classic DSP application. Engineers understand the waveform structure: the grammar of how it is constructed. Rather than downloading a ResNet-50 and pruning it into a microcontroller, engineers can train a compact network, perhaps with 120,000 parameters, to understand the specific signal grammar of that waveform. The result is a model that is interpretable, embedded-friendly, and robust. "Understand the signal structure and create the model. Do not download a model, prune it, cut it, that is a massacre. And at the end, it is a blame game."

Sanjeev tied it together with a concept central to the handbook: "We are moving away from pure data intelligence to augmenting it with human intelligence, the DSP algorithms, with data intelligence. In the end, you really have edge intelligence."

Autonomous driving and the certification gap 

The discussion turned to autonomous driving . This is one of the sectors pushing hardest on AI as a decision-making layer, and one where the gap between capability and certification is particularly visible.

Sanjeev was candid: "The legislation,  IEC standards and automotive SIL requirements, most of it hasn't caught up with the technology. The strictest levels say: it must be completely explainable. Until we can explain how these models work and prove they are safe in difficult situations, we have a problem. Society as a whole has this suspicion: if I take my hands off the steering wheel, am I going to be alive?"

Reinhard added an industry perspective. ISO 26262 is not evolving fast enough, but ASPICE, the Automotive SPICE process framework, has moved faster and already defines capability levels for AI in safety-critical automotive devices. He also pointed to a pattern emerging in medical device AI: running a classic algorithm in parallel with an AI algorithm and preferring the AI output only when both deliver consistent results. "The AI result is typically better and more precise, but you only trust it when the classical algorithm agrees." Bosch's Etas division has already qualified a TensorFlow Lite Micro runtime built on Arm's CMSIS-NN library for safety-qualified automotive applications.

Sanjeev's view: "I think it will come, but it will take forever. There is so much cool stuff happening in the world, but the certification bodies are very conservative. They really will take a lot of pushing."

The chapter on safety and mission-critical systems in the Real-Time Edge Intelligence Solutions Handbook covers this territory in the detail it deserves.

The chapter Sanjeev almost cut

Towards the close of the session, Jayant asked Sanjeev a question that revealed a lot about how the handbook was assembled: What is one topic you almost did not include, but are glad you did?

Without hesitation, he answered: Kalman filtering.

"I sat there for a couple of days thinking: how can I explain something so mathematically heavy in a simple way? You could easily write an entire book just on Kalman filtering." He nearly cut it to keep scope under control. Then he thought about how many engineers building real-time edge systems already use Kalman filters with sensor fusion, and how little practical guidance exists on choosing between the linear, Unscented, and Extended variants, or comparing them to AI-based approaches.

The result is a chapter that covers all of those variants, draws an explicit comparison between Kalman filtering and AI methods, and gives developers the tools to make an informed choice. Audience feedback has been consistently positive; it is the kind of chapter that demonstrates what makes the Real-Time Edge Intelligence Solutions Handbook different from a textbook. 

Final advice: Four perspectives, one session

The panel closed with practical advice for engineers building intelligent embedded systems. Four people, four distinct takes:

Sanjeev Sarpal: "Do not assume AI can do everything for you. It is fantastic technology, but please do not give up critical thinking, structured workflows, and working with specifications and requirements. Always do your homework."

Reinhard Keil: "Do not rush to hardware. Get the functionality right first, even on a more powerful platform than your target, and make sure you have the right top-level architecture. Architecture decisions are very hard to undo."

Joseph Yiu: "Whether you are doing this professionally or for fun, just start. Use a Raspberry Pi 5 for basic ML experimentation. Learn the technologies. When you move to production, think about the optimal solution; but the learning curve starts with doing, not waiting."

Jayakumar Singaram: "Understand the signal structure. Create the grammar, create the dictionaries, and then bring the learning model to that. Younger engineers today are lucky. They have silicon that can do things we could not imagine 20 or 30 years ago. Use it well."

Get the book and start building 

If this session raised questions you want to explore in more detail, the Real-Time Edge Intelligence Solutions Handbook provides a useful starting point. It is written by a practitioner, reviewed by Arm engineers, and covers topics ranging from foundational DSP principles to building production edge AI systems on Arm silicon.

Real-Time Edge Intelligence Solutions Handbook

To explore the Arm tools and frameworks discussed during the session, visit http://developer.arm.com.  These resources include CMSIS, CMSIS-NN, MDK v6, Fixed Virtual Platforms, SDS, and Ethos-U NPU support.

Stay connected

Real-time embedded intelligence will not be built by models alone. It will be built by engineers who understand the signals, respect the constraints, test continuously, and learn from each other. Stay connected with engineers innovating in this space through the Arm Community Forums and the Edge AI channel on Discord. Join Arm developer events and workshops, and learn from ambassadors and experts like Sanjeev, Jayakumar, Reinhard, Joseph, and the wider ecosystem.

This is a community where developers can learn, share experience, and collaborate with others working on similar challenges.