Smarter than your average dog
Built by Texas A&M engineering students, this AI-powered robotic dog sees, remembers and responds with human-like precision making it a powerful ally in search-and-rescue missions.
video: Meet the robotic dog with a memory like an elephant and the instincts of a seasoned first responder.
Credit: Logan Jinks/Texas A&M Engineering
Meet the robotic dog with a memory like an elephant and the instincts of a seasoned first responder.
Developed by Texas A&M University engineering students, this AI-powered robotic dog doesn’t just follow commands — it sees, remembers and thinks. Designed to navigate chaos with precision, the robot could revolutionize search-and-rescue missions, disaster response and many other emergency operations.
Sandun Vitharana, an engineering technology master’s student, and Sanjaya Mallikarachchi, an interdisciplinary engineering doctoral student, spearheaded the invention of the robotic dog that never forgets where it’s been and what it’s seen. It understands voice commands and uses AI and camera input to perform path planning and identify objects.
A roboticist would describe it as a terrestrial robot that uses a memory-driven navigation system powered by a multimodal large language model (MLLM). This system interprets visual inputs and generates routing decisions, integrating environmental image capture, high-level reasoning, and path optimization, combined with a hybrid control architecture that enables both strategic planning and real-time adjustments.
Robot navigation has evolved from simple landmark-based methods to complex computational systems integrating various sensory sources. However, navigating in unpredictable and unstructured environments like disaster zones or remote areas has remained difficult in autonomous exploration, where efficiency and adaptability are critical.
While robot dogs and large language model-based navigation exist in different contexts, it is a unique concept to combine a custom MLLM with a visual memory-based system, especially in a general-purpose and modular framework.
“Some academic and commercial systems have integrated language or vision models into robotics,” said Vitharana. “However, we haven’t seen an approach that leverages MLLM-based memory navigation in the structured way we describe, especially with custom pseudocode guiding decision logic.”
Mallikarachchi and Vitharana began by exploring how an MLLM could interpret visual data from a camera in a robotic system. With support from the National Science Foundation, they combined this idea with voice commands to build a natural and intuitive system to show how vision, memory and language can come together interactively.
Like humans, the robot uses reactive and deliberative behaviors and thoughtful decision-making. It quickly responds to avoid a collision and handles high-level planning by using the custom MLLM to analyze its current view and plan how best to proceed.
“Moving forward, this kind of control structure will likely become a common standard for human-like robots,” Mallikarachchi explained.
The robot’s memory-based system allows it to recall and reuse previously traveled paths, making navigation more efficient by reducing repeated exploration. This ability is critical in search-and-rescue missions, especially in unmapped areas and GPS-denied environments.
The potential applications could extend well beyond emergency response. Hospitals, warehouses and other large facilities could use the robots to improve efficiency. Its advanced navigation system might also assist people with visual impairments, explore minefields or perform reconnaissance in hazardous areas.
Dr. Isuru Godage, assistant professor in the Department of Engineering Technology and Industrial Distribution, advised the project.
“The core of our vision is deploying MLLM at the edge, which gives our robotic dog the immediate, high-level situational awareness and emotional intelligence previously impossible,” said Godage. “This allows the system to bridge the interaction gap between humans and machines seamlessly. Our goal is to ensure this technology is not just a tool, but a truly empathetic partner, making it the most sophisticated and first responder-ready system for any unmapped environment.”
Nuralem Abizov, Amanzhol Bektemessov and Aidos Ibrayev from Kazakhstan’s International Engineering and Technological University developed the ROS2 infrastructure for the project. HG Chamika Wijayagrahi from the UK’s Coventry University supported the map design and the analysis of experimental results.
Vitharana and Mallikarachchi presented the robot and demonstrated its capabilities at the recent 22nd International Conference on Ubiquitous Robots. The research was published in "A Walk to Remember: MLLM Memory-Driven Visual Navigation."
By Jennifer Nichols, Texas A&M University College of Engineering
###