Satellite Learns to Find Objects Autonomously in Orbit, Powered by AI

For the first time in history, an Earth observation satellite has autonomously identified objects of interest without human intervention. This groundbreaking achievement, reported in April, marks the inaugural use of a vision-language model (VLM) in orbit, signaling a fundamental shift in the capabilities and value of space-based sensors powered by artificial intelligence. The milestone was achieved aboard Yam-9, a spacecraft developed by space infrastructure company Loft Orbital, utilizing a sophisticated software package from NASA's Jet Propulsion Laboratory. Traditionally, Earth observation involves satellites downloading vast amounts of raw data to ground-based analysts, who then employ machine learning or manual inspection to extract insights. However, the Yam-9 mission reversed this paradigm. The onboard software, powered by Google DeepMind's Gemma 3 VLM – purpose-built for edge applications on limited hardware – can process sensor data and respond to natural language queries. Researchers successfully tasked the model with classifying areas where natural environments meet human development and identifying infrastructure around railway hubs, demonstrating its advanced contextual understanding and imagery analysis capabilities. This demonstration carries significant implications. In the near term, it promises to drastically enhance the utility of space sensors by performing initial data triage directly in orbit. This reduces the overwhelming flood of raw data that analysts currently contend with, allowing for more efficient and targeted information delivery. Looking further ahead, this success serves as a crucial proof point for deploying more extensive AI infrastructure in space. Paul Lasserre, Loft Orbital's head of AI, envisions "always-on, patrol layers in space," enabling satellites to monitor specific areas for suspicious activity and interact dynamically with ground teams. Loft Orbital operates on an "infrastructure-as-a-service" model, providing platforms for third-party customers. The Yam-9 satellite, launched in late 2025, acts as a pathfinder for the company's ambitious orbital AI projects and is equipped with a powerful Nvidia Jetson Orrin AGX GPU, a leading chip for space computing. NASA JPL's AI group, led by Juan Delfa Victoria, developed NAVI-Orbital, the software harness for the Gemma 3 VLM. While Gemma 3 is an off-the-shelf model, significant engineering effort was invested to streamline the software package, optimizing it for the constrained memory and processing capabilities of orbital hardware. The success of Yam-9 is expected to catalyze a broader trend within the space industry. Companies like Planet Labs, already flying satellites with Jetson Orin processors for simpler object detection, are actively researching more advanced AI applications, including VLMs. Kepler Communications, operating a substantial array of GPUs in space, has hinted at undisclosed AI use cases since its latest spacecraft launch. Loft Orbital aims to expand its constellation to 50-100 satellites, like Yam-9, to ensure real-time global coverage. The lessons learned from deploying these smaller models will be vital for future endeavors involving larger-scale compute infrastructure in space, particularly concerning power and memory management. Beyond Earth observation, the underlying concept for NAVI-Space originated from JPL researcher Taran Cyriac John's vision for digital assistants aiding astronauts on lunar or Martian missions, offering interactive AI support for complex tasks without requiring keyboard input.