Western’s neuromorphic cameras mimic the way that photoreceptors in our eyes only send signals to the brain when activated by light, cutting out vast quantities of non-essential information. The hope is that they will bring much needed cost, power and data efficiencies to space research. 

For space-based applications, where power and data transmission are at a premium, “the amount of data conventional cameras produced is just way more than you could ever stand to bring down from the space station,” explains Matthew McHarg, a research director at the United States Air Force Academy (USAFA) in Colorado Springs, who has been working with Neuro Falcon to ensure the cameras can reach and survive space conditions.

Researchers have previously studied sprites using high-speed digital cameras on satellites or aeroplanes. These cameras contain an array of semiconductor chips that convert light into electrical signals, which are periodically gathered together to create a single image frame. “But if you’re looking at the sky and nothing’s happening, a normal camera produces whole frames of information that tell you nothing,” says Cohen. High-speed, high resolution cameras can produce up to 100,000 frames per second and a vast quantity of useless data.

An array of pixels have been fitted to Neuro Falcon’s cameras, with each pixel wired to operate like an individual camera, only delivering data when there is a change in light intensity falling upon it. “If something moves or suddenly gets brighter or darker — like a lightning strike, for example — those pixels tell you that something’s changed,” Cohen explains. 

Falcon Neuro’s cameras are each about the size of a conventional single lens reflex camera and feature far fewer pixels than a smartphone camera, but they are sufficient to capture sprites. Their primary task is to map exactly where sprites occur, how often they are formed, and what might trigger them. This may also help us to understand more about the origins of conventional lightning, which could feed into weather and climate models. 

Each pixel in Cohen’s neuromorphic camera has dedicated circuitry to process its signals, so that it can immediately report changes in light and time-stamp them with microsecond precision. 

“And if you reduce the amount of data, you don’t just reduce the power draw of the camera, you reduce the amount of power in the processing and the transmission of data back to Earth,” explains Cohen. 

Elsewhere, neurons are also the inspiration behind highly efficient, low power processors, which are attracting significant early innovation investment from NASA, Boeing, Intel, IBM, General Motors and others.  

McHarg runs a research team of cadets at USAFA’s Space Physics and Atmospheric Research Centre who build science payloads destined for space. “We didn’t really know much about neuromorphic cameras,” explains McHarg “So, I was very excited to team up with Greg on this.” One goal, he says, is simply to find out how the cameras operate in the harsh space environment. “We hope that this is the start of a whole bunch of uses of these cameras in space,” he adds.

“It’s a ludicrously expensive business, launching and installing equipment on the ISS,” Cohen points out, recalling that he offered to travel to the ISS himself to fit the cameras. But NASA “very kindly rejected my offer,” he says, laughing. “I said to them once, ‘What if I installed a thumbprint reader, so that you needed me to activate them?’ And they said, ‘We’ll absolutely send your thumb up for you’.”

Western’s neuromorphic first has been made possible through a $5.4 million contract from Australia’s Defence Innovation Hub, awarded in 2019, to develop neuromorphic cameras for use beyond Earth’s atmosphere, the largest ever contract awarded by the funder. 

Gregory Cohen is an Associate Professor in Neuromorphic Systems at the International Centre for Neuromorphic Systems (ICNS) at 91���� and program lead for neuromorphic algorithms and space applications.  

He received a BSc(Eng) Electrical and Computer engineering, a MSc (Eng) and a BCom(Hons) in Finance and Portfolio Management from the University of Cape Town, Cape Town, South Africa in 2007, 2008, 2010, respectively, and a joint PhD in signal processing and neuromorphic engineering from 91����, Sydney Australia and the University of Pierre and Marie Curie in Paris, France.  

Prior to returning to research from industry, he worked in several start-ups and established engineering and consulting firms including working as a consulting engineer in the field of large-scale HVAC from 2007 to 2009, as an electronic design engineer from 2009 to 2011, and as an expert consultant for Kaiser Economic Development Practice in 2012.