Is anything possible? Playing with AI

Is anything possible? Playing with AI

At LEAP 2023, Jack McCauley (Co-Founder of Oculus, Professor of Engineering at UC Berkeley) spoke about his experience of playing with the potential of AI, and developing different capabilities that could change the world we live in.

He described his private R&D facility, equipped with state-of-the-art technologies and highly skilled engineers, where he explores and tests ideas – including ideas he feels sceptical about: “if I doubt something, or something seems like hysteria, I will go and do the reading and the work myself.”

McCauley told us how, around five years earlier, he and his team had built a homegrown neural network. They wanted to find out if they could track something in flight using AI.

It started with an understanding of the human inner ear

McCauley asked the LEAP audience to hold a finger out in front of them. He told them to focus on the fingernail of that hand, and then move their heads from side to side; noticing that as the head moved the eyes stayed focused on the fingernail – but the eyes moved in the opposite direction to the head.

And it was this basic reflex that he and his team used to start creating inner ear electronics when he was leading the engineering team at Oculus.

“The inner ear has structures in it,” he said, “and one of the structures is a series of tubes that has a fluid in it, and that fluid tickles hair inside these tubes, and gives the brain feedback on what the head position is and what the body position it. And it feeds this cranial nerve into a cluster of neutrons — about a thousand of them, out of 86 billion, are dedicated to balance. So it seemed easy to us at the time to replicate that.”

Using a combination of hardware purchased from online retailers, and hardware that had to be built from scratch, the team created a neural network that could track a ball being bounced up and down – much like a human eye can track a ball.

They were doing the inner ear research to find a solution to the nausea that 10% of people experience when they use a VR headset. But McCauley explained that the results of this exploration can also be translated to AI-powered automotive vehicles – taking the ability to track an object and using that to map the road ahead of a vehicle.

The neural network has the capacity to learn

A neural network can be trained, or conditioned, to do certain things – responding to input sets that act like teachers.

“We wanted to teach this thing we built to track something in flight.” McCauley said. To do this, he and his team had to throw a ball in the air in front of a little eyeball they’d built; “and we’d throw the ball over and over again, and eventually the eyeball started catching it and then tracking it.”

It was a slow process – they bounced that ball 50,000 times or more – but eventually, the neural network did something amazing: it learned.

“The beauty of AI like this is I didn’t really have to sit down at a keyboard and type into the keyboard what to do — it sort of taught itself what to do. It was able to do this without my intervention.”

This device, McCauley added, can be used as part of a cluster of devices on the front of a car to track objects ahead of the vehicle; and even determine exactly where and how far away an object is.

But putting use cases aside for a moment, there’s something else to highlight here. The neural network can learn. Seemingly impossible feats of technology can become possible – through experimentation, play, and the power of curiosity. And that makes the future of technology an incredibly exciting prospect.

To find out more on AI...👀

Head over to DeepFest, the dedicated AI stream at LEAP from 4-7 March 2024. DeepFest is where the global AI community comes together to learn, connect, and explore the latest innovations in AI.

Register now to attend LEAP 2024.


Precision therapeutics: Informed by genes and enabled by tech

Technological development is driving major leaps forward in human genomics and healthcare. Scaleable sequencing technologies and AI-powered automated data analysis are expediting discoveries in the field of life sciences; and driving a new wave of healthtech companies that have real potential to solve the most serious health challenges faced by

How tech talks to the world

Natalia Brzezinski (Founder and CEO at Absolute Strategies) is the former Global Business Development lead at Klarna, and the former CEO of Brilliant Minds Foundation – bringing together new disruptors and established business groups to explore the relationship between tech and humanity.  She understands that communication is at the heart of

Tech in the brain: A mission to advance BCIs

Professor Thomas Oxley (CEO at Synchron) is a vascular and interventional neurologist – and a world-leading expert in brain computer interfaces (BCIs). He has performed more than 1,600 endovascular neurosurgical procedures, giving him a unique perspective into the human brain.  As founding CEO of Synchron, he’s on a mission