AI in the Home: Andrew Paulson in Conversation with Jonny Cox

Last time we spoke to LightField founder Jonny Cox, he shared his vision for an in-home immersive experience inspired by the holodeck in Star Trek. Just five months later, that vision is now set to become a reality, with LightField building a prototype room, ‘Sensorium’, in its own office.

As part of LA London’s collaboration with LightField to bring in-home innovation to our clients, partner Andrew Paulson spoke to Jonny Cox about the potential of this new technology for our homes, schools and workplaces.

Jonny Cox (L), founder of LightField and Light Architecture. Andrew Paulson (R), managing partner at LA London.

Andrew Paulson  We’ve been talking recently about innovation in the Internet of Things and AI, and how developments in these might benefit our clients at LA London. Many of us already have an Alexa or similar to manage home systems like lighting, heating and entertainment, but what do you think is next when it comes to AI in the home?

Jonny Cox  I’ve just downloaded Chat GPT-4o1, which you can have a normal conversation with, and its latency and speed are really good. You can interrupt it and redirect it as it responds to you. Open AI claims it can reason. I’m using it as much as I can, trying to push its capabilities and see how far I can take it.

AP  What sort of thing are you using it for?

JC  We plan to introduce an AI avatar into Sensorium. We’re building our own large language models to allow us to interact with the avatars on screen. The more data and systems that feed into them, the more they’re going to be aware of what’s actually happening.

We can install sensors that monitor how many people go into the room and track what they’re doing. We’ve recently worked with Foster + Partners on their new building for JP Morgan in New York, which will be completely powered by renewable energy. The building constantly monitors itself for efficiencies, knowing who’s in the building and where, locating lights in one room and turning them off in others.

AP  Similarly, we use building management systems that monitor different aspects of the building. So, for example, it identifies if there is less pressure in a pipe than usual, which might indicate that there’s a leak.  

There are property management companies that look after absentee owners’ properties in London. The owners might only be there for two or three weeks a year, but they need to know that at any moment they can arrive and everything will work properly.

“The data’s all there and it can be distilled into whatever level of detail the client wants.”

JC  That’s a really good point. You need to be able to bring that scale and sophistication of a 60-storey office building like JP Morgan’s to a single apartment, and have that granular level of data that can be played with by the AI.

AP  Some people are more techy than others, but the data’s all there and it can be distilled into whatever level of detail the client wants. Speaking for myself, there isn’t a room in my house that doesn’t have an Alexa! If I’m going away overnight, I give a particular command and then the system is set so that it also arms the security cameras. Things like that are already possible but they can be made a lot more accessible and customisable.

JC  Exactly. We need to bridge that gap for people who may not be so confident with new technology. AI is going to help that shift, to bring it to people in an easy way. Alexa and Siri have started to do that, but it still feels clunky.

AP  Where do you think AR (augmented reality) is going to take things?

Jonny Cox and Light Architecture co-founder Hernan Kraviez.

JC  Mark Zuckerberg has done what I wanted Tim Cook to do. When Apple released the Vision Pro they seemed like just another set of goggles, and they cost $5,000. Meta has just extended its partnership with EssilorLuxottica, the company that makes RayBan lenses. They’re making glasses where every piece of the frame is tech, but they look like a normal pair of RayBans.  

At the moment, we have to use too much bandwidth with a phone – we’re giving it our time and our attention, pressing buttons. These AR glasses would remove that barrier by using an overlay that can see what you’re seeing. And you could do simultaneous translation so you can have live conversations in different languages. I’m a firm believer that glasses with an AR overlay will be the next step. 

AP  Can you tell us more about Sensorium?

JC  Sensorium is a next-generation, multi-dimensional experiential room designed for the home of the future. In essence, it creates infinite moods, environments, destinations, memories, thoughts, and ideas in your personal, real-time 3D digital space.

I’ve been a computer gamer for a long time. I always wanted this ability to transform a space into anything.

I’ve been a computer gamer for a long time and it was always something I wanted, this ability to transform a space into anything. It’s like Star Trek, where you’ve got a group of people on a spaceship millions of miles from home, and the Holodeck lets them transport themselves back to earth. It’s a reminder of the places they loved.

Now we’ve built one in our studio at LightField – a floor-to-ceiling projection room, six metres by six metres, as a case study. For a residential client it might be in a basement space with a set of screens connected to our system, Unreal Engine. Clients could go down there for a yoga or meditation session and transform the room into whatever environment they wish: a Himalayan viewpoint, the desert, or the beach.  

But it’s not just a visual experience – we want to stimulate other senses too, with 360-degree sound and smell – to get people as close to the real experience as possible. From a wellness perspective, we also know that infra-red light, i.e. the light that you can’t see, is what makes us feel good, so we’re working on introducing infra-red light into the space.

The Sensorium experience.

AP  And then there’s the floor, which you can walk or run on.

JC  Yes, Disney has developed a technology called the Holotile, which allows you to walk without actually going anywhere. It’s an omni-directional floor, so multiple people can be walking independently, and you can even use it to move furniture around.

AP  What’s exciting is that there’s so much potential beyond residential. We were talking about healthcare and education.

“The idea is that you’ll be able to speak to the technology and instruct it. The AI will create the space.”

JC  Cruise liners, oil rigs, prisons. It’s super exciting. The technology could be installed in a hospital where there’s dead space and used for staff training or patient rehabilitation. Anywhere where you have to stay inside for long periods.

Our goal is to enrich people’s experience. So, for example, you could use the space as a virtual library. We’re working with publishers to feed books into our large language models so that you can visualise a book as you’re reading it. Reading Oliver Twist, for example, you could be transported to Victorian London. The idea is that you’ll be able to speak to the technology and instruct it. The AI will create the space.

AP  I can imagine so many of our clients wanting this. If you’re the first person to have this in your home, you’re going to have a lot of guests wanting to come to visit.

JC  For me it has to be a curated experience – both in terms of how we create the content and the architectural element, working with LA London and the team. It’s curated to each individual client.

AP  How long do you think it’s going to take?

JC  Creating content for the client is the straightforward part. Now we’re looking for the right partner for the AI aspect, the interaction with the avatars. We could be looking at a year from now. The prototype room’s nearly done. The technology is here. It’s just a case of joining the dots.

For more information on LA London’s technological innovations in the home, view our portfolio or contact us.