Smart things connect to the Internet, smart robots clean carpets, a smart computer controls smart ventilation – so all homes become smart. But what happens when you add AR to the mix?
The concept of the smart home is by no means new. Back in the late 90’s Bill Gates turned one of his estates into a smart house and the electronic equipment alone for this purpose cost a fabulous $10 million (and that Gates did not have normal voice-recognition technology). Now everything that 20 years ago could only be afforded by the richest person in the world at the time is truly available to everyone – thanks to a revolution in home ecosystems, consisting of many small devices. Smart speakers, Wi-Fi lights and outlets, robot vacuum cleaners – all of these are slowly but surely making their way into our apartments and becoming a full-fledged basis for home automation. Creating new meanings and new experiences in the home will be the next step in the development of the smart home, and this step will be helped by AR.
In this article, we will break down several concepts of AR applications in the home and provide interesting examples of how close home automation technology has come to fantasy concepts from books and movies.
Why a smart home needs AR
There are two ways to assemble a smart home ecosystem today. The first one is hardcore and implies designing communications and installing control modules in all important nodes: power, meters, plumbing, lighting, security systems, and so on. The architecture of such a smart house consists of three key components: information gathering devices (various peripheral sensors, cameras, meters), actor devices (those that execute instructions and actually react to triggers) and a single processor – a control device, a hub that defines all the logic of a smart house. As a rule, this device has a “human” interface: a remote control or a mobile application window.
The second way is based on the use of “smart” things – outlets, light bulbs, vacuum cleaners and any other equipment with IoT-modules (Internet of Things). They can be built into the existing infrastructure of the house, and the main communication will be the usual home wi-fi.
AR, regardless of the chosen home automation concept, allows the user to visually interact with the environment. In this very space, it is possible to see in real time a variety of information about the state of the home and will be a natural evolution of the concept of smart home.
What exactly AR brings to a smart home:
- Visual linking of functional device readings. Point the camera to the room and see what switches are on, how much charge is left in the battery of the robot vacuum cleaner, how much rpm the smart fan is humming and how many degrees the air conditioner is giving out. This is a hundred times more visual than a single remote control or a mobile application, the interface of which the average user still has to understand.
- Visualization of electricity and water consumption indicators. Point your smartphone at the meter and get on the screen all the information about costs and unpaid bills.
- Visualization of climatic data in the room. AR shows a complete heat map of the room: in which corner it is cold, where it is warm, how fresh the air is and what is the CO2 content. Useful, for example, for planning of rearrangements and sleeping arrangements.
- Visualizing utilities. When it’s time to hang a picture or add another outlet to the wall, AR will show you where you can drill and where you’ll run into wiring.
- Help in emergency situations. For example, in total darkness, the smartphone will show you where the switch or valve is. And there is another trick, which is more suitable for smart buildings – if the wiring is worn out or the pipe is leaking, the sensor can show the specific place where you need to fix the leak.
The more devices in the ecosystem, the more the control hub collects data that can be processed, interpreted and output in human-readable form – so the number of scenarios for AR applications in a smart home will increase.
The challenges of integrating AR into the smart home
The main challenge of integrating AR into the smart home ecosystem is the inevitable diversity of platforms. They support different data transfer formats (Wi-Fi, Z-Wave, Zigbee) and are controlled by different solutions – OpenHAB, developed in Java, or Home Assistant, written in Python. An AR solution must be able to connect to each platform, which is quite difficult to achieve without the use of universal integration buses.
Another problem is positioning. The AR solution must clearly identify the user’s location. You can put visual markers on every element, but what is suitable for industrial areas and the steel shop, will look out of place in residential areas, where you do not want to spoil the interior with technical details. A more complicated variant is to use acoustic sensors (ALPS), IPS solutions (Indoor positioning system, an analogue of GPS navigation for rooms), Bluetooth beacons, as well as advanced solutions for Wi-Fi navigation by triangulation method – when positioning is based on three points in the room and sensors can be, for instance, Wi-Fi lights.
More advanced solutions use the whole complex of navigation systems: both Wi-Fi triangulation and GPS/GLONASS systems, and sensors of the smartphone itself. Such so-called synergistic positioning is used in the SmartShell system.
Large developers of home automation equipment, such as Samsung SmartThings or Philips, offer their solutions for AR navigation based on positioning within a single smart system. It is enough to select a single synchronization point in the room, such as a bright picture in the hallway or a wall-mounted grandmother clock with a cuckoo, from which the navigation is built throughout the space.
Application developers created on the ARKit and ARCore platforms can use this technology in their solutions. This, for example, is how Smart AR Home works.
A new experience in AR glasses
In the examples listed above, using a smartphone makes interaction with systems more visual at best, but essentially provides the same experience as a regular handy app or remote control. What really changes the paradigm of smart home interaction is the use of AR glasses. Now almost all popular glasses from major manufacturers – Microsoft Hololens, Epson Moverio, Google Glass, Vuzix Blade – can use open home smart system protocols. There are also universal solutions like HoloHome, which can combine AR glasses with home systems.
Using AR glasses in a home environment really changes the experience of interacting with a smart home. For example, with just a glance at the desired device and a slight gesture, you can dim the lights or turn on the air conditioner.
Just like cell phones, AR glasses can work with smart devices through open protocols integrated into digital ecosystems. As computer vision develops, AR glasses will be able to autonomously recognize and control smart devices directly through an ecosystem app.
The combination of AR glasses and voice interfaces (smart speakers like our popular Alice and some less popular solutions from Google or Amazon) may turn the automation of your home into a conceptually new direction and lead to the appearance of home virtual assistants. Ideologically, the market is already ready for this. For example, the Cottage home expert concept from miLAB IDC, which is developing a prototype of a smart assistant, is attracting keen interest.
The development of the home assistant concept not only complements the capabilities of the smart home, but also improves the quality of life of people who, for various reasons, experience difficulties in everyday life. Users with vision problems, cognitive and motor impairments get a tool to help them with everyday tasks, make it easier to navigate and simplify everyday life. This is a very important social aspect of using AR in smart home ecosystems.
Growth points
The use of AR in smart homes is still held back by the fact that the concept of home automation itself is not spreading as fast as we would like it to. Until recently, a smart home was the domain of either very rich people buying turnkey ecosystems or geek-enthusiasts assembling them themselves from opensource solutions. The emergence of home ecosystems from companies like Yandex, Samsung, Phillips, Xiaomi, and other electronics suppliers is greatly accelerating the process.
The emergence of unified integration solutions, like Connected Home over IP (CHIP), created by Amazon, Apple, Google and the Zigbee Alliance, also contributes to the development. The program solves the problem of a single standard for the smart home and allows devices of different manufacturers to form a unified home network without additional dancing with tambourine. In our hemisphere, Yandex and Xiaomi, for example, are doing something similar, and many of their smart devices are compatible with each other.
Another important issue remains the security of home networks. Like any other structure, a smart home can be hacked, disrupted or used for criminal activity. Devices with IoT modules themselves remain vulnerable to hacking and can be used to form networks of bots.
However, according to analysts, these are all temporary difficulties. It is now clear that the market for home automation devices will grow vigorously in the coming years. For example, the European market for smart home appliances is projected to grow from $23.2 billion in 2020 to $39.6 billion in five years. This growth will be due to various factors: the increasing popularity of IoT devices, the rapid development of home smart speakers, the spread of the concept of conscious consumption. In addition, various futuristic concepts will inevitably spark interest in AR in the home.
For example, smart windows and mirrors with built-in displays, which will soon occupy a niche in the market of solutions not only for elite houses and apartments, but also for our usual “man-houses”.
The mass market for smart home solutions still has many virtually empty niches where AR can be used. This includes total home gamification (it’s cool to put a kettle on the vacuum cleaner), energy efficiency and environmental control in living spaces (CO2, lighting), and security and surveillance systems – for children and pets. And perhaps some of our readers right now are already working on a product to solve one of those tasks that we can only fantasize about now.
