Smart Mirrors: a Makeover for Consumer Research and Physical Retail

Smart home

Between 2015 and 2017, the number of connected households, or smart homes, grew by a CAGR of 31% in the U.S. In it's simplest form, a smart home is a household that contains connected devices. In it's most mature form, all of these connected devices are integrated into a seamlessly automated environment. Connected devices interact with their surroundings via sensors and with external resources via internet connectivity, resulting in a household that is responsive to its occupants needs. For retailers of consumer products, the smart home opens up unique opportunities to understand how consumers engage with their products, and to interact with consumers in their home environment. Smart home devices can include speakers, heating and lighting controls, kitchen appliances, and even ordinarily inanimate furniture, such as a mirror.

Smart mirrors

Several prototype forms of the Smart Mirror technology have been developed with different functionalities. Products such as the Savvy Smart Mirror (from Electric Mirror) serve as a non-specialized smart device, from which various applications can be accessed and viewed directly on the mirror/screen. The user can coordinate their other smart home devices, post on social media, get news or weather reports and more. The mirror essentially serves as an extension of the smartphone, which is integrated into the household.

Beyond being a smart home hub, the Smart Mirror is an intuitive candidate for applications in beauty and cosmetics as these products are almost always used in front of a mirror. The potential for providing a personalized user experience with cosmetics products has not gone unnoticed, as demonstrated with the development of smart mirrors for the home with integrated AR capability for trying on make-up, microphone and speaker for virtual beauty consultations and even software for visual skin analysis, as well as smart mirrors in cosmetics stores.

Smart mirrors in the home

Smart mirrors in the home have the unique advantage of being fully integrated into the user's home and their day-to-day routine. The smart mirror can track the products in the users skincare routine and the condition of their skin in response to these products. The smart mirror could even be used to give personalized beauty consultations in the home. They also represent an interesting opportunity for cosmetics brands to understand how consumers interact with their products, the benefits they see and any adverse reactions. This information can be used to improve products, determine their position within a range and conduct real-time market research.

An example of this technology is the HiMirror, a smart mirror product that is completely adapted for beauty applications with features such as interactive beauty consultations, skin analysis and skincare routine tracking. The technology was developed by a subsidiary of the New Kinpo Group, a multinational Taiwanese electronics company. It has an inbuilt camera, microphone and speaker, enabling it to perform analysis on the user's image and interact with the user. The skin analysis capability collects data on various aspects of the skin, including wrinkles, fine lines, dark circles, dark spots, red spots, roughness, pores and complexion. It can detect changes in the skin over time, the presence of skin conditions and give guidance for skin care. The mirror also has in-built memory to store details of the user's skincare routine, allowing them to track changes in their skin in relation to the products they are using. The most recent models also integrate AR and make-up tutorial capabilities.

While this technology is relatively new, further extensions of this technology could also pick up more abstract factors such as emotion. The capability to detect emotions through facial recognition and micro-expression detection exists in fields such as psychological research and healthcare. In consumer product research, this technology could be applied to understand how the user feels when they engage with the product. For instance, frustration or anger while trying to remove the lid could be detected and used to optimize the packaging of a product.

The convergence of digital and physical retail

The birth of digital retail marked the start of a new competitive era between e-commerce and brick-and-mortar stores. Indeed, the convenience of purchasing at the touch of a button and receiving products at home is hard to beat for a busy consumer. The past ten years have seen terms like the "retail apocalypse" and "death of retail" being associated with the closure of high-street stores, while e-commerce giants flourish. With augmented reality capabilities, e-commerce can begin to deliver on the more personal aspect of product purchasing, usually reserved only for physical retail. On the other hand, physical retail has begun to see some technological innovations of its own with the introduction of smart mirrors into cosmetics stores, marking a convergence of the virtual and physical experience. While the consumer can benefit from the customized convenience of AR technologies, they can also benefit from the full sensory experience of picking up products and interacting with a beauty consultant in person.

Smart stores in action

Charlotte Tilbury

One of the first beauty companies to introduce the smart mirror technology in physical stores was Charlotte Tilbury. The "Magic Mirror" was developed in partnership with Holition, a creative innovation company specializing in digital retail. The AR-enabled smart mirror was launched at two London stores in 2016, and has since been introduced at 11 further locations around the world. The mirror allows the user to try-on up to 10 pre-programmed looks, from which they can select one for a makeover given by a beauty consultant. The flexibility and instantaneous nature of the AR technology is married with the indulgent and personalized experience of a professional makeover, creating a multi-dimensional experience for the customer.

MAC

MAC Cosmetics launched a virtual try-on mirror in stores across the US in 2017. The smart mirror uses ModiFace (now c/o L'Oreal) facial recognition and make-up application technology. It allows the user to test different combinations of MAC products. The user can select lipstick, eyeshadow or blush, and mix-and-match for themselves. The mirror is loaded with 29 pre-set eyeshadow looks which can be selected from in a menu across the bottom of the screen. Here, the smart mirror allows the customer to see how products will look on their face without the hygiene concerns associated with physical testers.

Coty

In early 2018, Coty followed the Smart Mirror trend with a "blended reality" mirror, which was also developed in partnership with Holition. The smart mirror was unveiled at the Bourjois boutique in Paris. Unlike the Charlotte Tilbury Magic Mirror, Coty's version includes another level of integration between the virtual experience and the physical. When a shopper picks up a product and stands in front of the mirror, it automatically appears on their image in the mirror, without the need to search for the product. Coty states "putting the physical product in shoppers’ hands has been found to increase attitudes and purchase intentions of the product, propelling a more seamless path to purchase". Indeed, when surveyed 72% of shoppers said they want a mix of virtual and physical experience. Most recently, Coty have developed a smart mirror that allows customers of the Wella Professionals salons to test different hair colors. This mirror was created in partnership with the beauty tech company, CareOS, and was launched at CES 2019.

Connecting the dots

The cosmetics industry is undergoing an extensive digital renovation. AR is introducing an entirely new aspect of personalization and convenience to both the physical and digital shopping experience. However, the full potential of connected technologies lies in the seamless integration of the augmented virtual world with the physical. Much like the smart home concept, the most mature form is attained when the home becomes one fully integrated device, rather than a fragmented collection of isolated devices. This is a common complaint from consumers when it comes to the smart home - that it yields a "fragmented experience" which does not deliver on the expected convenience. The key to reaching the full potential of connected technologies, both in retail and in the smart home, will be the creation of entirely connected environments.