Augmented reality: a digital makeover for the beauty industry

An Evolving Retail Landscape

Beauty stores Ulta and Sephora have not only survived the longstanding “retail apocalypse” and cascading effects of the COVID-19 pandemic but are actually expanding in 2021: Sephora is opening more than 60 new retail stores and Ulta is opening 40 [1]. While it is estimated that the beauty industry declined by as much as 35% in 2020, it is predicted to grow from $511 billion in 2021 to $784.6 billion by 2027 [2,3]. Ulta and Sephora are resilient companies that have evolved during the COVID-19 pandemic by embracing the digital marketplace. One way these beauty companies have stayed relevant is through innovations such as augmented reality (AR).

Augmented Reality

Users of Snapchat, Instagram, or TikTok are already acquainted with AR technology. These applications allow users to add “filters” to their photos or videos. The user therefore sees the real-life environment with a special effect overlaid onto the image. Some examples of Instagram filters include features such as real-time selfie images of the user with different hair colors, or freckles added to their face, or butterflies moving around their head. These filters are complex enough to allow the special effects to move with the user’s movement, so, for example, devil horns and silly sunglasses will stay in the appropriate areas of the head and face. AR is different from virtual reality (VR) because it relies on adding layers to existing real images, whereas VR is composed of completely artificial images [4]. VR technology also typically relies on users

A Digital Makeover for Beauty Brands

The French beauty company L’Oréal has been at the forefront of incorporating AR technology into its advertising and digital strategy. In 2016, it was the first cosmetics company to partner with Snapchat to create a filter for advertising their new Silkissime Eyeliner [5,6]. The filter allowed Snapchat users to see themselves with distinctive, winged eyeliner. Since then, L’Oréal has dived headfirst into the AR realm by acquiring the AR technology company ModiFace in 2018 [7]. As a result, on L’Oréal’s website, anyone can “try on” makeup or hair color virtually.

Most major beauty brands have realized the benefits of AR and have worked to incorporate this technology into their digital platforms, especially as consumers increasingly value immersive experiences with the products they purchase. Following the lead of L’Oréal, Ulta also acquired an AR technology company, GlamST [8]. GlamST technology is incorporated into Ulta’s mobile beauty application, and it too allows users to try on makeup virtually. Similarly, Sephora worked with ModiFace to develop their own in-house virtual makeup applications [6].

The Latest Innovations in AR

1. Advancement with existing AR technology

AR technology has been refined in recent years to ensure that makeup shades are realistic representations of how they appear in real life. One of the main criticisms of virtual try-ons is the fear that the virtual color does not match the real color. To create accurate depictions of makeup colors, companies must take a great deal of pictures of cosmetic “swatches … in different lighting and on different skin tones” [8]. As a result, the virtual try-on feature is becoming increasingly precise.

2. Magic mirrors

Again, showing itself to be an AR leader, L’Oréal has added special AR mirrors to three of its Chinese boutiques. Like mobile application technology, users can virtually try on makeup in real time by looking into a mirror instead of a front-facing cell phone camera [9]. While other AR technology allows users to interact with the beauty brand in their home, this enables an interactive, digital experience in a retail store. Other companies have developed their own “magic mirrors,” such as the Memory Mirror produced by MemoMi and Sephora’s own AR mirror [10].

3. Incorporation of artificial intelligence with AR

Users of virtual makeup applications may have already realized that it can be overwhelming to try hundreds of beauty products. Artificial intelligence (AI) technology has helped streamline this selection process. Examples of AI incorporated into digital beauty platforms are Sephora’s Color IQ (foundation finder) and Lip IQ (lipstick finder) [6]. For Color IQ, users can plug in their known foundation type and the program will generate all the other foundation brands and colors that match the original foundation, allowing users to see other products that will also match their skin tone.

Combining AI and AR technology can further enhance the digital experience of beauty brands by optimizing and virtually testing new products for consumers. One example is Nutox, a Malaysian skincare brand, who developed in partnership with Ministry XR a “skin analysis tool powered by deep learning and computer vision” that is available as a mobile application [11]. Users can upload a selfie to the Nutox mobile application, and the program can detect several skin conditions such as wrinkles or redness. Using deep learning, the program will instantly generate feedback about the user’s skin condition and what products they should invest in [11-12].

The technology company PerfectCorp uses a combination of AI and AR for its application YouCam. PerfectCorp relies on facial recognition and color matching. Both a mobile and in-store application, it allows users to try on makeup from popular brands, including e.l.f. Cosmetics, Benefit, Tarte, Estee Lauder, NYX, and MAC [13-14]. PerfectCorp has also used its technology with Neutrogena in a similar way as Nutox and Ministry XR to analyze skin and offer product recommendations [15].

AR in Cosmetics and the COVID-19 Pandemic

Prior to the COVID-19 shutdown, 85% of beauty product purchases occurred in store as opposed to online [2]. Consumers prefer to try on beauty products in person to feel confident about how the colors or product characteristics interact with their unique skin tones. Beauty product buyers, particularly millennials, insist on testing products before purchasing [16]. The March 2020 shutdown completely disrupted the beauty industry as nonessential stores closed and online shopping was the only option.

When beauty stores were allowed to reopen in June 2020, cosmetic “testers” were removed from the stores due to safety concerns. This is when the previously developed AR tools came in handy. Virtual makeup try-ons allowed consumers a safe, contactless opportunity to try new cosmetic products. The AR tools increased consumer purchasing confidence as they removed “guesswork” from determining correct makeup foundation colors, for example.

Virtual makeup try-on technology is also more environmentally friendly than alternate options. Since cosmetic testers are unavailable, beauty brands feel pressured to increase the production of samples that can be distributed to potential buyers [17]; these single-use samples can generate unnecessary plastic waste.

Future Directions of AR within the Beauty Industry

The incorporation of AR into the beauty sphere is here to stay. L’Oréal has stated that the brand is intending to move from a “beauty company to a beauty technology company” [11]. Many companies will follow L’Oréal’s lead and continue to invest in novel technologies. It is expected that virtual makeup testing will become increasingly utilized within the cosmetics industry. In the post-pandemic economy, “tester” cosmetics may be a thing of the past as we have collectively become more safety-conscious and accepting of contactless transactions. AR will serve as both an interactive and informative experience for beauty brand consumers as they increasingly interact with beauty brands via digital channels.

Bibliography

[1] Thomas, L. (2021, March 21). These 10 retailers are in expansion mode: Here’s who is opening new stores. CNBC. https://www.cnbc.com/2021/03/2...

[2] Gerstell, E., Marchessou, S., Schmidt, J., & Spagnuolo, E. (2020, December 14). How COVID-19 is changing the world of beauty. McKinsey & Company. https://www.mckinsey.com/indus... consumer-packaged-goods/our-insights/how-covid-19-is-changing-the-world-of-beauty

[3] 2021 Beauty Industry Trends & Cosmetics Marketing: Statistics and Strategies for Your Ecommerce Growth. (2019, October 19). Common Thread Collective. https://commonthreadco.com/blo... coachs-corner/beauty-industry-cosmetics-marketing-ecommerce

[4] What’s the Difference Between AR, VR, and MR? (2020, January 7). The Franklin Institute. https://www.fi.edu/difference-...

[5] Caldwell, G. (2016, June 6). Beauty first: L’Oréal Paris runs Snapchat lens ad to promote new eyeliner. Global Cosmetics News. https://www.globalcosmeticsnew...

[6] Winker, K. (2018, December 12). Virtual Reality vs. Augmented Reality in the Beauty Industry. Digital Beauty. https://digitalbeauty.com/virt...

[7] L’Oréal offers first line of virtual makeup for social media, video calls. (2020, November 6). Marketing Dive. https://www.marketingdive.com/...

[8] Metz, R. (2019, February 19). Virtual makeovers are better than ever. Beauty companies are trying to cash in. CNN Business. https://edition.cnn.com/2019/0...

[9] WBR Insights (2019, September 6). Here’s How L’Oréal is Using Augmented and Virtual Reality to

Create In-Store Experiences. Future Stores Seattle 2022. https://futurestores.wbresearc... blog/loreal-augmented-reality-virtual-reality-in-store-experience-strategy

[10] Marr, B. (2019, October 4). The Magic Of Smart Mirrors: Artificial Intelligence, Augmented Reality

And The Internet of Things. Forbes. https://www.forbes.com/sites/b... the-magic-of-smart-mirrors-artificial-intelligence-augmented-reality-and-the-internet-of-things/?sh=3a463710615b

[11] Low, J. J., & Low, J. J. (2020, June 3). Is the beauty industry leading with AR experiences? TechHQ.

https://techhq.com/2020/06/is-...

[12] Skincare Efficacy System — Ministry XR. (n.d.). Ministry XR. Retrieved April 13, 2021, from https://www.ministryxr.com/wor...

[13] Prinzivalli, L. (2018, July 4). How YouCam Makeup is building a beauty AR empire. Glossy. https://www.glossy.co/new-face...

[14] Beauty AR Company and Makeup AR Technology Platform. (n.d.). PerfectCorp. Retrieved April 13, 2021, from https://www.perfectcorp.com/bu...

[15] A.I. in the beauty industry: How the pandemic finally made consumers care about it. (2021, January 12). Fortune. https://fortune.com/2021/01/11...

[16] Shapouri, B. (2016, May 26). The Way We Buy Beauty Now. Racked. https://www.racked.com/2016/5/...

[17] DeFino, J. (2020, August 18). End of In-Store Makeup Testers and Samples in Post-COVID-19 World — Report. Allure. https://www.allure.com/story/f...