Adrian Whelan, Intel: Augmented Reality Brings 3-D to Retail

December 14, 2011

Lego's Familiar Building Bricks Come Alive Using In-Store Technology the Danish Construction Toy Maker Hopes will Enhance the Customer Experience

Using augmented reality, a customer in the Sacramento, Calif. Lego Store gets a better look at the Millennium Falcon set from Lego's Star Wars collection.

As brick-and-mortar stores compete not only with themselves, but the convenience of e-commerce, a company whose core product is bricks without the mortar is set on improving an already unique shopping experience.

Digital Box, an interactive kiosk jazzing up branded Lego stores that allows customers to see fully assembled Lego products come alive on screen, was one of the first examples of augmented reality in the retail sector when it debuted in 2008. In a refresh for all Lego stores, older systems are now being replaced with a second-generation model geared to improve the in-store experience for consumers and net sales for the company.

"The Lego Digital Box is unique in that it was one of the first massively implemented in-store AR experiences," said Dr. Thomas Alt, CEO and co-founder of Munich-based Metaio, software provider for both generations. "The initial trial period was so successful in increasing important benchmarks, like customer satisfaction and product sales, that now there is an in-store kiosk in every Lego-branded store in the world."

Augmented reality is the technique of overlaying graphics on a real-world image so the graphics enhance and recontextualize the scene. With Digital Box, customers can see how the Lego products, some with thousands of pieces, will come together without ever opening an actual box. It's not streaming video, but a real-time visual controlled by the individual. When a customer holds up a product to the screen what pops up is a 3-D animation that changes as the box is moved around. Hold up the City Corner set to the camera and the box comes alive with a scene of Lego Minifigures getting on a public bus, patronizing a pizzeria and skateboarding on the sidewalk, just as it might be played when built.

Besides the fun factor, Digital Box can help assess potential return on investment. Is the $240 Lego London Tower Bridge worth the money? What about the investment of time to build from a whopping 4,287 pieces? A 3-D rendering of one of the most ambitious and priciest of Lego's projects, as opposed to just the 2-D box photo, might win over a shopper.

Spending more than $200 on a Lego set wasn't what Shawn Anbiah had in mind when he shopped at the Sacramento, Calif. Lego store on Black Friday, but the high school senior, an admitted technology fan, recognized the value of Digital Box.

"I was just browsing today, but if I was looking for toys to buy as a gift it would definitely help me make a decision," said Anbiah, 17, of Folsom.

Second-Generation Improvements

The most noticeable difference between the first and second generations is capacity. The 3-D experience that has been limited to 24 Lego products as powered by an Intel Xeon Core processor will accommodate 200 with the upgrade to Intel's second-generation Core processor.

"We will now be able to include virtually all of our products, meaning that a consumer will be able to pick products up from pretty much anywhere in the store and have a fun experience with the virtual contents of the box," said Justin Tripp, vice president of Lego retail stores.

One of the first examples of augmented reality in the retail space, Digital Box (along the right-side wall) is a standard feature in Lego brand stores worldwide, including this Denmark location.

The latest rev of Digital Box, planned for integration in all 80 Lego brand stores as early as mid- 2012, also improves performance while shrinking the unit's size, according to the Denmark-based company.

"We've increased content and at the same time created an improved customer experience," said Olav Gjerlufsen, digital director of Lego 3-D Flow. "You don't need the huge case with the new design."

Gjerlufsen said he no longer will use "bulky" and "limiting" to describe Digital Box.

"Working closely with Intel's developers helps us create the augmented reality system of the future," said the Lego director. "The built-in graphics of the 'Sandy Bridge' platform improves the overall performance and overcomes some of the bottlenecks we had with the first generation."

For the near-10-fold increase in showcased product, Gjerlufsen said, digital 3-D graphics from Lego Digital, software from Metaio and the multi-threaded Intel chipset have to work in parallel. Doing so overcomes such challenges as recognizing and tracking hundreds of boxes and rendering the multimedia animations smoothly and fast. "Tracking," in this instance, means using the image of the packaging as optical reference for true-to-scale and true-to-position display of the referenced information.

The Bottom Line

Cool as the technology is, it's more important for the international toy giant to keep customers interested in the bricks and plates inside a Lego box than the nuts and bolts inside Digital Box. If the common sight of customers clustered in front of the interactive kiosks is any indication, the Lego Group's foray into augmented reality is a hit.

"Research within stores indicates that customers love the technology and confess that it definitely influences their purchase," Tripp said, adding that the company hasn't taken specific measures to learn if Digital Box drives sales on particular product lines.

Research conducted by Intel in the early stage of development of the second-generation Digital Box found that it's more a marketing tool than a sales tool.

"Digital Box gives a different experience in the store. It's branding, it's more interaction with the public," said Adrian Whelan, Intel's embedded new business director in Europe. "Kids have fun with it, parents can physically see what the product looks like before buying, and sales people are able to get customers of all ages more excited about the products."

No argument from Alt, who said interactive, camera-based experiences have some of the most potent stopping power in the industry.

"Consumers spend up to seven times longer engaged with AR than a poster or print ad," he said. "Also, initial studies show that people are 64 percent more likely to purchase a product after engaging in an AR retail experience."

Growth in the Retail Space

The ink on the final specs of the second-generation Digital Box is barely dry, yet discussion is already underway for a third version that can augment reality for yet another 100 Lego products, according to Whelan. For now, at least as far as the customer-facing side of the Lego Group is concerned, it's all about rolling out Digital Box II.


"Depending on the five-store trial and the budget, I would hope to have [the second generation] in all of our stores by mid-year 2012," Tripp said. "As fast a rollout as possible is desired as our objective at Lego is to innovate and capture children's imagination with both our stores and our products, the Digital Box helps us on this journey."

Lego is hardly alone in using technology to develop engaging retail experiences. Augmented reality is also a reality at Macy's stores this holiday season. As a twist to its 3-year-old "Believe" campaign that invites children to send letters to Santa using special mailboxes in stores, customers with iOS and select Android smartphones can download a mobile app from Metaio that allows them to interact with characters from the animated TV special, "Yes, Virginia." Customers can step in the frame and take a 3-D photo, which can also be uploaded onto a holiday card template to share via email or Facebook.

An example of augmented reality in eRetail is Ray-Ban's Virtual Mirror, which allows online shoppers to virtually try on and sample eyewear while sitting in front of a computer.

Whether augmented reality will become ubiquitous in the retail space is debatable, but its growth is less so, according to Alt, adding that he sees the industry's use of the technology as more of a paradigm than a trend.

"AR is an interface for retail, a visual platform to display information about any given retail product," Alt said. "It's no more of a trend than digital signs, online shopping or even mannequins. Retailers are always looking for efficient, useful ways of displaying their products whether it's in-store, online or mobile. Augmented reality provides a new, engaging way of positioning merchandise in all three venues."