The worker pulls on his tool belt and snaps down the visor of his headset. It powers on, loading the 3D model of what he needs to build—in this case, the metal framing for a hospital bathroom pod. The 3D image snaps into place, staying affixed to the floor and ceiling as he moves around it. Without the aid of a tape measure or layout laser, the worker begins assembling the framing, matching the pieces to their ghostly hologram outlines and fastening connections where the model dictates.

It was only a proof-of-concept demonstration, set up in a controlled space with proper lighting for the Microsoft HoloLens to function correctly. But the results of the test by Gardena, Calif.-based contractor Martin Bros. were promising, says Cody Nowak, the company’s VDC and BIM manager.

“We did the test with the president of Martin Bros., who hadn’t picked up a tool belt in 20 years,” he says. “We put the HoloLens on him, and he built that bathroom pod.” Nowak says that, although the pilot project was in a controlled environment, building a bathroom pod that would normally be prefabricated off-site, it shows real potential for mixed reality. “We’re one of the first to utilize holograms for construction layouts,” says Nowak. “It’s still early, but someday we will be able to build directly from the BIM model on the site.”

Mixed-reality and augmented-reality systems are distinguished from the isolation of virtual reality by the ability to see what is actually around the viewer. “In VR, the context is wholly fabricated by design—what you see is completely fictional and might be based on something that is real, but you’re in a world that’s wholly for your eyes,” explains Alan Robles, associate at Gensler in charge of interactive design. “With augmented-reality and mixed-reality systems, having a device that can load your digital assets into the built environment is hugely important to being able to communicate intent.”

Devices that are unaware of the ambient environment and simply project floating images into the user’s field of vision are generally classified as augmented reality, whereas mixed-reality systems have sensors that can detect nearby surfaces and project images onto them. This contextual awareness allows a model to stay in place on a table as a user walks around it or a 1:1 scale model to be mapped onto a room’s dimensions. And since MR doesn’t isolate the user like VR, it allows for multiple headset wearers to consider and discuss the same model at once, in person.

Moving Beyond Virtual Reality

Virtual reality devices, such as the Oculus Rift and the HTC Vive, can provide immersive experiences, but it’s at the cost of situational awareness. “In 2008, we convinced our company to build one of the first VR ‘CAVEs’ in Spain,” recalls Daniel Garciá Espinel, technology transfer manager for Madrid-based ACCIONA. “We took a lot of people in to see their projects. It was a good commercial tool.” Workflows were largely unchanged by VR, so VR became more of a tool to show to clients. “But this mixed reality and the HoloLens involve a jump forward because, in this case, it’s beyond a commercial tool. It’s a real useful tool for understanding sites,” he says.

The challenge now is to get the data condensed into a form the headsets can handle, says Aviad Almagor, director of Trimble’s mixed-reality program, which includes the HoloLens pilot projects. “The goal is to have one-click mixed reality,” he says.

“Mixed-reality devices will go through a standard adoption cycle,” predicts Stacey Scopano, AEC senior strategist with Autodesk. “I don’t think our industry realizes how disruptive immersion as a deliverable will become in the next decade.” Autodesk’s mixed-reality efforts have seen some success, and Scopano expects its large investment in VR technology to translate to MR platforms as they get more mature. “We’re moving data into HoloLens and into Apple’s and Google’s projects,” he says. “We’re fortunate that enough of our market has a 30-year run of digitizing content, our workflows are baked, and our customers know this is the natural evolution of what they are already doing.”

The collaborative aspect of MR is one of its main attractions, according to Chris Thorn, principal technology and process consultant at AECOM. “For us, it’s the quality of communication,” he says. Engineers usually tell their issues to a senior engineer, who has to explain them to another team during a visit. “But by using mixed reality, the engineers can have those conversations directly. That quality of communication can be improved. We have fewer conversations about which grid line we’re talking about,” Thorn says. Unlike in VR, Thorn says, teams can engage more directly in MR.

Thorn is running a mixed-reality pilot project out of AECOM’s Plymouth, U.K., office in collaboration with Trimble and Microsoft. He also has developed a quick-start guide for MR users as well as a workflow for using MR in the design and review process. “We used it for our work on the Serpentine Galleries, these prestigious architectural projects done each year in Hyde Park [London],” he notes. Thorn points to the design of an elaborate, ribbonlike sculpture for the event that required complex steel supports. “The advantages of working with HoloLens is that you can open it up in a big-enough conference room and see it at 1:1 scale. You can review the fabrication model in the room, looking around it and walking around it, with a sort of natural spatial awareness. We found additional problems that we couldn’t find on the screen because you couldn’t get the view just where you wanted it,” he says.  

The HoloLens required a bit of a learning curve for the team—and not just the busywork of building simplified models that the headset could handle from the main design pipeline, according to Thorn. He recalled finding a HoloLens-wearing senior team member lying on his back on a conference-room floor to get a closer look at one detail of the holographic 3D model—unaware the MR model-viewer controls allowed for zoom and rotation.

When Augmented Reality Is Enough

Not everyone is waiting for the HoloLens to be jobsite-ready. The contextual awareness of mixed-realty systems has drawn a great deal of interest in the industry, but there are already augmented-reality solutions available or coming soon that can be used in the field. ODG smart glasses from Osterhaus Design Group can provide augmented reality in a lightweight, glasses-like form factor. Others are trying to mimic safety helmets. The Daqri Smart Helmet (see story, p. 51) has received OSHA PPE certification. While it initially will target industrial settings such as petrochemical facilities, it also could be adapted to construction environments, offering heads-up displays of useful information and possessing a sensor array for basic mixed-reality.

But augmented- and mixed-reality systems such as these and the HoloLens are held back by not being safety-certified. Even those that eventually get approval likely will draw the ire of safety managers who object to jobsite workers with constant distractions in their fields of view.

Due to these safety concerns, the first AR and MR systems to make it to the jobsite will likely be tablet-based. Tech start-up Scope AR already has launched its augmented-reality training software for industrial and manufacturing workers (ENR 6/13/16 p. 47) and is working to bring its technology to construction. The company recently announced a partnership with equipment manufacturing giant Caterpillar to bring its tablet-based augmented-reality systems to fleet maintenance. “You now have the opportunity to have an expert back in the office with expertise you don’t have,” says John Carpenter, Cat’s manager of global construction technology and solutions. “For example, there’s a visual aspect being drawn into the tablet view as the expert brings in that expertise you don’t have.”

When Will It Be Ready for Construction?

At the mixed-reality demo area of the biennial Trimble Dimensions conference, held in Las Vegas on Nov. 7-10, attendees lined up to try out SketchUp Viewer for HoloLens. Putting on the headset, Nick Burkel caught on quickly and soon was zooming through the model in a series of quick gestures. Burkel, a preconstruction manager with Sarasota, Fla.-based Willis Smith Construction, said it was worth the wait. “I think we’re all going to be using it five years from now, probably,” he told ENR. “I would build a project in that. Imagine having somebody on site walking around, seeing that everything is where it’s supposed to go.”

So if the hardware improves, what’s next for MR? “One of the lowest-hanging fruit is bridging the gap from the office to the construction site, seeing that BIM information out there in the field and having that overlay in the right location,” says Martin Bros.’ Nowak. “It could mitigate a lot of arguments we’re having in the field right now between contractors and subcontractors.”

The MR demo Nowak ran internally was beset by its share of  technical glitches, and getting mixed reality into construction will require clearing some hurdles, he told ENR. The headset often lost tracking, and the model would jump inches or feet around the room; the level of detail was limited, and they elected to go with simple wire frames that wouldn’t confuse the operator or strain the hardware to the breaking point.

 Despite these problems, Nowak remains optimistic about mixed reality’s potential to change the average jobsite. “I’m not saying it’s ready for prime time in construction next week, next month, next year. But this is a paradigm shift. We won’t have to look down at screens all the time. All the project’s information can be right in front of our eyes when we need it.”