SQL, or Structured Query Language, is a programming language for organizing data in an indexed way that is more searchable. DiPalma and PJ Dick got their 20 years of project data as they wanted it, but moving it to another platform as planned was impractical. They ended up putting the SQL database in Amazon AWS cloud storage.

“When we need it, we basically have to pull it down from Amazon, give it to our project team on a shared network drive, and they have to figure out where what they need is,” DiPalma says.

This is just one of many scenarios where construction technologists are being asked to break with the old ways of saving project data—in filing cabinets and bankers’ boxes—and find new ways that incorporate project data stored in cloud-based, construction project management platforms from vendors such as Autodesk, Bentley, Oracle or Procore. 

Contractors must not only consider how they are going to optimize data and make it searchable for future project teams, but they also figure out how to gain business intelligence insights from all of the raw project data stored either on a technology provider’s cloud, their own rented cloud space or their local servers. All the while, they must still consider what to keep and how to limit risk while staying ahead of rapid changes in data storage and analytics technologies.

“The licensing language has two elements: access to the data and access for integration,” says Christian Burger, president of Burger Consulting Group in Chicago, which advises construction clients on how to negotiate better agreements with technology vendors. “Firms are trying to put in place a data warehouse strategy now. The question becomes, how does the data from this cloud solution get to your data warehouse?”

Both Autodesk and Procore executives say they want to help their customers achieve their project goals, but they also have priorities of their own. The two CPM cloud providers are keen to use aggregated, anonymized customer data to provide industrywide insights, such as regional employment data based on hours worked as recorded in their platforms. By offering such views into the economic health of the entire construction sector, both say they can assist industry groups and modernize construction’s business practices.

“It’s one thing to digitize project data,” says Kris Lengieza, vice president of global partnerships and alliances at Procore, who had been a virtual design and construction executive for Stiles and the Weitz Co. before joining the technology company. “We’ve all made a big step and a big stride there from pen and paper and Excel. Now, we’ve got to talk about normalizing data. Getting it arranged, getting it organized, and then we have to figure out how to analyze it and then visualize it.”

Lengieza says the contractors he worked for in the early years of this millennium were still in the process of understanding and utilizing project data to inform future decisions. “It’s not necessarily related to size. It’s related to how invested contractors are and how important their data maturity is,” he explains.

Refining construction processes, offering visibility into project performance during construction and helping deliver high-quality structures to owners are key goals for many technology providers, since contractors are not their only customers. Many serve designers, owners and contractors across multiple industries. They look at the construction process as one that can and should be refined and project data as something that can inform those refinements.

“If you are an owner, the holy grail is your ability to ‘round-trip’ a lot of the relevant operational data of the building back into the design process,” says Sidharth Haksar, head of construction strategy and industry partnerships at Autodesk. He says in manufacturing, a customer creates machine parts, then puts them to work out in the field with sensors transmitting information back. Designers look at that information to gain insight about the next revision of their design. “Because we start with the design authoring tools all the way through construction, we believe we are best positioned” to bring that operational data back to the design stage for the next iteration or to inform the next design, improving on costs and scheduling, he says.

A persistent concern for contractors is limiting risk. While having project data accessible via a vendor’s cloud is convenient, it still represents an ongoing liability, no matter how secure it may be. One major contractor ENR spoke to for this article said its mandate was to decide what information it wanted to keep and what could be discarded after project handover.

Mortenson sought to standardize how it keeps the information from the various cloud products it uses on its many projects in Sharepoint and Office 365
Graphic courtesy of Mortenson
*Click the image for greater detail

The Cloud-Connected Filing Cabinet

Minneapolis-based Mortenson has 13 offices and ranks at No. 19 on the 2022 ENR Top 400 Contractors list. Its projects run the gamut from local bank branches to the $2-billion Allegiant Stadium, home of the Las Vegas Raiders.

The company’s move to standardize project documents, file sharing, and collaboration best practices is a far-reaching effort being co-led by Alison Hart, Mortenson senior manager of project solutions. Since early 2016, the project solutions team there has been pushing to modernize the firm’s project management technology, focusing not just on CPM platforms, but on everything that touches a project site from CAD or BIM files to measurements taken by pencil in the field.

In late 2019, Hart’s team, along with other Mortenson information technology leaders, started an effort to migrate project documents and file collaboration during project delivery from in-house network file servers to the cloud, using the Office 365 platform with Microsoft Teams and SharePoint as the new standard interface. This included simplifying folder structures across Mortenson’s many operating groups, as well as incorporating the access and permissions needed for various internal and external project stakeholders. The team had to do this while ensuring there was enough flexibility in the system for project teams to manage and organize documents based on the unique needs and challenges of the project.

Microsoft Teams became the visual interface of a project and Mortenson used channels hosted on a separate SharePoint site to be the external facing platform for project partners.

Moving to Teams and SharePoint offered Mortenson’s project solutions group and IT team the ability to standardize and separate project data for internal stakeholders in a private channel, and external collaboration through the general default channel. External channels were open to anyone on the project.

“There’s no sensitive documentation—this is for you, for your trade partners, your owners, and your designers.” Hart says.

Internally, the team went to great lengths to keep confidential information private, putting restrictions and security controls in place to safeguard sensitive project details. Project teams would still use a variety of other tools to manage the project, but the collaboration and final deliverables were all collected through Microsoft Teams and SharePoint. This approach allows Mortenson’s 13 groups and all of its project designers, owners and trade partners to create their own experience in the cloud and not have to rely on the contractor's servers as the main point of access to data.

“What is more intuitive for you?” Hart asks. “Is it accessing it via SharePoint? Is it looking at it from Teams? Is it syncing it down to your OneDrive, which is a similar experience to [Windows] File Explorer?”

Hart says now that Mortenson has used Microsoft Teams and SharePoint for a few years, its project document management team, based at company’s headquarters, was tasked in 2020 with further improving on the original design and concept of the system. Construction file types such as BIM, CAD and other rich data files continue to change, and user feedback has prompted Mortenson to revisit how it uses the Microsoft platform.

“We took what I would call the ‘file cabinet’ way of document management and we’ve modernized it to take into consideration all of the collaboration applications used in the field office today,” Hart says. “There’s still a lot of redundancy and we’re starting to put some definition around it.”

Prescriptive guidelines for what needs to be saved and where, along with clarifying the close out processes for both project files and the various applications used during the course of construction are still being established as new standard procedures.

Construction project management platforms such as Autodesk Build are increasingly offering analysis of the construction process itself.
Dashboard screenshot courtesy of Autodesk
*Click the image for greater detail

From Cloud to Data Warehouse

Southfield, Mich.-based Barton Malow, which ranks at No. 30 on ENR’s Top 400 Contractors list, is taking a different approach toward its data, not simply keeping it on a vendor cloud nor stashing it on a private, internal cloud. As longtime users of Autodesk Construction Cloud, the firm uses Autodesk Build to capture data across project teams. Director of Business Transformation Ted Jennings began working with the vendor in 2019 to connect its project data to analytics tools like Power BI to glean project insights.

“We use Autodesk Build, which is where project management data is created. We have set up a systematic way to flow the data into our environment to be used in business intelligence reports,” he says. Partnering with Autodesk to set up a data connector, Jennings says “this connector has given us the ability to extract our data. We leverage Autodesk Platform Services to programmatically schedule jobs to bring the data directly into our data environment.”

Barton Malow also uses tools from Microsoft Azure and SAP to manage its data warehouse, Jennings says. “We want to be better stewards of our data so that the data that we’re generating on projects today is not only useful today, but there are subsets of that data that we want to leverage in two years, five years, ten years down the road.," he says. "That is currently the journey we are on.” 

Construction schedules is one area of its business in which Barton Malow has learned how to better leverage data. “There’s a lot of data and information that goes into a construction schedule. Learning to capture and use all information from a data analytics perspective can be challenging,” Jennings says.

The manager and Barton Malow’s application development team worked with Autodesk’s Construction IQ team to take advantage of its business intelligence capabilities for the entire construction lifecycle. “One way Autodesk is helping us is that they’ve built machine-learning classification tools to look at issues,” Jennings says. Autodesk’s Construction IQ can analyze the text in an issue and then classify it and append useful contextual tags.

“The text that a team member enters into the description field of that design issue is analyzed by a set of machine-learning algorithms, then Construction IQ applies tagging based on the machine-learning model," he says. "The benefit to us is that this additional information helps improve reporting and downstream analytics.”

Using Construction IQ inside Autodesk Build is how Barton Malow harnesses machine learning to generate insights from its data. Previously, categorizing issues was a manual effort, requiring subjective judgements that can be hard to standardize across an organization. With Construction IQ, that tagging is automated, affording a degree of standardization and insight for data analytics.

Pre-pandemic, contractors’ methods of creating project data were siloed and manual. Remote office work and communication to sites from home accelerated the transition to cloud storage.
Graph courtesy of dodge data and analytics
*Click the image for greater detail

Remaining in the Cloud

From a legal perspective, contractors generally have two ways to approach project data retention even if there’s potential insights to be had after the project is completed.

Contractors have to make a choice—do they want to keep their data for a long period of time to protect themselves in potential disputes and lawsuits, or do they feel that keeping that data presents more risk than opportunity?

Aaron Geiger, chief technology officer of contractor Alberici in St. Louis that ranks at No. 35 on ENR’s Top 400 list, is firmly in the opportunity camp. “If a company feels the data presents a risk, they dispose of it," he says. "At Alberici, we believe that data supports evidence that we did the right thing, so we have a longer than average retention policy on our data.”

The firm uses Procore’s CPM platform and finds the ability to revisit closed projects in the cloud to be valuable. Because the vendor charges customers based on construction volume, once a project is closed out that data is retained in its platform in perpetuity, at no additional cost.

“All of our projects are stored in secure cloud solutions, such as Egnyte and Procore,” Geiger says.

For Alberici’s work with government clients, it uses systems tuned to comply with NIST/CMMC security standards. Storing data in a platform like Procore and other construction technology tools, such as Oracle Primavera P6 for scheduling or StructionSite for jobsite data capture, is valuable, according to Geiger.

“If you’re looking for data from three years ago, these platforms let you see all the projects that were worked on three years ago, regardless of whether they’re active or closed,” he says.

Alberici has a small amount of legacy systems still hosted in its own data center, but Geiger says cloud data storage is so inexpensive today that the cost to retain actual project information from a technology perspective is very low. Alberici’s core cloud storage provider across the company is Egnyte. Both operations and support groups store files in Egnyte, which, once inactive, are moved into a lower-cost archive domain based on company rules. Once the data is older than the retention period, it is automatically deleted.

When Alberici performs the handover of a project to a client owner, it uses the data capabilities in Procore to keep everything from that project.

“We’ve found it’s not worth the time to cull through project files and data to delete things,” Geiger says. “[We focus on] retaining and archiving data based on business value. Our clients have been well served with this strategy, as we’re able to use past successes and best practices on projects today.”

Alberici is between the crawl and walk stages in using business intelligence across its project data, he says, if only because most construction data is still not automatically generated and requires manual input by field personnel. Recent advancements in technology that tackle that limitation are a major area of investment for many contractors now, according to Geiger.

“Right now, we’re doing basic comparisons across different project types and different markets we work in,” he says, adding that the firm also is cognizant of not creating even more onerous data entry tasks for field personnel.

Contractors say Procore’s analytics dashboard is useful during the job and is also available in the cloud for years afterwards for project comparison purposes.
Screenshot courtesy of Procore
*Click the image for greater detail

Where Does Construction Data Go Next?

While project data is less siloed than it has ever been, contractors and construction technology providers are still grappling with how to get needed data from one party to another. Many vendors are charging for integrations that use their APIs, which can be a very expensive added cost for contractors, says Burger.

Construction technology providers can also limit a contractor’s bandwidth when accessing the cloud, a practice known as "throttling," which makes getting project data out of a platform time-consuming.

The strategies contractors are deploying to deal with project data after closeouts show an industry still struggling with how to store and discard data, even as the era of storing everything on paper in a box or filing cabinet is rapidly fading.

The various pricing models used for construction project management systems are changing as well. “If we’re going to peg it to your revenue, that’s, well, kind of on the honor system,” says PJ Dick’s DiPalma.

He notes that a given technology provider will not always see all of the contract volume that a contractor has going at any given time. With many costs and project elements not tracked in the construction management platform, determining the overall volume can be a bit of a shell game. But it all comes back to the negotiations between vendors and contractors.

Lengieza says Procore pricing based on volume put-in-place is ultimately better for contractors, since data storage after a project closes is then free to stay in the construction management platform indefinitely. “With our pricing model you can manage your projects during the preconstruction phase for free," he says. "Any projects that don’t go under construction won’t count towards your volume, so you don’t have to pay for them.”

Haksar says Autodesk’s pricing models have evolved over the last few years, providing increased flexibility and choice around how customers purchase the software-as-a-service. “We’ve been focused on giving customers flexibility in the manner in which they would prefer to buy and consume our solutions,” he says. “You want a model tied to construction volume with unlimited user access? Go for it. If a per-user pricing model makes more sense for the way you manage your construction business, go for that. If you want to take more of a consumption model, pay-for-what-you-use approach, we have that too.” 

Haksar adds that Autodesk is not auditing the construction volume during the term of a contract.

Despite the range of approaches to the data storage problem, all of the contractor technologists ENR spoke to for this article agree that available methods to store and better use project data are changing, and no one is sure what the perfect solution is today—or will be tomorrow.

“We are taking in what we’ve learned in the last couple years. We are still looking at the ways we consume data,” says Hart.