Statistics That Mislead
Is the Obsession With Recordable Injury Rates a Deadly Safety Distraction?
AFTERMATH Three days after the New Year's Day 2021 accident that killed laborer Mason Mack Harris, 25, in Lincoln, Neb., an investigator photographed the slab that crushed him and the bent guardrail that had been a tie-off anchor.
Photo: Labor Dept. investigation file
On the first morning of 2021, laborer Mason Mack Harris, 25, reported for work that would have qualified for extra holiday pay. On that New Year’s Day, the onsite manager for his employer, Midwest Demolition Co., assigned Harris and a workmate to complete demolition of a 9-ft-high concrete balcony slab at a children’s home renovation project in Lincoln, Neb. According to U.S. Labor Dept. records, they used a concrete saw since neighbors had complained about jackhammer noise from earlier work.
Both men were tied off and both, as a precaution against airborne particles while cutting, wore typical disposable surgical-style face masks. To keep as much concrete dust as possible out of the air, Harris’ task was to pump water from a small tank onto the Makita saw his workmate was using to cut. A third employee in a lift basket was simultaneously using a saw and hammer to demolish columns and a beam supporting the slab.
The bent guardrail that had been a tie-off anchor for Harris and a workmate.
Photo: U.S. Dept. of Labor Accident Investigation
Harris’ workmate had cut a 7-ft x 6-ft slab section on three sides, leaving it supported only at the edge of its fourth side. Harris may have taken a step forward, but the cut section suddenly opened like a trap door beneath him. The guardrail to which the harnesses were anchored bent over and part of the railing fell on top of Harris’ workmate, breaking his leg.
Harris fell through to the ground a fraction of a second ahead of the cut slab section, whose edge cut Harris’ safety line. A ton and a half of concrete—the slab section—landed on top of him and stayed there for several minutes more until the crew supervisor used a Bobcat to lift the slab so that they could drag Harris out from beneath it. He struggled to breathe for some time before he died.
In all U.S. construction, fatal on-the-job accidents have remained stubbornly consistent at just above 1,000 per year since 2017, with 1,034 in 2020, the last full year for which there is US Labor Dept. data. One of four brothers from a Texas family, Harris entered the record book as the first U.S. construction-related fatality of 2021.
Quantifying Past Safety Performance
Midwest Demolition’s total recordable injury rate for 2020 is shown to be 4, according to OSHA’s version of the document.
Image Courtesy Labor Dept. Investigation Records
Was there any way to know before taking a job how safe it was to work for Midwest Demolition?
The company’s Total Recordable Incident Rate (TRIR)—a workhorse snapshot of a contractor’s past safety performance (see story below)—was 4 in 2020, according to the US Occupational Safety and Health Administration (OSHA) accident investigation documents reviewed by ENR.
Looking for quick answers on construction and engineering topics?
Try Ask ENR, our new smart AI search tool.
Ask ENR →
While that is above the industrywide average for all trades—about 3.0, according to Labor Dept. statistics—other comparatively dangerous specialty trades in 2020 had worse rates. Midsize structural steel and precast concrete contractors averaged 4.2 and framing contractors of all sizes and types averaged 6.8.
Midwest Demolition’s safety performance in the last year may have been, if not stellar, good enough, judged against other higher-peril classes of construction specialties.
An outsider also could have been impressed by Midwest Demolition’s website claims about the company’s safety practices. It states that “a culture of safety is our absolute priority” and “over the years, we’ve developed a sophisticated system that goes beyond simple injury prevention.”
But on New Year’s Day 2021, according to OSHA records, Midwest Demolition was violating three important safety regulations. One was the federal respiratory safety rule during the concrete cutting. The next was failure to choose adequate anchorages for the tie-offs. And the third was a failure to perform an engineering survey by a competent person to determine the possibility of unplanned collapse and train the crew in the hazards of premature collapses—a key threat to demolition workers—so as not to expect a concrete slab cut on three sides into a cantilever to support its own weight.
Relied on for decades as trailing indicators of an employer's safety success or failure, injury and fatality rates—along with government enforcement—have since the 1970s helped improve construction’s safety practices, saving many lives and preventing injuries.
Of these gauges, the TRIR—which measures a work-related injury or illness requiring medical treatment beyond first aid, or that results in loss of consciousness, days away from work, restricted work or transfer to another job—stands out. It is deeply enshrined in federal labor statistic-gathering. For companies that practice and ponder safety and its effects on business, it looms large. An owner or prime contractor might believe that a company that stays at or below the national average was doing okay when it comes to protecting jobsite craft labor.
How a Formula Falls Short
Matthew R. Hallowell, professor of civil engineering at University of Colorado, Boulder.
As common as hardhats and safety harnesses, the total recordable safety incident rate has been called many things: The gold standard. Ubiquitous. And for the statistically inclined, it has sometimes been conceived of as an “ideal response variable,” according to authors who studied the rate for the Construction Safety Research Alliance.
There’s also a beguiling simplicity to the formula: TRIR= Number of Recordable Incidents x 200,000 ÷ Number of Worker Hours. The 200,000 equates to 100 workers toiling 40 hours a week for 50 weeks a year. For convenience, the U.S. Bureau of Labor Statistics posts a calculator tool on its website.
Click on the image for greater detail
The rate “is an institution in itself,” says CSRA founder and executive director, Matthew R. Hallowell, professor of civil engineering at the University of Colorado, Boulder, “and it’s not perfect.”
There are too few safety incidents in many cases at the company level to meet the standards of statistical validity, so using it to compare companies, years or managers is misguided. How to improve TRIR? Another problem Hallowell notes is the binary nature—currently an incident either is or isn’t recordable, so weighting for different severity levels may be better. For example, “How many points for first aid or a medical case?” Hallowell wonders, adding: “We could argue these all day long.”
By Richard Korman
However, there are reasons to reconsider the role of TRIR in company-level safety measurement. A research project funded by the Construction Safety Research Alliance (CSRA) has shown how serious the problem is.
When based on fewer than 300 million work hours recorded, TRIR is unreliable and misleading, the researchers argue, and recordable incident rates should not be used to distinguish the value or success of a safety program or to compare one company’s safety performance against another’s.
But that is exactly how the rate is often used.
Originally published as a study in the journal Professional Safety in 2020, the authors also delivered another blow to a long-believed safety theory that assumes that by controlling less severe injuries and small, non-injurious lapses, employers create a firm foundation that also will limit fatalities. That belief is represented, visually and psychologically, in two well-known safety pyramids illustrating the connection.
One was developed by occupational safety pioneer Herbert W. Heinrich in 1931, who produced the incident/accident pyramid diagram sometimes referred to as Heinrich’s Law. He wrote that in a group of 330 similar accidents, 300 will produce no injury, 29 will produce lesser injuries and one will produce serious injury. Prevent the accidents and the injuries will take care of themselves, Heinrich wrote. A refinement of the pyramid using different ratios was published by insurance researcher Frank E. Bird in 1966.
The gathering of safety incident data rates by the Labor Dept.’s Bureau of Labor Statistics (BLS) began before World War II but took a major step forward with the passage of the Occupational Safety & Health Act of 1972, which created OSHA within the department. That year, BLS started its Survey of Occupational Injuries and Illnesses, which the agency states on its website “continues to be a great resource to the safety and health community” as it decides how to “allocate prevention resources.” After a 1987 National Academy of Sciences study showing the BLS survey was unable to produce reliable fatality counts, it added its Census of Fatal Occupational Injuries.
Doubts about the assumed link between lesser accidents and more severe ones are not new. Of special concern was that the relationship seemed to emphasize individual behavior as the cause rather than poor management or the existence of hazards.
Insurance researcher Fred Manuele, writing in a 2008 article in the Journal of Safety Research, noted that there had been a remarkable decline in workers’ compensation claim frequency, which had fallen off 30% in the prior decade, but there had been a larger decline in the frequency of smaller lost-time claims than in the frequency of larger lost-time claims.
In 2016, Bob Fitzgerald, at that time a safety and health principal at Birmingham, Ala.-based electric utility Southern Co., wrote that a narrow focus on achieving safety numbers alone “does not meet the true goal of a safety” culture. He cited the problematic definition of a recordable incident, where a dose of prescription pain medication qualifies, as does a fractured femur.
“Ideally, injury severity should be considered because incident rates alone may not paint the clearest picture” of holistic safety performance, he wrote.
Other safety experts were already discounting TRIR in favor of leading indicators such as safety programs, safety culture and monitoring and root-cause analysis of incidents. They were seen as better predictors of safety performance.
The CSRA study, titled the Statistical Invalidity of TRIR as a Measure of Safety Performance, involved 10 firms that provided access to more than 3 trillion work-hours of data. The result “debunks the notion that reducing TRIR is a surrogate for mitigating the risk of high-impact events,” the authors wrote.
In other words, a good TRIR gives no clue of the likelihood of someone being severely injured or killed. And that, in turn, raises the question about whether the nearly obsessive quest to lower and limit TRIR is a hindrance or distraction from preventing severe injury and death.
(Continued below)
Despite Emphasis on Rate, Chronic Underreporting
Unlike with this worker injured in an accident in New York City in 2020, many injuries are believed to go unrecorded in required government logs.
Photo Credit: John Nacion / SOPA Images/Sipa USA)(Sipa via AP Images
Submitting OSHA 300 logs to the U.S. Labor Dept. with information about serious injuries is a March ritual for many employers. The logs contain data about serious injuries that need more than first aid, require days away from work or meet other criteria. Ambiguity in the definitions adds to the temptation to underreport or manipulate the data. Companies recently have lagged in complying with other reporting rules, too.
On April 5, the US Occupational Safety and Health Administration announced a crackdown related to electronic submission, part of a decades-long effort to improve overall compliance. Specifically, the enforcement program will identify employers who fail to submit Form 300A data through the agency’s Injury Tracking Application. Employers with staffs of 250 of more must make annual electronic submissions as do establishments with 20-249 employees in industries with high injury and illness rates, including construction.
Attorney Travis Livermore, a division environmental health and safety manager for Irving, Texas-based Shemco Industries, sees underreporting of injuries as a chronic, widespread problem. In 2020, his article in the American Bar Association’s Journal of Labor & Employment Law contained an exhaustive critique of OSHA’s “exploitable record-keeping standard.” In construction, he wrote, a “good” safety manager must “possess this skill, along with the ability to find doctors who understand what makes a case recordable and are willing to provide treatment in a way that avoids such an outcome.”
The issue is not new. A 2008 report by the U.S. House of Representatives’ education and labor committee found that the vast majority of workplace injuries and illnesses—69%—are never reported. It faulted OSHA’s reliance on self-reporting by companies, noting an inherent conflict, given that workers’ compensation rates, bonuses and lucrative government contracts may all hinge on the number. BLS has recognized the problem and regularly updates a webpage about its search for solutions to undercounting during its Survey of Occupational Injury and Illnesses. Exactly where most underreporting occurs is not yet clear, says BLS.
The AFL-CIO’s annual “Death on the Job” report for 2021 also found that underreporting of serious injuries and illnesses in the workplace remains as problematic as ever.
The quality of the report information, which includes the injury type, location, trade and name of the injured worker, varies. While elite companies have good-quality data that can be used for comparison with national averages, the variation in OSHA logs prepared by other, smaller companies is broad, says Tomasz Dering, corporate safety director for construction manager Plaza Construction. He says he reads hundreds of OSHA 300 logs a year and one-third are very good, another one-third need help, “and the last 33% are useless.”
Employers walk a fine line as they apply a case-management approach to each injury. For example, one labor supply company shares on its website “legal and ethical” recordable injury tips that include: “Don’t over treat [injuries] when less treatment will provide proper care” and “consider use of over-the-counter medication instead of prescriptions, butterfly bandages instead of dermal adhesive and suture support devices instead of immobilization.”
Another example: one of OSHA’s criteria for reporting an illness or injury is that it needs to be tied to a specific “exposure” or “event.” Those terms offer leeway for a company to not report the injury altogether “if an employee complains of pain in his arm, but cannot recall when the pain began, and cannot identify an event leading to the pain, other than by speculation,” Livermore wrote in his journal article. Other OSHA regulatory language opens the door to keeping injuries off the books if it involves “minor musculoskeletal discomfort,” with the word “minor.”
In addition, an employer would also have to get a medical recommendation that the employee can perform “routine job functions,” but by forming relationships with various health care professionals, that potential barrier can also be cleared relatively easily, Livermore wrote.
The safety manager, for example, can offer to put the employee in a recovery program to limit his or her work activities, thereby paving the way for the doctor to recommend the worker perform routine activities. However, well before that determination is made, the safety manager is likely to have discussed the matter with the doctor in a way designed to prevent it becoming a recordable incident.
By Scott Van Voorhis
(Continued from above)
Matthew R. Hallowell, professor of civil engineering at the University of Colorado, Boulder, the study’s lead author and executive director of CSRA, says that “although we have made great strides in TRIR as an industry [where it is statistically meaningful], we have not made similar progress in fatality rates,” which are flat.
“Safety nerds,” as many dedicated safety pros see themselves, greeted the study with appreciative hosannas. They had sensed that recordable injury rates, as used, often created an illusion of validity and safety. To some, the study signaled that more focus should be directed toward fatalities and severe injuries and somewhat less on broken arms and turned ankles.
“Using data to question tradition, love that,” wrote T.J. Lyons, a safety specialist for DPR Construction, in an article on the website of the International Risk Management Institute. “Lost time is a good indicator, but I suspect it’s a mix of inspections, number of lost work days and incident reviews that include the energy involved” that are better predictors of trouble. Inspections, he wrote, are more significant if they catch missing railings, not missing safety glasses or gloves.
Others agree. The study “was tremendously validating,” says Jeremy Hakes, a safety manager at Adolfson and Peterson Construction. He sees it as an opportunity to refocus construction safety on high-impact events. “Should we obsess on PPE? Yes, it’s important to use it or people will lose eyes and fingers,” he says. But “PPE doesn’t prevent fatalities; yet that’s often how we judge a jobsite.”
“The reassessment of the importance of TRIR,” adds Hakes, “has the ability to change how we protect our people from serious injury and fatality events instead of worrying about sprains and strains.”
Kevin Hildebrandt, safety director for Miron Construction, says “many people have understood this problem for a long time, but there has not been scientific data on it.”
Mountains of Safety Data
In dissecting TRIR’s statistical validity, the study authors looked at their mountains of data in different ways. In one theoretical test case, the researchers looked at a company that had one recordable injury in 200,000 hours of work. Rather than expressing the rate as a single number, the most accurate way to depict the result with 95% certainty was to state it as a range of .18 to 5.66 with a most likely true value of 1.
If the same company experienced two recordable injuries in 200,000 hours of work, the range should be expressed as .55 to 7.29 per 200,000 hours. The injury count may be double but it “does not reveal anything significant about the difference in the safety system between the two periods,” wrote the study authors.
“Safety data doesn’t work according to the normal bell curve distributions that we’re used to seeing in other types of statistics,” said Drew Rae, a senior lecturer in safety science at Griffith University, during a podcast discussing the CSRA study.
New safety programs, the study suggests, which seem like a success or failure based on a year’s work by a few dozen employees, may really be only the result of random events. To calculate TRIR to a single decimal point, .1, requires at least 300 million hours of worker exposure time. The study authors indicate that unless vast numbers of work hours are involved, in no way can a traditionally calculated TRIR ever be pinpointed, much less take it to mean that a company has its arms around potentially fatal accidents. “Since TRIR has remained the most pervasive measure of safety for nearly fifty years, this study underscores the need to scientifically test even the most basic assumptions of the safety profession,” the authors wrote.
The CSRA study, of course, doesn’t have all the answers about how safety should be measured or practiced. Since most, but not all, deadly construction accidents occur at smaller employers, how should larger contractors influence them? Also, what’s a better way to define what “safe” means? Emphasize high-risk tasks and work? And doesn’t elimination and analysis of small injuries and incidents help produce and sustain a safety culture?
That’s what some safety experts argue: near-misses and the small injuries still matter. In a blog post in 2019 titled, Heinrich’s Pyramid-Does It Hold Up 90 Year’s Later?, loss control and safety expert Randy Klatt said yes, in the sense that injuries matter because “stopping ‘minor’ issues will help promote a culture of injury prevention and improved productivity.”
An approach offered by Mike Morris, health, safety and environment vice president of Austin Industries, is based on evaluating multiple data points. He says he has never counted on data about injuries in the hope that it would help prevent fatalities. High-risk potential incidents need to be “extracted” from the theoretical safety pyramid and focused on so that information about them can be combined with leading initiatives and goal-setting.
And when in the past there was an uptick in ladder incidents on Austin projects, for example, the company decided to use safer platform ladder designs.
As for total recordable incident rates, Austin folds all its subs’ rates into its own and has set .43 as a goal, says Morris.
When he evaluates subs, “I would never use one year. I would use five years of OSHA logs and five years of EMRs [workers compensation insurance experience modification ratings], and if EMRs are near or above 1, I would ask for more information, and not just their rate. It would be illogical to evaluate on one year of TRIR.” When Austin reaches a goal, “we evaluate whether our upstream processes are truly driving the result.”
This partial screenshot is from a dignitymemorial.com page with dozens of photos and messages of love for Harris and his family.
Image: A Partial Screenshot of dignity Memorial.com page for Mason Harris
*Click on the image for greater detail
OSHA Investigation in Lincoln
In the days after the death of Mason Mack Harris, an OSHA investigator took notes during his meetings with Midwest Demolition’s managers, the site supervisor on the day of the accident and Harris’ workmate. According to those notes, the project site supervisor had some knowledge of safety and training and had seen to it that Harris and his workmate on the balcony had fall protection tie-offs and various protective equipment. Midwest Demolition did not respond to phone or email requests for comment.
But the OSHA investigation claimed the site supervisor didn’t formally test the adequacy of the tie-off and the investigation summary said Midwest Demolition didn’t think to support the slab, as it was being cut, with bracing or blocking of any kind. A company manager said in essence, according to OSHA, that the firm relied on its site supervisor to carry out safety in the field. The project’s general contractor was not on site and was not cited for any safety violations.
Eventually, records show, OSHA and the company settled on a fine of $21,000 to be paid in installments.
The pain of Harris’ death last year generated numerous social media postings (his parents declined to comment for this article) paying tribute to the 25-year-old’s work ethic, warm smile and loving heart.
A former high school football player in Iowa Park, Texas, Harris had spent much of 2020 employed by a telecommunications services company in the Lincoln area.
Along with the posted photos of Harris with his fiancée, parents, brothers and friends, he is also shown in a slideshow on dignitymemorial.com in the midst of a work day, out in the field, prepared for fall protection with a safety harness strapped on. Tyler Mull, another employee of the telecommunications services company, wrote that Harris “was one of the hardest working men on our crew” giving “110% every day” during the nine months he worked for the company.
“He overcame fears of the tower instantly,” wrote Mull, “and kept everybody in high spirits no matter the task.”
