A widely anticipated report on the Compliance Safety and Accountability program (CSA) was released this summer by the National Academy of Sciences (NAS). This report was required by Congress as part of the FAST Act signed by President Obama in December 2015.

There had been growing evidence and criticism by carriers, shippers and brokers that CSA data, and specifically Safety Measurement System (SMS) BASIC percentiles, were misleading and inaccurate when applied to individual carriers.

As a result, Congress required the Federal Motor Carrier Safety Administration (FMCSA) to take down the BASIC percentiles from public view. They tasked the NAS with a full review of the SMS methodology. Here are some of the important conclusions in the report:

More Visibility Needed For Carriers and Drivers

• The report validates many carrier complaints of SMS, including: poor quality data, old data, and inconsistent data.
• Carriers have long complained about a lack of visibility to methodology. In other words, they may think corrective actions will improve scores, but when the new scores come out, carriers are surprised at the lack of impact. This reports recommends that FMCSA produce better predictive models for carriers so they know what actions will have direct statistical impacts. Carriers, prepare to add a data scientist to your team.
• The report highlights issues related to relative rankings versus absolute measures (should peer groups be used or not?) The report says both should be used.
• The panel recommends a meaningful change in methodology to an Item Response Theory (IRT) model. This would be tested over the next two years alongside current SMS methodology. In plain language, this means that CSA methodology will continue to evolve over the next few years. The measurement of carriers and drivers adherence to the rules will keep changing.

More Research Needed For Shippers and Brokers

• Congress asked the NAS panel to look at whether the BASIC percentiles should be publicly available or not. In the study, the panel recommends another study to look at this more in depth and acknowledges “the disadvantages of public reporting stem from the fact that SMS is not a perfect discriminator between the carriers that need to improve their safety performance and those that do not.’’
• The panel specifically highlights the difficulty of using small sample sizes to predict outcomes for small carriers. They state in the report “there is a particular concern about false negatives and false positives among smaller carriers, which results from not having much data with which to judge them.’’
• It appears that BASIC percentiles will remain hidden from public view for quite some time as yet another study tries to resolve whether BASIC percentiles are good indicators of an individual carrier’s crash risk.
• This report will likely lead to more calls from shippers and brokers to pass legislation to establish uniform federal standards when qualifying motor carriers. This legislation has been introduced in both the US House (HR 1568) and the Senate (S 1345)

You will likely see many more articles in the near future regarding the analysis of this report. We will provide updates as new details emerge.

Enjoying our insights?

Subscribe to our newsletter to keep up with the latest industry trends and developments.

Stay Informed

(Jason Craig is manager of government affairs for C.H. Robinson. He can be reached at 1-800-323-7598 or www.solutions@chrobinson.com. Based in Eden Prairie, Minn., C.H. Robinson is a leading third-party logistics provider offering freight transportation and logistics, outsource solutions, and information services.)