August 20, 2018

The Federal Motor Carrier Safety Administration (FMCSA) announced it has delivered a required ‘Correlation Study Corrective Action Plan’ for its compliance, safety and accountability program to Congress. ‘Correlation Study Corrective Action Plan’ FMCSA’s new corrective action plan outlines how the agency will address recommendations made in the National Academy of Sciences’ (NAS) “Improving Motor Carrier Safety Measurement’’ report, which examines the effectiveness of the use of the percentile ranks produced by the academy’s Safety Measurement System (SMS) for identifying high-risk carriers. And if not, what alternatives might be preferred. Accuracy and Sufficiency. In addition, that report evaluates the accuracy and sufficiency of the data used by SMS to assess whether other approaches to identifying unsafe carriers would identify high-risk carriers more effectively. The report also seeks to address how members of the public use the SMS, and what effect making the SMS information public has had on reducing crashes. FMCSA noted that it has pulled from public view a preview website that displayed changes that the agency had proposed to its SMS. Those changes had been “released prior to the NAS report,” so they are now outmoded. The NAS study and subsequent report was commissioned by FMCSA to comply with a provision of the FAST Act highway bill of 2015 that mandated a study be conducted of the agency’s compliance, safety and accountability program and its SMS component. According to the agency, the report to Congress presents FMCSA’s corrective action plan for addressing the six recommendations presented in the NAS study. Item Response Theory. A key element of the NAS report is the recommendation that FMSCA study a more scientific modeling approach known as “item response theory” (IRT) and then consider implementing it to make the SMS function more accurately. Joe DeLorenzo, FMCSA director of compliance and enforcement The hope is that the IRT method would help develop “an estimate of the measure of safety culture’’ for each carrier. That estimate could be used to monitor and identify carriers in need of intervention rather than just zero in on violations, said Joe DeLorenzo, FMCSA’s director of compliance and enforcement. DeLorenzo made his comments during an educational session at the American Trucking Associations (ATA) annual management conference and exhibition in Orlando, Fla. FMCSA Actions List. Actions that FMCSA stated it will take in its 10-page corrective action plan include: Moving forward to develop and test an IRT model. That model will be used to inform the agency’s work, with opportunities for public input. As recommended by NAS, if the new IRT model performs well, it will replace the existing SMS. FMCSA will not substantively modify SMS while the IRT model is being tested to ensure there is “a stable comparison to evaluate effectiveness.” Agreeing that more frequent and more detailed Vehicle Miles Traveled (VMT) data from motor carriers would reduce the need for FMCSA to use substitute values and would improve the quality of the data in SMS. FMCSA currently only collects carrier VMT data every two years. Access to this data by state on a monthly basis is not currently feasible, but the agency will continue to identify possible sources for this important data. Agreeing that additional information about carrier operations might improve the agency’s analysis and identification of non-compliant motor carriers. However, the collection of this data would come at a cost, and the benefits are unknown. Agreeing that there could be benefits from making motor carrier data available to researchers and carriers. As a result, the agency’s first effort will be to improve data availability. FMCSA will develop a web page where researchers, carriers, safety consultants, and the public can obtain simplified motor carrier data snapshots. Gathering public input from motor carriers, insurance companies and shippers regarding the ways in which the public uses SMS data. Using the data collected in these public listening sessions, FMCSA will scope and complete a study specific to the issue of percentile ranks and the usability of public scores. Once IRT modeling is complete, FMCSA will evaluate the use of absolute measures for set intervention thresholds. Absolute measures and percentiles are products of the SMS system. At this time, it is not known how these would be affected once the agency completes the modeling recommended by NAS. Therefore, FMCSA defers action on this recommendation until it is confirmed this would be relevant. FMCSA Next Steps. As for its next steps, FMCSA said it will work with the input of the NAS Standing Committee to develop and run a small scale IRT model by September. After evaluating the results, the agency will run a full-scale IRT model by April of 2019. Additionally, the agency said it will work with the NAS Standing Committee on planning and will also schedule public meetings throughout IRT development to further discuss the recommendations. Federal Motor Carrier Safety Administration (FMCSA)