With an increasing need to be proactive in risk management, it might be time to look at your road testing practices. The transportation industry has adopted modern technology to manage its fleet, so why are companies still doing the same old road test?
What is the same old road test?
With a lack of direct guidance on this topic, companies are left to create their own road-testing requirements. These requirements vary by location and driver evaluator and, in many cases, follow the same practices used over 20 years ago. They start with a pre-trip inspection to gauge the driver’s understanding of a vehicle inspection and then move on to the road test. Driver evaluators typically use pen and paper to document observations, provide directions during the road test, and finish by delivering their subjective opinion of the driver’s ability to the company decision-makers.
So, what’s wrong with the same old road test?
Let’s highlight the gaps identified when performing the same old road test to answer this question.
- No two road courses are created equal.
- No two driver evaluators are equal.
- What is your road test not telling you?
No two road courses are created equal.
Road course design can vary by the terminal and by driver evaluator. Companies with multiple terminals may not realize how drastic the differences are from one road test to another. Even driver evaluators might have their preferred route compared to a company-designed course.
Driver evaluators are a significant factor in this inconsistency. Even if you have one terminal and only one driver evaluator, each road test may differ from the last. Driver evaluators might decide they know a better route or can tell if a driver is ‘good’ in 10 minutes.
Based on the Canadian Council of Motor Transportation Administration (CCMTA), National Safety Code (NSC) Standard 2, you should consider several pieces when developing road courses. Below are just a few of the items for consideration.
- Maximizing observations
- Uniformity of maneuvers
- Opportunity
- Objectivity
- Skill requirements
Maximizing observations: The various maneuvers differ greatly in the opportunities they provide to observe the performances that involve skill. For example, turning, particularly at uncontrolled intersections, permits much more opportunity to assess driver skill than straight driving. Routes need to be chosen in a way that will result in maneuvers that maximize the opportunities to observe scored performance.
Uniformity of maneuvers: All applicants must receive the same test, no matter when or where they take it. Applicants with the same ability should have the same probability of passing the test.
Opportunity: Situations requiring each performance must occur with sufficient regularity to assure that all applicants are scored on the same set of performances.
Objectivity: Performances that can be assessed objectively are preferred over those that require subjective judgment on the examiner’s part.
Skill requirements: Attentional, perceptual, and motor.
No two driver evaluators are equal.
Driver evaluators use subjectivity to determine risk. Every driver evaluator has a different interpretation of a good driver, the required skill level, and the perceived high-risk driver level. This interpretation falls under one of the 19 Unconscious Biases called Idiosyncratic rater bias.
Idiosyncratic rater bias affects the way we evaluate the performance of others. We often rate others based on our subjective interpretations of the assessment criteria and our definition of what “success” looks like.
Unconscious biases have also been linked to unintentional discrimination in case law. In these cases, it has been determined that there is no need to establish intention or motivation, no need for direct evidence, circumstantial evidence and inference are enough, and that stereotyping will usually be the result of subtle unconscious beliefs, biases, and prejudices.
So, when you look at your organization, what controls do you currently have to manage and mitigate these risks? Below are a few examples that carriers are using today.
- Driver evaluator qualification requirements
- Onboarding and ongoing training
- Driver evaluator performance tracking
- Analytical data
- Objective versus subjective scoring algorithms
What is your road test not telling you?
When evaluating a driver’s performance and abilities, ask yourself, what is my road test not telling me?
- Was it just a good day?
- Are we marking performance based on operational needs?
- Were the errors based on competency, or were they trainable?
- Is there any underlying impairment?
Drivers can be on their best behaviour during a road test. Is there a way to challenge the driver enough to ensure you get the most out of the road test?
Companies are seeing an increase in pressure from operations to hiring. Are you making decisions based on risk or filling a seat?
What system do you have to break down the type of errors? A driver evaluator’s interpretation of competency versus trainable may differ from others. What if the driver evaluator scores an error harshly that the company believes trainable? Did you just let a good driver walk out the door?
Are you measuring or screening for any underlying impairment? The CCMTA Standard 6 – Determining Fitness to Drive, identifies 12 cognitive functions that are needed for driving.
- Divided attention
- Selective attention
- Sustained attention (vigilance)
- Short-term or passive memory
- Working memory (the active component of short-term memory)
- Long-term memory
- Choice/complex reaction time
- Tracking
- Visuospatial abilities
- Executive functioning
- Central executive functioning
- Visual information processing
Cognitive impairment ranges from mild short-term to severe long-term. Below is a list, not all-inclusive, of medical conditions that can cause cognitive impairment.
- Untreated sleep apnea
- Untreated diabetes
- Hypertension
- Psychiatric illness
- Respiratory disease
- Dementia
- Developmental disorder
- Amnesia
- Substance-induced cognitive impairment
- Alzheimer’s disease
- Parkinson disease
- Brain trauma
- Frontotemporal degeneration
If someone is suffering from cognitive impairment, they may not even realize it, and a road test may not detect there is a concern present. A driver’s provincial medical assessment might also miss identifying mild and severe cognitive impairment.
Road test best practices.
Even though there’s a lack of direct guidance on how to develop a road test and conduct performance testing, there are two industry resources that do provide information and general guidance.
- Canadian Council of Motor Transportation Administrators (CCMTA), National Safety Code – Standard 2. Chapter 8: High-Class Vehicles (Class 1-4)
- United States Department of Transportation (US DOT), Federal Motor Carrier Safety Administration (FMCSA) – 49 CFR Part 391.31 – Road Test
The CCMTA NSA Standard 6, Determining Fitness to drive, also provides great insight into determining driver fitness, which includes medical assessments as well as functional assessments such as a cognitive screen, cognitive road test, eye tests and hearing tests.
Conclusion
With raising insurance costs, nuclear verdicts in the US, and a general responsibility to operate safely, why have we left risk tolerance to subjectivity? And in a world where drivers are in high demand, why risk losing someone due to bias when training and retention might be the best action plan currently available to combat the driver shortage crisis. It’s time to ask yourself, are you doing the same old road test?
Learn how we can support you in creating a standardized on-road evaluation: Commercial On-Road Evaluation
Chris Wilkinson
CHRP, Sales Manager, Impirica Inc.
Chris has established himself as a leader and innovator in the fitness for duty industry supporting companies in Canada and the USA. From program development and implementation, to training, program review, and program management, Chris has worked with companies in a variety of safety sensitive industries. Additional accolades include, training over 5,000 people since 2013, published online educator with 4.4/5 rating after 200 reviews, and published a white paper on opiate and opioid use in the trucking industry.