Ensure that ITS field operations tests use technologies and applications that are proven to be deployment ready.
Experience with a Field Operational Test for the Intelligent Vehicle Initiative in Minnesota
26 September 2003
Hutchinson,Minnesota,United States; Minnetonka,Minnesota,United States
The IVSS research culminated in a Field Operational Test (FOT), conducted in Minnesota from December 2002 to March 2003, of the safety benefits and driver acceptance of the systems. The FOT deployed four snowplows, an ambulance and a State Patrol squad vehicle that were equipped with the IVSS. The FOT occurred in the winter months to take advantage of the typically heavy snowfall in Minnesota and test the systems in the condition for which they were designed, namely, very low visibility. However, two limitations to the FOT emerged that diminished the capability of determining the safety benefits of IVSS. First, the winter turned out to be one of the mildest on record for the state without producing the desired (for the FOT) low visibility. Second, it emerged during the FOT that not all of the on-board technologies were not deployment ready and did not perform reliably. The evaluation of the FOT revealed lessons that are relevant to the design of ITS FOTs and include the following recommendations.
- Ensure the ITS technologies to be evaluated are deployment-ready. The IVSS FOT produced inconclusive results because the test conditions did not materialize due to the mild winter but also because the technologies did not perform as designed or reliably. One lesson from this experience is that an FOT will be compromised when the technologies under assessment are not deployment-ready. Although this point seems obvious, its importance must be reinforced. The pressure to continue a project as planned may compel agencies or personnel to move forward with a schedule even if systems are not demonstrably ready to be tested in the field. From a project manager's point of view, delaying a field test can incur negative consequences. However, there are lost opportunity costs and perhaps safety costs associated with following through with scheduled tests when systems are not ready. FOT plans should include steps and requirements for confirming that technologies meet a minimum standard of reliability and performance before deployment. It is also worth noting that the problem of system unreliability appeared to have damaged operator acceptance of the system. Interviews and questionnaire data of the operators show that operators were initially optimistic about using the system, but they lost trust in the system due to technical problems such as the intermittent signal loss and unstable images on the head-up display, among other problems.
- Develop contingency plans for field tests requiring specific weather conditions. The premise of the IVSS FOT was that the winter months in Minnesota would generate significant amounts of snowfall and create the conditions necessary for the evaluation of IVSS. The winter, having been atypically mild, did not generate the conditions necessary for the evaluation. Thus, the results were of limited value. This outcome underscores the risks involved when research designs are dependent upon externally-controlled events or conditions (such as weather). Thus, to the extent possible, FOT designs must include alternative test scenarios and options. For example, although inconvenient, an FOT requiring particular weather conditions should have flexible start dates so that testing can occur under the desired weather conditions. Test scenarios could utilize simulation and modeling opportunities if possible until the desired environmental conditions emerge. Another consideration is to partner with an agency that is located in an area that has the test conditions. For example, several operators in the IVSS FOT expressed the idea or concern that the IVSS was better suited for snowplows in the rural areas of Minnesota that has more severe and frequent low visibility conditions, as opposed to the location designated for the FOT.
- Involve end users early in the design phase for the operator interface. The snow plow, ambulance and patrol car operators in the FOT indicated in interviews and questionnaires that they approved of the concept of the IVSS but that they experienced problems with the interface and lost trust in the system due to its unreliable performance. For example, operators noted that the IVSS equipment installed in the cab was bulky and awkward, with the projector located in a way that meant the operator's shoulders frequently hit it. The ambulance operators in particular viewed the equipment as inhibiting communication with other emergency personnel in the vehicle and elected not to use the technologies (although it is possible that they may have had there been low visibility conditions). Similarly, operators disliked the fact that the auditory warning shuts the truck radio off, and many operators tried to lower the volume of the auditory warning and said that they would have disabled it if they could have. Involving end users in the design phase in straight-forward usability sessions or focus groups can prevent or mitigate problems with the interface that will be too late to change after a system is built, installed and deployed.
On-board ITS applications have the potential to improve the safety of specialty vehicles by assisting operators in low visibility situations due to inclement weather. However, realizing the safety benefits of these technologies requires further research and development, and field evaluations. This lesson learned entry highlights the importance of conducting FOTs with systems that are proven to be ready for field deployment, using flexibility in schedules and/or locations to conduct tests in the weather conditions needed for testing, and involving end users early in the design process to produce better operator interfaces.
Published By: U.S. Department of Transportation, Federal Highway Administration
Source Date: 26 September 2003
EDL Number: 14106URL: http://ntl.bts.gov/lib/jpodocs/repts_te/14106.htm
Average User Rating
Lesson of the Month for November, 2010 !
Intelligent Transportation Systems > Crash Prevention & Safety > Collision Avoidance > Obstacle Detection
Intelligent Transportation Systems > Crash Prevention & Safety > Collision Avoidance > Lane Departure Warning
Intelligent Transportation Systems > Driver Assistance > Lane Keeping Assistance
Intelligent Transportation Systems > Driver Assistance > Object Detection
Intelligent Transportation Systems > Driver Assistance > Vision Enhancement