Lesson

Ensure that experienced staff oversee the development of a complex software system and thoroughly follow systems engineering process.

Experience from iFlorida Model Deployment


01/30/2009
Florida,United States


Background (Show)

Lesson Learned

As noted previously, FDOT experienced significant difficulties in reaching its objectives with the iFlorida Model Deployment. Early in the deployment, the problems were centered on the deployed field equipment, with a large number of the arterial toll tag readers failing by the time the software systems were in place to use that data. As FDOT brought the field equipment back online, problems with the CRS software became more apparent. From November 2005 through November 2007, FDOT's efforts focused on eliminating the problems with the CRS software. Through this process, FDOT identified a number of lessons learned that might benefit others attempting to deploy a new (or upgrade an existing) traffic management system.
  • Ensure that experienced staff oversees the development of a complex software system like the CRS. FDOT D5 had no ITS staff with that experience and declined FHWA's offer to provide a software training course for the FDOT iFlorida staff.
  • Beware that one of the biggest sources of problems in a complex system is the interfaces between subsystems. With the CRS, long-standing problems occurred with the interfaces between the CRS and the FHP CAD system, the weather provider, the travel time server, and the DMS signs. Approaches for reducing the risk associated with these interfaces include:
    • Adopt ITS standards that have been used effectively in other, similar applications.
    • Develop the interfaces early in the development process and test the interfaces independently of the other parts of the system.
    • Include interface diagnostic tools that could sample data passing through the interface and assess whether the interface was operating correctly.
  • Include tools for diagnosing problems that might occur in individual subsystems.
    • When errors occurred with the arterial travel time system, it was difficult to identify whether the error was caused by the readers, the travel time server that computed travel times from the reader data, the CRS that used the computed travel times, or the interfaces and network connections between these systems.
    • When errors occurred with updating DMS messages, it was difficult to identify whether the CRS was sending incorrect data to the system that interfaced with the signs or the sign interface was not updating the signs correctly.
  • Incorporate alternate methods for accessing data and updating signs and 511 messages in case the primary software tools for doing so are not functioning correctly.
    • The I-4 loop detector data was available through the Cameleon 360 software after the CRS failed, enabling RTMC operators to continue to estimate I-4 travel times.
    • The backup interface for updating DMS messages allowed FDOT to disable the signs in the CRS and manage the sign messages through the backup interface when the CRS interface for updating the sign messages proved unreliable.
    • The backup interface for updating 511 messages allowed FDOT to continue to provide 511 services when the CRS failed.
  • Consider testing as a critical part of a software development project. While testing is primarily the responsibility of the contractor, the lead agency may want to review the test plans and documentation of test results during development, including unit and integration testing. The testing should include both the software and configuration information used to initialize the system. To achieve this, the contractual process must include details related to the visibility of the testing process.
    • Errors in the CRS travel time calculations were not discovered until OOCEA reviewed the resulting travel times, and tools to test them (such as the static tests) were not available until more than 6 months after the problem was first discovered. Better testing for configuration data validity might have prevented errors from reaching the production software. For example, tools could have been developed to verify computed travel times independently. Tools that generated a map-based display of the configuration data would have provided an alternate means of testing that data.
    • More detailed tests of the component that related FHP CAD incidents to roadways should have identified the fact that the software sometimes miscalculated incident locations.
    • Tests of the CRS interface to the DMSs should have revealed some of the problems that FDOT experienced with this interface, such as the fact that the Cameleon 360 and CRS software had an opposite interpretation of the meaning of sign priorities.
  • Include software system requirements related to configuration and administration of the system.
    • With the CRS, the configuration was performed by the CRS contractor. When errors with the configuration were identified, FDOT discovered that the configuration was too complex to correct without assistance from the CRS contractor. Even the CRS contractor failed to eliminate all of the configuration errors in the CRS. Requirements that described the configuration process might have resulted in a simpler, less error-prone configuration process.
The deployment experience also highlighted the challenges of taking a "top-down" approach rather than a "bottom-up" approach to development. FDOT wavered in its leadership over the contractors after expressing only a vision for the system operation. Guidance provided to contractors by lower level FDOT staff was often over-ridden by upper management, and lower-level staff had little or no voice in expressing concerns. FDOT also did not provide a well-developed document to describe the traffic and travel management operations the iFlorida systems were expected to support. This meant that some iFlorida contractors were provided with limited documentation regarding the requirements of the systems they were developing and received contradictory feedback from different FDOT staff. Many critical client-contractor relations suffered as a result, and the continued miscommunication magnified the errors of each successive phase of the development until it became too difficult to manage.

Despite these failures in developing the CRS, the existence of other methods of performing key operations, such as updating 511 and DMS messages, meant that FDOT did continue to perform these traffic management operations in spite of the failings of the CRS.


Lesson Comments

No comments posted to date

Comment on this Lesson

To comment on this lesson, fill in the information below and click on submit. An asterisk (*) indicates a required field. Your name and email address, if provided, will not be posted, but are to contact you, if needed to clarify your comments.



Source

iFlorida Model Deployment Final Evaluation Report

Author: Robert Haas (SAC); Mark Carter (SAIC); Eric Perry (SAIC); Jeff Trombly (SAIC); Elisabeth Bedsole (SAIC): Rich Margiotta (Cambridge Systematics)

Published By: United States Department of Transportation Federal Highway Administration 1200 New Jersey Avenue, SE Washington, DC 20590

Source Date: 01/30/2009

EDL Number: 14480

URL: http://ntl.bts.gov/lib/31000/31000/31051/14480.htm

Other Lessons From this Source

Lesson Contacts

Lesson Analyst:

Firoz Kabir
Noblis
202-863-2987
firoz.kabir@noblis.org


Rating

Average User Rating

0 ( ratings)

Rate this Lesson

(click stars to rate)


Lessons From This Source

Assess security risks, threats, vulnerabilities, and identify countermeasures to ensure operations of transportation management centers.

Be flexible to use data from various sources, such as the highway police patrol’s incident data, user feedback, and monitoring stations, to develop a statewide traveler information system.

Beware of challenges involved in developing an integrated statewide operations system for traffic monitoring, incident data capture, weather information, and traveler information—all seamlessly controlled by a central software system.

Beware of costs, utility, reliability, and maintenance issues in deploying a statewide transportation network monitoring system.

Beware of the limitations of using toll tags in order to calculate travel time on limited access roadways and arterials.

Beware that software development for ITS projects can be utterly complex, which demands avoiding pitfalls by following a rigorous systems engineering process.

Define a vision for software operations upfront and follow sound systems engineering practices for successfully deploying a complex software system.

Deploy a variable speed limit system only after the software systems required to support it are mature and reliable.

Design traffic video transmission systems around the constraints of bandwidth limitations and provide provisions for remote configuration of video compression hardware.

Develop an accurate, map-based fiber network inventory and engage ITS team in the construction approval process.

Develop an effective evacuation plan for special event that gathers a large audience and consider co-locating the responding agencies in a joint command center.

Ensure compatibility of data format of the field-weather monitoring sensors with the central software in the transportation management center.

Ensure that experienced staff oversee the development of a complex software system and thoroughly follow systems engineering process.

Ensure that Highway Patrol's CAD system operators enter key information needed by the transportation management center operators.

Establish a well defined process for monitoring and maintenance before expanding the base of field equipment.

Estimate life-cycle cost of ITS technologies as part of procurement estimates in order to assess the range of yearly maintenance costs.

In developing software for automated posting of messages on dynamic message signs, focus on the types of messages that are used often and changed frequently, and also include manual methods for posting.

Incorporate diagnostic tools to identify and verify problems in the transmission of video in a transit bus security system.

Perform adequate analyses and tests to design, calibrate and validate the capabilities of a bridge security monitoring system in order to reduce false alarms.

To support statewide traveler information services, design and implement reliable interface software processes to capture incident data from the local and highway patrol police’s computer aided dispatch systems.

Use simple menu choices for 511 traveler information and realize that the majority of callers are seeking en route information while already encountering congestion.

Application Areas

None defined

States

Florida

Countries

United States

Focus Areas

None defined

Goal Areas

Mobility

Keywords

None defined

Lesson ID: 2009-00489