IMSA Journal Feature Article
Sept/Oct 2002
IMSA Journal
By Roger L. Shaffer, Advanced Traffic Control, Inc.

Early traffic control devices

Traffic congestion has been a significant urban problem for over a hundred years. As early as 1868, the city of London recognized the need to control its traffic, (wagons, buggies and pedestrians) and installed what may be the world’s first traffic signal. Operated by a policeman, the signal device had semaphore arms and (for night use) a revolving gas lantern with red and green lights. Unfortunately, a month after its installation the signal light exploded injuring the officer.

     While the Duryea motor wagon was offered for sale in 1896, the historians among us will remember a certain Henry Ford who introduced the Model T, “a car for anybody and everybody” in 1908. His idea caught on?in a big way.

     America’s entry into the world of traffic control began in 1911 with the low tech but common sense painting of a centerline on a Michigan road. Cleveland boasts of the first electric traffic signal in 1913. The stop sign made its appearance in 1915, and Detroit installed the first 4-way electric traffic light at the intersection of Woodward Avenue and Fort Street in 1920. After witnessing an accident between an automobile and a horse drawn carriage, Cleveland inventor Garrett A. Morgan was motivated to design his own electric light and semaphore arm traffic signal. The first to actually apply for a patent on a traffic signal, Morgan’s patent was granted in 1923. The next year Morgan sold his rights to General Electric for forty thousand dollars.

     These early traffic control initiatives incorporated all three of the components necessary for an effective traffic safety program: Engineering, Enforcement, and Education. These control devices were engineered to not only regulate the new drivers, but also to educate them as they undertook their role in the solution to the particular traffic problem being ameliorated. The painted centerline made it clear that the road was divided and showed drivers how to stay on their own side and thus avoid collisions. Stop signs and traffic lights (or semaphore arms) taught them how to wait and take their turn at intersections. The ever-present police officers reinforced the lessons with traffic tickets, just as they do today.

ITS today

     Today’s computer controlled Intelligent Traffic Solution systems?trying hard to reduce the more than six million road crashes experienced annually in the US, while at the same time keeping the traffic flowing!?still place significant emphasis on communicating to the motoring public through traffic signs and signals. Also, many urban systems provide special traffic and weather broadcasts. Traffic conditions and weather reports are often available on-line before motorists start their journey. In today’s traffic control paradigm, real-time information rules.

    Aside from obvious technology differences?like LED’s, variable message signs, and computer controlled devices (instead of a police officer hand turning exploding gas operated signal lights or waving semaphore arms)?the major change is the traffic manager’s ability (and responsibility) to communicate the ever-changing traffic conditions to motorists and emergency personnel. Effective communication of fluid traffic conditions requires real-time, interactive control over a system’s dynamic signs and signals.

    Road tunnels have been an integral part of comprehensive traffic control solutions for years. Just four years after Garrett Morgan’s traffic signal was patented, the Holland Tunnel opened. But tunnels have unique problems and concerns?in addition to those of the surface highway traffic systems. Solving those problems is particularly difficult in aging tunnel systems. While many of the concepts and solutions discussed below will apply to any controlled traffic system, this article will focus on intelligent traffic solutions as they relate to the computer-controlled tunnels within our highway systems.
Tunnel control system issues

     Today tunnels handle large volumes of truck, SUV and automobile traffic moving at expressway speeds. In the narrow confines and restrictive environment of a tunnel even ostensibly minor incidents can soon become deadly serious since critical mass for a catastrophic incident may be achieved far more quickly than on surface highways. Time becomes a significant issue. To effectively minimize incidents through implementation of sign and signal plans and the dispatch of appropriate emergency services, tunnel control operators must have real-time data from their field equipment, and they must have real-time control over their equipment. And, like the highway systems they serve, tunnels are around the clock operations. Tunnel computer controlled systems can have no "planned downtime." That means the tunnel control system must also have constant availability fault tolerance.

      Administrators of many of today’s maturing tunnel control systems are hampered by high maintenance costs and missing functionality. A significant portion of the ever-increasing maintenance cost is attributable to aging computer systems and “orphan software.” Expertise drain is a direct consequence of an aging computer system. Where there are a diminishing number of people capable of maintaining older systems, costs go up. Moreover, where, as is often the case in any maturing traffic control center, the system uses proprietary code developed by a company that has either gone out of business, or one that no longer offers support for their product, Administrators are saddled with orphan software. Lacking both source code and expertise, they are unable to maintain their system. Administrators are also limited by their inability to enhance or expand the control system with new functionality, desirable as it may be, as it is often not compatible with the limitations of the aging system.

     Finally, while there will be the exchange of a legacy field device for a superior cutting edge device, the time honored saying “if it ain’t broke don’t fix it” still applies, and much of the performing legacy equipment already in place will be retained in a new system. That means an upgraded tunnel control system must be able to talk to both new, and legacy equipment in real-time.
     Advanced Traffic Control, Inc., (ATC) 1515 University Drive, Suite 105, Coral Springs, Florida, 33071, USA, has developed cost effective solutions that resolve all of these issues.

Intelligent tunnel solutions

Operating under the flexibility of RFPs, as opposed to the constraints of RFQs, in the fall of 1998, Frank L. Roark, (now ATC’s President) began designing and implementing upgraded tunnel control systems for Virginia’s Department of Transportation. The new Roark designed control systems, designated “newTON” by VDOT, are based on Microsoft? Windows? operating systems (specified by VDOT as part of their own COTS contribution to cost reduction). The newTON systems replaced the aging MODCOMP? driven TOMAC control systems then in use at the Hampton Roads Bridge Tunnel, the Elizabeth River Tunnel, and the Monitor-Merrimac Memorial Bridge Tunnel, all located in Virginia’s tidewater area.

     These innovative newTON systems rely on proven and supported commercial off-the-shelf components, (ATC’s basic SCADA engine is Intellution’s Ifix? product line) thereby solving the problem of expertise drain and orphan software inherent in the TOMAC systems they replaced. The newTON control system design was a resounding success (“Virginia DOT uses SCADA for tunnel traffic control. Better Roads (October 2000); and “A New Wind Blows at ITS?” Advanced Transportation Technology News (April 1999)). The success of the initial Hampton Roads Tunnel upgrade system garnered VDOT’s tunnel Administrators the prestigious Commissioner’s Award of Excellence, and Advanced Traffic Control went on to author the winning RFPs for both the Elizabeth River and the Monitor-Merrimac newTON upgrades. (“Under the river. Traffic Technology International (International Review Issue, 2002)).

     An integral component of the newTON system at Hampton Roads was a COTS data fault tolerance system. But data fault tolerance alone was found to be insufficient. Thus the Elizabeth River iteration called for a COTS hardware and data fault tolerance system, which was also retrofitted to the Hampton Roads tunnel. Ultimately this, too, was found wanting. The culprit was the threat of the occasional crashing of the OS (the blue screen of death), or the OS locking up and requiring a reboot.

     Occasional crashes and locking are familiar to many Windows users. Without a satisfactory fault tolerant solution, either one is unacceptable in a tunnel environment. However, while it is true that more stable and robust systems are available, Windows operating systems are well supported and offer both variety and convenience in developer tools. Thus, code for tunnel applications on Windows OS can be written in days, whereas code for the same applications in other systems may take months. That is a significant cost issue for the tunnel Administrator. Accordingly, it became clear that what was actually needed was solutions for all three: data, hardware, and software fault tolerance.

     Finding no commercially available off-the-shelf system that met the data, hardware, software, and real-time fault tolerant requirements of the tunnels, Advanced Traffic Control, Inc. designed a continuous availability system that does. It is our unique Advanced Fault Tolerant Solution (AFTS) that is the key to seamless communications with both the new and legacy field equipment. Moreover, with the AFTS system, computer maintenance is accomplished with no impact on tunnel operations. And, AFTS permits future expansion and development of the newTON operating system. Consistent with our commitment of avoiding orphan software, Advanced Traffic Control (for maintenance only) provides the source code for the firmware and PC side software, parts lists, and board layouts for this proprietary feature. These strategies assure Tunnel Administrators of a safe, supportable, expandable, yet cost-effective control system. (“Tolerating Faults Improves Control.” Advanced Transportation Technology News (February 2002).)

The AFTS system overview

     The Advanced Fault Tolerance Solution is a continuous availability system consisting of a combination of hardware and software that permits two independent computers to function as a redundant single computer.  One machine is designated primary and runs the newTON system. The other is a hot backup, ready to take over and run newTON in the event of a hardware or software fault in the primary. When the backup computer takes control of the system it appears to the network exactly as if it were the primary machine. This dual configuration effectively provides hardware fault tolerance for the system.

     Fault tolerance for the relational database is achieved through off-the-shelf mirroring software. This mirroring software copies files to the backup machine that have changed from specific directories on the primary, assuring that the backup RDB is always current. Additionally, the software dealing with the RDB is configured to immediately flush all writes to the database. These features prevent the data loss that could otherwise occur when data is in the cache and a system fails.

Dealing with legacy equipment

     Combining legacy field equipment with new field equipment in the same control system can present challenges as it may require transmitting multiple protocols over the same RS232 line (as it did on the Monitor-Merrimac project). Rather than developing control software compatible with the various legacy devices, this process is made fault tolerant through intelligent dual ported serial interfaces designed and built by Advanced Traffic Control. These smart switches use a dedicated hard real-time microprocessor to communicate to field devices while providing ‘on demand’ data to the PC. This avoids the requirement that the servers talking to the field equipment must conform to the field equipment’s various packet formats and timing requirements.

     Fault tolerance for this serial I/O is provided by these intelligent dual ported serial interfaces. These rack-mounted devices accept two serial ports from the serial controller card in the primary and backup computers, and provide a single surge suppressed serial port to connect the modems to the field equipment. They can be set to switch from one input channel to the other input channel should there be a loss of communications, or to synchronize to a master port so that all the serial ports switch at the same time. The cards are hot swappable and require no cable disconnection.

     The fault tolerant path to the iFIX process database by iFIX View Clients and other nodes, is accomplished by having both the Primary and Backup computers having identical Ethernet cards (including identical MAC addresses), but having only one card physically attached to the network at a time. Ethernet switching is accomplished through the AFTS Fault Tolerant Controller. To maintain current output states for all of the devices, the PDB clone runs between the two systems using a dedicated serial link that copies the output tags from the primary PDB to the backup PDB. This insures that there will not be a change of field devices during any failure, and thus accomplishes fault tolerance for the PDB.
Fault detection

     The AFTS's Fault Tolerant Controller is a microprocessor controlled device that is the core of the AFTS system. Rack mountable, the FTC provides two basic but critical functions. First, it controls which of the two computers the serial ports and network interface are connected to. Second, it serves as the heartbeat monitor. This monitoring function and the reporting of system status is done via serial communications to each computer. See the Fault Tolerant Monitor State Transition diagram.

     Critical custom tasks and workspace applications are monitored by a software heartbeat mechanism within the primary computer by the Task Monitor program. Each critical application must use an iFIX simulation PDB tag as a heartbeat. Each time the application runs it will increment the value of the PDB tag. The Task Monitor also checks
that all of these tags have incremented. If an application fails to increment its PDB tag, it will be considered to have failed and the backup system will assure continuous availability.

The bright light: continuous system expansion and development

     As the reader may have already recognized, in addition to no planned or unplanned downtime for maintenance, the AFTS system also gives the Administrator a safe and cost effective method for continuous expansion and development of the control system. System changes of every nature, whether Microsoft? service pack installations, software revisions, functionality additions or any other changes, can all be added and tested on the backup system without affecting the configuration or the operations of the primary.

     These changes are easily accomplished by installing the service pack, revision or new function on the backup machine. After testing, if necessary an offending product can be uninstalled. If the change is a success, the Administrator would then switch into the “improved” machine, (accomplished by hand throwing a switch on the Fault Tolerant Controller) thereby making it the primary, and then making the identical change on the other (now) backup computer.

     By this simple means, the tunnel Administrator can constantly maintain and upgrade his control system with no impact on tunnel operations. No more revision lock. Every service pack, every software revision, can be installed contemporaneous with its issue.  New functionality can be added without disruption of the tunnel operation. The control system can always be “up to date.” Expertise drain becomes a non-issue and maintenance costs remain manageable.

More on Garrett Morgan

     Prior to patenting his traffic signal, Garrett Morgan was personally involved in a tunnel crisis. On Jul 25, 1916, workmen digging a tunnel under Lake Erie inadvertently ignited a pocket of natural gas 250 below the lake. Eleven tunnel workers were killed and 10 rescuers were overcome. Two years earlier Morgan had patented his ‘safety hood and smoke protector’ device. Using his device, Morgan and his brother rescued two men and recovered the bodies of four more before the U. S. Bureau of mines halted further rescue and recovery efforts. Morgan’s gas mask was later modified for use by the U.S. Army in World War I. Born in Paris, Kentucky in 1877, Garrett A. Morgan was the son of a former slave.


IMSA Journal Feature Article
Sept/Oct 2002
IMSA Journal