FieldServer Setup, Hot Standby Mode 1 and 2

Project Overview: Using a FieldServer in a Hot Standby Mode

Hot Standby Mode 1, also known as true Hot Standby, is a redundancy configuration utilizing two FieldServers: an Active and a Passive unit. This mode offers simplicity in implementation but comes with inherent limitations.

Alternatively, Hot Standby Mode 2 (Dual Redundant Mode) provides a more sophisticated redundancy solution suitable for complex applications, albeit requiring a more intricate configuration process.

Hot Standby Mode 2 (Dual Redundant Mode)

In this mode, both the Primary and the Secondary system are continuously active and the Data Arrays of both systems are continually updated. Each system keeps an image of what is happening in its complementary system. Figure XII shows how the dual image allows for multiple data paths, which in turn provide a high level of data redundancy.

Possible FieldServer Configurations for Redundant, Hot Standby operation

Configuration 1: Single FieldServer, single connection to field node. This is not a hot standby or redundant configuration. It is included here for comprehensiveness.

Configuration 2: Single FieldServer, dual connection to field node. This mode of operation requires the field node to have two serial ports.

For any given node configured in the FieldServer, there is a “Backup Connection”. If there is a single communication failure, the kernel will attempt the next poll out the backup connection. If this poll is successful, then the backup connection is used from this point onwards. If the poll fails, then the kernel will alternate between the primary and the backup connection until the number of allowed retries is used up. Once this happens, the node is marked offline.

Configuration 3: Hot Standby FieldServer Pair, Ribbon Cable, single connection to field node. This is the classic hot standby mode, where the ribbon cable shares the FieldServer serial ports with a single field node serial port. If the primary FieldServer fails, the secondary FieldServer takes over. If a single port on the primary fails, at present, the primary FieldServer will continue to be the active FieldServer.

Configuration 4: Hot Standby FieldServer Pair, Ribbon Cable, backup connection to field node.

Combination of Configuration 3 and Configuration 2.

Configuration 5: Hot Standby FieldServer Pair, No Ribbon Cable, single connection to field node from each FieldServer. Although this is labeled as single connection to field nodes, each FieldServer in the pair is connected to one of two ports on field node. When the Active FieldServer fails, the standby FieldServer will automatically take over. The question is, what happens if a single cable is cut? We want to switch to the standby FieldServer and attempt to recover communication. If however a node fails, communication recovery is not possible, and we want to prevent flip-flopping between the FieldServers. This can be prevented by looking at the “Health Data” of the Standby FieldServer. (See section "Monitoring Standby FieldServer Health from the Active FieldServer")

Configuration 6: Hot Standby, No Ribbon Cable, backup connection to field node. This is not a legal configuration. Included for comprehensiveness. (It would require the PLC to have four serial ports).

Hot Standby FieldServer configuration issues

Configuring a FieldServer for Hot Standby

1. Establishing Hot Standby Operation
2. Defining a “Hot_Standby” connection in configuration file.
   The FieldServer can determine that it is configured to act as part of a Hot Standby Pair when a “hot_standby” connection is defined in the configuration file. At startup a particular FieldServer would now need to determine if it should become the Active or the Standby FieldServer. At startup the FieldServer will wait for a heartbeat message from an Active FieldServer for a random time of between 2.0 and 2.8 seconds. If the Failover Timeout has expired, the FieldServer will become the active FieldServer and start sending heartbeat signals on the Hot Standby connection. Typically this will be used in Configuration Example 3 above (Section Configuration 3:).
3. Explicit configuration.
   The FieldServer can be explicitly configured as either a Primary or Secondary FieldServer by way of a separate hot standby configuration file. This file will be stored on the FieldServer flash disk. If the FieldServer is configured as a Secondary FieldServer, then it will wait an additional one-second before starts up. This will give the Primary FieldServer (if one exists) time to become the Active FieldServer when the hot standby pair is switched on at the same time. If both FieldServers in the hot standby pair are configured the same way, then essentially the mode of operation reverts back to the automatic configuration explained above. Typically this will be used in the Configuration Example 5 above (Section Configuration 5:).
4. Heartbeat Message Operation.
   Heartbeat operation can be configured to work over a Serial Connection or over an Ethernet connection (new) or a combination thereof (new). This is achieved by setting the Connection_Mode of the relevant Connection to Hot_Standby. Two connections must be designated for Hot Standby operation.
5. Heartbeat Messages over a Serial Connection.
   Configuration of this is straightforward and requires two serial connection cables to run between the Hot Standby FieldServer pair.
6. Heartbeat messages over Ethernet.
   Sending heartbeat messages over Ethernet require that FieldServers of a particular Hot Standby pair be explicitly configured with a “Pair Name”. This will be done in the hot_stby.ini file. When a FieldServer wants to send a heartbeat message over Ethernet, it would first send an Ethernet “Who-Is Standby” broadcast message containing the FieldServer Hot Standby “Pair Name”. Any FieldServer that receives this message and has a matching Hot Standby Pair Name will respond with an “I-Am Standby” message. Subsequent heartbeat messages will be sent directly to the MAC address of the response message. Note that this mechanism does NOT USE IP ADDRESSING. This eliminates the need for annoying IP address configuration. It also means that the messages cannot be routed however, but this is OK since anybody who wants to put a router between the two FieldServers in a Hot Standby pair need to re-evaluate their career choices anyway.
7. Transferring Data Arrays to the Standby FieldServer.
   An integral part of Hot Standby operation is to transfer “live” FieldServer data from the Active FieldServer to the Standby FieldServer. For this to work the Active and Standby FieldServers must have identical data arrays configured. This transfer of data is achieved by using the heartbeat signal. If the Connection_Mode is set to Hot_Standby_Data, then the heartbeat message will continuously transfer all the Data Array data to the standby FieldServer, one section of data at a time. If the Connection_Mode is set to Hot_Standby_Data_On_Change then only the part of the Data Array that changes will be sent to the Standby FieldServer. Typical applications for this are when the Data Arrays are very large and the data changes very seldom, such as for a fire panel system. In particular this would make sense if the heartbeat message is send over Serial connections.
8. Monitoring Standby FieldServer Health from the Active FieldServer.
   Both FieldServers in a hot standby configuration pair can be configured with a Data Array containing the FieldServer status. This data will be send back to the Active FieldServer by including it into the heartbeat message acknowledge packet. Typically this would contain the last known status of all the nodes on the Standby FieldServer. This information could then be used by the Active FieldServer to decide if it should cause a FieldServer swap-around in case of a port/node failure.
9. Connecting and downloading to the Standby FieldServer.
   Ruinet can be used to connect to the Active as well as the Standby FieldServer. For this to happen it will be required to know the IP_Address of the relevant FieldServer. Ruiping (using the ‘h’) switch can be used to determine the FieldServer IP Address, Hot Standby Pair Name and Hot Standby status (i.e. active or standby)
10. Naming convention and operation of the hot_standby ini files.
• If the FieldServer is configured as a Primary FieldServer, the file hsb_p.ini must be created and downloaded to the bridge.
• If the FieldServer is configured as the Secondary FieldServer, the file hsb_s.ini must be created and downloaded to the FieldServer.
• The syntax of the hsb_x.ini files is exactly the same as that of the standard FieldServer configuration file.
• The Hot Standby Pair Name must be set as the “Bridge” Parameter “HS_Pair_Name”.
• Normally, the Hot Standby connections will also be configured in the ini files.

IP Addressing

Mode 1: Static Dual IP Addresses
In this mode, each FieldServer is provided with an IP Address via the normal configuration process. The IP Addresses remain static and the client software package has to detect that the Primary FieldServer has gone offline and that the Secondary FieldServer with another IP Address has taken over.

Client software such as FieldServer Technologies OPC driver will handle this occurrence seamlessly. Packages such as Intellution’s Fix, without FieldServer Technologies OPC driver, will handle the switchover automatically, but report a communications failure during this period. Packages such as GE Cimplicity do not handle this process at this time.

Mode 2: Dynamic Single IP Address
This mode is for the client packages that do not handle Mode 1. The Secondary FieldServer will assume the IP Address and MAC address of the Primary on switchover. Most client software will still report a communications failure during this interval.

Mode 3: Bumpless Transfer IP Address
>This mode is the same as Mode 2, but the Standby FieldServer closely shadows the state of the Active FieldServer’s socket by watching traffic on the network. If a switchover is required, the client software will not be able to detect that another FieldServer has taken over the communications.