To allow neighbour peers a transparent failover, between one SAPC to the mated peer, the following information is replicated:

• Subscriber data (including Subscriber Fair usage accumulated data).

• Provisioning data (policies, subscriber groups data, services, and profiles).

• Session information, notification data and time trigger data.

The SAPC does not replicate any other data which is not stored in the Database Service (DBS). It is responsibility of the operator, to align the non replicated data between both SAPC peers. This comprises the following data:

• The SAPC configuration data that is provided through COM, and through configuration (cfg) files. Therefore, the configuration data has to be provided to each SAPC individually.

• Licensing information has to be managed in each SAPC individually. For capacity licenses based on SAPC capacity measures, it shall be considered that both SAPC peers participate in each measure. For example, the SAPC mobileActiveSessions capacity measure, counts the total number of simultaneous active mobile sessions in both SAPC peers.

• Alarms, logging events and performance measurements are local to each SAPC.

• User management information and authentication credentials.

• Networking configuration.

Data mirroring is performed asynchronously for performance reasons. Hence, database changes in one SAPC are temporarily stored in memory while they are sent to the mated peer. Next, the received changes are saved and a confirmation is sent back to the mated peer. The SAPC can then apply the received changes in the local database, and the mated peer can release the transmitted data. These memory buffers in both SAPC peers, can be regarded as a backlog.

This procedure allows the SAPC to handle failures during data mirroring and ensure database consistency by forcing the same order of changes in both SAPC peers. Thus, having a backlog is a normal condition. However, there are situations when some transactions in the backlog cannot be processed immediately, for example, because of overload in either the sender or the receiver side, or disturbances in the Replication Channel. To handle this case in terms of resource use, it is possible to configure the maximum amount of memory used by the backlog.

Data mirroring synchronizes database changes between SAPC peers. However, there are a number of cases where the original contents of the databases are different, therefore synchronizing the changes does not make the contents identical. In the following cases, a complete synchronization of the database is needed:

• After initial startup of a SAPC, when the Replication Channel is up and the mated peer is available.

• When transactions have to be discarded because of memory and network capacity limitations.

• After split-brain situations (loss of network connectivity between SAPC peers).

• When data mirroring functionality is resumed after being disabled as a result of an operational procedure, for example when one SAPC is set to Halted state.

• After a geographical redundancy configuration change, like change the preferred SAPC to non-preferred.

Synchronization itself, transmits a consistent view of the database from one SAPC to the mated peer. Any changes done in the SAPC transmitting the data in the meantime, are also transferred as normal database changes, which are applied in the mated peer once the initial database is fully imported. The synchronization process is automatically triggered by the SAPC when it is required, no manual intervention is needed. The SAPC DBS component, sends notifications about the start and the end (successful or unsuccessful) of the synchronization process.

When a SAPC reloads, it starts synchronization from the still-running mated peer. However, in other situations where a complete synchronization is needed, it is not possible to know which of the SAPC peers holds the most up-to-date database. In those cases, for example after a split-brain situation, the database from the preferred SAPC is maintained, and the non-preferred SAPC synchronizes from the preferred one.

The SAPC that starts a complete synchronization of the database with the mated peer data, can not neither handle traffic nor provisioning operation until the synchronization is finished.

An Active-Active solution, means that any of the SAPC mated peers can handle traffic and provisioning requests from the client peers, maintaining a SAPC synchronization between them (see SAPC DBS Synchronization). It is required in the neighboring traffic and provisioning plane the following:

• diameter clients/DRA nodes shall support subscriber and session stickiness (preventing the latency from the database ownership), distributing homogeneously the traffic between both active SAPC peers. One of the possible solutions is that the diameter client/DRA distribute traffic based on subscribers range. Each SAPC should be assigned with a range, ensuring the traffic from each range is balanced.

• diameter clients/DRA nodes, shall be able to do failover to the SAPC that remains active in case of SAPC peer failure. Once the failed SAPC is again available, it shall be able to do failback and continue the homogenous traffic distribution between both SAPC peers.

• provisioning systems, should follow a primary/secondary priorities scheme, where all the provisioning orders have to be sent to the SAPC configured as preferred (primary), if it is available. In case of preferred SAPC failure, the provisioning system shall redirect the provisioning orders to the non-preferred SAPC (secondary). The provisioning priorities scheme, prevents the loss of data after split-brain scenarios, where the non-preferred SAPC synchronizes from the preferred one (all the provisioned data in the non-preferred node during the split-brain is not maintained).

• in network deployments with the CUDB or an external database function, both SAPC peers shall be configured in the CUDB or external database node, to broadcast the SOAP notifications..

The SAPC configuration, operation and maintenance procedures are performed in each SAPC individually.

One of the SAPC peers must be configured as the preferred node in geographical redundancy. This is used when resolving some faulty situations where is not possible to know which of the SAPC peers holds the most up-to-date database. It is also used to prevent loss of data in some Replication Channel failure scenarios. These are the possible situations where the preferred configuration is used:

• After recovery of the network connectivity (neither the Replication Channel nor the Application Channel are available) between both SAPC peers, the database from the preferred SAPC is maintained, and the database from the non-preferred is discarded.

• When both SAPC peers reload and come up nearly at the same time, the preferred SAPC takes the role of active and provides data to the not preferred SAPC.

• When the Replication Channel is down but the Application Channel is available, the preferred SAPC takes the role of active and the non-preferred goes to standby, to prevent loss of data after recovery of the Replication Channel.

Clients need to maintain one connection towards each SAPC in a redundant mated active-active pair. Seen from the external network, the two SAPC peers are as if they were two different standalone deployments, both of them with the capacity of handling traffic and provisioning. The redundant SAPC solution exposes two VIP addresses for traffic, another two VIP addresses for provisioning and two diameter host names.

Each SAPC handles the following mandatory VIP addresses:

And the following optional VIP addresses:

The following picture shows all the IP addresses involved in the geographical redundancy scenario with details about which IP addresses are available for each SAPC. As in standalone deployments, additional VIPs can be added if traffic separation between interfaces is required.

Figure 2   VIP addresses in the SAPC active-active geographical redundancy solution

The SAPC executes transitions from one state to another depending on the information provided by the supervision functions explained in Active-Active Geographical Redundancy Supervision and Control Functions.

Figure 3   SAPC Active-Active geographical redundancy state machine

The basic principle for the Active-Active Geographical Redundancy function, is that both SAPC peers process all incoming traffic and provisioning operations. Each SAPC keeps the state of the mated peer. However, there are situations that might cause one SAPC not to be synchronized with the peer. This chapter describes those scenarios and how they are handled.

In a standalone deployment, when the SAPC recovers from a restart, the database information recovered from the backup may not be fully up to date. Hence the SAPC increments its own Origin State Id and includes the new value in every response message alerting the peer diameter nodes about the loss of previous session state.

In an Active-Active Geographical Redundancy deployment, the Origin State Id information is replicated between both SAPC peers. Upon a SAPC restart, the Origin State Id is not incremented, it is obtained from the most up-to-date database information during synchronization with the mated peer. This enables to each SAPC to send the same Origin State Id value. Transitions between SAPC redundancy states do not increase the value of the SAPC Origin State Id, as those are transparent to the peer diameter nodes in the external network.

The SAPC only increments the Origin State Id if both SAPC peers restart. This is, when the SAPC recovers from a restart and cannot replicate the Origin State Id information (cannot sync with the mated peer, for example because of loss of network connectivity in the Replication Channel), the SAPC increments its own Origin State Id.

When one SAPC detects a split-brain situation, the own Origin State Id is not increased in order to provide service availability, as the session information is available in both SAPC peers. When the Replication Channel is re-established, the preferred SAPC continues providing service, but the Origin State Id is not increased (the non-preferred SAPC makes a complete synchronization with the mated peer and then, replicates the same Origin State Id).

If the SAPC restarts in a split-brain situation, the SAPC autonomously increments its own Origin State Id, and this information cannot be replicated in the mated peer. As a result each SAPC may send a different Origin State Id value as long as the Replication Channel is unavailable. When the Replication Channel is recovered, the Origin State Id in the preferred SAPC is maintained (the non-preferred SAPC makes a complete synchronization with the mated peer and then replicates the same Origin State Id).

When a diameter client requests for a new diameter session establishment, the SAPC Origin-Host included in the diameter answer, is the one from the SAPC peer that manages the session establishment. Each SAPC peer have a different Origin-Host value. In case of failure in the SAPC that managed the session establishment, the SAPC mated peer (that handles all the sessions while the failed SAPC recovers), always include as origin-host, the one used in the session establishment instead of its own, to avoid any problem in the diameter client.