Data Replication Queue

When database replication is unavailable, replication data is queued until the queue file has reached its size limit, or the node comes back online. You can modify:

You can set the queue limit to a fixed size, using the following settings:

You can set the queue limit to a percentage of disk, using the following parameters:

It is best practice to set these variables the same on all Bravura Security Fabric servers. After making changes, you must restart the Database Service or click Propagate and reload replication configuration on all servers on the replication configuration page.

During the replication process, the replicating servers attempt to send data to another server. If they fail to connect, they retry after 30 seconds until they succeed, or the queue reaches its limit. Once the limit is reached, the sending server suspends database commits, and stops accepting requests.

The server state when the queue is full is called DB COMMIT SUSPEND mode. One symptom of this mode is that every Bravura Security Fabric UI page shows the following message:

Changes to this instance are temporarily disallowed. Please contact the Bravura Security Fabric administrator. Due to a problem in the replication environment all pages except Database replication and System logs are temporarily disabled.

When the server is in DB COMMIT SUSPEND mode, only superusers can log into Bravura Security Fabric . They will only have access to the systems log and database replication pages.

This state triggers the DB COMMIT SUSPEND exit trap. If you adjust the queue limit, or free some disk space when the limit is set as a percentage of disk, to allow the server to come back to normal replication mode. This triggers the DB COMMIT RESUME exit trap.

You can configure Bravura Security Fabric to send email warnings, or some other notification, to administrators when replication failures occur.

Replication event action options can be accessed by navigating to Manage the system > Maintenance > System variables or Manage the system > Maintenance > Options.

By default some events are configured to run the pxnull interface program with the pxnull-replication script, located in the \<instance>\script\ directory, to notify users when the server has attempted to reconnect to the downed server a certain number of times. The script is set to use the settings set in the Email configuration page and contact the account in the RECIPIENT EMAIL field. You must install the pxnull program with the Connector Pack .

You can also send a warning email to administrators in the event of a short-lived replication problem, by configuring the DB REPLICATION CONN FAILURE event action (Manage the system > Maintenance > Options). The advantage of detecting short-lived replication problems is that administrators can react more quickly to problems. The disadvantage is that there may be many spurious replication problems, due to a busy server rather than any underlying problem in the environment, and these may generate too many emails. A carefully configured exit trap can check for a threshold number of problems before sending an email. See the sample script below.

function REPLICATION_FAILURE
{
  var $retryKey;
  var $retryVal;
  var $retryThreshold = 50;  #Default limit is 256 tries every minute before database is suspended.
  for( var $i = 0; $i < size( $sessdat ); $i = $i + 1 )
  {
      $retryKey = keyAt($sessdat, $i);
      $retryVal = valAt($sessdat, $i);
      if (strcmp($retryKey, "retry")==0)
      {
         if ($retryVal > $retryThreshold)
         {
          # Add code to generate and send email notifications, and open
          # monitoring tickets here
            log ("replication-failure-pxnull.cfg: Send emails about failure: " + $retryVal);
         }
         else
         {
          # Threshold has not been reached. Do nothing.
         }
      }
   }
   return 0;
}

The time interval between when replication stops – for example, due to network outage or hardware problem – and when the originator server becomes non-functional, depends on the rate of data updates:

The time available to resolve a problem before Bravura Security Fabric functionality fails depends on the frequency of logins, password changes on target systems, and passwords being checked out.

Bravura Security recommends that you periodically estimate the time available to resolve problems based on current metrics and the current queue size.