DRBD and the sync rate controller (8.3.9 and above)

The sync-rate controller is used for controlling the used bandwidth during resynchronization (not normal replication); it runs in the SyncTarget state, ie. on the (inconsistent) receiver side.

It’s configured as follows:

  • Set c-plan-ahead to approximately 10 times the RTT; so if ping from one node to the other says 200msec, configure 2 seconds (ie. a value of 20, as the unit is tenths of a second).[1. In 8.4 the default is already 20, so the dynamic rate controller is used by default.]
    Please note that the controller is only polling every 100msec; so c-plan-ahead values below 5 don’t make sense, as the controller hasn’t collected enough information to decide whether to request more data. We recommend to use at least 1 second (configured value is 10).
    This value specifies the “thinking ahead” time of the controller, ie. the time period the controller has to achieve the actual sync-rate.
  • Configure minimum and maximum values via c-min-rate and c-max-rate; these depend mostly on the available bandwidth per resource.
    The c-min-rate is the minimum bandwidth that will be used during a resync, whereas c-max-rate is the most bandwidth that can be used by a resync.
  • Now decide whether to use c-fill-target or c-delay-target – you can choose only one.

Difference between delay and fill based control

If you set c-fill-target to a non-zero value, DRBD will try to keep that much data on the wire; if application IO gets in, it will temporarily displace the synchronization traffic. This means that application data will have only a limited amount of synchronization data in the buffers before it, which helps latency a bit.
The data still has to fit into the socket buffers, along with the application IO, so using multi-MB sizes here doesn’t make sense; 100kByte is a good starting value.

With a proxy you should use c-delay-target, so set the c-fill-target value to zero. This way the time interval that the synchronization data is on the wire is measured; if application IO gets in, this triggers the controller, and it will turn back the synchronization speed, to keep the communication latency at the specified value. Use 5 times the RTT as a starting point.

2 replies
  1. Piotr Kandziora
    Piotr Kandziora says:

    Hi,

    So this is used when your volumes are in SyncTarget state. How to limit replication bandwidth when your volumes have been already synced (normal replication)?

    Reply
  2. Flip
    Flip says:

    Hello Piotr,

    when the Synchronization is done, the bandwidth usage is determined solely by your application – if it doesn’t write, there’s nothing to replicate to the other side.

    DRBD doesn’t offer any configuration settings here, because it could only lower the speed – and this is not what you want.

    Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *