Block storage drbd compression deduplication

Linux Data Deduplication and Compression: One more reason to use block level data replication.

Having recently returned from my 6th Red Hat Summit (RHS), I’m writing this blog to answer a common question: “why replicate at the block level?” Using block-level replication, we can easily add high availability or disaster recovery features to any application that doesn’t natively support them.

The most frequently asked question we heard at RHS was, “how do you compare to [insert application replication OR filesystem here?]”. In most cases, the answer was, “LINBIT’s replication software, DRBD, replicates data at the block level.” It would be an extreme task to run performance comparisons vs all of the other replication technologies on the market, so generally we provide background information, including:

 

  • DRBD can usually replicate with 1-3 percent overhead to the cluster’s backing disks, as measured by FIO
  • In dual-primary mode, overhead increases to 15-20 percent
  • DRBD is compatible with any application or Linux filesystem, and is effective at replicating multiple applications simultaneously.
  • DRBD has a read-balancing feature. If you are running a read intensive application, DRBD will pass through reads to secondary nodes once the primary is running at maximum capacity, enabling you to leverage all of your replicated systems. One test showed 1.7x the read performance compared to the advertised speed of the drive.

Deduplication and Compression

Generally, it comes down to efficiency. EMC, NetApp, and the other big storage players use block level replication in their appliances because this way the replication doesn’t need to go “all the way up the stack.” It enables flexibility, stability, and performance. And now, Red Hat has given us one more reason to replicate at the block level: Deduplication and Compression.

In the most recent Red Hat Enterprise Linux 7.5 release, Red Hat announced integration of Red Hat VDO, or Virtual Data Optimizer. VDO is used for deduplication and compression of Linux environments. Though it can be paired with other replication technologies, it can only be fully leveraged when the replication sits underneath the VDO device. Why? You want to deduplicate and compress your data before replicating it for efficiency gains.

Effective transfer times

According to Louis Imershein, Red Hat’s Principal Product Manager for data reduction technologies, “Solutions like LINBIT’s DRBD are able to capture data below the VDO layer.  This means that datasets that benefit from deduplication and compression get replicated in their dehydrated form. With less data to move, Red Hat Enterprise Linux customers with LINBIT DRBD can benefit from faster effective transfer times and reduced bandwidth requirements.”

So, as you’re thinking about underlying storage for your applications, ensure you are using a solution which allows you to maximize the benefit of the existing Linux utilities built in, and around, your Operating System. Thanks to Red Hat, block level replication is now more important than ever.

 

Greg Eckert on Linkedin
Greg Eckert
In his role as the Director of Business Development for LINBIT America and Australia, Greg is responsible for building international relations, both in terms of technology and business collaboration. Since 2013, Greg has connected potential technology partners, collaborated with businesses in new territories, and explored opportunities for new joint ventures.
2 replies
  1. Strahil Nikolov
    Strahil Nikolov says:

    If you want to reduce yoir bandwidth – you need to use VDO in async mode which might cause data loss, as it cheats the FileSystem that the information is actually on the HDD’s platters , while it’s still in memory.
    The safe approach will be to ise the sync method, but it will lead to higher bandwith requirements ,as first VDO writes the data (and guarantees the data won’t get lost) , then DRBD will replicate, and then VDO will deduplicate or compress – which will again be replicated.
    Benefits from VDO are saving storage and increased read speeds in specific scenarios (when deduplicated data is requested several times in short succession ) and with the ability to read from the other side of the DRBD – we can expect very high reads.

    Reply
  2. Dennis
    Dennis says:

    Efficient data storage is essential to running a business that stores a lot of information. The last thing I want is to have my customers wait even a few extra seconds to receive information, so I have invested plenty of capital on the best data storage. I always keep in mind that there is a product life cycle to every storage company, so keeping up to date on the latest and greatest options is essential. I’m looking forward to checking out LINBIT DRBD, and possibly switching to them in the near future.

    Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *