Posts

Piraeus datastore - port

LINBIT announces Piraeus Datastore – Software-Defined Storage (SDS) for Kubernetes

Piraeus Datastore offers high performance, highly reliable SDS solution for Persistent Volumes in Kubernetes

 

San Diego, November 18., 2019 – LINBIT, the inventor of the open-source software DRBD™ and leader in Linux storage software, has announced the project “Piraeus Datastore”. Piraeus Datastore offers a fast, stable way for users to provide Persistent Volumes for their Kubernetes applications. It is developed under the open-source development model and has been made publicly available via pre-built packages and containers in a joint-effort between LINBIT, Daocloud and the open-source community. 

 

Serving the rapidly growing containerized applications market, Piraeus Datastore fills a significant gap in the market by providing a stable Software-Defined Storage solution for Kubernetes applications requiring high-performance block storage. “When it comes to storage, Piraeus helps clients achieve better system availability than competitors in the space can offer ”, says  Philipp Reisner, CEO of LINBIT. “Most Kubernetes storage newcomers combined the data and control plane to push out a minimum viable product. By separating these components out, we ensure that controller failure doesn’t impact storage system availability.” 

 

DaoCloud has been a leading CloudNative computing vendor in China since 2014. They have extensive experience of Kubernetes production with Fortune Global 500 customers in manufacturing and finance industries like SAIC, Haier and SPDB bank. They believe that Piraeus fits their clients’ needs for a cloud-native, container attached storage that provides both reliability and performance. Roby Chen, CEO of DaoCloud, says: “It is very exciting that we have the Piraeus project to elevate DRBD technology into the cloud-native arena, where we believe it can play a key role.” 

Kubernetes is generating more and more buzz

Piraeus Datastore provides data persistence for elastic applications, which dynamically create or remove containers depending on the load. A recent survey of 390 IT professionals showed that 51% of participants acknowledge there has been an increase in Kubernetes adoption in the last six months. And 86% of respondents say they have now adopted Kubernetes. The number went up from 57% a half a year ago. Storage remains a requirement for enterprises that are beginning to move their applications over to Kubernetes and the Piraeus datastore solution is perfect for clients who need the combination of reliability and performance.

 

With Piraeus Datastore, two key Open Source technologies are packaged in a way that is easily accessible and consumable for Kubernetes users. They bring the highest performance by leveraging on the proven DRBD technology and real-world operable clusters by the way LINSTOR™ separates control and data paths.

Piraeus Datastore makes proven technologies cloud-native. DRBD is under development, improvements and optimizations for 19 Years. LINSTOR stands out in the field of SDS systems by having a separate and independent control plane that is independent from the data plane. This big advantage of this separation is it makes upgrades in a running storage cluster feasible, which saves downtime and therefore a lot of money. 

Piraeus Datastore is perfectly suited for databases, AI and analytics workloads, where the throughput and latency of primary storage is required.

 

LINBIT SDS™ is the industry’s fastest software-defined storage solution for enterprise, cloud, and container environments. LINBIT SDS leverages the DRBD™ and LINSTOR™ technology to provision, replicate, and manage data storage: independent of underlying hardware. The Piraeus containers intended for adoption by the community, while the LINBIT SDS containers are intended for consumption by corporate users. The LINBIT SDS product comes with enterprise support options, while Piraeus is supported by the community.

 

For a close comparison check out the following graphic:

 

Piraeus_logoPiraeus Datastore linbit_sdsLINBIT SDS
Container base image Debian_logodebian UBI
Pre-built available Publicly on Dockerhub, Quay For LINBIT customers drbd.io
Support community only ✅ enterprise, incl 24/7
Runs with OpenShift  without DRBD
OpenShift certified n.a.  
Kernel-module compile from source compile & pre-compiled
Contains DRBD logo, piraeus-operator, linstor-csi and

Linstor_logo

Licensing Open source software, GPL & Apache
Developed and verified for kubernetes container orchestration and redhat primed openshift
Platforms on the roadmap ibm cloud containerSuse Caas

 

Learn more:

 

About Linbit

 

LINBIT is the force behind DRBD and a leader in open-source Linux block storage software for enterprise and cloud computing. The LINBIT software has helped dozens of global companies such as Volkswagen, Intel, Cisco, Siemens, BBC to provide High Availability (HA), Geo Clustering for Disaster Recovery (DR), and Software-Defined Storage (SDS) for public and private clouds. Based in Vienna LINBIT partners with other companies like Redhat, Intel, IBM or DaoCloud to accelerate Linux storage software. For more information, visit linbit.com or follow @linbit

 

Social Media Channels:

 

LINBIT on Twitter

LINBIT on Linkedin

LINBIT on Youtube

LINBIT on Facebook

 

About DaoCloud

 

DaoCloud is a leading CloudNative computing vendor in China since 2014. They have extensive experience of Kubernetes production with Fortune Global 500 customers in manufacturing and finance industries like SAIC, Haier and SPDB bank. 

 

PR contact:

 

Sebastian Schinhammer

Marketing Manager

sebastian.schinhammer(@)linbit.com

Phone: 0043 1 817 82 92 -64

LINBIT HA-Solutions GmbH

Vivenotgasse 48

1120 Wien

 

Container-kubernetes-linsto

Containerize LINSTOR

LINBIT and its Software-Defined Storage (SDS) solution LINSTOR has provided integration with Linux containers for quite some time. These range from a Docker volume plugin, to a Flexvolume plugin, and recently, a CSI plugin for Kubernetes. While we always provided excellent integration to the container world, most of our software itself was not available as a container/base image. Containerizing our services is a non-trivial task. As you probably know, the core of the DRBD software consists of a Linux kernel module and user space utilities that interact via netlink with this kernel module. Additionally, our software needs to create LVM devices and DRBD block devices within a container. These tasks are interesting and challenging to put into containers. For this article, we assume 3 nodes, one node that acts as a LINSTOR controller, and two that act as satellites. We tested this with recent Centos7 machines and with a current version of Docker.

Prerequisites

In this article, we assume access to our Docker registry hosted on drbd.io. On all hosts you should run the following commands:

docker login drbd.io
Username: YourUserName
Password: YourPassword
Login Succeeded

Installing the DRBD kernel modules

We need the DRBD kernel module and its dependencies on the LINSTOR satellites (the controller does not need access to DRBD). For that we provide a solution for the most common platforms, namely Centos7/RHEL7 and Ubuntu Bionic.

docker run --privileged -it --rm \
  -v /lib/modules:/lib/modules drbd.io/drbd9:rhel7
DRBD modul sucessfully loaded 

What this does is check which kernel is actually executed on the host, then found it the most appropriate package in the container and installed it. We ship the same, unmodified rpm/deb packages in the container as we provide in our customer repositories. If you are using Ubuntu Bionic, you should use the drbd.io/drbd9:bionic container.

Running a LINSTOR controller

docker run -d --name=linstor-controller \
  -p 3376:3376 -p 3377:3377 drbd.io/linstor-controller

The controller does not have any special requirements, it just needs to be accessible to the client via TCP/IP. Please note that in this configuration the controller’s database is not persisted. One possibility is to bind-mount the directory used for the controller’s database by adding
-v /some/dir:/var/lib/linstor .

Running a LINSTOR satellite

docker run -d --name=linstor-satellite --net=host \
 --privileged drbd.io/linstor-satellite 

The satellite is the component that creates actual block devices. On one hand the backing devices (usually LVM) and the actual DRBD block devices. Therefore this container needs access to/dev, and it needs to share the host networking. Host networking is required for the communication between drbdsetup and the actual kernel module.

Configuring the Cluster

We have to set up LINSTOR as usual, which fortunately, is an easy task and has to be done only once. In the spirit of this blog post, let’s use a containerized LINSTOR client as well. As the client obviously has to talk to the controller, we need to tell the client in the container where to find the controller. This is done by setting the environment variable LS_CONTROLLERS.

docker run -it --rm -e LS_CONTROLLERS=Controller \ 
  drbd.io/linstor-client interactive
  ...
- volume-definition (vd)
LINSTOR ==> node create Satellite1 172.42.42.10
LINSTOR ==> node create Satellite2 172.42.42.20
LINSTOR ==> storage-pool-definition create drbdpool
LINSTOR ==> storage-pool create lvm Satellite1 drbdpool drbdpool
LINSTOR ==> storage-pool create lvm Satellite2 drbdpool drbdpool 

Creating a replicated DRBD resource

So far we loaded the kernel module on the satellites, started the controller and satellite containers and configured the LINSTOR cluster. Now it is time to actually create resources.

docker run -it --rm -e LS_CONTROLLERS=Controller \ 
  drbd.io/linstor-client interactive
  ... 
- volume-definition (vd)
 LINSTOR ==> resource-definition create demo
 LINSTOR ==> volume-definition create demo 1G
 LINSTOR ==> resource create demo --storage-pool drbdpool --auto-place 2 

If you have drbd-utils installed on the host, you can now see the DRBD resource as usual viadrbdsetup status. But we can also use a container to do that. On one of the satellites you can run a throw-away linstor-satellite container which contains drbd-utils:

docker run -it --rm --net=host --privileged \
 --entrypoint=/bin/bash drbd.io/linstor-satellite
$ drbdsetup status
$ lvs

Note that by default you will not see the symbolic links for the backing devices created by LVM/udev in the LINSTOR satellite container. That is expected. In the container you will see something like /dev/drbdpool/demo_00000, while on the host you will only see/dev/dm-X, and  lvs will not show the LVs. If you really want to see the LVs on the host, you could execute  lvscan -a --cache, but there is no actual reason for that. One might also map the lvmetad socket to the container.

Summary

As you can see, LINBIT’s container story is now complete. It is now possible to deploy the whole stack via containers. This ranges from the lowest level of providing the kernel modules to the highest level of LINSTOR SDS including the client, the controller, and satellites.

Roland Kammerer
Software Engineer at Linbit
Roland Kammerer studied technical computer science at the Vienna University of Technology and graduated with distinction. Currently, he is a PhD candidate with a research focus on time-triggered realtime-systems and works for LINBIT in the DRBD development team.