Searching a cluster filesystem (file storage) that is up-to-date and runs on both FreeBSD and Linux well

Question

I have a FreeBSD 12.1-RELEASE server and a CentOS 7 server. Both run on amd64.

I would like to set up a cluster file system, that runs on both platforms well. It should have CentOS 7 packages and FreeBSD packages. The solutions should be open-source software and "free of use".

After a little research, I found the following, but nontheless I always encountered drawbacks:

• MooseFS3: Works on FreeBSD and CentOS, has packages for both, but only the MooseFS3 Pro version, which is commercial, has the functionality of real cluster functionality such as the possibility of mounting the file system from several nodes. Also I had locking problems with files that where access by my dovecot imap server daemon, when I run dovecot from the file system.

• GlusterFS: Seems to work well, but there are no packages for the most current version of 8.x for FreeBSD. FreeBSD provides only a port for GlusterFS 3.x as of today. Different versions of GlusterFS can not operate together.

• Ceph: Is very complex to configure, and I couldn't execute all of the steps of the official FreeBSD documentation for it, since the tool ceph-disk is deprecated in favor of ceph-volume. With Ceph-volume, though, I could not get it running with my zfs pool on FreeBSD, since the plugin for zfs for ceph-volume seemed to have some Linux code in it when it was ported to FreeBSD or similiar, so it might only run with ZFSOnLinux on Linux itsself.

• OCFS2: I don't have much experience with that one, but its releases seem a bit outdated.

• Lustre: No packages for FreeBSD and no acurate and up-to-date documentation how to set it up on a recent FreeBSD system

• BeeGFS (Fraunhofer): No packages for FreeBSD, only for Linux

• Hadoop MapR filesystem: Has a use case more for BigData storage than for a UNIX cluster filesystem, I don't know if it has FreeBSD packages.

So I don't find a good solution for a Cluster filesystem that runs on both FreeBSD and CentOS Linux. Even I'm planning to migrate the CentOS server to Fedora Server, so it should run there as well.

Anyone who can recommend me a recent compatible cluster file system that I could use on both FreeBSD and CentOS/Fedora Server and that allows real cluster file system features like replication and HA?

Or is there currently no cluster filesystem that fulfills my needs and I have to migrate the two machines running the same OS?

Thank you in advance.

Best regards, rforberger

Answer 1

MooseFS3: Works on FreeBSD and CentOS, has packages for both, but only the MooseFS3 Pro version, which is commercial, has the functionality of real cluster functionality such as the possibility of mounting the file system from several nodes.

This is not true, you can mount MooseFS Community from as many nodes as you wish.

Answer 2

Glusterfs may be worth to try, it is based on fuse, which is available on FreeBSD, so you need only to build the userspace part, which may not be impossible, if it is not available for you OS version. On Linux it is definitely the simplest one to set up, since it comes packaged with most of the distros.

Lustre, despite supporting replicated directories, is more of a parallel filesystem oriented to HPC and high I/O performances, than a clustered filesystem oriented at redundancy, so I would not even consider it if redundancy is your purpose.

I have no experience with the other ones.