sysutils/moosefs3-chunkserver/moosefs-3.0.116

MooseFS - A Petabyte Distributed File System

MooseFS is a Petabyte Open Source Network Distributed File System. It is easy
to deploy and maintain, highly reliable, fault tolerant, highly performing,
easily scalable and POSIX compliant.

MooseFS spreads data over a number of commodity servers, which are visible
to the user as one resource. For standard file operations MooseFS acts
like ordinary Unix-like file system:

  * A hierarchical structure – directory tree
  * Stores POSIX file attributes – permissions, last access and modification
    times, etc.
  * Supports ACLs
  * Supports POSIX and BSD file locks – including support for distributed
    file locking
  * Supports special files – block and character devices, pipes and sockets
  * Supports symbolic links – file names pointing to target files,
    not necessarily on MooseFS
  * Supports hard links – different names of files which refer to the same
    data on MooseFS


Distinctive MooseFS features:
=============================

  * High reliability – files are stored in several copies on separate servers.
    The number of copies is a configurable parameter, even per each file
  * No Single Point of Failure – all hardware and software components may be
    redundant
  * Parallel data operations – many clients can access many files concurrently
  * Capacity can be dynamically expanded by simply adding new servers/disks
    on the fly
  * Retired hardware may be removed on the fly
  * Deleted files are retained for a configurable period of time (a file system
    level "trash bin")
  * Coherent, "atomic" snapshots of files, even while the files are being
    written/accessed
  * Access to the file system can be limited based on IP address and/or password
    (similarly as in NFS)
  * Data tiering – supports different storage policies for different
    files/directories in Storage Classes mechanism
  * Per-directory, "project" quotas – configurable per RAW space, usable space,
    number of inodes with hard and soft quotas support
  * Apart from file system storage, MooseFS also provides block storage (mfsbdev)
  * Efficient, pure C implementation
  * Ethernet support


Supported platforms
===================

MooseFS can be installed on any POSIX compliant operating system including
various Linux distributions, FreeBSD and macOS:

  * Ubuntu
  * Debian
  * RHEL / CentOS
  * OpenSUSE
  * FreeBSD
  * macOS
  * Raspbian – Raspberry Pi 3

MooseFS Linux Client uses FUSE (https://github.com/libfuse/libfuse).
MooseFS macOS Client uses FUSE for macOS (https://github.com/osxfuse/osxfuse).

There is a separate MooseFS Client for Microsoft Windows available,
built on top of Dokany (https://github.com/dokan-dev/dokany).


Getting started
===============

You can install MooseFS using your favorite package manager on one
of the following platforms using officially supported repositories
(https://moosefs.com/download):

  * Ubuntu 16 / 18 / 20
  * Debian 8 / 9 / 10
  * RHEL / CentOS 7 / 8
  * FreeBSD 11 / 12
  * macOS 10.11+
  * Raspbian 8 / 9 – Raspberry Pi 3

Packages for Ubuntu 14 and CentOS 6 are also available, but no longer supported.

Minimal set of packages, which are needed to run MooseFS:
  * moosefs-master – MooseFS Master Server for metadata servers,
  * moosefs-chunkserver – MooseFS Chunkserver for data storage servers,
  * moosefs-client – MooseFS Client – client side package to mount
    the filesystem.


Source code
-----------

Feel free to download the source code from our GitHub code repository!

Install the following dependencies before building MooseFS from sources:
  * Debian/Ubuntu: sudo apt install build-essential libpcap-dev zlib1g-dev libfuse3-dev pkg-config
  (if you don't have FUSE v. 3 in your system, use:
  sudo apt install build-essential libpcap-dev zlib1g-dev libfuse-dev pkg-config)
  * CentOS/RHEL: sudo yum install gcc gcc-c++ make libpcap-devel zlib-devel fuse3-devel pkgconfig
  (if you don't have FUSE v. 3 in your system, use:
  sudo yum install gcc make libpcap-devel zlib-devel fuse-devel pkgconfig)

Recommended packages:
  * Debian/Ubuntu: sudo apt install fuse3
  (if you don't have FUSE v. 3 in your system, use: sudo apt install fuse)
  * CentOS/RHEL: sudo yum install fuse3
  (if you don't have FUSE v. 3 in your system, use: sudo yum install fuse)

Building MooseFS on Linux can be easily done by running ./linux_build.sh.
Similarly, use ./freebsd_build.sh in order to build MooseFS on FreeBSD
and respectively ./macosx_build.sh on macOS. Remember that these scripts
do not install binaries (i.e. do not run make install) at the end. Run this
command manually.

Minimal setup
-------------

Just three steps to have MooseFS up and running:

1. Install at least one Master Server

   1. Install moosefs-master package
   2. Prepare default config (as root):
        cd /etc/mfs
        cp mfsmaster.cfg.sample mfsmaster.cfg
        cp mfsexports.cfg.sample mfsexports.cfg
   3. Prepare the metadata file (as root):
        cd /var/lib/mfs
        cp metadata.mfs.empty metadata.mfs
        chown mfs:mfs metadata.mfs
        rm metadata.mfs.empty
   4. Run Master Server (as root): mfsmaster start
   5. Make this machine visible under 'mfsmaster' name, e.g. by adding
      a DNS entry (recommended) or by adding it in /etc/hosts on all servers
      that run any of MooseFS components


2. Install at least two Chunkservers

   1. Install moosefs-chunkserver package
   2. Prepare default config (as root):
        cd /etc/mfs
        cp mfschunkserver.cfg.sample mfschunkserver.cfg
        cp mfshdd.cfg.sample mfshdd.cfg
   3. At the end of mfshdd.cfg file make one or more entries containing paths
      to HDDs / partitions designated for storing chunks, e.g.:
        /mnt/chunks1
        /mnt/chunks2
        /mnt/chunks3
      It is recommended to use XFS as an underlying filesystem
      for disks designated to store chunks.
   4. Change the ownership and permissions to mfs:mfs to above
      mentioned locations:
        chown mfs:mfs /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
        chmod 770 /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
   5. Start the Chunkserver: mfschunkserver start

   Repeat the steps above for second (third, ...) Chunkserver.
   More than two Chunkservers are strongly recommended.


3. Client side: mount MooseFS filesystem

   1. Install moosefs-client package
   2. Mount MooseFS (as root):
        mkdir /mnt/mfs
        mount -t moosefs mfsmaster: /mnt/mfs
      or:
        mfsmount -H mfsmaster /mnt/mfs
   3. You can also add an /etc/fstab entry to mount MooseFS during
      the system boot:
        mfsmaster: /mnt/mfs moosefs defaults,mfsdelayedinit 0 0

   There are more configuration parameters available but most
   of them may stay with defaults. We do our best to keep MooseFS easy
   to deploy and maintain.

   MooseFS, for testing purposes, can even be installed on a single machine!


Additional tools
----------------

Setting up moosefs-cli or moosefs-cgi with moosefs-cgiserv is also
recommended – it gives you a possibility to monitor the cluster online:
  1. Install moosefs-cli moosefs-cgi moosefs-cgiserv packages
     (they are typically set up on the Master Server)
  2. Run MooseFS CGI Server (as root): mfscgiserv start
  3. Open http://mfsmaster:9425 in your web browser

It is also strongly recommended to set up at least one Metalogger on a different
machine than Master Server (e.g. on one of Chunkservers). Metalogger constantly
synchronizes and backups the metadata:
  1. Install moosefs-metalogger package
  2. Prepare default config (as root):
       cd /etc/mfs
       cp mfsmetalogger.cfg.sample mfsmetalogger.cfg
  3. Run Metalogger (as root): mfsmetalogger start

Refer to installation guides (https://moosefs.com/support/#documentation)
for more details.


Some facts
==========

  * Date of the first public release: 2008-05-30
  * The project web site: https://moosefs.com
  * Installation and using MooseFS: https://moosefs.com/support
  * (Old) Sourceforge project site: http://sourceforge.net/projects/moosefs


Contact us
==========

  * Reporting bugs: GitHub issue (https://github.com/moosefs/moosefs/issues)
    or support@moosefs.pro
  * General: contact@moosefs.pro


Copyright
=========

Copyright (c) 2008-2021 Jakub Kruszona-Zawadzki, Core Technology Sp. z o.o.

This file is part of MooseFS.

MooseFS is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, version 2 (only).

MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
MooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St,
Fifth Floor, Boston, MA 02111-1301, USA or visit
http://www.gnu.org/licenses/gpl-2.0.html.
Name		Date	Size	#Lines	LOC
..		03-May-2022	-
debian/	H	10-Aug-2021	-	3,691	2,697
freebsd/	H	10-Aug-2021	-	724	473
m4/	H	10-Aug-2021	-	9,393	8,488
macosx/	H	10-Aug-2021	-	36	27
mfschunkserver/	H	10-Aug-2021	-	15,729	13,484
mfsclient/	H	10-Aug-2021	-	37,306	31,815
mfscommon/	H	10-Aug-2021	-	14,459	10,658
mfsdata/	H	10-Aug-2021	-	1,168	967
mfsmanpages/	H	10-Aug-2021	-	4,738	4,461
mfsmaster/	H	10-Aug-2021	-	43,526	38,129
mfsmetalogger/	H	10-Aug-2021	-	2,160	1,877
mfsmetatools/	H	10-Aug-2021	-	3,290	3,008
mfsnetdump/	H	10-Aug-2021	-	1,701	1,524
mfsscripts/	H	03-May-2022	-	9,789	9,221
mfssupervisor/	H	10-Aug-2021	-	486	415
mfstests/	H	10-Aug-2021	-	1,969	1,608
rpm/	H	10-Aug-2021	-	913	663
systemd/	H	10-Aug-2021	-	702	611
COPYING	H A D	10-Aug-2021	14.6 KiB	98	49
INSTALL	H A D	10-Aug-2021	9.2 KiB	235	177
Makefile.am	H A D	10-Aug-2021	2.1 KiB	84	68
Makefile.in	H A D	10-Aug-2021	28.6 KiB	908	807
NEWS	H A D	10-Aug-2021	79.2 KiB	1,764	1,275
README	H A D	10-Aug-2021	8.8 KiB	244	188
README.md	H A D	10-Aug-2021	8.7 KiB	190	145
aclocal.m4	H A D	10-Aug-2021	53.1 KiB	1,479	1,340
ar-lib	H A D	10-Aug-2021	5.7 KiB	271	210
compile	H A D	10-Aug-2021	7.2 KiB	349	259
config.guess	H A D	10-Aug-2021	41.9 KiB	1,421	1,229
config.h.in	H A D	10-Aug-2021	16.3 KiB	614	424
config.sub	H A D	10-Aug-2021	34.8 KiB	1,791	1,653
configure	H A D	03-May-2022	580.9 KiB	20,360	17,127
configure.ac	H A D	10-Aug-2021	26.8 KiB	771	682
debian_sysv_to_systemd.sh	H A D	10-Aug-2021	729	18	11
depcomp	H A D	10-Aug-2021	23 KiB	792	502
freebsd_build.sh	H A D	10-Aug-2021	188	5	2
install-sh	H A D	10-Aug-2021	12.9 KiB	508	340
linux_build.sh	H A D	10-Aug-2021	341	9	6
ltmain.sh	H A D	10-Aug-2021	316.6 KiB	11,150	7,980
macosx_build.sh	H A D	10-Aug-2021	84	5	2
missing	H A D	10-Aug-2021	6.7 KiB	216	143
sanitize_build.sh	H A D	10-Aug-2021	698	11	4
test-driver	H A D	10-Aug-2021	4.2 KiB	140	84