1<img align="right" alt="MooseFS logo" src="https://moosefs.com/Content/Images/moosefs.png" /> 2 3# MooseFS – A Petabyte Distributed File System 4MooseFS is a Petabyte Open Source Network Distributed File System. It is easy to deploy and maintain, highly reliable, fault tolerant, highly performing, easily scalable and POSIX compliant. 5 6MooseFS spreads data over a number of commodity servers, which are visible to the user as one resource. For standard file operations MooseFS acts like ordinary Unix-like file system: 7 8* A hierarchical structure – **directory tree** 9* Stores **POSIX file attributes** – permissions, last access and modification times, etc. 10* Supports **ACLs** 11* Supports POSIX and BSD **file locks** – including support for **distributed file locking** 12* Supports **special files** – block and character devices, pipes and sockets 13* Supports **symbolic links** – file names pointing to target files, not necessarily on MooseFS 14* Supports **hard links** – different names of files which refer to the same data on MooseFS 15 16Distinctive MooseFS features: 17 18* **High reliability** – files are stored in several copies on separate servers. The number of copies is a configurable parameter, even per each file 19* **No Single Point of Failure** – all hardware and software components may be redundant 20* **Parallel** data operations – many clients can access many files concurrently 21* Capacity can be **dynamically expanded** by simply adding new servers/disks on the fly 22* Retired hardware **may be removed on the fly** 23* Deleted files are retained for a configurable period of time (a **file system level "trash bin"**) 24* **Coherent, "atomic" snapshots** of files, even while the files are being written/accessed 25* **Access to the file system can be limited** based on IP address and/or password (similarly as in NFS) 26* **Data tiering** – supports different storage policies for different files/directories in Storage Classes mechanism 27* Per-directory, **"project" quotas** – configurable per RAW space, usable space, number of inodes with hard and soft quotas support 28* Apart from file system storage, MooseFS also provides **block storage** (`mfsbdev`) 29* Efficient, **pure C** implementation 30* **Ethernet** support 31 32## Supported platforms 33MooseFS can be installed on any POSIX compliant operating system including various Linux distributions, FreeBSD and macOS: 34 35* Ubuntu 36* Debian 37* RHEL / CentOS 38* OpenSUSE 39* FreeBSD 40* macOS 41* Raspbian – Raspberry Pi 3 42 43MooseFS Linux Client uses [FUSE](https://github.com/libfuse/libfuse). MooseFS macOS Client uses [FUSE for macOS](https://github.com/osxfuse/osxfuse). 44 45There is a separate MooseFS Client for Microsoft Windows available, built on top of [Dokany](https://github.com/dokan-dev/dokany). 46 47## Getting started 48You can install MooseFS using your favourite package manager on one of the following platforms using [officially supported repositories](https://moosefs.com/download): 49 50* Ubuntu 16 / 18 / 20 51* Debian 8 / 9 / 10 52* RHEL / CentOS 7 / 8 53* FreeBSD 11 / 12 54* macOS 10.11+ 55* Raspbian 8 / 9 – Raspberry Pi 3 56 57Packages for Ubuntu 14 and CentOS 6 are also available, but no longer supported. 58 59Minimal set of packages, which are needed to run MooseFS: 60 61* `moosefs-master` MooseFS Master Server for metadata servers, 62* `moosefs-chunkserver` MooseFS Chunkserver for data storage servers, 63* `moosefs-client` MooseFS Client – client side package to mount the filesystem. 64 65### Source code 66Feel free to download the source code from our GitHub code repository! 67 68Install the following dependencies before building MooseFS from sources: 69 70* Debian/Ubuntu: `sudo apt install build-essential libpcap-dev zlib1g-dev libfuse3-dev pkg-config` 71(if you don't have FUSE v. 3 in your system, use `sudo apt install build-essential libpcap-dev zlib1g-dev libfuse-dev pkg-config`) 72* CentOS/RHEL: `sudo yum install gcc make libpcap-devel zlib-devel fuse3-devel pkgconfig` 73(if you don't have FUSE v. 3 in your system, use `sudo yum install gcc make libpcap-devel zlib-devel fuse-devel pkgconfig`) 74 75Recommended packages: 76 77* Debian/Ubuntu: `sudo apt install fuse3` 78(if you don't have FUSE v. 3 in your system, use `sudo apt install fuse`) 79* CentOS/RHEL: `sudo yum install fuse3` 80(if you don't have FUSE v. 3 in your system, use `sudo yum install fuse`) 81 82Building MooseFS on Linux can be easily done by running `./linux_build.sh`. Similarly, use `./freebsd_build.sh` in order to build MooseFS on FreeBSD and respectively `./macosx_build.sh` on macOS. Remember that these scripts do not install binaries (i.e. do not run `make install`) at the end. Run this command manually. 83 84### Minimal setup 85Just three steps to have MooseFS up and running: 86 87#### 1. Install at least one Master Server 881. Install `moosefs-master` package 892. Prepare default config (as `root`): 90``` 91cd /etc/mfs 92cp mfsmaster.cfg.sample mfsmaster.cfg 93cp mfsexports.cfg.sample mfsexports.cfg 94``` 953. Prepare the metadata file (as `root`): 96``` 97cd /var/lib/mfs 98cp metadata.mfs.empty metadata.mfs 99chown mfs:mfs metadata.mfs 100rm metadata.mfs.empty 101``` 1024. Run Master Server (as `root`): `mfsmaster start` 1035. Make this machine visible under `mfsmaster` name, e.g. by adding a DNS entry (recommended) or by adding it in `/etc/hosts` on **all** servers that run any of MooseFS components. 104 105#### 2. Install at least two Chunkservers 1061. Install `moosefs-chunkserver` package 1072. Prepare default config (as `root`): 108``` 109cd /etc/mfs 110cp mfschunkserver.cfg.sample mfschunkserver.cfg 111cp mfshdd.cfg.sample mfshdd.cfg 112``` 1133. At the end of `mfshdd.cfg` file make one or more entries containing paths to HDDs / partitions designated for storing chunks, e.g.: 114``` 115/mnt/chunks1 116/mnt/chunks2 117/mnt/chunks3 118``` 119It is recommended to use XFS as an underlying filesystem for disks designated to store chunks. More than two Chunkservers are **strongly** recommended. 120 1214. Change the ownership and permissions to `mfs:mfs` to above mentioned locations: 122``` 123chown mfs:mfs /mnt/chunks1 /mnt/chunks2 /mnt/chunks3 124chmod 770 /mnt/chunks1 /mnt/chunks2 /mnt/chunks3 125``` 1265. Start the Chunkserver: `mfschunkserver start` 127 128Repeat steps above for second (third, ...) Chunkserver. 129 130#### 3. Client side: mount MooseFS filesystem 1311. Install `moosefs-client` package 1322. Mount MooseFS (as `root`): 133``` 134mkdir /mnt/mfs 135mount -t moosefs mfsmaster: /mnt/mfs 136``` 137or: `mfsmount -H mfsmaster /mnt/mfs` if the above method is not supported by your system 138 1393. You can also add an `/etc/fstab` entry to mount MooseFS during the system boot: 140``` 141mfsmaster: /mnt/mfs moosefs defaults,mfsdelayedinit 0 0 142``` 143 144There are more configuration parameters available but most of them may stay with defaults. We do our best to keep MooseFS easy to deploy and maintain. 145 146MooseFS, for testing purposes, can even be installed on a single machine! 147 148#### Additional tools 149Setting up `moosefs-cli` or `moosefs-cgi` with `moosefs-cgiserv` is also recommended – it gives you a possibility to monitor the cluster online: 150 1511. Install `moosefs-cli moosefs-cgi moosefs-cgiserv` packages (they are typically set up on the Master Server) 1522. Run MooseFS CGI Server (as `root`): `mfscgiserv start` 1533. Open http://mfsmaster:9425 in your web browser 154 155It is also **strongly** recommended to set up at least one Metalogger on a different machine than Master Server (e.g. on one of Chunkservers). Metalogger constantly synchronizes and backups the metadata: 156 1571. Install `moosefs-metalogger` package 1582. Prepare default config (as `root`): 159``` 160cd /etc/mfs 161cp mfsmetalogger.cfg.sample mfsmetalogger.cfg 162``` 1633. Run Metalogger (as `root`): `mfsmetalogger start` 164 165 166 167Refer to [installation guides](https://moosefs.com/support/#documentation) for more details. 168 169## Some facts 170* Date of the first public release: 2008-05-30 171* The project web site: https://moosefs.com 172* Installation and using MooseFS: https://moosefs.com/support 173* (Old) Sourceforge project site: http://sourceforge.net/projects/moosefs 174 175## Contact us 176* Reporting bugs: [GitHub issue](https://github.com/moosefs/moosefs/issues) or [support@moosefs.pro](mailto:support@moosefs.pro) 177* General: [contact@moosefs.pro](mailto:contact@moosefs.pro) 178 179 180## Copyright 181Copyright (c) 2008-2021 Jakub Kruszona-Zawadzki, Core Technology Sp. z o.o. 182 183This file is part of MooseFS. 184 185MooseFS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2 (only). 186 187MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. 188 189You should have received a copy of the GNU General Public License along with MooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02111-1301, USA or visit http://www.gnu.org/licenses/gpl-2.0.html. 190
