1MooseFS - A Petabyte Distributed File System
2
3MooseFS is a Petabyte Open Source Network Distributed File System. It is easy
4to deploy and maintain, highly reliable, fault tolerant, highly performing,
5easily scalable and POSIX compliant.
6
7MooseFS spreads data over a number of commodity servers, which are visible
8to the user as one resource. For standard file operations MooseFS acts
9like ordinary Unix-like file system:
10
11  * A hierarchical structure – directory tree
12  * Stores POSIX file attributes – permissions, last access and modification
13    times, etc.
14  * Supports ACLs
15  * Supports POSIX and BSD file locks – including support for distributed
16    file locking
17  * Supports special files – block and character devices, pipes and sockets
18  * Supports symbolic links – file names pointing to target files,
19    not necessarily on MooseFS
20  * Supports hard links – different names of files which refer to the same
21    data on MooseFS
22
23
24Distinctive MooseFS features:
25=============================
26
27  * High reliability – files are stored in several copies on separate servers.
28    The number of copies is a configurable parameter, even per each file
29  * No Single Point of Failure – all hardware and software components may be
30    redundant
31  * Parallel data operations – many clients can access many files concurrently
32  * Capacity can be dynamically expanded by simply adding new servers/disks
33    on the fly
34  * Retired hardware may be removed on the fly
35  * Deleted files are retained for a configurable period of time (a file system
36    level "trash bin")
37  * Coherent, "atomic" snapshots of files, even while the files are being
38    written/accessed
39  * Access to the file system can be limited based on IP address and/or password
40    (similarly as in NFS)
41  * Data tiering – supports different storage policies for different
42    files/directories in Storage Classes mechanism
43  * Per-directory, "project" quotas – configurable per RAW space, usable space,
44    number of inodes with hard and soft quotas support
45  * Apart from file system storage, MooseFS also provides block storage (mfsbdev)
46  * Efficient, pure C implementation
47  * Ethernet support
48
49
50Supported platforms
51===================
52
53MooseFS can be installed on any POSIX compliant operating system including
54various Linux distributions, FreeBSD and macOS:
55
56  * Ubuntu
57  * Debian
58  * RHEL / CentOS
59  * OpenSUSE
60  * FreeBSD
61  * macOS
62  * Raspbian – Raspberry Pi 3
63
64MooseFS Linux Client uses FUSE (https://github.com/libfuse/libfuse).
65MooseFS macOS Client uses FUSE for macOS (https://github.com/osxfuse/osxfuse).
66
67There is a separate MooseFS Client for Microsoft Windows available,
68built on top of Dokany (https://github.com/dokan-dev/dokany).
69
70
71Getting started
72===============
73
74You can install MooseFS using your favorite package manager on one
75of the following platforms using officially supported repositories
76(https://moosefs.com/download):
77
78  * Ubuntu 16 / 18 / 20
79  * Debian 8 / 9 / 10
80  * RHEL / CentOS 7 / 8
81  * FreeBSD 11 / 12
82  * macOS 10.11+
83  * Raspbian 8 / 9 – Raspberry Pi 3
84
85Packages for Ubuntu 14 and CentOS 6 are also available, but no longer supported.
86
87Minimal set of packages, which are needed to run MooseFS:
88  * moosefs-master – MooseFS Master Server for metadata servers,
89  * moosefs-chunkserver – MooseFS Chunkserver for data storage servers,
90  * moosefs-client – MooseFS Client – client side package to mount
91    the filesystem.
92
93
94Source code
95-----------
96
97Feel free to download the source code from our GitHub code repository!
98
99Install the following dependencies before building MooseFS from sources:
100  * Debian/Ubuntu: sudo apt install build-essential libpcap-dev zlib1g-dev libfuse3-dev pkg-config
101  (if you don't have FUSE v. 3 in your system, use:
102  sudo apt install build-essential libpcap-dev zlib1g-dev libfuse-dev pkg-config)
103  * CentOS/RHEL: sudo yum install gcc gcc-c++ make libpcap-devel zlib-devel fuse3-devel pkgconfig
104  (if you don't have FUSE v. 3 in your system, use:
105  sudo yum install gcc make libpcap-devel zlib-devel fuse-devel pkgconfig)
106
107Recommended packages:
108  * Debian/Ubuntu: sudo apt install fuse3
109  (if you don't have FUSE v. 3 in your system, use: sudo apt install fuse)
110  * CentOS/RHEL: sudo yum install fuse3
111  (if you don't have FUSE v. 3 in your system, use: sudo yum install fuse)
112
113Building MooseFS on Linux can be easily done by running ./linux_build.sh.
114Similarly, use ./freebsd_build.sh in order to build MooseFS on FreeBSD
115and respectively ./macosx_build.sh on macOS. Remember that these scripts
116do not install binaries (i.e. do not run make install) at the end. Run this
117command manually.
118
119Minimal setup
120-------------
121
122Just three steps to have MooseFS up and running:
123
1241. Install at least one Master Server
125
126   1. Install moosefs-master package
127   2. Prepare default config (as root):
128        cd /etc/mfs
129        cp mfsmaster.cfg.sample mfsmaster.cfg
130        cp mfsexports.cfg.sample mfsexports.cfg
131   3. Prepare the metadata file (as root):
132        cd /var/lib/mfs
133        cp metadata.mfs.empty metadata.mfs
134        chown mfs:mfs metadata.mfs
135        rm metadata.mfs.empty
136   4. Run Master Server (as root): mfsmaster start
137   5. Make this machine visible under 'mfsmaster' name, e.g. by adding
138      a DNS entry (recommended) or by adding it in /etc/hosts on all servers
139      that run any of MooseFS components
140
141
1422. Install at least two Chunkservers
143
144   1. Install moosefs-chunkserver package
145   2. Prepare default config (as root):
146        cd /etc/mfs
147        cp mfschunkserver.cfg.sample mfschunkserver.cfg
148        cp mfshdd.cfg.sample mfshdd.cfg
149   3. At the end of mfshdd.cfg file make one or more entries containing paths
150      to HDDs / partitions designated for storing chunks, e.g.:
151        /mnt/chunks1
152        /mnt/chunks2
153        /mnt/chunks3
154      It is recommended to use XFS as an underlying filesystem
155      for disks designated to store chunks.
156   4. Change the ownership and permissions to mfs:mfs to above
157      mentioned locations:
158        chown mfs:mfs /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
159        chmod 770 /mnt/chunks1 /mnt/chunks2 /mnt/chunks3
160   5. Start the Chunkserver: mfschunkserver start
161
162   Repeat the steps above for second (third, ...) Chunkserver.
163   More than two Chunkservers are strongly recommended.
164
165
1663. Client side: mount MooseFS filesystem
167
168   1. Install moosefs-client package
169   2. Mount MooseFS (as root):
170        mkdir /mnt/mfs
171        mount -t moosefs mfsmaster: /mnt/mfs
172      or:
173        mfsmount -H mfsmaster /mnt/mfs
174   3. You can also add an /etc/fstab entry to mount MooseFS during
175      the system boot:
176        mfsmaster: /mnt/mfs moosefs defaults,mfsdelayedinit 0 0
177
178   There are more configuration parameters available but most
179   of them may stay with defaults. We do our best to keep MooseFS easy
180   to deploy and maintain.
181
182   MooseFS, for testing purposes, can even be installed on a single machine!
183
184
185Additional tools
186----------------
187
188Setting up moosefs-cli or moosefs-cgi with moosefs-cgiserv is also
189recommended – it gives you a possibility to monitor the cluster online:
190  1. Install moosefs-cli moosefs-cgi moosefs-cgiserv packages
191     (they are typically set up on the Master Server)
192  2. Run MooseFS CGI Server (as root): mfscgiserv start
193  3. Open http://mfsmaster:9425 in your web browser
194
195It is also strongly recommended to set up at least one Metalogger on a different
196machine than Master Server (e.g. on one of Chunkservers). Metalogger constantly
197synchronizes and backups the metadata:
198  1. Install moosefs-metalogger package
199  2. Prepare default config (as root):
200       cd /etc/mfs
201       cp mfsmetalogger.cfg.sample mfsmetalogger.cfg
202  3. Run Metalogger (as root): mfsmetalogger start
203
204Refer to installation guides (https://moosefs.com/support/#documentation)
205for more details.
206
207
208Some facts
209==========
210
211  * Date of the first public release: 2008-05-30
212  * The project web site: https://moosefs.com
213  * Installation and using MooseFS: https://moosefs.com/support
214  * (Old) Sourceforge project site: http://sourceforge.net/projects/moosefs
215
216
217Contact us
218==========
219
220  * Reporting bugs: GitHub issue (https://github.com/moosefs/moosefs/issues)
221    or support@moosefs.pro
222  * General: contact@moosefs.pro
223
224
225Copyright
226=========
227
228Copyright (c) 2008-2021 Jakub Kruszona-Zawadzki, Core Technology Sp. z o.o.
229
230This file is part of MooseFS.
231
232MooseFS is free software; you can redistribute it and/or modify it under
233the terms of the GNU General Public License as published by the Free Software
234Foundation, version 2 (only).
235
236MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY
237WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
238FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
239
240You should have received a copy of the GNU General Public License along with
241MooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St,
242Fifth Floor, Boston, MA 02111-1301, USA or visit
243http://www.gnu.org/licenses/gpl-2.0.html.
244