README.md
1# SFTPGo
2
3
4[](https://codecov.io/gh/drakkan/sftpgo/branch/main)
5[](https://www.gnu.org/licenses/agpl-3.0)
6[](https://hub.docker.com/r/drakkan/sftpgo)
7[](https://github.com/avelino/awesome-go)
8
9Fully featured and highly configurable SFTP server with optional HTTP, FTP/S and WebDAV support, written in Go.
10Several storage backends are supported: local filesystem, encrypted local filesystem, S3 (compatible) Object Storage, Google Cloud Storage, Azure Blob Storage, SFTP.
11
12## Features
13
14- Support for serving local filesystem, encrypted local filesystem, S3 Compatible Object Storage, Google Cloud Storage, Azure Blob Storage or other SFTP accounts over SFTP/SCP/FTP/WebDAV.
15- Virtual folders are supported: a virtual folder can use any of the supported storage backends. So you can have, for example, an S3 user that exposes a GCS bucket (or part of it) on a specified path and an encrypted local filesystem on another one. Virtual folders can be private or shared among multiple users, for shared virtual folders you can define different quota limits for each user.
16- Configurable [custom commands and/or HTTP hooks](./docs/custom-actions.md) on file upload, pre-upload, download, pre-download, delete, pre-delete, rename, mmkdir, rmdir on SSH commands and on user add, update and delete.
17- Virtual accounts stored within a "data provider".
18- SQLite, MySQL, PostgreSQL, CockroachDB, Bolt (key/value store in pure Go) and in-memory data providers are supported.
19- Chroot isolation for local accounts. Cloud-based accounts can be restricted to a certain base path.
20- Per user and per directory virtual permissions, for each exposed path you can allow or deny: directory listing, upload, overwrite, download, delete, rename, create directories, create symlinks, change owner/group/file mode.
21- [REST API](./docs/rest-api.md) for users and folders management, data retention, backup, restore and real time reports of the active connections with possibility of forcibly closing a connection.
22- [Web based administration interface](./docs/web-admin.md) to easily manage users, folders and connections.
23- [Web client interface](./docs/web-client.md) so that end users can change their credentials, manage and share their files.
24- Public key and password authentication. Multiple public keys per user are supported.
25- SSH user [certificate authentication](https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.certkeys?rev=1.8).
26- Keyboard interactive authentication. You can easily setup a customizable multi-factor authentication.
27- Partial authentication. You can configure multi-step authentication requiring, for example, the user password after successful public key authentication.
28- Per user authentication methods.
29- Two-factor authentication based on time-based one time passwords (RFC 6238) which works with Authy, Google Authenticator and other compatible apps.
30- Custom authentication via external programs/HTTP API.
31- [Data At Rest Encryption](./docs/dare.md).
32- Dynamic user modification before login via external programs/HTTP API.
33- Quota support: accounts can have individual quota expressed as max total size and/or max number of files.
34- Bandwidth throttling, with distinct settings for upload and download.
35- Per-protocol [rate limiting](./docs/rate-limiting.md) is supported and can be optionally connected to the built-in defender to automatically block hosts that repeatedly exceed the configured limit.
36- Per user maximum concurrent sessions.
37- Per user and global IP filters: login can be restricted to specific ranges of IP addresses or to a specific IP address.
38- Per user and per directory shell like patterns filters: files can be allowed or denied based on shell like patterns.
39- Automatically terminating idle connections.
40- Automatic blocklist management using the built-in [defender](./docs/defender.md).
41- Atomic uploads are configurable.
42- Per user files/folders ownership mapping: you can map all the users to the system account that runs SFTPGo (all platforms are supported) or you can run SFTPGo as root user and map each user or group of users to a different system account (\*NIX only).
43- Support for Git repositories over SSH.
44- SCP and rsync are supported.
45- FTP/S is supported. You can configure the FTP service to require TLS for both control and data connections.
46- [WebDAV](./docs/webdav.md) is supported.
47- Two-Way TLS authentication, aka TLS with client certificate authentication, is supported for REST API/Web Admin, FTPS and WebDAV over HTTPS.
48- Per user protocols restrictions. You can configure the allowed protocols (SSH/FTP/WebDAV) for each user.
49- [Prometheus metrics](./docs/metrics.md) are exposed.
50- Support for HAProxy PROXY protocol: you can proxy and/or load balance the SFTP/SCP/FTP/WebDAV service without losing the information about the client's address.
51- Easy [migration](./examples/convertusers) from Linux system user accounts.
52- [Portable mode](./docs/portable-mode.md): a convenient way to share a single directory on demand.
53- [SFTP subsystem mode](./docs/sftp-subsystem.md): you can use SFTPGo as OpenSSH's SFTP subsystem.
54- Performance analysis using built-in [profiler](./docs/profiling.md).
55- Configuration format is at your choice: JSON, TOML, YAML, HCL, envfile are supported.
56- Log files are accurate and they are saved in the easily parsable JSON format ([more information](./docs/logs.md)).
57- SFTPGo supports a [plugin system](./docs/plugins.md) and therefore can be extended using external plugins.
58
59## Platforms
60
61SFTPGo is developed and tested on Linux. After each commit, the code is automatically built and tested on Linux, macOS and Windows using a [GitHub Action](./.github/workflows/development.yml). The test cases are regularly manually executed and passed on FreeBSD. Other *BSD variants should work too.
62
63## Requirements
64
65- Go as build only dependency. We support the Go version(s) used in [continuous integration workflows](./tree/main/.github/workflows).
66- A suitable SQL server to use as data provider: PostgreSQL 9.4+ or MySQL 5.6+ or SQLite 3.x or CockroachDB stable.
67- The SQL server is optional: you can choose to use an embedded bolt database as key/value store or an in memory data provider.
68
69## Installation
70
71Binary releases for Linux, macOS, and Windows are available. Please visit the [releases](https://github.com/drakkan/sftpgo/releases "releases") page.
72
73An official Docker image is available. Documentation is [here](./docker/README.md).
74
75Some Linux distro packages are available:
76
77- For Arch Linux via AUR:
78 - [sftpgo](https://aur.archlinux.org/packages/sftpgo/). This package follows stable releases. It requires `git`, `gcc` and `go` to build.
79 - [sftpgo-bin](https://aur.archlinux.org/packages/sftpgo-bin/). This package follows stable releases downloading the prebuilt linux binary from GitHub. It does not require `git`, `gcc` and `go` to build.
80 - [sftpgo-git](https://aur.archlinux.org/packages/sftpgo-git/). This package builds and installs the latest git `main` branch. It requires `git`, `gcc` and `go` to build.
81- Deb and RPM packages are built after each commit and for each release.
82- For Ubuntu a PPA is available [here](https://launchpad.net/~sftpgo/+archive/ubuntu/sftpgo).
83
84SFTPGo is also available on [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=6e849ab8-70a6-47de-9a43-13c3fa849335), purchasing from there will help keep SFTPGo a long-term sustainable project.
85
86On FreeBSD you can install from the [SFTPGo port](https://www.freshports.org/ftp/sftpgo).
87
88On Windows you can use:
89
90- The Windows installer to install and run SFTPGo as a Windows service.
91- The portable package to start SFTPGo on demand.
92- The [Chocolatey package](https://community.chocolatey.org/packages/sftpgo) to install and run SFTPGo as a Windows service.
93
94You can easily test new features selecting a commit from the [Actions](https://github.com/drakkan/sftpgo/actions) page and downloading the matching build artifacts for Linux, macOS or Windows. GitHub stores artifacts for 90 days.
95
96Alternately, you can [build from source](./docs/build-from-source.md).
97
98[Getting Started Guide for the Impatient](./docs/howto/getting-started.md).
99
100## Configuration
101
102A full explanation of all configuration methods can be found [here](./docs/full-configuration.md).
103
104Please make sure to [initialize the data provider](#data-provider-initialization-and-management) before running the daemon.
105
106To start SFTPGo with the default settings, simply run:
107
108```bash
109sftpgo serve
110```
111
112Check out [this documentation](./docs/service.md) if you want to run SFTPGo as a service.
113
114### Data provider initialization and management
115
116Before starting the SFTPGo server please ensure that the configured data provider is properly initialized/updated.
117
118For PostgreSQL, MySQL and CockroachDB providers, you need to create the configured database. For SQLite, the configured database will be automatically created at startup. Memory and bolt data providers do not require an initialization but they could require an update to the existing data after upgrading SFTPGo.
119
120SFTPGo will attempt to automatically detect if the data provider is initialized/updated and if not, will attempt to initialize/ update it on startup as needed.
121
122Alternately, you can create/update the required data provider structures yourself using the `initprovider` command.
123
124For example, you can simply execute the following command from the configuration directory:
125
126```bash
127sftpgo initprovider
128```
129
130Take a look at the CLI usage to learn how to specify a different configuration file:
131
132```bash
133sftpgo initprovider --help
134```
135
136You can disable automatic data provider checks/updates at startup by setting the `update_mode` configuration key to `1`.
137
138You can also reset your provider by using the `resetprovider` sub-command. Take a look at the CLI usage for more details:
139
140```bash
141sftpgo resetprovider --help
142```
143
144## Create the first admin
145
146To start using SFTPGo you need to create an admin user, you can do it in several ways:
147
148- by using the web admin interface. The default URL is [http://127.0.0.1:8080/web/admin](http://127.0.0.1:8080/web/admin)
149- by loading initial data
150- by enabling `create_default_admin` in your configuration file and setting the environment variables `SFTPGO_DEFAULT_ADMIN_USERNAME` and `SFTPGO_DEFAULT_ADMIN_PASSWORD`
151
152## Upgrading
153
154SFTPGo supports upgrading from the previous release branch to the current one.
155Some examples for supported upgrade paths are:
156
157- from 1.2.x to 2.0.x
158- from 2.0.x to 2.1.x and so on.
159
160For supported upgrade paths, the data and schema are migrated automatically, alternately you can use the `initprovider` command.
161
162So if, for example, you want to upgrade from a version before 1.2.x to 2.0.x, you must first install version 1.2.x, update the data provider and finally install the version 2.0.x. It is recommended to always install the latest available minor version, ie do not install 1.2.0 if 1.2.2 is available.
163
164Loading data from a provider independent JSON dump is supported from the previous release branch to the current one too. After upgrading SFTPGo it is advisable to regenerate the JSON dump from the new version.
165
166## Downgrading
167
168If for some reason you want to downgrade SFTPGo, you may need to downgrade your data provider schema and data as well. You can use the `revertprovider` command for this task.
169
170As for upgrading, SFTPGo supports downgrading from the previous release branch to the current one.
171
172So, if you plan to downgrade from 2.0.x to 1.2.x, before uninstalling 2.0.x version, you can prepare your data provider executing the following command from the configuration directory:
173
174```shell
175sftpgo revertprovider --to-version 4
176```
177
178Take a look at the CLI usage to see the supported parameter for the `--to-version` argument and to learn how to specify a different configuration file:
179
180```shell
181sftpgo revertprovider --help
182```
183
184The `revertprovider` command is not supported for the memory provider.
185
186Please note that we only support the current release branch and the current main branch, if you find a bug it is better to report it rather than downgrading to an older unsupported version.
187
188## Users and folders management
189
190After starting SFTPGo you can manage users and folders using:
191
192- the [web based administration interface](./docs/web-admin.md)
193- the [REST API](./docs/rest-api.md)
194
195To support embedded data providers like `bolt` and `SQLite` we can't have a CLI that directly write users and folders to the data provider, we always have to use the REST API.
196
197Full details for users, folders, admins and other resources are documented in the [OpenAPI](/openapi/openapi.yaml) schema. If you want to render the schema without importing it manually, you can explore it on [Stoplight](https://sftpgo.stoplight.io/docs/sftpgo/openapi.yaml).
198
199## Tutorials
200
201Some step-to-step tutorials can be found inside the source tree [howto](./docs/howto "How-to") directory.
202
203## Authentication options
204
205### External Authentication
206
207Custom authentication methods can easily be added. SFTPGo supports external authentication modules, and writing a new backend can be as simple as a few lines of shell script. More information can be found [here](./docs/external-auth.md).
208
209### Keyboard Interactive Authentication
210
211Keyboard interactive authentication is, in general, a series of questions asked by the server with responses provided by the client.
212This authentication method is typically used for multi-factor authentication.
213
214More information can be found [here](./docs/keyboard-interactive.md).
215
216## Dynamic user creation or modification
217
218A user can be created or modified by an external program just before the login. More information about this can be found [here](./docs/dynamic-user-mod.md).
219
220## Custom Actions
221
222SFTPGo allows you to configure custom commands and/or HTTP hooks to receive notifications about file uploads, deletions and several other events.
223
224More information about custom actions can be found [here](./docs/custom-actions.md).
225
226## Virtual folders
227
228Directories outside the user home directory or based on a different storage provider can be exposed as virtual folders, more information [here](./docs/virtual-folders.md).
229
230## Other hooks
231
232You can get notified as soon as a new connection is established using the [Post-connect hook](./docs/post-connect-hook.md) and after each login using the [Post-login hook](./docs/post-login-hook.md).
233You can use your own hook to [check passwords](./docs/check-password-hook.md).
234
235## Storage backends
236
237### S3 Compatible Object Storage backends
238
239Each user can be mapped to the whole bucket or to a bucket virtual folder. This way, the mapped bucket/virtual folder is exposed over SFTP/SCP/FTP/WebDAV. More information about S3 integration can be found [here](./docs/s3.md).
240
241### Google Cloud Storage backend
242
243Each user can be mapped with a Google Cloud Storage bucket or a bucket virtual folder. This way, the mapped bucket/virtual folder is exposed over SFTP/SCP/FTP/WebDAV. More information about Google Cloud Storage integration can be found [here](./docs/google-cloud-storage.md).
244
245### Azure Blob Storage backend
246
247Each user can be mapped with an Azure Blob Storage container or a container virtual folder. This way, the mapped container/virtual folder is exposed over SFTP/SCP/FTP/WebDAV. More information about Azure Blob Storage integration can be found [here](./docs/azure-blob-storage.md).
248
249### SFTP backend
250
251Each user can be mapped to another SFTP server account or a subfolder of it. More information can be found [here](./docs/sftpfs.md).
252
253### Encrypted backend
254
255Data at-rest encryption is supported via the [cryptfs backend](./docs/dare.md).
256
257### Other Storage backends
258
259Adding new storage backends is quite easy:
260
261- implement the [Fs interface](./vfs/vfs.go#L28 "interface for filesystem backends").
262- update the user method `GetFilesystem` to return the new backend
263- update the web interface and the REST API CLI
264- add the flags for the new storage backed to the `portable` mode
265
266Anyway, some backends require a pay per use account (or they offer free account for a limited time period only). To be able to add support for such backends or to review pull requests, please provide a test account. The test account must be available for enough time to be able to maintain the backend and do basic tests before each new release.
267
268## Brute force protection
269
270The [connection failed logs](./docs/logs.md) can be used for integration in tools such as [Fail2ban](http://www.fail2ban.org/). Example of [jails](./fail2ban/jails) and [filters](./fail2ban/filters) working with `systemd`/`journald` are available in fail2ban directory.
271
272You can also use the built-in [defender](./docs/defender.md).
273
274## Account's configuration properties
275
276Details information about account configuration properties can be found [here](./docs/account.md).
277
278## Performance
279
280SFTPGo can easily saturate a Gigabit connection on low end hardware with no special configuration, this is generally enough for most use cases.
281
282More in-depth analysis of performance can be found [here](./docs/performance.md).
283
284## Release Cadence
285
286SFTPGo releases are feature-driven, we don't have a fixed time based schedule. As a rough estimate, you can expect 1 or 2 new releases per year.
287
288## Acknowledgements
289
290SFTPGo makes use of the third party libraries listed inside [go.mod](./go.mod).
291
292We are very grateful to all the people who contributed with ideas and/or pull requests.
293
294Thank you [ysura](https://www.ysura.com/) for granting me stable access to a test AWS S3 account.
295
296## Sponsors
297
298I'd like to make SFTPGo into a sustainable long term project and your [sponsorship](https://github.com/sponsors/drakkan) will really help :heart:
299
300Thank you to our sponsors!
301
302[<img src="https://www.7digital.com/wp-content/themes/sevendigital/images/top_logo.png" alt="7digital logo">](https://www.7digital.com/)
303
304## License
305
306GNU AGPLv3
307
){kind=link}
){kind=link}
){kind=link}
){kind=link}
