README.md
1# Pichi 
2
3Pichi is a flexible rule-based proxy.
4
5## Build Status
6
7### Unix-like
8
9| OS | macOS 10.15 | macOS 10.14 | Alpine 3.11 |
10|:----------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
11| **Toolchain** | Xcode 11.3 | Xcode 11.1 | GCC 8.3 |
12| **Status** | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=9&branchName=master) | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=3&branchName=master) | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=5&branchName=master) |
13
14### Windows
15
16| OS | Windows Server 2019 | Windows Server 2016 |
17|:-------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
18| **Toolchain** | Visual Studio 2019 | Visual Studio 2017 |
19| **Status** | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=1&branchName=master) | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=2&branchName=master) |
20
21### Mobile
22
23| OS | iOS/tvOS | Android |
24|:-------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
25| **Toolchain** | Xcode 11.3 | NDK r20b |
26| **Status** | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=8&branchName=master) | [](https://dev.azure.com/pichi-ci/pichi/_build/latest?definitionId=7&branchName=master) |
27
28## Overview
29
30Pichi is designed
31
321. **to support common proxy protocols**: HTTP(S)/Socks5(s)/Shadowsocks;
331. **to dynamically and flexibly control the proxy route**: just use RESTful APIs;
341. **developer oriented**: NO GUI, but can be easily integrated into other GUIs;
351. **for personal usage**: performance is important, but not the first priority;
361. **for multiple platforms**: at least Windows, POSIX-compatible, Android and iOS.
37
38
39
40### Motivation
41
42Proxy is widely applied to traverse through the firewall, hide or change the original address, expose internal service port, etc. But if we iterate some famous proxy tools, it's easily recognized that each of them has at least one of these cons list below:
43
44* Not support all of HTTP, Socks5, Shadowsocks
45* Not support multiple ingresses or egresses
46* No rule-based routing
47
48As a result, the motivation of pichi is to provide a tool, which can
49
501. support sufficient proxy protocols,
511. be easily integrated with GUIs/APPs/scripts, which might have their own rule database,
521. control rule-based routing at runtime.
53
54### Use cases
55
56#### Alternative to PAC
57
58If an individual user is using a proxy, it's very common that the network traffic is probably split into 2 pieces of flow, one going through the proxy and another going directly.
59[PAC](https://en.wikipedia.org/wiki/Proxy_auto-config) is a good choice if web browsing is heavily used. But unfortunately, not all applications support PAC, such as most of [MUA](https://en.wikipedia.org/wiki/Email_client), [IM](https://en.wikipedia.org/wiki/Instant_messaging), and so on.
60
61Pichi is an alternative choice for this situation. It separates routing rules from every application.
62
63
64
65#### Unify proxy configuration
66
67If the configuration for remote proxies is volatile, such as changing IP/Port periodically, it's a nightmare that lots of clients are using it.
68Pichi can centralize the varies, rather than editing the configuration client by client.
69
70
71
72#### Trasparent proxy for DNS
73
74Transparent proxy is very useful if you want to use some DNS servers which might be already poisoned or blocked. Pichi provides *tunnel* ingress to act as a transparent proxy. Furthermore, the outgoing egress for each destination will be chosen by following the user-defined rules.
75
76
77
78### Supported protocols
79
80#### Ingress protocols
81
82* Tunnel: TCP tunnel to multiple destinations to be chosen by pre-defined load balance algorithms
83* HTTP Proxy: defined by [RFC 2068](https://www.ietf.org/rfc/rfc2068.txt)
84* HTTP Tunnel: defined by [RFC 2616](https://www.ietf.org/rfc/rfc2817.txt)
85* SOCKS5: defined by [RFC 1928](https://www.ietf.org/rfc/rfc1928.txt)
86* Shadowsocks: defined by [shadowsocks.org](https://shadowsocks.org/en/spec/Protocol.html)
87* Trojan: defined by [trojan-gfw](https://trojan-gfw.github.io/trojan/protocol) and its websocket extension defined by [trojan-go](https://github.com/p4gefau1t/trojan-go)
88* VMess: defined by [V2Ray](https://v2ray.com/developer/protocols/vmess.html)
89
90#### Egress protocols
91
92* Direct: connecting to destination directly
93* Reject: rejecting request immediately or after a fixed/random delay
94* HTTP Proxy: defined by [RFC 2068](https://www.ietf.org/rfc/rfc2068.txt)
95* HTTP Tunnel: defined by [RFC 2616](https://www.ietf.org/rfc/rfc2817.txt)
96* SOCKS5: defined by [RFC 1928](https://www.ietf.org/rfc/rfc1928.txt)
97* Shadowsocks: defined by [shadowsocks.org](https://shadowsocks.org/en/spec/Protocol.html)
98* Trojan: defined by [trojan-gfw](https://trojan-gfw.github.io/trojan/protocol) and its websocket extension defined by [trojan-go](https://github.com/p4gefau1t/trojan-go)
99* VMess: defined by [V2Ray](https://v2ray.com/developer/protocols/vmess.html)
100
101**NOTE:** HTTP egress would like to try [HTTP CONNECT](https://www.ietf.org/rfc/rfc2817.txt) first. HTTP proxy will be chosen if the previous handshake is failed.
102
103## Get started
104
105### Installation
106
107#### Linux
108
109Please use [Docker](https://www.docker.com):
110
111```
112$ docker pull pichi/pichi:latest
113$ docker run --rm pichi/pichi:latest pichi <options>
114```
115
116#### macOS
117
118Please use [Homebrew](https://brew.sh):
119
120```
121$ brew tap pichi-router/pichi
122$ brew install pichi
123$ pichi <options>
124```
125
126#### FreeBSD
127
128Please use [FreeBSD ports](https://ports.freebsd.org).
129
130```
131$ pkg install pichi
132```
133
134#### Windows or others
135
136Please refer to **Build** section.
137
138### Run
139
140```
141$ pichi -h
142Allow options:
143 -h [ --help ] produce help message
144 -l [ --listen ] arg (=::1) API server address
145 -p [ --port ] arg API server port
146 -g [ --geo ] arg GEO file
147 --json arg Initail configration(JSON format)
148 -d [ --daemon ] daemonize
149 -u [ --user ] arg run as user
150 --group arg run as group
151```
152
153`--port` and `--geo` are mandatory. `--json` option can take a JSON file as an Initial configuration to specify ingresses/egresses/rules/route. The initial configuration format looks like:
154
155```
156{
157 "ingresses": {
158 "ingress-0": {/* ingress configuration */},
159 "ingress-1": {/* ingress configuration */}
160 },
161 "egresses": {
162 "egress-0": {/* egress configuration */},
163 "egress-1": {/* egress configuration */}
164 },
165 "rules": {
166 "rule-0": {/* rule configuration */},
167 "rule-1": {/* rule configuration */}
168 },
169 "route": {/* route configuration */}
170}
171```
172
173Please refer to **Using Pichi API** section for the details of configurations of ingress/egress/rule/route.
174
175Furthermore, Pichi server reloads JSON configuration on `SIGHUP` received if OS supports.
176
177## Using Pichi API
178
179### Resources
180
181* **Ingress**: defines an incoming network adapter, containing protocol type, listening address/port, and protocol-specific configurations.
182* **Egress**: defines an outgoing network adapter, containing protocol type, next hop address/port, and protocol-specific configurations.
183* **Rule**: contains a group of conditions, such as IP ranges, domain regular expressions, the countries of the destination IP, and so on, that the incoming connection matching ANY conditions means the one matching this rule.
184* **Route**: indicates a priority ordered sequence of \[rule, egress\] pairs, and a default egress which would be forwarded to if none of the rules matched.
185
186### API Specification
187
188[Pichi API](https://app.swaggerhub.com/apis/pichi-router/pichi-api/1.4)
189
190### Examples
191
192#### Proxy like ss-local(shadowsocks-libev)
193
194```
195$ curl -i -X PUT -d '{ \
196> "type":"socks5", \
197> "bind":[ \
198> {"host":"127.0.0.1","port":1080} \
199> ] \
200> }' http://pichi-router:port/ingresses/socks5
201HTTP/1.1 204 No Content
202
203$ curl -i -X PUT -d '{ \
204> "type":"ss", \
205> "server":{"host":"my-ss-server","port":8388}, \
206> "option":{"method":"rc4-md5","password":"my-password"} \
207> }' http://pichi-router:port/egresses/shadowsocks
208HTTP/1.1 204 No Content
209
210$ curl -i -X PUT -d '{"default":"shadowsocks"}' http://pichi-router:port/route
211HTTP/1.1 204 No Content
212
213```
214
215#### HTTP proxy except intranet
216
217```
218$ curl -i -X PUT -d '{ \
219> "type":"http", \
220> "bind":[ \
221> {"host":"::","port":8080}
222> ] \
223> }' http://pichi-router:port/ingresses/http
224HTTP/1.1 204 No Content
225
226$ curl -i -X PUT -d '{ \
227> "type":"http", \
228> "server":{"host":"http-proxy","port":8080} \
229> }' http://pichi-router:port/egresses/http
230HTTP/1.1 204 No Content
231
232$ curl -i -X PUT -d '{ \
233> "range":[ \
234> "::1/128", \
235> "127.0.0.1/32", \
236> "10.0.0.0/8", \
237> "172.16.0.0/12", \
238> "192.168.0.0/16", \
239> "fc00::/7" \
240> ], \
241> "domain":["local"], \
242> "pattern":["^localhost$"] \
243> }' http://pichi-router:port/rules/intranet
244HTTP/1.1 204 No Content
245
246$ curl -i -X PUT -d '{ \
247> "default":"http", \
248> "rules":[ \
249> ["intranet","direct"] \
250> ] \
251> }' http://pichi-router:port/route
252HTTP/1.1 204 No Content
253
254```
255
256#### 100 shadowsocks servers
257
258```
259$ for((i=20000;i<20100;++i)); do \
260> curl -X PUT \
261> -d "{ \
262> \"type\":\"ss\", \
263> \"bind\":[ \
264> {\"host\":\"::\",\"port\":$i} \
265> ], \
266> \"option\":{ \
267> \"method\":\"rc4-md5\", \
268> \"password\":\"pw-$i\" \
269> } \
270> }" \
271> "http://pichi-router:port/ingresses/$i"; \
272> done
273
274```
275
276#### dark web
277
278```
279$ curl -i -X PUT -d '{ \
280> "type":"socks5", \
281> "server":{"host":"localhost","port":9050} \
282> }' http://pichi-router:port/egresses/tor
283HTTP/1.1 204 No Content
284
285$ curl -i -X PUT -d '{ \
286> "type":"http", \
287> "server":{"host":"localhost","port":4444} \
288> }' http://pichi-router:port/egresses/i2p
289HTTP/1.1 204 No Content
290
291$ curl -i -X PUT -d '{"domain":["onion"]}' http://pichi-router:port/rules/onion
292HTTP/1.1 204 No Content
293
294$ curl -i -X PUT -d '{"domain":["i2p"]}' http://pichi-router:port/rules/i2p
295HTTP/1.1 204 No Content
296
297$ curl -i -X PUT -d '{ \
298> "rules":[ \
299> ["onion","tor"], \
300> ["i2p","i2p"] \
301> ] \
302> }' http://pichi-router:port/route
303HTTP/1.1 204 No Content
304
305```
306
307#### socks5 server with TLS certificate issued by Let's encrypt CA
308
309```
310$ curl -i -X PUT -d '{ \
311> "type":"socks5", \
312> "bind":[ \
313> {"host":"::1","port":1080} \
314> ], \
315> "tls": { \
316> "key_file": "/etc/letsencrypt/live/example.com/privkey.pem", \
317> "cert_file": "/etc/letsencrypt/live/example.com/fullchain.pem" \
318> } \
319> }' http://pichi-router:port/ingresses/socks5s
320HTTP/1.1 204 No Content
321
322```
323
324#### DNS-over-TLS proxy
325
326```
327$ curl -i -X PUT -d '{ \
328> "type":"tunnel", \
329> "bind":[ \
330> {"host":"::1","port":853} \
331> ], \
332> "option":{ \
333> "destinations": [ \
334> { "host": "2606:4700:4700::1111", "port": 853 }, \
335> { "host": "2606:4700:4700::1001", "port": 853 }, \
336> { "host": "1.1.1.1", "port": 853 }, \
337> { "host": "1.0.0.1", "port": 853 } \
338> ], \
339> "balance":"random" \
340> } \
341> }' http://pichi-router:port/ingresses/cloudflare
342HTTP/1.1 204 No Content
343
344```
345
346#### More examples
347
348Please refer to the [folder](schemas/examples) to find more examples.
349
350## Build
351
352### Requirements
353
354* C++17
355* [Boost](https://www.boost.org) 1.72.0
356* [MbedTLS](https://tls.mbed.org) 2.7.0
357* [libsodium](https://libsodium.org) 1.0.12
358* [RapidJSON](http://rapidjson.org/) 1.1.0
359* [libmaxminddb](http://maxmind.github.io/libmaxminddb/) 1.3.0
360* [OpenSSL](https://www.openssl.org) or [LibreSSL](https://www.libressl.org)
361
362### CMake options
363
364* `BUILD_SERVER`: Build pichi application, the default is **ON**.
365* `BUILD_TEST`: Build unit test cases, the default is **ON**.
366* `STATIC_LINK`: Generate static library, the default is **ON**.
367* `INSTALL_DEVEL`: Install development files, the default is **OFF**.
368
369### Build and run tests
370
371Build and run on Unix-like:
372
373```
374$ cmake -B /path/to/build /path/to/pichi
375$ cmake --build /path/to/build
376$ cmake --build /path/to/build --target test
377```
378
379Build and run on Windows with [Vcpkg](https://github.com/Microsoft/vcpkg):
380
381```
382PS C:\pichi> cmake -D CMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake `
383>> -D VCPKG_TARGET_TRIPLET="x64-windows-static" -D CMAKE_BUILD_TYPE=MinSizeRel -B build .
384PS C:\pichi> cmake --build build --config Release
385PS C:\pichi> cmake --build build --config Release --target test
386```
387
388### Docker
389
390The pre-built docker image can be found on [Docker Hub](https://hub.docker.com/r/pichi/pichi),
391which is automatically generated according to `docker/pichi.dockerfile`.
392Furthermore, `docker/builder.dockerfile` is intended to provide a docker environment
393for development.
394
395```
396$ docker pull pichi/pichi
397$ docker run -d --name pichi --net host --restart always pichi/pichi \
398> pichi -g /usr/share/pichi/geo.mmdb -p 1024 -l 127.0.0.1
399c51b832bd29dd0333b0d32b0b0563ddc72821f7301c36c7635ae47d00a3bb902
400$ docker ps -n 1
401CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
402c51b832bd29d pichi/pichi "pichi -g /usr/share…" 1 seconds ago Up 1 seconds pichi
403```
404
405### Build library for iOS/Android
406
407Pichi is designed to run or be embedded into some APPs on iOS/Android. `deps-build` directory gives some helping scripts to build Pichi's dependencies for iOS/Android.
408
409#### iOS
410
411CMake provides the [detailed documents](https://cmake.org/cmake/help/v3.16/manual/cmake-toolchains.7.html#cross-compiling-for-ios-tvos-or-watchos) for iOS/tvOS/watchOS cross compiling. Let's follow it.
412
413```
414$ cmake -G Xcode -D CMAKE_SYSTEM_NAME=iOS -D CMAKE_OSX_DEPLOYMENT_TARGET=8.0 \
415> -D CMAKE_INSTALL_PREFIX=/path/to/sysroot [other options] \
416> -B /path/to/build /path/to/pichi
417$ cmake --build /path/to/build --config Release -- -sdk iphoneos # Build for iPhone
418$ cmake --build /path/to/build --config Release -- -sdk iphonesimulator # Build for iPhone simulator
419```
420
421On the other hand, `deps-build/boost.sh` can generate libraries for iOS/tvOS if below environment variables are set:
422
423* **PLATFORM**: to specify target OS(*iOS, tvOS*);
424* **ARCH**: to specify the target CPU architecture(it has to be corresponding to PLATFORM);
425* **SYSROOT**: to specify root install directory of headers/libraries.
426
427For example:
428
429```
430$ # Build for iOS simulator
431$ export PLATFORM=iOS
432$ export ARCH=x86_64
433$ export SYSROOT=/path/to/ios/sysroot
434$ bash deps-build/boost.sh /path/to/boost/source
435...
436```
437
438#### Android
439
440The usage of `deps-build/boost.sh` is very similar to iOS one, except environment variables:
441
442* **PLATFORM**: *android-\<API\>s* are available;
443* **ARCH**: to specify target CPU architecture(*arm, arm64, i386, x86_64*);
444* **SYSROOT**: to specify root install directory of headers/libraries.
445
446Android NDK kindly provides `build/tools/make_standalone_toolchain.py` script to generate a cross-compiling toolchain for any version of Android.
447
448```
449$ # Generate standalone toolchain
450$ export NDK_ROOT=/path/to/android/ndk
451$ export TOOLCHAIN_ROOT=/path/to/standalone/toolchain
452$
453$ # Create cross toolchain
454$ python ${NDK_ROOT}/build/tools/make_standalone_toolchain.py --arch arm64 --api 29 \
455> --install-dir ${TOOLCHAIN_ROOT}
456$
457$ # Build for android-29/arm64
458$ export PLATFORM=android-29
459$ export ARCH=64
460$ export SYSROOT=${TOOLCHAIN_ROOT}/sysroot
461$
462$ # Build boost
463$ bash deps-build/boost.sh /path/to/boost/source
464$
465$ # Build other dependent libraries
466$ cmake -D CMAKE_SYSROOT=${SYSROOT} -D CMAKE_INSTALL_PREFIX=${SYSROOT} \
467> -D CMAKE_C_COMPILER=${TOOLCHAIN_ROOT}/bin/clang \
468> -D CMAKE_CXX_COMPILER=${TOOLCHAIN_ROOT}/bin/clang++ \
469> [other options] -B /path/to/build/library /path/to/other/library
470$ cmake --build /path/to/build/libraries --target install
471$
472$ # Build pichi
473$ cmake -D CMAKE_SYSROOT=${SYSROOT} -D CMAKE_INSTALL_PREFIX=${SYSROOT} \
474> -D CMAKE_C_COMPILER=${TOOLCHAIN_ROOT}/bin/clang \
475> -D CMAKE_CXX_COMPILER=${TOOLCHAIN_ROOT}/bin/clang++ \
476> [other options] -B /path/to/build /path/to/pichi
477$ cmake --build /path/to/build --target install
478```
479
480#### Cross-Compiling for other architecture
481
482`deps-build/boost.sh` doesn't provide any cross-compiling steps except iOS/Android,
483but it's not very complicated to cross-compile Boost libraries.
484There's only one thing important. Boost.Context requires that `target-os`, `abi`, `binary-format`, `architecture` and `address-model` [must be explicitly specified correctly while cross-compiling](https://www.boost.org/doc/libs/release/libs/context/doc/html/context/requirements.html).
485
486On the contrary, other libraries can be cross-complied very easily by using CMake.
487For example:
488
489```
490$ # Cross-Compiling for aarch64-freebsd12.0
491$ export AARCH64_SYSROOT=/path/to/aarch64/base/system
492$ export CROSS_FLAGS="-target aarch64-unknown-freebsd12.0 --sysroot=${AARCH64_SYSROOT}"
493$
494$ # Build Boost
495$ cd /path/to/boost
496$ ./bootstrap.sh
497$ cat > project-config.jam <<EOF
498> using clang :
499> : /usr/bin/clang++ -std=c++17 ${CROSS_FLAGS}
500> ;
501> EOF
502$ ./b2 --with-context --with-filesystem --with-program_options --with-system \
503 --prefix=${AARCH64_SYSROOT} target-os=freebsd abi=aapcs binary-format=elf \
504 architecture=arm address-model=64 variant=release link=static install
505$
506$ # Build other libraries
507$ cmake -D CMAKE_C_COMPILER=clang -D CMAKE_C_FLAGS="${CROSS_FLAGS}" \
508 -D CMAKE_INSTALL_PREFIX=${AARCH64_SYSROOT} \
509 -B /path/to/build/library /path/to/library
510$ cmake --build /path/to/build --target install
511$
512$ # Build pichi
513$ cmake -D CMAKE_CXX_COMPILER=clang++ -D CMAKE_CXX_FLAGS="${CROSS_FLAGS}" \
514 -D CMAKE_INSTALL_PREFIX=${AARCH64_SYSROOT} -B /path/to/build /path/to/pichi
515$ cmake --build /path/to/build
516$ file /path/to/build/server/pichi
517build/server/pichi: ELF 64-bit LSB executable, ARM aarch64, version 1 (FreeBSD),
518dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.0 (1200086),
519FreeBSD-style, with debug_info, not stripped
520```
521
522## Integration with pichi
523
524There are 2 ways to integrate with pichi:
525
526* **Standalone**: pichi runs in its own process,
527* **In-Process**: pichi runs in its supervisor process.
528
529Regardless of any mode, the supervisor must communicate with pichi via RESTful APIs.
530
531### Standalone
532
533Standalone mode requires `BUILD_SERVER` CMake option, which builds code in `server` directory. For example:
534
535```
536$ cmake -D CMAKE_INSTALL_PREFIX=/usr -D CMAKE_BUILD_TYPE=MinSizeRel -D BUILD_SERVER=ON -B build .
537$ cmake --build build --target install/strip
538```
539
540### In-Process
541
542In-Process mode is suitable for the scenarios that the standalone process is prohibited or unnecessary, such as iOS/Android, or the supervisor prefers to run pichi in its own process. There are 2 types of interface to run pichi.
543
544#### C function
545
546C function can be invoked by lots of program languages. It's defined in `include/pichi.h`:
547
548```C
549/*
550 * Start PICHI server according to
551 * - bind: server listening address, NOT NULL,
552 * - port: server listening port,
553 * - mmdb: IP GEO database, MMDB format, NOT NULL.
554 * The function doesn't return if no error occurs, otherwise -1.
555 */
556extern int pichi_run_server(char const* bind, uint16_t port, char const* mmdb);
557```
558
559`pichi_run_server` will block the caller thread if no error occurs.
560
561#### C++ class
562
563C++ class might sometimes be friendly while the supervisor is written in C++. It's defined in `include/pichi/api/server.hpp`:
564
565```C++
566class Server {
567public:
568 Server(boost::asio::io_context&, char const* mmdb);
569 void listen(std::string_view bind, uint16_t port);
570};
571```
572
573`pichi::api::Server` accepts a `boost::asio::io_context` object reference, which is shared by the supervisor. Furthermore, `Server::listen` **doesn't** block the caller thread. It means that the supervisor can invoke `io_context::run()` right where it wants to do. Here's a simple code snippet:
574
575```C++
576#include <pichi/api/server.hpp>
577
578auto io = boost::asio::io_context{};
579
580auto server = pichi::api::Server{io, mmdb};
581server.listen(bind, port);
582
583// Setup other ASIO services
584
585io.run(); // Thread blocked
586
587```
588
589## Donation
590
591Please offer me a cup of coffee if you like pichi.
592
593[](
594https://www.blockchain.com/btc/payment_request?address=33CEcNHjKpyHSq4gfVHDhkDjYwWjLQt3Qo
595)
596
){kind=link}
){kind=link}