1# Contributing 2 3This document provides information for developers who want to contribute to the 4RLS or run it in a heavily customised configuration. 5 6The RLS is open source and we'd love you to contribute to the project. Testing, 7reporting issues, writing documentation, writing tests, writing code, and 8implementing clients are all extremely valuable. 9 10Here is the list of known [issues](https://github.com/rust-lang/rls/issues). 11These are [good issues to start on](https://github.com/rust-lang/rls/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22). 12 13A good resource on how RLS works can be found [here](architecture.md). 14 15We're happy to help however we can. The best way to get help is either to 16leave a comment on an issue in this repo, or to ping us (nrc) in #rust-dev-tools 17on IRC. 18 19We'd love for existing and new tools to use the RLS. If that sounds interesting 20please get in touch by filing an issue or on IRC. 21 22If you want to implement RLS support in an editor, see [clients.md](clients.md). 23 24## Building 25 26Note, you don't need to build the `rls` to use it. Instead, you can install 27via `rustup`, which is the currently preferred method. See the 28[readme](README.md) for more information. 29 30### Step 1: Install build dependencies 31 32On Linux, you will need [cmake](https://cmake.org/), [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/) 33and [zlib](http://zlib.net/): 34 35- On Ubuntu run: `sudo apt-get install cmake pkg-config zlib1g-dev libssl-dev` 36- On Fedora run: `sudo dnf install cmake pkgconfig zlib-devel openssl-devel` 37 38On Windows, you will need to have [cmake](https://cmake.org/) installed. 39 40### Step 2: Clone and build the RLS 41 42Since the RLS is closely linked to the compiler and is in active development, 43you'll need a recent nightly compiler to build it. 44 45``` 46git clone https://github.com/rust-lang/rls.git 47cd rls 48cargo build --release 49``` 50 51#### If RLS couldn't be built with clippy 52 53Sometimes nightly toolchain changes break the `clippy_lints` dependency. 54Since RLS depends on `clippy_lints` by default, those changes can also break RLS itself. 55In this case, you can build RLS like this: 56 57`cargo build --no-default-features` (disabling the clippy feature) 58 59And sometimes git revision of `clippy` submodule in the Rust repo (https://github.com/rust-lang/rust/tree/master/src/tools) and `clippy_lints` dependency of RLS is different. 60In this case, submit a PR here updating the `clippy_lints` dependency to the git revision pulled from the Rust tree. 61 62### Step 3: Connect the RLS to your compiler 63 64If you're using recent versions of rustup, you will also need to make sure that 65the compiler's dynamic libraries are available for the RLS to load. You can see 66where they are using: 67 68``` 69rustc --print sysroot 70``` 71 72This will show you where the compiler keeps the dynamic libs. In Windows, this 73will be in the `bin` directory under this path. On other platforms, it will be 74in the `lib` directory. 75 76Next, you'll make the compiler available to the RLS: 77 78#### Windows 79 80On Windows, make sure this path (plus `bin`) is in your PATH. For example: 81 82``` 83set PATH=%PATH%;C:\Users\appveyor\.multirust\toolchains\nightly-i686-pc-windows-gnu\bin 84``` 85 86#### Mac 87 88For Mac, you need to set the DYLD_LIBRARY_PATH. For example: 89 90``` 91export DYLD_LIBRARY_PATH=$(rustc --print sysroot)/lib 92``` 93 94#### Linux 95 96For Linux, this path is called LD_LIBRARY_PATH. 97 98``` 99export LD_LIBRARY_PATH=$(rustc --print sysroot)/lib 100``` 101 102### Step 4: Download standard library metadata 103 104Finally, we need to get the metadata for the standard library. This lets 105us get additional docs and types for all of `std`. The command is currently only 106supported on the nightly compilers, though we hope to remove this restriction in 107the future. 108 109``` 110rustup component add rust-analysis 111``` 112 113If you've never set up Racer before, you may also need to follow the 114[Racer configuration steps](https://github.com/racer-rust/racer#configuration) 115 116## Running and testing 117 118You can run the rls by hand with: 119 120``` 121cargo run 122``` 123 124Though more commonly, you'll use an IDE plugin to invoke it for you 125(see [README.md](README.md) for details). 126 127We recommend using https://github.com/rust-lang/rls-vscode in VSCode. 128You can configure `rls-vscode` to use custom built binary by changing the 129`rust-client.rlsPath` setting to a full path to the binary you want to use. 130 131Anything the RLS writes to stderr is redirected to the output pane in 132VSCode - select "Rust Language Server" from the drop down box ("Rust Language 133Server" will only show up if there is any debugging output from RLS). Do not 134write to stdout, that will cause LSP errors (this means you cannot 135`println`). You can enable logging using 136[RUST_LOG](https://docs.rs/env_logger/) environment variable 137(e.g. `RUST_LOG=rls=debug code`). For adding your own, temporary logging you may 138find the `eprintln` macro useful. 139 140Test using `cargo test`. 141 142Testing is unfortunately minimal. There is support for regression tests, but not 143many actual tests exists yet. There is significant [work to do](https://github.com/rust-lang/rls/issues/12) 144before we have a comprehensive testing story. 145 146### CLI 147 148You can run RLS in the command line mode which is useful for debugging and 149testing, especially to narrow down a bug to either the RLS or a client. 150 151You need to run it in the root directory of the project to be analyzed with the 152`--cli` flag, e.g., `cargo run -- --cli`. This should initialize the RLS (which 153will take some time for large projects) and then give you a `>` prompt. During 154initialization RLS will print out a number of progress messages to the console 155(that might hide the prompt) during which some of the commands may not work 156properly. Look for the final message that will signal the end of the 157initialization phase which will look something like: 158 159``` 160{"jsonrpc":"2.0","method":"window/progress","params":{"done":true,"id":"progress_0","message":null,"percentage":null,"title":"Indexing"}} 161``` 162 163Type `help` (or just `h`) to see the [commands available][CLI_COMMANDS]. Note 164that the [positions][LSP_POSITION] in the requests and the responses are 165_zero-based_ (contrary to what you'll normally see in the IDE line numbers). 166 167[LSP_POSITION]: https://github.com/Microsoft/language-server-protocol/blob/gh-pages/specification.md#position 168 169[CLI_COMMANDS]: https://github.com/rust-lang/rls/blob/6d99a32d888a427250ff06229b6030b7dc276eac/rls/src/cmd.rs#L390-L424 170 171## Standard library support 172 173The way it works is that when the libraries are built, the compiler can emit all 174the data that the RLS needs. This can be read by the RLS on startup and used to 175provide things like type on hover without having access to the source code for 176the libraries. 177 178The compiler gives every definition an id, and the RLS matches up these ids. In 179order for the RLS to work, the id of a identifier used in the IDE and the id of 180its declaration in a library must match exactly. Since ids are very unstable, 181the data used by the RLS for libraries must match exactly with the crate that 182your source code links with. 183 184You need a version of the above data which exactly matches the standard 185libraries you will use with your project. Rustup takes care of this for you and 186is the preferred (and easiest) method for installing this data. If you want to 187use the RLS with a Rust compiler/libraries you have built yourself, then you'll 188need to take some extra steps. 189 190 191### Install with rustup 192 193You'll need to be using [rustup](https://www.rustup.rs/) to manage your Rust 194compiler toolchains. The RLS does not yet support cross-compilation - your 195compiler host and target must be exactly the same. 196 197You must be using nightly (you need to be using nightly for the RLS to work at 198the moment in any case). To install a nightly toolchain use `rustup install 199nightly`. To switch to using that nightly toolchain by default use `rustup 200default nightly`. 201 202Add the RLS data component using `rustup component add rust-analysis`. 203 204Everything should now work! You may need to restart the RLS. 205 206 207### Build it yourself 208 209When you build Rust, run it with a `RUSTC_SAVE_ANALYSIS=api` environment variable, e.g. with: 210 211``` 212RUSTC_SAVE_ANALYSIS=api ./x.py build 213``` 214 215When the build has finished, you should have a bunch of JSON data in a directory like 216`~/rust1/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/save-analysis`. 217 218You need to copy all those files (should be around 16) into a new directory: 219`~/rust1/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/analysis` 220(assuming you are running the stage 2 compiler you just built. You'll need to 221modify the root directory (`~/rust1` here) and the host triple 222(`x86_64-unknown-linux-gnu` in both places)). 223 224 225Finally, to run the RLS you'll need to set things up to use the newly built 226compiler, something like: 227 228``` 229export RUSTC="~/rust1/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" 230``` 231 232Either before you run the RLS, or before you run the IDE which will start the 233RLS. 234 235 236### Details 237 238Rustup (or you, manually) will install the rls data (which is a bunch of json 239files) into `$SYSROOT/lib/rustlib/$TARGET_TRIPLE/analysis`, where `$SYSROOT` is 240your Rust sysroot, this can be found using `rustc --print=sysroot`. 241`$TARGET_TRIPLE` is the triple which defines the compilation target. Since the 242RLS currently does not support cross-compilation, this must match your host 243triple. It will look something like `x86_64-unknown-linux-gnu`. 244 245For example, on my system RLS data is installed at: 246 247``` 248/home/ncameron/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/analysis 249``` 250 251This data is only for the standard libraries, project-specific data is stored 252inside your project's target directory. 253 254 255## Implementation overview 256 257The goal of the RLS project is to provide an awesome IDE experience *now*. That 258means not waiting for incremental compilation support in the compiler. However, 259Rust is a somewhat complex language to analyze and providing precise and 260complete information about programs requires using the compiler. 261 262The RLS has two data sources - the compiler and Racer. The compiler is always 263right, and always precise. But can sometimes be too slow for IDEs. Racer is 264nearly always fast, but can't handle some constructs (e.g., macros) or can only 265handle them with limited precision (e.g., complex generic types). 266 267The RLS tries to provide data using the compiler. It sets a time budget and 268queries both the compiler and Racer. If the compiler completes within the time 269budget, we use that data. If not, we use Racer's data. 270 271We link both Racer and the compiler into the RLS, so we don't need to shell out 272to either (though see notes on the build process below). We also customise our 273use of the compiler (via standard APIs) so that we can read modified files 274directly from memory without saving them to disk. 275 276### Building 277 278The RLS tracks changes to files, and keeps the changed file in memory (i.e., the 279RLS does not need the IDE to save a file before providing data). These changed 280files are tracked by the 'Virtual File System' (which is a bit of a grandiose 281name for a pretty simple file cache at the moment, but I expect this area to 282grow significantly in the future). The VFS is in a [separate 283-crate](https://github.com/nrc/rls-vfs). 284 285We want to start building before the user needs information (it would be too 286slow to start a build when data is requested). However, we don't want to start a 287build on every keystroke (this would be too heavy on user resources). Nor is 288there any point starting multiple builds when we would throw away the data from 289some of them. We therefore try to queue up and coalesce builds. This is further 290documented in [src/build.rs](src/build.rs). 291 292When we do start a build, we may also need to build dependent crates. We 293therefore do a full `cargo build`. However, we do not compile the last crate 294(the one the user is editing in the IDE). We only run Cargo to get a command 295line to build that crate. Furthermore, we cache that command line, so for most 296builds (where we don't need to build dependent crates, and where we can be 297reasonably sure they haven't changed since a previous build) we don't run Cargo 298at all. 299 300The command line we got from Cargo, we chop up and feed to the in-process 301compiler. We then collect error messages and analysis data in JSON format 302(although this is inefficient and [should 303change](https://github.com/rust-lang/rls/issues/25)). 304 305### Analysis data 306 307From the compiler, we get a serialized dump of its analysis data (from name 308resolution and type checking). We combine data from all crates and the standard 309libraries and combine this into an index for the whole project. We cross- 310reference and store this data in HashMaps and use it to look up data for the 311IDE. 312 313Reading, processing, and storing the analysis data is handled by the 314[rls-analysis crate](https://github.com/nrc/rls-analysis) 315 316### Communicating with IDEs 317 318The RLS communicates with IDEs via 319the [Language Server protocol](https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md). 320 321The LS protocol uses JSON sent over stdin/stdout. The JSON is rather dynamic - 322we can't make structs to easily map to many of the protocol objects. The client 323sends commands and notifications to the RLS. Commands must get a reply, 324notifications do not. Usually the structure of the reply is dictated by the 325protocol spec. The RLS can also send notifications to the client. So for a long 326running task (such as a build), the RLS will reply quickly to acknowledge the 327request, then send a message later with the result of the task. 328 329Associating requests with replies is done using an id which must be handled by 330the RLS. 331 332 333### Extensions to the Language Server Protocol 334 335The RLS uses some custom extensions to the Language Server Protocol. 336These are all sent from the RLS to an LSP client and are only used to 337improve the user experience by showing progress indicators. 338 339* `window/progress`: notification, `title: "Building"`. Sent before build starts. 340* `window/progress`: notification with `title: "Building"`, repeated for each compile target. 341 * When total amount of work is not known, has field `message` set to the current crate name. 342 * When total amount of work is known, has field `percentage` set to how much of build has started. 343* `window/progress`: notification, `title: "Building"`, `"done": true`. Sent when build ends. 344* `window/progress`: notification, `title: "Indexing"`. Sent before analysis of build starts. 345* ... standard LSP `publishDiagnostics` 346* `window/progress`: notification, `title: "Indexing"`, `"done": true`. Sent when analysis ends. 347