1ycmd: a code-completion & comprehension server 2============================================== 3 4[![Build status](https://dev.azure.com/YouCompleteMe/YCM/_apis/build/status/ycm-core.ycmd?branchName=master)](https://dev.azure.com/YouCompleteMe/YCM/_build/latest?definitionId=4&branchName=master) 5[![Coverage status](https://img.shields.io/codecov/c/github/ycm-core/ycmd/master.svg)](https://codecov.io/gh/ycm-core/ycmd) 6 7ycmd is a server that provides APIs for code-completion and other 8code-comprehension use-cases like semantic GoTo commands (and others). For 9certain filetypes, ycmd can also provide diagnostic errors and warnings. 10 11ycmd was originally part of [YouCompleteMe][ycm]'s codebase, but has been split 12out into a separate project so that it can be used in editors other than Vim. 13 14Check [the API documentation][api-docs] if you want to implement a client. A 15good way to learn how to interact with ycmd is by reading through (and running) 16the [`example_client.py`][example-client] file. See the [README for the 17examples][example-readme] folder for details on how to run the example client. 18 19Known ycmd clients: 20------------------ 21 22- [YouCompleteMe][ycm]: Vim client, stable and exposes all ycmd features. 23- [emacs-ycmd][]: Emacs client. 24- [you-complete-me][atom-you-complete-me]: Atom client. 25- [YcmdCompletion][sublime-ycmd-completion]: Sublime client 26- [sublime-ycmd][sublime-ycmd]: Sublime Text 3 client. 27- [kak-ycmd][]: Kakoune client. 28- [you-complete-me][vscode-you-complete-me]: VSCode client. 29- [gycm][]: Geany client. 30- [nano-ycmd][]: GNU nano client. 31 32Feel free to send a pull request adding a link to your client here if you've 33built one. 34 35Building 36-------- 37**If you're looking to develop ycmd, see the [instructions for running the 38tests][test-setup].** 39 40This is all for Ubuntu Linux. Details on getting ycmd running on other OS's can 41be found in [YCM's instructions][ycm-install] (ignore the Vim-specific parts). 42Note that **ycmd runs on Python 3.5.1+.** 43 44First, install the minimal dependencies: 45``` 46sudo apt install build-essential cmake python3-dev 47``` 48 49Next, install the language specific dependencies you need: 50- `sudo apt install golang-go` for Go. 51- `sudo apt install npm` for JavaScript and TypeScript. 52- `sudo apt install mono-devel` for C#. 53- `sudo apt install openjdk-8-jre` for Java. 54 55When you first clone the repository you'll need to update the submodules: 56``` 57git submodule update --init --recursive 58``` 59 60Then run `python3 build.py --all` or any of the specific completers listed by 61`python3 build.py --help`. This should get you going. 62 63For more detailed instructions on building ycmd, see [YCM's 64instructions][ycm-install] (ignore the Vim-specific parts). 65 66Supported compilers 67------------------- 68 69- GCC 8 and later 70- Clang 7 and later 71- Microsoft Visual Studio 2017 v 15.7 and later 72 73API notes 74--------- 75 76- All strings going into and out of the server are UTF-8 encoded. 77- All lines end with `\n`. 78- All line and column numbers are 1-based, not 0-based. They are also byte 79 offsets, _not_ Unicode codepoint offsets. 80- All file paths are full, absolute paths. 81- All requests to the server _must_ include an [HMAC][] in the `x-ycm-hmac` HTTP 82 header. The HMAC is computed from the shared secret passed to the server on 83 startup and the request/response body. The digest algorithm is SHA-256. The 84 server will also include the HMAC in its responses; you _must_ verify it 85 before using the response. See [`example_client.py`][example-client] to see 86 how it's done. 87- API is documented in swagger and published on [the website][api-docs]. 88 89How ycmd works 90-------------- 91 92There are several completion engines in ycmd. The most basic one is an 93identifier-based completer that collects all of the identifiers in the file 94provided in the completion request, other files of the same filetype that were 95provided previously and any tags files produced by ctags. This engine is 96non-semantic. 97 98There are also several semantic engines in YCM. There's [clangd][clangd]-based 99completer that both provide semantic completion for C-family languages. 100There's also a Jedi-based completer for semantic completion for Python, an 101OmniSharp-based completer for C#, a [gopls][gopls]-based completer for Go 102(using [gopls][gopls] for jumping to definitions), a TSServer-based completer 103for JavaScript and TypeScript, a [jdt.ls][jdtls]-based server for Java, and a 104[RLS][]-based completer for Rust. More will be added with time. 105 106 107There are also other completion engines, like the filepath completer (part of 108the identifier completer). 109 110The server will automatically detect which completion engine would be the best 111in any situation. On occasion, it queries several of them at once, merges the 112outputs and presents the results. 113 114Semantic engines are triggered only after semantic "triggers" are inserted in 115the code. If the request received shows that the user's cursor is after the last 116character in `string foo; foo.` in a C# file, this would trigger the semantic 117engine to 118examine members of `foo` because `.` is a [default semantic 119trigger][trigger-defaults] for C# (triggers can be changed dynamically). If the 120text were `string foo; foo.zoo`, semantic completion would still be triggered 121(the trigger is behind the `zoo` word the user is typing) and the results would 122be filtered with the `zoo` query. 123 124Semantic completion can also be forced by setting `force_semantic: true` in 125the JSON data for the completion request, _but you should only do this if the 126user explicitly requested semantic completion with a keyboard shortcut_; 127otherwise, leave it up to ycmd to decide when to use which engine. 128 129The reason why semantic completion isn't always used even when available is 130because the semantic engines can be slow and because most of the time, the 131user doesn't actually need semantic completion. 132 133There are two main use-cases for code-completion: 134 1351. The user knows which name they're looking for, they just don't want to type 136 the whole name. 1372. The user either doesn't know the name they need or isn't sure what the name 138 is. This is also known as the "API exploration" use-case. 139 140The first use case is the most common one and is trivially addressed with the 141identifier completion engine (which BTW is blazing fast). The second one needs 142semantic completion. 143 144### Completion string filtering 145 146A critical thing to note is that the completion **filtering is NOT based on 147the input being a string prefix of the completion** (but that works too). The 148input needs to be a _[subsequence][] match_ of a completion. This is a fancy way 149of saying that any input characters need to be present in a completion string in 150the order in which they appear in the input. So `abc` is a subsequence of 151`xaybgc`, but not of `xbyxaxxc`. 152 153### Completion string ranking 154 155The subsequence filter removes any completions that do not match the input, but 156then the sorting system kicks in. It's a bit involved, but roughly speaking 157"word boundary" (WB) subsequence character matches are "worth" more than non-WB 158matches. In effect, this means given an input of "gua", the completion 159"getUserAccount" would be ranked higher in the list than the "Fooguxa" 160completion (both of which are subsequence matches). A word-boundary character 161are all capital characters, characters preceded by an underscore and the first 162letter character in the completion string. 163 164### Auto-shutdown if no requests for a while 165 166If the server hasn't received any requests for a while (controlled by the 167`--idle_suicide_seconds` ycmd flag), it will shut itself down. This is useful 168for cases where the process that started ycmd dies without telling ycmd to die 169too or if ycmd hangs (this should be extremely rare). 170 171If you're implementing a client for ycmd, ensure that you have some sort of 172keep-alive background thread that periodically pings ycmd (just call the 173`/healthy` handler, although any handler will do). 174 175You can also turn this off by passing `--idle_suicide_seconds=0`, although that 176isn't recommended. 177 178### Exit codes 179 180During startup, ycmd attempts to load the `ycm_core` library and exits with one 181of the following return codes if unsuccessful: 182 183- 3: unexpected error while loading the library; 184- 4: the `ycm_core` library is missing; 185- 7: the version of the `ycm_core` library is outdated. 186- 8: server is started with python; recompile with python3. 187 188User-level customization 189----------------------- 190 191You can provide settings to ycmd on server startup. There's a 192[`default_settings.json`][def-settings] file that you can tweak. See the 193[_Options_ section in YCM's _User Guide_][options] for a description on what 194each option does. Pass the path to the modified settings file to ycmd as an 195`--options_file=/path/to/file` flag. Note that you must set the `hmac_secret` 196setting (encode the value with [base64][]). Because the file you are passing 197contains a secret token, ensure that you are creating the temporary file in a 198secure way (the [`mkstemp()`][mkstemp] Linux system call is a good idea; use 199something similar for other OS's). 200 201After it starts up, ycmd will _delete_ the settings file you provided after 202it reads it. 203 204The settings file is something your editor should produce based on values your 205user has configured. There's also an extra file (`.ycm_extra_conf.py`) your user 206is supposed to provide to configure certain semantic completers. More 207information on it can also be found in the [corresponding section of YCM's _User 208Guide_][extra-conf-doc]. 209 210### `.ycm_extra_conf.py` specification 211 212The `.ycm_extra_conf.py` module may define the following functions: 213 214#### `Settings( **kwargs )` 215 216This function allows users to configure the language completers on a per project 217basis or globally. Currently, it is required by the libclang-based completer and 218optional for other completers. The following arguments can be retrieved from 219the `kwargs` dictionary and are common to all completers: 220 221- `language`: an identifier of the completer that called the function. Its value 222 is `python` for the Python completer and `cfamily` for the C-family completer. 223 This argument is useful to configure several completers at once. For 224 instance: 225 226 ```python 227 def Settings( **kwargs ): 228 language = kwargs[ 'language' ] 229 if language == 'cfamily': 230 return { 231 # Settings for the libclang and clangd-based completer. 232 } 233 if language == 'python': 234 return { 235 # Settings for the Python completer. 236 } 237 return {} 238 ``` 239 240- `filename`: absolute path of the file currently edited. 241 242- `client_data`: any additional data supplied by the client application. 243 See the [YouCompleteMe documentation][extra-conf-vim-data-doc] for an 244 example. 245 246The return value is a dictionary whose content depends on the completer. 247 248#### LSP based completers 249 250LSP servers often support user configuration via the initialise request. These 251are usually presented as options in the UI. Ycmd supports this using the 252`.ycm_extra_conf.py` by allowing the user to specify the exact dictionary of 253settings that are passed in the server initialise message. These options are 254returned from `Settings` under the `ls` key. The python dictionary is converted 255to json and included verbatim in the LSP initialize request. In order to 256determine the set of options for a server, consult the server's documentation or 257`package.json` file. 258A `config_sections` object is a dictionary whose keys are "sections" and values 259are pieces of settings (usually found in the `ls` object) corresponding to 260those sections. This is even more underspecified and requires trial and error 261to make it work. It is optional and only useful if you explicitly enable 262`workspace/configuration` support. 263 264Example of LSP configuration: 265```python 266def Settings( **kwargs ): 267 if kwargs[ 'language' ] == 'java': 268 return { 'ls': { 'java.rename.enabled' : False }, 269 # `config_sections` is not used for java... 270 'config_sections': { 'section0': {} } 271``` 272 273In addition, ycmd can use any language server, given a file type and a command 274line. A user option `language_server` can be used to plug in a LSP server ycmd 275wouldn't usually know about. The value is a list of dictionaries containing: 276 277- `name`: the string representing the name of the server 278- `cmdline`: the list representing the command line to execute the server 279 (optional; mandatory if port not specified) 280- `port`: optional. If specified, a TCP connection is used to this port. If set 281 to `*`, an unused locall port is selected and made availble in the `cmdline` 282 as `${port}` (see below examples). 283- `filetypes`: list of supported filetypes. 284- `project_root_files`: Tells ycmd which files indicate project root. 285- `capabilities'`: Overrides the default LSP capabilities of ycmd. 286 - If you enable `workspace/configuration` support, check the extra conf 287 details, relevant to LSP servers. 288 289```json 290{ 291 "language_server": [ { 292 "name": "gopls", 293 "cmdline": [ "/path/to/gopls", "-rpc.trace" ], 294 "filetypes": [ "go" ], 295 "project_root_files": [ "go.mod" ] 296 } ] 297} 298``` 299 300Or, to use a TCP connection: 301 302```json 303{ 304 "language_server": [ { 305 "name": "godot", 306 "port": "6008", 307 "filetypes": [ "gdscript" ] 308 } ] 309} 310``` 311 312Or, to use an unused local port, set `port` to `*` and use `${port}` in the 313`cmdline`: 314 315```json 316{ 317 "language_server": [ { 318 "name": "someserver", 319 "cmdline": [ "/path/to/some/server", "--port", "${port}" ], 320 "port": "*", 321 "filetypes": [ "somethign" ], 322 "project_root_files": [ "somethingfile" ] 323 } ] 324} 325``` 326 327 328When plugging in a completer in this way, the `kwargs[ 'language' ]` will be set 329to the value of the `name` key, i.e. `gopls` in the above example. 330 331LSP completers currently supported without `language_server`: 332 333- Java 334- Rust 335- Go 336- C-family 337 338One can also override the root directory, with `project_directory`. 339 340```python 341def Settings( **kwargs ): 342 return { 'project_directory': 'src/' } # The path may be absolute as well. 343``` 344 345##### C-family settings 346 347The `Settings` function is called by the libclang and clangd-based completers to 348get the compiler flags to use when compiling the current file. The absolute path 349of this file is accessible under the `filename` key of the `kwargs` dictionary. 350 351The return value expected by both completers is a dictionary containing the 352following items: 353 354- `flags`: (mandatory for libclang, optional for clangd) a list of compiler 355 flags. 356 357- `include_paths_relative_to_dir`: (optional) the directory to which the include 358 paths in the list of flags are relative. Defaults to ycmd working directory 359 for the libclang completer and `.ycm_extra_conf.py`'s directory for the 360 clangd completer. 361 362- `do_cache`: (optional) a boolean indicating whether or not the result of 363 this call (i.e. the list of flags) should be cached for this file name. 364 Defaults to `True`. If unsure, the default is almost always correct. 365 366The libclang-based completer also supports the following items: 367 368- `override_filename`: (optional) a string indicating the name of the file to 369 parse as the translation unit for the supplied file name. This fairly advanced 370 feature allows for projects that use a 'unity'-style build, or for header 371 files which depend on other includes in other files. 372 373- `flags_ready`: (optional) a boolean indicating that the flags should be 374 used. Defaults to `True`. If unsure, the default is almost always correct. 375 376A minimal example which simply returns a list of flags is: 377 378```python 379def Settings( **kwargs ): 380 return { 381 'flags': [ '-x', 'c++' ] 382 } 383``` 384 385##### Formatting settings 386 387The configuration for `Format` subcommand can be specified with an extra conf for 388[the java subserver][jdtls] and for the typescript subserver. 389The formatter options can be found below: 390 391- [Java configuration][jdt-formatter] 392- [TSServer configuration][ts-formatter], 393 394These servers support custom formatting options to be supplied in a different 395way than the rest. For this purpose the `Settings` function can return a 396`formatter` property. 397 398An example of the formatter configuration would be: 399 400```python 401def Settings( **kwargs ): 402 return { 403 'formatting_options': { 404 'org.eclipse.jdt.core.formatter.lineSplit': 30, 405 } 406 } 407``` 408 409##### Python settings 410 411The `Settings` function allows users to specify the Python interpreter and 412the `sys.path` used by the completer to provide completion and code 413comprehension. No additional arguments are passed. 414 415The return value expected by the completer is a dictionary containing the 416following items: 417 418- `interpreter_path`: (optional) path to the Python interpreter. `~` and 419 environment variables in the path are expanded. If not an absolute path, it 420 will be searched through the `PATH`. 421 422- `sys_path`: (optional) list of paths prepended to `sys.path`. 423 424Usage example: 425 426```python 427def Settings( **kwargs ): 428 return { 429 'interpreter_path': '~/project/virtual_env/bin/python', 430 'sys_path': [ '~/project/third_party/module' ] 431 } 432``` 433 434#### `PythonSysPath( **kwargs )` 435 436Optional for Python support. 437 438This function allows further customization of the Python path `sys.path`. Its 439parameters are the possible items returned by the `Settings` function for the 440Python completer: 441 442- `interpreter_path`: path to the Python interpreter. 443 444- `sys_path`: list of Python paths from `sys.path`. 445 446The return value should be the modified list of Python paths. 447 448See [ycmd's own `.ycm_extra_conf.py`][ycmd-extra-conf] for an example. 449 450### Global extra conf file specification 451 452The global extra module must expose the same functions as the 453`.ycm_extra_conf.py` module with the following additions: 454 455#### `YcmCorePreLoad()` 456 457Optional. 458 459This method, if defined, is called by the server prior to importing the c++ 460python plugin. It is not usually required and its use is for advanced users 461only. 462 463#### `Shutdown()` 464 465Optional. 466 467Called prior to the server exiting cleanly. It is not usually required and its 468use is for advanced users only. 469 470Backwards compatibility 471----------------------- 472 473ycmd's HTTP+JSON interface follows [SemVer][]. While ycmd has seen extensive use 474over the last several years as part of YCM, the version number is below 1.0 475because some parts of the API _might_ change slightly as people discover 476possible problems integrating ycmd with other editors. In other words, the 477current API might unintentionally be Vim-specific. We don't want that. 478 479Note that ycmd's internal API's (i.e. anything other than HTTP+JSON) are **NOT** 480covered by SemVer and _will_ randomly change underneath you. **DON'T** interact 481with the Python/C++/etc code directly! 482 483FAQ 484--- 485 486### Is HMAC auth for requests/responses really necessary? 487 488Without the HMAC auth, it's possible for a malicious website to impersonate the 489user. Don't forget that evil.com can send requests to servers listening on 490localhost if the user visits evil.com in a browser. 491 492**This is not merely a theoretical concern**; a working proof-of-concept remote 493code execution exploit [was created][exploit] for ycmd running on localhost. The 494HMAC auth was added to block this attack vector. 495 496 497Contributor Code of Conduct 498--------------------------- 499 500Please note that this project is released with a [Contributor Code of 501Conduct][ccoc]. By participating in this project you agree to abide by its 502terms. 503 504Contact 505------- 506 507If you have questions about the plugin or need help, please use the 508[ycmd-users][] mailing list. 509 510The author's homepage is <http://val.markovic.io>. 511 512License 513------- 514 515This software is licensed under the [GPL v3 license][gpl]. 516© 2015-2019 ycmd contributors 517 518[ycmd-users]: https://groups.google.com/forum/?hl=en#!forum/ycmd-users 519[ycm]: http://ycm-core.github.io/YouCompleteMe/ 520[atom-you-complete-me]: https://atom.io/packages/you-complete-me 521[sublime-ycmd-completion]: https://packagecontrol.io/packages/YcmdCompletion 522[sublime-ycmd]: https://packagecontrol.io/packages/YouCompleteMe 523[semver]: http://semver.org/ 524[hmac]: http://en.wikipedia.org/wiki/Hash-based_message_authentication_code 525[exploit]: https://groups.google.com/d/topic/ycm-users/NZAPrvaYgxo/discussion 526[example-client]: https://github.com/ycm-core/ycmd/blob/master/examples/example_client.py 527[example-readme]: https://github.com/ycm-core/ycmd/blob/master/examples/README.md 528[trigger-defaults]: https://github.com/ycm-core/ycmd/blob/master/ycmd/completers/completer_utils.py#L143 529[subsequence]: http://en.wikipedia.org/wiki/Subsequence 530[ycm-install]: https://github.com/ycm-core/YouCompleteMe/blob/master/README.md#mac-os-x 531[def-settings]: https://github.com/ycm-core/ycmd/blob/master/ycmd/default_settings.json 532[base64]: http://en.wikipedia.org/wiki/Base64 533[mkstemp]: http://man7.org/linux/man-pages/man3/mkstemp.3.html 534[options]: https://github.com/ycm-core/YouCompleteMe#options 535[extra-conf-doc]: https://github.com/ycm-core/YouCompleteMe#c-family-semantic-completion 536[emacs-ycmd]: https://github.com/abingham/emacs-ycmd 537[gpl]: http://www.gnu.org/copyleft/gpl.html 538[kak-ycmd]: https://github.com/mawww/kak-ycmd 539[ccoc]: https://github.com/ycm-core/ycmd/blob/master/CODE_OF_CONDUCT.md 540[dev-setup]: https://github.com/ycm-core/ycmd/blob/master/DEV_SETUP.md 541[test-setup]: https://github.com/ycm-core/ycmd/blob/master/TESTS.md 542[extra-conf-vim-data-doc]: https://github.com/ycm-core/YouCompleteMe#the-gycm_extra_conf_vim_data-option 543[vscode-you-complete-me]: https://marketplace.visualstudio.com/items?itemName=RichardHe.you-complete-me 544[gycm]: https://github.com/jakeanq/gycm 545[nano-ycmd]: https://github.com/orsonteodoro/nano-ycmd 546[jdtls]: https://github.com/eclipse/eclipse.jdt.ls 547[jdt-formatter]: https://github.com/eclipse/eclipse.jdt.ls/blob/master/org.eclipse.jdt.ls.core/.settings/org.eclipse.jdt.core.prefs 548[ts-formatter]: https://github.com/Microsoft/TypeScript/blob/master/lib/protocol.d.ts#L2384-L2421 549[api-docs]: https://ycm-core.github.io/ycmd/ 550[ycmd-extra-conf]: https://github.com/ycm-core/ycmd/blob/master/.ycm_extra_conf.py 551[clangd]: https://clang.llvm.org/extra/clangd.html 552[gopls]: https://github.com/golang/tools/ 553[RLS]: https://github.com/rust-lang-nursery/rls 554