1= Hacking the compiler �� 2 3This document is a work-in-progress attempt to provide useful 4information for people willing to inspect or modify the compiler 5distribution's codebase. Feel free to improve it by sending change 6proposals for it. 7 8If you already have a patch that you would like to contribute to the 9official distribution, please see link:CONTRIBUTING.md[]. 10 11=== Your first compiler modification 12 131. Create a new git branch to store your changes. 14+ 15---- 16git checkout -b my-modification 17---- 18 192. Consult link:INSTALL.adoc[] for build instructions. Here is the gist of it: 20+ 21---- 22./configure 23make world.opt 24---- 25 263. Try the newly built compiler binaries `ocamlc`, `ocamlopt` or their 27`.opt` version. To try the toplevel, use: 28+ 29---- 30make runtop 31---- 32 334. Hack frenetically and keep rebuilding. 34 355. Run the testsuite from time to time. 36+ 37---- 38make tests 39---- 40 415. Install in a new opam switch to try things out: 42+ 43---- 44opam compiler-conf install 45---- 46 476. You did it, Well done! Consult link:CONTRIBUTING.md[] to send your contribution upstream. 48 49See our <<Development tips and tricks>> for various helpful details, 50for example on how to automatically <<opam compiler script,create an 51opam switch>> from a compiler branch. 52 53=== What to do 54 55There is always a lot of potential tasks, both for old and 56newcomers. Here are various potential projects: 57 58* http://caml.inria.fr/mantis/view_all_bug_page.php[The OCaml 59 bugtracker] contains reported bugs and feature requests. Some 60 changes that should be accessible to newcomers are marked with the 61 tag 62 http://caml.inria.fr/mantis/search.php?project_id=1&sticky_issues=1&sortby=last_updated&dir=DESC&highlight_changed=24&hide_status_id=90&tag_string=junior_job[junior_job]. 63 64* The 65 https://github.com/ocamllabs/compiler-hacking/wiki/Things-to-work-on[OCaml 66 Labs compiler-hacking wiki] contains various ideas of changes to 67 propose, some easy, some requiring a fair amount of work. 68 69* Documentation improvements are always much appreciated, either in 70 the various `.mli` files or in the official manual 71 (See link:manual/README.md[]). If you invest effort in understanding 72 a part of the codebase, submitting a pull request that adds 73 clarifying comments can be an excellent contribution to help you, 74 next time, and other code readers. 75 76* The https://github.com/ocaml/ocaml[github project] contains a lot of 77 pull requests, many of them being in dire need of a review -- we 78 have more people willing to contribute changes than to review 79 someone else's change. Picking one of them, trying to understand the 80 code (looking at the code around it) and asking questions about what 81 you don't understand or what feels odd is super-useful. It helps the 82 contribution process, and it is also an excellent way to get to know 83 various parts of the compiler from the angle of a specific aspect or 84 feature. 85+ 86Again, reviewing small or medium-sized pull requests is accessible to 87anyone with OCaml programming experience, and helps maintainers and 88other contributors. If you also submit pull requests yourself, a good 89discipline is to review at least as many pull requests as you submit. 90 91== Structure of the compiler 92 93The compiler codebase can be intimidating at first sight. Here are 94a few pointers to get started. 95 96=== Compilation pipeline 97 98==== The driver -- link:driver/[] 99 100The driver contains the "main" function of the compilers that drive 101compilation. It parses the command-line arguments and composes the 102required compiler passes by calling functions from the various parts 103of the compiler described below. 104 105==== Parsing -- link:parsing/[] 106 107Parses source files and produces an Abstract Syntax Tree (AST) 108(link:parsing/parsetree.mli[] has lot of helpful comments). See 109link:parsing/HACKING.adoc[]. 110 111The logic for Camlp4 and Ppx preprocessing is not in link:parsing/[], 112but in link:driver/[], see link:driver/pparse.mli[], 113link:driver/pparse.mli[]. 114 115==== Typing -- link:typing/[] 116 117Type-checks the AST and produces a typed representation of the program 118(link:parsing/typedtree.mli[] has some helpful comments). See 119link:typing/HACKING.adoc[]. 120 121==== The bytecode compiler -- link:bytecomp/[] 122 123==== The native compiler -- link:middle_end/[] and link:asmcomp/[] 124 125=== Runtime system 126 127=== Libraries 128 129link:stdlib/[]:: The standard library. Each file is largely 130independent and should not need further knowledge. 131 132link:otherlibs/[]:: External libraries such as `unix`, `threads`, 133`dynlink`, `str` and `bigarray`. 134 135=== Tools 136 137link:lex/[]:: The `ocamllex` lexer generator. 138 139link:yacc/[]:: The `ocamlyacc` parser generator. We do not recommend 140using it for user projects in need of a parser generator. Please 141consider using and contributing to 142link:http://gallium.inria.fr/~fpottier/menhir/[menhir] instead, which 143has tons of extra features, lets you write more readable grammars, and 144has excellent documentation. 145 146=== Complete file listing 147 148 Changes:: what's new with each release 149 configure:: configure script 150 CONTRIBUTING.md:: how to contribute to OCaml 151 HACKING.adoc:: this file 152 INSTALL.adoc:: instructions for installation 153 LICENSE:: license and copyright notice 154 Makefile:: main Makefile 155 Makefile.nt:: Windows Makefile (deprecated) 156 Makefile.shared:: common Makefile 157 Makefile.tools:: used by manual/ and testsuite/ Makefiles 158 README.adoc:: general information on the compiler distribution 159 README.win32.adoc:: general information on the Windows ports of OCaml 160 VERSION:: version string 161 asmcomp/:: native-code compiler and linker 162 asmrun/:: native-code runtime library 163 boot/:: bootstrap compiler 164 bytecomp/:: bytecode compiler and linker 165 byterun/:: bytecode interpreter and runtime system 166 compilerlibs/:: the OCaml compiler as a library 167 config/:: configuration files 168 debugger/:: source-level replay debugger 169 driver/:: driver code for the compilers 170 emacs/:: editing mode and debugger interface for GNU Emacs 171 experimental/:: experiments not built by default 172 flexdll/:: git submodule -- see link:README.win32.adoc[] 173 lex/:: lexer generator 174 man/:: man pages 175 manual/:: system to generate the manual 176 middle_end/:: the flambda optimisation phase 177 ocamldoc/:: documentation generator 178 otherlibs/:: several additional libraries 179 parsing/:: syntax analysis -- see link:parsing/HACKING.adoc[] 180 stdlib/:: standard library 181 testsuite/:: tests -- see link:testsuite/HACKING.adoc[] 182 tools/:: various utilities 183 toplevel/:: interactive system 184 typing/:: typechecking -- see link:typing/HACKING.adoc[] 185 utils/:: utility libraries 186 yacc/:: parser generator 187 188== Development tips and tricks 189 190=== opam compiler script 191 192The separately-distributed script 193https://github.com/gasche/opam-compiler-conf[`opam-compiler-conf`] can 194be used to easily build opam switches out of a git branch of the 195compiler distribution. This lets you easily install and test opam 196packages from an under-modification compiler version. 197 198=== Useful Makefile targets 199 200Besides the targets listed in link:INSTALL.adoc[] for build and 201installation, the following targets may be of use: 202 203`make runtop` :: builds and runs the ocaml toplevel of the distribution 204 (optionally uses `rlwrap` for readline+history support) 205`make natruntop`:: builds and runs the native ocaml toplevel (experimental) 206 207`make partialclean`:: Clean the OCaml files but keep the compiled C files. 208 209`make depend`:: Regenerate the `.depend` file. Should be used each time new dependencies are added between files. 210 211`make -C testsuite parallel`:: see link:testsuite/HACKING.adoc[] 212 213=== Bootstrapping 214 215The OCaml compiler is bootstrapped. This means that 216previously-compiled bytecode versions of the compiler, dependency 217generator and lexer are included in the repository under the 218link:boot/[] directory. These bytecode images are used once the 219bytecode runtime (which is written in C) has been built to compile the 220standard library and then to build a fresh compiler. Details can be 221found in link:INSTALL.adoc#bootstrap[INSTALL.adoc]. 222 223=== Continuous integration 224 225==== Github's CI: Travis and AppVeyor 226 227==== INRIA's Continuous Integration (CI) 228 229INRIA provides a Jenkins continuous integration service that OCaml 230uses, see link:https://ci.inria.fr/ocaml/[]. It provides a wider 231architecture support (MSVC and MinGW, a zsystems s390x machine, and 232various MacOS versions) than the Travis/AppVeyor testing on github, 233but only runs on commits to the trunk or release branches, not on every 234PR. 235 236You do not need to be an INRIA employee to open an account on this 237jenkins service; anyone can create an account there to access build 238logs, enable email notifications, and manually restart builds. If you 239would like to do this but have trouble doing it, please contact Damien 240Doligez or Gabriel Scherer. 241 242==== Running INRIA's CI on a github Pull Request (PR) 243 244If you have suspicions that a PR may fail on exotic architectures 245(it touches the build system or the backend code generator, 246for example) and would like to get wider testing than github's CI 247provides, it is possible to manually start INRIA's CI on arbitrary git 248branches by pushing to a `precheck` branch of the main repository. 249 250This is done by pushing to a specific github repository that the CI 251watches, namely 252link:https://github.com/ocaml/precheck[ocaml/precheck]. You thus need 253to have write/push/commit access to this repository to perform this operation. 254 255Just checkout the commit/branch you want to test, then run 256 257 git push --force git@github.com:ocaml/precheck.git HEAD:trunk 258 259(This is the syntax to push the current `HEAD` state into the `trunk` 260reference on the specified remote.)