• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..30-Mar-2022-

.githooks/H30-Mar-2022-157

.github/workflows/H30-Mar-2022-454414

.metrics/H30-Mar-2022-329228

benchmarks/H03-May-2022-

js_parser/H30-Mar-2022-7,4575,678

jsparagus/H30-Mar-2022-9,5606,628

mozconfigs/H30-Mar-2022-1513

src/H30-Mar-2022-975798

tests/H30-Mar-2022-1,4481,223

.cargo-checksum.jsonH A D03-May-202227 11

.flake8H A D30-Mar-2022603 1612

CODE_OF_CONDUCT.mdH A D30-Mar-2022495 94

Cargo.lockH A D30-Mar-202226.2 KiB1,097979

Cargo.tomlH A D30-Mar-20221 KiB3730

LICENSEH A D30-Mar-2022636 1612

LICENSE-APACHE-2.0H A D30-Mar-202211.9 KiB219182

LICENSE-MITH A D30-Mar-20221 KiB84

MakefileH A D30-Mar-20223.4 KiB11280

README.mdH A D30-Mar-20226.4 KiB160112

journal.mdH A D30-Mar-202210.9 KiB273199

js-quirks.mdH A D30-Mar-202237.6 KiB1,037753

pgen.pgenH A D30-Mar-20221.7 KiB9075

smoosh_status.pyH A D30-Mar-20225.5 KiB194146

test.shH A D30-Mar-2022608 3222

update.shH A D30-Mar-2022886 275

update_stencil.pyH A D30-Mar-202219.7 KiB720537

update_unicode.pyH A D30-Mar-20229.5 KiB355277

README.md

1[![Rust][Rust Badge]][Rust CI Link]
2[![NotImplemented Counter][NotImplemented Badge]][NotImplemented Search]
3[![Fuzzbug days since][Fuzzbug Days Badge]][Fuzzbugs]
4[![Fuzzbug open][Fuzzbug Open Badge]][Open Fuzzbugs]
5[![SmooshMonkey Build Result][SmooshMonkey Build Badge]][SmooshMonkey Build TreeHerder]
6[![SmooshMonkey Test Result][SmooshMonkey Test Badge]][SmooshMonkey Test TreeHerder]
7
8# jsparagus - A JavaScript parser written in Rust
9
10jsparagus is intended to replace the JavaScript parser in Firefox.
11
12Current status:
13
14*   jsparagus is not on crates.io yet. The AST design is not stable
15    enough.  We do have a build of the JS shell that includes jsparagus
16    as an option (falling back on C++ for features jsparagus doesn't
17    support). See
18    [mozilla-spidermonkey/rust-frontend](https://github.com/mozilla-spidermonkey/rust-frontend).
19
20*   It can parse a lot of JS scripts, and will eventually be able to parse everything.
21    See the current limitations below, or our GitHub issues.
22
23*   Our immediate goal is to [support parsing everything in Mozilla's JS
24    test suite and the features in test262 that Firefox already
25    supports](https://github.com/mozilla-spidermonkey/jsparagus/milestone/1).
26
27Join us on Discord: https://discord.gg/tUFFk9Y
28
29
30## Building jsparagus
31
32To build the parser by itself:
33
34```sh
35make init
36make all
37```
38
39The build takes about 3 minutes to run on my laptop.
40
41When it's done, you can:
42
43*   Run `make check` to make sure things are working.
44
45*   `cd crates/driver && cargo run -- -D` to try out the JS parser and bytecode emitter.
46
47
48## Building and running SpiderMonkey with jsparagus
49
50*   To build SpiderMonkey with jsparagus, `configure` with `--enable-smoosh`.
51
52    This builds with a specific known-good revision of jsparagus.
53
54*   Building SpiderMonkey with your own local jsparagus repo, for
55    development, takes more work; see [the jsparagus + SpiderMonkey wiki
56    page](https://github.com/mozilla-spidermonkey/jsparagus/wiki/SpiderMonkey)
57    for details.
58
59**NOTE: Even after building with jsparagus, you must run the shell with
60`--smoosh`** to enable jsparagus at run time.
61
62
63
64## Benchmarking
65
66### Fine-grain Benchmarks
67
68Fine-grain benchmarks are used to detect regression by focusing on each part of
69the parser at one time, exercising only this one part. The benchmarks are not
70meant to represent any real code sample, but to focus on executing specific
71functions of the parser.
72
73To run this parser, you should execute the following command at the root of the
74repository:
75
76```sh
77cd crates/parser
78cargo bench
79```
80
81### Real-world JavaScript
82
83Real world benchmarks are used to track the overall evolution of performance over
84time. The benchmarks are meant to represent realistic production use cases.
85
86To benchmark the AST generation, we use SpiderMonkey integration to execute the
87parser and compare it against SpiderMonkey's default parser. Therefore, to run
88this benchmark, we have to first compile SpiderMonkey, then execute SpiderMonkey
89shell on the benchmark. (The following instructions assume that `~` is the
90directory where all projects are checked out)
91
92* Generate Parse Tables:
93
94  ```sh
95  cd ~/jsparagus/
96  make init
97  make all
98  ```
99
100* Compile an optimized version of [SpiderMonkey's JavaScript shell](https://github.com/mozilla/gecko-dev):
101
102  ```sh
103  cd ~/mozilla/js/src/
104  # set the jsparagus' path to the abosulte path to ~/jsparagus.
105  $EDITOR frontend/smoosh/Cargo.toml
106  ../../mach vendor rust
107  # Create a build directory
108  mkdir obj.opt
109  cd obj.opt
110  # Build SpiderMonkey
111  ../configure --enable-nspr-build --enable-smoosh --enable-debug-symbols=-ggdb3 --disable-debug --enable-optimize --enable-release --disable-tests
112  make
113  ```
114
115* Execute the [real-js-samples](https://github.com/nbp/real-js-samples/) benchmark:
116
117  ```sh
118  cd ~/real-js-samples/
119  ~/mozilla/js/src/obj.opt/dist/bin/js ./20190416.js
120  ```
121
122This should return the overall time taken to parse all the Script once, in the
123cases where there is no error. The goal is to minimize the number of
124nano-seconds per bytes.
125
126
127## Limitations
128
129It's *all* limitations, but I'll try to list the ones that are relevant
130to parsing JS.
131
132*   Features that are not implemented in the parser yet include `let`,
133    `import` and `export`, `async` functions, `yield` expressions, the
134    use of `await` and `yield` as identifiers, template strings,
135    `BigInt`, Unicode escape sequences that evaluate to surrogate code
136    points, legacy octal integer literals, legacy octal escape
137    sequences, some RegExp flags, strict mode code, `__proto__` in
138    object literals, some features of destructuring assignment.
139
140    Many more features are not yet supported in the bytecode emitter.
141
142*   Error messages are poor.
143
144We're currently working on parser performance and completeness, as well
145as the bytecode emitter and further integration with SpiderMonkey.
146
147
148[Rust Badge]: https://github.com/mozilla-spidermonkey/jsparagus/workflows/Rust/badge.svg
149[Rust CI Link]: https://github.com/mozilla-spidermonkey/jsparagus/actions?query=branch%3Amaster
150[NotImplemented Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fnot-implemented.json
151[NotImplemented Search]: https://github.com/mozilla-spidermonkey/jsparagus/search?q=notimplemented&unscoped_q=notimplemented
152[Fuzzbug days Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fsince-last-fuzzbug.json
153[Fuzzbug Open Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fopen-fuzzbug.json
154[Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/issues?utf8=%E2%9C%93&q=label%3AlibFuzzer+
155[Open Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/labels/libFuzzer
156[SmooshMonkey Build Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_build.json
157[SmooshMonkey Build TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-nonunified
158[SmooshMonkey Test Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_test.json
159[SmooshMonkey Test TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-smoosh
160