1[![Rust][Rust Badge]][Rust CI Link] 2[![NotImplemented Counter][NotImplemented Badge]][NotImplemented Search] 3[![Fuzzbug days since][Fuzzbug Days Badge]][Fuzzbugs] 4[![Fuzzbug open][Fuzzbug Open Badge]][Open Fuzzbugs] 5 6# Metrics 7 8This is the metrics directory. It follows the evolution of the repository separately from the 9repostory. You can find the actual metrics in the 10[`ci-results`](https://github.com/mozilla-spidermonkey/jsparagus/tree/ci-results) branch of the jsparagus project. This branch is automatically generated using the `create-ci-branch.sh` script found in this directory. If there are issues with your fork, you can remove the `ci-results` branch, and the ci will automatically rerun the `create-ci-branch` script to reset it. Do not push manula data to this repository, it will be lost. 11 12If you find that the `ci-results` branch has disappeared or been corrupted somehow, you can reset it by deleting it and recreating it. 13 14``` 15git branch -D ci-results 16cd .metrics 17./create-ci-branch.sh 18``` 19 20The `create-ci-branch.sh` file creates the branch, prepares it, and populates it with data from the past. 21 22## Making your own metrics 23Make sure you do not use data that can not be automatically recovered. We cannot rely on the `ci-results` branch always being present, therefore anything that you write must be recoverable on its own, either by relying on external APIs or through some other mechanism. 24 25Please update this README if you make any changes. 26 27## Types of CI Actions 28These actions are all found in the `.github/workflows` directory 29 301) `Rust.yml` - Run on Pull Request 31* runs every time there is a push to master, use for any metrics that are development related. Examples include linting, testing, etc. 322) `ci-push.yml` - Run on Push to `master` 33* runs on self contained metrics. An example is the number of `NotImplemented` errors in the codebase. This does not depend on anything external 343) `ci-daily.yml` - Run Daily 35* a cron task that runs daily. Useful for metrics that need daily updates 364) `ci-issue.yml` - Run on issue open 37* runs each time an issue is opened. Good for tracking types of issues. 38 39 40## Types of data 41 42These are the types of data that this metrics folder tracks. 43 441) Rust Passing 45 * Ensures our internal tests are passing 46 * Updates on every pull request to master. See [this 47 action](https://github.com/mozilla-spidermonkey/jsparagus/tree/master/.github/workflows/rust.yml) 48 492) NotImplemented Count 50 * counts number of NotImplemented errors in the codebase. This should slowly rundown to zero 51 * Updates on every push to master. See [this 52 action](https://github.com/mozilla-spidermonkey/jsparagus/tree/master/.github/workflows/ci-push.yml) 53 543) Days Since last Fuzzbug 55 * tracks the last fuzzbug we saw, if it does not exist, return ∞, otherwise return the last date regardless of state. 56 * Updates daily, regardless of push. See [this 57 action](https://github.com/mozilla-spidermonkey/jsparagus/tree/master/.github/workflows/ci-daily.yml) 58 594) Fuzzbug open count 60 * tracks the number of open fuzzbugs 61 * Updates on issue open. See [this action](https://github.com/mozilla-spidermonkey/jsparagus/.github/workflows/ci-issues.yml) 62 635) Percentage of tests passing with SmooshMonkey 64 * TODO: tracks the number of tests passing without fallback. We should use the try api for this. 65 * Updates daily, regardless of push. See [this 66 action](https://github.com/mozilla-spidermonkey/jsparagus/tree/master/.github/workflows/ci-daily.yml) 67 686) Percentage of JS compilable with SmooshMonkey 69 * TODO: see comment about writing bytes to a file in [this repo](https://github.com/nbp/seqrec) 70 * implementation is dependant on how we get the data. We need a robust solution for importing this data. 71 72[Rust Badge]: https://github.com/mozilla-spidermonkey/jsparagus/workflows/Rust/badge.svg 73[Rust CI Link]: https://github.com/mozilla-spidermonkey/jsparagus/actions?query=branch%3Amaster 74[NotImplemented Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fnot-implemented.json 75[NotImplemented Search]: https://github.com/mozilla-spidermonkey/jsparagus/search?q=notimplemented&unscoped_q=notimplemented 76[Fuzzbug days Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fsince-last-fuzzbug.json 77[Fuzzbug Open Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fopen-fuzzbug.json 78[Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/issues?utf8=%E2%9C%93&q=label%3AlibFuzzer+ 79[Open Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/labels/libFuzzer 80