1# Schedule
2
3For each CI config change, we need to follow:
4 * scope of work (what will run, how frequently)
5 * capacity planning (cost, physical space limitations)
6 * will this replace anything or is this 100% new
7 * puppet/deployment scripts or documentation
8 * setup pool on try server
9 * documented updated on this page, communicate with release management and others as appropriate
10
11
12## Current / Future CI config changes
13
14Start Date | Completed | Tracking Bug | Description
15--- | --- | --- | ---
16October 2020  | TBD | [Bug 1665012](https://bugzilla.mozilla.org/show_bug.cgi?id=1665012) | add samsung S7 phones for perf testing
17November 2020 | TBD | [Bug 1676850](https://bugzilla.mozilla.org/show_bug.cgi?id=1676850) | Windows tests migrate from AWS -> Datacenter/Azure and 1803 -> 1903
18November 2020 | TBD | TBD | upgrade datacenter linux perf machines from ubuntu 16.04 to 18.04
19TBD  | TBD | [Bug 1665012](https://bugzilla.mozilla.org/show_bug.cgi?id=1665012) | Android phones upgrade from version 7 -> 10
20October 2020  | TBD | [Bug 1673067](https://bugzilla.mozilla.org/show_bug.cgi?id=1673067) | Run tests on MacOSX BigSur (subset in parallel)
21October 2020  | TBD | [Bug 1673067](https://bugzilla.mozilla.org/show_bug.cgi?id=1673067) | Run tests on MacOSX Aarch64 (subset in parallel)
22December 2020 | TBD | TBD | Migrate OSX from Mac Mini R7, OSX 10.14 (Mojave) -> Mac Mini R8, OSX 10.15 (Catalina)
23TBD  | TBD | TBD | Migrate more coverage of OSX from 10.14 to BigSur/aarch64
24TBD  | TBD | TBD | Upgrade ubuntu from 18.04 to 20.04
25TBD  | TBD | TBD | Upgrade android emulators to modern version
26September 2020 | TBD | [Bug 1548264](https://bugzilla.mozilla.org/show_bug.cgi?id=1548264) | Python 2.7 -> 3.6 migration in CI
27TBD  | TBD | [Bug 1665010](https://bugzilla.mozilla.org/show_bug.cgi?id=1665010) | Add more android phone hardware (replace moto g5 and probably pixel 2)
28TBD  | TBD | TBD | Upgrade datacenter hardware for windows/linux (primarily perf)
29TBD  | TBD | TBD | Add Linux ARM64 worker in AWS (as it is close to Apple Silicon)
30
31
32## Completed CI config changes
33
34Start Date | Completed | Tracking Bug | Description
35--- | --- | --- | ---
36July 2020 | October 2020| [Bug 1653344](https://bugzilla.mozilla.org/show_bug.cgi?id=1653344) | Remove EDID dongles from MacOSX machines
37August 2020 | September 2020 | [Bug 1643689](https://bugzilla.mozilla.org/show_bug.cgi?id=1643689) | Schedule tests by test selection/manifest
38June 2020 | August 2020 | [Bug 1486004](https://bugzilla.mozilla.org/show_bug.cgi?id=1486004) | Android hardware tests running without rooted phones
39August 2019 | January 2020 | [Bug 1572242](https://bugzilla.mozilla.org/show_bug.cgi?id=1572242) | Upgrade Ubuntu from 16.04 to 18.04 (finished in January)
40
41
42## Appendix:
43 * *OS*: base operating system such as Android, Linux, Mac OSX, Windows
44 * *Hardware*: specific cpu/memory/disk/graphics/display/inputs that we are using, could be physical hardware we own or manage, or it could be a cloud provider.
45 * *Platform*: a combination of hardware and OS
46 * *Configuration*: what we change on a platform (can be runtime with flags), installed OS software updates (service pack), tools (python/node/etc.), hardware or OS settings (anti aliasing, display resolution, background processes, clipboard), environment variables,
47 * *Test Failure*: a test doesn’t report the expected result (if we expect fail and we crash, that is unexpected).  Typically this is a failure, but it can be a timeout, crash, not run, or even pass
48 * *Greening up*: Assuming all tests return expected results (passing), they are green.  When tests fail, they are orange.  We need to find a way to get all tests green by investigating test failures.
49