README.md
1# XR Instrumentation Tests
2
3## TL;DR For Most Local Repros
4
51. Get a rooted Pixel device of some sort.
62. Make sure "VR Services" is up to date in the Playstore.
73. Set lock screen timeout to at least 5 minutes. If screen is locked or device
8 goes to sleep while tests are still running, they will fail.
94. Run `ninja -C out/Debug chrome_public_test_vr_apk
10 && out/Debug/bin/run_chrome_public_test_vr_apk
11 --num-retries=0
12 --shared-prefs-file=//chrome/android/shared_preference_files/test/vr_ddview_skipdon_setupcomplete.json
13 --test-filter=<failing test case>`
14 Don't touch phone while the tests are running.
15
16If you are reproducing an issue with the AR tests, run
17`export DOWNLOAD_VR_TEST_APKS=1 && gclient runhooks` in order to get the
18playback datasets that are necessary.
19
20**NOTE** The message "Main Unable to find package info for org.chromium.chrome"
21 is usually displayed when the test package is being installed and does
22 not indicate any problem.
23
24## Introduction
25
26This directory contains all the Java-side infrastructure for running
27instrumentation tests for [WebXR][webxr_spec]
28(VR/Virtual Reality and AR/Augmented Reality) features currently in Chrome.
29
30These tests are integration/end-to-end tests run in the full Chromium browser on
31actual Android devices.
32
33## Directories
34
35These are the files and directories that are relevant to XR instrumentation
36testing.
37
38### Subdirectories
39
40* `mock/` - Contains all the classes for mock implementations of XR classes.
41* `rules/` - Contains all the XR-specific JUnit4 rules for handling
42functionality such as running tests multiple times in different activities and
43handling the fake VR pose tracker service.
44* `util/` - Contains utility classes with code that is used by multiple test
45classes and that does not make sense to include in the core test framework.
46
47### Other Directories
48
49* [`//chrome/android/shared_preferences_files/test`][shared_prefs_dir] -
50Contains the VrCore settings files for running VR instrumentation tests (see the
51"Building and Running" section for more information).
52* [`//chrome/test/data/xr/e2e_test_files/`][html_dir] - Contains the JavaScript
53and HTML files for XR instrumentation tests.
54* [`//third_party/gvr-android-sdk/test-apks`][vr_test_apks] - Contains the VR
55APKs used for testing, such as VrCore. You must have `DOWNLOAD_VR_TEST_APKS` set
56as an environment variable when you run gclient runhooks in order to actually
57download these from storage.
58* [`//third_party/gvr-android-sdk/test-libraries`][vr_test_libraries] - Contains
59third party VR testing libraries. Currently, only has the Daydream controller
60test library used for sending controller events to VrCore using intents.
61* [`//third_party/arcore-android-sdk/test-apks`][ar_test_apks] - Contains the AR
62APKs used for testing, such as ArCore. You must have `DOWNLOAD_VR_TEST_APKS` set
63as an environment variable when you run gclient runhooks in order to actually
64download these from storage.
65
66## Building
67
68### AR
69
70The AR instrumentation tests can be built with the
71`monochrome_public_test_ar_apk` target, which will also build
72`monochrome_public_apk` to test with.
73
74### VR
75
76The VR instrumentation tests can be built with the `chrome_public_test_vr_apk`
77target, which will also build `chrome_public_apk` to test with.
78
79## Running
80
81Both the VR and AR tests are run using the generated script in your build output
82directory's `bin/` directory, e.g. `out/foo/bin/run_chrome_public_test_vr_apk`
83to run the VR tests. You will likely need to pass some or all of the following
84arguments in order for the tests to run properly, though.
85
86**NOTE** The instrumentation tests can only be run on rooted devices.
87
88### Common Arguments
89
90#### additional-apk
91
92`--additional-apk path/to/apk/to/install`
93
94Installs the specified APK before running the tests. No-ops if the provided APK
95is already installed on the device and their versions match.
96
97For VR tests, you'll likely want to use `--additional-apk
98third_party/gvr-android-sdk/test-apks/vr_services/vr_services_current.apk` to
99ensure that the VrCore version used is the one used for automated testing at
100whatever your current git revision is.
101
102For AR tests, you'll likely want to use `--additional-apk
103third_party/arcore-android-sdk/test-apks/arcore/arcore_current.apk` to ensure
104that the ArCore version used is the one used for automated testing at whatever
105your current git revision is.
106
107**NOTE** Using this argument for VR on most Pixel devices will fail, as VrCore
108is pre-installed as a system app. This can be dealt with in the following ways:
109
110* Remove VrCore as a system app by following the instructions
111 [here](go/vrcore/building-and-running). This will permanently solve the issue
112 unless you reflash your device.
113* Use `--replace-system-package
114 com.google.vr.vrcore,//third_party/gvr-android-sdk/test-apks/vr_services/vr_services_current.apk`
115 instead. This will take significantly longer, as it requires rebooting, and
116 must be done every time you run the tests.
117* Skip this argument entirely and just ensure that the VrCore version on the
118 device is up to date via the Play Store.
119
120#### test-filter
121
122`--test-filter TestClass#TestCase`
123
124Allows you to limit the set of tests run to a particular test class or subset of
125tests within a test class. Use of the `*` wildcard is supported, e.g.
126`--test-filter VrBrowserTransitionTest#*` will run all tests in the
127VrBrowserTransitionTest class.
128
129#### local-output/json-results-file
130
131`--local-output --json-results-file output.json`
132
133Sets the test runner to generate a local results summary after running all tests
134and print out a file URL pointing to the summary. This allows you to view both
135logcat output for a particular test and its post-failure screenshot.
136
137#### num-retries
138
139`--num-retries <#>`
140
141Sets the test runner to retry failed tests a certain number of times. The
142default is 2, resulting in a max of 3 test runs. Usually used as `--num-retries
1430` when debugging to reduce test runtime and make flakiness more visible.
144
145#### repeat
146
147`--repeat <#>`
148
149Sets the test runner to repeat the tests a certain number of times. The default
150is 0, resulting in only one iteration. Usually used to repeat a test many times
151in order to check for or reproduce flakiness.
152
153### VR-Specific Arguments
154
155#### shared-prefs-file
156
157`--shared-prefs-file path/to/preference/json/file`
158
159Configures VrCore according to the provided file, e.g. changing the paired
160headset. The currently supported files are:
161
162* `//chrome/android/shared_preference_files/test/vr_cardboard_skipdon_setupcomplete.json`
163 will cause all cardboard-compatible tests to run. This will pair the device
164 with a Cardboard headset and disable controller emulation.
165* `//chrome/android/shared_preference_files/test/vr_ddview_skipdon_setupcomplete.json`
166 will cause most Daydream View-compatible tests to run, with the exception of
167 those that require the DON flow to be enabled. This will pair the device with
168 a Daydream View headset, set the DON flow to be skipped, and enable controller
169 emulation.
170* `//chrome/android/shared_preference_files/test/vr_ddview_don_setupcomplete.json`
171 combined with the extra `--vr-don-enabled` and
172 `--annotation=Restriction=VR_DON_Enabled` flags will cause all tests that
173 are using the `RESTRICTION_TYPE_VR_DON_ENABLED` restriction to run. This runs
174 tests that expect to have the DON flow enabled.
175
176The test runner will automatically revert any changed settings back to their
177pre-test values after the test suite has completed. If for whatever reason you
178want to manually apply settings outside of a test, you can do so with
179`//build/android/apply_shared_preference_file.py`.
180
181## Adding New Tests
182
183See [adding_new_tests.md][adding_new_tests].
184
185[webxr_spec]: https://immersive-web.github.io/webxr-samples/explainer.html
186[shared_prefs_dir]:
187https://chromium.googlesource.com/chromium/src/+/master/chrome/android/shared_preference_files/test
188[html_dir]: https://chromium.googlesource.com/chromium/src/+/master/chrome/test/data/xr/e2e_test_files
189[vr_test_apks]: https://chromium.googlesource.com/chromium/src/+/master/third_party/gvr-android-sdk/test-apks
190[vr_test_libraries]: https://chromium.googlesource.com/chromium/src/+/master/third_party/gvr-android-sdk/test-libraries
191[ar_test_apks]: https://chromium.googlesource.com/chromium/src/+/master/third_party/arcore-android-sdk/test-apks
192[adding_new_tests]:
193https://chromium.googlesource.com/chromium/src/+/master/chrome/android/javatests/src/org/chromium/chrome/browser/vr/adding_new_tests.md
194