1# Switch Access (for developers) 2 3Switch Access is a Chrome OS feature to control the computer with 1 or 2 4**switches**, or button-like inputs. It is targeted at supporting users with 5motor or mobility impairments, for whom other methods of controlling the device 6are not feasible. 7 8 9## Using Switch Access 10 11Go to Chrome OS settings > Accessibility settings > "Manage accessibility 12features" > Enable "Switch Access". You can assign switches to actions and 13enable or disable automatic scanning on the Switch Access settings subpage. 14For most development purposes, buttons on either the built-in keyboard or an 15external keyboard can be used as switches, rather than needing a dedicated 16switch device. 17 18With this feature enabled, you can navigate to and interact with **actionable** 19elements (or **nodes**) onscreen. Because there are so many nodes onscreen at 20any given moment, they are organized into a nested system of **groups** based on 21proximity and other semantic information. 22 23 24### Navigation 25 26Switch Access supports two methods of navigation between nodes: manual scanning 27and automatic scanning (or **auto-scan**). Manual scanning means the user 28navigates from one element to the next by pressing one of their switches. 29Automatic scanning means that Switch Access moves from one element to the next 30after a set period of time (the **scanning speed**). 31 32The user can identify which element is currently focused because it is 33surrounded by a **focus ring**, which is two concentric rounded rectangles of 34contrasting colors (currently fixed at light blue and black) that surround the 35element. Sometimes, a second focus ring will appear onscreen, which has dashed 36rather than solid lines. This dashed focus ring provides a preview of which node 37will be focused, if the user selects the current node. It also provides a hint 38that the current node is a navigational node, and is not directly actionable. 39 40There are two types of navigational nodes: nodes that group other nodes 41together, and the **back button**. The back button allows a user to exit the 42current group, and move outwards towards the top-level node. 43 44 45### Selection 46 47All users have one switch dedicated to selecting elements. For some users, this 48is the only input they have (they rely on auto-scan to advance to the next 49node). So a user needs to be able to perform any action based on some series of 50select actions. To support this, when multiple actions are available for a 51single actionable node, we open the **action menu**, which displays the 52available actions for the given node. The user can then navigate to the action 53they wish to perform, and when they select an action the action is performed, 54and in most cases the menu is closed. 55 56Sometimes there is only one action available for a given node. In this case, the 57action menu does not open. Instead, the available action is performed 58automatically when the user presses *select*. 59 60 61## Reporting bugs 62 63Use bugs.chromium.org, filing bugs under the component 64[OS>Accessibility>SwitchAccess](https://bugs.chromium.org/p/chromium/issues/list?sort=-opened&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&q=component%3AOS%3EAccessibility%3ESwitchAccess%20&can=2). 65 66## Developing 67 68### Code location 69 70Switch Access code lives mainly in four places: 71 72- A component extension to do the bulk of the logic and processing, 73`chrome/browser/resources/chromeos/accessibility/switch_access/` 74 75- In the `AccessibilityEventRewriter`, 76`ash/events/accessibility_event_rewriter.h` 77 78- The Switch Access menu and back button code, in `ash/system/accessibility/` 79 80- The Switch Access settings page, 81`chrome/browser/resources/settings/chromeos/os_a11y_page/switch_access_subpage.*` 82 83 84### Tests 85 86Tests are in `unit_tests`, `ash_unittests`, and `browser_tests`: 87 88``` 89out/Release/unit_tests --gtest_filter="*SwitchAccess*" 90out/Release/ash_unittests --gtest_filter="*SwitchAccess*" 91out/Release/browser_tests --gtest_filter="*SwitchAccess*" 92``` 93 94 95### Debugging 96 97Developers can add log lines to any of the C++ files and see output in the 98console. To debug the Switch Access extension, the easiest way is from an 99external browser. Start Chrome OS on Linux with this command-line flag: 100 101``` 102out/Release/chrome --remote-debugging-port=9222 103``` 104 105Now open http://localhost:9222 in a separate browser, and debug the Switch 106Access extension background page from there. 107 108 109## How it works 110 111Like [Chromevox](chromevox.md) and [Select to Speak](select_to_speak.md), 112Switch Access is implemented mainly as a component Chrome extension which 113is always loaded and running in the background when enabled, and unloaded 114when disabled. Unlike Chromevox and Select to Speak, the settings for 115Switch Access are not located within the component extension. Instead, 116they are found with the other settings in the Settings app. 117 118There are a handful of tasks which, for various reasons, are handled in 119C++ code. These are: 120 121- Event forwarding. In order to capture the events before other system 122functions, we use an Event Rewriter (shared with Chromevox). 123 124- Rendering the menu. To allow the menu UI to match other system UI, the process 125of creating the menu UI is done in C++. 126 127The Switch Access extension does the following, at a high level: 128 1291. Listens for SwitchAccessCommands, which are the user-initiated commands like 130"select" or "next" that the user performs by pressing their switch. 131 1322. Maps those commands to the appropriate behavior: 133 134 - If it is a navigation command, the focused node is changed and focus rings 135 are updated. 136 137 - If it is a select command, the available actions are determined. 138 139If there is only one, it is performed (Note that for navigational nodes, this 140means entering or exiting a group). If more than one action is available, the 141menu is opened and focus jumps into the menu. 142 1433. Listens for focus events, and moves Switch Access focus to follow system 144focus. 145 1464. Listens for location changes and other tree changes that affect the current 147node or its parent or siblings, and update the focused node and focus rings when 148necessary. 149 150 151## Switch Access extension structure 152 153### Managers 154 155Switch Access divides tasks by function, each being handled by a manager class. 156This section details what those managers are, their guiding principle, and what 157tasks they are responsible for. 158 159#### ActionManager 160 161The ActionManager is responsible for what happens after a user presses their 162*Select* switch. This includes determining what actions are available, 163performing actions, and specifying what actions to display in and where to show 164the action menu and sub-menus when needed. 165 166The specifics of opening and closing the menu are not handled here, but are 167encapsulated in the MenuManager. However, all calls to open or close the menu 168should be made on the ActionManager, rather than the MenuManager directly, 169because the ActionManager maintains state about sub-menus and similar, which 170needs to remain in sync with the menu displayed. 171 172#### AutoScanManager 173 174The AutoScanManager handles the process of automatically scanning from one 175element to the next. If the user has enabled auto-scan, it sets an interval 176to call `NavigationManager.moveForward()` periodically. It also resets the 177interval each time a command is received, or a focus event causes the focused 178node to change. 179 180#### FocusRingManager 181 182The FocusRingManager determines what focus rings should be drawn and where to 183show them, and calls the `accessibilityPrivate` API. 184 185#### MenuManager 186 187The MenuManager handles the details of displaying the action menu, given a 188list of actions and a location. It also waits for the menu to load, and jumps 189Switch Access focus to the menu when it is ready. 190 191Calls to open or close a menu should be handled by the ActionManager, rather 192than by the MenuManager directly, to keep state in sync. 193 194#### NavigationManager 195 196The NavigationManager handles tracking and changing the current Switch Access 197focus. This includes handling moving forward or backward from a user command, 198moving forward from auto-scan, jumping to a UI element when opened (such as the 199keyboard or action menu), and following system focus. It also handles entering 200and exiting groups when navigational nodes are selected. 201 202The NavigationManager also has a method `getTreeForDebugging()` which can be 203used to print either the current group or the entire tree to the console, to 204help with debugging. 205 206#### PreferenceManager 207 208The PreferenceManager handles accessing user preferences, and updating 209Switch Access' behavior accordingly. It also verifies whether the user has 210a configuration that allows for full navigation. 211 212#### TextNavigationManager 213 214The TextNavigationManager handles tasks surrounding navigating through text. 215Currently only text within editable text fields is supported, and only behind 216the flag `enable-experimental-accessibility-switch-access-text`. 217 218 219### Nodes 220 221In addition to dividing the tasks by function, Switch Access delegates many 222tasks to the nodes themselves. This allows specific behaviors to be changed 223by changing what nodes are created based on the circumstances, and keeps the 224ActionManager and NavigationManager from getting bloated with special cases, 225and all the permutations of those special cases. 226 227All of the nodes used by Switch Access have some relation to an **automation 228node**, the underlying tree structure shared by all accessibility component 229extensions. Most have a one-to-one association (one automation node for each 230Switch Access node), but sometimes there are multiple Switch Access nodes for 231the same automation node (such as when breaking a long list into smaller 232groups). 233 234#### SAChildNode and SARootNode 235 236These are the two base types. They define the functions available on nodes 237when they are functioning as either a selectable node, or the current group. 238 239SAChildNode is an abstract class, but SARootNode can be used directly to 240represent a group that does not have an underlying automation node. 241 242#### BasicNode and BasicRootNode 243 244These types implement a default behavior for nodes that have a one-to-one 245association with an automation node. Information is derived directly from 246the automation node to implement the interface provided by SAChildNode and 247SARootNode. 248 249#### BackButtonNode 250 251The BackButtonNode is special because it is part of Switch Access' UI, and 252it is shown for each group. Therefore, it automatically finds the automation 253node corresponding to the back button by searching the tree, and instead 254takes the SARootNode for its group in its constructor. It uses this to 255determine where to show the back button, and what to do if it is selected. 256 257#### ComboBoxNode 258 259ComboBoxNode extends BasicNode, and only overrides a small number of functions 260where combo boxes have special behavior. Specifically, combo boxes require some 261special logic around moving focus into the dropdown when it opens. 262 263#### DesktopNode 264 265Represents the largest group possible, the entire computer desktop. Behaves 266mostly like a SARootNode, but does not have a back button. 267 268#### EditableTextNode 269 270Editable text nodes have a very different set of actions. Currently supported 271are opening the keyboard and using dictation, but many more actions (such as 272selection and copy/paste) are available with the flag 273`enable-experimental-accessibility-switch-access-text`. 274 275#### GroupNode 276 277A GroupNode represents a subset of the children of a single automation node, 278used to break a single node with many children into intermediate nodes for 279easier navigation. 280 281Currently the only place this is used is to break the keyboard into rows, but 282ideally this should be done for any group with more than some number of 283children. 284 285#### KeyboardNode and KeyboardRootNode 286 287A KeyboardNode represents a button within the virtual keyboard. Because the 288keys do not support actions through the automation API, the KeyboardNode 289simulate a mouse press at the center of the button. 290 291The KeyboardRootNode represents the keyboard as a whole. It handles finding 292the buttons from the keyboard and grouping them into rows, as well as 293finding the automation node representing the virtual keyboard and monitoring 294the visibility of the keyboard, so if the user opens/closes the keyboard 295other than via Switch Access we still detect it. 296 297#### ModalDialogRootNode 298 299Currently used when a menu is opened, it behaves exactly like a BasicRootNode 300except that when the user exits, it fires an ESC key event to close the menu 301or modal dialog. 302 303#### SliderNode 304 305Adds support for custom sliders by sending left/right arrow key events to 306decrement/increment. 307 308#### TabNode and ActionableTabNode 309 310These two classes are primarily to allow the close button to both be 311accessible via Switch Access while clearly displaying visually how to select 312the tab. This is done by having TabNode be a group (when it contains a button), 313and create an ActionableTabNode as a child whose location is just the portion of 314the tab that doesn't overlap with the close button. 315 316#### WindowRootNode 317 318The WindowRootNode focuses a window when it is entered. Otherwise it behaves 319exactly like a BasicRootNode. 320 321 322### Other 323 324There are some other classes that support Switch Access without being a manager 325or node type directly. 326 327#### background.js 328 329This file is the first one run. Its primary job is to create an instance of 330SwitchAccess, although it als overifies that there is not more than one instance 331of Switch Access running simultaneously (this would normally happen on the sign 332in page). 333 334#### Commands 335 336Commands translates a SwitchAccessCommand from the user into a call to the 337appropriate function. 338 339#### History 340 341History helps store and recover state about what the current node and group are. 342When a group is exited, the history provides the previous position to return to 343(after verifying that it is still valid). It also builds a new history when 344focus moves, capturing how the user would have gotten to that same node. 345 346#### Metrics 347 348A utility for recording metrics. 349 350#### SACache 351 352Implements a dynamic programming strategy for when walking the automation tree to 353find interesting nodes. A cache is created for a single query, and should not be 354used in nonconsecutive function calls, as the underlying data can change and the 355SACache does not account for this. 356 357#### SAConstants 358 359This file contains most or all of the constants used throughout Switch Access. 360 361#### SwitchAccess 362 363Primarily, this class creates the managers that need to be explicitly initialized, 364as well as the Commands object. It also handles creating errors (so we can track 365metrics on how often different errors happen), and has a function to find a node 366matching a predicate (or wait for it to be created). 367 368#### SwitchAccessPredicate 369 370Has a variety of predicates determining things about automation nodes, all 371generally specific to Switch Access. Many of these functions utilize SACache. 372It also creates the restrictions that can be passed to AutomationTreeWalker to 373find the appropriate children for a given group node. 374 375 376## For Googlers 377 378For more, Googlers could check out the Switch Access feature design docs for 379more details on design as well as UMA. 380 381- Overall product design, [go/cros-switch](go/cros-switch) 382 383- The navigation strategy, [go/cros-switch-navigation](go/cros-switch-navigation) 384 385- The action menu, [go/cros-switch-menu](go/cros-switch-menu) 386 387- The "preview" dashed focus ring, [go/cros-switch-dashed-focus](go/cros-switch-dashed-focus) 388 389- The UX description slideshow, [go/cros-switch-access-ux](go/cros-switch-access-ux) 390 391- The UI specification, [go/cros-switch-spec](go/cros-switch-spec) 392 393- Testing, [go/cros-switch-testing](go/cros-switch-testing) 394 395- Improved Text Input (still experimental), [go/cros-switch-text-input](go/cros-switch-text-input) 396 397- Point Scanning (still under development), [go/cros-switch-point-scanning](go/cros-switch-point-scanning) 398