1--- 2stage: Secure 3group: Dynamic Analysis 4info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers 5type: howto 6--- 7 8# HTTP Archive format **(ULTIMATE)** 9 10HTTP Archive (HAR) format files are an industry standard for exchanging information about HTTP 11requests and HTTP responses. A HAR file's content is JSON formatted, containing browser interactions 12with a web site. The file extension `.har` is commonly used. 13 14The HAR files can be used to perform [web API Fuzz Testing](index.md#http-archive-har) as part of 15your [GitLab CI/CD](../../../ci/index.md) pipelines. 16 17WARNING: 18A HAR file stores information exchanged between web client and web server. It could also 19store sensitive information such as authentication tokens, API keys, and session cookies. We 20recommend that you review the HAR file contents before adding them to a repository. 21 22## HAR file creation 23 24You can create HAR files manually or by using a specialized tool for recording web sessions. We 25recommend using a specialized tool. However, it is important to make sure files created by these 26tools do not expose sensitive information, and can be safely used. 27 28The following tools can be used generate a HAR file based on your network activity. They 29automatically record your network activity and generate the HAR file: 30 311. [GitLab HAR Recorder](#gitlab-har-recorder). 321. [Insomnia API Client](#insomnia-api-client). 331. [Fiddler debugging proxy](#fiddler-debugging-proxy). 341. [Safari web browser](#safari-web-browser). 351. [Chrome web browser](#chrome-web-browser). 361. [Firefox web browser](#firefox-web-browser). 37 38WARNING: 39HAR files may contain sensitive information such as authentication tokens, API keys, and 40session cookies. We recommend that you review the HAR file contents before adding them to a 41repository. 42 43### GitLab HAR Recorder 44 45[GitLab HAR Recorder](https://gitlab.com/gitlab-org/security-products/har-recorder) is a command 46line tool for recording HTTP messages and saving them to HTTP Archive (HAR) files. For more details 47about the GitLab HAR Recorder, see the [homepage](https://gitlab.com/gitlab-org/security-products/har-recorder). 48 49#### Install GitLab HAR Recorder 50 51Prerequisites: 52 53- Install Python 3.6 or greater. 54- For Microsoft Windows, you must also install `Microsoft Visual C++ 14.0`. It's included with 55 *Build Tools for Visual Studio* from [Visual Studio Downloads page](https://visualstudio.microsoft.com/downloads/). 56- Install HAR Recorder. 57 58Install GitLab HAR Recorder: 59 60 ```shell 61 pip install gitlab-har-recorder --extra-index-url https://gitlab.com/api/v4/projects/22441624/packages/pypi/simple 62 ``` 63 64#### Create a HAR file with GitLab HAR Recorder 65 661. Start recorder with the proxy port and HAR filename. 671. Complete the browser actions, using the proxy. 68 1. Make sure proxy is used! 691. Stop the recorder. 70 71To verify the HAR contains all requests, use an online HAR viewer, for example: 72 73- [HAR Viewer](http://www.softwareishard.com/har/viewer/) 74<!-- vale gitlab.Admin = NO --> 75- [Google Admin Toolbox HAR Analyzer](https://toolbox.googleapps.com/apps/har_analyzer/) 76<!-- vale gitlab.Admin = YES --> 77 78### Insomnia API Client 79 80[Insomnia API Client](https://insomnia.rest/) is an API design tool that among many uses, helps 81you to design, describe, and test your API. You can also use it to generate HAR files that can be 82used in [Web API Fuzz Testing](index.md#http-archive-har). 83 84#### Create a HAR file with the Insomnia API Client 85 861. Define or import your API. 87 - Postman v2. 88 - Curl. 89 - OpenAPI v2, v3. 901. Verify each API call works. 91 - If you imported an OpenAPI specification, go through and add working data. 921. Select **API > Import/Export**. 931. Select **Export Data > Current Workspace**. 941. Select requests to include in the HAR file. 951. Select **Export**. 961. In the **Select Export Type** dropdown select **HAR -- HTTP Archive Format**. 971. Select **Done**. 981. Enter a location and filename for the HAR file. 99 100### Fiddler debugging proxy 101 102[Fiddler](https://www.telerik.com/fiddler) is a web debugger tool. It captures HTTP and HTTP(S) 103network traffic and allows you to examine each request. It also lets you export the requests and 104responses in HAR format. 105 106#### Create a HAR file with Fiddler 107 1081. Go to the [Fiddler home page](https://www.telerik.com/fiddler) and sign in. If you don't already 109have an account, first create an account. 1101. Browse pages that call an API. Fiddler automatically captures the requests. 1111. Select one or more requests, then from the context menu, select **Export > Selected Sessions**. 1121. In the **Choose Format** dropdown select **HTTPArchive v1.2**. 1131. Enter a filename and select **Save**. 114 115Fiddler shows a popup message confirming the export has succeeded. 116 117### Safari web browser 118 119[Safari](https://www.apple.com/safari/) is a web browser maintained by Apple. As web development 120evolves, browsers support new capabilities. With Safari you can explore network traffic and 121export it as a HAR file. 122 123#### Create a HAR file with Safari 124 125Prerequisites: 126 127- Enable the `Develop` menu item. 128 1. Open Safari's preferences. Press <kbd>Command</kbd>+<kbd>,</kbd> or from the menu, select 129 **Safari > Preferences...**. 130 1. Select **Advanced** tab, then select `Show Develop menu item in menu bar`. 131 1. Close the **Preferences** window. 132 1331. Open the **Web Inspector**. Press <kbd>Option</kbd>+<kbd>Command</kbd>+<kbd>i</kbd>, or from the 134 menu, select **Develop > Show Web Inspector**. 1351. Select the **Network** tab, and select **Preserve Log**. 1361. Browse pages that call the API. 1371. Open the **Web Inspector** and select the **Network** tab 1381. Right-click on the request to export and select **Export HAR**. 1391. Enter a filename and select **Save**. 140 141### Chrome web browser 142 143[Chrome](https://www.google.com/chrome/) is a web browser maintained by Google. As web development 144evolves, browsers support new capabilities. With Chrome you can explore network traffic and 145export it as a HAR file. 146 147#### Create a HAR file with Chrome 148 1491. From the Chrome context menu, select **Inspect**. 1501. Select the **Network** tab. 1511. Select **Preserve log**. 1521. Browse pages that call the API. 1531. Select one or more requests. 1541. Right click and select **Save all as HAR with content**. 1551. Enter a filename and select **Save**. 1561. To append additional requests, select and save them to the same file. 157 158### Firefox Web Browser 159 160[Firefox](https://www.mozilla.org/en-US/firefox/new/) is a web browser maintained by Mozilla. As web 161development evolves, browsers support new capabilities. With Firefox you can explore network traffic 162and export it as a HAR file. 163 164#### Create a HAR file with Firefox 165 1661. From the Firefox context menu, select **Inspect**. 1671. Select the **Network** tab. 1681. Browse pages that call the API. 1691. Check the **Network** tab and confirm requests are being recorded. If there is a message 170 `Perform a request or Reload the page to see detailed information about network activity`, 171 select **Reload** to start recording requests. 1721. Select one or more requests. 1731. Right click and select **Save All As HAR**. 1741. Enter a filename and select **Save**. 1751. To append additional requests, select and save them to the same file. 176 177## HAR verification 178 179Before using HAR files it's important to make sure they don't expose any sensitive information. 180 181For each HAR file you should: 182 183- View the HAR file's content 184- Review the HAR file for sensitive information 185- Edit or remove sensitive information 186 187### View HAR file contents 188 189We recommend viewing a HAR file's content in a tool that can present its content in a structured 190way. Several HAR file viewers are available online. If you would prefer not to upload the HAR file, 191you can use a tool installed on your computer. HAR files used JSON format, so can also be viewed in 192a text editor. 193 194Tools recommended for viewing HAR files include: 195 196- [HAR Viewer](http://www.softwareishard.com/har/viewer/) - (online) 197<!-- vale gitlab.Admin = NO --> 198- [Google Admin Toolbox HAR Analyzer](https://toolbox.googleapps.com/apps/har_analyzer/) - (online) 199<!-- vale gitlab.Admin = YES --> 200- [Fiddler](https://www.telerik.com/fiddler) - local 201- [Insomnia API Client](https://insomnia.rest/) - local 202 203## Review HAR file content 204 205Review the HAR file for any of the following: 206 207- Information that could help to grant access to your application, for example: authentication 208 tokens, authentication tokens, cookies, API keys. 209- [Personally Identifiable Information (PII)](https://en.wikipedia.org/wiki/Personal_data). 210 211We strongly recommended that you [edit or remove it](#edit-or-remove-sensitive-information) any 212sensitive information. 213 214Use the following as a checklist to start with. Note that it's not an exhaustive list. 215 216- Look for secrets. For example: if your application requires authentication, check common locations 217 or authentication information: 218 - Authentication related headers. For example: cookies, authorization. These headers could contain 219 valid information. 220 - A request related to authentication. The body of these requests might contain information such 221 as user credentials or tokens. 222 - Session tokens. Session tokens could grant access to your application. The location of these 223 token could vary. They could be in headers, query parameters or body. 224- Look for Personally Identifiable Information 225 - For example, if your application retrieves a list of users and their personal data: phones, 226 names, emails. 227 - Authentication information might also contain personal information. 228 229## Edit or remove sensitive information 230 231Edit or remove sensitive information found during the [HAR file content review](#review-har-file-content). 232HAR files are JSON files and can be edited in any text editor. 233 234After editing the HAR file, open it in a HAR file viewer to verify its formatting and structure are 235intact. 236 237The following example demonstrates use of [Visual Studio Code](https://code.visualstudio.com/) text 238editor to edit an Authorization token found in a header. 239 240![Authorization token edited in Visual Studio Code](img/vscode_har_edit_auth_header.png) 241