1--- 2stage: none 3group: unassigned 4info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments 5--- 6 7# API style guide 8 9This style guide recommends best practices for API development. 10 11## Instance variables 12 13Please do not use instance variables, there is no need for them (we don't need 14to access them as we do in Rails views), local variables are fine. 15 16## Entities 17 18Always use an [Entity](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/entities) to present the endpoint's payload. 19 20## Documentation 21 22Each new or updated API endpoint must come with documentation, unless it is internal or behind a feature flag. 23The docs should be in the same merge request, or, if strictly necessary, 24in a follow-up with the same milestone as the original merge request. 25 26See the [Documentation Style Guide RESTful API page](documentation/restful_api_styleguide.md) for details on documenting API resources in Markdown as well as in OpenAPI definition files. 27 28## Methods and parameters description 29 30Every method must be described using the [Grape DSL](https://github.com/ruby-grape/grape#describing-methods) 31(see [`environments.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/environments.rb) 32for a good example): 33 34- `desc` for the method summary. You should pass it a block for additional 35 details such as: 36 - The GitLab version when the endpoint was added. If it is behind a feature flag, mention that instead: _This feature is gated by the :feature\_flag\_symbol feature flag._ 37 - If the endpoint is deprecated, and if so, its planned removal date 38 39- `params` for the method parameters. This acts as description, 40 [validation, and coercion of the parameters](https://github.com/ruby-grape/grape#parameter-validation-and-coercion) 41 42A good example is as follows: 43 44```ruby 45desc 'Get all broadcast messages' do 46 detail 'This feature was introduced in GitLab 8.12.' 47 success Entities::BroadcastMessage 48end 49params do 50 optional :page, type: Integer, desc: 'Current page number' 51 optional :per_page, type: Integer, desc: 'Number of messages per page' 52end 53get do 54 messages = BroadcastMessage.all 55 56 present paginate(messages), with: Entities::BroadcastMessage 57end 58``` 59 60## Declared parameters 61 62> Grape allows you to access only the parameters that have been declared by your 63`params` block. It filters out the parameters that have been passed, but are not 64allowed. 65 66– <https://github.com/ruby-grape/grape#declared> 67 68### Exclude parameters from parent namespaces 69 70> By default `declared(params)`includes parameters that were defined in all 71parent namespaces. 72 73– <https://github.com/ruby-grape/grape#include-parent-namespaces> 74 75In most cases you should exclude parameters from the parent namespaces: 76 77```ruby 78declared(params, include_parent_namespaces: false) 79``` 80 81### When to use `declared(params)` 82 83You should always use `declared(params)` when you pass the parameters hash as 84arguments to a method call. 85 86For instance: 87 88```ruby 89# bad 90User.create(params) # imagine the user submitted `admin=1`... :) 91 92# good 93User.create(declared(params, include_parent_namespaces: false).to_h) 94``` 95 96NOTE: 97`declared(params)` return a `Hashie::Mash` object, on which you must 98call `.to_h`. 99 100But we can use `params[key]` directly when we access single elements. 101 102For instance: 103 104```ruby 105# good 106Model.create(foo: params[:foo]) 107``` 108 109## Array types 110 111With Grape v1.3+, Array types must be defined with a `coerce_with` 112block, or parameters, fails to validate when passed a string from an 113API request. See the [Grape upgrading 114documentation](https://github.com/ruby-grape/grape/blob/master/UPGRADING.md#ensure-that-array-types-have-explicit-coercions) 115for more details. 116 117### Automatic coercion of nil inputs 118 119Prior to Grape v1.3.3, Array parameters with `nil` values would 120automatically be coerced to an empty Array. However, due to [this pull 121request in v1.3.3](https://github.com/ruby-grape/grape/pull/2040), this 122is no longer the case. For example, suppose you define a PUT `/test` 123request that has an optional parameter: 124 125```ruby 126optional :user_ids, type: Array[Integer], coerce_with: ::API::Validations::Types::CommaSeparatedToIntegerArray.coerce, desc: 'The user ids for this rule' 127``` 128 129Normally, a request to PUT `/test?user_ids` would cause Grape to pass 130`params` of `{ user_ids: nil }`. 131 132This may introduce errors with endpoints that expect a blank array and 133do not handle `nil` inputs properly. To preserve the previous behavior, 134there is a helper method `coerce_nil_params_to_array!` that is used 135in the `before` block of all API calls: 136 137```ruby 138before do 139 coerce_nil_params_to_array! 140end 141``` 142 143With this change, a request to PUT `/test?user_ids` causes Grape to 144pass `params` to be `{ user_ids: [] }`. 145 146There is [an open issue in the Grape tracker](https://github.com/ruby-grape/grape/issues/2068) 147to make this easier. 148 149## Using HTTP status helpers 150 151For non-200 HTTP responses, use the provided helpers in `lib/api/helpers.rb` to ensure correct behavior (like `not_found!` or `no_content!`). These `throw` inside Grape and abort the execution of your endpoint. 152 153For `DELETE` requests, you should also generally use the `destroy_conditionally!` helper which by default returns a `204 No Content` response on success, or a `412 Precondition Failed` response if the given `If-Unmodified-Since` header is out of range. This helper calls `#destroy` on the passed resource, but you can also implement a custom deletion method by passing a block. 154 155## Using API path helpers in GitLab Rails codebase 156 157Because we support [installing GitLab under a relative URL](../install/relative_url.md), one must take this 158into account when using API path helpers generated by Grape. Any such API path 159helper usage must be in wrapped into the `expose_path` helper call. 160 161For instance: 162 163```haml 164- endpoint = expose_path(api_v4_projects_issues_related_merge_requests_path(id: @project.id, issue_iid: @issue.iid)) 165``` 166 167## Custom Validators 168 169In order to validate some parameters in the API request, we validate them 170before sending them further (say Gitaly). The following are the 171[custom validators](https://GitLab.com/gitlab-org/gitlab/-/tree/master/lib/api/validations/validators), 172which we have added so far and how to use them. We also wrote a 173guide on how you can add a new custom validator. 174 175### Using custom validators 176 177- `FilePath`: 178 179 GitLab supports various functionalities where we need to traverse a file path. 180 The [`FilePath` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/file_path.rb) 181 validates the parameter value for different cases. Mainly, it checks whether a 182 path is relative and does it contain `../../` relative traversal using 183 `File::Separator` or not, and whether the path is absolute, for example 184 `/etc/passwd/`. By default, absolute paths are not allowed. However, you can optionally pass in an allowlist for allowed absolute paths in the following way: 185 `requires :file_path, type: String, file_path: { allowlist: ['/foo/bar/', '/home/foo/', '/app/home'] }` 186 187- `Git SHA`: 188 189 The [`Git SHA` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/git_sha.rb) 190 checks whether the Git SHA parameter is a valid SHA. 191 It checks by using the regex mentioned in [`commit.rb`](https://gitlab.com/gitlab-org/gitlab/-/commit/b9857d8b662a2dbbf54f46ecdcecb44702affe55#d1c10892daedb4d4dd3d4b12b6d071091eea83df_30_30) file. 192 193- `Absence`: 194 195 The [`Absence` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/absence.rb) 196 checks whether a particular parameter is absent in a given parameters hash. 197 198- `IntegerNoneAny`: 199 200 The [`IntegerNoneAny` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/integer_none_any.rb) 201 checks if the value of the given parameter is either an `Integer`, `None`, or `Any`. 202 It allows only either of these mentioned values to move forward in the request. 203 204- `ArrayNoneAny`: 205 206 The [`ArrayNoneAny` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/array_none_any.rb) 207 checks if the value of the given parameter is either an `Array`, `None`, or `Any`. 208 It allows only either of these mentioned values to move forward in the request. 209 210- `EmailOrEmailList`: 211 212 The [`EmailOrEmailList` validator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators/email_or_email_list.rb) 213 checks if the value of a string or a list of strings contains only valid 214 email addresses. It allows only lists with all valid email addresses to move forward in the request. 215 216### Adding a new custom validator 217 218Custom validators are a great way to validate parameters before sending 219them to platform for further processing. It saves some back-and-forth 220from the server to the platform if we identify invalid parameters at the beginning. 221 222If you need to add a custom validator, it would be added to 223it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators) directory. 224Since we use [Grape](https://github.com/ruby-grape/grape) to add our API 225we inherit from the `Grape::Validations::Base` class in our validator class. 226Now, all you have to do is define the `validate_param!` method which takes 227in two parameters: the `params` hash and the `param` name to validate. 228 229The body of the method does the hard work of validating the parameter value 230and returns appropriate error messages to the caller method. 231 232Lastly, we register the validator using the line below: 233 234```ruby 235Grape::Validations.register_validator(<validator name as symbol>, ::API::Helpers::CustomValidators::<YourCustomValidatorClassName>) 236``` 237 238Once you add the validator, make sure you add the `rspec`s for it into 239it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/api/validations/validators) directory. 240 241## Internal API 242 243The [internal API](internal_api/index.md) is documented for internal use. Please keep it up to date so we know what endpoints 244different components are making use of. 245 246## Avoiding N+1 problems 247 248In order to avoid N+1 problems that are common when returning collections 249of records in an API endpoint, we should use eager loading. 250 251A standard way to do this within the API is for models to implement a 252scope called `with_api_entity_associations` that preloads the 253associations and data returned in the API. An example of this scope can 254be seen in 255[the `Issue` model](https://gitlab.com/gitlab-org/gitlab/-/blob/2fedc47b97837ea08c3016cf2fb773a0300a4a25/app%2Fmodels%2Fissue.rb#L62). 256 257In situations where the same model has multiple entities in the API 258(for instance, `UserBasic`, `User` and `UserPublic`) you should use your 259discretion with applying this scope. It may be that you optimize for the 260most basic entity, with successive entities building upon that scope. 261 262The `with_api_entity_associations` scope also [automatically preloads 263data](https://gitlab.com/gitlab-org/gitlab/-/blob/19f74903240e209736c7668132e6a5a735954e7c/app%2Fmodels%2Ftodo.rb#L34) 264for `Todo` _targets_ when returned in the [to-dos API](../api/todos.md). 265 266For more context and discussion about preloading see 267[this merge request](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/25711) 268which introduced the scope. 269 270### Verifying with tests 271 272When an API endpoint returns collections, always add a test to verify 273that the API endpoint does not have an N+1 problem, now and in the future. 274We can do this using [`ActiveRecord::QueryRecorder`](query_recorder.md). 275 276Example: 277 278```ruby 279def make_api_request 280 get api('/foo', personal_access_token: pat) 281end 282 283it 'avoids N+1 queries', :request_store do 284 # Firstly, record how many PostgreSQL queries the endpoint will make 285 # when it returns a single record 286 create_record 287 288 control = ActiveRecord::QueryRecorder.new { make_api_request } 289 290 # Now create a second record and ensure that the API does not execute 291 # any more queries than before 292 create_record 293 294 expect { make_api_request }.not_to exceed_query_limit(control) 295end 296``` 297 298## Testing 299 300When writing tests for new API endpoints, consider using a schema [fixture](testing_guide/best_practices.md#fixtures) located in `/spec/fixtures/api/schemas`. You can `expect` a response to match a given schema: 301 302```ruby 303expect(response).to match_response_schema('merge_requests') 304``` 305 306Also see [verifying N+1 performance](#verifying-with-tests) in tests. 307 308## Include a changelog entry 309 310All client-facing changes **must** include a [changelog entry](changelog.md). 311This does not include internal APIs. 312