1--- 2stage: none 3group: unassigned 4info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments 5--- 6 7# GitLab architecture overview 8 9## Software delivery 10 11There are two software distributions of GitLab: 12 13- The open source [Community Edition](https://gitlab.com/gitlab-org/gitlab-foss/) (CE). 14- The open core [Enterprise Edition](https://gitlab.com/gitlab-org/gitlab/) (EE). 15 16GitLab is available under [different subscriptions](https://about.gitlab.com/pricing/). 17 18New versions of GitLab are released from stable branches, and the `main` branch is used for 19bleeding-edge development. 20 21For more information, visit the [GitLab Release Process](https://about.gitlab.com/handbook/engineering/releases/). 22 23Both distributions require additional components. These components are described in the 24[Component details](#components) section, and all have their own repositories. 25New versions of each dependent component are usually tags, but staying on the `main` branch of the 26GitLab codebase gives you the latest stable version of those components. New versions are 27generally released around the same time as GitLab releases, with the exception of informal security 28updates deemed critical. 29 30## Components 31 32A typical install of GitLab is on GNU/Linux, but growing number of deployments also use the 33Kubernetes platform. The largest known GitLab instance is on GitLab.com, which is deployed using our 34[official GitLab Helm chart](https://docs.gitlab.com/charts/) and the [official Linux package](https://about.gitlab.com/install/). 35 36A typical installation uses NGINX or Apache as a web server to proxy through 37[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) and into the [Puma](https://puma.io) 38application server. GitLab serves web pages and the [GitLab API](../api/index.md) using the Puma 39application server. It uses Sidekiq as a job queue which, in turn, uses Redis as a non-persistent 40database backend for job information, metadata, and incoming jobs. 41 42By default, communication between Puma and Workhorse is via a Unix domain socket, but forwarding 43requests via TCP is also supported. Workhorse accesses the `gitlab/public` directory, bypassing the 44Puma application server to serve static pages, uploads (for example, avatar images or attachments), 45and pre-compiled assets. 46 47The GitLab application uses PostgreSQL for persistent database information (for example, users, 48permissions, issues, or other metadata). GitLab stores the bare Git repositories in the location 49defined in [the configuration file, `repositories:` section](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example). 50It also keeps default branch and hook information with the bare repository. 51 52When serving repositories over HTTP/HTTPS GitLab uses the GitLab API to resolve authorization and 53access and to serve Git objects. 54 55The add-on component GitLab Shell serves repositories over SSH. It manages the SSH keys within the 56location defined in [the configuration file, `GitLab Shell` section](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example). 57The file in that location should never be manually edited. GitLab Shell accesses the bare 58repositories through Gitaly to serve Git objects, and communicates with Redis to submit jobs to 59Sidekiq for GitLab to process. GitLab Shell queries the GitLab API to determine authorization and access. 60 61Gitaly executes Git operations from GitLab Shell and the GitLab web app, and provides an API to the 62GitLab web app to get attributes from Git (for example, title, branches, tags, or other metadata), 63and to get blobs (for example, diffs, commits, or files). 64 65You may also be interested in the [production architecture of GitLab.com](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/). 66 67## Adapting existing and introducing new components 68 69There are fundamental differences in how the application behaves when it is installed on a 70traditional Linux machine compared to a containerized platform, such as Kubernetes. 71 72Compared to [our official installation methods](https://about.gitlab.com/install/), some of the 73notable differences are: 74 75- Official Linux packages can access files on the same file system with different services. 76 [Shared files](shared_files.md) are not an option for the application running on the Kubernetes 77 platform. 78- Official Linux packages by default have services that have access to the shared configuration and 79 network. This is not the case for services running in Kubernetes, where services might be running 80 in complete isolation, or only accessible through specific ports. 81 82In other words, the shared state between services needs to be carefully considered when 83architecting new features and adding new components. Services that need to have access to the same 84files, need to be able to exchange information through the appropriate APIs. Whenever possible, 85this should not be done with files. 86 87Since components written with the API-first philosophy in mind are compatible with both methods, all 88new features and services must be written to consider Kubernetes compatibility **first**. 89 90The simplest way to ensure this, is to add support for your feature or service to 91[the official GitLab Helm chart](https://docs.gitlab.com/charts/) or reach out to 92[the Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/distribution/#how-to-work-with-distribution). 93 94Refer to the [process for adding new service components](adding_service_component.md) for more details. 95 96### Simplified component overview 97 98This is a simplified architecture diagram that can be used to 99understand the GitLab architecture. 100 101A complete architecture diagram is available in our 102[component diagram](#component-diagram) below. 103 104![Simplified Component Overview](img/architecture_simplified.png) 105 106<!-- 107To update this diagram, GitLab team members can edit this source file: 108https://docs.google.com/drawings/d/1fBzAyklyveF-i-2q-OHUIqDkYfjjxC4mq5shwKSZHLs/edit. 109 --> 110 111### Component diagram 112 113```mermaid 114%%{init: {"flowchart": { "useMaxWidth": false } }}%% 115graph LR 116 %% Anchor items in the appropriate subgraph. 117 %% Link them where the destination* is. 118 119 subgraph Clients 120 Browser((Browser)) 121 Git((Git)) 122 end 123 124 %% External Components / Applications 125 Geo{{GitLab Geo}} -- TCP 80, 443 --> HTTP 126 Geo -- TCP 22 --> SSH 127 Geo -- TCP 5432 --> PostgreSQL 128 Runner{{GitLab Runner}} -- TCP 443 --> HTTP 129 K8sAgent{{GitLab Agent}} -- TCP 443 --> HTTP 130 131 %% GitLab Application Suite 132 subgraph GitLab 133 subgraph Ingress 134 HTTP[[HTTP/HTTPS]] 135 SSH[[SSH]] 136 NGINX[NGINX] 137 GitLabShell[GitLab Shell] 138 139 %% inbound/internal 140 Browser -- TCP 80,443 --> HTTP 141 Git -- TCP 80,443 --> HTTP 142 Git -- TCP 22 --> SSH 143 HTTP -- TCP 80, 443 --> NGINX 144 SSH -- TCP 22 --> GitLabShell 145 end 146 147 subgraph GitLab Services 148 %% inbound from NGINX 149 NGINX --> GitLabWorkhorse 150 NGINX -- TCP 8090 --> GitLabPages 151 NGINX -- TCP 8150 --> GitLabKas 152 NGINX --> Registry 153 %% inbound from GitLabShell 154 GitLabShell --TCP 8080 -->Puma 155 156 %% services 157 Puma["Puma (GitLab Rails)"] 158 Puma <--> Registry 159 GitLabWorkhorse[GitLab Workhorse] <--> Puma 160 GitLabKas[GitLab Agent Server] --> GitLabWorkhorse 161 GitLabPages[GitLab Pages] --> GitLabWorkhorse 162 Mailroom 163 Sidekiq 164 end 165 166 subgraph Integrated Services 167 %% Mattermost 168 Mattermost 169 Mattermost ---> GitLabWorkhorse 170 NGINX --> Mattermost 171 172 %% Grafana 173 Grafana 174 NGINX --> Grafana 175 end 176 177 subgraph Metadata 178 %% PostgreSQL 179 PostgreSQL 180 PostgreSQL --> Consul 181 182 %% Consul and inbound 183 Consul 184 Puma ---> Consul 185 Sidekiq ---> Consul 186 Migrations --> PostgreSQL 187 188 %% PgBouncer and inbound 189 PgBouncer 190 PgBouncer --> Consul 191 PgBouncer --> PostgreSQL 192 Sidekiq --> PgBouncer 193 Puma --> PgBouncer 194 end 195 196 subgraph State 197 %% Redis and inbound 198 Redis 199 Puma --> Redis 200 Sidekiq --> Redis 201 GitLabWorkhorse --> Redis 202 Mailroom --> Redis 203 GitLabKas --> Redis 204 205 %% Sentinel and inbound 206 Sentinel <--> Redis 207 Puma --> Sentinel 208 Sidekiq --> Sentinel 209 GitLabWorkhorse --> Sentinel 210 Mailroom --> Sentinel 211 GitLabKas --> Sentinel 212 end 213 214 subgraph Git Repositories 215 %% Gitaly / Praefect 216 Praefect --> Gitaly 217 GitLabKas --> Praefect 218 GitLabShell --> Praefect 219 GitLabWorkhorse --> Praefect 220 Puma --> Praefect 221 Sidekiq --> Praefect 222 Praefect <--> PraefectPGSQL[PostgreSQL] 223 %% Gitaly makes API calls 224 %% Ordered here to ensure placement. 225 Gitaly --> GitLabWorkhorse 226 end 227 228 subgraph Storage 229 %% ObjectStorage and inbound traffic 230 ObjectStorage["Object Storage"] 231 Puma -- TCP 443 --> ObjectStorage 232 Sidekiq -- TCP 443 --> ObjectStorage 233 GitLabWorkhorse -- TCP 443 --> ObjectStorage 234 Registry -- TCP 443 --> ObjectStorage 235 GitLabPages -- TCP 443 --> ObjectStorage 236 end 237 238 subgraph Monitoring 239 %% Prometheus 240 Grafana -- TCP 9090 --> Prometheus[Prometheus] 241 Prometheus -- TCP 80, 443 --> Puma 242 RedisExporter[Redis Exporter] --> Redis 243 Prometheus -- TCP 9121 --> RedisExporter 244 PostgreSQLExporter[PostgreSQL Exporter] --> PostgreSQL 245 PgBouncerExporter[PgBouncer Exporter] --> PgBouncer 246 Prometheus -- TCP 9187 --> PostgreSQLExporter 247 Prometheus -- TCP 9100 --> NodeExporter[Node Exporter] 248 Prometheus -- TCP 9168 --> GitLabExporter[GitLab Exporter] 249 Prometheus -- TCP 9127 --> PgBouncerExporter 250 Prometheus --> Alertmanager 251 GitLabExporter --> PostgreSQL 252 GitLabExporter --> GitLabShell 253 GitLabExporter --> Sidekiq 254 255 %% Alertmanager 256 Alertmanager -- TCP 25 --> SMTP 257 end 258 %% end subgraph GitLab 259 end 260 261 subgraph External 262 subgraph External Services 263 SMTP[SMTP Gateway] 264 LDAP 265 266 %% Outbound SMTP 267 Sidekiq -- TCP 25 --> SMTP 268 Puma -- TCP 25 --> SMTP 269 Mailroom -- TCP 25 --> SMTP 270 271 %% Outbound LDAP 272 Puma -- TCP 369 --> LDAP 273 Sidekiq -- TCP 369 --> LDAP 274 275 %% Elasticsearch 276 Elasticsearch 277 Puma -- TCP 9200 --> Elasticsearch 278 Sidekiq -- TCP 9200 --> Elasticsearch 279 end 280 subgraph External Monitoring 281 %% Sentry 282 Sidekiq -- TCP 80, 443 --> Sentry 283 Puma -- TCP 80, 443 --> Sentry 284 285 %% Jaeger 286 Jaeger 287 Sidekiq -- UDP 6831 --> Jaeger 288 Puma -- UDP 6831 --> Jaeger 289 Gitaly -- UDP 6831 --> Jaeger 290 GitLabShell -- UDP 6831 --> Jaeger 291 GitLabWorkhorse -- UDP 6831 --> Jaeger 292 end 293 %% end subgraph External 294 end 295 296click Alertmanager "./architecture.html#alertmanager" 297click Praefect "./architecture.html#praefect" 298click Geo "./architecture.html#gitlab-geo" 299click NGINX "./architecture.html#nginx" 300click Runner "./architecture.html#gitlab-runner" 301click Registry "./architecture.html#registry" 302click ObjectStorage "./architecture.html#minio" 303click Mattermost "./architecture.html#mattermost" 304click Gitaly "./architecture.html#gitaly" 305click Jaeger "./architecture.html#jaeger" 306click GitLabWorkhorse "./architecture.html#gitlab-workhorse" 307click LDAP "./architecture.html#ldap-authentication" 308click Puma "./architecture.html#puma" 309click GitLabShell "./architecture.html#gitlab-shell" 310click SSH "./architecture.html#ssh-request-22" 311click Sidekiq "./architecture.html#sidekiq" 312click Sentry "./architecture.html#sentry" 313click GitLabExporter "./architecture.html#gitlab-exporter" 314click Elasticsearch "./architecture.html#elasticsearch" 315click Migrations "./architecture.html#database-migrations" 316click PostgreSQL "./architecture.html#postgresql" 317click Consul "./architecture.html#consul" 318click PgBouncer "./architecture.html#pgbouncer" 319click PgBouncerExporter "./architecture.html#pgbouncer-exporter" 320click RedisExporter "./architecture.html#redis-exporter" 321click Redis "./architecture.html#redis" 322click Prometheus "./architecture.html#prometheus" 323click Grafana "./architecture.html#grafana" 324click GitLabPages "./architecture.html#gitlab-pages" 325click PostgreSQLExporter "./architecture.html#postgresql-exporter" 326click SMTP "./architecture.html#outbound-email" 327click NodeExporter "./architecture.html#node-exporter" 328``` 329 330### Component legend 331 332- ✅ - Installed by default 333- ⚙ - Requires additional configuration 334- ⤓ - Manual installation required 335- ❌ - Not supported or no instructions available 336- N/A - Not applicable 337 338Component statuses are linked to configuration documentation for each component. 339 340### Component list 341 342| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) | [CE/EE](https://about.gitlab.com/install/ce-or-ee/) | 343|-------------------------------------------------------|----------------------------------------------------------------------|:--------------:|:--------------:|:------------:|:----------------:|:----------:|:------:|:---:|:-------:| 344| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | ✅ | ✅ | ✅ | ⚙ | ✅ | ⚙ | ⚙ | CE & EE | 345| [Consul](#consul) | Database node discovery, failover | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only | 346| [Database Migrations](#database-migrations) | Database migrations | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE | 347| [Elasticsearch](#elasticsearch) | Improved search within GitLab | ⤓ | ⚙ | ⤓ | ⤓ | ✅ | ⤓ | ⚙ | EE Only | 348| [Gitaly](#gitaly) | Git RPC service for handling all Git calls made by GitLab | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE | 349| [GitLab Exporter](#gitlab-exporter) | Generates a variety of GitLab metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE | 350| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | ⚙ | ⚙ | ❌ | ❌ | ✅ | ❌ | ⚙ | EE Only | 351| [GitLab Pages](#gitlab-pages) | Hosts static websites | ⚙ | ⚙ | ❌ | ❌ | ✅ | ⚙ | ⚙ | CE & EE | 352| [GitLab Agent](#gitlab-agent) | Integrate Kubernetes clusters in a cloud-native way | ⚙ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | EE Only | 353| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | ⚙ | ⚙ | ✅ | ⚙ | ✅ | ❌ | ❌ | CE & EE | 354| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | ✅ | ✅ | ⚙ | ⤓ | ✅ | ❌ | ⚙ | CE & EE | 355| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | ❌ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | CE & EE | 356| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | ✅ | ✅ | ✅ | ⚙ | ✅ | ❌ | ⚙ | CE & EE | 357| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | ⤓ | ⤓ | ⤓ | ❌ | ✅ | ⤓ | ⤓ | CE & EE | 358| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE | 359| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE | 360| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | ⤓ | ⤓ | ⚙ | ⤓ | ✅ | ⤓ | ⤓ | CE & EE | 361| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⚙ | EE Only | 362| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | ⤓ | ⤓ | ⤓ | ⤓ | ❌ | ⤓ | ⚙ | CE & EE | 363| [Mattermost](#mattermost) | Open-source Slack alternative | ⚙ | ⚙ | ⤓ | ⤓ | ⤓ | ❌ | ⚙ | CE & EE | 364| [MinIO](#minio) | Object storage service | ⤓ | ⤓ | ✅ | ✅ | ✅ | ❌ | ⚙ | CE & EE | 365| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | ✅ | ✅ | ✅ | ⚙ | ✅ | ⤓ | ⚙ | CE & EE | 366| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | ✅ | ✅ | N/A | N/A | ✅ | ❌ | ❌ | CE & EE | 367| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | ⤓ | ⤓ | ⚙ | ⤓ | ✅ | ⤓ | ⤓ | CE & EE | 368| [Patroni](#patroni) | Manage PostgreSQL HA cluster leader selection and replication | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only | 369| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | CE & EE | 370| [PgBouncer](#pgbouncer) | Database connection pooling, failover | ⚙ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | EE Only | 371| [PostgreSQL Exporter](#postgresql-exporter) | Prometheus endpoint with PostgreSQL metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE | 372| [PostgreSQL](#postgresql) | Database | ✅ | ✅ | ✅ | ✅ | ✅ | ⤓ | ✅ | CE & EE | 373| [Praefect](#praefect) | A transparent proxy between any Git client and Gitaly storage nodes. | ✅ | ✅ | ⚙ | ❌ | ✅ | ⚙ | ✅ | CE & EE | 374| [Puma (GitLab Rails)](#puma) | Handles requests for the web interface and API | ✅ | ✅ | ✅ | ✅ | ✅ | ⚙ | ✅ | CE & EE | 375| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE | 376| [Redis](#redis) | Caching service | ✅ | ✅ | ✅ | ✅ | ✅ | ⤓ | ✅ | CE & EE | 377| [Registry](#registry) | Container registry, allows pushing and pulling of images | ⚙ | ⚙ | ✅ | ✅ | ✅ | ⤓ | ⚙ | CE & EE | 378| [Runner](#gitlab-runner) | Executes GitLab CI/CD jobs | ⤓ | ⤓ | ✅ | ⚙ | ✅ | ⚙ | ⚙ | CE & EE | 379| [Sentry integration](#sentry) | Error tracking for deployed apps | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | ⤓ | CE & EE | 380| [Sidekiq](#sidekiq) | Background jobs processor | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | CE & EE | 381 382### Component details 383 384This document is designed to be consumed by systems administrators and GitLab Support Engineers who want to understand more about the internals of GitLab and how they work together. 385 386When deployed, GitLab should be considered the amalgamation of the below processes. When troubleshooting or debugging, be as specific as possible as to which component you are referencing. That should increase clarity and reduce confusion. 387 388**Layers** 389 390GitLab can be considered to have two layers from a process perspective: 391 392- **Monitoring**: Anything from this layer is not required to deliver GitLab the application, but allows administrators more insight into their infrastructure and what the service as a whole is doing. 393- **Core**: Any process that is vital for the delivery of GitLab as a platform. If any of these processes halt, a GitLab outage results. For the Core layer, you can further divide into: 394 - **Processors**: These processes are responsible for actually performing operations and presenting the service. 395 - **Data**: These services store/expose structured data for the GitLab service. 396 397#### Alertmanager 398 399- [Project page](https://github.com/prometheus/alertmanager/blob/master/README.md) 400- Configuration: 401 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) 402 - [Charts](https://github.com/helm/charts/tree/master/stable/prometheus) 403- Layer: Monitoring 404- Process: `alertmanager` 405- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 406 407[Alert manager](https://prometheus.io/docs/alerting/latest/alertmanager/) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or Opsgenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue #45740](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45740) about what we alert on. 408 409#### Certificate management 410 411- Project page: 412 - [Omnibus](https://github.com/certbot/certbot/blob/master/README.rst) 413 - [Charts](https://github.com/jetstack/cert-manager/blob/master/README.md) 414- Configuration: 415 - [Omnibus](https://docs.gitlab.com/omnibus/settings/ssl.html) 416 - [Charts](https://docs.gitlab.com/charts/installation/tls.html) 417 - [Source](../install/installation.md#using-https) 418 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/https.md) 419- Layer: Core Service (Processor) 420- GitLab.com: [Secrets Management](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#secrets-management) 421 422#### Consul 423 424- [Project page](https://github.com/hashicorp/consul/blob/master/README.md) 425- Configuration: 426 - [Omnibus](../administration/consul.md) 427 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#postgresql) 428- Layer: Core Service (Data) 429- GitLab.com: [Consul](../user/gitlab_com/index.md#consul) 430 431Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scalable. 432 433#### Database migrations 434 435- Configuration: 436 - [Omnibus](https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration) 437 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/migrations/) 438 - [Source](../update/upgrading_from_source.md#10-install-libraries-migrations-etc) 439- Layer: Core Service (Data) 440 441#### Elasticsearch 442 443- [Project page](https://github.com/elastic/elasticsearch/) 444- Configuration: 445 - [Omnibus](../integration/elasticsearch.md) 446 - [Charts](../integration/elasticsearch.md) 447 - [Source](../integration/elasticsearch.md) 448 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/elasticsearch.md) 449- Layer: Core Service (Data) 450- GitLab.com: [Get Advanced Search working on GitLab.com (Closed)](https://gitlab.com/groups/gitlab-org/-/epics/153) epic. 451 452Elasticsearch is a distributed RESTful search engine built for the cloud. 453 454#### Gitaly 455 456- [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md) 457- Configuration: 458 - [Omnibus](../administration/gitaly/index.md) 459 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/gitaly/) 460 - [Source](../install/installation.md#install-gitaly) 461- Layer: Core Service (Data) 462- Process: `gitaly` 463- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 464 465Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's README](https://gitlab.com/gitlab-org/gitaly). 466 467#### Praefect 468 469- [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md) 470- Configuration: 471 - [Omnibus](../administration/gitaly/index.md) 472 - [Source](../install/installation.md#install-gitaly) 473- Layer: Core Service (Data) 474- Process: `praefect` 475- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 476 477Praefect is a transparent proxy between each Git client and the Gitaly coordinating the replication of 478repository updates to secondary nodes. 479 480#### GitLab Geo 481 482- Configuration: 483 - [Omnibus](../administration/geo/setup/index.md) 484 - [Charts](https://docs.gitlab.com/charts/advanced/geo/) 485 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/geo.md) 486- Layer: Core Service (Processor) 487 488Geo is a premium feature built to help speed up the development of distributed teams by providing one or more read-only mirrors of a primary GitLab instance. This mirror (a Geo secondary site) reduces the time to clone or fetch large repositories and projects, or can be part of a Disaster Recovery solution. 489 490#### GitLab Exporter 491 492- [Project page](https://gitlab.com/gitlab-org/gitlab-exporter) 493- Configuration: 494 - [Omnibus](../administration/monitoring/prometheus/gitlab_exporter.md) 495 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/gitlab-exporter/index.html) 496- Layer: Monitoring 497- Process: `gitlab-exporter` 498- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 499 500GitLab Exporter is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's README](https://gitlab.com/gitlab-org/gitlab-exporter). 501 502#### GitLab Agent 503 504- [Project page](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) 505- Configuration: 506 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) 507 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/kas/index.html) 508 509The [GitLab Agent](../user/clusters/agent/index.md) is an active in-cluster 510component for solving GitLab and Kubernetes integration tasks in a secure and 511cloud-native way. 512 513You can use it to sync deployments onto your Kubernetes cluster. 514 515#### GitLab Pages 516 517- Configuration: 518 - [Omnibus](../administration/pages/index.md) 519 - [Charts](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/37) 520 - [Source](../install/installation.md#install-gitlab-pages) 521 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/pages.md) 522- Layer: Core Service (Processor) 523- GitLab.com: [GitLab Pages](../user/gitlab_com/index.md#gitlab-pages) 524 525GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab. 526 527You can use it either for personal or business websites, such as portfolios, documentation, manifestos, and business presentations. You can also attribute any license to your content. 528 529#### GitLab Runner 530 531- [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md) 532- Configuration: 533 - [Omnibus](https://docs.gitlab.com/runner/) 534 - [Charts](https://docs.gitlab.com/runner/install/kubernetes.html) 535 - [Source](https://docs.gitlab.com/runner/) 536 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/runner.md) 537- Layer: Core Service (Processor) 538- GitLab.com: [Runners](../ci/runners/index.md) 539 540GitLab Runner runs jobs and sends the results to GitLab. 541 542GitLab CI/CD is the open-source continuous integration service included with GitLab that coordinates the testing. The old name of this project was `GitLab CI Multi Runner` but please use `GitLab Runner` (without CI) from now on. 543 544#### GitLab Shell 545 546- [Project page](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/README.md) 547- Configuration: 548 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) 549 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/) 550 - [Source](../install/installation.md#install-gitlab-shell) 551 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 552- Layer: Core Service (Processor) 553- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 554 555[GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) is a program designed at GitLab to handle SSH-based `git` sessions, and modifies the list of authorized keys. GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. 556 557#### GitLab Workhorse 558 559- [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md) 560- Configuration: 561 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) 562 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/webservice/) 563 - [Source](../install/installation.md#install-gitlab-workhorse) 564- Layer: Core Service (Processor) 565- Process: `gitlab-workhorse` 566- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 567 568[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Puma. You can read more about the [historical reasons for developing](https://about.gitlab.com/blog/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. 569 570#### Grafana 571 572- [Project page](https://github.com/grafana/grafana/blob/master/README.md) 573- Configuration: 574 - [Omnibus](../administration/monitoring/performance/grafana_configuration.md) 575 - [Charts](https://docs.gitlab.com/charts/charts/globals#configure-grafana-integration) 576- Layer: Monitoring 577- GitLab.com: [GitLab triage Grafana dashboard](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) 578 579Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus, and InfluxDB. 580 581#### Jaeger 582 583- [Project page](https://github.com/jaegertracing/jaeger/blob/master/README.md) 584- Configuration: 585 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4104) 586 - [Charts](https://docs.gitlab.com/charts/charts/globals#tracing) 587 - [Source](../development/distributed_tracing.md#enabling-distributed-tracing) 588 - [GDK](../development/distributed_tracing.md#using-jaeger-in-the-gitlab-development-kit) 589- Layer: Monitoring 590- GitLab.com: [Configuration to enable Tracing for a GitLab instance](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4104) issue. 591 592Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. 593It can be used for monitoring microservices-based distributed systems. 594 595For monitoring deployed apps, see [Jaeger tracing documentation](../operations/tracing.md) 596 597#### Logrotate 598 599- [Project page](https://github.com/logrotate/logrotate/blob/master/README.md) 600- Configuration: 601 - [Omnibus](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate) 602- Layer: Core Service 603- Process: `logrotate` 604 605GitLab is comprised of a large number of services that all log. We bundle our own Logrotate 606to make sure we were logging responsibly. This is just a packaged version of the common open source offering. 607 608#### Mattermost 609 610- [Project page](https://github.com/mattermost/mattermost-server/blob/master/README.md) 611- Configuration: 612 - [Omnibus](../integration/mattermost/index.md) 613 - [Charts](https://docs.mattermost.com/install/install-mmte-helm-gitlab-helm.html) 614- Layer: Core Service (Processor) 615- GitLab.com: [Mattermost](../user/project/integrations/mattermost.md) 616 617Mattermost is an open source, private cloud, Slack-alternative from <https://mattermost.com>. 618 619#### MinIO 620 621- [Project page](https://github.com/minio/minio/blob/master/README.md) 622- Configuration: 623 - [Omnibus](https://min.io/download) 624 - [Charts](https://docs.gitlab.com/charts/charts/minio/) 625 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/object_storage.md) 626- Layer: Core Service (Data) 627- GitLab.com: [Storage Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#storage-architecture) 628 629MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups, and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. 630 631#### NGINX 632 633- Project page: 634 - [Omnibus](https://github.com/nginx/nginx) 635 - [Charts](https://github.com/kubernetes/ingress-nginx/blob/master/README.md) 636- Configuration: 637 - [Omnibus](https://docs.gitlab.com/omnibus/settings/) 638 - [Charts](https://docs.gitlab.com/charts/charts/nginx/) 639 - [Source](../install/installation.md#9-nginx) 640- Layer: Core Service (Processor) 641- Process: `nginx` 642- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 643 644NGINX has an Ingress port for all HTTP requests and routes them to the appropriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver. 645 646#### Node Exporter 647 648- [Project page](https://github.com/prometheus/node_exporter/blob/master/README.md) 649- Configuration: 650 - [Omnibus](../administration/monitoring/prometheus/node_exporter.md) 651 - [Charts](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1332) 652- Layer: Monitoring 653- Process: `node-exporter` 654- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 655 656[Node Exporter](https://github.com/prometheus/node_exporter) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project. 657 658#### Patroni 659 660- [Project Page](https://github.com/zalando/patroni) 661- Configuration: 662 - [Omnibus](../administration/postgresql/replication_and_failover.md#patroni) 663- Layer: Core Service (Data) 664- Process: `patroni` 665- GitLab.com: [Database Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#database-architecture) 666 667#### PgBouncer 668 669- [Project page](https://github.com/pgbouncer/pgbouncer/blob/master/README.md) 670- Configuration: 671 - [Omnibus](../administration/postgresql/pgbouncer.md) 672 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#postgresql) 673- Layer: Core Service (Data) 674- GitLab.com: [Database Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#database-architecture) 675 676Lightweight connection pooler for PostgreSQL. 677 678#### PgBouncer Exporter 679 680- [Project page](https://github.com/prometheus-community/pgbouncer_exporter/blob/master/README.md) 681- Configuration: 682 - [Omnibus](../administration/monitoring/prometheus/pgbouncer_exporter.md) 683 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#postgresql) 684- Layer: Monitoring 685- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 686 687Prometheus exporter for PgBouncer. Exports metrics at 9127/metrics. 688 689#### PostgreSQL 690 691- [Project page](https://github.com/postgres/postgres/blob/master/README) 692- Configuration: 693 - [Omnibus](https://docs.gitlab.com/omnibus/settings/database.html) 694 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#postgresql) 695 - [Source](../install/installation.md#6-database) 696- Layer: Core Service (Data) 697- Process: `postgresql` 698- GitLab.com: [PostgreSQL](../user/gitlab_com/index.md#postgresql) 699 700GitLab packages the popular Database to provide storage for Application meta data and user information. 701 702#### PostgreSQL Exporter 703 704- [Project page](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md) 705- Configuration: 706 - [Omnibus](../administration/monitoring/prometheus/postgres_exporter.md) 707 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#postgresql) 708- Layer: Monitoring 709- Process: `postgres-exporter` 710- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 711 712[`postgres_exporter`](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that delivers data about PostgreSQL to Prometheus for use in Grafana Dashboards. 713 714#### Prometheus 715 716- [Project page](https://github.com/prometheus/prometheus/blob/master/README.md) 717- Configuration: 718 - [Omnibus](../administration/monitoring/prometheus/index.md) 719 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#prometheus) 720- Layer: Monitoring 721- Process: `prometheus` 722- GitLab.com: [Prometheus](../user/gitlab_com/index.md#prometheus) 723 724Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service. 725 726#### Redis 727 728- [Project page](https://github.com/antirez/redis/blob/unstable/README.md) 729- Configuration: 730 - [Omnibus](https://docs.gitlab.com/omnibus/settings/redis.html) 731 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#redis) 732 - [Source](../install/installation.md#7-redis) 733- Layer: Core Service (Data) 734- Process: `redis` 735- GitLab.com: [Service Architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/#service-architecture) 736 737Redis is packaged to provide a place to store: 738 739- session data 740- temporary cache information 741- background job queues 742 743See our [Redis guidelines](redis.md) for more information about how GitLab uses Redis. 744 745#### Redis Exporter 746 747- [Project page](https://github.com/oliver006/redis_exporter/blob/master/README.md) 748- Configuration: 749 - [Omnibus](../administration/monitoring/prometheus/redis_exporter.md) 750 - [Charts](https://docs.gitlab.com/charts/installation/deployment.html#redis) 751- Layer: Monitoring 752- Process: `redis-exporter` 753- GitLab.com: [Monitoring of GitLab.com](https://about.gitlab.com/handbook/engineering/monitoring/) 754 755[Redis Exporter](https://github.com/oliver006/redis_exporter) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana. 756 757#### Registry 758 759- [Project page](https://github.com/docker/distribution/blob/master/README.md) 760- Configuration: 761 - [Omnibus](../update/upgrading_from_source.md#10-install-libraries-migrations-etc) 762 - [Charts](https://docs.gitlab.com/charts/charts/registry/) 763 - [Source](../administration/packages/container_registry.md#enable-the-container-registry) 764 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/registry.md) 765- Layer: Core Service (Processor) 766- GitLab.com: [GitLab Container Registry](../user/packages/container_registry/index.md#build-and-push-by-using-gitlab-cicd) 767 768The registry is what users use to store their own Docker images. The bundled 769registry uses NGINX as a load balancer and GitLab as an authentication manager. 770Whenever a client requests to pull or push an image from the registry, it 771returns a `401` response along with a header detailing where to get an 772authentication token, in this case the GitLab instance. The client then 773requests a pull or push auth token from GitLab and retries the original request 774to the registry. Learn more about [token authentication](https://docs.docker.com/registry/spec/auth/token/). 775 776An external registry can also be configured to use GitLab as an auth endpoint. 777 778#### Sentry 779 780- [Project page](https://github.com/getsentry/sentry/) 781- Configuration: 782 - [Omnibus](https://docs.gitlab.com/omnibus/settings/configuration.html#error-reporting-and-logging-with-sentry) 783 - [Charts](https://docs.gitlab.com/charts/charts/globals#sentry-settings) 784 - [Source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 785 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 786- Layer: Monitoring 787- GitLab.com: [Searching Sentry](https://about.gitlab.com/handbook/support/workflows/500_errors.html#searching-sentry) 788 789Sentry fundamentally is a service that helps you monitor and fix crashes in real time. 790The server is in Python, but it contains a full API for sending events from any language, in any application. 791 792For monitoring deployed apps, see the [Sentry integration docs](../operations/error_tracking.md) 793 794#### Sidekiq 795 796- [Project page](https://github.com/mperham/sidekiq/blob/master/README.md) 797- Configuration: 798 - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) 799 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/) 800 - [Minikube Minimal](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) 801 - [Source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 802 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 803- Layer: Core Service (Processor) 804- Process: `sidekiq` 805- GitLab.com: [Sidekiq](../user/gitlab_com/index.md#sidekiq) 806 807Sidekiq is a Ruby background job processor that pulls jobs from the Redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background. 808 809#### Puma 810 811Starting with GitLab 13.0, Puma is the default web server. 812 813- [Project page](https://gitlab.com/gitlab-org/gitlab/-/blob/master/README.md) 814- Configuration: 815 - [Omnibus](../administration/operations/puma.md) 816 - [Charts](https://docs.gitlab.com/charts/charts/gitlab/webservice/) 817 - [Source](../install/installation.md#configure-it) 818 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 819- Layer: Core Service (Processor) 820- Process: `puma` 821- GitLab.com: [Puma](../user/gitlab_com/index.md#puma) 822 823[Puma](https://puma.io/) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often this displays in process output as `bundle` or `config.ru` depending on the GitLab version. 824 825#### LDAP Authentication 826 827- Configuration: 828 - [Omnibus](../administration/auth/ldap/index.md) 829 - [Charts](https://docs.gitlab.com/charts/charts/globals.html#ldap) 830 - [Source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 831 - [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/ldap.md) 832- Layer: Core Service (Processor) 833- GitLab.com: [Product Tiers](https://about.gitlab.com/pricing/#gitlab-com) 834 835#### Outbound Email 836 837- Configuration: 838 - [Omnibus](https://docs.gitlab.com/omnibus/settings/smtp.html) 839 - [Charts](https://docs.gitlab.com/charts/installation/command-line-options.html#outgoing-email-configuration) 840 - [Source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 841 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 842- Layer: Core Service (Processor) 843- GitLab.com: [Mail configuration](../user/gitlab_com/index.md#mail-configuration) 844 845#### Inbound Email 846 847- Configuration: 848 - [Omnibus](../administration/incoming_email.md) 849 - [Charts](https://docs.gitlab.com/charts/installation/command-line-options.html#incoming-email-configuration) 850 - [Source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 851 - [GDK](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example) 852- Layer: Core Service (Processor) 853- GitLab.com: [Mail configuration](../user/gitlab_com/index.md#mail-configuration) 854 855## GitLab by request type 856 857GitLab provides two "interfaces" for end users to access the service: 858 859- Web HTTP Requests (Viewing the UI/API) 860- Git HTTP/SSH Requests (Pushing/Pulling Git Data) 861 862It's important to understand the distinction as some processes are used in both and others are exclusive to a specific request type. 863 864### GitLab Web HTTP request cycle 865 866When making a request to an HTTP Endpoint (think `/users/sign_in`) the request takes the following path through the GitLab Service: 867 868- NGINX - Acts as our first line reverse proxy. 869- GitLab Workhorse - This determines if it needs to go to the Rails application or somewhere else to reduce load on Puma. 870- Puma - Since this is a web request, and it needs to access the application, it routes to Puma. 871- PostgreSQL/Gitaly/Redis - Depending on the type of request, it may hit these services to store or retrieve data. 872 873### GitLab Git request cycle 874 875Below we describe the different paths that HTTP vs. SSH Git requests take. There is some overlap with the Web Request Cycle but also some differences. 876 877### Web request (80/443) 878 879Git operations over HTTP use the stateless "smart" protocol described in the 880[Git documentation](https://git-scm.com/docs/http-protocol), but responsibility 881for handling these operations is split across several GitLab components. 882 883Here is a sequence diagram for `git fetch`. Note that all requests pass through 884NGINX as well as any other HTTP load balancers, but are not transformed in any 885way by them. All paths are presented relative to a `/namespace/project.git` URL. 886 887```mermaid 888sequenceDiagram 889 participant Git on client 890 participant NGINX 891 participant Workhorse 892 participant Rails 893 participant Gitaly 894 participant Git on server 895 896 Note left of Git on client: git fetch<br/>info-refs 897 Git on client->>+Workhorse: GET /info/refs?service=git-upload-pack 898 Workhorse->>+Rails: GET /info/refs?service=git-upload-pack 899 Note right of Rails: Auth check 900 Rails-->>-Workhorse: Gitlab::Workhorse.git_http_ok 901 Workhorse->>+Gitaly: SmartHTTPService.InfoRefsUploadPack request 902 Gitaly->>+Git on server: git upload-pack --stateless-rpc --advertise-refs 903 Git on server-->>-Gitaly: git upload-pack response 904 Gitaly-->>-Workhorse: SmartHTTPService.InfoRefsUploadPack response 905 Workhorse-->>-Git on client: 200 OK 906 907 Note left of Git on client: git fetch<br/>fetch-pack 908 Git on client->>+Workhorse: POST /git-upload-pack 909 Workhorse->>+Rails: POST /git-upload-pack 910 Note right of Rails: Auth check 911 Rails-->>-Workhorse: Gitlab::Workhorse.git_http_ok 912 Workhorse->>+Gitaly: SmartHTTPService.PostUploadPack request 913 Gitaly->>+Git on server: git upload-pack --stateless-rpc 914 Git on server-->>-Gitaly: git upload-pack response 915 Gitaly-->>-Workhorse: SmartHTTPService.PostUploadPack response 916 Workhorse-->>-Git on client: 200 OK 917``` 918 919The sequence is similar for `git push`, except `git-receive-pack` is used 920instead of `git-upload-pack`. 921 922### SSH request (22) 923 924Git operations over SSH can use the stateful protocol described in the 925[Git documentation](https://git-scm.com/docs/pack-protocol#_ssh_transport), but 926responsibility for handling them is split across several GitLab components. 927 928No GitLab components speak SSH directly - all SSH connections are made between 929Git on the client machine and the SSH server, which terminates the connection. 930To the SSH server, all connections are authenticated as the `git` user; GitLab 931users are differentiated by the SSH key presented by the client. 932 933Here is a sequence diagram for `git fetch`, assuming [Fast SSH key lookup](../administration/operations/fast_ssh_key_lookup.md) 934is enabled. Note that `AuthorizedKeysCommand` is an executable provided by 935[GitLab Shell](#gitlab-shell): 936 937```mermaid 938sequenceDiagram 939 participant Git on client 940 participant SSH server 941 participant AuthorizedKeysCommand 942 participant GitLab Shell 943 participant Rails 944 participant Gitaly 945 participant Git on server 946 947 Note left of Git on client: git fetch 948 Git on client->>+SSH server: ssh git fetch-pack request 949 SSH server->>+AuthorizedKeysCommand: gitlab-shell-authorized-keys-check git AAAA... 950 AuthorizedKeysCommand->>+Rails: GET /internal/api/authorized_keys?key=AAAA... 951 Note right of Rails: Lookup key ID 952 Rails-->>-AuthorizedKeysCommand: 200 OK, command="gitlab-shell upload-pack key_id=1" 953 AuthorizedKeysCommand-->>-SSH server: command="gitlab-shell upload-pack key_id=1" 954 SSH server->>+GitLab Shell: gitlab-shell upload-pack key_id=1 955 GitLab Shell->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1 956 Note right of Rails: Auth check 957 Rails-->>-GitLab Shell: 200 OK, { gitaly: ... } 958 GitLab Shell->>+Gitaly: SSHService.SSHUploadPack request 959 Gitaly->>+Git on server: git upload-pack request 960 Note over Git on client,Git on server: Bidirectional communication between Git client and server 961 Git on server-->>-Gitaly: git upload-pack response 962 Gitaly -->>-GitLab Shell: SSHService.SSHUploadPack response 963 GitLab Shell-->>-SSH server: gitlab-shell upload-pack response 964 SSH server-->>-Git on client: ssh git fetch-pack response 965``` 966 967The `git push` operation is very similar, except `git receive-pack` is used 968instead of `git upload-pack`. 969 970If fast SSH key lookups are not enabled, the SSH server reads from the 971`~git/.ssh/authorized_keys` file to determine what command to run for a given 972SSH session. This is kept up to date by an [`AuthorizedKeysWorker`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/authorized_keys_worker.rb) 973in Rails, scheduled to run whenever an SSH key is modified by a user. 974 975[SSH certificates](../administration/operations/ssh_certificates.md) may be used 976instead of keys. In this case, `AuthorizedKeysCommand` is replaced with an 977`AuthorizedPrincipalsCommand`. This extracts a username from the certificate 978without using the Rails internal API, which is used instead of `key_id` in the 979[`/api/internal/allowed`](internal_api/index.md) call later. 980 981GitLab Shell also has a few operations that do not involve Gitaly, such as 982resetting two-factor authentication codes. These are handled in the same way, 983except there is no round-trip into Gitaly - Rails performs the action as part 984of the [internal API](internal_api/index.md) call, and GitLab Shell streams the 985response back to the user directly. 986 987## System layout 988 989When referring to `~git` in the pictures it means the home directory of the Git user which is typically `/home/git`. 990 991GitLab is primarily installed within the `/home/git` user home directory as `git` user. Within the home directory is where the GitLab server software resides as well as the repositories (though the repository location is configurable). 992 993The bare repositories are located in `/home/git/repositories`. GitLab is a Ruby on rails application so the particulars of the inner workings can be learned by studying how a Ruby on rails application works. 994 995To serve repositories over SSH there's an add-on application called GitLab Shell which is installed in `/home/git/gitlab-shell`. 996 997### Installation folder summary 998 999To summarize here's the [directory structure of the `git` user home directory](../install/installation.md#gitlab-directory-structure). 1000 1001### Processes 1002 1003```shell 1004ps aux | grep '^git' 1005``` 1006 1007GitLab has several components to operate. It requires a persistent database 1008(PostgreSQL) and Redis database, and uses Apache `httpd` or NGINX to `proxypass` 1009Puma. All these components should run as different system users to GitLab 1010(for example, `postgres`, `redis`, and `www-data`, instead of `git`). 1011 1012As the `git` user it starts Sidekiq and Puma (a simple Ruby HTTP server 1013running on port `8080` by default). Under the GitLab user there are normally 4 1014processes: `puma master` (1 process), `puma cluster worker` 1015(2 processes), `sidekiq` (1 process). 1016 1017### Repository access 1018 1019Repositories get accessed via HTTP or SSH. HTTP cloning/push/pull uses the GitLab API and SSH cloning is handled by GitLab Shell (previously explained). 1020 1021## Troubleshooting 1022 1023See the README for more information. 1024 1025### Init scripts of the services 1026 1027The GitLab init script starts and stops Puma and Sidekiq: 1028 1029```plaintext 1030/etc/init.d/gitlab 1031Usage: service gitlab {start|stop|restart|reload|status} 1032``` 1033 1034Redis (key-value store/non-persistent database): 1035 1036```plaintext 1037/etc/init.d/redis 1038Usage: /etc/init.d/redis {start|stop|status|restart|condrestart|try-restart} 1039``` 1040 1041SSH daemon: 1042 1043```plaintext 1044/etc/init.d/sshd 1045Usage: /etc/init.d/sshd {start|stop|restart|reload|force-reload|condrestart|try-restart|status} 1046``` 1047 1048Web server (one of the following): 1049 1050```plaintext 1051/etc/init.d/httpd 1052Usage: httpd {start|stop|restart|condrestart|try-restart|force-reload|reload|status|fullstatus|graceful|help|configtest} 1053 1054$ /etc/init.d/nginx 1055Usage: nginx {start|stop|restart|reload|force-reload|status|configtest} 1056``` 1057 1058Persistent database: 1059 1060```plaintext 1061$ /etc/init.d/postgresql 1062Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [version ..] 1063``` 1064 1065### Log locations of the services 1066 1067GitLab (includes Puma and Sidekiq logs): 1068 1069- `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `puma.stdout.log`, `git_json.log` and `puma.stderr.log` normally. 1070 1071GitLab Shell: 1072 1073- `/home/git/gitlab-shell/gitlab-shell.log` 1074 1075SSH: 1076 1077- `/var/log/auth.log` auth log (on Ubuntu). 1078- `/var/log/secure` auth log (on RHEL). 1079 1080NGINX: 1081 1082- `/var/log/nginx/` contains error and access logs. 1083 1084Apache `httpd`: 1085 1086- [Explanation of Apache logs](https://httpd.apache.org/docs/2.2/logs.html). 1087- `/var/log/apache2/` contains error and output logs (on Ubuntu). 1088- `/var/log/httpd/` contains error and output logs (on RHEL). 1089 1090Redis: 1091 1092- `/var/log/redis/redis.log` there are also log-rotated logs there. 1093 1094PostgreSQL: 1095 1096- `/var/log/postgresql/*` 1097 1098### GitLab specific configuration files 1099 1100GitLab has configuration files located in `/home/git/gitlab/config/*`. Commonly referenced 1101configuration files include: 1102 1103- `gitlab.yml`: GitLab configuration 1104- `puma.rb`: Puma web server settings 1105- `database.yml`: Database connection settings 1106 1107GitLab Shell has a configuration file at `/home/git/gitlab-shell/config.yml`. 1108 1109### Maintenance tasks 1110 1111[GitLab](https://gitlab.com/gitlab-org/gitlab/-/tree/master) provides Rake tasks with which you see version information and run a quick check on your configuration to ensure it is configured properly within the application. See [maintenance Rake tasks](../administration/raketasks/maintenance.md). 1112In a nutshell, do the following: 1113 1114```shell 1115sudo -i -u git 1116cd gitlab 1117bundle exec rake gitlab:env:info RAILS_ENV=production 1118bundle exec rake gitlab:check RAILS_ENV=production 1119``` 1120 1121It's recommended to sign in to the `git` user using either `sudo -i -u git` or 1122`sudo su - git`. Although the `sudo` commands provided by GitLab work in Ubuntu, 1123they don't always work in RHEL. 1124 1125## GitLab.com 1126 1127The [GitLab.com architecture](https://about.gitlab.com/handbook/engineering/infrastructure/production/architecture/) 1128is detailed for your reference, but this architecture is only useful if you have 1129millions of users. 1130