Gitlab

This guide provides step-by-step instructions for installing a self-hosted GitLab instance on Ubuntu 20.04 or 22.04, as well as how to perform backups and restores of your GitLab data. Additionally, this guide includes instructions for migrating GitLab to a new instance, such as when using an Amazon EC2 instance.

Table of Contents:

1. Install and Configure Dependencies
2. Add GitLab Repository and Install Package
3. Initial Login and Password Setup
4. Migrating GitLab to a New Instance & Performing Backups and Restores
5. Configuring gitlab.rb
6. Configuring crontab

1. Install and Configure Dependencies

Open a terminal and execute the following commands to update the package list and install the necessary dependencies:

sudo apt-get update
sudo apt-get install -y curl openssh-server ca-certificates tzdata perl

2. Add GitLab Repository and Install Package

1. To add the GitLab package repository and install the package, run the following commands:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash
sudo apt-get install gitlab-ee # [for enterprise edition]
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo apt-get install gitlab-ce # [for community edition]

3. Initial Login and Password Setup

After installation, open a web browser and navigate to your server’s hostname. Set up an initial password:

If you didn’t provide a custom password during installation, you can find the randomly generated password at `/etc/gitlab/initial_root_password`.

Use the username `root` and this password to log in.

4. Migrating GitLab to a New Instance & Performing Backups and Restores

If you are migrating GitLab to a new instance, such as on an Amazon EC2 instance, follow these steps:

1. Create a new EC2 instance from an appropriate AMI.
2. Copy your GitLab configuration and data files to the new instance.

aws s3 cp s3://xxxxxxxxxxx-gitlab-backup/1666083747_2022_10_18_15.4.2_gitlab_backup.tar .

Copy files to the appropriate path

tar -xzvf gitlab_config_1691596803_2023_08_09.tar
remove all files from /etc/gitlab
copy untar gitlab_config file to the /etc/gitlab
copy 1666083747_2022_10_18_15.4.2_gitlab_backup.tar to /var/opt/gitlab/backups

Stop gitlab

1. Below are the commands:

sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop puma
sudo gitlab-ctl stop sidekiq

2. Start the restore process with backup commands:

sudo gitlab-backup restore BACKUP= 1668186028_2022_11_11_15.4.2_gitlab_backup
sudo gitlab-ctl start unicorn
sudo gitlab-ctl start puma
sudo gitlab-ctl start sidekiq 
sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart

Note: Restore will be created only to the version backup is created

5. Gitlab.rb Configuration

Configure the external URL for GitLab

To display the correct repository clone links to our users, we must provide GitLab with the URL our users use to reach the repository. We can use the IP of our server, but we are using ALB.

external_url 'http://dev-gitlab-alb-966713332.ap-south-1.elb.amazonaws.com'

1. To give ssh host

gitlab_rails['gitlab_ssh_host'] = 'gitlab.abhi.in'

2. SMTP settings

If we would rather send application email via an SMTP server instead of via Sendmail or Postfix, add the following configuration information to /etc/gitlab/gitlab.rb for our own SMTP server & run gitlab-ctl reconfigure

gitlab_rails['smtp_enable'] = true 
gitlab_rails['smtp_address'] = "emailrelay.xxxx.in" 
gitlab_rails['smtp_port'] = 25 
gitlab_rails['smtp_authentication'] = "login" 
gitlab_rails['smtp_domain'] = "http://xxxxx.in " 
gitlab_rails['gitlab_email_from'] = 'xxxx@xxxx.in' 
gitlab_rails['gitlab_email_reply_to'] = 'abhishek.xxx@xxx.com'

3. Artifact Deletion

Our instances are giving 500 errors because of the full disk and artifact size being so big so we integrated the artifacts auto deletion cron job, added the following configuration in gitlab.rb, and ran gitlab-ctl reconfigure.

gitlab_rails['expire_build_artifacts_worker_cron'] = "*/2 * * * *"

4. Artifact backup in S3 bucket

We are taking a backup of artifacts in the S3 bucket, adding the following configuration in gitlab. rb, and running gitlab-ctl reconfigure.

gitlab_rails['artifacts_enabled'] = true 
gitlab_rails['artifacts_object_store_enabled'] = true 
gitlab_rails['artifacts_object_store_proxy_download'] = false 
gitlab_rails['artifacts_object_store_remote_directory'] = "s3-bucket-name" 
gitlab_rails['artifacts_object_store_connection'] = { 
   'provider' => 'AWS', 
   'region' => 'xxxxxxx', 
   'aws_access_key_id' => 'XXXXXXXXXXXXXXXXXXXXXXXXXX', // give only s3 access 
   'aws_secret_access_key' => 'XXXXXXXXXXXXXXXXXXXXX' // give only s3 access 
}

1. Taking backup of Gitlab repository, artifact, and users.
2. We are taking a backup of all gitlab data except configuration backup in the S3 bucket by adding the following configuration in gitlab.rb, and running gitlab-ctl reconfigure.

gitlab_rails['manage_backup_path'] = true  
gitlab_rails['backup_path'] = "/newvolume"  
gitlab_rails['backup_gitaly_backup_path'] = "/opt/gitlab/embedded/bin/gitaly-backup"  
gitlab_rails['backup_archive_permissions'] = 0644  
gitlab_rails['backup_pg_schema'] = 'public'  
gitlab_rails['backup_keep_time'] = 3600  
gitlab_rails['backup_upload_connection'] = {'provider' => 'AWS','region' => 'ap-south-1', 'aws_access_key_id' => 'XXXXXXXXXXXXXXXXXXXXXXX','aws_secret_access_key' => 'XXXXXXXXXXXXXXXXXXXXXXXXXXXX'}
gitlab_rails['backup_upload_remote_directory'] = 's3-bucket-name'  
gitlab_rails['backup_multipart_chunk_size'] = 1048576000

6. Crontab Configuration

Add the following Crontab Configuration to the Crontab file to take a backup of data and configuration and delete the backup from the server daily

0 15 * * * root  gitlab-backup create 
0 16 * * * root  gitlab-ctl backup-etc && cd /etc/gitlab/config_backup && aws s3 cp "$(ls -tp | grep -v /$ | head -1)" s3://xxxxxxxxxx-gitlab-backup/ 
0 18 * * * root  cd /newvolume/ && aws s3 cp "$(ls -tp | grep -v /$ | head -1)" s3://bucket-name/gitlab-backup/ 
0 19 * * * root  cd /newvolume/ && rm "$(ls -tp | grep -v /$ | head -1)"

Area of Improvement:

1. Remove the access key and secret key and use IAM Roles
2. Monitoring GitLab
3. External PostgreSQL integration