[分享] 极狐Helm Charts安装GitLab常见配置汇总

使用极狐helm charts安装GitLab场景汇总

添加极狐GitLab helm 仓库

首先,我们必须能够安装 gitlab-jh/gitlab。为此,我们必须将仓库添加到 helm 的配置中:

helm repo add gitlab-jh https://charts.gitlab.cn

快速安装

只需一个命令即可安装,并配置 SSL。

为了正确配置 chart,我们需要以下配置:

  1. 将在其下运行极狐GitLab 的域名或子域。
  2. 您的电子邮件地址,Let’s Encrypt 可以颁发证书。

安装 chart 的 helm install 命令示例如下,以安装 5.9.1 版本 chart 为例(应用版本为 14.9.1):

helm install gitlab gitlab-jh/gitlab \
  --version 5.9.1 \
  --set global.hosts.domain=DOMAIN \
  --set certmanager-issuer.email=me@example.com 

自定义配置

指定values文件

新建一个values文件 gitlab-values.yaml,该示例文件指定了以下几项内容:

  • gitlab域名和ip地址
  • 是否使用Cert-manager以及Cert-manager 关联邮箱地址
  • 绑定腾讯云的clb(该clb提供了上述的Ip地址)
  • 有状态组件所需要的存储大小
global:
  hosts:
    domain: secondary.devxops.cn 
    externalIP: "1.14.223.191" 
  ingress:
    configureCertmanager: true
certmanager-issuer:
  email: devops@bigbige.com # use your real email address here
nginx-ingress:
  controller:
    service:
      annotations:
        service.kubernetes.io/tke-existed-lbid: lb-3zdkb2em  # 绑定建好的clb
postgresql:
  persistence:
    size: 20Gi
redis:
  master:
    persistence:
      size: 20Gi
prometheus:
  server:
    persistentVolume:
      size: 20Gi
gitlab:
  gitaly:
    persistence:
      size: 20Gi

安装

helm -n gitlab upgrade --install gitlab gitlab-jh/gitlab --version 5.9.2 \
  --timeout 600s \
 -f gitlab-values.yaml

Https or Http

禁用https

新建一个values文件 gitlab-values.yaml

global:
  hosts:
    domain: secondary.devxops.cn 
    externalIP: "1.14.223.191" 
nginx-ingress:
  controller:
    service:
      annotations:
        service.kubernetes.io/tke-existed-lbid: lb-3zdkb2em  # 绑定建好的clb
postgresql:
  persistence:
    size: 20Gi
redis:
  master:
    persistence:
      size: 20Gi
prometheus:
  server:
    persistentVolume:
      size: 20Gi
gitlab:
  gitaly:
    persistence:
      size: 20Gi

部署命令指定以下几个值(也可以添加到上述values文件中):

helm -n gitlab upgrade --install gitlab gitlab-jh/gitlab  \
  --timeout 600s \
  --version 5.9.2 \
  --set global.ingress.configureCertmanager=false \
  --set global.ingress.tls.enabled=false \
  --set certmanager.install=false \
  --set global.hosts.https=false 

禁用Cert-Manager,单独维护证书

需要准备三个域名的证书,泛域名也可以。以devxops.cn为例:

或者:

  • *.devxops.cn

创建secrets:

kubectl -n gitlab create secret tls gitlab-devxops-ssl --cert=gitlab.devxops.cn.pem --key=gitlab.devxops.cn.key
kubectl -n gitlab create secret tls registry-devxops-ssl --cert=registry.devxops.cn.pem --key=registry.devxops.cn.key
kubectl -n gitlab create secret tls minio-devxops-ssl --cert=minio.devxops.cn.pem --key=minio.devxops.cn.key

# 或者泛域名
kubectl -n gitlab create secret tls gitlab-common-ssl --cert=devxops.cn.pem --key=devxops.cn.key

新建一个values文件 gitlab-values.yaml

global:
  hosts:
    domain: devxops.cn 
    externalIP: "1.14.223.191" 
nginx-ingress:
  controller:
    service:
      annotations:
        service.kubernetes.io/tke-existed-lbid: lb-3zdkb2em  # 绑定建好的clb
postgresql:
  persistence:
    size: 20Gi
redis:
  master:
    persistence:
      size: 20Gi
prometheus:
  server:
    persistentVolume:
      size: 20Gi
gitlab:
  gitaly:
    persistence:
      size: 20Gi

部署命令指定以下几个值(也可以添加到上述values文件中):

独立域名情况:

helm -n gitlab upgrade --install gitlab gitlab/gitlab  --version 5.9.2 \
  --timeout 600s \
  -f gitlab-values.yaml \
  --set global.ingress.configureCertmanager=false \ 
  --set gitlab.webservice.ingress.tls.secretName=gitlab-devxops-ssl \ 
  --set registry.ingress.tls.secretName=registry-devxops-ssl \
  --set minio.ingress.tls.secretName=minio-devxops-ssl 

泛域名情况:

helm -n gitlab upgrade --install gitlab gitlab/gitlab  --version 5.9.2 \
  --timeout 600s \
  -f gitlab-values.yaml \
  --set global.ingress.configureCertmanager=false \ 
  --set global.ingress.tls.secretName=gitlab-common-ssl

功能相关

禁用registry

修改values文件,确保以下值存在:

registry:
  enabled: false

gitlab:
	sidekiq:
	  registry: 
	    enabled: false
	
gitlab:
	webservice:
	  registry: 
	    enabled: false

runner绑定到某个node节点

values中可定义的值可以参考https://gitlab.com/gitlab-org/charts/gitlab-runner/blob/main/values.yaml

以gitlab自带的runner为例:

将runner绑定到某个node节点,修改values文件,添加以下内容:

gitlab-runner:
  nodeSelector:
    "kubernetes.io/hostname": "node name"

将runner启动的并发job pod也绑定到某个node节点,修改values文件,添加以下内容:

gitlab-runner:
  runners:
    config: |
      [[runners]]
        ...
        ...

        [runners.kubernetes.node_selector]
          "kubernetes.io/hostname" = "node name"

注册额外的runner,并打上tag

列出支持的runner版本

helm search repo -l gitlab-jh/gitlab-runner

创建 gitlab-runner.yaml,并为runner打上 kubernetesdevops的tag:

runners:
  # runner configuration, where the multi line strings is evaluated as
  # template so you can specify helm values inside of it.
  #
  # tpl: https://helm.sh/docs/howto/charts_tips_and_tricks/#using-the-tpl-function
  # runner configuration: https://docs.gitlab.com/runner/configuration/advanced-configuration.html
  config: |
    [[runners]]
      [runners.kubernetes]
        namespace = "{{.Release.Namespace}}"
        image = "ubuntu:16.04"
  tags: "kubernetes,devops" ## 给runner打tag

安装

helm upgrade --install  gitlab-runner-devops --set gitlabUrl=https://gitlab.xxx.cn --set runnerRegistrationToken=xxxxxxx gitlab-jh/gitlab-runner --version 0.39.0 -f gitlab-runner.yaml

更多配置,参考GitLab Runner Helm Chart

Gitlab runner在k8s中部署时使用dind

修改 gitlab-runner.yaml并helm安装

runners:
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:20.04"
        privileged = true
      [[runners.kubernetes.volumes.empty_dir]]
        name = "docker-certs"
        mount_path = "/certs/client"
        medium = "Memory"

.gitlab-ci.yml 如下定义:

image: docker:19.03.13

variables:
  # When using dind service, you must instruct Docker to talk with
  # the daemon started inside of the service. The daemon is available
  # with a network connection instead of the default
  # /var/run/docker.sock socket.
  DOCKER_HOST: tcp://docker:2376
  #
  # The 'docker' hostname is the alias of the service container as described at
  # https://docs.gitlab.com/ee/ci/services/#accessing-the-services.
  # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier,
  # the variable must be set to tcp://localhost:2376 because of how the
  # Kubernetes executor connects services to the job container
  # DOCKER_HOST: tcp://localhost:2376
  #
  # Specify to Docker where to create the certificates. Docker
  # creates them automatically on boot, and creates
  # `/certs/client` to share between the service and job
  # container, thanks to volume mount from config.toml
  DOCKER_TLS_CERTDIR: "/certs"
  # These are usually specified by the entrypoint, however the
  # Kubernetes executor doesn't run entrypoints
  # https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4125
  DOCKER_TLS_VERIFY: 1
  DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"

services:
  - docker:19.03.13-dind

before_script:
  - docker info

build:
  stage: build
  script:
    - docker build -t my-docker-image .
    - docker run my-docker-image /script/to/run/tests

集成相关

LDAP

修改values文件,添加ldap配置,如下:

global:
  appConfig:
    ldap:
      servers:
        main:
          base: dc=example,dc=com
          bind_dn: cn=read-only-admin,dc=example,dc=com
          encryption: plain
          host: ldap.forumsys.com
          label: LDAP
          password:
            key: secret
            secret: gitlab-ldap-secret
          port: 389
          uid: uid

创建ldap的secrets:

kubectl create secret generic gitlab-ldap-secret \
  --from-literal=secret=password

执行helm安装

使用外部资源

使用你自己的nginx ingress

  1. 修改你的nginx ingress配置,启用22端口:

    1. 首先创建configmap:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: tcp-configmap-example
    data:
      22: "gitlab/mygitlab-gitlab-shell:22"
    

    b. 然后修改nginx-ingress的启动参数:

    args:
      - /nginx-ingress-controller
      - --tcp-services-configmap=gitlab/tcp-configmap-example
    
  2. helm 更新gitlab,指定以下参数

    --set global.ingress.class=myingressclass \
    --set nginx-ingress.enabled=false \
    --set certmanager.install=false \
    --set global.ingress.configureCertmanager=false
    

使用外部数据库

创建secrets,包含数据库密码

kubectl create secret generic gitlab-postgresql-password \
--from-literal=postgresql-password=xxxxxx -n gitlab

修改values.yaml文件,添加以下内容:

  • postgresql.install: false
  • global.psql.host: 10.10.0.14
  • global.psql.password.secret: gitlab-postgresql-password
  • global.psql.password.key: postgresql-password

以下参数根据实际情况修改:

  • global.psql.port: 默认是5432
  • global.psql.database: gitlabhq_production
  • global.psql.username: gitlab

使用外部对象存储

使用外部存储(比如s3,Azure Blob)时,配置会主要分为三类:

  • registry
  • LFS, Artifacts, Uploads, Packages, External Diffs, Pseudonymizer, Terraform State, Dependency Proxy
  • backup

配置之前,先创建需要的存储桶,参考https://docs.gitlab.com/charts/charts/globals.html#specify-buckets

Registry存储配置

# vim registry.s3.yaml
s3:
  bucket: gitlab-registry-storage
  accesskey: BOGUS_ACCESS_KEY
  secretkey: BOGUS_SECRET_KEY
  region: us-east-1
  # regionendpoint: "https://minio.example.com:9000"
  v4auth: true

helm安装时指定以下参数:

--set registry.storage.secret=registry-storage \
--set registry.storage.key=config \
--set global.registry.bucket=bucket-name

Rails存储配置和备份配置(以腾讯云为例),其它云可以参考 https://docs.gitlab.com/ee/administration/object_storage.html#s3-compatible-connection-settings

# vim rails.s3.yaml
provider: AWS
region: eu-central-1
aws_access_key_id: xxxxxx
aws_secret_access_key: xxxxxx
aws_signature_version: 2
host: cos.ap-guangzhou.myqcloud.com
# vim storage.config
[default]
host_base=cos.ap-guangzhou.myqcloud.com
host_bucket = %(bucket)s.cos.ap-guangzhou.myqcloud.com
access_key = xxxxx
secret_key = xxxxx
bucket_location = us-east-1
multipart_chunk_size_mb = 128

创建secrets

kubectl create secret generic gitlab-rails-storage --fromfile=connection=rails.s3.yaml
kubectl create secret generic storage-config --from-file=config=storage.config

按照以下内容修改values文件

  • global.minio.enabled: false
  • global.appConfig.object_store.enabled: true
  • global.appConfig.object_store.connection.secret: gitlab-rails-storage

helm 安装时指定以下参数 :

--set global.appConfig.object_store.connection.secret=gitlab-rails-storage \
--set global.appConfig.object_store.connection.key=connection \
--set global.appConfig.lfs.bucket=gitlab-lfs-1306773458 \
--set global.appConfig.artifacts.bucket=gitlab-artifacts-1306773458 \
--set global.appConfig.uploads.bucket=gitlab-uploads-1306773458 \
--set global.appConfig.packages.bucket=gitlab-packages-1306773458 \
--set global.appConfig.externalDiffs.bucket=gitlab-externaldiffs-1306773458 \
--set global.appConfig.terraformState.bucket=gitlab-tf-state-1306773458 \
--set global.appConfig.pseudonymizer.bucket=pseudonymizer-1306773458 \
--set global.appConfig.dependencyProxy.bucket=dependencyproxy-1306773458 \-
-set global.appConfig.backups.bucket=gitlab-backup-1306773458 \
--set global.appConfig.backups.tmpBucket=gitlab-tmp-1306773458 \
--set gitlab.toolbox.backups.objectStorage.config.secret=storage-config \
--set gitlab.toolbox.backups.objectStorage.config.key=config

使用独立的Gitaly Cluster

创建praefect secret

kubectl create secret generic gitlab-praefect-secret \
--from-literal=token=xxxxxx # 替换为PRAEFECT_EXTERNAL_TOKEN

参考下面的内容修改values

global:
  gitaly:
    enabled: true
    internal:
      names:
        - default
    external:
      - name: praefect
        hostname: lb_address
        port: 2305
        tlsEnabled: false
    authToken:
      secret: gitlab-praefect-secret
      key: token
    tls:
      enabled: false

备份相关

启用定时备份

设定以下参数:

  • gitlab.toolbox.backups.cron.enabled: true
  • gitlab.toolbox.backups.cron.schedule: “0 1 * * *”
  • gitlab.toolbox.backups.cron.extraArgs: 可选,比如 --skip db
1 个赞