Aller au contenu principal

Gitlab - Tenants Management & Visibility

User creation​

New User will be created in gitlab when a valid user connect through SSO. This new user will not be part of any group by default.

Tenant Creation​

In order to create a Tenant, create a new Group in gitlab named as your Tenant :

  • Left panel > Group
  • Click on New group > Create group
  • Name the group and set its visibility to Private.
  • Validate the creation, which will send you to the new group page

In order to add a Tenant administrator to your Tenant :

  • Left panel > Group
  • Pick the Tenant you wish to configure
  • Left panel > Manage > Members
  • Click on Invite members
  • Find your Tenant administrator and give it the Owner role
    • Optionally provide an expiration date
  • Finish the process by clicking on Invite

Adding a Tenant runner​

Introduction​

A runner is a pod that will spawn other pods when CI/CD pipelines are submitted in the gitlab UI. Adding a Tenant runner allow the Tenant to have a dedicated gitlab runner for their CI/CD needs.

Initial creation of the agent in Gitlab UI​

In order to create a runner :

  • Left panel > Group
  • Pick the Tenant you wish to configure (tenant0)
  • Left panel > Build > Runners
  • Click on New group Runner
  • Configure your runner
  • Pay attention to the runner authentication token displayed, it will NOT be shown ever again and you need it to register your runner in later steps.

Instantiate the runner in the cluster​

The runner can be deployed using helm in the cluster using the gitlab helm chart.

An exemple of values is provided below with some changes needed :

  • Put your token retrieved from the UI in the gitlab-runner.runnerToken field.
  • If using self-signed certificates (the exemple below assumes you do), you need to :
    • provide a CA that can be used to validate the gitlab KAS (Kubernetes Agent Service) Ingress in the gitlab-runner.runners.extraObjects key as a ConfigMap
      • the CA certificate can be found in the tls secret generated by cert-manager
    • provide the [[runners.kubernetes.volumes.config_map]] bloc
    • add the SSL_CERT_FILE environment variable and its value that match the configmap mount path
---
gitlab-runner:
    install: true
    gitlabUrl: http://gitlab-webservice-default.kosmos-dev.svc.cluster.local:8181
    securityContext:
        seccompProfile:
            type: RuntimeDefault
    rbac:
        create: true
    # Put your token retrieved from the UI here or pass it with the helm install param '--set config.token=${token}'
    runnerToken: PUT_YOUR_TOKEN_FROM_UI_HERE
    runners:
        locked: nil
        config: |
            [[runners]]
              clone_url = "http://gitlab-webservice-default.kosmos-dev.svc.cluster.local:8181"
              environment = ["FF_USE_ADVANCED_POD_SPEC_CONFIGURATION=true", "HOME=/tmp", "SSL_CERT_FILE=/tmp/kas/ca.crt"]
              [runners.kubernetes]
              helper_image = "docker.io/gitlab/gitlab-runner-helper:ubuntu-x86_64-bleeding"
              image = "ubuntu:22.04"
              namespace = "{{ .Release.Namespace }}"
              privileged = false
              cap_drop = ["ALL"]
              [[runners.kubernetes.volumes.config_map]]
                name = "kas-tls-ca"
                mount_path = "/tmp/kas/ca.crt"
                sub_path = "ca.crt"
              [runners.kubernetes.pod_security_context]
                run_as_non_root = true
                run_as_user = 1000
              [[runners.kubernetes.pod_spec]]
                name = "container security policies patch"
                patch_type = "strategic"
                patch = '''
                  initContainers:
                    - name: init-permissions
                      securityContext:
                        allowPrivilegeEscalation: false
                        capabilities:
                          drop:
                            - ALL
                        runAsNonRoot: true
                        seccompProfile:
                          type: RuntimeDefault
                  containers:
                    - name: build
                      securityContext:
                        allowPrivilegeEscalation: false
                        capabilities:
                          drop:
                            - ALL
                        runAsNonRoot: true
                        seccompProfile:
                          type: RuntimeDefault
                    - name: helper
                      securityContext:
                        allowPrivilegeEscalation: false
                        capabilities:
                          drop:
                            - ALL
                        runAsNonRoot: true
                        seccompProfile:
                          type: RuntimeDefault
                    - name: build
                      securityContext:
                        allowPrivilegeEscalation: false
                        capabilities:
                          drop:
                            - ALL
                        runAsNonRoot: true
                        seccompProfile:
                          type: RuntimeDefault
                '''
              {{- if .Values.global.minio.enabled }}
              [runners.cache]
                Type = "s3"
                Path = "gitlab-runner"
                Shared = true
                [runners.cache.s3]
                  ServerAddress = {{ include "gitlab-runner.cache-tpl.s3ServerAddress" . }}
                  BucketName = "runner-cache"
                  BucketLocation = "us-east-1"
                  Insecure = false
              {{ end }}
    extraObjects:
        - apiVersion: v1
          data: # Put your KAS CA here (or remove if you are not using self signed certificates)
              ca.crt: |
                  -----BEGIN CERTIFICATE-----
                  MIIBfjCCASWgAwIBAgIRAIaCvqh9Itd8dobZz26kCPEwCgYIKoZIzj0EAwIwHzEd
                  MBsGA1UEAxMUa29zbW9zLXNlbGZzaWduZWQtY2EwHhcNMjUwMzAzMTY0MTUwWhcN
                  MjUwNjAxMTY0MTUwWjAfMR0wGwYDVQQDExRrb3Ntb3Mtc2VsZnNpZ25lZC1jYTBZ
                  MBMGByqGSM49AgEGCCqGSM49AwEHA0IABAOPhAPdqPhOW97iU8Sp739eOQxAMq+w
                  f+vE0eQD4Mi8RCPWlLtBax+pB5wPy6MyumNarErIcnmuNQbjdRBXsZ2jQjBAMA4G
                  A1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRdy3/BJl8P
                  0uwoiy4eMm4J0DvjLzAKBggqhkjOPQQDAgNHADBEAiA2C7gZKWU7ZvgZ9WQAcG47
                  uDbofXBHlPOcoFrwl+dmiQIgGkdsoZJl93VcTZhsrcQWhxm0m/kl9GGpXl7ui/Cf
                  NKI=
                  -----END CERTIFICATE-----
          kind: ConfigMap
          metadata:
              name: kas-tls-ca
# Default value to disable every part of the gitlab chart we are not interested in
# Only the appConfig to provide s3 credentials & enabling minio might be needed.
global:
    edition: ce
    hosts:
        domain: kosmos.athea
        https: false
        tls:
            enabled: false
    ingress:
        configureCertmanager: false
        tls:
            enabled: false
    minio:
        enabled: false
    gitaly:
        enabled: false
        external:
            - name: default
              hostname: gitlab-gitaly.kosmos-dev.svc.cluster.local
              tlsEnabled: false
    kas:
        enabled: false
    redis: # necessary to prevent blocking templating check on unused resources
        redisYmlOverride: "---"
    appConfig:
        lfs:
            enabled: false
        artifacts:
            enabled: false
        #  connection:
        #    secret: gitlab-s3-secret
        #    key: connection
        uploads:
            enabled: false
        #  connection:
        #    secret: gitlab-s3-secret
        #    key: connection
        packages:
            enabled: false
        #  connection:
        #    secret: gitlab-s3-secret
        #    key: connection
    extraEnv:
        GITLAB_LOG_LEVEL: "info"

upgradeCheck:
    enabled: false

certmanager:
    installCRDs: false
    install: false

nginx-ingress: &nginx-ingress
    enabled: false

prometheus:
    install: false

postgresql:
    install: false
redis:
    install: false

gitlab:
    toolbox:
        enabled: false
    gitlab-shell:
        enabled: false
    sidekiq:
        enabled: false
    webservice:
        enabled: false
    #gitaly:
    #  enabled: false
    gitlab-exporter:
        enabled: false
    migrations:
        enabled: false
    kas:
        enabled: false

registry:
    enabled: false

shared-secrets:
    enabled: false
    rbac:
        create: false
    serviceAccount:
        enabled: false
        create: false

Once the runner has started and contacted gitlab you'll see it as Online in the runners list.

Adding a Tenant agent​

Introduction​

An agent is a pod that allow runners to interact with a kubernetes Cluster with specific RBAC rules. Adding a Tenant agent allow the Tenant to have a dedicated gitlab agent for their CI/CD needs.

For the exemple we'll consider that :

  • Your Tenant is tenant0
  • Your agent is named agent0 and created in the project agent-config
  • Your namespace is tenant0-dev

Creating the RBAC rules​

The permissions that will be granted to the runner in a cluster will be defined by the kubernetes RBAC rules given to the agent ServiceAccount.

Here is an exemple of RBAC rules that gives the right to get, list, watch & delete pods in the same namespace as the agent :

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tenant0-agent-gitlab-agent
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tenant0-agent-gitlab-agent
rules:
  # minimal RBAC for an agent to properly function
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - patch
      - create
  - apiGroups:
      - "coordination.k8s.io"
    resources:
      - leases
    verbs:
      - get
      - create
      - update
  # arbitrary RBAC as exemple
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - get
      - list
      - watch
      - delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tenant0-agent-gitlab-agent
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tenant0-agent-gitlab-agent
subjects:
  - kind: ServiceAccount
    name: tenant0-agent-gitlab-agent

Note : The two rules on events & leases ressources are necessary for the agent to properly work.

Initial creation of the agent in Gitlab UI​

In order to create an agent you'll need to create a new project (eg: agent-config) in the Tenant :

  • Left panel > Group
  • Pick the Tenant you wish to configure (tenant0)
  • Click on a project OR create a new one
  • In this project create the file .gitlab/agents/agent0/config.yaml with the content as below
    • this will give access to the CI in every subgroup and project of your Tenant
    • for more fine-grained access see official documentation
ci_access:
  groups:
    - id: tenant0
      default_namespace: "tenant0-dev"
      access_as:
        agent: {}

Now that your configuration file exists, create the agent itself :

  • Left panel > Operate > Kubernetes Cluster
  • Click on Connect a cluster
  • Name the agent agent0 and click Create and register
  • Pay attention to the Agent access token field, it will NOT be shown ever again and you need it to register your agent in later steps.

Instantiate the agent in the cluster​

The agent can be deployed using helm in the cluster using the gitlab-agent helm chart.

An exemple of values is provided below with some changes needed :

  • Put your token in the config.token field

Note: the service account used is the one provided as exemple above.

---
rbac:
  create: false

# -- serviceAccount settings
serviceAccount:
  create: false
  name: tenant0-agent-gitlab-agent

# -- configure the agent
config:
  kasAddress: 'ws://gitlab-kas.kosmos-dev.svc.cluster.local:8150'
  token: PUT_YOUR_TOKEN_FROM_UI_HERE

Check your agent availability​

You can check your agent availability in any project that ahs access to it :

  • Left panel > Operate > Kubernetes Cluster
  • Click on your agent

With the default configuration provided above you should see your agent in any of tenant0 projects.

Use the agent in your pipelines​

Create a new project (eg: test-agent) and create a pipeline configuration using kubectl :

  • Left panel > Group
  • Pick the Tenant you wish to configure (tenant0)
  • Create a new project inside the Tenant
  • Left panel > Build > Pipeline editor
  • Click on Configure pipeline
  • Copy/Paste the exemple configuration below and click Commit changes

The pipeline should be successful and the pods of namespace tenant0-dev should be listed in the logs.

stages:
  - deploy

deploy-job:
  stage: deploy
  image:
      name: docker.io/bitnami/kubectl:1.32.1
      entrypoint: ['']
  script:
    - kubectl config get-contexts
    - kubectl config use-context tenant0/agent-config:agent0
    - kubectl get pods