gitlab-ci基于kubernetes+harbor

著作：行癫 <盗版必究> ------ ![image-20220514010742541](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514010742541.png) ## 一：环境准备 #### 1.kubernetes集群 集群环境检查 ```shell [root@master ~]# kubectl get node NAME STATUS ROLES AGE VERSION master Ready control-plane,master 17d v1.23.1 node-1 Ready 17d v1.23.1 node-2 Ready 17d v1.23.1 node-3 Ready 17d v1.23.1 ``` 注意：node节点的运行内存需要大于5G以上 #### 2.harbor仓库 仓库检查 image-20220513232903669 #### 3.NFS提供PVC ```shell [root@nfs-harbor ~]# exportfs -rv exporting *:/data/storage/k8s/gitlab/gitlab exporting *:/data/storage/k8s/gitlab/redis exporting *:/data/storage/k8s/gitlab/postgresql ``` ## 二：gitlab-ce部署 #### 1.创建命名空间 ```shell [root@master gitlab]# kubectl create namespace kube-ops ``` #### 2.postgresql数据库 Deployment yaml文件： ```shell [root@master gitlab]# cat gitlab-postgresql.yaml apiVersion: apps/v1 kind: Deployment metadata: name: postgresql namespace: kube-ops labels: name: postgresql spec: replicas: 1 selector: matchLabels: name: postgresql template: metadata: name: postgresql labels: name: postgresql spec: containers: - name: postgresql image: daocloud.io/library/postgres:9.0.20 env: - name: DB_USER value: gitlab - name: DB_PASS value: passw0rd - name: DB_NAME value: gitlab_production - name: DB_EXTENSION value: pg_trgm ports: - name: postgres containerPort: 5432 volumeMounts: - mountPath: /var/lib/postgresql name: data volumes: - name: data nfs: server: 10.0.0.230 path: /data/storage/k8s/gitlab/postgresql readOnly: false --- apiVersion: v1 kind: Service metadata: name: postgresql namespace: kube-ops labels: name: postgresql spec: ports: - name: postgres port: 5432 targetPort: postgres selector: name: postgresql ``` 创建Depolyment： ```shell [root@master gitlab]# kubelet create -f gitlab-postgresql.yaml ``` 查看Deployment和Pod ```shell [root@master gitlab]# kubectl get deployment -n kube-ops NAME READY UP-TO-DATE AVAILABLE AGE postgresql 1/1 1 1 97m ``` ```shell [root@master gitlab]# kubectl get pod -n kube-ops NAME READY STATUS RESTARTS AGE postgresql-cccb54fff-2gczp 1/1 Running 0 99m ``` ![image-20220513233712291](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220513233712291.png) ![image-20220513233722978](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220513233722978.png) 查看service： ```shell [root@master gitlab]# kubectl get svc -n kube-ops NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE postgresql ClusterIP 10.103.7.249 5432/TCP 100m ``` #### 3.redis部署 创建Deployment yaml文件： ```shell [root@master gitlab]# cat gitlab-redis.yaml apiVersion: apps/v1 kind: Deployment metadata: name: redis namespace: kube-ops labels: name: redis spec: replicas: 1 selector: matchLabels: name: redis template: metadata: name: redis labels: name: redis spec: containers: - name: redis image: 10.0.0.230/xingdian/redis:v1 imagePullPolicy: IfNotPresent ports: - name: redis containerPort: 6379 volumeMounts: - mountPath: /var/lib/redis name: data livenessProbe: exec: command: - redis-cli - ping initialDelaySeconds: 30 timeoutSeconds: 5 readinessProbe: exec: command: - redis-cli - ping initialDelaySeconds: 5 timeoutSeconds: 1 volumes: - name: data nfs: server: 10.0.0.230 path: /data/storage/k8s/gitlab/redis readOnly: false --- apiVersion: v1 kind: Service metadata: name: redis namespace: kube-ops labels: name: redis spec: ports: - name: redis port: 6379 targetPort: redis selector: name: redis ``` 创建Deployment： ```shell [root@master gitlab]# kubectl create -f gitlab-redis.yaml ``` 查看Deployment和Pod： ```shell [root@master gitlab]# kubectl get deployment -n kube-ops NAME READY UP-TO-DATE AVAILABLE AGE redis 1/1 1 1 104m ``` ```shell [root@master gitlab]# kubectl get pod -n kube-ops NAME READY STATUS RESTARTS AGE redis-7786bc4f96-lxhlj 1/1 Running 0 104m ``` #### 4.gitlab-ce部署 创建Deployment yaml文件： ```shell [root@master gitlab]# cat gitlab.yaml apiVersion: apps/v1 kind: Deployment metadata: name: gitlab namespace: kube-ops labels: name: gitlab spec: replicas: 1 selector: matchLabels: name: gitlab template: metadata: name: gitlab labels: name: gitlab spec: containers: - name: gitlab image: 10.0.0.230/xingdian/gitlab-ce:v1 imagePullPolicy: IfNotPresent env: - name: TZ value: Asia/Shanghai - name: GITLAB_TIMEZONE value: Beijing - name: GITLAB_SECRETS_DB_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_SECRETS_SECRET_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_SECRETS_OTP_KEY_BASE value: long-and-random-alpha-numeric-string - name: GITLAB_ROOT_PASSWORD value: xingdian123 - name: GITLAB_ROOT_EMAIL value: zhuangyaovip@163.com - name: GITLAB_HOST value: 0.0.0.0:30004 - name: GITLAB_PORT value: "80" - name: GITLAB_SSH_PORT value: "22" - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS value: "true" - name: GITLAB_NOTIFY_PUSHER value: "false" - name: GITLAB_BACKUP_SCHEDULE value: daily - name: GITLAB_BACKUP_TIME value: 01:00 - name: DB_TYPE value: postgres - name: DB_HOST value: postgresql - name: DB_PORT value: "5432" - name: DB_USER value: gitlab - name: DB_PASS value: passw0rd - name: DB_NAME value: gitlab_production - name: REDIS_HOST value: redis - name: REDIS_PORT value: "6379" ports: - name: http containerPort: 80 - name: ssh containerPort: 22 volumeMounts: - mountPath: /home/git/data name: data livenessProbe: httpGet: path: / port: 80 initialDelaySeconds: 180 timeoutSeconds: 5 readinessProbe: httpGet: path: / port: 80 initialDelaySeconds: 5 timeoutSeconds: 1 volumes: - name: data nfs: server: 10.0.0.230 path: /data/storage/k8s/gitlab/gitlab readOnly: false --- apiVersion: v1 kind: Service metadata: name: gitlab namespace: kube-ops labels: name: gitlab spec: type: NodePort ports: - name: http port: 80 targetPort: http nodePort: 30004 - name: ssh port: 22 targetPort: ssh selector: name: gitlab ``` 创建Deployment： ```shell [root@master gitlab]# kubectl create -f gitlab.yaml ``` 查看Deployment和Pod： ```shell [root@master gitlab]# kubectl get deployment -n kube-ops NAME READY UP-TO-DATE AVAILABLE AGE gitlab 1/1 1 1 108m ``` ```shell [root@master gitlab]# kubectl get pod -n kube-ops NAME READY STATUS RESTARTS AGE gitlab-696d568999-8zg75 1/1 Running 1 (105m ago) 109m ``` 注意：gitlab启动较慢，我们需要耐心等待 ![image-20220513235023237](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220513235023237.png) ![image-20220513235044997](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220513235044997.png) ## 三：gitlab-runner部署 #### 1.创建configmap 创建config-configmap.yaml文件： ```shell [root@master gitlab-runner]# cat config-configmap.yaml apiVersion: v1 data: REGISTER_NON_INTERACTIVE: "true" REGISTER_LOCKED: "false" METRICS_SERVER: "0.0.0.0:9100" CI_SERVER_URL: "http://10.0.0.220:30004/ci" # k8s内gitlab服务的通信地址格式:svc.namespace.svc.cluster.local, 同时加上/ci这个prefix，这里也可以使用外网访问地址 RUNNER_REQUEST_CONCURRENCY: "4" RUNNER_EXECUTOR: "kubernetes" KUBERNETES_NAMESPACE: "kube-ops" # 服务运行的namespace KUBERNETES_PRIVILEGED: "true" KUBERNETES_CPU_LIMIT: "1" KUBERNETES_MEMORY_LIMIT: "1Gi" KUBERNETES_SERVICE_CPU_LIMIT: "1" KUBERNETES_SERVICE_MEMORY_LIMIT: "1Gi" KUBERNETES_HELPER_CPU_LIMIT: "500m" KUBERNETES_HELPER_MEMORY_LIMIT: "100Mi" KUBERNETES_PULL_POLICY: "if-not-present" KUBERNETES_TERMINATIONGRACEPERIODSECONDS: "10" KUBERNETES_POLL_INTERVAL: "5" KUBERNETES_POLL_TIMEOUT: "360" kind: ConfigMap metadata: labels: app: gitlab-ci-runner name: gitlab-ci-runner-cm namespace: kube-ops ``` 创建scripts-configmap.yaml文件： ```shell [root@master gitlab-runner]# cat scripts-configmap.yaml apiVersion: v1 data: run.sh: | #!/bin/bash unregister() { kill %1 echo "Unregistering runner ${RUNNER_NAME} ..." /usr/bin/gitlab-ci-multi-runner unregister -t "$(/usr/bin/gitlab-ci-multi-runner list 2>&1 | tail -n1 | awk '{print $4}' | cut -d'=' -f2)" -n ${RUNNER_NAME} exit $? } trap 'unregister' EXIT HUP INT QUIT PIPE TERM echo "Registering runner ${RUNNER_NAME} ..." /usr/bin/gitlab-ci-multi-runner register -r ${GITLAB_CI_TOKEN} sed -i 's/^concurrent.*/concurrent = '"${RUNNER_REQUEST_CONCURRENCY}"'/' /home/gitlab-runner/.gitlab-runner/config.toml echo "Starting runner ${RUNNER_NAME} ..." /usr/bin/gitlab-ci-multi-runner run -n ${RUNNER_NAME} & wait kind: ConfigMap metadata: labels: app: gitlab-ci-runner name: gitlab-ci-runner-scripts namespace: kube-ops ``` 创建configmap： ```shell [root@master gitlab-runner]# kubectl create -f scripts-configmap.yaml [root@master gitlab-runner]# kubectl create -f config-configmap.yaml ``` 查看： ```shell [root@master gitlab-runner]# kubectl get configmap -n kube-ops NAME DATA AGE gitlab-ci-runner-cm 18 22m gitlab-ci-runner-scripts 1 21m ``` #### 2.创建secret ​ 需要创建一个Secret用来存储gitlab的token 创建secret.yaml文件： ```shell [root@master gitlab-runner]# cat secret.yaml apiVersion: v1 kind: Secret metadata: name: gitlab-ci-token namespace: kube-ops labels: app: gitlab-ci-runner data: GITLAB_CI_TOKEN: R1IxMzQ4OTQxVVZVWS1yUjh4YXl6aE1zaVp3eDU= # 这是base64加密Gitlab runner token之后的字符串 ``` 生产tokne： ```shell [root@master gitlab-runner]# echo -n 'GR1348941UVUY-rR8xayzhMsiZwx5' | openssl base64 R1IxMzQ4OTQxVVZVWS1yUjh4YXl6aE1zaVp3eDU= ``` 创建secret： ```shell [root@master gitlab-runner]# kubectl create -f secret.yaml ``` 查看： ```shell [root@master gitlab-runner]# kubectl get secret -n kube-ops NAME TYPE DATA AGE default-token-5xql2 kubernetes.io/service-account-token 3 24h gitlab-ci-token Opaque 1 18m gitlab-ci-token-dwvd5 kubernetes.io/service-account-token 3 10m ``` #### 3.创建ServiceAccount ​ 创建一个`ServiceAccount`来有足够的权限做一些事情，因此我们创建一个`gitlab-ci`的`ServiceAccount` 创建ServiceAccount、Role、RoleBinding文件： ```shell [root@master gitlab-runner]# cat rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: gitlab-ci namespace: kube-ops --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: gitlab-ci namespace: kube-ops rules: - apiGroups: [""] resources: ["*"] verbs: ["*"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: gitlab-ci namespace: kube-ops subjects: - kind: ServiceAccount name: gitlab-ci namespace: kube-ops roleRef: kind: Role name: gitlab-ci apiGroup: rbac.authorization.k8s.io ``` 创建： ```shell [root@master gitlab-runner]# kubectl create -f rbac.yaml ``` 查看： ```shell [root@master gitlab-runner]# kubectl get serviceaccount -n kube-ops NAME SECRETS AGE default 1 24h gitlab-ci 1 13m ``` ```shell [root@master gitlab-runner]# kubectl get role -n kube-ops NAME CREATED AT gitlab-ci 2022-05-13T16:42:08Z ``` ```shell [root@master gitlab-runner]# kubectl get rolebinding -n kube-ops NAME ROLE AGE gitlab-ci Role/gitlab-ci 14m ``` #### 4.创建Statefulset 创建statefulset yaml文件： ```shell [root@master gitlab-runner]# cat statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: gitlab-ci-runner namespace: kube-ops labels: app: gitlab-ci-runner spec: selector: matchLabels: app: gitlab-ci-runner updateStrategy: type: RollingUpdate replicas: 1 serviceName: gitlab-ci-runner template: metadata: labels: app: gitlab-ci-runner spec: volumes: - name: gitlab-ci-runner-scripts projected: sources: - configMap: name: gitlab-ci-runner-scripts items: - key: run.sh path: run.sh mode: 0755 serviceAccountName: gitlab-ci securityContext: runAsNonRoot: true runAsUser: 999 supplementalGroups: [999] containers: - image: 10.0.0.230/xingdian/gitlab/gitlab-runner:latest name: gitlab-ci-runner command: - /scripts/run.sh envFrom: - configMapRef: name: gitlab-ci-runner-cm - secretRef: name: gitlab-ci-token env: - name: RUNNER_NAME valueFrom: fieldRef: fieldPath: metadata.name ports: - containerPort: 9100 name: http-metrics protocol: TCP volumeMounts: - name: gitlab-ci-runner-scripts mountPath: "/scripts" readOnly: true restartPolicy: Always ``` 创建： ```shell [root@master gitlab-runner]# kubectl create -f statefulset.yaml ``` 查看： ```shell [root@master gitlab-runner]# kubectl get statefulset -n kube-ops NAME READY AGE gitlab-ci-runner 1/1 19m ``` ```shell [root@master gitlab-runner]# kubectl get pod -n kube-ops NAME READY STATUS RESTARTS AGE gitlab-696d568999-8zg75 1/1 Running 1 3h2m gitlab-ci-runner-0 1/1 Running 0 18m postgresql-cccb54fff-2gczp 1/1 Running 0 3h3m redis-7786bc4f96-lxhlj 1/1 Running 0 3h2m ``` #### 5.gitlab查看注册Runner服务 ![image-20220514010639780](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514010639780.png) ## 四：gitlab-ce使用 #### 1.配置项目启动邮件 Menu---->your project--->admin(其中一个项目) ![image-20220514140944925](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514140944925.png) Configure intergrations---> Add an intergration---> Email on push ![image-20220514141021752](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141021752.png) 添加邮箱（可以先测试连接，后添加） ![image-20220514141234577](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141234577.png) #### 2.关联harbor仓库 仓库地址：http://10.0.0.230 这是一个本地地址，在kubernetes集群外 Menu---->Admin ![image-20220514141414315](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141414315.png) Settings---->Network ![image-20220514141453504](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141453504.png) 找到Outbound requests ![image-20220514141557089](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141557089.png) 选择允许本地网络 ![image-20220514141636331](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141636331.png) 注意： ​ 以上操作就是解决这个问题：Import url is blocked: Requests to the local network are not allowed Menu---->admin(项目)---->configure intergrations ![image-20220514141749612](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141749612.png) Add an integration---> Harbor ![image-20220514141844310](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514141844310.png) Harbor相关配置（仓库地址，仓库名称，用户名，密码） ![image-20220514142007413](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514142007413.png) #### 3.CI/CD pipelines ​ pipelines 管道，管道是持续集成、交付和部署的顶级组件，作业由runners执行。如果有足够的并发运行者，同一阶段的多个作业将并行执行；如果一个阶段中的*所有*作业都成功，则管道将进入下一个阶段；如果一个阶段中的任何作业失败，则（通常）不会执行下一个阶段，并且管道会提前结束；一般来说，管道是自动执行的，一旦创建就不需要干预；但是，有时您也可以手动与管道交互 管道包括： ​ 工作：它定义*了*要做什么。例如，编译或测试代码的作业 ​ 阶段：定义何时运行作业。例如，在编译代码的阶段之后运行测试的阶段 四个阶段： ​ 一个`build`舞台，有一个工作叫做`compile` ​ 一个`test`阶段，有两个作业称为`test1`和`test2` ​ 一个`staging`舞台，有一个工作叫做`deploy-to-stage` ​ 一个`production`舞台，有一个工作叫做`deploy-to-prod` #### 4.`.gitlab-ci.yaml`文件： 配置管道行为的全局关键字： | 关键词 | 描述 | | :----------------------------------------------------------- | :---------------------------------- | | [`default`](https://docs.gitlab.com/ee/ci/yaml/index.html#default) | 工作关键字的自定义默认值。 | | [`include`](https://docs.gitlab.com/ee/ci/yaml/index.html#include) | 从其他 YAML 文件导入配置。 | | [`stages`](https://docs.gitlab.com/ee/ci/yaml/index.html#stages) | 流水线阶段的名称和顺序。 | | [`variables`](https://docs.gitlab.com/ee/ci/yaml/index.html#variables) | 为管道中的所有作业定义 CI/CD 变量。 | | [`workflow`](https://docs.gitlab.com/ee/ci/yaml/index.html#workflow) | 控制运行什么类型的管道。 | 使用作业关键字配置的作业： | 关键词 | 描述 | | :----------------------------------------------------------- | :----------------------------------------------------------- | | [`after_script`](https://docs.gitlab.com/ee/ci/yaml/index.html#after_script) | 覆盖作业后执行的一组命令。 | | [`allow_failure`](https://docs.gitlab.com/ee/ci/yaml/index.html#allow_failure) | 允许作业失败。失败的作业不会导致管道失败。 | | [`artifacts`](https://docs.gitlab.com/ee/ci/yaml/index.html#artifacts) | 成功后附加到作业的文件和目录列表。 | | [`before_script`](https://docs.gitlab.com/ee/ci/yaml/index.html#before_script) | 覆盖在作业之前执行的一组命令。 | | [`cache`](https://docs.gitlab.com/ee/ci/yaml/index.html#cache) | 应在后续运行之间缓存的文件列表。 | | [`coverage`](https://docs.gitlab.com/ee/ci/yaml/index.html#coverage) | 给定作业的代码覆盖率设置。 | | [`dast_configuration`](https://docs.gitlab.com/ee/ci/yaml/index.html#dast_configuration) | 在作业级别使用 DAST 配置文件中的配置。 | | [`dependencies`](https://docs.gitlab.com/ee/ci/yaml/index.html#dependencies) | 通过提供要从中获取工件的作业列表来限制将哪些工件传递给特定作业。 | | [`environment`](https://docs.gitlab.com/ee/ci/yaml/index.html#environment) | 作业部署到的环境的名称。 | | [`except`](https://docs.gitlab.com/ee/ci/yaml/index.html#only--except) | 控制何时不创建作业。 | | [`extends`](https://docs.gitlab.com/ee/ci/yaml/index.html#extends) | 此作业继承的配置条目。 | | [`image`](https://docs.gitlab.com/ee/ci/yaml/index.html#image) | 使用 Docker 镜像。 | | [`inherit`](https://docs.gitlab.com/ee/ci/yaml/index.html#inherit) | 选择所有作业继承的全局默认值。 | | [`interruptible`](https://docs.gitlab.com/ee/ci/yaml/index.html#interruptible) | 定义一个作业是否可以在被较新的运行冗余时取消。 | | [`needs`](https://docs.gitlab.com/ee/ci/yaml/index.html#needs) | 在阶段排序之前执行作业。 | | [`only`](https://docs.gitlab.com/ee/ci/yaml/index.html#only--except) | 控制何时创建工作。 | | [`pages`](https://docs.gitlab.com/ee/ci/yaml/index.html#pages) | 上传作业结果以与 GitLab 页面一起使用。 | | [`parallel`](https://docs.gitlab.com/ee/ci/yaml/index.html#parallel) | 应并行运行多少个作业实例。 | | [`release`](https://docs.gitlab.com/ee/ci/yaml/index.html#release) | 指示运行器生成[释放](https://docs.gitlab.com/ee/user/project/releases/index.html)对象。 | | [`resource_group`](https://docs.gitlab.com/ee/ci/yaml/index.html#resource_group) | 限制作业并发。 | | [`retry`](https://docs.gitlab.com/ee/ci/yaml/index.html#retry) | 发生故障时可以自动重试作业的时间和次数。 | | [`rules`](https://docs.gitlab.com/ee/ci/yaml/index.html#rules) | 用于评估和确定作业的选定属性以及是否已创建的条件列表。 | | [`script`](https://docs.gitlab.com/ee/ci/yaml/index.html#script) | 由运行程序执行的 Shell 脚本。 | | [`secrets`](https://docs.gitlab.com/ee/ci/yaml/index.html#secrets) | CI/CD 是工作需要的秘密。 | | [`services`](https://docs.gitlab.com/ee/ci/yaml/index.html#services) | 使用 Docker 服务镜像。 | | [`stage`](https://docs.gitlab.com/ee/ci/yaml/index.html#stage) | 定义作业阶段。 | | [`tags`](https://docs.gitlab.com/ee/ci/yaml/index.html#tags) | 用于选择跑步者的标签列表。 | | [`timeout`](https://docs.gitlab.com/ee/ci/yaml/index.html#timeout) | 定义优先于项目范围设置的自定义作业级超时。 | | [`trigger`](https://docs.gitlab.com/ee/ci/yaml/index.html#trigger) | 定义下游管道触发器。 | | [`variables`](https://docs.gitlab.com/ee/ci/yaml/index.html#variables) | 在工作级别定义工作变量。 | | [`when`](https://docs.gitlab.com/ee/ci/yaml/index.html#when) | 何时运行作业。 | #### 5.案例 示例.gitlab-ci.yaml： ```shell default: image: ruby:3.0 rspec: script: bundle exec rspec rspec 2.7: image: ruby:2.7 script: bundle exec rspec ``` 构建管道： ![image-20220514143753535](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514143753535.png) ![image-20220514143808724](%E5%88%A9%E7%94%A8kubernetes%E9%83%A8%E7%BD%B2gitlab-ce.assets/image-20220514143808724.png)