注意：在集群内使用 Gitlab Runner 可能会导致内部网络暴露，请谨慎使用。
|
1. 说明
Gitlab Runner 需要如下镜像：
1
2
3
registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v15.5.0
registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-0d4137b8
docker.io/library/docker:20-dind
2. 导入 RBAC 规则
由于 Gitlab Runner 将使用 kubernetes 运行容器，将单独为其创建命名空间 RBAC 规则。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
apiVersion: v1
kind: ServiceAccount
metadata:
name: gitlab-ci
namespace: dev-ops
labels:
app: gitlab-ci
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: gitlab-ci
namespace: dev-ops
labels:
app: gitlab-ci
rules:
- apiGroups: [ "" ]
resources: [ "*" ]
verbs: [ "*" ]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: gitlab-ci
namespace: dev-ops
labels:
app: gitlab-ci
subjects:
- kind: ServiceAccount
name: gitlab-ci
namespace: dev-ops
roleRef:
kind: Role
name: gitlab-ci
apiGroup: rbac.authorization.k8s.io
3. 创建Gitlab配置
前往 https://gitlab.d7z.net/admin/runners,点击右上角，复制注册令牌。
Figure 1. 复制注册令牌
然后修改如下配置文件，将指示位置的令牌修改为你复制的令牌：
1
2
3
4
5
6
7
8
9
10
apiVersion: v1
kind: Secret
metadata:
name: secret-gitlab-ci-token
namespace: dev-ops
labels:
app: gitlab-ci
stringData:
# 请填写 Gitlab CI 的 Token
GITLAB_CI_TOKEN: '<请填写 Gitlab CI 的 Token>' (1)
其中
| 1 | 修改为你的令牌内容 |
4. 导入默认内部配置
Gitlab CI 需要指定脚本配置部分配置才可正常运行，此处脚本参考至 Gitlab CI/CD 实践一：Gitlab Runner 部署到 K8S 集群。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
apiVersion: v1
data:
default.toml: |
check_interval = 0
[session_server]
listen_address = "[::]:8093"
advertise_address = "{{pod.address}}:8093"
session_timeout = 1800
run.sh: |
#!/usr/bin/env bash
export CONF_PATH="/etc/gitlab-runner"
unregister() {
kill %1
echo "Unregistering runner ${RUNNER_NAME} ..."
GITLAB_TOKEN="$(/usr/bin/gitlab-ci-multi-runner list 2>&1 | tail -n1 | awk '{print $4}' | cut -d'=' -f2)"
for i in {1..5}; do
/usr/bin/gitlab-ci-multi-runner unregister -t "${GITLAB_TOKEN}" -n "${RUNNER_NAME}" && break
if [ "$i" -gt 4 ]; then
exit 1
fi
done
}
trap 'unregister' EXIT HUP INT QUIT PIPE TERM
cp -f /scripts/default.toml "$CONF_PATH"/config.toml
CURRENT_IP=$(/sbin/ip -o -4 addr list eth0 | awk '{print $4}' | cut -d/ -f1 | sed 's/\./-/g')
sed -i -e "s/{{pod.address}}/$CURRENT_IP.$KUBERNETES_NAMESPACE.pod.cluster.local/g" "$CONF_PATH"/config.toml
# add gitlab custom cert
if [ -f /certs/tls.crt ]; then
export CI_SERVER_TLS_CERT_FILE=/certs/tls.crt
fi
if [ -f /certs/ca.crt ]; then
export CI_SERVER_TLS_CA_FILE=/certs/ca.crt
fi
echo "Registering runner ${RUNNER_NAME} ..."
REGISTER_ARGS=(--env GIT_SSL_NO_VERIFY=true --env DOCKER_HOST=$DOCKER_HOST)
for i in {1..5}; do
/usr/bin/gitlab-ci-multi-runner register -r "${GITLAB_CI_TOKEN}" ${REGISTER_ARGS[*]} && break
if [ "$i" -gt 4 ]; then
echo "gitlab ci register fail ." 1>&2
exit 1
fi
sleep 5
done
sed -i "$CONF_PATH"/config.toml \
-e 's/^concurrent.*/concurrent = '"${RUNNER_REQUEST_CONCURRENCY}"'/'
echo "Starting runner ${RUNNER_NAME} ..."
/usr/bin/gitlab-ci-multi-runner run -n "${RUNNER_NAME}" &
wait
kind: ConfigMap
metadata:
labels:
app: gitlab-ci
name: conf-gitlab-ci-scripts
namespace: dev-ops
---
apiVersion: v1
data:
REGISTER_NON_INTERACTIVE: "true"
REGISTER_LOCKED: "false"
METRICS_SERVER: "0.0.0.0:9100"
CI_SERVER_URL: "http://svc-gitlab.core-app.svc.cluster.local/ci"
DOCKER_HOST: "tcp://svc-docker-dind.dev-ops.svc.cluster.local:2375"
RUNNER_REQUEST_CONCURRENCY: "4"
RUNNER_EXECUTOR: "kubernetes"
KUBERNETES_NAMESPACE: "dev-ops"
KUBERNETES_CPU_LIMIT: ""
KUBERNETES_CPU_LIMIT_OVERWRITE_MAX_ALLOWED: ""
KUBERNETES_MEMORY_LIMIT: ""
KUBERNETES_MEMORY_LIMIT_OVERWRITE_MAX_ALLOWED: ""
KUBERNETES_CPU_REQUEST: ""
KUBERNETES_CPU_REQUEST_OVERWRITE_MAX_ALLOWED: ""
KUBERNETES_MEMORY_REQUEST: ""
KUBERNETES_MEMORY_REQUEST_OVERWRITE_MAX_ALLOWED: ""
KUBERNETES_SERVICE_CPU_LIMIT: ""
KUBERNETES_SERVICE_MEMORY_LIMIT: ""
KUBERNETES_HELPER_CPU_LIMIT: ""
KUBERNETES_HELPER_MEMORY_LIMIT: ""
KUBERNETES_PULL_POLICY: "always"
KUBERNETES_TERMINATIONGRACEPERIODSECONDS: "10"
KUBERNETES_POLL_INTERVAL: "5"
KUBERNETES_POLL_TIMEOUT: "360"
RUNNER_NAME: "runner-kubernetes"
kind: ConfigMap
metadata:
labels:
app: gitlab-ci
name: conf-gitlab-ci
namespace: dev-ops
5. 导入 Docker dind 配置
| Docker dind 以特权模式运行容器, 可能会导致严重的安全问题。建议配置节点亲和性使其运行在专门的节点中，或者将其部署到集群外部。 |
复制并导入以下配置，启动 Docker dind。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: sts-docker-dind
namespace: dev-ops
labels:
app: docker-dind
spec:
replicas: 1
template:
metadata:
labels:
app: docker-dind
spec:
containers:
- name: "docker-dind"
image: "docker.io/library/docker:20-dind"
command: [ "dockerd", "--host", "tcp://0.0.0.0:2375" ,"--insecure-registry","registry.d7z.net","--registry-mirror","https://registry.d7z.net/" ]
securityContext:
privileged: true
volumeMounts:
- mountPath: '/etc/docker/certs.d/registry.d7z.net/'
name: cluster-certs
volumes:
- name: cluster-certs
secret:
secretName: 'tls-pub-d7z'
selector:
matchLabels:
app: docker-dind
serviceName: svc-docker-dind
---
apiVersion: v1
kind: Service
metadata:
labels:
app: docker-dind
name: "svc-docker-dind"
namespace: dev-ops
spec:
ports:
- name: docker-tcp
protocol: TCP
port: 2375
selector:
app: docker-dind
6. 导入运行配置
复制并导入以下配置，启动 Gitlab Runner。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: gitlab-ci-runner
namespace: dev-ops
labels:
app: gitlab-ci
spec:
selector:
matchLabels:
app: gitlab-ci
updateStrategy:
type: RollingUpdate
replicas: 2
serviceName: gitlab-ci-runner
template:
metadata:
labels:
app: gitlab-ci
spec:
volumes:
- name: gitlab-ci-runner-scripts
projected:
sources:
- configMap:
name: conf-gitlab-ci-scripts
items:
- key: run.sh
path: run.sh
mode: 0755
- key: default.toml
path: default.toml
mode: 0755
- name: gitlab-ci-runner-certs
projected:
sources:
- secret:
name: 'tls-pub-d7z'
items:
- key: 'tls.crt'
path: 'tls.crt'
mode: 0777
serviceAccountName: gitlab-ci
containers:
- image: 'registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v15.5.0'
name: gitlab-ci-runner
command:
- /scripts/run.sh
envFrom:
- configMapRef:
name: conf-gitlab-ci
- secretRef:
name: secret-gitlab-ci-token
ports:
- containerPort: 9100
name: http-metrics
protocol: TCP
volumeMounts:
- name: gitlab-ci-runner-scripts
mountPath: "/scripts"
readOnly: true
- name: gitlab-ci-runner-certs
mountPath: "/certs"
readOnly: true
restartPolicy: Always