Day10-作业讲解，Ingress规则编写及Jenkins集成K8S项目实战案例
Day10-作业讲解，Ingress规则编写及Jenkins集成K8S项目实战案例
- 1、作业:
- 1.1 作业分析:
- 1.2 复刻镜像
- 1.3 将游戏镜像上K8S集群
- 1.4 helm部署服务
- 1.5 ingress规则编写
- 2、项目1-jenkins和k8s集成
- 2.1 Jenkins实现k8s持续集成项目流程图解
- 2.2 Jenkins集成K8S-将代码推送到gitee
- 2.3 Jenkins集成K8S-部署Jenkins
- 2.4 Jenkins集成K8S-编写dockerfile
- 2.5 Jenkins集成K8S-部署应用到K8S
1、作业:
将"jasonyin2020/oldboyedu-games:v0.1"游戏镜像使用helm部署，请自行设计。
1.1 作业分析:
-
使用的技术栈:
- docker
- dockerfile
- docker build
- docker cp
- nginx
- harbor
- kubernetes
- configMap
- secret
- deployments
- service
- pv
- pvc
- sc
- helm
- docker
-
镜像的名称设计:
harbor.oldboyedu.com/games/tanke:v1
harbor.oldboyedu.com/games/bird:v1
harbor.oldboyedu.com/games/pingshu:v1
harbor.oldboyedu.com/games/pintai:v1
harbor.oldboyedu.com/games/chengbao:v1
1.2 复刻镜像
(1)运行镜像
[root@k8s231.oldboyedu.com ~]# docker run -d --name linux85-games jasonyin2020/oldboyedu-games:v0.1
(2)创建工作目录
[root@k8s231.oldboyedu.com homework]# mkdir scripts softwares config dockerfile
(3)拷贝源代码
[root@k8s231.oldboyedu.com homework]# docker cp linux85-games:/usr/local/nginx/html/ softwares/
(4)将代码打包
[root@k8s231.oldboyedu.com homework]# tar zcf softwares/oldboyedu-bird.tar.gz -C softwares/html bird
[root@k8s231.oldboyedu.com homework]# tar zcf softwares/oldboyedu-chengbao.tar.gz -C softwares/html chengbao
[root@k8s231.oldboyedu.com homework]# tar zcf softwares/oldboyedu-pingtai.tar.gz -C softwares/html pingtai
[root@k8s231.oldboyedu.com homework]# tar zcf softwares/oldboyedu-pinshu.tar.gz -C softwares/html pinshu
[root@k8s231.oldboyedu.com homework]# tar zcf softwares/oldboyedu-tanke.tar.gz -C softwares/html tanke
[root@k8s231.oldboyedu.com homework]# rm -rf softwares/html/
(5)制作配置文件
[root@k8s231 homework]# docker cp linux85-games:/usr/local/nginx/conf/nginx.conf config/
[root@k8s231 homework]# ll config/
总用量 4
-rw-r--r-- 1 root root 951 2022-02-22 19:51 nginx.conf
[root@k8s231 homework]# cp config/nginx.conf config/bird.conf
[root@k8s231 homework]# cp config/nginx.conf config/chengbao.conf
[root@k8s231 homework]# cp config/nginx.conf config/pingtai.conf
[root@k8s231 homework]# cp config/nginx.conf config/pinshu.conf
[root@k8s231 homework]# cp config/nginx.conf config/tanke.conf入门
[root@k8s231.oldboyedu.com homework]# cat config/bird.conf
worker_processes 1;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;# include /usr/local/nginx/conf/conf.d/*.conf;server {listen 80;root /usr/local/nginx/html/bird/;server_name brid.oldboyedu.com;}
}
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat config/chengbao.conf
worker_processes 1;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;# include /usr/local/nginx/conf/conf.d/*.conf;server {listen 80;root /usr/local/nginx/html/chengbao/;server_name chengbao.oldboyedu.com;}}
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat config/pingtai.conf
worker_processes 1;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;# include /usr/local/nginx/conf/conf.d/*.conf;server {listen 80;root /usr/local/nginx/html/pingtai/;server_name pingtai.oldboyedu.com;}
}
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat config/pinshu.conf
worker_processes 1;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;# include /usr/local/nginx/conf/conf.d/*.conf;server {listen 80;root /usr/local/nginx/html/pinshu/;server_name pinshu.oldboyedu.com;}}
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat config/tanke.conf
worker_processes 1;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;# include /usr/local/nginx/conf/conf.d/*.conf;server {listen 80;root /usr/local/nginx/html/tanke/;server_name tanke.oldboyedu.com;}
}
[root@k8s231 homework]# rm -rf config/nginx.conf
(6)编写dockerfile
[root@k8s231.oldboyedu.com homework]# cat dockerfile/bird.yaml
FROM harbor.oldboyedu.com/linux/alpine:latestLABEL school=oldboyedu \class=linux85 \auther=JasonYinEXPOSE 80ARG appsRUN sed -i 's#dl-cdn.alpinelinux.org#mirrors.aliyun.com#' /etc/apk/repositories && \apk update && apk add nginx && rm -rf /var/cacheADD softwares/oldboyedu-${apps}.tar.gz /usr/local/nginx/htmlCOPY config/${apps}.conf /etc/nginx/nginx.confCMD ["nginx","-g","daemon off;"]
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat dockerfile/chengbao.yaml
FROM harbor.oldboyedu.com/linux/alpine:latestLABEL school=oldboyedu \class=linux85 \auther=JasonYinEXPOSE 80ARG appsRUN sed -i 's#dl-cdn.alpinelinux.org#mirrors.aliyun.com#' /etc/apk/repositories && \apk update && apk add nginx && rm -rf /var/cacheADD softwares/oldboyedu-${apps}.tar.gz /usr/local/nginx/htmlCOPY config/${apps}.conf /etc/nginx/nginx.confCMD ["nginx","-g","daemon off;"]
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat dockerfile/pingtai.yaml
FROM harbor.oldboyedu.com/linux/alpine:latestLABEL school=oldboyedu \class=linux85 \auther=JasonYinEXPOSE 80ARG appsRUN sed -i 's#dl-cdn.alpinelinux.org#mirrors.aliyun.com#' /etc/apk/repositories && \apk update && apk add nginx && rm -rf /var/cacheADD softwares/oldboyedu-${apps}.tar.gz /usr/local/nginx/htmlCOPY config/${apps}.conf /etc/nginx/nginx.confCMD ["nginx","-g","daemon off;"]
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat dockerfile/pinshu.yaml
FROM harbor.oldboyedu.com/linux/alpine:latestLABEL school=oldboyedu \class=linux85 \auther=JasonYinEXPOSE 80ARG appsRUN sed -i 's#dl-cdn.alpinelinux.org#mirrors.aliyun.com#' /etc/apk/repositories && \apk update && apk add nginx && rm -rf /var/cacheADD softwares/oldboyedu-${apps}.tar.gz /usr/local/nginx/htmlCOPY config/${apps}.conf /etc/nginx/nginx.confCMD ["nginx","-g","daemon off;"]
[root@k8s231.oldboyedu.com homework]#
[root@k8s231.oldboyedu.com homework]# cat dockerfile/tanke.yaml
FROM harbor.oldboyedu.com/linux/alpine:latestLABEL school=oldboyedu \class=linux85 \auther=JasonYinEXPOSE 80ARG appsRUN sed -i 's#dl-cdn.alpinelinux.org#mirrors.aliyun.com#' /etc/apk/repositories && \apk update && apk add nginx && rm -rf /var/cacheADD softwares/oldboyedu-${apps}.tar.gz /usr/local/nginx/htmlCOPY config/${apps}.conf /etc/nginx/nginx.confCMD ["nginx","-g","daemon off;"]
[root@k8s231.oldboyedu.com homework]#
(7)编写编译脚本并推送镜像到harbor仓库
[root@k8s231.oldboyedu.com homework]# cat scripts/build.sh
#!/bin/bash
# school: www.oldboyedu.com
# class: linux85
# auther: JasonYin# docker build --build-arg apps=bird -f dockerfile/bird.yaml -t harbor.oldboyedu.com/games/bird:v1 /root/dockerfile/homework
# docker build --build-arg apps=chengbao -f dockerfile/chengbao.yaml -t harbor.oldboyedu.com/games/chengbao:v1 /root/dockerfile/homework
# docker build --build-arg apps=pingtai -f dockerfile/pingtai.yaml -t harbor.oldboyedu.com/games/pingtai:v1 /root/dockerfile/homework
# docker build --build-arg apps=pinshu -f dockerfile/pinshu.yaml -t harbor.oldboyedu.com/games/pinshu:v1 /root/dockerfile/homework
# docker build --build-arg apps=tanke -f dockerfile/tanke.yaml -t harbor.oldboyedu.com/games/tanke:v1 /root/dockerfile/homeworkAPPS=("bird" "chengbao" "pingtai" "pinshu" "tanke")
WORKDIR=/root/dockerfile/homework
PROJECT=harbor.oldboyedu.com/games
VERSION=v1
USERNAME=admin
PASSWORD=1
SERVER=harbor.oldboyedu.comdocker login -u $USERNAME -p $PASSWORD $SERVERfor app in ${APPS[@]}
dodocker build --build-arg apps=${app} -f dockerfile/${app}.yaml -t ${PROJECT}/${app}:${VERSION} ${WORKDIR}docker push harbor.oldboyedu.com/games/${app}:v1
donedocker logout $SERVER[root@k8s231 homework]# chmod +x scripts/build.sh
# 构建镜像并推送
[root@k8s231 homework]# scripts/build.sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded
.....
The push refers to repository [harbor.oldboyedu.com/games/tanke]
08485c0bfabc: Pushed
2e4e3fc89d11: Pushed
68ff0f3db9ba: Pushed
8d3ac3489996: Mounted from games/pinshu
v1: digest: sha256:13e08cb8f1c075f271366119687724c6f6c9e86c4a96b16ae36e41cf044470b7 size: 1156
Removing login credentials for harbor.oldboyedu.com
1.3 将游戏镜像上K8S集群
(1)创建工作目录
[root@k8s231 homework]# mkdir bird chengbao pingtai pinshu tanke
(2)创建secret秘钥（harbor.oldboyedu.com创建私有仓库）
[root@k8s231 bird]# kubectl create secret docker-registry harbor-games --docker-username=linux85 --docker-password=Linux85@2023 --docker-email=linux85@oldboyedu.com --docker-server=harbor.oldboyedu.com
(3)准备环境
[root@k8s231 bird]# mkdir /oldboyedu/data/kubernetes/homework
[root@k8s231 homework]# ll
总用量 0
drwxr-xr-x 2 root root 101 2024-07-16 09:49 config
drwxr-xr-x 2 root root 101 2024-07-16 09:53 dockerfile
drwxr-xr-x 2 root root 22 2024-07-16 09:54 scripts
drwxr-xr-x 2 root root 161 2024-07-16 09:36 softwares
[root@k8s231 homework]# tar xf softwares/oldboyedu-bird.tar.gz -C /oldboyedu/data/kubernetes/homework/
[root@k8s231 homework]# tar xf softwares/oldboyedu-chengbao.tar.gz -C /oldboyedu/data/kubernetes/homework/
[root@k8s231 homework]# tar xf softwares/oldboyedu-pingtai.tar.gz -C /oldboyedu/data/kubernetes/homework/
[root@k8s231 homework]# tar xf softwares/oldboyedu-pinshu.tar.gz -C /oldboyedu/data/kubernetes/homework/
[root@k8s231 homework]# tar xf softwares/oldboyedu-tanke.tar.gz -C /oldboyedu/data/kubernetes/homework/
(3)编写资源清单
[root@k8s231 bird]# pwd
/manifests/homework/bird
[root@k8s231 bird]# cat deployments.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: oldboyedu-linux85-deploy-bird
spec:replicas: 3selector:matchExpressions:- key: appsoperator: Existstemplate:metadata:labels:apps: birdspec:volumes:- name: nginxconfigMap:name: bird-confitems:- key: nginx.confpath: nginx.conf- name: code# nfs:# server: 10.0.0.231# path: /oldboyedu/data/kubernetes/homework/birdpersistentVolumeClaim:claimName: oldboyedu-games-birdimagePullSecrets:- name: harbor-gamescontainers:- name: birdimage: harbor.oldboyedu.com/games/bird:v1volumeMounts:- name: nginxmountPath: /etc/nginx/nginx.confsubPath: nginx.conf- name: codemountPath: /usr/share/nginx/html/bird/lifecycle:postStart:exec:# 此处为了简单起见，直接从镜像中拷贝数据，生产环境中可以考虑去镜像仓库拉取镜像# 为了不污染业务镜像，对于复杂的操作，可以考虑使用初始化容器拉取代码，这样就不会对业务容器造成侵入性！command: - "/bin/sh"- "-c"- "cp -r /usr/local/nginx/html/bird/ /usr/share/nginx/html"[root@k8s231 bird]# cat configMap.yaml
apiVersion: v1
kind: ConfigMap
metadata:name: bird-conf
data:nginx.conf: |worker_processes 1;events {worker_connections 1024;}http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;server {listen 80;# root /usr/local/nginx/html/bird/;root /usr/share/nginx/html/bird/;server_name bird.oldboyedu.com;}}[root@k8s231 bird]# cat services.yaml
apiVersion: v1
kind: Service
metadata:name: oldboyedu-linux85-bird
spec:selector:apps: birdtype: NodePortports:- port: 80targetPort: 80nodePort: 30080[root@k8s231 bird]# cat persistentvolumeclaims.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: oldboyedu-games-bird
spec:storageClassName: managed-nfs-storageaccessModes:- ReadWriteManyresources:limits:storage: 4Girequests:storage: 3Gi[root@k8s231 bird]# mkdir /oldboyedu/data/kubernetes/homework
[root@k8s231 bird]# kubectl get po
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-c494888bb-xx8gl 1/1 Running 2 (4h39m ago) 3d23h
traefik-5f98b98777-jh29z 1/1 Running 1 (4h39m ago) 23h
[root@k8s231 bird]# kubectl apply -f .
configmap/bird-conf created
deployment.apps/oldboyedu-linux85-deploy-bird created
persistentvolumeclaim/oldboyedu-games-bird created
service/oldboyedu-linux85-bird created
[root@k8s231 bird]# kubectl get po
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-c494888bb-xx8gl 1/1 Running 2 (4h39m ago) 3d23h
oldboyedu-linux85-deploy-bird-6d796cdb98-47gtz 1/1 Running 0 4s
oldboyedu-linux85-deploy-bird-6d796cdb98-6lswj 1/1 Running 0 4s
oldboyedu-linux85-deploy-bird-6d796cdb98-vtnnr 1/1 Running 0 4s
traefik-5f98b98777-jh29z 1/1 Running 1 (4h39m ago) 23h
[root@k8s231 bird]# kubectl exec -it oldboyedu-linux85-deploy-bird-6d796cdb98-47gtz -- sh
/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # ls
bird
/usr/share/nginx/html # l bird/
sh: l: not found
/usr/share/nginx/html # ls bird/
2000.png 21.js icon.png img index.html sound1.mp3
/usr/share/nginx/html # ls /usr/local/nginx/html/bird/
2000.png 21.js icon.png img index.html sound1.mp3
/usr/share/nginx/html #
[root@k8s231 bird]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
managed-nfs-storage oldboyedu/linux85 Delete Immediate false 4d20h
[root@k8s231 bird]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.200.0.1 <none> 443/TCP 3d23h
linux-headless-volume ClusterIP None <none> 80/TCP 3d23h
oldboyedu-linux-sts-svc ClusterIP 10.200.250.235 <none> 80/TCP 3d23h
oldboyedu-linux85-bird NodePort 10.200.106.134 <none> 80:30080/TCP 6m32s
traefik NodePort 10.200.216.33 <none> 80:20087/TCP,443:43993/TCP 23h
1.4 helm部署服务
[root@k8s231 homework]# helm create bird-chart
Creating bird-chart
[root@k8s231 homework]# rm -rf bird-chart/templates/*
[root@k8s231 homework]# cat bird-chart/templates/NOTES.txt
###############################
# 老男孩教育官网： #
# www.oldboyedu.com ##############################恭喜您，"{{ .Values.apps.image }}:{{ .Values.apps.tag }}"服务部署成功。
[root@k8s231 homework]# vim bird-chart/values.yaml
replicas: 2apps:image: harbor.oldboyedu.com/games/birdtag: v1
[root@k8s231 homework]# cp bird/* bird-chart/templates/
[root@k8s231 homework]# ll bird-chart/templates/
总用量 20
-rw-r--r-- 1 root root 523 2024-07-16 13:59 configMap.yaml
-rw-r--r-- 1 root root 1417 2024-07-16 13:59 deployments.yaml
-rw-r--r-- 1 root root 218 2024-07-16 13:56 NOTES.txt
-rw-r--r-- 1 root root 240 2024-07-16 13:59 persistentvolumeclaims.yaml
-rw-r--r-- 1 root root 181 2024-07-16 13:59 services.yaml
[root@k8s231 homework]# helm install bird bird-chart
NAME: bird
LAST DEPLOYED: Tue Jul 16 14:08:11 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
###############################
# 老男孩教育官网： #
# www.oldboyedu.com #
###############################恭喜您，"harbor.oldboyedu.com/games/bird:v1"服务部署成功。
[root@k8s231 homework]# helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
bird default 1 2024-07-16 14:08:11.794678949 +0800 CST deployed bird-chart-0.1.0 1.16.0
traefik default 1 2024-07-15 13:54:35.443868935 +0800 CST deployed traefik-22.1.0 v2.9.10
root@k8s231 homework]# cat bird-chart/Chart.yaml
apiVersion: v2
name: bird-chart
description: A Helm chart for Kubernetes# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.Versions are expected to follow Semantic Versioning (https://semver.org/)
version: v1# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "v1"
至此，bird程序已经部署完
- 创建helm部署模板
[root@k8s231 homework]# cat bird-chart/values.yaml
replicas: 2secret:name: harbor-gamesapps: name: oldboyedu-linux85-deploy-birdlabel: bird image: harbor.oldboyedu.com/games/birdtag: v1cmname: bird-confpvcname: oldboyedu-games-birdcontainername: birdcode:srcpath: /usr/local/nginx/html/bird/destpath: /usr/share/nginx/html/bird/cm: name: bird-confservername: bird.oldboyedu.compvc: name: oldboyedu-games-birdsc: managed-nfs-storagesvc:name: oldboyedu-linux85-birdport: 80targetport: 80nodeport: 30080type: NodePort[root@k8s231 homework]# cat bird-chart/templates/deployments.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: {{ .Values.apps.name }}
spec:replicas: {{ .Values.replicas }}selector:matchExpressions:- key: appsoperator: Existstemplate:metadata:labels:apps: {{ .Values.apps.label }}spec:volumes:- name: nginxconfigMap:name: {{ .Values.apps.cmname }}items:- key: nginx.confpath: nginx.conf- name: code# nfs:# server: 10.0.0.231# path: /oldboyedu/data/kubernetes/homework/birdpersistentVolumeClaim:claimName: {{ .Values.apps.pvcname }}imagePullSecrets:- name: {{ .Values.secret.name }}containers:- name: {{ .Values.apps.containername }}image: {{ .Values.apps.image }}:{{ .Values.apps.tag }} volumeMounts:- name: nginxmountPath: /etc/nginx/nginx.confsubPath: nginx.conf- name: codemountPath: /usr/share/nginx/htmllifecycle:postStart:exec:# 此处为了简单起见，直接从镜像中拷贝数据，生产环境中可以考虑去镜像仓库拉取镜像# 为了不污染业务镜像，对于复杂的操作，可以考虑使用初始化容器拉取代码，这样就不会对业务容器造成侵入性！command: - "/bin/sh"- "-c"- "cp -r {{ .Values.code.srcpath }} {{ .Values.code.destpath }} "
[root@k8s231 homework]# cat bird-chart/templates/configMap.yaml
apiVersion: v1
kind: ConfigMap
metadata:name: {{ .Values.cm.name }}
data:nginx.conf: |worker_processes 1;events {worker_connections 1024;}http {include mime.types;default_type application/octet-stream;sendfile on;keepalive_timeout 65;server {listen 80;# root /usr/local/nginx/html/bird/;# root /usr/share/nginx/html/bird/;root {{ .Values.code.destpath }};# server_name bird.oldboyedu.com;server_name {{ .Values.cm.servername }};}}
[root@k8s231 homework]# cat bird-chart/templates/services.yaml
apiVersion: v1
kind: Service
metadata:name: {{ .Values.svc.name }}
spec:selector:apps: {{ .Values.apps.label }}type: {{ .Values.svc.type }}ports:- port: {{ .Values.svc.port }}targetPort: {{ .Values.svc.targetport }}nodePort: {{ .Values.svc.nodeport }}
[root@k8s231 homework]# cat bird-chart/templates/persistentvolumeclaims.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: {{ .Values.pvc.name }}
spec:storageClassName: {{ .Values.pvc.sc }}accessModes:- ReadWriteManyresources:limits:storage: 4Girequests:storage: 3Gi
[root@k8s231 homework]# [root@k8s231 homework]# helm install bird bird-chart
NAME: bird
LAST DEPLOYED: Tue Jul 16 14:56:02 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
###############################
# 老男孩教育官网： #
# www.oldboyedu.com #
###############################恭喜您，"harbor.oldboyedu.com/games/bird:v1"服务部署成功。
[root@k8s231 homework]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-c494888bb-xx8gl 1/1 Running 2 (5h54m ago) 4d
oldboyedu-linux85-deploy-bird-b6679755f-bv57m 0/1 CrashLoopBackOff 1 (2s ago) 5s
oldboyedu-linux85-deploy-bird-b6679755f-v2vsm 0/1 CrashLoopBackOff 1 (3s ago) 5s
traefik-5f98b98777-jh29z 1/1 Running 1 (5h55m ago) 25h(4)其他服务仿照模板创建
[root@k8s231 homework]# cp -r bird-chart/ chengbao-chert
[root@k8s231 homework]# cp -r bird-chart/ pinshu-chert
[root@k8s231 homework]# cp -r bird-chart/ tanke-chert
[root@k8s231 homework]# cp -r bird-chart/ pingtai-chert[root@k8s231 homework]# cat chengbao-chert/values.yaml
replicas: 2secret:name: harbor-gamesapps: name: oldboyedu-linux85-deploy-chengbaolabel: chengbao image: harbor.oldboyedu.com/games/chengbaotag: v1cmname: chengbao-confpvcname: oldboyedu-games-chengbaocontainername: chengbaocode:srcpath: /usr/local/nginx/html/chengbao/destpath: /usr/share/nginx/html/chengbao/cm: name: chengbao-confservername: chengbao.oldboyedu.compvc: name: oldboyedu-games-chengbaosc: managed-nfs-storagesvc:name: oldboyedu-linux85-chengbaoport: 80targetport: 80nodeport: 30081type: NodePort# 其余文件复制进去之后进行修改
# helm模板部署
[root@k8s231 homework]# helm install chengbao chengbao-chert
[root@k8s231 homework]# helm install pingtai pingtai-chert
[root@k8s231 homework]# helm install pinshu pinshu-chert
[root@k8s231 homework]# helm install tanke tanke-chert
(5) 查看服务状态
[root@k8s231 homework]# kubectl get po
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-c494888bb-xx8gl 1/1 Running 2 (8h ago) 4d3h
oldboyedu-linux85-deploy-bird-5597d95f55-cmcvh 1/1 Running 0 139m
oldboyedu-linux85-deploy-bird-5597d95f55-hr6vf 1/1 Running 0 139m
oldboyedu-linux85-deploy-chengbao-7c557f9b8f-kvmrl 1/1 Running 0 113m
oldboyedu-linux85-deploy-chengbao-7c557f9b8f-rxk79 1/1 Running 0 113m
oldboyedu-linux85-deploy-pingtai-64d9568fc6-bwtjm 1/1 Running 0 107m
oldboyedu-linux85-deploy-pingtai-64d9568fc6-pzrgp 1/1 Running 0 107m
oldboyedu-linux85-deploy-pinshu-8db6f59f9-7fp2x 1/1 Running 0 105m
oldboyedu-linux85-deploy-pinshu-8db6f59f9-zvvwc 1/1 Running 0 105m
oldboyedu-linux85-deploy-tanke-68fd88ffb9-4szwp 1/1 Running 0 104m
oldboyedu-linux85-deploy-tanke-68fd88ffb9-vlk9x 1/1 Running 0 104m
traefik-5f98b98777-jh29z 1/1 Running 1 (8h ago) 27h
1.5 ingress规则编写
[root@k8s231 ingress]# cat 01-ingress-games.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: oldboyedu-games-ingressesannotations:kubernetes.io/ingress.class: traefik # 指定Ingress 控制器为"traefik"
spec:# 定义Ingress规则rules:# 访问的主机名- host: bird.oldboyedu.com# 定义http的相关规则http:paths:- backend:service: name: oldboyedu-linux85-birdport:number: 80path: "/"pathType: "Prefix"- host: chengbao.oldboyedu.comhttp:paths:- backend:service: name: oldboyedu-linux85-chengbaoport: number: 80path: "/"pathType: "Prefix"- host: tanke.oldboyedu.comhttp:paths:- backend:service:name: oldboyedu-linux85-tankeport: number: 80path: "/"pathType: "Prefix"- host: pinshu.oldboyedu.comhttp:paths:- backend:service:name: oldboyedu-linux85-pinshuport: number: 80path: "/"pathType: "Prefix"- host: pingtai.oldboyedu.comhttp:paths:- backend:service:name: oldboyedu-linux85-pingtaiport: number: 80path: "/"pathType: "Prefix"[root@k8s231 ingress]# kubectl apply -f 01-ingress-games.yaml
[root@k8s231 ingress]# kubectl get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
oldboyedu-games-ingresses <none> bird.oldboyedu.com,chengbao.oldboyedu.com,tanke.oldboyedu.com + 2 more... 80 100m# traefik使用宿主机网络
[root@k8s231 adds-on]# vim traefik/values.yaml
....
# If hostNetwork is true, runs traefik in the host network namespace
# To prevent unschedulabel pods due to port collisions, if hostNetwork=true
# and replicas>1, a pod anti-affinity is recommended and will be set if the
# affinity is left as default.
hostNetwork: true # 默认是false，这里修改为true[root@k8s231 adds-on]# helm uninstall traefik
[root@k8s231 adds-on]# helm install traefik traefik
[root@k8s231 adds-on]# kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP
nfs-client-provisioner-c494888bb-xx8gl 1/1 Running 3 (109m ago) 4d20h 10.100.2.94
oldboyedu-linux85-deploy-bird-5597d95f55-cmcvh 1/1 Running 1 (109m ago) 19h 10.100.2.95
oldboyedu-linux85-deploy-bird-5597d95f55-hr6vf 1/1 Running 1 (109m ago) 19h 10.100.1.112
oldboyedu-linux85-deploy-chengbao-7c557f9b8f-kvmrl 1/1 Running 1 (109m ago) 18h 10.100.1.107
oldboyedu-linux85-deploy-chengbao-7c557f9b8f-rxk79 1/1 Running 1 (109m ago) 18h 10.100.2.96
oldboyedu-linux85-deploy-pingtai-64d9568fc6-bwtjm 1/1 Running 1 (109m ago) 18h 10.100.2.98
oldboyedu-linux85-deploy-pingtai-64d9568fc6-pzrgp 1/1 Running 1 (109m ago) 18h 10.100.1.115
oldboyedu-linux85-deploy-pinshu-8db6f59f9-7fp2x 1/1 Running 1 (109m ago) 18h 10.100.1.113
oldboyedu-linux85-deploy-pinshu-8db6f59f9-zvvwc 1/1 Running 1 (109m ago) 18h 10.100.2.92
oldboyedu-linux85-deploy-tanke-68fd88ffb9-4szwp 1/1 Running 1 (109m ago) 18h 10.100.2.93
oldboyedu-linux85-deploy-tanke-68fd88ffb9-vlk9x 1/1 Running 1 (109m ago) 18h 10.100.1.108
traefik-75bd8494cd-nvmkg 1/1 Running 0 13s 10.0.0.232
测试访问，浏览器输入
http://tanke.oldboyedu.com:8000/ #等其余
2、项目1-jenkins和k8s集成
2.1 Jenkins实现k8s持续集成项目流程图解
项目背景:应公司领导要求，将测试环境上线K8S集群，并集成Jenkins实现自动的代码发布和回滚。项目职责:(1)参与集群规划，项目部署，文档整理，跨部门沟通，领导交代的其他任务;(2)负责gitlab的高可用环境部署，备份，恢复，及技术文档编写，日常故障处理等工作;(3)负责Jenkins环境搭建，高可用环境部署，技术文档编写，日常故障处理;(4)负责Harbor仓库的高可用部署，及账号管理，docker-registry镜像仓库迁移，Dockerfile，dokcer-compose编写等工作;(5)负责K8S集群高可用环境部署，集群维护，监控，技术文档编写，以及集成Jenkins实现代码的自动上线，技术文档编写等工作;项目收获:对Jenkins，K8S,Gitlab，Harbor等云原生技术有一定生产环境的经验积累，尤其是对K8S日常运维有了一定的使用心得。
2.2 Jenkins集成K8S-将代码推送到gitee
(1)下载软件包
[root@k8s231.oldboyedu.com jenkins]# wget http://192.168.15.253/Kubernetes/day10-/softwares/jenkins-k8s.zip
(2)解压软件包
[root@k8s231.oldboyedu.com jenkins]# unzip jenkins-k8s.zip
(3)解压开发代码
[root@k8s231.oldboyedu.com jenkins]# cd jenkins-k8s/
[root@k8s231.oldboyedu.com jenkins-k8s]# unzip yiliaoqixie.zip
(4)将代码推送到gitee
[root@k8s231.oldboyedu.com jenkins-k8s]# cd yiliaoqixie/
[root@k8s231.oldboyedu.com yiliaoqixie]# yum -y install git
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git config --global user.name "jasonyin2020"
[root@k8s231.oldboyedu.com yiliaoqixie]# git config --global user.email "y1053419035@qq.com"
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git init
Initialized empty Git repository in /manifests/project/jenkins/jenkins-k8s/yiliaoqixie/.git/
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git add .
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git commit -m 'first commit'
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git tag -a v0.1 -m 'first commit'
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git tag -l
v0.1
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git remote add origin https://gitee.com/jasonyin2020/oldboyedu-linux85-yiliao.git
[root@k8s231 yiliaoqixie]# git push -u origin "master"
Username for 'https://gitee.com': dws123456
Password for 'https://dws123456@gitee.com': dwsDWS@19960720
Counting objects: 90, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (90/90), done.
Writing objects: 100% (90/90), 1.48 MiB | 1.68 MiB/s, done.
Total 90 (delta 10), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/dws123456/oldboyedu-linux85-yiliao.git* [new branch] master -> master
分支 master 设置为跟踪来自 origin 的远程分支 master。[root@k8s231 yiliaoqixie]# git tag -a v0.2 -m 'add dockerfile'
root@k8s231 yiliaoqixie]# git push origin master --tags
Username for 'https://gitee.com': dws123456
Password for 'https://dws123456@gitee.com':
Counting objects: 1, done.
Writing objects: 100% (1/1), 159 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/dws123456/oldboyedu-linux85-yiliao.git* [new tag] v0.2 -> v0.2[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git push -u origin "master" --tags
Username for 'https://gitee.com': jasonyin2020
Password for 'https://jasonyin2020@gitee.com':
Counting objects: 1, done.
Writing objects: 100% (1/1), 160 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/jasonyin2020/oldboyedu-linux85-yiliao.git* [new tag] status -> status* [new tag] v0.1 -> v0.1
Branch master set up to track remote branch master from origin.
[root@k8s231.oldboyedu.com yiliaoqixie]#
2.3 Jenkins集成K8S-部署Jenkins
(1)安装JDK环境，如上图所示
[root@k8s231.oldboyedu.com jenkins-k8s]# rpm -ivh jdk-8u102-linux-x64.rpm && java -version
(2)解压tomcat软件包
[root@k8s231.oldboyedu.com jenkins-k8s]# mkdir -pv /oldboyedu/softwares && tar xf apache-tomcat-8.0.27.tar.gz -C /oldboyedu/softwares
(3)删除tomcat的初始数据
[root@k8s231.oldboyedu.com jenkins-k8s]# rm -rf /oldboyedu/softwares/apache-tomcat-8.0.27/webapps/*
(4)将Jenkins的war包放入到tomcat的应用目录，注意，复制该名称
[root@k8s231.oldboyedu.com jenkins-k8s]# cp jenkins.war /oldboyedu/softwares/apache-tomcat-8.0.27/webapps/ROOT.war
(5)解压jenkins数据到"/root"目下，会创建一个隐藏目录(.jenkins)
[root@k8s231.oldboyedu.com jenkins-k8s]# tar xf jenkins-data.tar.gz -C /root/
(6)启动tomcat服务
[root@k8s231.oldboyedu.com jenkins-k8s]# /oldboyedu/softwares/apache-tomcat-8.0.27/bin/startup.sh
(7)检查端口是否存在
[root@k8s231.oldboyedu.com jenkins-k8s]# ss -ntl | grep 8080
LISTEN 0 100 [::]:8080 [::]:*
[root@k8s231.oldboyedu.com jenkins-k8s]#
(9)验证Jenkins的WebUI是否可以正常访问
http://10.0.0.231:8080/
2.4 Jenkins集成K8S-编写dockerfile
(1)编写dockerfile及编译脚本
[root@k8s231.oldboyedu.com yiliaoqixie]# cat Dockerfile
FROM harbor.oldboyedu.com/web/apps:v1LABEL school=oldboyedu \class=linux85 \auther=JasonYinADD . /usr/share/nginx/html
[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# cat build.sh
#!/bin/bashVERSION=$1docker login -u admin -p 1 harbor.oldboyedu.com
docker build -t harbor.oldboyedu.com/project/yiliaoqixie:v0.${VERSION:-1} .
docker push harbor.oldboyedu.com/project/yiliaoqixie:v0.${VERSION:-1}
docker logout harbor.oldboyedu.com
[root@k8s231 yiliaoqixie]# chmod +x build.sh
[root@k8s231 yiliaoqixie]# ./build.sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded
Sending build context to Docker daemon 4.015MB
Step 1/3 : FROM harbor.oldboyedu.com/update/apps:v1---> 5caeb29b76b3
Step 2/3 : LABEL school=oldboyedu class=linux85 auther=JasonYin---> Running in 2e199a1d4441
Removing intermediate container 2e199a1d4441---> 3a2f7f32aaa9
Step 3/3 : ADD . /usr/share/nginx/html---> 654fa2d1337f
Successfully built 654fa2d1337f
Successfully tagged harbor.oldboyedu.com/project/yiliaoqixie:v0.1
The push refers to repository [harbor.oldboyedu.com/project/yiliaoqixie]
dc256e1300e9: Pushed
be2b839367f1: Mounted from update/apps
9d5b000ce7c7: Mounted from update/apps
b8dbe22b95f7: Mounted from update/apps
c39c1c35e3e8: Mounted from update/apps
5f66747c8a72: Mounted from update/apps
15d7cdc64789: Mounted from update/apps
7fcb75871b21: Mounted from update/apps
v0.1: digest: sha256:62d93cc9ec5cd5305edca04a9f1645e6185eb0a6265a311af4b60045324d2449 size: 1986
Removing login credentials for harbor.oldboyedu.com# 创建测试
[root@k8s231 yiliaoqixie]# docker run -d -p 88:80 harbor.oldboyedu.com/project/yiliaoqixie:v0.1
e8212cdea34b6d18922b1b94183477a0898c30af8411a61a94bc0c25fcac78f2
[root@k8s231 yiliaoqixie]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8212cdea34b harbor.oldboyedu.com/project/yiliaoqixie:v0.1 "/docker-entrypoint.…" 44 seconds ago Up 43 seconds 0.0.0.0:88->80/tcp, :::88->80/tcp wizardly_dijkstra
[root@k8s231 yiliaoqixie]# docker rm -f wizardly_dijkstra
wizardly_dijkstra
(2)推送代码
[root@k8s231.oldboyedu.com yiliaoqixie]# git add .
[root@k8s231 yiliaoqixie]# git commit -m 'add build.sh and dockerfile'
[master ac198b0] add build.sh and dockerfile2 files changed, 19 insertions(+)create mode 100644 Dockerfilecreate mode 100755 build.sh
[root@k8s231 yiliaoqixie]# git push origin master
Username for 'https://gitee.com': dws123456
Password for 'https://dws123456@gitee.com':
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 614 bytes | 0 bytes/s, done.
Total 4 (delta 1), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/dws123456/oldboyedu-linux85-yiliao.git7084466..ac198b0 master -> master[root@k8s231.oldboyedu.com yiliaoqixie]# git commit -m 'update build.sh'
[root@k8s231.oldboyedu.com yiliaoqixie]# git push -u origin --all
Username for 'https://gitee.com': jasonyin2020
Password for 'https://jasonyin2020@gitee.com':
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 329 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/jasonyin2020/oldboyedu-linux85-yiliao.git4c4fcac..c96d1de master -> master
Branch master set up to track remote branch master from origin.[root@k8s231.oldboyedu.com yiliaoqixie]#
[root@k8s231.oldboyedu.com yiliaoqixie]# git push -u origin --tags
Username for 'https://gitee.com': jasonyin2020
Password for 'https://jasonyin2020@gitee.com':
Counting objects: 1, done.
Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/jasonyin2020/oldboyedu-linux85-yiliao.git* [new tag] v0.3 -> v0.3
[root@k8s231.oldboyedu.com yiliaoqixie]#
(3)手动创建资源清单
[root@k8s231.oldboyedu.com jenkins]# cat deploy-yiliaoqixie.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: oldboyedu-linux-yiliao
spec:replicas: 3selector:matchExpressions:- key: appsoperator: Existstemplate:metadata:labels:apps: yiliaospec:containers:- name: yiliaoimage: harbor.oldboyedu.com/project/yiliaoqixie:v0.4---apiVersion: v1
kind: Service
metadata:name: oldboyedu-linux-yiliao-svc
spec:selector:apps: yiliaotype: ClusterIPports:- port: 80targetPort: 80---apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: oldboyedu-linux-yiliaoannotations:kubernetes.io/ingress.class: traefik # 指定Ingress 控制器为"traefik"
spec:rules:- host: yiliao.oldboyedu.comhttp:paths:- backend:service:name: oldboyedu-linux-yiliao-svcport:number: 80path: "/"pathType: "Prefix"
[root@k8s231.oldboyedu.com jenkins]#
[root@k8s231.oldboyedu.com jenkins]# kubectl apply -f deploy-yiliaoqixie.yaml
(4)更新应用
[root@k8s231 yiliaoqixie]# vim index.html
<!DOCTYPE html><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><meta http-equiv="X-UA-Compatible" content="IE=edge" /><meta name="viewport" content="width=device-width, initial-scale=1" /><meta name="applicable-device" content="pc,mobile" /><title>老男孩IT教育医疗器械有限公司</title>
....
[root@k8s231 yiliaoqixie]# git add .
[root@k8s231 yiliaoqixie]# git commit -m 'update index.html'
[master e051152] update index.html2 files changed, 1 insertion(+), 1 deletion(-)create mode 100644 master
[root@k8s231 yiliaoqixie]# git push -u origin --all
Username for 'https://gitee.com': dws123456
Password for 'https://dws123456@gitee.com':
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 355 bytes | 0 bytes/s, done.
Total 4 (delta 2), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/dws123456/oldboyedu-linux85-yiliao.gitac198b0..e051152 master -> master
分支 master 设置为跟踪来自 origin 的远程分支 master。[root@k8s231 yiliaoqixie]# git add .
[root@k8s231 yiliaoqixie]# git commit -m 'update build.sh'
[master dc54669] update build.sh1 file changed, 1 insertion(+), 1 deletion(-)
[root@k8s231 yiliaoqixie]# git push -u origin --all
Username for 'https://gitee.com': dws123456
Password for 'https://dws123456@gitee.com':
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 284 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Powered by GITEE.COM [GNK-6.4]
To https://gitee.com/dws123456/oldboyedu-linux85-yiliao.gite051152..dc54669 master -> master
分支 master 设置为跟踪来自 origin 的远程分支 master。
2.5 Jenkins集成K8S-部署应用到K8S
参考资源清单。
今日作业:
- 完成课堂的所有练习并整理思维导图;
- 请完善"build.sh"脚本，实现部署和升级;
- 使用Jenkins的webhook功能，当gitee有代码变更时，自动部署更新到K8S集群，版本对应的当前构建的编号ID。