Skip to the content.

jinja和jenkins结合做为kubernetes的服务发布平台

公司使用java开发语言,使用kubernetes集群运行这些java服务,为了试发布便捷,使用jenkins构建,利用jinja对模板渲染,利用Jenkins做一套kubernetes的发布平台。流程如下:

gitlab提交代码–jenkins构建(maven(jar包),Dockerfile(docker包))–镜像推送到harbor仓库–生成Deployment模板–kubectl create pod.yaml

jinja渲染模板配置,为生成deployment 配置文件

pip安装jinja2

yum install python-pip python-devel 
pip install jinja2

kubernetes pod的jinja模板

# cat /opt/templatekind: Deployment
apiVersion: apps/v1beta2
metadata:
  name: 
  labels:
    k8s-app: 
  annotations:
    kubernetes.io/change-cause: 
spec:
  replicas: 
  revisionHistoryLimit: 1
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  selector:
    matchLabels:
      k8s-app: 
  template:
    metadata:
      labels:
        k8s-app: 
    spec:
      containers:
      - name: 
        image: :
        imagePullPolicy: IfNotPresent
        resources:
          limits:
            memory: Mi
            #cpu: 
        ports:
        - containerPort: 
        livenessProbe:
          tcpSocket:
            port: 
          initialDelaySeconds: 15
          periodSeconds: 20
        volumeMounts:
          - name: --data-storage
            mountPath: /opt
      imagePullSecrets:
        - name: harbor-auth
      volumes:
      - name: --data-storage
        hostPath:
          path: "/tmp/data"   #线上为gfs集群,非本地存储
 
---
kind: Service
apiVersion: v1
metadata:
  name: -com
  labels:
    k8s-app: 
spec:
  selector:
    k8s-app: 
  ports:
  - port: 
    targetPort: 

简单说明一下上面deployment配置:

cpu不做限制是因为限制了cpu核数,服务启动会特别慢

各个变量都是通过jenkins传过来的,比如branch、name、port、memory

kubernetes内部各个服务中调用是通过-com, 要全部小写,并且不能有特殊字符和 .,这是因为DNS的限制,因为实际DNS内部地址是

-com.default.svc.cluster.local

程序之间相互调用是-com:

python发布脚本

# cat /opt/public.py 
#!/usr/bin/env python
#encoding=utf8
import sys
from jinja2 import Template
 
#template
def RenderTemplate(name="",image="",version="",replicas_num="",port="",branch="",namespace="",env_name="",env_value="",cpu="",memory=""):
    if namespace == "dev":
        with open('/opt/template', "r") as f:
            print(Template(f.read()).render(name=name,image=image,version=version,replicas_num=replicas_num,port=port,branch=branch,namespace=namespace,env_name=env_name,env_value=env_value,cpu=cpu,memory=memory))
 
#value
d = {}
jenkinsfile="/tmp/{0}jenkins".format(sys.argv[1])
with open(jenkinsfile,'r') as f:
    for line in f:
        (k,v) = line.split(':')
        if k == '':
            d[str(k)] = ""
        else:
            d[str(k)] = v.replace('\n','')
 
RenderTemplate(**d)

下面是jenkins配置

安装Jenkins和插件

下载最新 Java SE Development Kit 8u161:

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html?ssSourceSiteId=otnpt

下载最新Jenkins:

https://jenkins.io/download/

rpm -ivh jdk-8u161-linux-x64.rpm
rpm -ivh jenkins-2.89.4-1.1.noarch.rpm
systemctl status jenkins
systemctl enable jenkins
systemctl start jenkins

Jenkins启动后浏览器打开默认的8080端口,安装Build With Parameters Plugin插件,此插件可以设置自定义变量

Jenkins新建项目配置

General
  参数化构建过程
    下面添加branch、port、replicas_num、cpu、memory变量,变量可以为Choices,string类型
 
中间添加git代码仓库、maven打jar包忽略
 
Pre Steps
  Execute shell
    sed "s/9999/$port/" /opt/Dockerfile > $WORKSPACE/Dockerfile
    cp /opt/docker-entrypoint.sh $WORKSPACE/docker-entrypoint.sh
 
Post Steps  添加对变量的整理,并把jar包打为docker包
  Execute shell
    echo JobName=`echo $JOB_NAME|tr '[A-Z]' '[a-z]'`  > $WORKSPACE/target/${JOB_NAME}_env
    echo branch="$branch"  >> $WORKSPACE/target/${JOB_NAME}_env
    current_date=`TZ='UTC-8' date +'%Y%m%d%H%M'`
    docker_version=${branch}-$current_date-${GIT_COMMIT:0:8}
    echo Version="${docker_version}" >> $WORKSPACE/target/${JOB_NAME}_env
    echo gitinfo=\"`git log -n 1 --pretty=format:"%h - %an, %ar : %s"`\" >> $WORKSPACE/target/${JOB_NAME}_env
 
  Inject environment variables
    $WORKSPACE/target/${JOB_NAME}_env
 
  Execute shell
    cd $WORKSPACE
    docker build -t harbor.bbotte.com/${branch}/$JobName:$Version .
    docker push harbor.bbotte.com/${branch}/$JobName:$Version
 
  Send file or execute commands over SSH
    SSH server 选择模板和脚本所在的主机
    Exec command
      /opt/init_public.sh $JobName ${branch}/$JobName $Version $replicas_num $port $cpu $memory $branch dev "" ""

jenkins-config

# cat /opt/Dockerfile 
FROM openjdk:8-alpine
WORKDIR /
ADD ./target/*.jar app.jar
EXPOSE 9999
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["java","-server","-Duser.timezone=GMT+08","-jar","/app.jar"]

设置如上变量,如果没有则为空,下面是python使用jinja渲染模板,kubernetes创建服务

# cat /opt/init_public.sh 
#!/bin/bash
cat /dev/null > /tmp/${9}jenkins
echo name:$1 > /tmp/${9}jenkins
echo image:$2 >> /tmp/${9}jenkins
echo version:$3 >> /tmp/${9}jenkins
echo replicas_num:$4 >> /tmp/${9}jenkins
echo port:$5 >> /tmp/${9}jenkins
echo cpu:$6 >> /tmp/${9}jenkins
echo memory:$7 >> /tmp/${9}jenkins
echo branch:$8 >> /tmp/${9}jenkins
echo namespace:$9 >> /tmp/${9}jenkins
echo env_name:${10} >> /tmp/${9}jenkins
echo env_value:${11} >> /tmp/${9}jenkins
 
python /opt/public.py ${9} > /opt/yaml/$8-$1.yaml
kubectl apply -f /opt/yaml/$8-$1.yaml --record=false

下面是Jenkins添加Blue Ocean插件执行结果

jenkins-blue-ocean

上面配置即可完成Jenkins调用gitlab,通过maven打包后,发送给自建的docker仓库,并传送变量生成kubernetes deployment配置模板,从而创建服务的需要

2018年03月10日 于 linux工匠 发表