k8s学习(二十九) RocketMq集群(双主双从-同步)

2024-06-20 08:18
文章标签 云原生 集群 学习 k8s rocketmq 同步 二十九 双主双

本文主要是介绍k8s学习(二十九) RocketMq集群(双主双从-同步)，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

确认安装前已经安装好StorageClass，可参考：https://blog.csdn.net/u011943534/article/details/100887530

1、准备jdk镜像，并拷贝到服务器

docker pull java:8-alpine

2、制作rocketmq镜像
下载地址：https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/
将”rocketmq-all-4.8.0-bin-release.zip”拷贝至服务器，解压
修改bin/runbroker.sh,将内存调整为适当（比如调小）

JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"

编写Dockerfile-rocketmq:

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0 \
&& mkdir -p /data/rocketmq/storeCMD ["/bin/bash"]

docker build -f Dockerfile-rocketmq -t 172.16.10.190:80/library/rocketmq:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq:4.8.0

3、制作nameserver镜像
使用第三步的rocketmq-all-4.8.0-bin-release
在conf/下添加namesrv.properties
内容为：

listenPort=20901

编写Dokerfile-namesrv:

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0WORKDIR /usr/local/rocketmq-4.8.0
CMD ["/usr/local/rocketmq-4.8.0/bin/mqnamesrv","-c","/usr/local/rocketmq-4.8.0/conf/namesrv.properties"]
docker build -f Dockerfile-nameserver -t 172.16.10.190:80/library/rocketmq-namesrv:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq-namesrv:4.8.0

4、制作rocketmq-ui镜像
下载源码：
git clone https://github.com/apache/rocketmq-externals.git
下载maven:
wget http://mirrors.hust.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

tar -zvxf rocketmq-externals.tar.gz

将本目录下的”application.properties“中内容替换到rocketmq-externals/rocketmq-console/src/main/resources/application.properties中
编写Dockerfile-ui

FROM java:8-alpineRUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezoneENV LANG en_US.UTF-8
ENV MAVEN_HOME /usr/local/maven
ENV PATH $PATH:$MAVEN_HOME/binADD rocketmq-externals /usr/local/rocketmq-externals
ADD apache-maven-3.3.9 /usr/local/apache-maven-3.3.9
RUN mv /usr/local/apache-maven-3.3.9 /usr/local/mavenWORKDIR /usr/local/rocketmq-externals/rocketmq-console/
RUN mvn clean package -Dmaven.test.skip=trueWORKDIR /usr/local/rocketmq-externals/rocketmq-console/target
EXPOSE 8080
CMD ["nohup","java","-jar","rocketmq-console-ng-1.0.0.jar","&"]

5、准备configmap
broker-a.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER

broker-a-s.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

broker-b.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER
flushDiskType=SYNC_FLUSH

broker-b-s.properties:

brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH

根据以上文件生成configmap:

kubectl create configmap rocketmq-config --from-file=broker-a.properties --from-file=broker-b.properties --from-file=broker-a-s.properties --from-file=broker-b-s.properties  -n default

6、准备服务文件
broker-a-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-aname: broker-anamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30911selector:app: broker-a
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-anamespace: default
spec:serviceName: broker-areplicas: 1selector:matchLabels:app: broker-atemplate:metadata:labels:app: broker-aspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-atopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-aimage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-a.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-a.propertiessubPath: broker-a.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-config- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-a-s-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-a-sname: broker-a-snamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30912selector:app: broker-a-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-a-snamespace: default
spec:serviceName: broker-a-sreplicas: 1selector:matchLabels:app: broker-a-stemplate:metadata:labels:app: broker-a-sspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-a-stopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-a-simage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-a-s.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-a-s.propertiessubPath: broker-a-s.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-configitems:- key: broker-a-s.propertiespath: broker-a-s.properties- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-b-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-bname: broker-bnamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30913selector:app: broker-b
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-bnamespace: default
spec:serviceName: broker-breplicas: 1selector:matchLabels:app: broker-btemplate:metadata:labels:app: broker-bspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-btopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-bimage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-b.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-b.propertiessubPath: broker-b.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-config- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

broker-b-s-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: broker-b-sname: broker-b-snamespace: default
spec:type: NodePortports:- port: 20911targetPort: 20911name: broker-portnodePort: 30914selector:app: broker-b-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: broker-b-snamespace: default
spec:serviceName: broker-b-sreplicas: 1selector:matchLabels:app: broker-b-stemplate:metadata:labels:app: broker-b-sspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- broker-b-stopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: broker-b-simage: 172.16.10.190:80/library/rocketmq:4.8.0imagePullPolicy: Alwayscommand: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker  -c /usr/local/rocketmq-4.8.0/conf/broker-b-s.properties"]volumeMounts:- mountPath: /root/logsname: rocketmq-datasubPath: mq-brokeroptlogs- mountPath: /data/rocketmqname: rocketmq-datasubPath: mq-brokeroptstore- name: broker-configmountPath: /usr/local/rocketmq-4.8.0/conf/broker-b-s.propertiessubPath: broker-b-s.properties- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20911initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: broker-configconfigMap:name: rocketmq-configitems:- key: broker-b-s.propertiespath: broker-b-s.properties- name: timehostPath:path: /usr/share/zoneinfo/Asia/ShanghaivolumeClaimTemplates:- metadata:name: rocketmq-dataannotations:volume.beta.kubernetes.io/storage-class: "course-nfs-storage"spec:accessModes:- ReadWriteManyresources:requests:storage: 4Gi

namesrv-statefulset.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: mq-namesrvname: mq-namesrvnamespace: default
spec:type: NodePortports:- port: 20901targetPort: 20901name: namesrv-portnodePort: 30915selector:app: mq-namesrv
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: mq-namesrvnamespace: default
spec:serviceName: mq-namesrvreplicas: 1selector:matchLabels:app: mq-namesrvtemplate:metadata:labels:app: mq-namesrvspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- mq-namesrvtopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: mq-namesrvimage: 172.16.10.190:80/library/rocketmq-namesrv:4.8.0imagePullPolicy: AlwaysvolumeMounts:- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 20901initialDelaySeconds: 15timeoutSeconds: 5periodSeconds: 20volumes:- name: timehostPath:path: /usr/share/zoneinfo/Asia/Shanghai

rocketmq-externals-deployment.yaml：

apiVersion: v1
kind: Service
metadata:labels:app: mq-externalsname: mq-externalsnamespace: default
spec:type: NodePortports:- port: 8080targetPort: 8080name: console-portnodePort: 30916selector:app: mq-externals
---
apiVersion: apps/v1
kind: Deployment
metadata:name: mq-externalsnamespace: default
spec:replicas: 1selector:matchLabels:app: mq-externalstemplate:metadata:labels:app: mq-externalsspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- mq-externalstopologyKey: "kubernetes.io/hostname"imagePullSecrets:- name: harborsecretcontainers:- name: mq-externalsimage: 172.16.10.190:80/library/rocketmq-externals:v3imagePullPolicy: AlwaysvolumeMounts:- name: timemountPath: /etc/localtimelifecycle:postStart:exec:command: ["/bin/sh","-c","touch /tmp/health"]livenessProbe:exec:command: ["test","-e","/tmp/health"]initialDelaySeconds: 5timeoutSeconds: 5periodSeconds: 10readinessProbe:tcpSocket:port: 8080initialDelaySeconds: 15volumes:- name: timehostPath:path: /usr/share/zoneinfo/Asia/Shanghai

7、部署服务
将6步的文件拷贝至K8s

kubectl apply -f .

8、查看
http://172.16.10.160:30916
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

这篇关于k8s学习(二十九) RocketMq集群(双主双从-同步)的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！


http://www.chinasem.cn/article/1077590。

相关文章

canal实现mysql数据同步的详细过程

canal实现mysql数据同步的详细过程

《canal实现mysql数据同步的详细过程》：本文主要介绍canal实现mysql数据同步的详细过程,本文通过实例图文相结合给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的... 目录1、canal下载2、mysql同步用户创建和授权3、canal admin安装和启动4、canal
阅读更多...
Go学习记录之runtime包深入解析

Go学习记录之runtime包深入解析

《Go学习记录之runtime包深入解析》Go语言runtime包管理运行时环境,涵盖goroutine调度、内存分配、垃圾回收、类型信息等核心功能,：本文主要介绍Go学习记录之runtime包的... 目录前言：一、runtime包内容学习1、作用：① Goroutine和并发控制：② 垃圾回收：③ 栈和
阅读更多...
Linux实现线程同步的多种方式汇总

Linux实现线程同步的多种方式汇总

《Linux实现线程同步的多种方式汇总》本文详细介绍了Linux下线程同步的多种方法,包括互斥锁、自旋锁、信号量以及它们的使用示例,通过这些同步机制,可以解决线程安全问题,防止资源竞争导致的错误,示例... 目录什么是线程同步？一、互斥锁（单人洗手间规则）适用场景：特点：二、条件变量（咖啡厅取餐系统）工作流
阅读更多...
Mysql的主从同步/复制的原理分析

Mysql的主从同步/复制的原理分析

《Mysql的主从同步/复制的原理分析》：本文主要介绍Mysql的主从同步/复制的原理分析,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录为什么要主从同步？mysql主从同步架构有哪些？Mysql主从复制的原理/整体流程级联复制架构为什么好？Mysql主从复制注意
阅读更多...
Android学习总结之Java和kotlin区别超详细分析

Android学习总结之Java和kotlin区别超详细分析

《Android学习总结之Java和kotlin区别超详细分析》Java和Kotlin都是用于Android开发的编程语言,它们各自具有独特的特点和优势,：本文主要介绍Android学习总结之Ja... 目录一、空安全机制真题 1：Kotlin 如何解决 Java 的 NullPointerExceptio
阅读更多...
Mac备忘录怎么导出/备份和云同步? Mac备忘录使用技巧

Mac备忘录怎么导出/备份和云同步? Mac备忘录使用技巧

《Mac备忘录怎么导出/备份和云同步?Mac备忘录使用技巧》备忘录作为iOS里简单而又不可或缺的一个系统应用，上手容易，可以满足我们日常生活中各种记录的需求，今天我们就来看看Mac备忘录的导出、... 「备忘录」是 MAC 上的一款常用应用，它可以帮助我们捕捉灵感、记录待办事项或保存重要信息。为了便于在不同
阅读更多...
查看MySql主从同步的偏移量方式

查看MySql主从同步的偏移量方式

《查看MySql主从同步的偏移量方式》：本文主要介绍查看MySql主从同步的偏移量方式,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 1.mysql的主从同步方案mysqlphp为了在实现读写分离，主库写，从库读mysql的同步方案主要是通过从库读取主库的binl
阅读更多...
Nginx使用Keepalived部署web集群（高可用高性能负载均衡）实战案例

Nginx使用Keepalived部署web集群（高可用高性能负载均衡）实战案例

《Nginx使用Keepalived部署web集群（高可用高性能负载均衡）实战案例》本文介绍Nginx+Keepalived实现Web集群高可用负载均衡的部署与测试,涵盖架构设计、环境配置、健康检查、... 目录前言一、架构设计二、环境准备三、案例部署配置 前端 Keepalived配置 前端 Nginx
阅读更多...
重新对Java的类加载器的学习方式

重新对Java的类加载器的学习方式

《重新对Java的类加载器的学习方式》：本文主要介绍重新对Java的类加载器的学习方式,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录1、介绍1.1、简介1.2、符号引用和直接引用1、符号引用2、直接引用3、符号转直接的过程2、加载流程3、类加载的分类3.1、显示
阅读更多...
Redis高可用-主从复制、哨兵模式与集群模式详解

Redis高可用-主从复制、哨兵模式与集群模式详解

《Redis高可用-主从复制、哨兵模式与集群模式详解》：本文主要介绍Redis高可用-主从复制、哨兵模式与集群模式的使用,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝... 目录Redis高可用-主从复制、哨兵模式与集群模式概要一、主从复制（Master-Slave Repli
阅读更多...

Copyright China编程 23002807@qq.com

沪ICP备2023033072号-2