Seatunnel-2.3.3 自打包 docker部署(含web)

2024-04-29 15:04

本文主要是介绍Seatunnel-2.3.3 自打包 docker部署(含web),希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

前言

此篇重点是,自己将源码编译后,将打包文件部署在docker里(也可以直接用官网的)

如果也有人是希望,将自己打包的源码部署了,可以参考可乐的这篇文章,这篇文章详细介绍了2.3.3的serve和web的源码启动流程

Apache Seatunnel本地源码构建编译运行调试

前期准备

  1. docker环境
  2. seatunnel-2.3.3  二进制执行文件(apache-seatunnel-2.3.4-bin.tar.gz解压后的文件)
  3. seatunnel-web-1.0.0 二进制可执行文件
  4. cp /opt/seatunnel/apache-seatunnel-2.3.3/config/hazelcast-client.yaml /opt/seatunnel/apache-seatunnel-web-1.0.0-bin/conf/

4的命令不用执行,只要是给大家一个参考,记得吧hazelcast-client.yaml文件覆盖到二进制里面去,同时hazelcast-client.yaml的内容改一下,主要把127或者localhost改为本机宿主机ip

提示:  这里我就默认大家,已经解决了所有,可乐文章里的疑难杂症,也就是说,

源码是可以添加数据源,可以添加任务执行单表操作的。

如果!!! 不能保障!!!请先看可乐文章后,再阅读本篇,不然的话,真的会很痛苦,要不断编译,不断部署!!!

文件目录介绍

--docer-compose.yml

--seatunnel

        --Dockerfile

        --Dockerfile_web

        --apache-seatunnel-2.3.3

        --apache-seatunnel-web-1.0.0

文件内容

docker-compose.yml

version: '3'
services:seatunnel-serve:build:context: ./seatunnel/dockerfile: Dockerfilecontainer_name: seatunnel_serveports:- 5801:5801command: nohup sh bin/seatunnel-cluster.sh 2>&1 &volumes:- ./seatunnel/apache-seatunnel-2.3.3/:/opt/seatunnel/restart: alwaysnetworks:- appseatunnel-web:build:context: ./seatunnel/dockerfile: Dockerfile_webvolumes:- ./seatunnel/apache-seatunnel-web-1.0.0/:/opt/seatunnel_web/container_name: seatunnel_webentrypoint: /opt/seatunnel_web/bin/chmod-daemon.shports:- 8801:8801restart: alwaysnetworks:- app
volumes:data:
networks:app:external: true

Dockerfile

这个其实也可以不用,但是有个小瑕疵没解决,先预留

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#FROM openjdk:8
ENV SEATUNNEL_HOME="/opt/seatunnel"
RUN  mkdir -p SEATUNNEL_HOME/logs
WORKDIR /opt/seatunnel

Dockerfile_web

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#FROM openjdk:8
user root
ENV SEATUNNEL_HOME="/opt/seatunnel_web"
COPY ./apache-seatunnel-web-1.0.0/ ${SEATUNNEL_HOME}/
RUN  mkdir -p SEATUNNEL_HOME/logs
RUN chmod +x /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh
RUN chmod +x /opt/seatunnel_web/bin/chmod-daemon.sh
WORKDIR /opt/seatunnel_web
#CMD ["chmod"," +x", "bin/seatunnel-backend-daemon.sh"]
#CMD ["sh", "/opt/seatunnel_web/bin/seatunnel-backend-daemon.sh", "start"]CMD ["tail"," -f", "nohup.out"]

chmod-daemon.sh

这里有个小瑕疵

 sh /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh start权限问题一直没有解决

所以我在seatunnel-backend-daemon.sh打了echo,把参数都打印出来了,打印出来后直接贴到了这里,这个后期我解决了,我会上报解决方案

注意!!!chmod-daemon.sh会存在sh文件格式的问题

映射进去的时候可能是CRLF的格式,linux识别不来,所以在docker-compose里的entrypoint查找文件的时候,会报no such file or directory

注意文件格式是LF还是CRLF

LF   linux

CRLF  window

#!/bin/sh#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#echo test111111111
chmod 777 /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh
echo 111111111111111111111111123/usr/local/openjdk-8/bin/java -server -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof -Dseatunnel-web.logs.path=/opt/seatunnel_web/bin/../logs -cp /opt/seatunnel_web/bin/../conf:/opt/seatunnel_web/bin/../libs/*:/opt/seatunnel_web/bin/../datasource/* -Dspring.config.name=application.yml -Dspring.config.location=classpath:application.yml org.apache.seatunnel.app.SeatunnelApplication
# sh /opt/seatunnel_web/bin/seatunnel-backend-daemon.sh start

还有一种解决方案(这是可乐提供的解决方案,不过没有测试,先贴在这里)

sh 如果是在windows下直接打开复制过去的,会有让shell文件的格式有问题,镜像运行的时候报初始化脚本\r问题,所以需要在wsl进入centOs子系统中执行如下:
sed -i ‘s/\r$//’ filename
将xxxx.sh格式转为linux下的格式

这篇关于Seatunnel-2.3.3 自打包 docker部署(含web)的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/946412

相关文章

Python程序打包exe,单文件和多文件方式

《Python程序打包exe,单文件和多文件方式》:本文主要介绍Python程序打包exe,单文件和多文件方式,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录python 脚本打成exe文件安装Pyinstaller准备一个ico图标打包方式一(适用于文件较少的程

python web 开发之Flask中间件与请求处理钩子的最佳实践

《pythonweb开发之Flask中间件与请求处理钩子的最佳实践》Flask作为轻量级Web框架,提供了灵活的请求处理机制,中间件和请求钩子允许开发者在请求处理的不同阶段插入自定义逻辑,实现诸如... 目录Flask中间件与请求处理钩子完全指南1. 引言2. 请求处理生命周期概述3. 请求钩子详解3.1

SpringBoot项目Web拦截器使用的多种方式

《SpringBoot项目Web拦截器使用的多种方式》在SpringBoot应用中,Web拦截器(Interceptor)是一种用于在请求处理的不同阶段执行自定义逻辑的机制,下面给大家介绍Sprin... 目录一、实现 HandlerInterceptor 接口1、创建HandlerInterceptor实

Maven项目打包时添加本地Jar包的操作步骤

《Maven项目打包时添加本地Jar包的操作步骤》在Maven项目开发中,我们经常会遇到需要引入本地Jar包的场景,比如使用未发布到中央仓库的第三方库或者处理版本冲突的依赖项,本文将详细介绍如何通过M... 目录一、适用场景说明​二、核心操作命令​1. 命令格式解析​2. 实战案例演示​三、项目配置步骤​1

Web技术与Nginx网站环境部署教程

《Web技术与Nginx网站环境部署教程》:本文主要介绍Web技术与Nginx网站环境部署教程,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐教... 目录一、Web基础1.域名系统DNS2.Hosts文件3.DNS4.域名注册二.网页与html1.网页概述2.HTML概述3.

Python使用Reflex构建现代Web应用的完全指南

《Python使用Reflex构建现代Web应用的完全指南》这篇文章为大家深入介绍了Reflex框架的设计理念,技术特性,项目结构,核心API,实际开发流程以及与其他框架的对比和部署建议,感兴趣的小伙... 目录什么是 ReFlex?为什么选择 Reflex?安装与环境配置构建你的第一个应用核心概念解析组件

Nginx使用Keepalived部署web集群(高可用高性能负载均衡)实战案例

《Nginx使用Keepalived部署web集群(高可用高性能负载均衡)实战案例》本文介绍Nginx+Keepalived实现Web集群高可用负载均衡的部署与测试,涵盖架构设计、环境配置、健康检查、... 目录前言一、架构设计二、环境准备三、案例部署配置 前端 Keepalived配置 前端 Nginx

Spring Boot项目打包和运行的操作方法

《SpringBoot项目打包和运行的操作方法》SpringBoot应用内嵌了Web服务器,所以基于SpringBoot开发的web应用也可以独立运行,无须部署到其他Web服务器中,下面以打包dem... 目录一、打包为JAR包并运行1.打包为可执行的 JAR 包2.运行 JAR 包二、打包为WAR包并运行

ubuntu如何部署Dify以及安装Docker? Dify安装部署指南

《ubuntu如何部署Dify以及安装Docker?Dify安装部署指南》Dify是一个开源的大模型应用开发平台,允许用户快速构建和部署基于大语言模型的应用,ubuntu如何部署Dify呢?详细请... Dify是个不错的开源LLM应用开发平台,提供从 Agent 构建到 AI workflow 编排、RA

ubuntu16.04如何部署dify? 在Linux上安装部署Dify的技巧

《ubuntu16.04如何部署dify?在Linux上安装部署Dify的技巧》随着云计算和容器技术的快速发展,Docker已经成为现代软件开发和部署的重要工具之一,Dify作为一款优秀的云原生应用... Dify 是一个基于 docker 的工作流管理工具,旨在简化机器学习和数据科学领域的多步骤工作流。它