本文主要是介绍启动Spark有时成功有时报错,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
环境:spark-2.4.8+CDH-6.3.2+scala-1.11.12
报错信息:
Application application_1669708409043_0014 failed 2 times due to AM Container for appattempt_1669708409043_0014_000002 exited with exitCode: 1Failing this attempt.Diagnostics: [2022-11-29 18:01:11.661]Exception from container-launch.Container id: container_1669708409043_0014_02_000001Exit code: 1[2022-11-29 18:01:11.663]Container exited with a non-zero exit code 1. Error file: prelaunch.err.Last 4096 bytes of prelaunch.err :Last 4096 bytes of stderr :Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher[2022-11-29 18:01:11.663]Container exited with a non-zero exit code 1. Error file: prelaunch.err.Last 4096 bytes of prelaunch.err :Last 4096 bytes of stderr :Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
原因:路径jar包没有识别读取到
解决办法:spark配置文件:spark-default.conf 中的spark.yarn.jars=路径找不到,检查hdfs或本地路径是否正确。注意:路径结尾加*。
这篇关于启动Spark有时成功有时报错的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!