[Bug]: java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerAdapter

[leiandpan]

What happened?

在使用HDFSReader时，出现java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerAdapter报错。
我不太明白Java，是打包的问题吗。

Version

4.1.3 (Default)

OS Type

Linux (Default)

Java JDK Version

Oracle JDK 1.8.0

Relevant log output

{
	"setting":{
		"speed":{
			"channel":2,
			"bytes":-1
		}
	},
	"content":{
		"reader":{
			"name":"hdfsreader",
			"parameter":{
				"path":"/test1/*",
				"defaultFS":"hdfs://localhost:9000",
				"column":[
					{
						"index":0,
						"type":"string"
					}
				],
				"fileType":"parquet",
				"encoding":"UTF-8",
				"fieldDelimiter":","
			}
		},
		"writer":{
			"name":"streamwriter",
			"parameter":{
				"print":true
			}
		}
	}
}

2024-03-13 18:36:00.914 [        main] INFO  JobContainer         - The jobContainer begins to process the job.
2024-03-13 18:36:01.000 [       job-0] INFO  HdfsReader$Job       - init() begin...
2024-03-13 18:36:01.146 [       job-0] INFO  HdfsReader$Job       - init() ok and end...
2024-03-13 18:36:01.148 [       job-0] INFO  JobContainer         - The Reader.Job [hdfsreader] perform prepare work .
2024-03-13 18:36:01.149 [       job-0] INFO  HdfsReader$Job       - prepare(), start to getAllFiles...
2024-03-13 18:36:01.149 [       job-0] INFO  DFSUtil              - get HDFS all files in path = [/test1/*]
java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerAdapter
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.commons.logging.LogFactory.createFactory(LogFactory.java:419)
	at org.apache.commons.logging.LogFactory.lambda$newFactory$3(LogFactory.java:1432)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.commons.logging.LogFactory.newFactory(LogFactory.java:1431)
	at org.apache.commons.logging.LogFactory.getFactory(LogFactory.java:928)
	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:987)
	at org.apache.hadoop.fs.FileSystem.<clinit>(FileSystem.java:137)
	at com.wgzhao.addax.plugin.reader.hdfsreader.DFSUtil.getHDFSAllFiles(DFSUtil.java:208)
	at com.wgzhao.addax.plugin.reader.hdfsreader.DFSUtil.getAllFiles(DFSUtil.java:185)
	at com.wgzhao.addax.plugin.reader.hdfsreader.HdfsReader$Job.prepare(HdfsReader.java:185)
	at com.wgzhao.addax.core.job.JobContainer.prepareJobReader(JobContainer.java:619)
	at com.wgzhao.addax.core.job.JobContainer.prepare(JobContainer.java:270)
	at com.wgzhao.addax.core.job.JobContainer.start(JobContainer.java:124)
	at com.wgzhao.addax.core.Engine.start(Engine.java:61)
	at com.wgzhao.addax.core.Engine.entry(Engine.java:112)
	at com.wgzhao.addax.core.Engine.main(Engine.java:139)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.spi.LoggerAdapter
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 18 more
2024-03-13 18:36:01.162 [       job-0] ERROR Engine               - java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerAdapter

[wgzhao]

可以尝试下载 log4j-api-2.17.1.jar ，然后放到 plugin/reader/hfdsreader/libs 目录下，然后再执行看看

[leiandpan]

放过去的时候提示文件已存在了，覆盖之后重跑还是这个报错

[wgzhao]

把 plugin/reader/hfdsreader/libs 下面 log4j 开头的几个文件拷贝 $ADDAX_HOME/lib 目录下应该可以解决这个问题

[wgzhao]

从 4.1.1 版本一直追溯到 4.1.4 版本，发现是在 4.1.4 中对 commons-logging 从 1.2.0 升级到了1.3.0 就出现了这个问题。
简单的办法是删除 $ADDAX_HOME/lib/comons-logging-1.3.0.jar 文件即可。

[wgzhao]

Fixed by 6c7ea46