rotatelogfileswithprefix: command not found
jinhshi opened this issue · comments
你好,感谢贡献。
另外,README 中 (2) 将上述生成的jar文件copy至cloudera /parcel-repo子目录下, 这个应该是放在cloudera /csd下吧。
除了上边那个,其余按你的步骤来,是可以将flink集成到cdh中。
但是在启动的时候遇到问题:
rotatelogfileswithprefix: command not found
ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/1321-flink_on_yarn-FLINK_YARN
查资料看到很多人说是 flink-shaded-hadoop包的问题,尝试了几个版本:flink-shaded-hadoop-2-uber-2.7.5-10.0.jar, flink-shaded-hadoop-2-uber-3.0.0-cdh6.3.0-7.0.jar, flink-shaded-hadoop-3-uber-3.1.1.7.2.9.0-173-9.0.jar 还是相同的问题,请问有没有相关的解决方法 ?
cdh:6.3.2
flink:1.13.1
maven:3.6.3
java:1.8
flink-yarn.sh 这个脚本有问题,没这个方法
/opt/cloudera/parcels/FLINK/lib/flink/bin/config.sh:391
加入
rotateLogFilesWithPrefix() {
dir=$1
prefix=$2
while read -r log ; do
rotateLogFile "$log"
# find distinct set of log file names, ignoring the rotation number (trailing dot and digit)
done < <(find "$dir" ! -type d -path "${prefix}*" | sed s/\.[0-9][0-9]*$// | sort | uniq)
}
# 旋转日志文件
rotateLogFile() {
log=$1;
num=$MAX_LOG_FILE_NUMBER
if [ -f "$log" -a "$num" -gt 0 ]; then
while [ $num -gt 1 ]; do
prev=`expr $num - 1`
[ -f "$log.$prev" ] && mv "$log.$prev" "$log.$num"
num=$prev
done
mv "$log" "$log.$num";
fi
}
最后出现 rotateLogFilesWithPrefix 这个函数的版本是 1.11.6,后面的版本就把这个函数删了,这个库应该是不支持 1.12.0 以及之后的版本的。
https://github.com/apache/flink/blob/release-1.11.6/flink-dist/src/main/flink-bin/bin/config.sh#L367
# Auxilliary functions for log file rotation
rotateLogFilesWithPrefix() {
dir=$1
prefix=$2
while read -r log ; do
rotateLogFile "$log"
# find distinct set of log file names, ignoring the rotation number (trailing dot and digit)
done < <(find "$dir" ! -type d -path "${prefix}*" | sed s/\.[0-9][0-9]*$// | sort | uniq)
}
rotateLogFile() {
log=$1;
num=$MAX_LOG_FILE_NUMBER
if [ -f "$log" -a "$num" -gt 0 ]; then
while [ $num -gt 1 ]; do
prev=`expr $num - 1`
[ -f "$log.$prev" ] && mv "$log.$prev" "$log.$num"
num=$prev
done
mv "$log" "$log.$num";
fi
}
所以应该咋解决呢?
上面不是写了解决方案了吗