big-data-europe / docker-hive

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how can i post text file to HDFS

baisui1981 opened this issue · comments

when i was post text to HDFS by java client from local:

try (TISFSDataOutputStream outputStream = fileSystem.create(p, true)) {
            org.apache.commons.io.IOUtils.write(IOUtils.loadResourceFromClasspath(DataXHdfsWriter.class
                    , "hdfs-datax-writer-assert-without-option-val.json"), outputStream, TisUTF8.get());
        }

however throw an exception form server side:

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/mozhenghua/com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory@29d80d2b/test/test could only be replicated to 0 nodes instead of minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1628)

have export an port 8082, on namenode :

    ports:
      - "50070:50070"
      - "8020:8020"

and I have found a historical issue #15, it seems that has been fixed. is there anyone can help me, thanks