hcoona / WebHdfs.Extensions.FileProviders

This is a file provider for HDFS through WebHDFS protocol for Microsoft.Extensions.FileProviders

WebHdfs File Provider

This is a file provider for HDFS through WebHDFS protocol for Microsoft.Extensions.FileProviders.

Moved to https://github.com/hcoona/OneDotNet/tree/1246c63/MicrosoftExtensions/FileProviders/WebHdfs

Getting Started

This project support .NET framework 4.5+ & .NET standard 1.1+

Installing

# Add the WebHdfs.Extensions.FileProviders package to the default project
Install-Package WebHdfs.Extensions.FileProviders

# Add the WebHdfs.Extensions.FileProviders package to a project named UtilitiesLib that is not the default
Install-Package WebHdfs.Extensions.FileProviders -ProjectName UtilitiesLib

Samples

var fileFileInfo = new WebHdfsFileInfo(nameNodeUri, settings.FilePath);
Assert.True(fileFileInfo.Exists);
Assert.True(fileFileInfo.Length > 0);
Assert.True(fileFileInfo.LastModified > DateTimeOffset.Parse("2010/1/1"));
Assert.False(fileFileInfo.IsDirectory);

string content;
using (var reader = new StreamReader(fileFileInfo.CreateReadStream())) {
    content = reader.ReadToEnd();
}

Build

msbuild.exe

dotnet build

Todo

1. Support OAuth

2. Support watch globbing files

3. Add CI for both Windows & Linux

License

This project is licensed under the GPL3.0 License — see the LICENSE.txt file for details