Question: Planned support for mono?

Question

Question: Planned support for mono?

alvinauyoung opened this issue 10 years ago · comments

Hi Naiad team,

I'm a bit late to the party, but after finally watching the presentation video from SOSP, I was very excited to learn that Naiad is available open source. I'm excited to try it out.

My main development environment is not a windows machine, but I noticed there is a script called "build_mono.sh". I was wondering if you've had any success building it? It might be something else I am doing wrong, but when I try to build using this script, it fails because it cannot find "EventTask", which should be under System.Diagnostics.Tracing. I'm using xbuild version 12.0.0, and Mono 3.10.0.0. As further evidence, the xamarin API doesn't seem to have this class.

Finally, in light of the somewhat recent and very shocking announcement from Microsoft about MSR-SV, do you have any idea what impact this might have on future support for Naiad? I'm very sorry to all of the folks who were impacted.

Thanks,
Alvin

Michael Isard · Answer 1 · Sat Oct 25 2014 00:24:28 GMT+0800 (China Standard Time)

Hi Alvin,

The quick answer, to get you started, is that if you clone the Release_0.4 branch things should work fine on (recent versions of) Mono. I just pushed Release_0.5 a few days ago and I’m afraid I didn’t test the Mono build; I can probably get it fixed and pushed shortly. (I think that just commenting out some tracing code for the non-Windows build should suffice.)

You are right that the closure of MSR-SV will probably have some impact on Naiad development, since the lead developers aren’t working at Microsoft any more. There will be at least one more release of feature branches with unstable code that either was already under development, or I am working on now, but I don’t know if or when there will be another stable version coming out of Microsoft. Of course one never knows if an external developer might clone the repository and build on it :)

I hope you will play with the system despite the uncertainty. The released version is pretty stable, and works well, and I’m happy to try to answer any questions.

Michael.

From: alvinauyoung [mailto:notifications@github.com]
Sent: October 23, 2014 7:31 PM
To: MicrosoftResearch/Naiad
Subject: [Naiad] Question: Planned support for mono? (#16)

Hi Naiad team,

I'm a bit late to the party, but after finally watching the presentation video from SOSP, I was very excited to learn that Naiad is available open source. I'm excited to try it out.

My main development environment is not a windows machine, but I noticed there is a script called "build_mono.sh". I was wondering if you've had any success building it? It might be something else I am doing wrong, but when I try to build using this script, it fails because it cannot find "EventTask", which should be under System.Diagnostics.Tracing. I'm using xbuild version 12.0.0, and Mono 3.10.0.0. As further evidence, the xamarin API doesn't seem to have this class.

Finally, in light of the somewhat recent and very shocking announcement from Microsoft about MSR-SV, do you have any idea what impact this might have on future support for Naiad? I'm very sorry to all of the folks who were impacted.

Thanks,
Alvin

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/16.

Alvin AuYoung · Answer 2 · Sat Oct 25 2014 06:25:27 GMT+0800 (China Standard Time)

Thanks for the quick response Michael: I've grabbed version 0.4.2, and I can see that the simple base examples work with my Mono installation. I do intend to continue trying it out despite the uncertainty, and thanks for keeping this project going, for at least one more release. :)

Alvin

Arjun Narayan · Answer 3 · Wed Dec 03 2014 07:17:02 GMT+0800 (China Standard Time)

Hi MIchael,

I'm still getting build errors on mono using release_0.5. Release 0.4 works fine, as mentioned above.
Here's the error I get:

Errors:

/root/Naiad/Naiad.sln (default targets) ->
(Build target) ->

    /root/Naiad/Naiad.sln: error : Could not find the project file '/root/Naiad/frameworks/Storage/Storage.csproj'
    /root/Naiad/Naiad.sln: error : Could not find the project file '/root/Naiad/frameworks/HdfsSupport/HdfsSupport.csproj'

/root/Naiad/Naiad.sln (default targets) ->
(Build target) ->
/root/Naiad/Frameworks/WebHdfsSupport/WebHdfsSupport.csproj (default targets) ->
/root/Naiad/.nuget/NuGet.targets (RestorePackages target) ->

    /root/Naiad/.nuget/NuGet.targets: error : Command 'mono --runtime=v4.0.30319 "/root/Naiad/.nuget/NuGet.exe" install "/root/Naiad/Frameworks/WebHdfsSupport/packages.config" -source ""   -RequireConsent -solutionDir "/root/Naiad/"' exited with code: 1.

     6 Warning(s)
     3 Error(s)

To enable a portable replication, I created a Dockerfile to replicate this error. The Dockerfile is below. If you're not using docker, the Dockerfile pretty much just runs all commands that begin with "RUN" in a fresh Ubuntu 14.04 VM. (The reason I'm working in Docker is I'm trying to eventually get Naiad running in containerized format in a multi-machine setup with automatic failure restoration, etc.).

FROM ubuntu:14.04
# turns out Naiad wont compile on wheezy. I won't bother debugging that.
MAINTAINER arjunravinarayan@gmail.com

RUN apt-key adv --keyserver pgp.mit.edu --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
RUN echo "deb http://download.mono-project.com/repo/debian wheezy main" > /etc/apt/sources.list.d/mono-xamarin.list
RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt-get -y install curl wget
RUN apt-get -y install build-essential
RUN apt-get -y install mono-complete fsharp
RUN apt-get -y install git

WORKDIR /root/
RUN git clone -b release_0.5 https://github.com/MicrosoftResearch/Naiad.git
RUN chmod a+x /root/Naiad/build_mono.sh
WORKDIR /root/Naiad/
RUN /root/Naiad/build_mono.sh
WORKDIR /root/Naiad/Examples/bin/Debug/
RUN mkbundle --static --deps Examples.exe *.dll -o example
RUN cp /root/Naiad/Examples/bin/Debug/example /usr/bin/naiadexamples

ENTRYPOINT /usr/bin/naiadexamples wordcount

If you change the git clone line from release_0.5 to release_0.4, everything works. I'm currently working in release_0.4, and am pretty happy, but noticed that release_0.5 claims to have "numerous minor fixes", and that would be nice to have.

Michael Isard · Answer 4 · Thu Dec 04 2014 01:48:16 GMT+0800 (China Standard Time)

Sorry about this. I believe the problem is that the .sln file uses lower case ‘frameworks’ when the directory structure is upper case ‘Frameworks.’ That edit should get you going again. I am on vacation currently but will push an update when I get a chance.

Michael.

From: Arjun Ravi Narayan [mailto:notifications@github.com]
Sent: December 2, 2014 6:17 PM
To: MicrosoftResearch/Naiad
Cc: Michael Isard
Subject: Re: [Naiad] Question: Planned support for mono? (#16)

Hi MIchael,

I'm still getting build errors on mono using release_0.5. Release 0.4 works fine, as mentioned above.
Here's the error I get:

Errors:

/root/Naiad/Naiad.sln (default targets) ->

(Build target) ->

/root/Naiad/Naiad.sln: error : Could not find the project file '/root/Naiad/frameworks/Storage/Storage.csproj'

/root/Naiad/Naiad.sln: error : Could not find the project file '/root/Naiad/frameworks/HdfsSupport/HdfsSupport.csproj'

/root/Naiad/Naiad.sln (default targets) ->

(Build target) ->

/root/Naiad/Frameworks/WebHdfsSupport/WebHdfsSupport.csproj (default targets) ->

/root/Naiad/.nuget/NuGet.targets (RestorePackages target) ->

/root/Naiad/.nuget/NuGet.targets: error : Command 'mono --runtime=v4.0.30319 "/root/Naiad/.nuget/NuGet.exe" install "/root/Naiad/Frameworks/WebHdfsSupport/packages.config" -source ""   -RequireConsent -solutionDir "/root/Naiad/"' exited with code: 1.



 6 Warning(s)

 3 Error(s)

To enable a portable replication, I created a Dockerfile to replicate this error. The Dockerfile is below. If you're not using docker, the Dockerfile pretty much just runs all commands that begin with "RUN" in a fresh Ubuntu 14.04 VM. (The reason I'm working in Docker is I'm trying to eventually get Naiad running in containerized format in a multi-machine setup with automatic failure restoration, etc.).

FROM ubuntu:14.04

turns out Naiad wont compile on wheezy. I won't bother debugging that.

MAINTAINER arjunravinarayan@gmail.commailto:arjunravinarayan@gmail.com

RUN apt-key adv --keyserver pgp.mit.edu --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF

RUN echo "deb http://download.mono-project.com/repo/debian wheezy main" > /etc/apt/sources.list.d/mono-xamarin.list

RUN apt-get -y update

RUN apt-get -y upgrade

RUN apt-get -y install curl wget

RUN apt-get -y install build-essential

RUN apt-get -y install mono-complete fsharp

RUN apt-get -y install git

WORKDIR /root/

RUN git clone -b release_0.5 https://github.com/MicrosoftResearch/Naiad.git

RUN chmod a+x /root/Naiad/build_mono.sh

WORKDIR /root/Naiad/

RUN /root/Naiad/build_mono.sh

WORKDIR /root/Naiad/Examples/bin/Debug/

RUN mkbundle --static --deps Examples.exe *.dll -o example

RUN cp /root/Naiad/Examples/bin/Debug/example /usr/bin/naiadexamples

ENTRYPOINT /usr/bin/naiadexamples wordcount

If you change the git clone line from release_0.5 to release_0.4, everything works. I'm currently working in release_0.4, and am pretty happy, but noticed that release_0.5 claims to have "numerous minor fixes", and that would be nice to have.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-65324046.

Arjun Narayan · Answer 5 · Thu Dec 04 2014 02:44:08 GMT+0800 (China Standard Time)

Hi Michael,

That fixes the errors above, but a new one crops up:

/root/Naiad/Naiad.sln (default targets) ->
(Build target) ->
/root/Naiad/Frameworks/WebHdfsSupport/WebHdfsSupport.csproj (default targets) ->
/usr/lib/mono/4.5/Microsoft.CSharp.targets (CoreCompile target) ->

    WebHdfs.cs(29,26): error CS0234: The type or namespace name `WebHdfs' does not exist in the namespace `Microsoft.Research.Peloponnese'. Are you missing an assembly reference?

I apologize if these are fairly basic errors. I feel like I should be able to debug them myself, but I have no familiarity with the .NET stack.

Michael Isard · Answer 6 · Thu Dec 04 2014 04:07:49 GMT+0800 (China Standard Time)

No apologies needed: I'm sorry I didn't debug the mono build. The issue here is that the WebHdfs and Hdfs frameworks depend on the Peloponnese libraries. Visual Studio would automatically download those for you from NuGet, but the Mono build doesn't do that. If you aren't planning on reading and writing from HDFS, then the easiest thing to do is just comment out those frameworks from the build, and everything else will hopefully just work ok. If you want to use HDFS, that will require a bit more work. You will have to build Peloponnese from source (it is available at https://github.com/MicrosoftResearch/Peloponnese). However not all of Peloponnese will build on Mono since some of it uses C++/Java interop via managed C++. The WebHdfs library and its dependencies should build, at which point you could also build the Naiad WebHdfs framework, however the Hdfs framework (which uses Java interop) is not going to build without a bunch of work.

Arjun Narayan · Answer 7 · Thu Dec 04 2014 05:15:18 GMT+0800 (China Standard Time)

Thanks very much! I can confirm that removing HDFS, WebHDFS, and Storage fixes the build. A little over the top perhaps, but I wasn't using them. I'm now able to build release 0.5 with no issues.

Just a final tip for anyone building Naiad in mono in the future, you need to include the line
mozroots --import --machine --sync
before you build release 0.5, as NuGet complains about unverified signatures otherwise (and this gets buried in other error messages).

Gábor E. Gévay · Answer 8 · Thu Jul 26 2018 20:08:18 GMT+0800 (China Standard Time)

Hi!

Just for future reference to anyone who lands on this page:

Actually, Mono does download Peloponnese from NuGet, and there is only a case sensitivity issue. After renaming

Microsoft.Research.Peloponnese.WebHDfs.dll

to

Microsoft.Research.Peloponnese.WebHdfs.dll

the build succeeds. (And I also needed to do the abovementioned framework -> Framework fix in the sln file.) (I'm on Ubuntu 16.04 and installed mono by apt install mono-complete.)