rtulke / pocketcluster

PocketCluster related files.

One-Step Spark/Hadoop Installer v0.1.1

A one-step utility to setup single-node pseudo-distributed Spark/Hadoop cluster on OSX.

Download Installer

Current Packages Supported

Package	Version	Release Note
Apache Spark	1.4.1	RN 1.4.1
Apache Hadoop	2.4.0	RN 2.4.1

Pre-requisites

1. OSX Yosemite

This installer has only been tested on Yosemite 10.10. May work fine on El Capitan.

• OSX Yosemite

2. Java

Please use Java 8. More and more Apache projects drop supporting for old Java version.
You have two choices. Either one works just fine.

• Oracle Java 8
• Apple Java 6 2015-001

3. R (Optional)

Spark 1.4.0 comes with SparkR. In case you'd like to delve into it, install R.

• R 3.2.1

System Setup

In order for Spark/Hadoop to work in pseudo-distributed mode, you need to open SSH server just for yourself.

1. Open "System Preference"
2. Open "Sharing"
3. Check "Remote Login" for "Administators" only.

Install

Download installer

1. Unzip & execute installer.

2. Select "Install Package" from menu.
   

3. Click "Install" button.
   

4. Once completed, open "Terminal" or "iTerm"

5. To start, type start-service.sh

   • Check out Spark Console if Spark comes up online.
   • Same for Hadoop Namenode.

6. To stop, type stop-service.sh

F.A.Q

If you have a question, leave it to my blog post or tweet me.