mirror-github-bspeice/kiva-dig
1
0
Fork 0
mirror of https://github.com/bspeice/kiva-dig synced 2025-07-12 03:05:13 -04:00
Code Issues Releases Wiki Activity

Add a basic requirements file

Browse Source 
And also easy script for running a Spark notebook
This commit is contained in:
Bradlee Speice
2016-10-21 18:00:06 -04:00
parent 71627e160e
commit df0132d466
3 changed files with 13 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
start_pyspark.sh Executable file
View File

@ -0,0 +1,9 @@
SPARK_DIR="spark-2.0.1-bin-hadoop2.7"
SPARK_URL="http://apache.claz.org/spark/spark-2.0.1/spark-2.0.1-bin-hadoop2.7.tgz"
if [ ! -d "$SPARK_DIR" ]; then
echo "Downloading Spark distribution..."
wget "$SPARK_URL" -O - | tar xzf -
fi
PYSPARK_DRIVER_PYTHON="jupyter" PYSPARK_DRIVER_PYTHON_OPTS="notebook" "./$SPARK_DIR/bin/pyspark"