pyspark-mocked-s3.py won't start- "Java gateway process exited before sending its post number"

[Roni Yosofov]

doesn't seem like PySpark is able to download jackson-annotations or jackson-core.

I tried adding them to here:
os.environ[ "PYSPARK_SUBMIT_ARGS" ] = '--packages "com.amazonaws:aws-java-sdk:1.7.4,org.apache.hadoop:hadoop-aws:2.7.3" pyspark-shell'

but no help.

This is the error stack:

:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS Exception in thread "main" java.lang.RuntimeException: [download failed: com.fasterxml.jackson.core#jackson-annotations;2.2.3!jackson-annotations.jar, download failed : com.fasterxml.jackson.core#jackson-core;2.2.3!jackson-core.jar] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1306) at org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54) at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:315) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Traceback (most recent call last): File "testing_mocked_s3.py", line 29, in <module> spark = SparkSession.builder.getOrCreate() File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\sql\session.py", line 173, in getOrCreate sc = SparkContext.getOrCreate(sparkConf) File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\context.py", line 367, in getOrCreate SparkContext(conf=conf or SparkConf()) File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\context.py", line 133, in __init__ SparkContext._ensure_initialized(self, gateway=gateway, conf=conf) File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\context.py", line 316, in _ensure_initialized SparkContext._gateway = gateway or launch_gateway(conf) File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\java_gateway.py", line 46, in launch_gateway return _launch_gateway(conf) File "D:\simon-app\appscripts\venv\lib\site-packages\pyspark\java_gateway.py", line 108, in _launch_gateway raise Exception("Java gateway process exited before sending its port number") Exception: Java gateway process exited before sending its port number