PySparkRuntimeError: [JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number

Question

I'm trying to run pyspark code using visual studio code 2, on my MacBook M1 Pro with SONOMA OS and I'm getting the error:

PySparkRuntimeError: [JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number.

I found several references saying that I should make changes to my .zshenv file and include the lines below:

if which java > /dev/null; then export JAVA_HOME=$(/usr/libexec/java_home); fi

if which pyspark > /dev/null; then
   export SPARK_HOME="/usr/local/Cellar/apache-spark/2.1.0/libexec/"
   export PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/build:$PYTHONPATH
   export PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.10.4-src.zip:$PYTHONPATH
fi

export JAVA_HOME=$(/usr/libexec/java_home -v21)
export PYSPARK_SUBMIT_ARGS="--master local[3] pyspark-shell"

but I still have errors. Does anyone have any different ideas on how to resolve this?

run pyspark code on mac