Spark Encryption and Data Integrity in Transit

Introduction

To use Oracle Native Network Encryption and Oracle Network Data Integrity for all connections from Spark the following should all be set.

Configure Spark Options

Add the following entries to $SPARK_HOME/conf/spark-defaults.conf:

spark.driver.extraJavaOptions=-Doracle.net.crypto_checksum_client=REQUESTED -Doracle.net.crypto_checksum_types_client=SHA1 -Doracle.net.encryption_client=REQUIRED -Doracle.net.encryption_types_client=AES256
spark.executor.extraJavaOptions=-Doracle.net.crypto_checksum_client=REQUESTED -Doracle.net.crypto_checksum_types_client=SHA1 -Doracle.net.encryption_client=REQUIRED -Doracle.net.encryption_types_client=AES256

Note

If spark.driver.extraJavaOptions or spark.executor.extraJavaOptions are already present in the spark-defaults.conf file then append the values above rather than overwriting the existing entries.

Restart Spark

For the changes to take effect Spark must be restarted. The examples below show how to restart Spark Standalone.

To stop Spark Standalone issue the following commands:

$ $SPARK_HOME/sbin/stop-all.sh
$ $SPARK_HOME/sbin/stop-history-server.sh

To start Spark Standalone manually, issue the following commands:

$ $SPARK_HOME/sbin/start-all.sh
$ $SPARK_HOME/sbin/start-history-server.sh

Documentation Feedback

Send feedback on this documentation to: feedback@gluent.com