Spark Encryption and Data Integrity in Transit¶
Table of Contents
Introduction¶
To use Oracle Native Network Encryption and Oracle Network Data Integrity for all connections from Spark the following should all be set.
Configure Spark Options¶
Add the following entries to $SPARK_HOME/conf/spark-defaults.conf:
spark.driver.extraJavaOptions=-Doracle.net.crypto_checksum_client=REQUESTED -Doracle.net.crypto_checksum_types_client=SHA1 -Doracle.net.encryption_client=REQUIRED -Doracle.net.encryption_types_client=AES256
spark.executor.extraJavaOptions=-Doracle.net.crypto_checksum_client=REQUESTED -Doracle.net.crypto_checksum_types_client=SHA1 -Doracle.net.encryption_client=REQUIRED -Doracle.net.encryption_types_client=AES256
Note
If spark.driver.extraJavaOptions or spark.executor.extraJavaOptions are already present in the spark-defaults.conf file then append the values above rather than overwriting the existing entries.
Restart Spark¶
For the changes to take effect Spark must be restarted. The examples below show how to restart Spark Standalone.
To stop Spark Standalone issue the following commands:
$ $SPARK_HOME/sbin/stop-all.sh
$ $SPARK_HOME/sbin/stop-history-server.sh
To start Spark Standalone manually, issue the following commands:
$ $SPARK_HOME/sbin/start-all.sh
$ $SPARK_HOME/sbin/start-history-server.sh