Spark User Interface Security¶
Table of Contents
Introduction¶
This document includes the steps to secure the user interface for a Spark Standalone installation on the Gluent Node.
Authentication¶
Configure Spark Standalone¶
The following parameter in /opt/gluent/transport/spark/conf/spark-defaults.conf sets the username and password for the Spark User Interface:
Parameter |
Value |
|---|---|
|
|
The values for gluent and change_me should be replaced with chosen values.
Restart Spark Standalone¶
For the changes to take effect Spark Standalone must be restarted.
To stop Spark Standalone issue the following commands:
$ $SPARK_HOME/sbin/stop-all.sh
$ $SPARK_HOME/sbin/stop-history-server.sh
To start Spark Standalone manually, issue the following commands:
$ $SPARK_HOME/sbin/start-all.sh
$ $SPARK_HOME/sbin/start-history-server.sh
A simple HTTP user/password will now be prompted for all Spark UIs.
SSL¶
The most basic steps to configure the key stores and the trust store for a Spark Standalone deployment mode is as follows:
Generate a key pair for each node
Export the public key of the key pair to a file on each node
Import all exported public keys into a single trust store
Distribute the trust store to the cluster nodes
Generate Certificate¶
Create a self-signed certificate as follows:
$ mkdir -p /opt/gluent/spark-cert
$ cd /opt/gluent/spark-cert
$ $JAVA_HOME/bin/keytool -genkeypair -alias $(hostname -s) -storepass <password> -keystore spark-$(hostname -s).jks \
-keyalg RSA -keysize 2048 -validity 3650 \
-dname "cn=$(hostname -f)" -keypass <password>
$ $JAVA_HOME/bin/keytool -export -alias $(hostname -s) -storepass <password> -keystore spark-$(hostname -s).jks \
-file spark-$(hostname -s).cer
$ $JAVA_HOME/bin/keytool -importcert -alias $(hostname -s) -storepass <password> -keystore spark-$(hostname -s).truststore \
-file spark-$(hostname -s).cer -noprompt
The value for <password> should be replaced with a chosen value.
Configure Spark Standalone¶
Add the following parameters and values to /opt/gluent/transport/spark/conf/spark-defaults.conf:
Parameter |
Value |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
The value for <password> should be replaced with the chosen value when generating the certificate.
The value for <hostname> should be replaced with the result from hostname -s from the server.
Restart Spark Standalone¶
For the changes to take effect Spark Standalone must be restarted.
To stop Spark Standalone issue the following commands:
$ $SPARK_HOME/sbin/stop-all.sh
$ $SPARK_HOME/sbin/stop-history-server.sh
To start Spark Standalone manually, issue the following commands:
$ $SPARK_HOME/sbin/start-all.sh
$ $SPARK_HOME/sbin/start-history-server.sh
Spark UIs will now be accessible only via the HTTPS protocol.