...
Parameter | Value |
---|---|
spark.eventLog.enabled | This should be set to true . |
spark.eventLog.dir | Set <DAS_HOME>/work as the location to save the logs. |
Info |
---|
When you enable logging in a multi node DAS cluster, the DAS nodes can consume too much CPU and sometimes run out of memory as a result of too many log directories being created in the |
Monitoring via Spark UIs
Note |
---|
Before you enable Spark UIs, note that they are only supported with HTTP, and the Environment tab of Application UI may display security sensitive information such as the keystore password depending on your Spark Configuration. If you do not want to expose such information, you can disable Spark UIs as explained in Disabling Spark UI. |
...