Versions Compared

Old Version 4

changes.mady.by.user Former user

Saved on

compared with

New Version Current

changes.mady.by.user Former user

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Monitoring via Spark UIs

Note

Before you enable Spark UIs, note that they are only supported with HTTP, and the Environment tab of Application UI may display security sensitive information such as user IDs and keystore passwords. the keystore password depending on your Spark Configuration.

If you do not want to expose such information, it is recommended to disable these UIs.the following options are available:

  • Exclude the relevant Spark properties from being displayed in the Environment tab by editing your spark properties.

  • Disable the Spark UIs for your DAS deployment by setting the spark.ui.enabled property to false in the <DAS_HOME>/repository/conf/analytics/spark/spark-defaults.conf file as shown below.

    Code Block
    spark.ui.enabled false

Apache Spark provides a set of user interfaces (UI) that allow you to monitor and troubleshoot the issues in a Spark cluster. This section helps you to understand the information accessed from these UIs.

...

The following information is displayed for each Spark worker in its web UI under Running Executors.

Info

It is recommended to run only one executor per DAS worker. If you observe any memory or Spark execution time issues for that executor, you can increase the amount of memory and the number of CPU cores allocated for that executor. For more information about configuring Spark executors, see Spark Configurations - Executor configurations.

ColumnDescription
ExecutorIDThe ID of the executor to which the information applies.
CoresThe number of cores used by the executor.
StateThe current status of the executor.
MemoryThe memory used by the executor.
Job Details

This displays the following:

  • The ID of the job performed by the executor.
  • The name of the job.
  • The ID of the user who submitted the job.
LogsThis lists the IDs of logs generated for the Spark worker. To view a specific log, click on the relevant ID.

...

This tab displays detailed information about the executrors executors of the selected Spark application.

...

This tab displays detailed information about the SQL queries of the selected Spark application.


Spark issues in a production envirionment

The following are three issues that may occur when you work with Spark in a multi node DAS cluster:

Info

The following issues only occur when the DAS cluster is running in RedHat Linux environments.

  • The DAS nodes consuming too much CPU processing power.
  • DAS nodes running out of memory.
  • Too many log directories being created in the  <DAS_HOME>/work  directory.

All of the above issues can be created as a result of he symbolic link not being correctly resolved in the operating system. To address this, you are required to update the  <DAS_HOME>/bin/wso2server.sh  file with the following entry so that the  <DAS_HOME>  is exported.  

Export CARBON_HOME=<symbolic link