...
The ports that are available for all the above 3 modes are listed below.
| Description | Port number |
|---|---|
| spark.ui.port | 4040 |
| spark.history.ui.port | 18080 |
| spark.blockManager.port | 12000 |
| spark.broadcast.port | 12500 |
| spark.driver.port | 13000 |
| spark.executor.port | 13500 |
| spark.fileserver.port | 14000 |
| spark.replClassServer.port | 14500 |
Ports available for the cluster mode
The ports that are available only for the cluster mode are listed below.
| Description | Port number |
|---|---|
| spark.master.port | 7077 |
| spark.master.rest.port | 6066 |
| spark.master.webui.port | 8081 |
| spark.worker.port | 11000 |
| spark.worker.webui.port | 11500 |
In addition, one or more ports in the 1200x range need to opened for communication between WSO2 DAS nodes based on the number of cores allocated for Spark.
When WSO2 DAS uses Apache Spark for batch processing, the processing is carried out by executors. Each time a new core is added for Spark, a new port in the 1200x range is needed to run the executor created for it.
e.g., When 1 core is allocated in a clustered deployment, one executor is run in each node. In the Spark master node, the port 12000 is used to run the Spark driver. Therefore, port 12001 met be opened to run the executor. However, the executor in Spark worker node can be run on port 12000. If you add the 2nd core, you need to open port 12002 to run the 2nd executor for the Spark master.
For a complete set of information on port configurations of WSO2 DAS and related WSO2 products, see Default Ports of WSO2 Products.