The following deployment patterns are supported for WSO2 ML.
...
| Value | Description |
|---|---|
| local | Runs Spark locally with one worker thread. There will be no multiple threads running in parallel. |
| local[k] | Runs Spark locally with k number of threads. (K is ideally the number of cores in your machine). |
| local[*] | Runs Spark locally with a number of worker threads that equals the number of logical cores in your machine. |
With external Spark clusterAnchor External Spark Cluster External Spark Cluster
| External Spark Cluster | |
| External Spark Cluster |
By default, WSO2 ML runs with an inbuilt Apache Spark instance. However, when working with big data, you can handle those large data sets in a distributed environment through WSO2 ML. You can carry out data pre-processing and model building processes on an Apache Spark cluster to share the workload between the nodes of the cluster. Using a Spark cluster optimizes the performance and reduces the time consumed to build and train a machine learning model for a large data set.
Follow the steps below to run the ML jobs by connecting WSO2 ML to an external Apache Spark cluster.
...