Scatter-Gather

This section explains, through an example scenario, how the Scatter-Gather EIP can be implemented using WSO2 ESB. The following topics are covered:

Introduction to Scatter-Gather

The Scatter-Gather EIP maintains the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply back. For more information, refer to http://www.eaipatterns.com/BroadcastAggregate.html.

Figure 1: Scatter-Gather EIP

Example scenario

This example scenario demonstrates an implementation of Scatter-Gather EIP, which broadcasts a message to multiple recipients, using WSO2 ESB. The ESB uses an Aggregator to collect the responses and distill them into a single response message.

We use a sample Stock Quote service as the service provided by the vendors. In this scenario, you send a quote request to 3 vendors, get quotes for certain items, and return the best quote to the client. We assume that all 3 vendors implement the same service contract. If the service contracts are different, appropriate transformations must be done before sending the requests to the vendor services, and then transform the responses back. The XSLT mediator can be used to do that.

The diagram below depicts how to simulate the example scenario using the WSO2 ESB.

Figure 2: Scatter-Gather Example Scenario

Before digging into implementation details, let's take a look at the relationship between the example scenario and the Scatter-Gather EIP by comparing their core components.

Environment setup

Start the ESB server and log into its management console UI (https://localhost:9443/carbon). In the management console, navigate to Main Menu, click Services, then Add and then click Proxy Service. Next, copy and paste the following configuration, which helps you explore the example scenario, to a new Pass Through Proxy Service named ScatterGatherProxy. 

<proxy xmlns="http://ws.apache.org/ns/synapse" name="ScatterGatherProxy" transports="https http" startOnLoad="true" trace="disable">  
    <description/>  
    <target>  
        <inSequence>  
            <clone>  
                <target>  
                    <endpoint name="vendorA">  
                        <address uri="http://localhost:9000/services/SimpleStockQuoteService/"/>  
                    </endpoint>  
                </target>  
                <target>  
                    <endpoint name="vendorB">  
                        <address uri="http://localhost:9001/services/SimpleStockQuoteService/"/>  
                    </endpoint>  
                </target>  
                <target>  
                    <endpoint name="vendorC">  
                        <address uri="http://localhost:9002/services/SimpleStockQuoteService/"/>  
                    </endpoint>  
                </target>  
            </clone>  
        </inSequence>  
        <outSequence>  
            <log level="full"/>  
            <aggregate>  
                <completeCondition>  
                    <messageCount min="3"/>  
                </completeCondition>  
                <onComplete xmlns:m1="http://services.samples/xsd" xmlns:m0="http://services.samples" expression="//m0:return">  
                    <enrich>  
                        <source xmlns:m1="http://services.samples/xsd" clone="true" xpath="//m0:return[not(preceding-sibling::m0:return/m1:last &lt;= m1:last) and not(following-sibling::m0:return/m1:last &lt; m1:last)]"/>  
                        <target type="body"/>  
                    </enrich>  
                    <send/>  
                </onComplete>  
            </aggregate>  
        </outSequence>  
    </target>  
</proxy>  

Simulating the sample scenario

1. Use a client like SOAP UI to send the following request to the ScatterGatherProxy service.

   <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://services.samples">  
      <soapenv:Header/>  
      <soapenv:Body>  
         <ser:getSimpleQuote>  
            <ser:symbol>Foo</ser:symbol>  
         </ser:getSimpleQuote>  
      </soapenv:Body>  
   </soapenv:Envelope>  

2. Since log mediator is enabled inside the outSequence, there will be 3 responses from the 3 vendors. The logs will be similar to the following:
   

   

3. In SOAP UI, you will get the response from the vendor providing the best quote as follows:
   

   

4. Comparison of the logged response messages with the response received by the client shows that the ScatterGatherProxy service returns the best quote to the client (soapUI).

How the implementation works

Let's investigate the elements of the ESB configuration in detail. The line numbers below are mapped with the ESB configuration illustrated in step 3 above.

• clone [line 6 in ESB config] - The Clone mediator is similar to the Splitter EIP. It clones the incoming request and passes the requests parallelly to several endpoints.   
• aggregate [line 26 in ESB config] - The Aggregate mediator aggregates response messages for requests made by the Iterate or Clone mediators. The completion condition specifies the minimum or maximum number of messages to be collected. When all messages are aggregated, the sequence inside onComplete will be run. 
• In the inSequence of the ScatterGatherProxy service, we are simply cloning, or making 3 copies of the request to be sent by the client using the Clone mediator. Those requests are then forwarded to the 3 vendor services (SimpleStockeQuoteService). The responses to those 3 requests are received at the outSequence.
• All received responses are logged before aggregating, using the Aggregate mediator of the ESB. The onComplete sequence of the Aggregate mediator is called once all 3 responses are received or the specified completion condition is met. The responses are aggregated based on the value of the return element in the response.
• Next, the Enrich mediator is used to extract the response, which contains the best quote. The following XPath 1.0 expression is used for this purpose:
  //m0:return[not(preceding-sibling::m0:return/m1:last <= m1:last) and not(following-sibling::m0:return/m1:last < m1:last)]

In essence, this message instructs to pick the response, which has the lowest last value. Using XPath 2.0 min function, the complexity of the above command can be reduced to a greater extent. However, XPath 1.0 is the default supported by WSO2 ESB at the moment.