JSON Support

WSO2 ESB provides support for JavaScript Object Notation (JSON) payloads in messages. The following sections describe how to work with JSON in the ESB:

JSON message builders and formatters

The ESB provides two types of message builders and formatters for JSON. The default builder and formatter keep the JSON representation intact without converting it to XML. You can access the payload content using JSON Path or XPath and convert the payload to XML at any point in the mediation flow.

• org.apache.synapse.commons.json.JsonStreamBuilder

• org.apache.synapse.commons.json.JsonStreamFormatter

If you want to convert the JSON representation to XML before the mediation flow begins, use the following builder and formatter instead. Note that some data loss can occur during the JSON to XML to JSON conversion process.

• org.apache.synapse.commons.json.JsonBuilder

• org.apache.synapse.commons.json.JsonFormatter

The builders and formatters are configured in the messageBuilders and messageFormatters sections, respectively, of the Axis2 configuration files located in the <ESB_HOME>/repository/conf/axis2 directory. Both types of JSON builders use StAXON as the underlying JSON processor.

The following builders and formatters are also included for compatibility with previous versions of the ESB:

• org.apache.axis2.json.JSONBuilder/JSONMessageFormatter

• org.apache.axis2.json.JSONStreamBuilder/JSONStreamFormatter

• org.apache.axis2.json.JSONBadgerfishOMBuilder/JSONBadgerfishMessageFormatter

If you want the ESB to handle JSON payloads that are sent using a media type other than application/json, you must register the JSON builder and formatter for that media type in the following two files at minimum (for best results, register them in all Axis2 configuration files found in the <ESB_HOME>/repository/conf/axis2 directory):

• <ESB_HOME>/repository/conf/axis2/axis2.xml
• <ESB_HOME>/repository/conf/axis2/axis2_blocking_client.xml

For example, if the media type is text/javascript, register the message builder and formatter as follows:

<messageBuilder contentType="text/javascript" 
               class="org.apache.synapse.commons.json.JsonStreamBuilder"/>

<messageFormatter contentType="text/javascript" 
                class="org.apache.synapse.commons.json.JsonStreamFormatter"/> 

XML representation of JSON payloads

When building the XML tree, JSON builders attach the converted XML infoset to a special XML element that acts as the root element of the final XML tree. If the original JSON payload is of type object, the special element is <jsonObject/>. If it is an array, the special element is <jsonArray/>. Following are examples of JSON and XML representations of various objects and arrays.

Null objects

JSON:

{"object":null}

XML:

<jsonObject>
    <object></object>
</jsonObject>

Empty objects

JSON:

{"object":{}}

XML:

<jsonObject>
   <object></object>
</jsonObject>

Empty strings

JSON:

{"object":""}

XML:

<jsonObject>
   <object></object>
</jsonObject>

Empty array

JSON:

[]

XML (JsonStreamBuilder):

<jsonArray></jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
</jsonArray>

Named arrays

JSON:

{"array":[1,2]}

XML (JsonStreamBuilder):

<jsonObject>
   <array>1</array>
   <array>2</array>
</jsonObject>

XML (JsonBuilder):

<jsonObject>
   <?xml-multiple array?>
   <array>1</array>
   <array>2</array>
</jsonObject>

JSON:

{"array":[]}

XML (JsonStreamBuilder):

<jsonObject></jsonObject>

XML (JsonBuilder):

<jsonObject>
   <?xml-multiple array?>
</jsonObject>

Anonymous arrays

JSON:

[1,2]

XML (JsonStreamBuilder):

<jsonArray>
   <jsonElement>1</jsonElement>
   <jsonElement>2</jsonElement>
</jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
   <jsonElement>1</jsonElement>
   <jsonElement>2</jsonElement>
</jsonArray>

JSON:

[1, []]

XML (JsonStreamBuilder):

<jsonArray>
   <jsonElement>1</jsonElement>
   <jsonElement>
       <jsonArray></jsonArray>
   </jsonElement>
</jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
   <jsonElement>1</jsonElement>
   <jsonElement>
       <jsonArray>
           <?xml-multiple jsonElement?>
       </jsonArray>
   </jsonElement>
</jsonArray>

XML processing instructions (PIs)

Note the addition of xml-multiple processing instructions to the XML payloads whose JSON representations contain arrays. JsonBuilder (via StAXON) adds these instructions to the XML payload that it builds during the JSON to XML conversion so that during the XML to JSON conversion, JsonFormatter can reconstruct the arrays that are present in the original JSON payload. JsonFormatter interprets the elements immediately following a processing instruction to construct an array.

Special characters

When building XML elements, the ESB handles the ‘$’ character and digits in a special manner when they appear as the first character of a JSON key. Following are examples of two such occurrences. Note the addition of the _JsonReader_PS_ and _JsonReader_PD_ prefixes in place of the ‘$’ and digit characters, respectively.

JSON:

{"$key":1234}

XML:

<jsonObject>
   <_JsonReader_PS_key>1234</_JsonReader_PS_key>
</jsonObject>

JSON:

{"32X32":"image_32x32.png"}

XML:

<jsonObject>
   <_JsonReader_PD_32X32>image_32x32.png</_JsonReader_PD_32X32>
</jsonObject>

Converting a payload between XML and JSON

To convert an XML payload to JSON, set the messageType property to application/json in the axis2 scope before sending message to an endpoint. Similarly, to convert a JSON payload to XML, set the messageType property to application/xml or text/xml. For example:

<proxy xmlns="http://ws.apache.org/ns/synapse"
      name="tojson"
      transports="https,http"
      statistics="disable"
      trace="disable"
      startOnLoad="true">
  <target>
     <inSequence>
        <property name="messageType" value="application/json" scope="axis2"/>
        <respond/>
     </inSequence>
  </target>
  <description/>
</proxy>

Use the following command to invoke this proxy service:

curl -v -X POST -H "Content-Type:application/xml" -d@request1.xml "http://localhost:8280/services/tojson"

If the request payload is as follows:

<coordinates>
   <location>
       <name>Bermuda Triangle</name>
       <n>25.0000</n>
       <w>71.0000</w>
   </location>
   <location>
       <name>Eiffel Tower</name>
       <n>48.8582</n>
       <e>2.2945</e>
   </location>
</coordinates>

The response payload will look like this:

{
  "coordinates":{
     "location":[
        {
           "name":"Bermuda Triangle",
           "n":25.0000,
           "w":71.0000
        },
        {
           "name":"Eiffel Tower",
           "n":48.8582,
           "e":2.2945
        }
     ]
  }
}

Note that we have used the Property mediator to mark the outgoing payload to be formatted as JSON:

<property name="messageType" value="application/json" scope="axis2"/>

Accessing content from JSON payloads

There are two ways to access the content of a JSON payload within the ESB.

• JSONPath expressions (with json-eval() method)

• XPath expressions

JSONPath allows you to access fields of JSON payloads with faster results and less processing overhead. Although it is possible to evaluate XPath expressions on JSON payloads by assuming the XML representation of the JSON payload, we recommend that you use JSONPath to query JSON payloads. It is also possible to evaluate both JSONPath and XPath expressions on a payload (XML/JSON) at the same time.

You can use JSON path expressions with following mediators:

<log>
    <property name="location" 
              expression="json-eval($.coordinates.location[0].name)"/>
</log>

<property name="location" 
              expression="json-eval($.coordinates.location[0].name)"/>

<payloadFactory media-type="json">
    <format>{"RESPONSE":"$1"}</format>
    <args>
        <arg evaluator="json" expression="$.coordinates.location[0].name"/>
    </args>
</payloadFactory>

<switch source="json-eval($.coordinates.location[0].name)">

<filter source="json-eval($.coordinates.location[0].name)" 
        regex="Eiffel.*">

JSON path syntax

Suppose we have the following payload:

{ 
  "id": 12345,
  "id_str": "12345",
  "array": [ 1, 2, [ [], [{"inner_id": 6789}] ] ],
  "name": null,
  "object": {},
  "$schema_location": "unknown",
  "12X12": "image12x12.png"
}

The following table summarizes sample JSONPath expressions and their outputs:

$.

{ "id":12345, "id_str":"12345", "array":[1, 2, [[],[{"inner_id":6789}]]], "name":null, "object":{}, "$schema_location":"unknown", "12X12":"image12x12.png"}

$.id

12345

$.name

null

$.object

{}

$.['$schema_location']

unknown

$.12X12

image12x12.png

$.array

[1, 2, [[],[{"inner_id":6789}]]]

$.array[2][1][0].inner_id

6789

You can learn more about JSONPath syntax here.

Logging JSON payloads

To log JSON payloads as JSON, use the Log mediator as shown below. The json-eval() method returns the java.lang.String representation of the existing JSON payload.

<log>
    <property name="JSON-Payload" expression="json-eval($.)"/>
</log>

To log JSON payloads as XML, use the Log mediator as shown below:

<log level="full"/>

For more information on logging, see Troubleshooting, debugging, and logging below.

Constructing and transforming JSON payloads

To construct and transform JSON payloads, you can use the PayloadFactory mediator or Script mediator as described in the rest of this section.

PayloadFactory mediator

The PayloadFactory mediator provides the simplest way to work with JSON payloads. Suppose we have a service that returns the following response for a search query:

{
   "geometry":{
      "location":{
         "lat":-33.867260,
         "lng":151.1958130
      }
   },
   "icon":"bar-71.png",
   "id":"7eaf7",
   "name":"Biaggio Cafe",
   "opening_hours":{
      "open_now":true
   },
   "photos":[
      {
         "height":600,
         "html_attributions":[
         ],
         "photo_reference":"CoQBegAAAI",
         "width":900
      }
   ],
   "price_level":1,
   "reference":"CnRqAAAAtz",
   "types":[
      "bar",
      "restaurant",
      "food",
      "establishment"
   ],
   "vicinity":"48 Pirrama Road, Pyrmont"
}

We can create a proxy service that consumes the above response and creates a new response containing the location name and tags associated with the location based on several fields from the above response.

<proxy xmlns="http://ws.apache.org/ns/synapse"
     name="singleresponse"
     transports="https,http"
     statistics="disable"
     trace="disable"
     startOnLoad="true">
     <target>
         <outSequence>
             <payloadFactory media-type="json">
                 <format>{
                             "location_response" : {
                                 "name" : "$1",
                                 "tags" : $2
                         }}
                 </format>
                 <args>
                     <arg evaluator="json" expression="$.name"/>
                     <arg evaluator="json" expression="$.types"/>
                 </args>
             </payloadFactory>
             <send/>
         </outSequence>
         <endpoint>
             <address uri="http://localhost:8280/location"/>
         </endpoint>
     </target>
 <description/>
</proxy>

Use the following command to invoke this service:

curl -v -X GET "http://localhost:8280/services/singleresponse"

The response payload would look like this:

{
   "location_response":{
      "name":"Biaggio Cafe",
      "tags":["bar", "restaurant", "food", "establishment"]
   }
}

Note the following aspects of the proxy service configuration:

• We use the payloadFactory mediator to construct the new JSON payload.
• The media-type attribute is set to json.
• Because JSONPath expressions are used in arguments, the json evaluators are specified.

Configuring the payload format

The <format> section of the proxy service configuration defines the format of the response. Notice that in the example above, the name and tags field values are enclosed by double quotes ("), which creates a string value in the final response. If you do not use quotes, the value that gets assigned uses the real type evaluated by the expression (boolean, number, object, array, or null).

It is also possible to instruct the PayloadFactory mediator to load a payload format definition from the registry. This approach is particularly useful when using large/complex payload formats in the definitions. To load a format from the registry, click Pick From Registry instead of Define inline when defining the PayloadFactory mediator.

For example, suppose we have saved the following text content in the registry under the location conf:/repository/esb/transform. (The resource name is “transform”.)

{
    "location_response" : {
        "name" : "$1",
        "tags" : $2
    }
}

We can now modify the definition of the PayloadFactory mediator to use this format text saved as a registry resource as the payload format. The new configuration would look as follows (note that the <format> element now uses the key attribute to point to the registry resource key):

<payloadFactory media-type="json">
  <format key="conf:/repository/esb/transform"/>
  ... 
</payloadFactory>

Script mediator

The Script mediator in JavaScript is useful when you need to create payloads that have recurring structures such as arrays of objects. The Script mediator defines the following important methods that can be used to manipulate payloads in many different ways:

• getPayloadJSON
• setPayloadJSON
• getPayloadXML
• setPayloadXML

By combining any of the setters with a getter, we can handle almost any type of content transformation within the ESB. For example, by combining getPayloadXML and setPayloadJSON, we can easily implement an XML to JSON transformation scenario. In addition, we can perform various operations (such as deleting individual keys, modifying selected values, and inserting new objects) on JSON payloads to transform from one JSON format to another JSON format by using the getPayloadJSON and setPayloadJSON methods. Following is an example of a JSON to JSON transformation performed by the Script mediator.

Suppose a second service returns the following response:

{
   "results" : [
      {
         "geometry" : {
            "location" : {
               "lat" : -33.867260,
               "lng" : 151.1958130
            }
         },
         "icon" : "bar-71.png",
         "id" : "7eaf7",
         "name" : "Biaggio Cafe",
         "opening_hours" : {
            "open_now" : true
         },
         "photos" : [
            {
               "height" : 600,
               "html_attributions" : [],
               "photo_reference" : "CoQBegAAAI",
               "width" : 900
            }
         ],
         "price_level" : 1,
         "reference" : "CnRqAAAAtz",
         "types" : [ "bar", "restaurant", "food", "establishment" ],
         "vicinity" : "48 Pirrama Road, Pyrmont"
      },
      {
         "geometry" : {
            "location" : {
               "lat" : -33.8668040,
               "lng" : 151.1955790
            }
         },
         "icon" : "generic_business-71.png",
         "id" : "3ef98",
         "name" : "Doltone House",
         "photos" : [
            {
               "height" : 600,
               "html_attributions" : [],
               "photo_reference" : "CqQBmgAAAL",
               "width" : 900
            }
         ],
         "reference" : "CnRrAAAAV",
         "types" : [ "food", "establishment" ],
         "vicinity" : "48 Pirrama Road, Pyrmont"
      }
   ],
   "status" : "OK"
}

The following proxy service shows how we can transform the above response using JavaScript with the Script mediator.

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="locations"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <outSequence>
         <script language="js"
                 key="conf:/repository/esb/transform.js"
                 function="transform"/>
         <send/>
      </outSequence>
      <endpoint>
         <address uri="http://localhost:8280/locations"/>
      </endpoint>
   </target>
   <description/>
</proxy>

The registry resource transform.js contains the JavaScript function that performs the transformation:

function transform(mc) {
    payload = mc.getPayloadJSON();
    results = payload.results;
    var response = new Array();
    for (i = 0; i < results.length; ++i) {
        location_object = results[i];
        l = new Object();
        l.name = location_object.name;
        l.tags = location_object.types;
        l.id = "ID:" + (location_object.id);
        response[i] = l;
    }
    mc.setPayloadJSON(response);
}

mc.getPayloadJSON() returns the current JSON payload as a JavaScript object. This object can be manipulated as a normal JavaScript variable within a script as shown in the above JavaScript code. The mc.setPayloadJSON() method can be used to replace the existing payload with a new payload. In the above script, we build a new array object by using the fields of the incoming JSON payload and set that array object as the new payload (see the response payload returned by the final proxy service below.)

Use the following command to invoke the proxy service:

curl -v -X GET "http://ggrky:8280/services/locations"

The response payload would look like this:

[
    {
        "id":"ID:7eaf7", 
        "tags":["bar", "restaurant", "food", "establishment"], 
        "name":"Biaggio Cafe"
    }, 
    {
       "id":"ID:3ef98", 
       "tags":["food", "establishment"], 
       "name":"Doltone House"
    }
]

If you want to get the response in XML instead of JSON, you would modify the out sequence by adding the Property mediator as follows:

<outSequence>
    <script language="js" 
            key="conf:/repository/esb/transform.js" 
            function="transform"/>
    <property name="messageType" value="application/xml" scope="axis2"/>
    <send/>
</outSequence>

The response will then look like this:

<jsonArray>
   <jsonElement>
      <id>ID:7eaf7</id>
      <tags>bar</tags>
      <tags>restaurant</tags>
      <tags>food</tags>
      <tags>establishment</tags>
      <name>Biaggio Cafe</name>
   </jsonElement>
   <jsonElement>
      <id>ID:3ef98</id>
      <tags>food</tags>
      <tags>establishment</tags>
      <name>Doltone House</name>
   </jsonElement>
</jsonArray>

If you are not getting the results you want when the Script mediator converts the JSON payload directly into XML, you can build the XML payload iteratively with the Script mediator as shown in the following script.

function transformXML(mc) {
    payload = mc.getPayloadJSON();
    results = payload.results;
    var response = <locations/>;
    for (i = 0; i < results.length; ++i) {
        var elem = results[i];
        response.locations += <location>
            <id>{elem.id}</id>
            <name>{elem.name}</name>
            <tags>{elem.types}</tags>
        </location>
    }
    mc.setPayloadXML(response);
}

The response would now look like this:

<locations>
   <location>
      <id>7eaf7</id>
      <name>Biaggio Cafe</name>
      <tags>bar,restaurant,food,establishment</tags>
   </location>
   <location>
      <id>3ef98</id>
      <name>Doltone House</name>
      <tags>food,establishment</tags>
   </location>
</locations>

Finally, let's look at how you can perform delete, modify, and add field operations on JSON payloads with the Script mediator in JavaScript. Let's send the JSON message returned by the locations proxy service as the request for the following proxy service, transformjson:

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="transformjson"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <script language="js"><![CDATA[
           payload = mc.getPayloadJSON();
           for (i = 0; i &lt; payload.length; ++i) {
               payload[i].id_str = payload[i].id;
               delete payload[i].id;
               payload[i].tags[payload[i].tags.length] = "pub";
           }
           mc.setPayloadJSON(payload);
         ]]></script>
         <log>
            <property name="JSON-Payload" expression="json-eval($.)"/>
         </log>
         <respond/>
      </inSequence>
   </target>
   <description/>
</proxy>

The proxy service will convert the request into the following format:

[
   {
      "name":"Biaggio Cafe",
      "tags":["bar", "restaurant", "food", "establishment", "pub"],
      "id_str":"ID:7eaf7"
   },
   {
      "name":"Doltone House",
      "tags":["food", "establishment", "pub"],
      "id_str":"ID:3ef98"
   }
]

Note that the transformation (line 9 through 17) has added a new field id_str and removed the old field id from the request, and it has added a new tag pub to the existing tags list of the payload.

For additional examples that demonstrate different ways to manipulate JSON payloads within the ESB mediation flow, see the following samples:

• Sample 440: Converting JSON to XML Using XSLT
• Sample 441: Converting JSON to XML Using JavaScript

Troubleshooting, debugging, and logging

To assist with troubleshooting, you can enable debug logging at several stages of the mediation of a JSON payload within the ESB by adding one or more of the following loggers to the <ESB_HOME>/repository/conf/log4j.properties file and restarting the ESB.

Following are the available loggers:

Message builders and formatters

• log4j.logger.org.apache.synapse.commons.json.JsonStreamBuilder=DEBUG
• log4j.logger.org.apache.synapse.commons.json.JsonStreamFormatter=DEBUG
• log4j.logger.org.apache.synapse.commons.json.JsonBuilder=DEBUG
• log4j.logger.org.apache.synapse.commons.json.JsonFormatter=DEBUG

JSON utility class

log4j.logger.org.apache.synapse.commons.json.JsonUtil=DEBUG

PayloadFactory mediator

log4j.logger.org.apache.synapse.mediators.transform.PayloadFactoryMediator=DEBUG

JSONPath evaluator

log4j.logger.org.apache.synapse.util.xpath.SynapseJsonPath=DEBUG