Transforming API Message Payload

When a request comes to the API Manager, it sends the response in the same format of the request. For example, the API Manager handles JSON to JSON transformations out of the box. If the backend does not accept messages of the same content type of the request message, it must be transformed to a different format. The API Gateway of the API Manager handles these transformations using message builders and formatters.

When a message comes in to the API Gateway, the receiving transport selects a message builder based on the message's content type. It uses that builder to process the message's raw payload data and convert it into JSON. Conversely, when sending a message out from the Gateway, a message formatter is used to build the outgoing stream from the message. As with message builders, the message formatter is selected based on the message's content type.

Note that if you edit an API's synapse configuration as mentioned in this guide and then go back to the API Publisher and save the API, your changes will be overwritten. Therefore, we do not recommend changing the API's synapse configuration directly. The recommended way to extend an API's mediation flow is by engaging In/Out sequences.

Also see the following sections in the WSO2 ESB documentation. WSO2 ESB is used to implement the API Gateway through which API messages are transformed:

JSON message builders and formatters

There are two types of message builders and formatters for JSON. The default builder and formatter keep the JSON representation intact without converting it to XML. You can access the payload content using the JSON Path or XPath and convert the payload to XML at any point in the mediation flow.

org.apache.synapse.commons.json.JsonStreamBuilder
org.apache.synapse.commons.json.JsonStreamFormatter

If you want to convert the JSON representation to XML before the mediation flow begins, use the following builder and formatter instead. Note that some data loss can occur during the JSON -> XML -> JSON conversion process.

org.apache.synapse.commons.json.JsonBuilder
org.apache.synapse.commons.json.JsonFormatter

The builders and formatters are configured respectively in the messageBuilders and messageFormatters sections of the Axis2 configuration files located in the <PRODUCT_HOME>/repository/conf/axis2 directory. Both types of JSON builders use StAXON as the underlying JSON processor.

The following builders and formatters are also included for compatibility with older API Manager versions:

org.apache.axis2.json.JSONBuilder/JSONMessageFormatter
org.apache.synapse.commons.json.JsonStreamBuilder/JSONStreamFormatter
org.apache.axis2.json.JSONBadgerfishOMBuilder/JSONBadgerfishMessageFormatter

Always use the same type of builder and formatter combination. Mixing different builders and formatters will cause errors at runtime.

If you want to handle JSON payloads that are sent using a media type other than application/json, you must register the JSON builder and formatter for that media type in the following two files at minimum (for best results, register them in all Axis2 configuration files found in the <PRODUCT_HOME>/repository/conf/axis2 directory):

<PRODUCT_HOME>/repository/conf/axis2/axis2.xml
<PRODUCT_HOME>/repository/conf/axis2/axis2_blocking_client.xml

For example, if the media type is text/javascript, register the message builder and formatter as follows:

<messageBuilder contentType="text/javascript" 
               class="org.apache.synapse.commons.json.JsonStreamBuilder"/>

<messageFormatter contentType="text/javascript" 
                class="org.apache.synapse.commons.json.JsonStreamFormatter"/>

To support having spaces inside JSON attributes, change the default JSON builder and formatter to the following pair in either <APIM_HOME>/repository/conf/axis2/axis2.xml (super tenant scenario) or <APIM_HOME>/repository/conf/axis2/tenant-axis2.xml (tenant scenario) file:

<messageBuilder contentType="application/json" class="org.apache.synapse.commons.json.JsonStreamBuilder>
<messageFormatter contentType="application/json" class="org.apache.synapse.commons.json.JsonStreamFormatter”/>

To support use cases for JSON payloads with arrays, change the default JSON builder and formatter to the following pair in either <APIM_HOME>/repository/conf/axis2/axis2.xml (super tenant scenario) or <APIM_HOME>/repository/conf/axis2/tenant-axis2.xml (tenant scenario) file:

<messageBuilder contentType="application/json" class="org.apache.synapse.commons.json.JsonStreamBuilder>
<messageFormatter contentType="application/json" class="org.apache.synapse.commons.json.JsonStreamFormatter”/>

Else, in JSON to XML conversion, there might be issues as below:

JSON

"phoneNumbers":[
      {
        "phoneNumber":"4027161289",
        "phoneNumberType":"home"
      }
]

Converted XML

<?xml-multiple phoneNumbers?>
<phoneNumbers>
   <phoneNumber>4027161289</phoneNumber>
   <phoneNumberType>home</phoneNumberType>
</phoneNumbers>

When you modify the builders/formatters in Axis2 configuration, make sure that you have enabled only one correct message builder/formatter pair for a given media type.

XML representation of JSON payloads

When building the XML tree, JSON builders attach the converted XML infoset to a special XML element that acts as the root element of the final XML tree. If the original JSON payload is of type object, the special element is <jsonObject/>. If it is an array, the special element is <jsonArray/>. Following are examples of JSON and XML representations of various objects and arrays.

Null objects

JSON:

{"object":null}

XML:

<jsonObject>
    <object></object>
</jsonObject>

Empty objects

JSON:

{"object":{}}

XML:

<jsonObject>
   <object></object>
</jsonObject>

Empty strings

JSON:

{"object":""}

XML:

<jsonObject>
   <object></object>
</jsonObject>

Empty array

JSON:

[]

XML (JsonStreamBuilder):

<jsonArray></jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
</jsonArray>

Named arrays

JSON:

{"array":[1,2]}

XML (JsonStreamBuilder):

<jsonObject>
   <array>1</array>
   <array>2</array>
</jsonObject>

XML (JsonBuilder):

<jsonObject>
   <?xml-multiple array?>
   <array>1</array>
   <array>2</array>
</jsonObject>

JSON:

{"array":[]}

XML (JsonStreamBuilder):

<jsonObject></jsonObject>

XML (JsonBuilder):

<jsonObject>
   <?xml-multiple array?>
</jsonObject>

Anonymous arrays

JSON:

[1,2]

XML (JsonStreamBuilder):

<jsonArray>
   <jsonElement>1</jsonElement>
   <jsonElement>2</jsonElement>
</jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
   <jsonElement>1</jsonElement>
   <jsonElement>2</jsonElement>
</jsonArray>

JSON:

[1, []]

XML (JsonStreamBuilder):

<jsonArray>
   <jsonElement>1</jsonElement>
   <jsonElement>
       <jsonArray></jsonArray>
   </jsonElement>
</jsonArray>

XML (JsonBuilder):

<jsonArray>
   <?xml-multiple jsonElement?>
   <jsonElement>1</jsonElement>
   <jsonElement>
       <jsonArray>
           <?xml-multiple jsonElement?>
       </jsonArray>
   </jsonElement>
</jsonArray>

XML processing instructions (PIs)

Note that the addition of xml-multiple processing instructions to the XML payloads whose JSON representations contain arrays. JsonBuilder (via StAXON) adds these instructions to the XML payload that it builds during the JSON to XML conversion so that during the XML to JSON conversion, JsonFormatter can reconstruct the arrays that are present in the original JSON payload. JsonFormatter interprets the elements immediately following a processing instruction to construct an array.

Special characters

When building XML elements, the ‘$’ character and digits are handled in a special manner when they appear as the first character of a JSON key. Following are examples of two such occurrences. Note the addition of the _JsonReader_PS_ and _JsonReader_PD_ prefixes in place of the ‘$’ and digit characters, respectively.

JSON:

{"$key":1234}

XML:

<jsonObject>
   <_JsonReader_PS_key>1234</_JsonReader_PS_key>
</jsonObject>

JSON:

{"32X32":"image_32x32.png"}

XML:

<jsonObject>
   <_JsonReader_PD_32X32>image_32x32.png</_JsonReader_PD_32X32>
</jsonObject>

Converting a payload between XML and JSON

To convert an XML payload to JSON, set the messageType property to application/json in the axis2 scope before sending message to an endpoint. Similarly, to convert a JSON payload to XML, set the messageType property to application/xml or text/xml. For example:

 <api name="admin--TOJSON" context="/tojson" version="1.0" version-type="url">
        <resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
            <inSequence>
                <property name="POST_TO_URI" value="true" scope="axis2"/>
                <property name="messageType" value="application/json" scope="axis2"/>
                <filter source="$ctx:AM_KEY_TYPE" regex="PRODUCTION">
                    <then>
                        <send>
                            <endpoint name="admin--Test_APIproductionEndpoint_0">
                                <http uri-template="http://localhost:9767/services/StudentService">
                                    <timeout>
                                        <duration>30000</duration>
                                        <responseAction>fault</responseAction>
                                    </timeout>
                                    <suspendOnFailure>
                                        <errorCodes>-1</errorCodes>
                                        <initialDuration>0</initialDuration>
                                        <progressionFactor>1.0</progressionFactor>
                                        <maximumDuration>0</maximumDuration>
                                    </suspendOnFailure>
                                    <markForSuspension>
                                        <errorCodes>-1</errorCodes>
                                    </markForSuspension>
                                </http>
                            </endpoint>
                        </send>
                    </then>
                    <else>
                        <sequence key="_sandbox_key_error_"/>
                    </else>
                </filter>
            </inSequence>
            <outSequence>
                <send/>
            </outSequence>
        </resource>
        <handlers>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
                <property name="id" value="A"/>
                <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
            </handler>
            <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>
            <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtGoogleAnalyticsTrackingHandler"/>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler"/>
        </handlers>
    </api>

An example command to invoke above API:

curl -v -X POST -H "Content-Type:application/xml" -H "Authorization: Bearer xxx" -d@request1.xml "http://10.100.1.110:8280/tojson/1.0"

If the request payload is as follows:

<coordinates>
   <location>
       <name>Bermuda Triangle</name>
       <n>25.0000</n>
       <w>71.0000</w>
   </location>
   <location>
       <name>Eiffel Tower</name>
       <n>48.8582</n>
       <e>2.2945</e>
   </location>
</coordinates>

The response payload will look like this:

{
  "coordinates":{
     "location":[
        {
           "name":"Bermuda Triangle",
           "n":25.0000,
           "w":71.0000
        },
        {
           "name":"Eiffel Tower",
           "n":48.8582,
           "e":2.2945
        }
     ]
  }
}

Note that we have used the Property mediator to mark the outgoing payload to be formatted as JSON. For more information about the Property Mediator, see the Property Mediator page on WSO2 ESB documentation.

<property name="messageType" value="application/json" scope="axis2"/>

Similarly if the response message needs to be transformed, set the messageType property in the outSequence.

 <api name="admin--TOJSON" context="/tojson" version="1.0" version-type="url">
        <resource methods="POST GET DELETE OPTIONS PUT" url-mapping="/*">
            <inSequence>
                <property name="POST_TO_URI" value="true" scope="axis2"/>
                <filter source="$ctx:AM_KEY_TYPE" regex="PRODUCTION">
                    <then>
                        <send>
                            <endpoint name="admin--Test_APIproductionEndpoint_0">
                                <http uri-template="http://localhost:9767/services/StudentService">
                                    <timeout>
                                        <duration>30000</duration>
                                        <responseAction>fault</responseAction>
                                    </timeout>
                                    <suspendOnFailure>
                                        <errorCodes>-1</errorCodes>
                                        <initialDuration>0</initialDuration>
                                        <progressionFactor>1.0</progressionFactor>
                                        <maximumDuration>0</maximumDuration>
                                    </suspendOnFailure>
                                    <markForSuspension>
                                        <errorCodes>-1</errorCodes>
                                    </markForSuspension>
                                </http>
                            </endpoint>
                        </send>
                    </then>
                    <else>
                        <sequence key="_sandbox_key_error_"/>
                    </else>
                </filter>
            </inSequence>
            <outSequence>
                <property name="messageType" value="application/json" scope="axis2"/>
                <send/>
            </outSequence>
        </resource>
        <handlers>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
                <property name="id" value="A"/>
                <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
            </handler>
            <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>
            <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtGoogleAnalyticsTrackingHandler"/>
            <handler class="org.wso2.carbon.apimgt.gateway.handlers.ext.APIManagerExtensionHandler"/>
        </handlers>
    </api>

XML to JSON Transformation Parameters

See JSON Transformation Parameters for additional parameters for converting XML to JSON.