Error Handling
When errors/exceptions occur in the system, the API Manager throws XML-based error responses to the client by default. To change the format of these error responses, you change the relevant XML file in the <API-M_HOME>/repository/deployment/server/synapse-configs/default/sequences
directory. The directory includes multiple XML files, named after the type of errors that occur. You must select the correct file.
For example, to change the message type of authorization errors, open the <API-M_HOME>/repository/deployment/server/synapse-configs/default/sequences/_auth_failure_handler.xml
file and change application/xml
to something like application/json
.
<sequence name="_auth_failure_handler_" xmlns="http://ws.apache.org/ns/synapse"> <property name="error_message_type" value="application/json"/> <sequence key="_cors_request_handler_"/> </sequence>
Similarly, to change the error messages of throttling errors (e.g., quota exceeding), change the _throttle_out_handler_.xml
file; resource mismatch errors, the _resource_mismatch_handler_.xml
file, etc.
Given below are some error codes and their meanings.
API handlers error codes
Error code | Error Message | Description | Example |
---|---|---|---|
| API blocked | This API has been blocked temporarily. Please try again later or contact the system administrators. | Invoke an API which is in the BLOCKED lifecycle state |
900800 | Message throttled out | The maximum number of requests that can be made to the API within a designated time period is reached and the API is throttled for the user. | Invoke an API exceeding the tier limit |
900801 | Hard limit exceeded | Hard throttle limit has been reached | Invoke an API exceeding the hard throttle limit |
900802 | Resource level throttle out | Message is throttled out because resource level has exceeded | Sending/Receiving messages beyond authorized resource level |
900803 | Application level throttle out | Message is throttled out because application level is exceeded | Sending/Receiving messages beyond authorized application level |
900804 | Subscription level throttled out | Message throttled out due to subscription level throttling limit reached. | Sending/Receiving messages beyond configured throttling limit of subscription level policy. |
900805 | Message blocked | Accessing an API which is blocked on user, IP, application, or API Context. | An admin user can block API invocations in real time by user, IP, application, or API context. The API invocation meets the blocked condition. |
900806 | Custom policy throttled out | Message throttled out due to exceeding the limit configured through the custom throttling policy rules. | The API invocations meet custom throttle policy rules, exceeding the limits of the configured custom policy. |
900807 | Message throttled out | Messaged throttled out because of exceeding the burst control/rate limit (requests per second) in the subscription level policy. | Sending/Receiving messages exceeding the configured burst control/rate limit within second. |
900900 | Unclassified authentication failure | An unspecified error has occurred | Backend service for key validation is not accessible when trying to invoke an API |
900901 | Invalid credentials | Invalid authentication information provided The error codes | When the access token is invalid or inactive. |
900902 | Missing credentials | No authentication information provided | Accessing an API without Authorization: Bearer header |
900905 | Incorrect access token type is provided | The access token type used is not supported when invoking the API. The supported access token types are application and user accesses tokens. See /wiki/spaces/SHAN/pages/22544564. | Invoke an API with application token, where the resource only allows application user tokens |
900906 | No matching resource found in the API for the given request | A resource with the name in the request can not be found in the API. | Invoke an API resource that is not available |
900907 | The requested API is temporarily blocked | Happens when the API user is blocked. | Invoke API resource with a subscription that has been blocked by the API publisher |
900908 | Resource forbidden | The user invoking the API has not been granted access to the required resource. | Invoke an unsubscribed API |
900909 | The subscription to the API is inactive | The status of the API has changed to an inaccessible/unavailable state. | Invoke an API resource with a subscription that has not yet been approved by the administrator. |
900910 | The access token does not allow you to access the requested resource | Can not access the required resource with the provided access token. Check the valid resources that can be accessed with this token. | Invoke API resource with an access token that is not generated to be used with the resource's scope. |
102511 | Incomplete payload | The payload sent with the request is too large and the client is unable to keep the connection alive until the payload is completely transferred to the API Gateway | Sending a large PDF file with the POST request |
Sequences error codes
Error code | Description |
---|---|
900901 | Production/sandbox key offered to the API with no production/sandbox endpoint |
400 | Server cannot process the request due to an error in the request sent by the client |
403 | No matching resource found in the API for the given request |
In addition to the above error codes, we have engaged Synapse-level error codes to the default fault sequence and custom fault sequences (e.g., _token_fault_.xml
) of the API Manager. For information, see Error Handling in WSO2 Enterprise Integrator (WSO2 EI) documentation.
The HTTP Status Codes and the corresponding error codes from the error responses are given below.
HTTP Status Code | Error Code |
---|---|
400 | 102511 |
401 | 900901, 900902, 900905, 900907, 900909 |
403 | 900906, 900908, 900910 |
429 | 900800, 900802, 900803, 900804, 900805, 900806, 900807 |
500 | 900900 |
503 | 700700, 900801 |
Transport error codes
Error Code | Detail |
---|---|
101000 | Receiver input/output error sending |
101001 | Receiver input/output error receiving |
101500 | Sender input/output error sending |
101501 | Sender input/output error receiving |
101503 | Connection failed |
101504 | Connection timed out (no input was detected on this connection over the maximum period of inactivity) |
101505 | Connection closed |
101506 | NHTTP protocol violation |
101507 | Connection canceled |
101508 | Request to establish new connection timed out |
101509 | Send abort |
101510 | Response processing failed |
HTTP PassThrough Transport Errors
Log messages for HTTP PassThrough transport errors have been improved to include more detail and the following error codes where applicable. Note that these improvements are only available as a WUM update and is effective from 28th February 2020 (2020-02-28). For more information on updating WSO2 API Manager, see Updating WSO2 API Manager.
Error Category | Error Code | Detail | Sample Log |
---|---|---|---|
SourceHandler Timeout Errors | N/A | The connection between the client and API-M has timed out after server accepting the request headers and the request body (inputReady()) The Socket timeout configured for the HTTP listener has been exceeded. For recommendations on socket timeout values see General APIM level recommendations | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Socket Timeout occurred after server accepting the request headers and the request body, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between the client and the EI timeout, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, SOCKET_TIMEOUT = 60000, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 |
N/A | The connection between the client and API-M has timed out. Socket timeout has occurred after reading request headers but while reading the request body (executing the method requestReceived() or inputReady() before the updateState). For recommendations on socket timeout values see General APIM level recommendations | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Socket Timeout occurred after reading request headers but while reading the request body from the client, INTERNAL_STATE = REQUEST_BODY/REQUEST_HEAD, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between the client and the EI timeout, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, SOCKET_TIMEOUT = 60000, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 | |
Response Path | N/A | The connection between the client and the API-M has timed out after the server has written the response headers but while writing the response body to the client (while executing method responseReady() or outputReady() before invoking response.write()) For recommendations on socket timeout values see General APIM level recommendations | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Socket Timeout occurred after server writing the response headers but while writing the response body to the client, INTERNAL_STATE = RESPONSE_BODY/RESPONSE_HEAD, DIRECTION = RESPONSE, CAUSE_OF_ERROR = Connection between the client and the EI timeout, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, SOCKET_TIMEOUT = 60000, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 |
TargetHandler TimeOut Errors | 101504 | Connection between the API-M and the backend has timedout after the server has written the request headers and the request body to the backend. The Request message has been written but Response message hasn’t been received by the API-M prior to socket timeout For recommendations on socket timeout values see General APIM level recommendations | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101504, STATE_DISCRIPTION = Socket Timeout occurred after Server written the request headers and the request body to the backend, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between the EI and the BackEnd timeouts, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME |
TargetHandler Connection Closed Exceptions | 101505 | The backend has closed the connection prior to API-M sending the request to it. The headers have been written (the requestReady() method has been executed) but writing the message body has not been completed (outputReady() method is not completed) | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101505, STATE_DISCRIPTION = Connection closed by target host after Server written the request headers but while writing the request body to the backend, INTERNAL_STATE = REQUEST_HEAD/REQUEST_BODY, DIRE-CTION = REQUEST, CAUSE_OF_ERROR = Connection between the EI and the BackEnd has been closed, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME |
101505 | The backend has closed the connection after reading the request prior to sending the response to the API-M. The outputReady() method is done | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101505, STATE_DISCRIPTION = Connection closed by target host after Server written the request headers and the request body to the backend, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between the EI and the BackEnd has been closed, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME | |
101505 | Connection closed by the backend after the server read the response headers but before completing to read the response body from the backend. The responseReceived() method has been executed so the headers have been read but the message body is still being received. inputReady() method is not completed | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101505, STATE_DISCRIPTION = Connection closed by target host after Server read the response headers but while reading the response body from the backend, INTERNAL_STATE = RESPONSE_HEAD/RESPONSE_BODY, DIRECTION = RESPONSE, CAUSE_OF_ERROR = Connection between the EI and the BackEnd has been closed, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME | |
SourceHandler Connection Closed Exceptions | N/A | Client has closed the connection after reading the request headers(request Received() method has been executed) but while reading the request body from the client( inputReady() method is not completed) | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Client has closed the connection after reading the request headers but while reading the request body from the client, INTERNAL_STATE = REQUEST_HEAD/REQUEST_BODY, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between EI and the Client has been closed, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 |
N/A | Client has closed the connection after the server has written the response headers (responseReady() method has been executed ) but while writing the response body to the client (outputReady() method is not completed) | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Client has closed the connection after Server writing the response headers but while writing the response body to the client, INTERNAL_STATE = RESPONSE_HEAD/RESPONSE_BODY, DIRECTION = RESPONSE, CAUSE_OF_ERROR = Connection between EI and the Client has been closed, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 | |
N/A | The client closed the connection after API-M read the request. The inputReady() method is done | WARN {org.apache.synapse.transport.passthru.SourceHandler} - STATE_DISCRIPTION = Client has closed the connection after server accepting the request headers and the request body, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection between EI and the Client has been closed, HTTP_URL = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, CLIENT_ADDRESS = /127.0.0.1:34422, CONNECTION = http-incoming-1 | |
TargetHandler Unexpected Exceptions | 101500 | Backend immediately closed the connection (RST) before API-M sends the request to it (headers written or body has been written prematurely). triggered by I/O Exception DefaultNHttpClientConnection in consumeInput(). | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101500, STATE_DISCRIPTION = Exception occurred after Server written the request headers but while writing the request body to the backend, INTERNAL_STATE = REQUEST_HEAD/REQUEST_BODY, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection reset by peer, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME |
101500 | Backend immediately closed the connection (RST) before API-M read the response from it triggered by I/O Exception DefaultNHttpClientConnection in consumeInput(). | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101500, STATE_DISCRIPTION = Exception occurred after Server written the request headers and the request body to the backend, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection reset by peer, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME | |
101500 | Backend immediately closed the connection (RST) before API-M read the response (headers read or body has been read prematurely). Triggered by I/O Exception DefaultNHttpClientConnection in consumeInput(). | WARN {org.apache.synapse.transport.passthru.TargetHandler} - ERROR_CODE = 101500, STATE_DISCRIPTION = Exception occurred after Server read the response headers but while reading the response body from the backend, INTERNAL_STATE = REQUEST_DONE, DIRECTION = REQUEST, CAUSE_OF_ERROR = Connection reset by peer, TARGET_HOST = 127.0.0.1, TARGET_PORT = 8280, TARGET_CONTEXT = v2/5a0e34ad30000067114334d, HTTP_METHOD = POST, TRIGGER_TYPE = API, TRIGGER_NAME = API_NAME |
Error Category | Method | Detail | Sample Logs |
---|---|---|---|
ConnectCallback Errors These errors could occur when DileveryAgent tries to get a connection in the submit() method and tries to connect | failed() | Triggered on the unsuccessful completion of a SessionRequest. Ex: If the backend is not available at the point where API-M is trying to create a connection The SessionRequest.getException() method can now be used to obtain the cause of the error. | Connection refused or failed for : localhost/127.0.0.1:8281 |
timeout() | Triggered if a SessionRequest timed out. | Timeout connecting to : localhost/127.0.0.1:8281 | |
cancelled() | Triggered on cancellation of a SessionRequest. | Connection cancelled for : localhost/127.0.0.1:8281 |
There is a state machine in the transport sender side, where the protocol state changes according to the phase of the message.
Following are the possible protocol states and the description for each:
Protocol State | Description |
---|---|
REQUEST_READY (0) | Connection is at the initial stage ready to send a request |
REQUEST_HEAD(1) | Sending the request headers through the connection |
REQUEST_BODY(2) | Sending the request body |
REQUEST_DONE(3) | Request is completely sent |
RESPONSE_HEAD(4) | The connection is reading the response headers |
RESPONSE_BODY(5) | The connection is reading the response body |
RESPONSE_DONE(6) | The response is completed |
CLOSING(7) | The connection is closing |
CLOSED(8) | The connection is closed |
Custom error messages
To send a custom message with a custom HTTP status code, you execute an additional sequence that can generate a new error message. You then override the message body, HTTP status code and other values.
The following steps demonstrate how to override a throttled-out message's HTTP status code as a custom error message:
Start the WSO2 API Manager.
Go to
<API-M_HOME>
/repository/deployment/server/synapse-configs/default/sequences
directory and create the fileconvert.xml
as follows.<sequence xmlns="http://ws.apache.org/ns/synapse" name="convert"> <payloadFactory media-type="xml"> <format> <am:fault xmlns:am="http://wso2.org/apimanager"> <am:code>$1</am:code> <am:type>Status report</am:type> <am:message>Runtime Error</am:message> <am:description>$2</am:description> </am:fault> </format> <args> <arg evaluator="xml" expression="$ctx:ERROR_CODE"/> <arg evaluator="xml" expression="$ctx:ERROR_MESSAGE"/> </args> </payloadFactory> <property name="RESPONSE" value="true"/> <header name="To" action="remove"/> <property name="HTTP_SC" value="555" scope="axis2"/> <property name="NO_ENTITY_BODY" scope="axis2" action="remove"/> <property name="ContentType" scope="axis2" action="remove"/> <property name="Authorization" scope="transport" action="remove"/> <property name="Access-Control-Allow-Origin" value="*" scope="transport"/> <property name="Host" scope="transport" action="remove"/> <property name="Accept" scope="transport" action="remove"/> <property name="X-JWT-Assertion" scope="transport" action="remove"/> <property name="messageType" value="application/json" scope="axis2"/> <send/> </sequence>
Alternatively, you can use the Source View of the API-M Management Console as follows to edit the synapse configuration:
- Sign in to the Management Console. (
https://<Server Host>:9443/carbon
). - Go to Manager -> Source View.
- Copy the content of the sequence in
convert.xml
, paste it as a new sequence in the source view and update it.
- Sign in to the Management Console. (
Check the terminal logs to see whether there are issues in the deployment.
If the deployment is successful, you see a message similar to the following in the system logs:INFO - DependencyTracker Sequence : convert was added to the Synapse configuration successfully INFO - SequenceDeployer Sequence named 'convert' has been deployed from file : <API-M_HOME>/repository/deployment/server/synapse-configs/default/sequences/convert.xml
Include the sequence that you just deployed in a sequence of your choice.
_auth_failure_handler_
sequence.<sequence name="_auth_failure_handler_" xmlns="http://ws.apache.org/ns/synapse"> ... <sequence key="convert"/> <drop/> </sequence>
Check the terminal and see whether there are any errors with the
_auth_failure_handler_
sequence deployment.
If the deployment is successful, you see a message similar to the following in the system logs:INFO - DependencyTracker Sequence : _auth_failure_handler_ was added to the Synapse configuration successfully INFO - SequenceDeployer Sequence: _auth_failure_handler_ has been updated from the file: <API-M_HOME>/repository/deployment/server/synapse-configs/default/sequences/_auth_failure_handler_.xml
Invoke the API with the respective criteria in order to trigger the sequence.
In this example, let's view the menu on the PizzaShack API and invoke the API with an incorrect token.
WSO2 API Manager has the following default fault sequences located in <API-M_HOME>
/repository/deployment/server/synapse-configs/default/sequences
directory.
Fault Sequence | Description |
---|---|
fault.xml | This is the primary fault sequence that gets invoked when an error occurs during the execution of an API resources |
main.xml | This sequence is called when the endpoint being called does not exist |
_auth_failure_handler.xml | This sequence is called when an API authentication error is encountered |
_production_key_error.xml | This sequence is called when a Production key is used to invoke an API that does not have a Production endpoint defined |
_sandbox_key_error.xml | This sequence is called when a Sandbox key is used to invoke an API that does not have a Sandbox endpoint defined |
_throttle_out_handler.xml | This sequence is called when a given request to an API gets throttled out |
_token_fault.xml | This sequence is called when there is an error in invoking the token API |
_resource_mismatch_handler.xml | This sequence is called when a matching resource cannot be found by the gateway to the corresponding resource being invoked |
| This sequence enables sending CORS specific headers when the CORS specific configuration (CORSConfiguration ) is enabled in WSO2 API Manager in the <API-M_HOME>/repository/conf/api-manager.xml file. |
_threat_fault_.xml | This sequence is called to send error messages with regard to threat detection. |
dispatchSeq.xml | This sequence is defined as a default handler for any inbound WebSocket calls. |
outDispatchSeq.xml | This sequence is defined to handle any outbound WebSocket calls. |
The default sequences can also be customized as shown in the section above.