com.atlassian.confluence.content.render.xhtml.migration.exceptions.UnknownMacroMigrationException: The macro 'next_previous_link3' is unknown.

Packaging Artifacts as a C-App Archive

Owned by Former user (Deleted)
Last updated: Apr 08, 2019
9 min read

C-App namely Carbon Application is an archive format collection of different artifacts bundled to a single deployable component. C-App files have CAR extensions and can be deployed to different runtimes. Each runtime will only deploy the artifacts which match with the role that the runtime is playing.

Supported C-App types

Given below are the type of the carbon applications artifacts that are supported by WSO2 DAS.

Generic artifacts

Artifact Type
Event Streams event/streams
Event Receivers event/receiver
Event Publishers event/publisher

Real time analytics specific artifacts

ArtifactType
Execution Plan event/execution-plan

Batch analytics specific artifacts

ArtifactType
Event stores analytics/eventstore
Spark scripts analytics/spark
Data purging scriptsanalytics/dataPurging

Visualization artifacts

ArtifactType
Dashboardsdashboards/dashboard
Layoutsdashboards/layout
Gadgetsdashboards/gadget

If the above mentioned artifacts are being deployed using a C-App, you are restricted on editing or deleting them. Therefore, if you need to edit the C-App, you need to re-pack and re-deploy it.

Creating a C-App

Follow the steps below to create a C-App to be deployed in WSO2 DAS. In this below section, we are going to focus on adding the carbon application with 5 different kind of artifact event streams, event receiver, event publisher, event store, and Spark scripts.

├── artifacts.xml
├── Eventreceiver_1.0.0
│   ├── artifact.xml
│   └── TestWso2EventReceiver.xml
├── EventPublisher_1.0.0
	├──artifact.xml
	└──TestLoggerPublisher.xml
├── Eventstore_1.0.0
│   ├── artifact.xml
│   └── ORG_WSO2_TEST.xml
├── Eventstream_1.0.0
│   ├── artifact.xml
│   └── org.wso2.testinputstream_1.0.0.json
├── Eventstream_1.0.0
│   ├── artifact.xml
│   └── org.wso2.testoutputstream_1.0.0.json
├── ExecutionPlan_1.0.0
	├──artifact.xml
	└──ExecutionPlan.xml
└── Sparkscripts_1.0.0
|	├── artifact.xml
|	└── sample_script.xml
└── DataPurging_1.0.0
	├── artifact.xml
	└──data_purging.xml
└── Dashboard_1.0.0
|	├── artifact.xml
|	└── appman.json
|── Gadget_1.0.0
|	├── artifact.xml
|	└── Test_Gadget
|── Layout_1.0.0
|	├── artifact.xml
|	└── Test_Layout

  1. Create the top level artifacts.xml file of the C-App which defines the set of folders included in it as shown below. 

    <artifacts>
	<artifact name="DASTestCApp" version="1.0.0" type="carbon/application">
    	<dependency artifact="Eventstore" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Eventreceiver" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
		<dependency artifact="EventPublisher" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Eventstream" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
		<dependency artifact="ExecutionPlan" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Sparkscripts" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
		<dependency artifact="DataPurging" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Dashboard" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Gadget" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Layout" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
	</artifact>
</artifacts>

  2. Create separate directories in the top level of the C-App for the above mentioned dependencies (Eventreceiver _1.0.0, EventPublisher_1.0.0 ,  Eventstore_1.0.0, Eventstream_1.0.0, ExecutionPlan_1.0.0, Sparkscripts_1.0.0, and Data_Purging_1.0.0 )   defined in the artifacts.xml file.

    You can have multiple dependencies as required with a directory for each of them in the same level as the artifacts.xml file in the C-App. Include the name and the version of the artifact in the name of the directory.

  3.  Create an artifact.xml file inside all dependency directories as follows.

    Eventstore_1.0.0 artifact .xml 

    <artifact name="Eventstore" version="1.0.0" type="analytics/eventstore" serverRole="DataAnalyticsServer">
	<file>ORG_WSO2_TEST.xml</file>
</artifact> 

    Eventstream_1.0.0 artifact.xml

    <artifact name="Eventstream" version="1.0.0" type="event/stream" serverRole="DataAnalyticsServer">
	<file>org.wso2.test_1.0.0.json</file>
</artifact>

    Eventreceiver_1.0.0 artifact.xml

    <artifact name="Eventreceiver" version="1.0.0" type="event/receiver" serverRole="DataAnalyticsServer">
	<file>TextWso2EventReceiver.xml</file>
</artifact>

    EventPublisher_1.0.0 artifact.xml


    <artifact name="EventPublisher" version="1.0.0" type="event/publisher" serverRole="DataAnalyticsServer">
    <file>TestLoggerPublisher.xml</file>
</artifact>

    ExecutionPlan_1.0.0 artifact.xml

    <artifact name="Executionplan" version="1.0.0" type="event/execution-plan" serverRole="DataAnalyticsServer">
        <file>TestExecutionPlan</file>
</artifact>


    Sparkscripts_1.0.0 artifact.xml

    <artifact name="Sparkscripts" version="1.0.0" type="analytics/spark" serverRole="DataAnalyticsServer">
	<file>sample_script.xml</file>
</artifact>

    DataPurging_1.0.0 artifact.xml


    <artifact name="DataPurging" version="1.0.0" type="analytics/dataPurging" serverRole="DataAnalyticsServer">
    <file>data_purging.xml</file>
</artifact>


    Dashboard_1.0.0 artifact.xml

    <artifact name="Dashboard" version="1.0.0" type="dashboards/dashboard" serverRole="DataAnalyticsServer">
    <file>appman.json</file>
</artifact>

    Gadget_1.0.0 artifact.xml

    <artifact name="Gadget" version="1.0.0" type="dashboards/gadget" serverRole="DataAnalyticsServer">
    <file>Test_Gadget</file>
</artifact>

    Layout_1.0.0 artifact.xml

    <artifact name="Layout" version="1.0.0" type="dashboards/layout" serverRole="DataAnalyticsServer">
    <file>Test_Layout</file>
</artifact>

  4. Create the actual artifact which was specified in the  artifact.xml above. For example, you need to create an event store configuration named  ORG_WSO2_TEST.xml , two event streams named  org.wso2.testinputstream_1.0.0.json and org.wso2.testoutputstream_1.0.0.json, an event receiver configuration named  TextWso2EventReceiver.xml , a Spark script configuration named   sample_script.xml , a dashboard configuration named  appman.json , a gadget configuration named  gadget.json ,   and a layout configuration named  layout.json . See below for each of the configuration files.

    Stream - org.wso2.testinputstream_1.0.0.json 

    {
  "name": "org.wso2.testinputstream",
  "version": "1.0.0",
  "nickName": "TestInputStream",
  "description": "A test input stream",
  "metaData": [
    {
      "name": "remote_host",
      "type": "STRING"
    },
    {
      "name": "tenant_id",
      "type": "INT"
    }
  ],
  "correlationData": [
    {
      "name": "activity_id",
      "type": "STRING"
    }
  ],
  "payloadData": [
    {
      "name": "operation_name",
      "type": "STRING"
    },
    {
      "name": "service_name",
      "type": "STRING"
    }
  ]
}

    Stream - org.wso2.testoutputstream_1.0.0.json

    {
  "name": "org.wso2.testoutputstream",
  "version": "1.0.0",
  "nickName": "TestOutputStream",
  "description": "A test output stream",
  "metaData": [
    {
      "name": "remote_host",
      "type": "STRING"
    },
    {
      "name": "tenant_id",
      "type": "INT"
    }
  ],
  "correlationData": [
    {
      "name": "activity_id",
      "type": "STRING"
    }
  ],
  "payloadData": [
    {
      "name": "operation_name",
      "type": "STRING"
    },
    {
      "name": "service_name",
      "type": "STRING"
    }
  ]
}

    Event Store - ORG_WSO2_TEST.xml 

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<EventStoreConfiguration>
<Source>
       <StreamId>org.wso2.test:1.0.0</StreamId>
   </Source>
   <TableSchema>
         <ColumnDefinition>
           <Name>meta_remote_host</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>meta_tenant_id</Name>
           <EnableIndexing>false</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>INTEGER</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>correlation_activity_id</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>FACET</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>operation_name</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>service_name</Name>
           <EnableIndexing>false</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
   </TableSchema>
   </EventStoreConfiguration>

    Event receiver - TestWso2EventReceiver.xml 

    <eventReceiver name="TestWso2EventReceiver" statistics="disable"
    trace="disable" xmlns="http://wso2.org/carbon/eventreceiver">
    <from eventAdapterType="wso2event">
        <property name="events.duplicated.in.cluster">false</property>
    </from>
    <mapping customMapping="disable" type="wso2event"/>
    <to streamName="org.wso2.testinputstream" version="1.0.0"/>
</eventReceiver>

    Event publisher - TestLoggerPublisher.xml 


    <eventPublisher name="TestLoggerPublisher" processing="enable"
  statistics="disable" trace="disable" xmlns="http://wso2.org/carbon/eventpublisher">
  <from streamName="org.wso2.testoutputstream" version="1.0.0"/>
  <mapping customMapping="disable" type="json"/>
  <to eventAdapterType="logger">
    <property name="uniqueId">TestLoggerPublisher</property>
  </to>
</eventPublisher>

    Execution Plan - ExecutionPlan_1.0.0.xml

    /* Enter a unique ExecutionPlan */
@Plan:name('ExecutionPlan')

/* Enter a unique description for ExecutionPlan */
-- @Plan:description('ExecutionPlan')

/* define streams/tables and write queries here ... */

@Import('org.wso2.testinputstream:1.0.0')
define stream InputStream (meta_remote_host string, meta_tenant_id int, correlation_activity_id string, operation_name string, service_name string);

@Export('org.wso2.testoutputstream:1.0.0')
define stream OutputStream (meta_remote_host string, meta_tenant_id int, correlation_activity_id string, operation_name string, service_name string);

From InputStream
Select meta_remote_host, meta_tenant_id, correlation_activity_id, operation_name, service_name
Insert Into OutoutStream

    Spark script - sample_script.xml 


    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Analytics>
	<Name>AddNewScriptTestWithouTask</Name>
	<Script>create temporary table TEST_CAPP USING CarbonAnalytics OPTIONS (tableName "TEST_CAPP", schema "server_name string, ip STRING, tenant INTEGER, sequence LONG, summary STRING");

insert into table TEST_CAPP select "MyServer", "10.10.10.10", 1, 12, "test";</Script>
	<CronExpression>0 * * * * ?</CronExpression>
</Analytics>

    Data purging script - data_purging.xml


    <analytics-data-purging>
    <cron-expression>0 0 0 * * ?</cron-expression>
    <purge-include-tables>
        <table>org.wso2.test</table>
        <table>TEST_CAPP</table>        
    </purge-include-tables>
    <data-retention-days>1</data-retention-days>
</analytics-data-purging>


    Dashboard - appman.json


    {"id":"appman","title":"appman","description":"","permissions":{"viewers":[],"editors":["Internal/everyone"]},"pages":[{"id":"landing","title":"My Dashboard","layout":{"id":"layout-3","title":"Layout 3","description":"This is a sample grid","thumbnail":"local://store/layout/layout-3/index.jpg","url":"local://store/layout/layout-3/index.hbs","content":"\n   <div class=\"row\">\n        <div id=\"a\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"b\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"c\" class=\"col-md-4 ues-component-box\"></div>\n    </div>\n    <div class=\"row\">\n        <div id=\"d\" class=\"col-md-10 ues-component-box\"></div>\n        <div id=\"e\" class=\"col-md-2 ues-component-box\"></div>\n    </div>\n    <div class=\"row\">\n        <div id=\"f\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"g\" class=\"col-md-4 ues-component-box\"></div>\n \t<div id=\"h\" class=\"col-md-4 ues-component-box\"></div>\n    </div>\n\n"},"content":{"d":[{"id":"24w6ukj9olz69a4i","content":{"id":"Temperature_By_City","title":"Temperature By City","type":"gadget","thumbnail":"local://store/gadget/usa-business-revenue/index.png","data":{"url":"local://store/gadget/Temperature_By_City/index.xml"},"styles":{"title":"Temperature By City","borders":true},"options":{"dataSource":{"type":"STRING","title":"Data Source","value":"/portal/gadgets/bar-chart/datasource/dataFile4.jag","options":[],"required":false},"updateGraph":{"type":"STRING","title":"Update Interval (s)","value":"No","options":[],"required":false}}}}]}}],"landing":"landing"}

    Gadget - gadget.json


    {"id":"Test_Gadget","title":"Temperature By City","type":"gadget","thumbnail":"local://store/gadget/usa-business-revenue/index.png","data":{"url":"local://store/gadget/Temperature_By_City/index.xml"}}

    Layout - layout.json

    {
    "id": "Test_Layout",
    "title": "Test_Layout",
    "description": "This is a sample grid",
    "thumbnail": "local://store/layout/layout-3/index.jpg",
    "url": "local://store/layout/layout-3/index.hbs"
}

Deploying a C-App

Follow the steps below to deploy a C-App.

  1. Log in to WSO2 CEP/DAS management console using admin/admin credentials.

  2. Click Main , and then click Add in the Carbon Applications menu.

  3. Click Choose File as shown below.

    It will take few seconds up to 10 seconds by default to deploy. You view below logs in the CEP/DAS server.


    [2015-05-26 18:05:44,541]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Deploying Carbon Application : DASTestCApp.car...
[2015-05-26 18:05:45,831]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.test:1.0.0
[2015-05-26 18:05:45,849]  INFO {org.wso2.carbon.event.input.adapter.core.internal.CarbonInputAdapterRuntime} -  Connecting receiver TextWso2EventReceiver
[2015-05-26 18:05:45,852]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.test:1.0.0
[2015-05-26 18:05:45,853]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state : TextWso2EventReceiver
[2015-05-26 18:05:45,853]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Deploying analytics event store :ORG_WSO2_TEST.xml
[2015-05-26 18:05:45,885]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  WSO2EventConsumer added to the junction. Stream:org.wso2.test:1.0.0
[2015-05-26 18:05:45,886]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Deployed successfully analytics event store :ORG_WSO2_TEST.xml
[2015-05-26 18:05:45,889]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Deploying spark script: sample_script.xml for tenant : -1234
[2015-05-26 18:05:45,932]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Deployed Carbon Application : DASTestCApp_1.0.0 {super-tenant}

  4.   Click Main , then click Carbon Applications, and then click List view, to see the uploaded Carbon application as shown below.  

  5. If you want to download a Carbon application, click Download as shown below.
     

    When SSO is enabled for WSO2 DAS 3.0.1, it is not possible to download Carbon applications by default. If you want to download a Carbon application while SSO is enabled, do the following configurations.

    • Comment the following ServerURL entry in the <DAS_HOME>/repository/conf/carbon.xml file as shown below.

      <!--ServerURL>local:/${carbon.context}/services/</ServerURL-->

    • Uncomment the following entry in the same file.

      <ServerURL>https://${carbon.local.ip}:${carbon.management.port}${carbon.context}/services/</ServerURL>

  6. Click on the Delete option to delete the Carbon application as shown below.

    Also, this removes the Carbon application within 10 seconds, and you view below logs in the server side during the successful undeployment.


    [2015-05-26 18:06:46,024]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Undeploying Carbon Application : DASTestCApp_1.0.0...
[2015-05-26 18:06:46,030]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream Definition was undeployed successfully : org.wso2.test_1.0.0.json
[2015-05-26 18:06:46,033]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver undeployed successfully : TextWso2EventReceiver.xml
[2015-05-26 18:06:46,034]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event receiver deployment held back and in inactive state :TextWso2EventReceiver.xml, Stream validation exception :Stream org.wso2.test:1.0.0 does not exist
[2015-05-26 18:06:46,034]  INFO {org.wso2.carbon.event.receiver.core.internal.CarbonEventReceiverService} -  Event receiver : TextWso2EventReceiver in inactive state because event stream dependency  could not be found : org.wso2.test:1.0.0
[2015-05-26 18:06:46,035]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver undeployed successfully : TextWso2EventReceiver.xml
[2015-05-26 18:06:46,035]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Undeploying analytics event store : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/Eventstore_1.0.0/ORG_WSO2_TEST.xml
[2015-05-26 18:06:46,036]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Undeployed successfully analytics event store : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/Eventstore_1.0.0/ORG_WSO2_TEST.xml
[2015-05-26 18:06:46,036]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Undeploying spark script : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/sparkscripts_1.0.0/sample_script.xml for tenant id : -1234
[2015-05-26 18:06:46,042]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Undeployed Carbon Application : DASTestCApp_1.0.0 {super-tenant}
com.atlassian.confluence.content.render.xhtml.migration.exceptions.UnknownMacroMigrationException: The macro 'next_previous_links2' is unknown.