com.atlassian.confluence.content.render.xhtml.migration.exceptions.UnknownMacroMigrationException: The macro 'next_previous_links' is unknown.

Packaging Artifacts as a C-App Archive

Owned by Former user (Deleted)
Last updated: Mar 02, 2017 by Former user (Deleted)
7 min read

C-App namely Carbon Application is an archive format collection of different artifacts bundled to a single deployable component. C-App files have CAR extensions and can be deployed to different runtimes. Each runtime will only deploy the artifacts which match with the role that the runtime is playing.

Supported C-App types

Given below are the type of the carbon applications artifacts that are supported by WSO2 DAS.

Generic artifacts

ArtifactType
Event Streamsevent/streams
Event Receiversevent/receiver
Event Publishersevent/publisher

Real time analytics specific artifacts

ArtifactType
Execution Planevent/execution-plan

Batch analytics specific artifacts

ArtifactType
Event storesanalytics/eventstore
Spark scriptsanalytics/spark
Data purging scriptsanalytics/dataPurging

Visualization artifacts

ArtifactType
Dashboardsdashboards/dashboard
Layoutsdashboards/layout
Gadgetsdashboards/gadget

If the above mentioned artifacts are being deployed using a C-App, you are restricted on editing or deleting them. Therefore, if you need to edit the C-App, you need to re-pack and re-deploy it.

Creating a C-App

Follow the steps below to create a C-App to be deployed in WSO2 DAS. In this below section, we are going to focus on adding the carbon application with 5 different kind of artifact event streams, event receiver, event publisher, event store, and Spark scripts.

├── artifacts.xml
├── Eventreceiver_1.0.0
│   ├── artifact.xml
│   └── TestWso2EventReceiver.xml
├── EventPublisher_1.0.0
	├──artifact.xml
	└──TestLoggerPublisher.xml
├── Eventstore_1.0.0
│   ├── artifact.xml
│   └── ORG_WSO2_TEST.xml
├── Eventstream_1.0.0
│   ├── artifact.xml
│   └── org.wso2.test_1.0.0.json
└── Sparkscripts_1.0.0
|	├── artifact.xml
|	└── sample_script.xml
└── DataPurging_1.0.0
	├── artifact.xml
	└──data_purging.xml
└── Dashboard_1.0.0
|	├── artifact.xml
|	└── appman.json
|── Gadget_1.0.0
|	├── artifact.xml
|	└── Test_Gadget
|── Layout_1.0.0
|	├── artifact.xml
|	└── Test_Layout

  1. Create the top level artifacts.xml file of the C-App which defines the set of folders included in it as shown below.

    <?xml version="1.0" encoding="UTF-8"?><artifacts>
	<artifact name="DASTestCApp" version="1.0.0" type="carbon/application">
    	<dependency artifact="Eventstore" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Eventreceiver" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
		<dependency artifact="EventPublisher" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Eventstream" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
    	<dependency artifact="Sparkscripts" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
		<dependency artifact="DataPurging" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Dashboard" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Gadget" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
        <dependency artifact="Layout" version="1.0.0" include="true" serverRole="DataAnalyticsServer"/>
	</artifact>
</artifacts>

  2. Create separate directories in the top level of the C-App for the above mentioned dependencies (Eventreceiver_1.0.0, EventPublisher_1.0.0, Eventstore_1.0.0, Eventstream_1.0.0, Sparkscripts_1.0.0 and Data_Purging_1.0.0) defined in the artifacts.xml file.

    You can have multiple dependencies as required with a directory for each of them in the same level as the artifacts.xml file in the C-App. Include the name and the version of the artifact in the name of the directory.

  3.  Create an artifact.xml file inside all dependency directories as follows.

    Eventstore_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?>
<artifact name="Eventstore" version="1.0.0" type="analytics/eventstore" serverRole="DataAnalyticsServer">
	<file>ORG_WSO2_TEST.xml</file>
</artifact> 

    Eventstream_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?><artifact name="Eventstream" version="1.0.0" type="event/stream" serverRole="DataAnalyticsServer">
	<file>org.wso2.test_1.0.0.json</file>
</artifact>

    Eventreceiver_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?><artifact name="Eventreceiver" version="1.0.0" type="event/receiver" serverRole="DataAnalyticsServer">
	<file>TextWso2EventReceiver.xml</file>
</artifact>

    EventPublisher_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?>
<artifact name="EventPublisher" version="1.0.0" type="event/publisher" serverRole="DataAnalyticsServer">
    <file>TestLoggerPublisher.xml</file>
</artifact>

    Sparkscripts_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?><artifact name="Sparkscripts" version="1.0.0" type="analytics/spark" serverRole="DataAnalyticsServer">
	<file>sample_script.xml</file>
</artifact>

    DataPurging_1.0.0 artifact.xml

     

    <artifact name="DataPurging" version="1.0.0" type="analytics/dataPurging" serverRole="DataAnalyticsServer">
    <file>data_purging.xml</file>
</artifact>

     

    Dashboard_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?>
<artifact name="Dashboard" version="1.0.0" type="dashboards/dashboard" serverRole="DataAnalyticsServer">
    <file>appman.json</file>
</artifact>

    Gadget_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?>
<artifact name="Gadget" version="1.0.0" type="dashboards/gadget" serverRole="DataAnalyticsServer">
    <file>Test_Gadget</file>
</artifact>

    Layout_1.0.0 artifact.xml

    <?xml version="1.0" encoding="UTF-8"?>
<artifact name="Layout" version="1.0.0" type="dashboards/layout" serverRole="DataAnalyticsServer">
    <file>Test_Layout</file>
</artifact>

  4. Create the actual artifact which was specified in the artifact.xml above. For example, you need to create an event store configuration named ORG_WSO2_TEST.xml, an event stream named org.wso2.test_1.0.0.json, an event receiver configuration named TextWso2EventReceiver.xml, a Spark script configuration named  sample_script.xml, a dashboard configuration named appman.json, a gadget configuration named gadget.json, and a layout configuration named layout.json. See below for each of the configuration files.

    Stream - org.wso2.test_1.0.0.json

    {
  "name": "org.wso2.test",
  "version": "1.0.0",
  "nickName": "Test Stream",
  "description": "A test stream",
  "metaData": [
	{
  	"name": "remote_host",
  	"type": "STRING"
	},
	{
  	"name": "tenant_id",
  	"type": "INT"
	}
  ],
  "correlationData": [
	{
  	"name": "activity_id",
  	"type": "STRING"
	}
  ],
  "payloadData": [
	{
  	"name": "operation_name",
  	"type": "STRING"
	},
	{
  	"name": "service_name",
  	"type": "STRING"
	}
  ]
}

    Event Store - ORG_WSO2_TEST.xml

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<EventStoreConfiguration>
<Source>
       <StreamId>org.wso2.test:1.0.0</StreamId>
   </Source>
   <TableSchema>
         <ColumnDefinition>
           <Name>meta_remote_host</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>meta_tenant_id</Name>
           <EnableIndexing>false</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>INTEGER</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>correlation_activity_id</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>FACET</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>operation_name</Name>
           <EnableIndexing>true</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
       <ColumnDefinition>
           <Name>service_name</Name>
           <EnableIndexing>false</EnableIndexing>
           <IsPrimaryKey>false</IsPrimaryKey>
           <EnableScoreParam>false</EnableScoreParam>
           <Type>STRING</Type>
       </ColumnDefinition>
   </TableSchema>
   </EventStoreConfiguration>

    Event receiver - TestWso2EventReceiver.xml

     <?xml version="1.0" encoding="UTF-8"?>
<eventReceiver name="TestWso2EventReceiver" statistics="disable"
	trace="disable" xmlns="http://wso2.org/carbon/eventreceiver">
	<from eventAdapterType="wso2event">
    	<property name="events.duplicated.in.cluster">false</property>
	</from>
	<mapping customMapping="disable" type="wso2event"/>
	<to streamName="org.wso2.test" version="1.0.0"/>
</eventReceiver>

    Event publisher - TestLoggerPublisher.xml

     

     

     
    <?xml version="1.0" encoding="UTF-8"?>
<eventPublisher name="TestLoggerPublisher" statistics="disable"
  trace="disable" xmlns="http://wso2.org/carbon/eventpublisher">
  <from streamName="org.wso2.test" version="1.0.0"/>
  <mapping customMapping="disable" type="json"/>
  <to eventAdapterType="logger">
    <property name="uniqueId">TestLoggerPublisher</property>
  </to>
</eventPublisher>
     

     

     

    Spark script - sample_script.xml

     

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Analytics>
	<Name>AddNewScriptTestWithouTask</Name>
	<Script>define table TEST_CAPP (server_name string, ip STRING, tenant INTEGER, sequence LONG, summary STRING);SELECT ip FROM TEST_CAPP;SELECT server_name, count(*) FROM TEST_CAPP GROUP BY server_name;</Script>
	<CronExpression>0 * * * * ?</CronExpression>
</Analytics>

    Dashboard - appman.json

     

    {"id":"appman","title":"appman","description":"","permissions":{"viewers":[],"editors":["Internal/everyone"]},"pages":[{"id":"landing","title":"My Dashboard","layout":{"id":"layout-3","title":"Layout 3","description":"This is a sample grid","thumbnail":"local://store/layout/layout-3/index.jpg","url":"local://store/layout/layout-3/index.hbs","content":"\n   <div class=\"row\">\n        <div id=\"a\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"b\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"c\" class=\"col-md-4 ues-component-box\"></div>\n    </div>\n    <div class=\"row\">\n        <div id=\"d\" class=\"col-md-10 ues-component-box\"></div>\n        <div id=\"e\" class=\"col-md-2 ues-component-box\"></div>\n    </div>\n    <div class=\"row\">\n        <div id=\"f\" class=\"col-md-4 ues-component-box\"></div>\n        <div id=\"g\" class=\"col-md-4 ues-component-box\"></div>\n \t<div id=\"h\" class=\"col-md-4 ues-component-box\"></div>\n    </div>\n\n"},"content":{"d":[{"id":"24w6ukj9olz69a4i","content":{"id":"Temperature_By_City","title":"Temperature By City","type":"gadget","thumbnail":"local://store/gadget/usa-business-revenue/index.png","data":{"url":"local://store/gadget/Temperature_By_City/index.xml"},"styles":{"title":"Temperature By City","borders":true},"options":{"dataSource":{"type":"STRING","title":"Data Source","value":"/portal/gadgets/bar-chart/datasource/dataFile4.jag","options":[],"required":false},"updateGraph":{"type":"STRING","title":"Update Interval (s)","value":"No","options":[],"required":false}}}}]}}],"landing":"landing"}

    Gadget - gadget.json

     

    {"id":"Test_Gadget","title":"Temperature By City","type":"gadget","thumbnail":"local://store/gadget/usa-business-revenue/index.png","data":{"url":"local://store/gadget/Temperature_By_City/index.xml"}}

    Laoyut - layout.json

    {
    "id": "Test_Layout",
    "title": "Test_Layout",
    "description": "This is a sample grid",
    "thumbnail": "local://store/layout/layout-3/index.jpg",
    "url": "local://store/layout/layout-3/index.hbs"
}

Deploying a C-App

Follow the steps below to deploy a C-App.

  1. Log in to WSO2 CEP/DAS management console using admin/admin credentials.

  2. Click Main, and then click Add in the Carbon Applications menu.

  3. Click Choose File as shown below.
    It will take few seconds up to 10 seconds by default to deploy. You view below logs in the CEP/DAS server.

     

    [2015-05-26 18:05:44,541]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Deploying Carbon Application : DASTestCApp.car...
[2015-05-26 18:05:45,831]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.test:1.0.0
[2015-05-26 18:05:45,849]  INFO {org.wso2.carbon.event.input.adapter.core.internal.CarbonInputAdapterRuntime} -  Connecting receiver TextWso2EventReceiver
[2015-05-26 18:05:45,852]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.test:1.0.0
[2015-05-26 18:05:45,853]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state : TextWso2EventReceiver
[2015-05-26 18:05:45,853]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Deploying analytics event store :ORG_WSO2_TEST.xml
[2015-05-26 18:05:45,885]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  WSO2EventConsumer added to the junction. Stream:org.wso2.test:1.0.0
[2015-05-26 18:05:45,886]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Deployed successfully analytics event store :ORG_WSO2_TEST.xml
[2015-05-26 18:05:45,889]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Deploying spark script: sample_script.xml for tenant : -1234
[2015-05-26 18:05:45,932]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Deployed Carbon Application : DASTestCApp_1.0.0 {super-tenant}

  4.  Click Main, then click Carbon Applications, and then click List view, to see the uploaded Carbon application as shown below. 

  5. Click on the Delete option to delete the Carbon application as shown below.

    Also, this removes the Carbon application within 10 seconds, and you view below logs in the server side during the successful undeployment.

     

    [2015-05-26 18:06:46,024]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Undeploying Carbon Application : DASTestCApp_1.0.0...
[2015-05-26 18:06:46,030]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream Definition was undeployed successfully : org.wso2.test_1.0.0.json
[2015-05-26 18:06:46,033]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver undeployed successfully : TextWso2EventReceiver.xml
[2015-05-26 18:06:46,034]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event receiver deployment held back and in inactive state :TextWso2EventReceiver.xml, Stream validation exception :Stream org.wso2.test:1.0.0 does not exist
[2015-05-26 18:06:46,034]  INFO {org.wso2.carbon.event.receiver.core.internal.CarbonEventReceiverService} -  Event receiver : TextWso2EventReceiver in inactive state because event stream dependency  could not be found : org.wso2.test:1.0.0
[2015-05-26 18:06:46,035]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver undeployed successfully : TextWso2EventReceiver.xml
[2015-05-26 18:06:46,035]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Undeploying analytics event store : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/Eventstore_1.0.0/ORG_WSO2_TEST.xml
[2015-05-26 18:06:46,036]  INFO {org.wso2.carbon.analytics.eventsink.AnalyticsEventStoreDeployer} -  Undeployed successfully analytics event store : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/Eventstore_1.0.0/ORG_WSO2_TEST.xml
[2015-05-26 18:06:46,036]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Undeploying spark script : /home/sinthuja/projects/my-git-repo/wso2/product-bam/modules/distribution/target/wso2das-3.0.0-SNAPSHOT/tmp/carbonapps/-1234/1432643744542DASTestCApp.car/sparkscripts_1.0.0/sample_script.xml for tenant id : -1234
[2015-05-26 18:06:46,042]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Undeployed Carbon Application : DASTestCApp_1.0.0 {super-tenant}
com.atlassian.confluence.content.render.xhtml.migration.exceptions.UnknownMacroMigrationException: The macro 'next_previous_links2' is unknown.