Releasing WSO2 Git Repositories

WSO2 Carbon Kernel is the base platform for all WSO2 products. With the new version of carbon kernel - 4.3.0, WSO2 has moved to the Git-based source code configuration management (SCM). All the code base is now moved to the WSO2 user account in github.

When the source code was moved from svn to github, the code base that was in svn had to be refactored into separate individual repositories. With git, everything is based on branches. So the above branching strategy using directories will not work. Therefore, the chunk based release of products has to be changed. For this reason, the platform repository was divided into multiple, individual git repositories by grouping common components. For example, carbon-deployment repository has grouped all components (including service stubs, components and features) related to the deployment of artifacts (webapps, axis2services). These individual repositories can then be released separately and the products depending on these repositories will be released afterwards.

Example: WSO2 Application Server depends on the following individual repositories:

• carbon4-kernel
  
• carbon-utils
• carbon-deployment
• carbon-commons
• carbon-multitenancy
• carbon-qos
• carbon-identity
• carbon-platform-integration-utils
• carbon-registry
• jaggery
• carbon-governance

Apart from the above, moving to git also has some added advantages:

1. Better management for each git repository (based on branches).

2. External contribution can be made easy with better code visibility.

3. Tools such as the maven-release-plugin and nexus staging repositories can be used easily.

Steps for releasing Git repositories

With the git based model, releases can be made easy using the maven-release-plugin and nexus staging repository. The following are the common guidelines for releasing from any git repository under a WSO2 user.

The following guidelines refer to carbon4-kernel as the sample project being released from git:

1. Create a “Repository Target” in http://maven.wso2.org/nexus/ that matches the groupID of the project and add a “Pattern Expression”. This pattern expression is used by nexus to automatically determine the staging profile. Shown below are the values used when creating a "Repository Target" for the carbon4-kernel project in git.
   
   Name: org.wso2.carbon
   Repository Type: Maven2
   Pattern Expression: .*/org/wso2/carbon/.*
   
   The main reason for creating a separate staging profile and repository target is for nexus to uniquely identify artifacts belonging to a staging profile. It uses the "groupID" of the artifacts. Nexus uses pattern matching for this purpose as explained above.

2. Create a nexus “Staging Profile” for the project in http://maven.wso2.org/nexus/, if it is not already created. The name of the profile should match the project's groupID. For example, the name of the profile for the carbon4-kernel project should be org.wso2.carbon.

   1. Select the "Repository Target" that was created in step 1 above as the "Repository Target" for this profile.

   2. Select the “Releases” repository in http://maven.wso2.org/nexus/content/repositories/releases/ as the “Release Repository” for all staging profiles.

3. Update the project parent pom with the correct SCM configuration as shown below.

   <scm>
      <url>https://github.com/wso2/carbon4-kernel.git</url>
      <developerConnection>scm:git:https://github.com/wso2/carbon4-kernel.git</developerConnection>
      <connection>scm:git:https://github.com/wso2/carbon4-kernel.git</connection>
      <tag>HEAD</tag>
   </scm>

4. Update project parent pom with the following plugins:

   1. Update the project root pom with the WSO2 Master parent pom as shown below. This pom holds all the common things that are used in almost all the repositories, such as distributionManagement, pluginManagement, repositories, pluginRepositories, etc.

      <parent>
          <groupId>org.wso2</groupId>
          <artifactId>wso2</artifactId>
          <version>1</version>
      </parent> 

   2. Add the given plugins to your build section. The versions of these will be inherited from the above parent pom.

      <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-release-plugin</artifactId>
             <configuration>
                 <preparationGoals>clean install</preparationGoals>
             </configuration>
         </plugin>
         <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-deploy-plugin</artifactId>
         </plugin>

   Note: You can add the autoVersionSubmodules configuration parameter to release the plugin configuration section, which will automatically version the sub modules. However, please note that this will cause issues with versioning if your project has an orbit sub module. This is because, for orbit modules, we follow a different versioning convention.

5. Update the project distribution management release repository as shown below. This step is not mandatory if the section on repositories is inherited from the WSO2 Parent Pom.

   <repository>
          <id>nexus-releases</id>
          <name>WSO2 Release Distribution Repository</name>      
   	   <url>http://maven.wso2.org/nexus/service/local/staging/deploy/maven2/</url>
        </repository>

6. Add a server config element in the maven configuration ($MVN_HOME/conf/settings.xml) for the nexus-releases server configuration given above. These are the nexus user credentials that will be used for remote artifact deployment.
   

   <server>
      <id>nexus-releases</id>
      <username>deployment</username>
      <password>deployment123</password>
   </server>

   Note: For the above step, you can request WSO2 Infra to create a user for the project in nexus.

7. Add another server config element that stores the SCM related credentials. This is an optional step, but will be useful to hide your SCM credentials when using the mvn-release-plugin.

   <server>
      <id>scm-server</id>
      <username>username</username>
      <password>password</password>
   </server>

   After adding the above, you have to update the parent pom properties section of your project with the following property: “project.scm.id”.
   

   <properties>
       <project.scm.id>scm-server</project.scm.id>
   </properties>

8. Create a git release branch from the master. The branch name would be "release-<release-version>".

   git checkout -b release-<release-version> master

9. Maven release plugin does not update some properties that we use, such as the osgi import and export versions. These properties also have the “SNAPSHOT” part in it. This has to be manually updated before performing the release preparation command. Also make sure that the project does not have any SNAPSHOT dependencies and update those with released versions. If there are any unreleased SNAPSHOT dependencies, we will have to release them separately. This will anyway be checked by the release plugin during the release:prepare stage. To test the above, we can use the “dryRun” option with the maven release plugin.

10. Issue the following release preparation command: mvn release:clean release:prepare. 
    Note: Please use an appropriate user name for the maven build. The build artifacts will have this username in its MANIFEST file. Give appropriate values for the release, development, and tag versions as shown below when prompted for the release preparation command.

    [INFO] Checking dependencies and plugins for snapshots …
    What is the release version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.0: : 4.3.0
    What is SCM release tag or label for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) carbon-4.3.0: : 4.3.0
    What is the new development version for "WSO2 Carbon Kernel"? (org.wso2.carbon:carbon-kernel) 4.3.1-SNAPSHOT: : 4.4.0-SNAPSHOT

11. Issue the release perform command: mvn release:perform 

12. When the above process succeeds, the artifacts will be deployed to a staging repo. The newly created staging repo will not be closed automatically when the artifacts are uploaded. This can be done through the release-manager. That is, by logging into the nexus UI the repo can be manually closed. When a staging repo is closed, it becomes available for public access. Eg: http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-037/

13. If there is a failure, the prepared release process can be rolled back using the following command: mvn release:rollback. 
    This will revert all the commits made during the preparation process. 

14. With the staging repo in effect, a release candidate VOTE should be called on dev@wso2.org using the template given below. This VOTE is essential for a product release. For other projects, this is optional.

    Subject : [VOTE] Release <Project Name> <Project Version> <RC #>

    <BEGIN>

    This is the <RC #> release candidate of <Project Name> <Project Version>

    Eg : WSO2 Carbon Kernel 4.3.0 rc1

    This release fixes the following issues:

    <URL to the fixed jira list>

    Please download, test and vote. Please refer the release verification guide for detailed information on verifying this release.

     

    Source & binary distribution files:

    <URL to the source and binary files>

     

    Maven staging repo:

    <URL to the maven nexus staging repo>

    Eg: http://maven.wso2.org/nexus/content/repositories/orgwso2carbon-1000/

     

    The tag to be voted upon:

    <URL to the release tag location>

    Eg: https://github.com/wso2/carbon4-kernel/releases/tag/carbon4-kernel-4.3.0-rc1

     

    KEYS file containing PGP keys we use to sign the release:

    <URL to the Keys used with signing the artifacts>

     

    Release verification guide:

    <If any>

     

    [ ] Broken - do not release (explain why)

    [ ] Stable - go ahead and release

    </BEGIN>

15. A release VOTE should be kept open for 72 hours. During this period, the developers should test the artifacts and then vote. When there are at least 3 binding +1 votes and no -1 votes, the vote is considered as a pass. Once the release vote is completed (the artifacts are tested and verified), the staging repo can be released, which will make the artifacts available in the public maven repo. Note that this should be done by the release manager. The released artifacts will be available in the WSO2 Releases maven repository at : http://maven.wso2.org/nexus/content/repositories/releases/

16. If the vote failed, then the staging repository should be dropped and the changes for the release branch should be reverted. This process should be started again from #9 onwards, after fixing the issues mentioned during the vote.