Discovering the unknown - IBM WAS Continuous Integration Process

1 Preface
IBM has not spent too much effort in publishing its continuous integration capabilities but this should not be underestimated for lack of support for continuous integration. IBM provides this with extensive support for Ant by a good set of Ant API calls. This is available in IBM’s online documentation.

This paper is first among three papers detailing continuous integration for IBM product WAS (Websphere Application Server). This, the first paper, details on setting up Ant for deployment


2 Prerequisites
1. IDE : RAD 7.0.0 (Rational Application Developer)
2. Server suite name : WebSphere Application Server 6.1.0.11
3. Ant version : Apache Ant 1.6.5 (The one installed with RAD)
4. Knowledge of RAD/WAS
5. Basic knowledge of ANT


3 Project Environment
The sample application that would be used to demo the project is XYZ project. The project hierarchy is as follows –
Business – The project that contains the business logic
DAO – The data layer access project
XYZ – RAD generated which contains the deployment descriptor
XYZWeb – The web project.


4 Requirement
The requirement is write ant scripts which would do help accomplish the following tasks needed for deployment. The actual code corresponding with each of the following sections can be found at section 6(Rendered Ant Script)
1. INSTALL_FRESH_APPLICATION_FROM_RAD
2. INSTALL_FRESH_APPLICATION
3. UNINSTALL_APPLICATION
4. RE-INSTALL_APPLICATION

4.1 INSTALL_FRESH_APPLICATION_FROM_RAD
One of the easiest ways to test the effectively of the post ear creating tasks is to use RAD’s export feature to create the ear and use that to check if all the other tasks run properly. During the time that this paper was written a 404 error from the application was assumed to be due to improper application start by Ant but a run of the Ant application start using the ear generated helped pinpoint the issue to improper ear creation.
4.2 INSTALL_FRESH_APPLICATION
The primary assumption for all the tasks is that WAS server is already running, else the separate ant task can be called to start the application server
4.3 UNINSTALL_APPLICATION
Uninstalls and restarts the application server
4.4 RE-INSTALL_APPLICATION
Assuming that application that we are going to install is already running the app server this ant task reinstalls it with the updated code available in the repository.


5 RAD/WAS specifics
5.1 Ant script issue
As mentioned in 7.1.1, since WebSphere ant tasks are not profile aware the tasks needs to be run from the ant folder in 5.9.4
5.2 Build problem for cyclic compilation
Ant builds also is not aware of the import locations and needs view to the corresponding compiled class files. In the sample application listed here, DAO and Business projects are closely coupled. While developing the application, DAO was added to the Business project’s build path by (Right Click on project) Build Path->Configure Build Path->Libraries->Add Class Folder – Add DAO source folders. Correspondingly, since Business project is dependant on DAO project we need to write the ant scripts to compile the DAO project and then add it to the classpath of the Business project’s ant javac compilation. The underlying message is that Ant needs to be specifically told of the cyclic compilation required and can be based on how the application’s build path is configured.
5.3 WAS server view issue
The server view from all perspectives would not reflect the deployed application. The only way to verify the deployment would be run the admin console (right click from the application server in server view). Applications->Enterprise Applications would list out the deployed application using Ant. Another way to verify the deployment is detailed in 5.4
5.4 WAS configuration issue when application is not present
Generally since the same workspace to do development and test the runs for ant might cause applications to conflict while deployment. As a result applications that might be undeployed/deployed through the server view in RAD might conflict with the installation done through Ant. This can be resolved by comparing the application present in the folder 5.9.2 and comparing it with 5.9.3 for the related entries in tag.
5.5 Ant ear diff with RAD ear
One clear cut way to check for correctness of the script or rather to fix issues like class not found is to generate the ear using Ant’s export and then running 4.1. If this runs fine and it usually should since RAD is generating the ear.
The generated ear from RAD can be compared with the ear generated by the ant script. There would definitely be difference during the initial runs of the Ant ear generation scripts. This can be resolved by comparing the files and calling the appropriate ant task. I would like to specifically mention the copy task since the issues usually revolve around improper placement of the META-INF directory , the directory not having complete contents or the classes for all the projects are not nested right.
5.6 Ant classpath issue
Some of the class path issues can be resolved with the help of the logs which is available as mentioned in 5.9.1
5.7 RAD Generated Ant
RAD also generates Ant scripts. This might be a good starting point to write your own scripts. The only problem being that the scripts are too convoluted and involves fine grained class path listing and cyclic target calls.
5.8 404 Error
This is a sample scenario based on the troubleshooting mechanism discussed in 5.5. If the application starts up but gives a 404 exception it’s time to start ripping apart the ear. Using a unzip utility break open the ear generated from RAD and replace the contents of the ear generated by Ant (assuming that the ear generated by RAD works. It should!!). This should eventually lead you to the problem.
For example, a 404 error occurs but the application runs fine when Ant scripts were used to start the application from the ear generated by RAD. By swapping the contents of the ear one at a time, it would found that the MANIFEST file was not filled in right which caused a 404 exception. The resulting differences were -
Ant generated manifest file
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.6.5
Created-By: 2.3 (IBM Corporation)
RAD generated manifest file
Manifest-Version: 1.0
Class-Path: Business.jar DAO.jar
As seen above the manifest file has the classpath attributes missing. Hence the script has been updated(see Rendered Ant Script) to include the Manifest attribute declarations.
More of a brute force method, to troubleshoot the issue but always effective.
5.9 Important paths
The paths mentioned below are for a sample WAS/RAD application setup for the project. The node names would vary but apart from that the path would almost be the same if the default paths of the installation for RAD/WAS are followed.
5.9.1 C:\Program Files\IBM\SDP70\runtimes\base_v61\profiles\AppSrv01\logs\ffdc\
5.9.2 C:\Program Files\IBM\SDP70\runtimes\base_v61\profiles\AppSrv01\config\cells\01hw081678Node01Cell\applications
5.9.3 C:\Program Files\IBM\SDP70\runtimes\base_v61\profiles\AppSrv01\config\cells\01hw081678Node01Cell\nodes\01hw081678Node01\serverindex.xml
5.9.4 C:\Program Files\IBM\SDP70\runtimes\base_v61\profiles\AppSrv01\bin>


6 Rendered Ant Script

<?xml version="1.0"?>
<project name="project" default="default" basedir=".">
 <description>  
Build/Deploy an EAR to WebSphere Application Server 6.1 (IDE used is RAD 7.0.0)  
</description>
 <property name="was_home" value="C:/Program Files/IBM/SDP70/runtimes/base_v61/" />
 <path id="was.runtime">
  <fileset dir="${was_home}/lib">
   <include name="**/*.jar" />
  </fileset>
  <fileset dir="${was_home}/plugins">
   <include name="**/*.jar" />
  </fileset> 
  <fileset dir="C:/Developer/Java_Develop/workspace/XYZ/XYZWeb/WebContent/WEB-INF/lib">
   <include name="**/*.jar" />
  </fileset>
 </path>
 <property name="was_cp" value="${toString:was.runtime}" />
 <property environment="env" />
 <property name="ear" value="C:/Developer/Java_Develop/workspace/XYZ/build/XYZ.ear" />

 <!-- If you need to configure a default task. Call it as an argument for depends -->
 <target name="default" depends="">
 </target>

 <target name="INSTALL_FRESH_APPLICATION" 
  depends="clean,generateEar,installApplication,stopServer,startServer,startApplication">
 </target>

 <!-- Assuming that a ear is manually created using RAD's export feature 
 and the ear is placed at the required location-->
 <target name="INSTALL_FRESH_APPLICATION_FROM_RAD" 
  depends="installApplication,stopServer,startServer,startApplication">
 </target>

 <target name="UNINSTALL_APPLICATION" 
  depends="stopApplication,uninstallApplication,stopServer,startServer">
 </target>

 <target name="RE-INSTALL_APPLICATION" depends="UNINSTALL_APPLICATION,INSTALL_FRESH_APPLICATION">
 </target>

 <target name="clean">
  <delete includeEmptyDirs="true">
   <fileset dir="../../build" />
  </delete>
  <mkdir dir="../../build/earbin/" />
  <mkdir dir="../../build/BusinessScratch" />
  <mkdir dir="../../build/DAOScratch" />
 </target>

 <target name="compileDAOClasses">
  <!--<echo message="was_cp=${was_cp}" />-->
  <javac srcdir="../../DAO/src" destdir="../../build/DAOScratch">
   <classpath>
    <pathelement path="${was_cp}" />
   </classpath>
  </javac>
  <copy includeemptydirs="false" todir="../../build/DAOScratch">
   <fileset dir="../../DAO/src" excludes="**/*.launch, **/*.java" />
  </copy>
 </target>

 <target name="compileBusinessClasses" depends="compileDAOClasses">
  <javac srcdir="../../Business/src" destdir="../../build/BusinessScratch" classpath="${was_cp}">
   <classpath>
    <pathelement path="${was_cp}" />
    <pathelement location="../../build/DAOScratch" />
   </classpath>
  </javac>
  <copy includeemptydirs="false" todir="../../build/BusinessScratch">
   <fileset dir="../../Business/src" excludes="**/*.launch, **/*.java" />
  </copy>
 </target>

 <target name="generateBusinessjar" depends="compileBusinessClasses">
  <jar destfile="../../build/earbin/Business.jar" basedir="../../build/BusinessScratch" excludes="**/*.java.bak">
  </jar>
 </target>

 <target name="generateDAOjar" depends="generateBusinessjar">
  <jar destfile="../../build/earbin/DAO.jar" basedir="../../build/DAOScratch" excludes="**/*.java.bak">
  </jar>
 </target>


 <target name="compileWarClasses">
  <javac srcdir="." destdir="../WebContent/WEB-INF/classes" classpath="${was_cp}">
  </javac>
 </target>

 <target name="generateWar" depends="compileWarClasses">
  <mkdir dir="../WebContent/WEB-INF/classes" />
  <copy includeemptydirs="false" todir="../WebContent/WEB-INF/classes">
   <fileset dir="." excludes="**/*.launch, **/*.java" />
  </copy>
  <jar destfile="../../build/XYZWeb.war">
      <manifest>
        <attribute name="Class-Path" value="Business.jar DAO.jar"/>
        <attribute name="Created-By" value="Developer"/>
      </manifest>   
   <fileset dir="../WebContent">
   </fileset>
  </jar>
 </target>

 <target name="generateEar" depends="generateDAOjar,generateWar">
  <mkdir dir="../../build/earbin/META-INF" />
  <move file="../../build/XYZWeb.war" todir="../../build/earbin" />
  <copy todir="../../build/earbin/META-INF">
   <fileset dir="../../XYZ/META-INF/" />
  </copy>
  <jar destfile="${ear}">
   <fileset dir="../../build/earbin" />
  </jar>
 </target>

 <target name="installApplication">
  <taskdef name="wsInstallApp" classname="com.ibm.websphere.ant.tasks.InstallApplication" classpath="${was_cp}" />
  <wsInstallApp ear="${ear}" failonerror="true" debug="true" washome="${was_home}" />
 </target>

 <target name="startApplication">
  <taskdef name="wsStartApplication" classname="com.ibm.websphere.ant.tasks.StartApplication" classpath="${was_cp}" />
  <wsStartApplication wasHome="${was_home}" server="server1" node="01hw081678Node01" application="XYZ" />
 </target>

 <target name="stopApplication">
  <taskdef name="wsStopApplication" classname="com.ibm.websphere.ant.tasks.StopApplication" classpath="${was_cp}" />
  <wsStopApplication wasHome="${was_home}" server="server1" node="01hw081678Node01" application="XYZ" />
 </target>

 <target name="uninstallApplication">
  <taskdef name="wsUninstallApp" classname="com.ibm.websphere.ant.tasks.UninstallApplication" classpath="${was_cp}" />
  <wsUninstallApp wasHome="${was_home}" application="XYZ" />
 </target>

 <target name="stopServer">
  <taskdef name="wsStopServer" classname="com.ibm.websphere.ant.tasks.StopServer" classpath="${was_cp}" />
  <wsStopServer server="server1" nowait="false" quiet="false" replaceLog="true" trace="true" timeout="3000" wasHome="${was_home}" failonerror="true" />
 </target>

 <target name="startServer">
  <taskdef name="wsStartServer" classname="com.ibm.websphere.ant.tasks.StartServer" classpath="${was_cp}" />
  <wsStartServer server="server1" nowait="false" quiet="false" replaceLog="true" trace="true" timeout="3000" wasHome="${was_home}" failonerror="true" />
 </target>
</project>

7 References
7.1.1 http://www-01.ibm.com/support/docview.wss?uid=swg1PK07628
7.1.2 http://ant.apache.org/manual/CoreTasks/jar.html
7.1.3 http://stackoverflow.com/questions/4046/can-someone-give-me-a-working-example-of-a-build-xml-for-an-ear-that-deploys-in-w
7.1.4 http://www.ibm.com/developerworks/websphere/library/techarticles/0505_weisz/0505_weisz.html
7.1.5 Rational Application Developer V7 Programming Guide