Developing Desktop AstroGrid

This page documents how to work with and develop DesktopAstroGrid.

Assembling the environment

Pre-requisites

JDK 1.5.

All the code is written to work with Java v1.5 or higher. We've recently dropped support for Java 1.4

Maven v1.0.2

Integrated build system.

• Set environment variable $JAVA_HOMEto point to the installation of JDK1.5
• You might find it helpful to set $JAVA_VERSION=1.5
• Set environment variable $MAVEN_HOMEto point to installation directory of maven.
• You may wish to increase the memory maven runs with. create a '~/.mavenrc' which contains the line MAVEN_OPTS="-Xmx1024m"
• A final release (v 1.1)of maven 1.x came out recently, but I've not tested our build against this yet - so stick with v1.0.2for now.

Getting access to the source

You need read & write permissions to the astrogrid CVS. If you don't already have this, email gg78@astrogrid.org

Checking out the source

The maven build system expects various projects to be in certain places, relative to each other.

1. Choose a base directory - I'll refer to this as $BASE
2. Check out the CVS project astrogrid/maven-baseinto $BASE/maven-base. This contains some global project definitions.
3. Check out the CVS project astrogrid/desktopinto $BASE/desktop. This contains the tree of desktop projects.

The following incantation for unix will leave all the sources in a $BASEdirectory called astrogrid- assuming you've got an account named bobon cvs.astrogrid.org and the correct ssh keys.

% cd ~
% cvs -d :ext:bob@cvs.astrogrid.org:/devel checkout -P
astrogrid/maven-base
% cvs -d :ext:bob@cvs.astrogrid.org:/devel checkout -P
astrogrid/desktop

If you're using Eclipse, see EclipseCVSSupport

---

Maven

Maven has the concept of a build project- which is usually a set of sources which should result in a single artifact- a jar file, for example. Maven is a build system which knows how to do things - unlike, say, Ant, which needs to be told how to do things. It comes with a load of pre-defined goals- e.g. javadoc, jar, which can be run once maven has been told a few things about the project.

A project is rooted within a single directory, and is defined by 3 files

project.xml

A declarative definition of the project - where the source and test code is, the project name and description, and a list of libraries required to compile the project

project.properties

settings that can be used to tweak details of the pre-defined goals

maven.xml

Used to define new goals, and extend the pre-defined goals with additional pre- and post- actions. Here be dragons. It's written in a combination of Ant tasks, jelly tags, xslt, and whatever else takes your fancy.

The final complication is that a maven project can inheritdefinitions from another project. This is indicated at the top of the project.xmlby an extendelement. This allows things to be defined once, but does mean you need to go hunting to find where something is defined.

Maven also takes settings from a ~/build.properties- this is a good place to provide passwords anth other personal data. To build the desktop sources, your ~/build.propertiesshould contain the following

# Required when SCM attempts to contact the AG CVS repository
maven.username=<i>your cvs username</i>

Maven - First Run

These commands apply to any maven project - try cd $BASE/desktop/implto try them out

maven -h

lists help and exits

maven -g

lists all predefined goals and exits. You'll see an overwhelming amount of stuff here.

maven -u

displays the description of the project, and lists any goals defined in the local maven.xml. A good place to start.

maven clean

removes all build products. You'll see maven start to download a lot of dependencies at this point - it will take some time, but should complete with no errors.

Desktop Project Structure

There's two maven projects in the cvs sources you just downloaded - they inherit from a template.

Early on, we agreed that it'd be a good idea to have a naming conventionfor goals across all AstroGridprojects - the desktop projects follows these conventions when meaningful:

maven-base

contains common astrogrid wide configuration. Nothing of much interest

desktop/template

contains common desktop-wide definitions - in particular version numbers for the other astrogrid libraries desktop depends upon, and the version number for the desktop project itself.

desktop/api

The public AstroRuntimeAPI. See #API_Project_build. Goals:

• astrogrid-clean- clean this project
• astrogrid-install-artifact- compiles the API jar, and builds the API distribution zip (jars, documentation, example code) and copies the results to your local repository ( ~/.maven/repository) from where they can be accessed by the desktop/implproject.

desktop/impl

The implementation of AstroRuntime. Goals:

• astrogrid-clean- clean this project
• astrogrid-build-artifact- compiles, tests, jars, strips the jar & runs smoke tests, See #Implementation_project_build.

API Project build

The apiproject compiles the interface java sources into a .jar, and then bundles this into a distribution .zip, along with supporting libraries, API Javadoc, and examples of using the AR from python, perl, etc. The .jaris used within the implproject build, and the distribution .zipforms the development kit for the AstroRuntimeAPI.

Issuing the command maven astrogrid-install-artifactin the apiproject causes both .jarand .zipto be built and placed in the localrepository - from where they can be accessed from the implbuild.

Issuing the command maven astrogrid-deploy-artifactin the apiproject causes both .jarand .zipto be built and deployed to the public astrogrid repository. The documentation site for the project is also built and deployed to the public webserver.

API Descriptors

One important quirk of the build is how a description of the API is extracted from the Java sources. AstroRuntime needs to be told about the public API - which interfaces, methods, and parameter types - so it can expose them via RMI, XML-RPC and HTTP. Some of this information couldbe determined just by introspecting onto the API classes - but this would produce only type information. AstroRuntime requires further metadata about the API - such as what the XML-RPC component names to use for each Java interface - and function documentation.

To achieve this, the build follows the process illustrated on the left. Javadocis run over the Java sources, with a custom docletwhich outputs XML rather than the usual HTML documentation.

This results in a single XML file which describes the API. It contains all the information extracted by Javadoc - method signatures, interface names, documentation comments and @tags.

A stylesheet generates an XML descriptor file for each 'module' of the API. Each file is in the format of a hivemind contribution for the framework.descriptorsconfiguration point. This supplies all the metadata needed to publish and document the API within AstroRuntime.

These descriptor files are packaged into the api .jaralong with the API interfaces, which is then included in the implproject. The descriptors are included into the main hivemind module files by using external entity references.

This approach means that the definition and documentation of the API can be managed in a single place - the Java source - and it is extracted and propagated throughout the system. All the required metadata is expressed in Javadoc with some additional tags. The most important addition is a new tag - @service modulename.servicename which must be added to the class documentation comment for each API interface. This tag defines the name of the module and service that this interface will be exposed as in the XML-RPC and HTML interfaces.

For a full description of the set of javadoc tags used for documenting the API documentation (and providing figures and code examples), see Javadoc Markup Notes.

Because the javadoc comments feed into this pipeline, it is important that if they contain any HTML formatting, it is well-formed XHTML. Otherwise the api build will fail.

Implementation project build

The maven build for the implementation project runs through the following steps.

1. compile the sources
2. run the JUnit tests
3. produces a jar from the compiled sources
4. builds a distribution zip, useful for developers who wish to embed AstroRuntime in their own projects. This contains -
   • the compiled jar
   • all dependent libraries
   • HTML Javadoc
   • HTML-formatted x-referenced sources
   • Hivedoc - (description of the hivemind container configuration) - generated by running the application itself
   • configuration documentation - a documented list of configuration settings - generated by running the application itself.
5. Strip unused classes and methods from classes and libraries, and optimize the remaining code in the compiled jar and dependent libraries. The reduces the download size and memory consumption of the code.
6. Merge the stripped compiled classes and libraries into a single jar file - the application jar. This is a standalone jar that requires no other resources to run.
7. If the build is being done on a Mac, build a .dmg file. This is a packaging format convenient for Mac OS X users, and can be offered as a download from the relevant download page. If the build is being done on a non-Mac system, the files required will be assembled, and instructions will be issued on how to generate the actual .dmg file (which must be done on a Mac). This step is done using the maven dmgfile goal.
8. Run smoke tests against the application jarto prevent against bad builds

The final product of the impl build are the application jar, distribution zip, and site documentation. As with the api build these can be produced locally using astrogrid-install-artifact or produced and deployed to the public astrogrid server using astrogrid-deploy-artifact

Version Numbering

Finally, version numbering requires a little care. The version number for the current build is defined by the key astrogrid.desktop.versionin $BASE/template/project.properties. The same version number is used for both the apiand implprojects, and the artifacts that they produce.

Finally, there must be a <release>in $BASE/impl/xdocs/changes.xmlwho's versionattribute matches the current version number. If this is not present, the implementation build will fail while generating documentation.

---

How do I do a release?

To produce a snapshot or interim release:

1. run maven cleanin $BASE/desktop/impland $BASE/desktop/api
2. edit the value of astrogrid.desktop.versionin file $BASE/desktop/template/project.properties
3. edit $BASE/desktop/impl/xdocs/changes.xmlto add a new <release>element with the versionmatching the new value of astrogrid.desktop.version
4. commit changes into CVS, and CVS tag from $BASE/desktopdownwards, using the version number (or a cvs friendly variant thereof)
5. in $BASE/desktop/apirun maven astrogrid-deploy-artifact
6. in $BASE/desktop/implrun maven astrogrid-deploy-artifact
7. Inform people

How do build a local copy of the application?

  cd api
  maven astrogrid-install-artifact
  cd ../impl
  maven astrogrid-build-artifact

After this, an executable jar file will be found in $BASE/desktop/impl/target

---

How do I just build the source?

You'll spend most time in $BASE/desktop/impl, although this advice also applies to the apiproject, mostly. For day-to-day use, here are the most useful maven commands

Command	Description
maven clean	cleans the project be deleting all build products
maven java:compile	compiles the sources
maven -Dmaven.compile.deprecation java:compile	compiles the sources, and displays deprecation warnings. Here's a list of other compiler flags
Testing Goals
maven run-now	compiles the sources,run the application
Testing Goals
maven test	compiles sources and tests, and runs the unit tests.
maven test:ui	Display the swing test runner, from which tests can be run interactively. Here's a list of other testing goals and flags
Documentation Goals
maven site	Generate development documentation in target/docs

IDE Integration.

If you're using Eclipse, you can say maven eclipseto generate an eclipse project that contains all the required dependencies (after which you import this project into eclipse using ( File > Import > Existing Projects into Workspace). This works nicely and is a lot more sensible that stitching in all the dependencies by hand. There's similar tasks for IDEA and JDeveloper too - consult the maven documentation.

To run DesktopAstroGridfrom within an IDE, run the main class org.astrogrid.VODesktop

---

Development Practice

Try to follow the spirit of this, if not the letter - it's to stop us treading on each other's toes.

1. Bugzilla
   1. Create a bugzilla bug for the task (or find the existing one relating to this task).
   2. Take a note of the bug number.
   3. Mark the bug as 'assigned'.
2. CVS - Branch the astrogrid/desktopproject, using a branch name something like username-desktop- bugnumber- e.g. nw-desktop-2312
3. Code away until the feature is implemented / failing tests pass
   • Testing and Testing Process
4. Record what you've done
   1. add an <action>to the top of $BASE/impl/xdocs/changes.xml
   2. Fill in further detail in the bugzilla bug, if needed
      • in particular, record the branch name
   3. Reference the url of the bugzilla bug in javadoccomments, if sensible.
5. Make sure you've checked all your changes back into that branch
6. Stop work on that branch
7. Mark the bugzilla bug as closed
8. Email whoever's doing review & merge ( noelat the moment) that the branch is ready to merge into HEAD.
   • the branch will get reviewed and will either be merged into HEAD (and the bug marked as verified) or re-opened with a description of what the problem is.

It's up to your own discretion how much work you do in each branch - it's best not to leave them branched too long, but repeating this process for many little fixes takes time. Sometimes it's sensible to create an 'umbrella bug' which depends on a set of related bugs - and tackle this in a single branch.

Bugzilla

• Record bugs and enhancement ideas in http://www.astrogrid.org/bugzilla/index.cgi
• List of open desktop bugs

Coding Standards

In general, try to fit in with what's already there - but don't be afraid to improve bits that aren't right.

Our coding standards are based on the Elements of Java Stylebook - which is in turn based on this PDF

Avoid AbbreviatedCodeand 'well known' shortcuts

Use Commons Loggingfor all log/debug messages

Javadoc

We use some additional tags:

@fixme

something to be fixed urgently

@todo

something to be done

@future

an idea for future development

@testme

a junit test should be written for this code

@modified

documents how, when & by whom code was modified

@fires

documents that calling this method fires the specified event

And in the api sources, an additional tag is used @service- See #API_Descriptors

---

Development Notes

AR API

The public API to AstroRuntimeis in the apiproject. This is the crown jewels - please don't add or change it without consultation. The code here is should just be public interfaces, plus supporting bean and exception objects. The code must:

be stable

unnecessary changes will irritate users

be backwards compatible

so existing client code doesn't break

be self-contained

not refer to any classes other than those in the JDK (and commons logging)

use @service javadoc tags

to define the XML-RPC names of services. See #API_Descriptors

have Javadoc comments

otherwise it won't be useful.

use valid XHTML in Javadoc

else the build will break See #API_Descriptors

not use overloaded method names

although this works fine in Java, it doesn't work for XML-RPC

only use simple parameter types

XML-RPC Clients of AR can't easily pass objects as function parameters.

only return simple types or dumb beans

XML-RPC Clients of AR which call a function that returns an object will return an equivalent structure. This will contain all the data of the object (provided the object follows the JavaBean pattern), but no functionality.

only pass and return types which are serializable

Inputs and outputs to API calls all need to be serializable, else RMI will fail.

only throw exceptions extending org.astrogrid.acr.ACRException

and certainly no runtime exceptions.

When changes are made, the version numberof the project mustbe altered.

HiveMind

The code relies on HiveMind to assemble and configure the application at runtime. There's lots of benefits to using HiveMind, but it is a new thing to learn - and theres a lot of depth to this subject.

See DependencyInjectionContainersfor explanation the benefits of container management

External libraries

As well as all the AstroGriddelegates and supporting libraries that DesktopAstroGridrelies upon, there's also some 3rd-party libraries which are particularly important to the implementation:

axis

SOAP toolkit, used by many of the astrogrid delegates

castor

XML marshalling framework - from schema, generates bean sources, and machinery to populate them. Used within the CEA framework. Would maybe like to replace with XBean

commons-collections

Used throughout

commons-lang

Used throughout

commons-vfs

a unified file system api, which operates over file, ftp, sftp, http, myspace, and vospace.

concurrent

very good concurrency abstractions - locks, flags, synchronized collections. The preferred concurrency method - more likely to be than coding this from scratch using synchronized, although java1.5 provides similar in the util.concurrentpackage - gradually migrate over to that.

easymock

See DesktopAstroGridTesting#EasyMock ^?

ehcache

Object cache. Used to cache vomon data, annotations, registry resources.

glazedlists

implementation of java.util.Listthat fires events on modification. Provides Swing List- and Table- models which display the contents of the list. Good to use in concurrent situations - especially when fetching data in a background worker and displaying it incrementally in the UI. Used throughout.

jedit-syntax

Syntax-highlighting editor pane.

jetty

Embeddable web server. Used to provide HTML and XML-RPC access to AR.

JGoodies Forms

A swing layout manager which is easier to work with than GridBag, but produces good results. Used throughout.

l2fprod-common

Additional swing widgets - taskPane, button bar, hyperlink-button. Used throughout the UI.

prefuse

vizualization toolkit, used in AstroScopeto draw swirly spidergram-type-diagrams. We're using the alpha - upgrading to the beta would require a large rewrite, as it's a different model.

StAX

a streaming pull-parser for XML. Has the efficiency of SAX and a more straightforward programming model. Used to parse registry resources, vomon service data, annotation metadata, etc.

stil

multi-format table parser. Used to handle VOTable responses from DAL services - within AstroScopeand also AR api.

xml beans

Another XML Marshalling framework, used within ADQL compiler & QueryRunner

xfire

Streaming SOAP client. Used to query the Registry Service. Would like to use it elsewhere too, instead of axis.

xstream

an XML serialization library - writes java objects out into a moderately-nice xml format - suitable for users to edit - with a minimum of fuss.

Astrogrid Delegates

Avoid using the astrogrid delegate classes directly. Instead call the equivalent AR API component that wraps these delegates in your own code. They're better integrated into the system and provide a stable API under which the delegates are free to change.

However, there are some AG libs that are useful to use directly - e.g. the utilities in the common project. All these are already available on the classpath.

UI Layer

There's a bunch of icons in the package org.astrogrid.desktop.icons. Please try to use icons consistently (e.g. reuse the same icon for save, etc), and if you need to add more icons to this package, take them from KDE's crystalsvg theme - so a consistent look and feel is maintained. The class org.astrogrid.desktop.icons.IconHelper makes it trivial to load an icon.

UIComponent

Most of the UI component implementations extend the class org.astrogrid.desktop.modules.ui.UIComponentImpl. This provides a standard frame with an indeterminate progress bar at the bottom, helper metods to show progress and dialogues, plus a worker class that can be extended to do background processing. See the implementation javadoc for more details.

Background Worker

As is typical in swing programming, any long running activity (e.g. calling any webservice) should be executed on a background thread so that the responsiveness of the UI is maintained. This is particularly important when there is more than one window displayed by the application.