h1. Portal 2.0.2 Design

{cagridroundpanel}
{pre:class=cagridheaderfont}Table of Contents{pre}
{toc:outline=true|exclude=Portal 2.0.2 Design|style=none}
{cagridroundpanel}
{cagridtable}
{table-row}
{th:class=confluenceTh|colspan=3}*Development Team*{th}
{table-row}
{table-row}
{th:class=confluenceTh}Development{th}
{th:class=confluenceTh}Programmer's Guide{th}
{th:class=confluenceTh}Program Managment{th}
{table-row}
{table-row}
{cagridtablecell-width}Joshua Phillips,Manav Kher{cagridtablecell-width}
{cagridtablecell-width}Joshua Phillips,Manav Kher{cagridtablecell-width}
{cagridtablecell-width}Arumani Manisundaram{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width} {cagridtablecell-width}
{cagridtablecell-width} {cagridtablecell-width}
{cagridtablecell-width}Avinash Shanbhag{cagridtablecell-width}
{table-row}
{table-row}
{cagridtable}
{cagridtable}
{table-row}
{th:class=confluenceTh|colspan=2}*Contacts and Support*{th}
{table-row}
{table-row}
{cagridtablecell-width}Enter training contact{cagridtablecell-width}
{cagridtablecell-width}Joshua Phillips (joshua.phillips@semanticbits.com)
Manav Kher (manav.kher@semanticbits.com)
{cagridtablecell-width}
{table-row}
{cagridtable}
h1. Introduction
----
The caGrid Portal is a web-based application whose primary role is to allow end users to discover and interact with the services that are available on the caGrid infrastructure. It serves as the primary visualization tool for the caGrid middleware and provides a standards-based platform for hosting caBIG™-related services. The portal also serves as a caBIG™ information source. Users of the portal have instant access to information about caBIG™ participants, caBIG™ Applications and Tools, caGrid points-of-contact, and caGrid-related news and events.

h1. Overview
----
This section describes system requirements and technology dependencies and provides a high-level view of the portal architecture. A more detailed description of the portal design is provided in the Design and Architecture section.

h2. System Requirements
----
The caGrid Portal consists of a set of components written in the Java language. The following components are required by the portal and are not distributed with it.

{cagridtable}
{table-row}
{th:class=confluenceTh}Component{th}
{th:class=confluenceTh}Version{th}
{table-row}
{table-row}
{cagridtablecell-width}Java SDK{cagridtablecell-width}
{cagridtablecell-width}1.5+{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Apache Ant{cagridtablecell-width}
{cagridtablecell-width}1.65{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}MySQL Database{cagridtablecell-width}
{cagridtablecell-width}5+{cagridtablecell-width}
{table-row}
{cagridtable}
h2. Technology Dependencies
----
Following is a partial list of the technologies that caGrid Portal depends on but which are not part of the caGrid project. For a complete list, see caGrid/projects/portal2/lib/README.txt

{cagridtable}
{table-row}
{th:class=confluenceTh|width=20%}Technology{th}
{th:class=confluenceTh|width=20%}Version{th}
{th:class=confluenceTh|width=30%}Description{th}
{table-row}
{table-row}
{cagridtablecell-width}Spring{cagridtablecell-width}
{cagridtablecell-width}2.0.5{cagridtablecell-width}
{cagridtablecell-width}Spring is used in various layers of caGrid Portal. In all layers, Spring provides dependency injection and object lifecycle management. In the presentation layer, both Spring MVC and Spring Portlet MVC are used. In the service and DAO layers, Spring provides declarative transaction demarcation.{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Hibernate Core{cagridtablecell-width}
{cagridtablecell-width}3.2.2{cagridtablecell-width}
{cagridtablecell-width}Provides persistence strategy{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Hibernate Annotations{cagridtablecell-width}
{cagridtablecell-width}3.3.0{cagridtablecell-width}
{cagridtablecell-width}Provides ORM configuration{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Velocity{cagridtablecell-width}
{cagridtablecell-width}1.4{cagridtablecell-width}
{cagridtablecell-width}Rendering of portal layouts and decorations{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}JSP{cagridtablecell-width}
{cagridtablecell-width}2.0{cagridtablecell-width}
{cagridtablecell-width}Rendering of portal views{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}jQuery{cagridtablecell-width}
{cagridtablecell-width}1.1.4{cagridtablecell-width}
{cagridtablecell-width}Client-side effects{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}DWR{cagridtablecell-width}
{cagridtablecell-width}2.0.2{cagridtablecell-width}
{cagridtablecell-width}AJAX{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Acegi Security{cagridtablecell-width}
{cagridtablecell-width}1.0.3{cagridtablecell-width}
{cagridtablecell-width}Secure channel processing{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Liferay{cagridtablecell-width}
{cagridtablecell-width}4.3.3{cagridtablecell-width}
{cagridtablecell-width}Portal platform{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}JBoss{cagridtablecell-width}
{cagridtablecell-width}4.0.5.GA{cagridtablecell-width}
{cagridtablecell-width}Application server{cagridtablecell-width}
{table-row}
{cagridtable}
h2. Architecture
----
The caGrid Portal architecture can be divided into three logical layers: presentation, service, and persistence, with dependencies traveling from upper to lower layers according to the following diagram.
{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddav0527fbd25df4a116a21531bfe00715a9.png|caption=_}The Model-View-Controller (MVC) design pattern is used in the presentation tier. Spring (Portlet) MVC is the main framework used in this layer. The service layer provides a high-level API for interacting with domain objects, tasks, and caGrid services. DAO objects in the persistence layer provide a thin abstraction over the Hibernate persistence API.

h1. Features
----
The portal's current feature set includes the following:
* Discovery
* Service Metadata Exploration
* Data Service Query
* Query Sharing
* Federated Query
* caGrid Account Registration
* Grid Status Overview
* caBIG® Tools
* News and Events

h2. Discovery
----
One of the goals of the caGrid infrastructure is to allow services to be easily discovered and invoked. To this end, caGrid services register with a central index service. This index service allows other applications to locate caGrid services and then retrieve the information that is necessary to interact with the located services. The portal uses caGrid component APIs to provide an interface that enables end users to query information gathered from one or more index services. This interface makes it convenient for both service developers and non-developers to determine what functionality is available on caGrid.

In addition to service information, the portal also provides information about caBIG® participant institutions and caGrid points-of-contact (POCs). The goal is to enable the end user to quickly determine who is involved in what activities, and then to provide the user with the necessary contact information.

We refer to the process of locating services, participants, and POCs as discovery. The portal provides several modes of discovery. First, on the front page of the portal, a map of the U.S. displays the geographic location of services and participant institutions. Users can click icons of these items and link to more detailed information. Second, services, participants, and POCs are grouped into directories. The user can browse these directories and click on hyperlinks that lead to more detailed information. Finally, we provide an intuitive, keyword search interface, which allows the user to enter one or more keywords and match against one or more fields. The keyword search interface supports simple, advancaed and EVS (concept) based searches.

h2. Service Metadata Exploration
----
To support the caBIG® goals of syntactic and semantic interoperability, caGrid services expose a rich, standard set of metadata. This metadata describes not only the operations that are supported by the service, but it also links the data types that are used by these operations to concepts that are defined in the NCICB's Enterprise Vocabulary Service (EVS), common data elements that are defined in the Cancer Data Standards Repository (caDSR), and XML schemas that are defined in the Global Metadata Exchange (GME) service. caGrid services that support the caGrid Query Language (CQL) also expose an XML representation of the UML domain model that the service supports. This model is also linked to the EVS, caDSR, and GME. While caGrid services support standard web service APIs for retrieving this metadata, the caGrid project provides client API components that facilitate this process.

The portal uses caGrid client APIs to build an interface that enables end users to explore caGrid service metadata. Users can view the semantic descriptions that come from the EVS, the data type descriptions from the caDSR, and XML schemas that are registered with the GME. The interface makes it easier for end users to determine if a service meets their needs and what is necessary to interact with the service.

h2. Data Service Query
----
caGrid services that support CQL are referred to as data services. They expose a UML domain model and support a query operation that processes CQL queries against the UML domain model and returns XML documents that contain serialized instances of the UML classes that are defined in the domain model. These serialized instances conform to the XML schemas that are registered with the GME.

The portal provides an interface which allows the user to select a service and target UML class, build a query, execute the query, and view the results in both tabular and XML form. From the discovery tool, or while exploring service metadata, users can select the service and UML class to be queried. Then the user can either use a graphical query-building interface to construct CQL, or type in the CQL text directly.

The portal maintains the complete history of a user's queries so that a user can always retrieve and work with previously executed queries. The results of all queries that are executed during a single session (while the browser is open) are available for viewing in tabular or XML format.

h2. Query Sharing
----
Although data service domain models are linked to semantic metadata and data type specifications, a fair amount of exploration is necessary to construct useful queries. The portal provides a query-sharing feature that allows users to share their query building knowledge with the rest of the community. Any portal user can publish CQL queries and supporting documentation. Other portal users can search for CQL queries based on the target service, the target UML class, the creator of the query, or the associated name and description of the query.

h2. Federated Query
----
caGrid supports federated queries through a infrastructure service called the Federated Query Processor (FQP), and a language know as Distributed caGrid Query Language (DCQL). The FQP service supports the execution of DCQL queries. Portal has a Graphical User Interface (GUI) to build DCQL queries, then asynchronously execute them against an available FQP service. There are two main types of DCQL queries that can be constructed in the Portal: aggregate queries and distributed joins.

*Aggregate Queries*

Aggregate queries run against two or more data services and return an aggregated result set. Aggregate queries are essentially CQL that is run against multiple data services and have the result returned from these data services aggregated into a single result set. For example: Get me all ArrayDesigns from the caArray data service hosted at Georgetown and the caArray data service hosted at NCICB.

*Distributed Joins*

Distributed Join queries link domain objects across multiple data services. These joins are done based on semantic equivalence of domain objects across data models. The Portal helps users to visualize the possible joins, then allows them to build DCQL queries that contain cross-service joins through a simple point and click interface.

h2. caGrid Account Registration
----
GAARDS, the caGrid security infrastructure, uses an authentication approach based on Public Key Infrastructure (PKI). PKI requires users to be provisioned with an X.509 certificate and private key, collectively referred to as grid credentials. In caGrid, the Dorian service is used to provision a user's grid credentials, after they are either registered with the Dorian service, or another Identity Provider (IdP) that has an established trust relationship.

Secure caGrid services apply an authorization policy to incoming requests based the client's (i.e. the user that is making the request) identity. This identity is determined through the authentication process by inspecting the client's grid credentials. So, in order to interact with secure services, caGrid users need to obtain grid credentials.

While nearly all the functionality that the portal provides is available to unauthenticated users, only authenticated users can share queries and maintain a query history across browser sessions. Since portal uses GAARDS to authenticate users, users need to obtain grid credentials to enjoy this functionality.

The portal allows a user to register for an account with the caGrid Dorian service hosted by the NCICB. This service provides a Level of Assurance 1 (LOA1), which means anyone can request an account and the request will automatically be approved. Once the user's request has been approved, the user can provide his username and password to login to the portal. The portal will then use the grid credentials obtained from Dorian when interacting with secure caGrid services.

h2. Grid Status Overview
----
The portal provides a quick, visual description of the scale and activity of caGrid. The map on the front page of the portal not only indicates where services are being hosted, but whether those services are up, or down. Icons of various colors are used to indicate service status. In addition to the map, a status portlet presents up-to-date counts of all participants, all services, data services, and analytical services. It also provides links to more information about the five most recently registered services.

h2. caBIG® Tools
----
In addition to discovery and query of caGrid services, Portal also visualizes a comprehensive list of caBIG® applications and Tools. These applications are grouped by caBIG® workspace, caGrid compatibility, area of focus etc. The actual content for the caBIG® Tools view is maintained by the [caBIG® website|http://cabig.nci.nih.gov|caBIG® website]

h2. News and Events
----
A news portlet on the front page of the portal displays a running list of the latest caGrid-related news items. Users can click on each item to link to the full news article. Users can also click on a link to subscribe to the news feed using their browser, or any other RSS 2.0 compliant news feed reader.

The portal provides a news feeder editor application that is available to the portal administrator. This application can be used to maintain one or more RSS news channels.

A full-featured calendaring application is also presented on the front page of the portal. Users can browse through caGrid related events by day, week, month, and year. A portal administrator can import and export calendar events using the standard [iCal|http://tools.ietf.org/html/rfc2445|iCal format]format.

h1. Portal Projects

----
This aggr project contains components that run as background processes. These components are referred to as tasks henceforth. Tasks are responsible for periodically aggregating information from caGrid core infrastructure services such as the NCICB Index Service, caDSR, GTS, and GME. They also aggregate metadata from non-core services. The information aggregated by these tasks is stored in a database that feeds the portal. Each task can be configured and run independently of the others. In the standard portal deployment, tasks run within the same Spring container as the rest of the portal, but they could also be run in separate containers.

The following list describes the current set of tasks.
* *Geocode:* Checks the database for objects that have associated address information. It calls out to a geocoding application to determine geocoordinates for these objects so they can be placed on the portal map.
* *Metachange:* Checks for changes in service metadata. When changes are detected, it will aggregate the associated information and update the database.
* *Regsvc:* Checks the Index Service for the presence of new caGrid services. When new services are detected, the associated metadata is aggregated and stored in the database.
* *Status:* Periodically checks for changes in service status. Changes are recorded in the service status history. This task also invokes a configurable banning strategy. The banning strategy determines if the services should be banned from the portal based on the configured policy.
* *Xmlschemas:* Checks the service metadata in the database for objects that have associated QName attributes. It then attempts to locate the referenced XML schema in the GME and associated the object with the schema. This supports the service development lifecycle in which XML schemas may be registered with the GME after service deployment.
* *Trust:* Periodically invokes the caGrid SyncGTS client to synchronize the portal's trust store with the caGrid trust fabric based on the configured sychronization policy.

h2. The _portlet_ project
----
This project contains JSR-168 compliant portlet applications and supporting servlet applications. These applications have no dependencies on the underlying portal platform. Also, while the portlets are by default configured to work together, interactions among portlets are mediated through an inter-portlet communication abstraction. This enables the possibility of deploying them separately, in separate web applications, or even in separate application services, though doing so would require implementing the component that manages inter-portlet communication across application servers.

The currently implemented portlet applications include the following:
* *Cagridquery*: Supports query caGrid data services and sharing query
* *Cagriddiscovery*: Supports discovery of services, participants, and POCs
* *Cagridmap*: Implements the map view of caGrid
* *Cagridstatus*: Implements the caGrid status view
* *Cagridnews*: Provides news feed functionality
* *Cagriddirectauthn*: Provides authentication using GAARDS
* *Cagridregistration*: Provides for caGrid account registration

In addition to these, there are several other auxiliary portlet applications.

AJAX is used extensively to provide a richer user experience for UI components including result sets and tree views. The Direct Web Remoting (DWR) framework is used, in which client-side JavaScript interacts with service facades that are implemented as servlet applications. A tab-based navigation motif is used to control the user's flow through the cagridquery and cagriddiscovery applications. The underlying tab model abstraction is integrated with portlet security so that the user is presented with only the appropriate functionality. It is also integrated with the inter-portlet communication component, which allows for an event-style approach to interface programming.

The following sub sections describe certain concepts and components that are relevant to this project.

h3. Inter-portlet Communication (IPC)
----
Portlets communicate with one another by sending/receive messages to/from logical topics. The input and output messages of a portlet are declared as portlet preferences in portlet.xml. Each input message is mapped to a "source" topic. Each output message is mapped to one or more "publish" topics. Defining messages and topics as portlet preferences enables the user to wire together portlets to suit his needs.

Three portlet preferences are required to map an input or output message to a queue. The mapping is achieved through a simple naming convention. An example will illustrate:
{code} <preference>
<name>MsgInput1Name</name>
<value>sgsId</value>
</preference>
<preference>
<name>MsgInput1SourceName</name>
<value>selectedGridService</value>
</preference>
<preference>
<name>MsgInput1SourceNamespace</name>
<value>gs</value>
</preference>
{code}In this case, the local message name "sgsId" is mapped to the queue "gs:selectedGridService".

If a portlet needed to publish this message to multiple other portlets, the following declaration would be used:
{code} <preference>
<name>MsgOutput1Name</name>
<value>sgsId</value>
</preference>
<preference>
<name>MsgOutput1PublishName1</name>
<value>selectedGridServiceId</value>
</preference>
<preference>
<name>MsgOutput1PublishNamespace1</name>
<value>gs</value>
</preference>
<preference>
<name>MsgOutput2Name</name>
<value>sgsId</value>
</preference>
<preference>
<name>MsgOutput1PublishName1</name>
<value>theGridServiceId</value>
</preference>
<preference>
<name>MsgOutput1PublishNamespace1</name>
<value>xs</value>
</preference>
{code}The implementation that we are currently using maintains topics in the PortletSession (see [here|http://www.doc.ic.ac.uk/~mo197/portlets/portlet_messaging/] for more information). However, to enable IPC via AJAX or to enable IPC among portlets that are deployed in different web applications, we may switch to an implementation that maintains topics in a database.

h3. Portlet Wiring
----
Another aspect of the design of the caGrid Portal's portlet applications is that portlets should be able to be wired together at run-time. That means, one should be able to configure how and which portlets communicate with one another. This is to support the goal of, at some point in the future, allowing users to customize their workspaces.

Two features of the InterPortletMessageAdapter implementation facilitate the wiring together of portlets. First, input and output queue names are configured as standard portlet preferences. This allows a portal user with the right privileges to modify these preferences at run-time, through the portal. So, for example, one could add a new portlet to the workspace and then configure it to listen to the messages that are produced be an existing portlet. Of course, issues of interoperability must be addressed. Second, the InterPortletMessageAdapter can be turned on or off through the value of another mode preference. This enables the portlet to run in STANDALONE or INTERPORTLET mode. The following snippet shows the mode preference used by the newssummary portlet.

{code} <preference>
<name>newsItemSelectMode</name>
<value>INTERPORTLET</value>
</preference>
{code}Figure 5: Configuration of inter-portlet communication mode

As this snippet shows, the newssummary portlet is configured to work in INTERPORTLET mode for newsItemSelect messages.

The 2.0 release of the portal doesn't provide the user interface that is necessary to make use of this run-time wiring feature, but future versions may.

h3. Tab Model
----
Both the cagriddiscovery and cagridquery portlets use tabs to manage UI navigation. The tab implementation needs to support several features:
* Users should be able to click on tabs and sub tabs to get to different functionality
* Tabs and sub tabs should be able to render one or more associated views
* Tabs should maintain the state of the interface across page refreshes (which are quite frequent in portlet applications)
* Tabs should be visible or invisible based on the authorization policy
* User actions in one portlet should be able to cause a particular tab or sub tab in another portlet to appear

The tabs themselves are rendered by by a JSP that is included in other views. In the cagridquery portlet, that file is portlets/src/war/WEB-INF/jsp/query/tabs.jspf. This JSP looks for a request attribute named tabModel which it interacts with the render the tabs. The TabModel object contains a list of Tab objects. Each of the Tab objects represents a top-level tab. Children of a Tab represent sub tabs. So, each Tab is really a tree, however TabModel will construct a row-based view of the tabs based on the current state.

Figure 6: Tab model
TabModel objects are constructed through Spring bean definitions. Look at the queryTabModel bean in portlets/src/war/WEB-INF/context/query-portlet.xml for an example.

h4. Tab Navigation
----
To allow users to navigate among tabs, tab labels are hyperlinked, and these hyperlinks create portlet render requests. Included in the render request is a parameter that indicates the new selected tab path. The request must be routed to the appropriate controller to render the view. Routing of requests in Spring Portlet MVC is accomplished by implementations of the HandlerMapping interface. HandlerMapping instances take a PortletRequest and return a HandlerExecutionChain that may include interceptors and must include the ultimate handler of the request. In the caGrid Portal portlets project, a hierarchy of custom HandlerMapping classes has been provided to facilitate the requirements of tabbed navigation. The result of this implementation is that a particular tab path is selected and that tab path is mapped to a controller that prepares the view.


Figure 7: Tabbed navigation components

By default, TabHandlerMapping will select the tab path that is specified by the selectedTabPath parameter. The SelectedPathDelegationTabHandlerMapping implementation delegates the responsibility of determining the selected tab path to instances of SelectedPathHandler, which can be chained. AbstractInterPortletMessageSelectedPathHandler uses an instance of InterPortletMessageReceiver to determine the selected tab path based on the presence or absence of a message. The diagram above shows one concrete implementation of this handler.

h4. Tab Views
----
By default, each tab leaf will be rendered as a single view with a single tab in the row below the row of its parent tab. However, sometimes you want render multiple views in the same sub tab. To achieve this using the tab model, you create invisible child tabs of the tab in which these views should be displayed. The invisible child tabs of a particular tab represent all of the possible views that a sub tab can display. Look at the query/builder sub tab in the queryTabModel bean in portlets/src/war/WEB-INF/context/query-portlet.xml for an example.

h4. Tab State
----
Since portal pages refresh quite often, it is necessary to maintain state information about which tab is currently selected. This is achieved by associated a TabModel object with the HTTP session. The Spring container maintains TabModel objects in the session scope, which maps to the standard PORTLET_SCOPE. The same AOP approach that was described in the section on inter-portlet communication, above, is used here. The TabHandlerMapping declares a reference to a TabModel object and selects the appropriate tab or sub tab based on the request parameters or inter-portlet messages.

h4. Tab Security
----
For the 2.0 release of the portal, the only security requirement that relates to tabs is that certain tabs (e.g. the My Shared Queries tab) must be visible only to authenticated users. Tab objects have a Boolean requiresAuthn property that the rendering JSP can use to determine if a tab should be displayed. See portlets/src/war/WEB-INF/jsp/query/tabs.jspf for an example. In this case, the JSP checks if a portalUser object exists in the request. The PortalUserRequestInterceptor object is responsible for putting the PortalUser object in the request scope.

h4. Tabs and Inter-Portlet Communication
----
As described in the Tab Navigation section, SelectedPathDelegatingTabHandlerMapping delegates to a chain of SelectedPathHandler objects. The first object in that chain to return a non-null path wins (i.e. that is the path that is used). Classes of that extend AbstractInterPortletMessageSelectedPathHandler return the selected tab path based on the presence or absence of an inter-portlet message. This allows events in one portlet to cause certain sub tabs in another portlet to appear.

h3. Render Phase and Action Phase Controller Separation
----
Spring handler mappings are processed twice during an action request: before the action phase and before the render phase. This allows one to handle the action phase with one controller and the render phase with another controller. This approach is used quite often in the caGrid Portal applications because it allows action and view preparation logic to be reused more easily. In order to have the render phase handled by a different controller than the action phase, typically the controller will include a parameter in the RenderResponse object which causes the second processing of the handler mapping to return a different controller. Since this approach is used so frequently in the portal code, a couple of abstract classes have been created to handle common logic.


Figure 8: Base classes for render and action phase controllers
Sub classes of AbstractActionResponseHandlerCommandController must provide the logic that processes the request (e.g. making state changes to model objects),. But, modification of the RenderResponse is delegated to an implementation of CommandActionResponseHandler. This is useful, for example if you want to use the same action logic both within the tab navigation scenario and outside of that scenario.

Together, these base classes reproduce some of the functionality provided by Spring's AbstractFormController and its sub classes. However, these classes make it easier to separate the action and render preparation logic. For simpler interactions, for example, editing and creating news feeds, classes that extend SimpleFormController are used to process both the action and render phases.

h3. Ajax
----
The portal currently uses AJAX for two things in the cagridquery portlet application: to update the status of running queries (History tab); to render tree views of CQL queries, UML classes, and service metadata. This section walks through how the former works step-by-step. The next section goes into further detail about tree rendering.

The portal uses the DWR framework to implement AJAX. Here's a quick overview of DWR. For more details, go to the [DWR documentation|http://getahead.org/dwr/documentation|DWR documentation].

DWR allows one to expose some Java class through Javascript. That means, you can write Javascript that calls methods defined on some Java class that you provide. The way this works is that the DWR servlet generates a Javascript object that has the same method signatures as your Java class. When your Javascript code calls these methods, DWR client-side code translates these into asynchronous HTTP requests to the DWR servlet, which in turn calls your Java class. DWR then serializes the results as Javascript objects and sends them back the client.

There is a fair amount of configuration necessary to get DWR, Spring, portlets, and servlets all working together. That's why this section walks through it.

h4. web.xml
----
The DWR servlet must be declared in web.xml. Look at the dwr-invoker servlet and servlet-mapping in portlets/src/war/WEB-INF/web.xml. Also look at the contextConfigLocation context-param. Notice the file named facades.xml is included in the expression. This file is described later.

h4. dwr.xml
----
The DWR servlet expects to find a file called dwr.xml in the WEB-INF directory. This file is used to configure what and how Java classes will be exposed. Look at portlets/src/war/WEB-INF/dwr.xml. Notice that there is a create element with a javascript attribute of QueryHistoryFacade. This is the name of the Javascript object that will be generated by DWR. Also notice that the value of the creator attribute is spring. This means that Spring will be used to instantiate the Java object which DWR will expose. The next param element indicates that a bean with the id of cqlQueryHistoryFacade should be used. DWR will check the servlet context for a WebApplicationContext and then use that to retrieved the required beans.

Also notice the converter elements. You can use DWR pass any Java object as parameter or result between client and server. But you must specifically tell DWR which objects are allow to serialized, unless they are in the java.\* package.

h4. facades.xml
----
The server-side Java objects that handle AJAX requests are referred to here as facades. The facades.xml file is just where the façade objects, and those that they directly use, are declared. But, since DWR uses the Spring container to retrieve Java objects, those objects have access to all the other objects that are managed by the Spring container.

h4. JSPs
----
Look at portlets/src/war/WEB-INF/jsp/query/history.jsp. At the top of this page you should see:

{code}<script type='text/javascript' src='<c:url value="/dwr/interface/QueryHistoryFacade.js"/>'></script>
<script type='text/javascript' src='<c:url value="/dwr/engine.js"/>'></script>
<script type='text/javascript' src='<c:url value="/dwr/util.js"/>'></script>
{code}The first line causes the DWR servlet to generate the QueryHistoryFacade Javascript object. The next lines include DWR utility code. Further down in that JSP, you'll see the code invoke methods on the QueryHistoryFacade object. In dwr.xml and facades.xml, you see that the QueryHistoryFacade Javascript object is implemented on the server side by the CQLQueryHistoryFacade Java class. The render method of this class uses the DWR WebContext interface to delegate the responsibility of rendering a CQLQueryInstance object to a JSP. In this case, the request goes directly to the JSP, but as described later, the request could also have been configured to go to a Spring DispatcherServlet in order to make use of the Spring MVC framework, for example, to manage Hibernate sessions.

This is the general pattern that is used in the portal to implement AJAX. DWR translates a client-side Javascript call to a server-side façade, which may invoke business logic and return Java objects, or delegate to a servlet to render some view.

h3. Trees
----
The portal renders service metadata (including domain models), working CQL queries, and UML classes (in the CQL query builder) as trees. This allows the user to navigate the data structure by expanding and collapsing nodes. Two functional requirements for these tree views are that 1) the trees must maintain state across page refreshes; and 2) the client-side HTML and Javascript that renders the tree must be generated incrementally. This second requirement is a result of the fact that it is impractical, in the case of a domain model, or impossible, in the case of the CQL query, to generate the entire tree structure initially. AJAX is used to incrementally render the tree when the user expands a previously unopened node. This saves the cost of re-rendering the entire portlet window.

A common set of components is used to render all of the trees. A listener pattern is used to provide customization.


Figure 9: Tree rendering components
The tree itself wraps portal domain objects with which the user is interacting. Each TreeNode object has a reference to a domain object or a collection of domain objects. DWR translates client-side calls to the server-side TreeFacade object. The call to openNode or closeNode includes the path to the node in the tree. TreeFacade calls the find method on the root node of the tree to retrieve the indicated node. Then TreeFacade iterates through the list of TreeNodeListener instances, calling either onOpen or onClose, and passing in a reference to the root node. It is the responsibility of the TreeNodeListeners to modify the state of the tree (e.g. adding nodes, change the state of a node). Once all listeners have received the event, TreeFacade calls the render method of the TreeNodeRenderer instance, which returns a string. This string is passed back to the client, parsed as HTML, and placed into the DOM of the page.

Spring is used to construct a different TreeFacade instance for each type of tree. The TreeNodeListener and TreeNodeRenderer implementations provide different behavior that is appropriate to the type of tree being displayed. There are three implementations of TreeNodeListener:

* CQLQueryTreeNodeListener
* ServiceMetadataTreeNodeListener
* UMLClassTreeNodeListener

ServletRenderer is the only implementation of TreeNodeRenderer. This implementation simply uses the DWR WebContext API to delegate rendering to a configured servlet. TreeNode rendering is handled by Spring MVC. The tree servlet is configured in web.xml and tree-servlet.xml. The Spring controllers that are configured in tree-servlet.xml are responsible for locating the object that should be rendered and populating the model with that object before passing it to the view technology, which is JSP.

h4. Custom JSP Tree Tag
----
JSPs are used to render tree views. Since AJAX is used to allow the tree to be built incrementally, usually only a single tree node must be rendered. Furthermore, client-side Javascript ensures that if a particular tree node has been previously rendered (e.g. the user has expanded and subsequently collapsed the node), it is not rendered again. However, when a full page is refreshed (e.g. by some action in another portlet), all the visible nodes of the tree must be re-rendered.

A custom JSP tag is provided so that a single JSP view can render the entire tree. That tag is defined in portlets/src/war/WEB-INF/tags/tree.tag. The tree tag contains recursive logic that will navigate through tree of TreeNode objects and render each node. The tag defines two attributes: nodeFragment and contentFragment. The values of these attributes are interpreted as JSP fragments, and invoke by the tree tag using the jsp:invoke action. These enable the pages that use this tag to customize how tree nodes and the contents of those nodes are rendered, while re-using common logic to render the basic tree structure and controls. The tag exposes two variables: currNode and currChildNode, which are TreeNode objects. These are available within the scope of the tag so that the nodeFragment and contentFragment JSP fragments can access these objects.

In the case of the service metadata tree, the tree nodes contain many different types of domain objects, each of which should be rendered differently. Since a single JSP view may need to render an entire tree structure, the JSP fragment that is rendering the content of a node must contain conditional logic to render the current node based on its type. Look at portlets/src/war/WEB-INF/jsp/disc/details/service/serviceDetails.jspf as an example. This fragment uses the JSTL choose control structure to include the appropriate JSP based on the type of the current node.

h4. Portlet-to-Servlet Communication
----
Both portlets and servlets are involved in preparing a model to be rendered in JSP views. In the case of trees, these portlets and servlets need access to the same model object, which is the tree of TreeNode objects. To achieve this, TreeFacadeObjects, which maintain a reference to the current tree, are placed in the Spring globalSession scope. This allows both portlets and servlets to reference the same objects. AOP proxying is used to ensure that controller instances, which reference them, operated on the right instances. That is, at most three TreeFacade instances are created per HTTP session. Each maintains the currently selected tree: service metadata, CQL query or UML class.

Another difficulty that arises when mixing portlet and servlet technology is the generation of portlet URLs by servlets (or the JSP views returned by Spring MVC controllers). For example, often a servlet will need to construct a URL that will invoke a portlet action. But, since the PortletContext is not available to within the servlet application, there isn't enough information available to construct this URL. In the caGrid Portal application, his problem has been addressed by constructing a base portlet URL from a portlet JSP view, and then passing that URL as a parameter to the servlet. The servlet JSP view can then use string manipulation to place parameter values into the URL.

An example of this can be seen in portlets/src/war/WEB-INF/jsp/query/builder.jsp and portlets/src/war/WEB-INF/jsp/query/builder/criteriaBean.jsp. The editCriterionUrl value is constructed in the portlet JSP view builder.jsp. Then it is passed in the AJAX call as a parameter and modified in the servlet JSP view criteriaBean.jsp.

h3. Cross-Site Scripting (XSS) Prevention
----
Since the portal allows user to enter information that will be displayed to other users, there is a risk of cross-site scripting attacks. To address this, the XSSFilterEditor custom bean property editor is used to bind parameter values to command objects in cases where an XSS attack could be attempted. Look at the initBinder method of UpdatedSharedQueryController for an example. If text that could be dangerous is detected, a binding exception is created and the user is warned not to enter text that could be interpreted as HTML, XML or some scripting language.

h2. The _portals_ Project
----
This project contains components that have dependencies on Liferay. These components include:
* Custom authentication plug-in
* Custom layout application
* Custom caGrid Portal theme application
* Customizations of the Liferay portal distribution
* Liferay installation/configuration code

h3. Authentication Plug-In
----
The caGrid WebSSO component was not available to be included in this release of the portal. However, it will be included in the next release. For this release, an implementation is provided that interacts with directly with an AuthenticationService and a Dorian service to authenticate the user and retrieve grid credentials.

Liferay provides a couple of different extension points where one can plug in a custom authentication system. It even supports SSO through CAS out of the box. However, this implementation assumes that user information will be pull from some LDAP server. See this [discussion|http://wiki.liferay.com/index.php/Develop_a_Custom_Authentication_System|Develop a Custom Authentication System]of Liferay authentication extension points. Also see this [example|http://wiki.liferay.com/index.php/Acegi_Security_for_Liferay|Acegi Security for Liferay]that uses Acegi Security with Liferay.

The approach used in the caGrid Portal is to provide an implementation of com.liferay.portal.security.auth.AutoLogin, which is found in portals/liferay/src/java/gov/nih/nci/cagrid/portal/liferay/security/DirectAutoLogin.java. This class is included in the auto-login chain that is configured in portals/liferay/src/war/WEB-INF/classes/portal-ext.properties. DirectAutoLogin checks the request for a ticket parameter. This ticket is then used to retrieve the authenticated user from the database.

The actual authentication is handled by the directauthn portlet application, which uses the AuthnService (from the security project) to authenticate a user to an Identity Federation Service (IFS), for example the NCICB's Dorian service, and Identity Provider (IdP), for example the NCICB's AuthenticationService.

h3. Custom Theme
----
Liferay defines a standard structure for custom themes. These themes can be deployed as separate web applications which are hot-deployed and automatically registered to the portal. A custom theme called cagrid-minimal has been created for the caGrid Portal. This theme is found under portals/liferay/src/themes/cagrid-minimal. Most of the work of creating the custom them involves editing the following files:
* templates/portal_normal.vm: renders the portal page container
* templates/navigation.vm: renders the top-level tabs
* templates/dock.vm: renders the doc
* css/custom.css: override styles in other CSS files

See the Liferay [wiki|http://wiki.liferay.com/index.php/Themes|Developing Themes]for more information about developing themes.

h3. Custom Layout
----
A layout determines how portlets can be arranged on the page, e.g. two columns versus three columns. A custom layout was created for caGrid Portal to permit three columns with widths of 20%, 60%, and 20%. This layout is found at portals/liferay/src/layouts.

h3. Liferay Archive Creation
----
The basic structure of the portal site is created through the Liferay portal interface. For example, the portal developer selects portlets to be displayed on the page, positions portals on the page, selects the theme and layout, add tabs. All of this is done through the portal interface. See the Liferay portal administrators guide for more details. When the site structure has been created, it can be exported and saved as a Liferay Archive (LAR) file. The LAR file can then be imported into another portal instance (of the same release number) to automatically set up the site structure.

h3. Secure Channel Processing
----
Certain portal pages should be encrypted (e.g. the login and registration pages) and other pages should not be encrypted (e.g. a page that is displaying a Google map, since Google doesn't support an HTTPS API). The Acegi ChannelProcessingFilter is used to handle switching between the HTTP and HTTPS protocols. The Acegi filter chain configuration is defined in portals/liferay/src/war/WEB-INF/applicationContext-acegi-security.xml.

h3. Servlet-to-Portlet Communication
----
Upon successful authentication, the DirectAutoLogin class must create a PortalUser object and put it in the request so that it is available to the portlet applications. The portal must be configured to enable portlets to retrieve objects that the portal has put into the session or the request. A description of this process is [here|http://wiki.liferay.com/index.php/Liferay_FAQ#How_do_we_use_the_request_to_share_attributes_between_portlets_deployed_as_individual_WAR|Process for sharing attributes between portlets deployed as individual WAR files]: [and here|http://wiki.liferay.com/index.php/Liferay_FAQ#How_do_we_use_the_request_to_share_attributes_between_portlets_deployed_as_individual_WARs.3F]

In portals/liferay/src/war/WEB-INF/classes/portal-ext.properties, you can see that the portal has been configured to share request attributes that start with the CAGRIDPORTAL_ATTS\_ prefix.

h2. The _security_ Project
----
This project contains security related components that are common among projects. For example, the portlets and portal project use the same encryption services.

h2. The _db_ Project
----
The db project contains domain objects and DAO implementations that are used by the other projects. The portal persistence layer uses Hibernate with annotations. Hibernate tools are used to generate the DDL. The DAOs uses Spring's Hibernate template abstraction to work with the Hibernate Session API.

The following diagram displays the main domain objects.

Figure 10: Main domain objects
The following diagram shows the domain objects that are involved in data service querying.


Figure 11: Data service query classes

As mentioned above, the DDL is generated by Hibernate tools. So the process of changing the schema involves first adding/editing domain model classes and annotations, and then running the Hibernate tools to regenerate the DDL. However, for completeness, the DDL as of the 2.0 release is included [here|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/~checkout~/cagrid-1-0/Documentation/docs/portal/portal2/design/arch/cagrid-portal-2.0-erd.jpg?rev=1.1;content-type=image%2Fjpeg;cvsroot=cagrid-1-0]

h1. Testing
----
h2. Unit Tests
----
Each project contains a suite of JUnit tests, which can be run individually using Ant targets like this: db:test, or aggr:test, etc. All tests can be run with "test" target. A consolidated HTML test report is placed in the docs/test/report directory. Database tests use the embedded HSQLDB, so no database set up is required.

Following is the list of JUnit test suites:
* db/test/src/java/gov/nih/nci/cagrid/portal/dao/GridServiceDaoTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/metachange/MetadataChangeTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/regsvc/RegisteredServiceMockTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/status/ServiceStatusTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/query/results/ExportResultsTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/util/HTMLInputFilterTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/util/PortletUtilsTest.java

We also use mock objects with testing utilities like [mockito|http://code.google.com/p/mockito/|Mockito code]to unit test individual projects.

h2. System Tests
----
The suite of system tests has been defined [here|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/~checkout~/cagrid-1-0/Documentation/docs/portal/portal2/testing/caGRID_Portal_2.0_test_cases.xls?rev=HEAD;content-type=application%2Foctet-stream;cvsroot=cagrid-1-0]

h1. References
----
h2. caGrid References
----
# Monitoring and Discovery Portal Enterprise Architect Project [GForge SCM Link|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/cagrid-1-0/caGrid/projects/portal/docs/project/Portal.EAP?cvsroot=cagrid-1-0;f=h;ln=1]
# caGrid 1.0 Gforge website [http://gforge.nci.nih.gov/projects/cagrid-1-0/]
# caGrid Homepage [https://cabig.nci.nih.gov/workspaces/Architecture/caGrid/]

h2. Technical Manuals/Articles
----
# Java Timer Tasks [http://java.sun.com/j2se/1.3/docs/api/java/util/TimerTask.html]
# Spring framework [http://www.springframework.org/]
# Java Bean Specification: [http://java.sun.com/products/javabeans/docs/spec.html]
# Foundations of Object-Relational Mapping: [http://www.chimu.com/publications/objectRelational/]
# Object-Relational Mapping articles and products: [http://www.service-architecture.com/object-relational-mapping/]
# Hibernate Reference Documentation: [http://www.hibernate.org/hib_docs/reference/en/html]
# Basic O/R Mapping: [http://www.hibernate.org/hib_docs/reference/en/html/mapping.html]
# Java Programming: [http://java.sun.com/learning/new2java/index.html]
# Javadoc tool: [http://java.sun.com/j2se/javadoc/]
# JUnit: [http://junit.sourceforge.net/]
# Extensible Markup Language: [http://www.w3.org/TR/REC-xml/]
# Mockito: [http://code.google.com/p/mockito/]

h2. caBIG® Material
----
# *caBIG®:* [http://cabig.nci.nih.gov/]
# *caBIG® Compatibility Guidelines*: [http://cabig.nci.nih.gov/guidelines_documentation]

h2. caCORE Material
----
# *caCORE:* [http://ncicb.nci.nih.gov/core]
# *caBIO:* [http://ncicb.nci.nih.gov/core/caBIO]
# *caDSR:* [http://ncicb.nci.nih.gov/core/caDSR]
# *EVS:* [http://ncicb.nci.nih.gov/core/EVS]
# *CSM:* [http://ncicb.nci.nih.gov/core/CSM]

h1. Glossary
----
|| Term || Definition ||
| API | Application Programming Interface |
| caArray | cancer Array Informatics |
| caBIG® | cancer Biomedical Informatics Grid® |
| caBIO | Cancer Bioinformatics Infrastructure Objects |
| caCORE | cancer Common Ontologic Representation Environment |
| caDSR | Cancer Data Standards Repository |
| caMOD | Cancer Models Database |
| cardinality | Cardinality describes the minimum and maximum number of associated objects within a set |
| CDE | Common Data Element |
| CGAP | Cancer Genome Anatomy Project |
| CMAP | Cancer Molecular Analysis Project |
| CN | Common Name |
| CS | Classification Scheme |
| CSI | Classification Scheme Item |
| CSM | Common Security Module |
| CTEP | Cancer Therapy Evaluation Program |
| CUI | Concept Unique Identifier |
| CVS | Concurrent Versions System |
| DAML | DARPA Agent Markup Language |
| DAO | Data Access Objects |
| DARPA | Defense Advanced Research Projects Agency |
| DAS | Distributed Annotation System |
| DL | Description Logic |
| EA | Enterprise Architect |
| EBI | European Bioinformatics Institute |
| EVS | Enterprise Vocabulary Services |
| GAI | CGAP Genetic Annotation Initiative |
| GEDP | Gene Expression Data Portal |
| GIS | Geographical Information System |
| HIPPA | Health Insurance Portability and Accountability Act |
| HLGT | High Level Group Term |
| HLT | High Level Term |
| HTTP | Hypertext Transfer Protocol |
| ISO | International Organization for Standardization |
| JAAS | Java Authentication and Authorization Service |
| JAR | Java Archive |
| Javadoc | Tool for generating API documentation in HTML format from doc comments in source code ([http://java.sun.com/j2se/javadoc/]) |
| JDBC | Java Database Connectivity |
| JET | Java Emitter Templates |
| JMI | Java Metadata Interface |
| JSP | JavaServer Pages |
| JUnit | A simple framework to write repeatable tests [http://junit.sourceforge.net/] |
| JSF | Java Server Faces |
| LDAP | Lightweight Directory Access Protocol |
| LLT | Lowest Level Term |
| LOINC | Logical Observation Identifier Names and Codes |
| MAGE | MicroArray and Gene Expression |
| MAGE-OM | MicroArray Gene Expression - Object Model |
| MedDRA | Medical Dictionary for Regulatory Activities |
| metadata | Definitional data that provides information about or documentation of other data. |
| MGED | Microarray Gene Expression Data |
| MMHCC | Mouse Models of Human Cancers Consortium |
| MO | MGED Ontology |
| multiplicity | Multiplicity of an association end indicates the number of objects of the class on that end may be associated with a single object of the class on the other end |
| NCI | National Cancer Institute |
| NCICB | National Cancer Institute Center for Bioinformatics |
| OIL | Ontology Inference Layer |
| OilEd | Ontology editor allowing you to build ontologies using DAML+OIL |
| OLLT | Obsolete Lower Level Terms |
| OMG | Object Management Group |
| ORM | Object Relational Mapping |
| PT | Preferred Term |
| RDBMS | Relational Database Management System |
| SDK | Software Development Kit |
| Semantic connector | A development kit to link model elements to NCICB EVS concepts. |
| SOC | System Organ Class |
| SPORE | Specialized Programs of Research |
| SQL | Structured Query Language |
| SSC | Special Search Categories |
| UI | User Interface |
| UID | User Identification |
| UML | Unified Modeling Language |
| UMLS | Unified Medical Language System |
| UPT | User Provisioning Tool |
| URL | Uniform Resource Locators |
| VD | Value Domain |
| WAR | Web Application Archive |
| WSDL | Web Services Description Language |
| XML | Extensible Markup Language ([http://www.w3.org/TR/REC-xml/]) - XML is a subset of Standard Generalized Markup Language (SGML). Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML. XML has been designed for ease of implementation and for interoperability with both SGML and HTML |