caBIG{^}™^ is an initiative of the National Cancer Institute, NIH, DHHSUpdated December 27, 2007

{cagridtable}
{table-row}
{th:class=confluenceTh|colspan=3}*Development Team*{th}
{table-row}
{table-row}
{th:class=confluenceTh}Development{th}
{th:class=confluenceTh}Programmer's Guide{th}
{th:class=confluenceTh}Program Managment{th}
{table-row}
{table-row}
{cagridtablecell-width}Joshua Phillips{cagridtablecell-width}
{cagridtablecell-width}Joshua Phillips{cagridtablecell-width}
{cagridtablecell-width}Yan Peter{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width} {cagridtablecell-width}
{cagridtablecell-width} {cagridtablecell-width}
{cagridtablecell-width}Avinash Shanbhag{cagridtablecell-width}
{table-row}
{table-row}
{table-row}
{cagridtablecell-width}^1^ Footnote contributing companies/ contractors/ divisions{cagridtablecell-width}
{cagridtablecell-width}^2^ department or division {cagridtablecell-width}
{cagridtablecell-width}^3^ company {cagridtablecell-width}
{table-row}
{cagridtable}

{cagridtable}
{table-row}
{th:class=confluenceTh|colspan=2}*Contacts and Support*{th}
{table-row}
{table-row}
{cagridtablecell-width}Enter training contact{cagridtablecell-width}
{cagridtablecell-width}Joshua Phillips (joshua.phillips@semanticbits.com){cagridtablecell-width}
{table-row}
{cagridtable}

h1. Introduction
----
The caGrid Portal is a web-based application whose primary role is to allow end users to discover and interact with the services that are available on the caGrid infrastructure. It serves as the primary visualization tool for the caGrid middleware and provides a standards-based platform for hosting caBIG-related tools. The portal also serves as a caBIG information source. Users of the portal have instant access to information about caBIG participants, caGrid points-of-contact, and caGrid-related news and events.

h1. Overview
----
This section describes system requirements, technology dependencies and provides a high-level view of the portal architecture. A more details description of the portal design is provide in the Design and Architecture section.

h2. System Requirements
----
The caGrid Portal consists of a set of components written in the Java language. The following components are required by the portal and are not distributed with it.


{cagridtable}
{table-row}
{th:class=confluenceTh}Component{th}
{th:class=confluenceTh}Version{th}
{table-row}
{table-row}
{cagridtablecell-width}Java SDK{cagridtablecell-width}
{cagridtablecell-width}1.5+{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Apache Ant{cagridtablecell-width}
{cagridtablecell-width}1.6.5{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}MySQL Database{cagridtablecell-width}
{cagridtablecell-width}5+{cagridtablecell-width}
{table-row}
{cagridtable}

h2. Technology Dependencies
----
Following is a partial list of the technologies that caGrid Portal depends on which are not part of the caGrid project. For a complete list, see caGrid/projects/portal2/lib/README.txt



{cagridtable}
{table-row}
{th:class=confluenceTh|width=20%}Technology{th}
{th:class=confluenceTh|width=20%}Version{th}
{th:class=confluenceTh|width=30%}Description{th}
{table-row}
{table-row}
{cagridtablecell-width}Spring{cagridtablecell-width}
{cagridtablecell-width}2.0.5{cagridtablecell-width}
{cagridtablecell-width}Spring is used in various layers of caGrid Portal. In all layers, Spring provides dependency injection and object lifecycle management. In the presentation layer, both Spring MVC, and Spring Portlet MVC are used. In the service and DAO layers, Spring provides declarative transaction demarcation.{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Hibernate Core{cagridtablecell-width}
{cagridtablecell-width}3.2.2{cagridtablecell-width}
{cagridtablecell-width}Provides persistence strategy{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Hibernate Annotations{cagridtablecell-width}
{cagridtablecell-width}3.3.0{cagridtablecell-width}
{cagridtablecell-width}Provides ORM configuration{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Velocity{cagridtablecell-width}
{cagridtablecell-width}1.4{cagridtablecell-width}
{cagridtablecell-width}Rendering of portal layouts and decorations{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}JSP{cagridtablecell-width}
{cagridtablecell-width}2.0{cagridtablecell-width}
{cagridtablecell-width}Rendering of portal views{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}jQuery{cagridtablecell-width}
{cagridtablecell-width}1.1.4{cagridtablecell-width}
{cagridtablecell-width}Client-side effects{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}DWR{cagridtablecell-width}
{cagridtablecell-width}2.0.2{cagridtablecell-width}
{cagridtablecell-width}AJAX{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Acegi Security{cagridtablecell-width}
{cagridtablecell-width}1.0.3{cagridtablecell-width}
{cagridtablecell-width}Secure channel processing{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}Liferay{cagridtablecell-width}
{cagridtablecell-width}4.3.3{cagridtablecell-width}
{cagridtablecell-width}Portal platform{cagridtablecell-width}
{table-row}
{table-row}
{cagridtablecell-width}JBoss{cagridtablecell-width}
{cagridtablecell-width}4.0.5.GA{cagridtablecell-width}
{cagridtablecell-width}Application server{cagridtablecell-width}
{table-row}
{cagridtable}

h2. Architecture
----
The caGrid Portal architecture can be divided into three logical layers: presentation, service, and persistence, with dependencies traveling from upper layers to lower layers according to the following diagram.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddav0527fbd25df4a116a21531bfe00715a9.png|caption=cFigure 1: Architectural layers}

The Model-View-Controller (MVC) design pattern is used in the presentation tier. Spring (Portlet) MVC is the main framework used in this layer. The service layer provides a high-level API for interacting with domain objects, tasks and caGrid services. DAO objects in the persistence layer provide a thin abstraction over the Hibernate persistence API.

h1. Features
----
The portal's current feature set includes the following:

* Discovery
* Service Metadata Exploration
* Data Service Query
* Query Sharing
* caGrid Account Registration
* Grid Status Overview
* News and Events

h2. Discovery
----
One of the goals of the caGrid infrastructure is to allow services to be easily discovered and invoked. To this end, caGrid services register with a central index service. This index service allows other applications to locate caGrid services and then retrieve the information that is necessary to interact with the located services. The portal uses caGrid component APIs to provide an interface that enables end users to query information gathered from one or more index services. This interface makes it convenient for both service developers and non-developers to determine what functionality is available on caGrid.

In addition to service information, the portal also provides information about caBIG participant institutions and caGrid points-of-contact (POCs). The goal is to enable the end user to quickly determine who is involved in what activities, and then to provide the user with the necessary contact information.

We refer to the process of locating services, participants, and POCs as discovery. The portal provides several modes of discovery. First, on the front page of the portal, a map of the U.S. displays the geographic location of services and participant institutions. Users can click icons of these items and link to more detailed information. Second, services, participants, and POCs are grouped into directories. The user can browse these directories and click on hyperlinks that lead to more detailed information. Finally, we provide an intuitive, keyword search interface, which allows the user to enter one or more keywords and match against one or more fields.

h2. Service Metadata Exploration
----
To support the caBIG goals of syntactic and semantic interoperability, caGrid services expose a rich, standard set of metadata. This metadata describes not only the operations that are supported by the service, but it also links the data types that are used by these operations to concepts that are defined in the NCICB's Enterprise Vocabulary Service (EVS), common data elements that are defined in the Cancer Data Standards Repository (caDSR), and XML schemas that are defined in the Global Metadata Exchange (GME) service. caGrid services that support the caGrid Query Language (CQL) also expose an XML representation of the UML domain model that the service supports. This model is also linked to the EVS, caDSR, and GME. While caGrid services support standard web service APIs for retrieving this metadata, the caGrid project provides client API components that facilitate this process.

The portal uses caGrid client APIs to build an interface that enables end users to explore caGrid service metadata. Users can view the semantic descriptions that come from the EVS, the data type descriptions from the caDSR, and XML schemas that are registered with the GME. The interface makes it easier for end users to determine if a service meets their needs and what is necessary to interact with the service.

h2. Data Service Query
----
caGrid services that support CQL are referred to as data services. They expose a UML domain model and support a query operation that processes CQL queries against the UML domain model and returns XML documents that contain serialized instances of the UML classes that are defined in the domain model. These serialized instances conform to the XML schemas that are registered with the GME.

The portal provides an interface which allows the user to select a service and target UML class, build a query, execute the query, and view the results in both tabular and XML form. From the discovery tool, or while exploring service metadata, users can select the service and UML class to be queried. Then the user can either use a graphical query-building interface to construct CQL, or type in the CQL text directly.

The portal maintains the complete history of a user's queries so that a user can always retrieve and work with previously executed queries. The results of all queries that are executed during a single session (while the browser is open) are available for viewing in tabular or XML format.

h2. Query Sharing
----
Although data service domain models are linked to semantic metadata and data type specifications, a fair amount of exploration is necessary to construct useful queries. The portal provides a query-sharing feature that allows users to share their query building knowledge with the rest of the community. Any portal user can publish CQL queries and supporting documentation. Other portal users can search for CQL queries based on the target service, the target UML class, the creator of the query, or the associated name and description of the query.

h2. caGrid Account Registration
----
The caGrid security infrastructure, known as GAARDS, uses an authentication approach that is based on Public Key Infrastructure (PKI). PKI requires users to be provisioned with an X.509 certificate and private key, what we refer to as grid credentials. The caGrid Dorian service is used to provision users with grid credentials. Before Dorian will grant a user grid credentials, the user must register either with the Dorian service or another Identity Provider (IdP) that has established a trust relationship with Dorian.

Secure caGrid services apply an authorization policy to incoming requests based the client's (i.e. the user that is making the request) identity. This identity is determined through the authentication process by inspecting the client's grid credentials. So, in order to interact with secure services, caGrid users need to obtain grid credentials.

While nearly all the functionality that the portal provides is available to unauthenticated users, only authenticated users can share queries and maintain a query history across browser sessions. The portal uses GAARDS to authenticate users. So, to use the query sharing and persistent query history features, users need to obtain grid credentials.

The portal provides an interface that allows users to register for an account with the caGrid Dorian service hosted by the NCICB. This service provides a Level of Assurance 1 (LOA1), which means anyone can request an account and the request will automatically be approved. Once the user's request has been approved, the user can provide his username and password to login to the portal. The portal will then use the grid credentials obtained from Dorian when interacting with secure caGrid services.

h2. Grid Status Overview
----
The portal provides a quick, visual description of the scale and activity of caGrid. The map on the front page of the portal not only indicates where services are being hosted, but whether those services are up or down. Icons of various colors are used to indicate service status. In addition to the map, a status portlet presents up-to-date counts of all participants, all services, data services, and analytical services. It also provides links to more information about the five most recently registered services.

h2. News and Events
----
A news portlet on the front page of the portal displays a running list of the latest caGrid-related news items. Users can click on each item to link to the full news article. Users can also click on a link to subscribe to the news feed using their browser, or any other RSS 2.0 compliant news feed reader.

The portal provides a news feeder editor application that is available to the portal administrator. This application can be used to maintain one or more RSS news channels.

A full-featured calendaring application is also presented on the front page of the portal. Users can browse through caGrid related events by day, week, month, and year. A portal administrator can import and export calendar events using the standard iCal format ([http://tools.ietf.org/html/rfc2445]).

h1. Portal Projects
----
The components that implement the architecture described in section 2.3 are organized into several sub projects. The dependencies among these sub projects are managed so as to reduce coupling among components and make it possible to use them independently of one another. The following diagram illustrates the dependencies among projects.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddava3eb161146a31785446c29add56c1874.png|caption=Figure 2: Portal projects}



h2. The _aggr_ Project
----
This aggr project contains components that run as background processes. These components are referred to as tasks henceforth. Tasks are responsible for periodically aggregating information from caGrid core infrastructure services such as the NCICB Index Service, caDSR, GTS, and GME. They also aggregate metadata from non-core services. The information aggregated by these tasks is stored in a database that feeds the portal. Each task can be configured and run independently of the others. In the standard portal deployment, tasks run within the same Spring container as the rest of the portal, but they could also be run in separate containers.

The following list describes the current set of tasks.

*Geocode{*}: Checks the database for objects that have associated address information. It calls out to a geocoding application to determine geocoordinates for these objects so they can be placed on the portal map.
*Metachange{*}: Checks for changes in service metadata. When changes are detected, it will aggregate the associated information and update the database.
*Regsvc{*}: Checks the Index Service for the presence of new caGrid services. When new services are detected, the associated metadata is aggregated and stored in the database.
*Status{*}: Periodically checks for changes in service status. Changes are recorded in the service status history. This task also invokes a configurable banning strategy. The banning strategy determines if the services should be banned from the portal based on the configured policy.
*Xmlschemas{*}: Checks the service metadata in the database for objects that have associated QName attributes. It then attempts to locate the referenced XML schema in the GME and associated the object with the schema. This supports the service development lifecycle in which XML schemas may be registered with the GME after service deployment.
*Trust:* Periodically invokes the caGrid SyncGTS client to synchronize the portal's trust store with the caGrid trust fabric based on the configured sychronization policy.

h2. The _portlets_ Project
----

This project contains JSR-168 compliant portlet applications and supporting servlet applications. These applications have no dependencies on the underlying portal platform. Also, while the portlets are by default configured to work together, interactions among portlets are mediated through an inter-portlet communication abstraction. This enables the possibility of deploying them separately, in separate web applications, or even in separate application services, though doing so would require implementing the component that manages inter-portlet communication across application servers.

The currently implemented portlet applications include the following:

*Cagridquery{*}: Supports query caGrid data services and sharing query
*Cagriddiscovery{*}: Supports discovery of services, participants, and POCs
*Cagridmap{*}: Implements the map view of caGrid
*Cagridstatus{*}: Implements the caGrid status view
*Cagridnews{*}: Provides news feed functionality
*Cagriddirectauthn{*}: Provides authentication using GAARDS
*Cagridregistration{*}: Provides for caGrid account registration

In addition to these, there are several other auxiliary portlet applications.

AJAX is used extensively to provide a richer user experience for UI components including result sets and tree views. The Direct Web Remoting (DWR) framework is used, in which client-side JavaScript interacts with service facades that are implemented as servlet applications. A tab-based navigation motif is used to control the user's flow through the cagridquery and cagriddiscovery applications. The underlying tab model abstraction is integrated with portlet security so that the user is presented with only the appropriate functionality. It is also integrated with the inter-portlet communication component, which allows for an event-style approach to interface programming.

The following sub sections describe certain concepts and components that are relevant to this project.

h3. Inter-portlet Communication

The caGrid Portal consists of several applications. An application consists of one or more portlets. When portlets work together to constitute an application, they need to be able to share state information. This sharing of information among portlets is referred to as _inter-portlet communication{_}. Unlike servlets, portlets are not URL-addressable, so you cannot use HTTP request parameters to pass information among portlets. In fact, there is no standard way to handle communication among portlets (JSR-286 will address this), but the caGrid Portal's portlets project defines a set of interfaces and default implementations for managing inter-portlet communication, and this approach will work in any JSR-168 compliant portlet container.

The approach used here models inter-portlet communication channels as logical topics. That is, a single portlets can produce a message that multiple portlets can consume. The default implementation uses multiple, inter-linked queues that are maintained in the scope of the user's session. The following diagram shows the interfaces and classes involved.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddav67c8152d7f6fcdec730f85040a8df37c.png|caption=Figure 3: Inter-portlet messaging framework}

In the typical scenario, a controller that needs to produce a message (i.e. the producer) will have a collaborator that is an instance of InterPortletMessageSender. Spring is used to inject an instance of InterPortletMessageAdapter into the producer. In the same manner, an instance of InterPortletMessageAdapter is injected into the consumer controller, to satisfy its dependency to an InterPortletMessageReceiver collaborator.

InterPortletMessageAdapter implements the logical topics based on inter-linked queues that are configured through standard portlet preferences. The following excerpt from portlet.xml shows how this communication is configured.
{code}
...
<preference>
<name>selectedNewsItemOutputQueueName</name>
<value>newssummary:selectedNewsItem</value>
</preference>
<preference>
<name>newssummary:selectedNewsItem</name>
<value>news:selectedNewsItem</value>
</preference>
\\
...
<preference>
<name>selectedNewsItemInputQueueName</name>
<value>news:selectedNewsItem</value>
</preference>
...
{code}
*Figure 4: Inter-portlet queue configuration*

In this case, the newssummary portlet needs to send a message to the news portlet that indicates which news item has been selected by the user. The first XML snippet configures two of the newssummery portlet's preferences: selectedNewsItemOutputQueueName and newssummary:selectedNewsItem. The first preference indicates what the name of the output queue is. The second preference links the output queue to one or more input queues. (Portlet preferences can have multiple values). The second XML snippet configures one of the news portlet's preferences: selectNewsItemInputQueueName. This preference indicates the name of the input queue.

In general, a portlet will send messages during the action phase and receive messages during the render phase of a request. The reason this is so is that the portlet standard does not specify any standard ordering of render request processing. However, it does specify 1) that at most one portlet can receive an action request during an HTTP request; 2) that all other portlets will receive a render request; and 3) that the action request will be handled prior to all render requests. By sending messages during the action phase of a request, we can ensure that all configured consumers actually receive the message.

Inter-portlet communication channels are implemented as queues that are maintained in the HTTP session by an instance of SessionBasedInterPortletMessageManager. Unlike servlets which have access to a single session scope, portlets have access to two session scopes: PORTLET_SCOPE and APPLICATION_SCOPE. Attributes in the PORTLET_SCOPE are not visible to other portlets while attributes in the APPLICATION_SCOPE are. Therefore, in order to allow different portlets to send to and receive from the same queue, the SessionBasedInterPortletMessageManager is created in the APPLICATION_SCOPE.

This is achieved through the scoping features provided by the Spring container. Specifically, a SessionBasedInterPortletMessageManager bean is declared as having the _globalSession_ scope, which equates to APPLICATION_SCOPE. That means, a single instance of this will be created for each HTTP session. The Spring container uses AOP to ensure that instances of InterPortletMessageAdapter, which have singleton scope and are therefore instantiated once per Spring application context, receive references to the SessionBasedInterPortletMessageManager in the current HTTP session. More details about this are provided in the Spring reference documentation [here|http://static.springframework.org/spring/docs/2.5.x/reference/beans.html#beans-factory-scopes-other-injection].

While this approach to inter-portlet communication works for portlet applications that have been deployed in the same web application, it would not work for portlets that are deployed in different web applications or in different application services (possibly running on different machines). Similarly, this approach doesn't allow for communication with WSRP producers. However, one could accommodate these kinds of scenarios by providing an different implementation of the InterPortletMessageManager interface, which can then be plugged in by modifying the Spring configuration.

h3. Portlet Wiring

Another aspect of the design of the caGrid Portal's portlet applications is that portlets should be able to be _wired together_ at run-time. That means, one should be able to configure how and which portlets communicate with one another. This is to support the goal of, at some point in the future, allowing users to customize their workspaces.

Two features of the InterPortletMessageAdapter implementation facilitate the wiring together of portlets. First, input and output queue names are configured as standard portlet preferences. This allows a portal user with the right privileges to modify these preferences at run-time, through the portal. So, for example, one could add a new portlet to the workspace and then configure it to listen to the messages that are produced be an existing portlet. Of course, issues of interoperability must be addressed. Second, the InterPortletMessageAdapter can be turned on or off through the value of another _mode_ preference. This enables the portlet to run in STANDALONE or INTERPORTLET mode. The following snippet shows the mode preference used by the newssummary portlet.

{code}
...
<preference>
<name>newsItemSelectMode</name>
<value>INTERPORTLET</value>
</preference> ...
{code}
*Figure 5: Configuration of inter-portlet communication mode*

As this snippet shows, the newssummary portlet is configured to work in INTERPORTLET mode for newsItemSelect messages.

The 2.0 release of the portal doesn't provide the user interface that is necessary to make use of this run-time wiring feature, but future versions may.

h3. Tab Model

Both the cagriddiscovery and cagridquery portlets use tabs to manage UI navigation. The tab implementation needs to support several features:
* Users should be able to click on tabs and sub tabs to get to different functionality
* Tabs and sub tabs should be able to render one or more associated views
* Tabs should maintain the state of the interface across page refreshes (which are quite frequent in portlet applications)
* Tabs should be visible or invisible based on the authorization policy
* User actions in one portlet should be able to cause a particular tab or sub tab in another portlet to be displayed

The tabs themselves are rendered by by a JSP that is included in other views. In the cagridquery portlet, that file is portlets/src/war/WEB-INF/jsp/query/tabs.jspf. This JSP looks for a request attribute named _tabModel_ which it interacts with the render the tabs. The TabModel object contains a list of Tab objects. Each of the Tab objects represents a top-level tab. Children of a Tab represent sub tabs. So, each Tab is really a tree, however TabModel will construct a row-based view of the tabs based on the current state.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddavec7eabac34646bc1665fc0018abbb126.png|caption=Figure 6: Tab model}

TabModel objects are constructed through Spring bean definitions. Look at the _queryTabModel_ bean in portlets/src/war/WEB-INF/context/query-portlet.xml for an example.

h4. Tab Navigation

To allow users to navigate among tabs, tab labels are hyperlinked, and these hyperlinks create portlet render requests. Included in the render request is a parameter that indicates the new selected tab path. The request must be routed to the appropriate controller to render the view. Routing of requests in Spring Portlet MVC is accomplished by implementations of the HandlerMapping interface. HandlerMapping instances take a PortletRequest and return a HandlerExecutionChain that may include interceptors and must include the ultimate handler of the request. In the caGrid Portal portlets project, a hierarchy of custom HandlerMapping classes has been provided to facilitate the requirements of tabbed navigation. The result of this implementation is that a particular tab path is selected and that tab path is mapped to a controller that prepares the view.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddav40f6fe0caf416ce7b86000abc8072316.png|caption=Figure 7: Tabbed navigation components}

By default, TabHandlerMapping will select the tab path that is specified by the selectedTabPath parameter. The SelectedPathDelegationTabHandlerMapping implementation delegates the responsibility of determining the selected tab path to instances of SelectedPathHandler, which can be chained. AbstractInterPortletMessageSelectedPathHandler uses an instance of InterPortletMessageReceiver to determine the selected tab path based on the presence or absence of a message. The diagram above shows one concrete implementation of this handler.

h4. Tab Views

By default, each tab leaf will be rendered as a single view with a single tab in the row below the row of its parent tab. However, sometimes you want render multiple views in the same sub tab. To achieve this using the tab model, you create _invisible_ child tabs of the tab in which these views should be displayed. The invisible child tabs of a particular tab represent all of the possible views that a sub tab can display. Look at the _query/builder_ sub tab in the _queryTabModel_ bean in portlets/src/war/WEB-INF/context/query-portlet.xml for an example.

h4. Tab State

Since portal pages refresh quite often, it is necessary to maintain state information about which tab is currently selected. This is achieved by associated a TabModel object with the HTTP session. The Spring container maintains TabModel objects in the _session_ scope, which maps to the standard PORTLET_SCOPE. The same AOP approach that was described in the section on inter-portlet communication, above, is used here. The TabHandlerMapping declares a reference to a TabModel object and selects the appropriate tab or sub tab based on the request parameters or inter-portlet messages.

h4. Tab Security

For the 2.0 release of the portal, the only security requirement that relates to tabs is that certain tabs (e.g. the My Shared Queries tab) must be visible only to authenticated users. Tab objects have a Boolean _requiresAuthn_ property that the rendering JSP can use to determine if a tab should be displayed. See portlets/src/war/WEB-INF/jsp/query/tabs.jspf for an example. In this case, the JSP checks if a _portalUser_ object exists in the request. The PortalUserRequestInterceptor object is responsible for putting the PortalUser object in the request scope.

h4. Tabs and Inter-portlet Communication

As described in the _Tab Navigation_ section, SelectedPathDelegatingTabHandlerMapping delegates to a chain of SelectedPathHandler objects. The first object in that chain to return a non-null path wins (i.e. that is the path that is used). Classes of that extend AbstractInterPortletMessageSelectedPathHandler return the selected tab path based on the presence or absence of an inter-portlet message. This allows events in one portlet to cause certain sub tabs in another portlet to be displayed.

h3. Render Phase and Action Phase Controller Separation

Spring handler mappings are processed twice during an action request: before the action phase and before the render phase. This allows one to handle the action phase with one controller and the render phase with another controller. This approach is used quite often in the caGrid Portal applications because it allows action and view preparation logic to be reused more easily. In order to have the render phase handled by a different controller than the action phase, typically the controller will include a parameter in the RenderResponse object which causes the second processing of the handler mapping to return a different controller. Since this approach is used so frequently in the portal code, a couple of abstract classes have been created to handle common logic.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddavfe594f0251add67fd10a6f41adc9849b.png|caption=Figure 8: Base classes for render and action phase controllers}

Sub classes of AbstractActionResponseHandlerCommandController must provide the logic that processes the request (e.g. making state changes to model objects),. But, modification of the RenderResponse is delegated to an implementation of CommandActionResponseHandler. This is useful, for example if you want to use the same action logic both within the tab navigation scenario and outside of that scenario.

Together, these base classes reproduce some of the functionality provided by Spring's AbstractFormController and its sub classes. However, these classes make it easier to separate the action and render preparation logic. For simpler interactions, for example, editing and creating news feeds, classes that extend SimpleFormController are used to process both the action and render phases.

h3. AJAX

The portal currently uses AJAX for two things in the cagridquery portlet application: to update the status of running queries (History tab); to render tree views of CQL queries, UML classes, and service metadata. This section walks through how the former works step-by-step. The next section goes into further detail about tree rendering.

The portal uses the DWR framework to implement AJAX. Here's a quick overview of DWR. For more details, go [here|http://getahead.org/dwr/documentation].

DWR allows one to expose some Java class through Javascript. That means, you can write Javascript that calls methods defined on some Java class that you provide. The way this works is that the DWR servlet generates a Javascript object that has the same method signatures as your Java class. When your Javascript code calls these methods, DWR client-side code translates these into asynchronous HTTP requests to the DWR servlet, which in turn calls your Java class. DWR then serializes the results as Javascript objects and sends them back the client.

There is a fair amount of configuration necessary to get DWR, Spring, portlets, and servlets all working together. That's why this section walks through it.

h4. web.xml

The DWR servlet must be declared in web.xml. Look at the dwr-invoker servlet and servlet-mapping in portlets/src/war/WEB-INF/web.xml. Also look at the contextConfigLocation context-param. Notice the file named facades.xml is included in the expression. This file is described later.

h4. dwr.xml

The DWR servlet expects to find a file called dwr.xml in the WEB-INF directory. This file is used to configure what and how Java classes will be exposed. Look at portlets/src/war/WEB-INF/dwr.xml. Notice that there is a create element with a javascript attribute of QueryHistoryFacade. This is the name of the Javascript object that will be generated by DWR. Also notice that the value of the creator attribute is spring. This means that Spring will be used to instantiate the Java object which DWR will expose. The next param element indicates that a bean with the id of cqlQueryHistoryFacade should be used. DWR will check the servlet context for a WebApplicationContext and then use that to retrieved the required beans.

Also notice the converter elements. You can use DWR pass any Java object as parameter or result between client and server. But you must specifically tell DWR which objects are allow to serialized, unless they are in the java.* package.

h4. facades.xml

The server-side Java objects that handle AJAX requests are referred to here as facades. The facades.xml file is just where the façade objects, and those that they directly use, are declared. But, since DWR uses the Spring container to retrieve Java objects, those objects have access to all the other objects that are managed by the Spring container.

h4. JSPs

Look at portlets/src/war/WEB-INF/jsp/query/history.jsp. At the top of this page you should see:

{code}
<script type='text/javascript' src='<c:url value="/dwr/interface/QueryHistoryFacade.js"/>'></script>
<script type='text/javascript' src='<c:url value="/dwr/engine.js"/>'></script>
<script type='text/javascript' src='<c:url value="/dwr/util.js"/>'></script>
{code}

The first line causes the DWR servlet to generate the QueryHistoryFacade Javascript object. The next lines include DWR utility code. Further down in that JSP, you'll see the code invoke methods on the QueryHistoryFacade object. In dwr.xml and facades.xml, you see that the QueryHistoryFacade Javascript object is implemented on the server side by the CQLQueryHistoryFacade Java class. The render method of this class uses the DWR WebContext interface to delegate the responsibility of rendering a CQLQueryInstance object to a JSP. In this case, the request goes directly to the JSP, but as described later, the request could also have been configured to go to a Spring DispatcherServlet in order to make use of the Spring MVC framework, for example, to manage Hibernate sessions.

This is the general pattern that is used in the portal to implement AJAX. DWR translates a client-side Javascript call to a server-side façade, which may invoke business logic and return Java objects, or delegate to a servlet to render some view.

h3. Trees

The portal renders service metadata (including domain models), working CQL queries, and UML classes (in the CQL query builder) as trees. This allows the user to navigate the data structure by expanding and collapsing nodes. Two functional requirements for these tree views are that 1) the trees must maintain state across page refreshes; and 2) the client-side HTML and Javascript that renders the tree must be generated incrementally. This second requirement is a result of the fact that it is impractical, in the case of a domain model, or impossible, in the case of the CQL query, to generate the entire tree structure initially. AJAX is used to incrementally render the tree when the user expands a previously unopened node. This saves the cost of re-rendering the entire portlet window.

A common set of components is used to render all of the trees. A listener pattern is used to provide customization.

{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddavb583497c5b85740411c1e04706148748.png|caption=Figure 9: Tree rendering components}


The tree itself wraps portal domain objects with which the user is interacting. Each TreeNode object has a reference to a domain object or a collection of domain objects. DWR translates client-side calls to the server-side TreeFacade object. The call to openNode or closeNode includes the path to the node in the tree. TreeFacade calls the find method on the root node of the tree to retrieve the indicated node. Then TreeFacade iterates through the list of TreeNodeListener instances, calling either onOpen or onClose, and passing in a reference to the root node. It is the responsibility of the TreeNodeListeners to modify the state of the tree (e.g. adding nodes, change the state of a node). Once all listeners have received the event, TreeFacade calls the render method of the TreeNodeRenderer instance, which returns a string. This string is passed back to the client, parsed as HTML, and placed into the DOM of the page.

Spring is used to construct a different TreeFacade instance for each type of tree. The TreeNodeListener and TreeNodeRenderer implementations provide different behavior that is appropriate to the type of tree being displayed. There are three implementations of TreeNodeListener:

* CQLQueryTreeNodeListener
* ServiceMetadataTreeNodeListener
* UMLClassTreeNodeListener

ServletRenderer is the only implementation of TreeNodeRenderer. This implementation simply uses the DWR WebContext API to delegate rendering to a configured servlet. TreeNode rendering is handled by Spring MVC. The tree servlet is configured in web.xml and tree-servlet.xml. The Spring controllers that are configured in tree-servlet.xml are responsible for locating the object that should be rendered and populating the model with that object before passing it to the view technology, which is JSP.

h4. Custom JSP Tree Tag

JSPs are used to render tree views. Since AJAX is used to allow the tree to be built incrementally, usually only a single tree node must be rendered. Furthermore, client-side Javascript ensures that if a particular tree node has been previously rendered (e.g. the user has expanded and subsequently collapsed the node), it is not rendered again. However, when a full page is refreshed (e.g. by some action in another portlet), all the visible nodes of the tree must be re-rendered.

A custom JSP tag is provided so that a single JSP view can render the entire tree. That tag is defined in portlets/src/war/WEB-INF/tags/tree.tag. The tree tag contains recursive logic that will navigate through tree of TreeNode objects and render each node. The tag defines two attributes: nodeFragment and contentFragment. The values of these attributes are interpreted as JSP fragments, and invoke by the tree tag using the jsp:invoke action. These enable the pages that use this tag to customize how tree nodes and the contents of those nodes are rendered, while re-using common logic to render the basic tree structure and controls. The tag exposes two variables: currNode and currChildNode, which are TreeNode objects. These are available within the scope of the tag so that the nodeFragment and contentFragment JSP fragments can access these objects.

In the case of the service metadata tree, the tree nodes contain many different types of domain objects, each of which should be rendered differently. Since a single JSP view may need to render an entire tree structure, the JSP fragment that is rendering the content of a node must contain conditional logic to render the current node based on its type. Look at portlets/src/war/WEB-INF/jsp/disc/details/service/serviceDetails.jspf as an example. This fragment uses the JSTL choose control structure to include the appropriate JSP based on the type of the current node.

h4. Portlet-to-Servlet Communication

Both portlets and servlets are involved in preparing a model to be rendered in JSP views. In the case of trees, these portlets and servlets need access to the same model object, which is the tree of TreeNode objects. To achieve this, TreeFacadeObjects, which maintain a reference to the current tree, are placed in the Spring globalSession scope. This allows both portlets and servlets to reference the same objects. AOP proxying is used to ensure that controller instances, which reference them, operated on the right instances. That is, at most three TreeFacade instances are created per HTTP session. Each maintains the currently selected tree: service metadata, CQL query or UML class.

Another difficulty that arises when mixing portlet and servlet technology is the generation of portlet URLs by servlets (or the JSP views returned by Spring MVC controllers). For example, often a servlet will need to construct a URL that will invoke a portlet action. But, since the PortletContext is not available to within the servlet application, there isn't enough information available to construct this URL. In the caGrid Portal application, his problem has been addressed by constructing a base portlet URL from a portlet JSP view, and then passing that URL as a parameter to the servlet. The servlet JSP view can then use string manipulation to place parameter values into the URL.

An example of this can be seen in portlets/src/war/WEB-INF/jsp/query/builder.jsp and portlets/src/war/WEB-INF/jsp/query/builder/criteriaBean.jsp. The editCriterionUrl value is constructed in the portlet JSP view builder.jsp. Then it is passed in the AJAX call as a parameter and modified in the servlet JSP view criteriaBean.jsp.

h3. Cross-Site Scripting (XSS) Prevention

Since the portal allows user to enter information that will be displayed to other users, there is a risk of cross-site scripting attacks. To address this, the XSSFilterEditor custom bean property editor is used to bind parameter values to command objects in cases where an XSS attack could be attempted. Look at the initBinder method of UpdatedSharedQueryController for an example. If text that could be dangerous is detected, a binding exception is created and the user is warned not to enter text that could be interpreted as HTML, XML or some scripting language.

h2. The portals Project
----
This project contains components that have dependencies on Liferay. These components include:

* Custom authentication plug-in
* Custom layout application
* Custom caGrid Portal theme application
* Customizations of the Liferay portal distribution
* Liferay installation/configuration code

h3. Authentication Plug-In

The caGrid WebSSO component was not available to be included in this release of the portal. However, it will be included in the next release. For this release, an implementation is provided that interacts with directly with an AuthenticationService and a Dorian service to authenticate the user and retrieve grid credentials.

Liferay provides a couple of different extension points where one can plug in a custom authentication system. It even supports SSO through CAS out of the box. However, this implementation assumes that user information will be pull from some LDAP server. A good discussion of Liferay authentication extension points is provided on the Liferay wiki [here|http://wiki.liferay.com/index.php/Developing_a_Custom_Authentication_System|http://wiki.liferay.com/index.php/Developing_a_Custom_Authentication_System].
An example of uses Acegi Security with Liferay is [here|http://wiki.liferay.com/index.php/Acegi_Security_for_Liferay].

The approach used in the caGrid Portal is to provide an implementation of com.liferay.portal.security.auth.AutoLogin, which is found in portals/liferay/src/java/gov/nih/nci/cagrid/portal/liferay/security/DirectAutoLogin.java. This class is included in the auto-login chain that is configured in portals/liferay/src/war/WEB-INF/classes/portal-ext.properties. DirectAutoLogin checks the request for a *ticket* parameter. This ticket is then used to retrieve the authenticated user from the database.

The actual authentication is handled by the directauthn portlet application, which uses the AuthnService (from the security project) to authenticate a user to an Identity Federation Service (IFS), for example the NCICB's Dorian service, and Identity Provider (IdP), for example the NCICB's AuthenticationService.

h3. Custom Theme

Liferay defines a standard structure for custom themes. These themes can be deployed as separate web applications which are hot-deployed and automatically registered to the portal. A custom theme called cagrid-minimal has been created for the caGrid Portal. This theme is found under portals/liferay/src/themes/cagrid-minimal. Most of the work of creating the custom them involves editing the following files:

* templates/portal_normal.vm: renders the portal page container
* templates/navigation.vm: renders the top-level tabs
* templates/dock.vm: renders the doc
* css/custom.css: override styles in other CSS files

More information about developing themes is [here|http://wiki.liferay.com/index.php/Themes].

h3. Custom Layout

A layout determines how portlets can be arranged on the page, e.g. two columns versus three columns. A custom layout was created for caGrid Portal to permit three columns with widths of 20%, 60%, and 20%. This layout is found at portals/liferay/src/layouts.

h3. Liferay Archive Creation

The basic structure of the portal site is created through the Liferay portal interface. For example, the portal developer selects portlets to be displayed on the page, positions portals on the page, selects the theme and layout, add tabs. All of this is done through the portal interface. See the Liferay portal administrators guide for more details. When the site structure has been created, it can be exported and saved as a Liferay Archive (LAR) file. The LAR file can then be imported into another portal instance (of the same release number) to automatically set up the site structure.

h3. Secure Channel Processing

Certain portal pages should be encrypted (e.g. the login and registration pages) and other pages should not be encrypted (e.g. a page that is displaying a Google map, since Google doesn't support an HTTPS API). The Acegi ChannelProcessingFilter is used to handle switching between the HTTP and HTTPS protocols. The Acegi filter chain configuration is defined in portals/liferay/src/war/WEB-INF/applicationContext-acegi-security.xml.

h3. Servlet-to-Portlet Communication

Upon successful authentication, the DirectAutoLogin class must create a PortalUser object and put it in the request so that it is available to the portlet applications. The portal must be configured to enable portlets to retrieve objects that the portal has put into the session or the request. A description of this process is [here|http://wiki.liferay.com/index.php/Liferay_FAQ#How_do_we_use_the_request_to_share_attributes_between_portlets_deployed_as_individual_WARs.3F].

In portals/liferay/src/war/WEB-INF/classes/portal-ext.properties, you can see that the portal has been configured to share request attributes that start with the CAGRIDPORTAL_ATTS_ prefix.

h2. The _security_ Project

This project contains security related components that are common among projects. For example, the portlets and portal project use the same encryption services.

h2. The _db_ Project

The db project contains domain objects and DAO implementations that are used by the other projects. The portal persistence layer uses Hibernate with annotations. Hibernate tools are used to generate the DDL. The DAOs uses Spring's Hibernate template abstraction to work with the Hibernate Session API.

The following diagram displays the main domain objects.
{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddavdd65681c3c562802945dbba504d8c6f9.png|caption=Figure 10: Main domain objects}

The following diagram shows the domain objects that are involved in data service querying.
{image-with-caption:tablealign=center|pagename=portal20:Design|attachmentname=worddav6d2b4f3f9812bd81dd35509384ed9d98.png|caption=Figure 11: Data service query classes}

As mentioned above, the DDL is generated by Hibernate tools. So the process of changing the schema involves first adding/editing domain model classes and annotations, and then running the Hibernate tools to regenerate the DDL. However, for completeness, the DDL as of the 2.0 release is included [here|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/~checkout~/cagrid-1-0/Documentation/docs/portal/portal2/design/arch/cagrid-portal-2.0-erd.jpg?rev=1.1;content-type=image%2Fjpeg;cvsroot=cagrid-1-0].

h1. Testing
----
h2. Unit Tests

Each project contains a suite of JUnit tests, which can be run individually using Ant targets like this: db:test, or aggr:test, etc. All tests can be run with "test" target. A consolidated HTML test report is placed in the docs/test/report directory. Database tests use the embedded HSQLDB, so no database set up is required.

Following is the list of JUnit test suites:

* db/test/src/java/gov/nih/nci/cagrid/portal/dao/GridServiceDaoTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/metachange/MetadataChangeTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/regsvc/RegisteredServiceMockTest.java
* aggr/test/src/java/gov/nih/nci/cagrid/portal/aggr/status/ServiceStatusTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/query/results/ExportResultsTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/util/HTMLInputFilterTest.java
* portlets/test/src/java/gov/nih/nci/cagrid/portal/portlet/util/PortletUtilsTest.java

h2. System Tests

The suite of system tests has been defined [here|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/~checkout~/cagrid-1-0/Documentation/docs/portal/portal2/testing/caGRID_Portal_2.0_test_cases.xls?rev=1.1;content-type=application%2Foctet-stream;cvsroot=cagrid-1-0].

h1. References
----
h2. caGrid References

# Monitoring and Discovery Portal Enterprise Architect Project [GForge SCM Link|http://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/cagrid-1-0/caGrid/projects/portal/docs/project/Portal.EAP?cvsroot=cagrid-1-0;f=h;ln=1]
# caGrid 1.0 Gforge website [http://gforge.nci.nih.gov/projects/cagrid-1-0/]
# caGrid Homepage [https://cabig.nci.nih.gov/workspaces/Architecture/caGrid/]

h2. Technical Manuals/Articles
-----
# Java Timer Tasks [http://java.sun.com/j2se/1.3/docs/api/java/util/TimerTask.html]
# Spring framework [http://www.springframework.org/]
# Java Bean Specification: [http://java.sun.com/products/javabeans/docs/spec.html]
# Foundations of Object-Relational Mapping: [http://www.chimu.com/publications/objectRelational/]
# Object-Relational Mapping articles and products: [http://www.service-architecture.com/object-relational-mapping/]
# Hibernate Reference Documentation: [http://www.hibernate.org/hib_docs/reference/en/html]
# Basic O/R Mapping: [http://www.hibernate.org/hib_docs/reference/en/html/mapping.html]
# Java Programming: [http://java.sun.com/learning/new2java/index.html]
# Javadoc tool: [http://java.sun.com/j2se/javadoc/]
# JUnit: [http://junit.sourceforge.net/]
# Extensible Markup Language: [http://www.w3.org/TR/REC-xml/]

h2. caBIG Material
----

# *caBIG:* [http://cabig.nci.nih.gov/]
# *caBIG Compatibility Guidelines*: [http://cabig.nci.nih.gov/guidelines_documentation]

h2. caCORE Material
----
# *caCORE:* [http://ncicb.nci.nih.gov/core]
# *caBIO:* [http://ncicb.nci.nih.gov/core/caBIO]
# *caDSR:* [http://ncicb.nci.nih.gov/core/caDSR]
# *EVS:* [http://ncicb.nci.nih.gov/core/EVS]
# *CSM:* [http://ncicb.nci.nih.gov/core/CSM]

h1. Glossary
----
||Term||Definition||
|API|Application Programming Interface|
|caArray|cancer Array Informatics|
|caBIG|cancer Biomedical Informatics Grid|
|caBIO|Cancer Bioinformatics Infrastructure Objects|
|caCORE|cancer Common Ontologic Representation Environment|
|caDSR|Cancer Data Standards Repository|
|caMOD|Cancer Models Database |
|cardinality|Cardinality describes the minimum and maximum number of associated objects within a set|
|CDE|Common Data Element|
|CGAP|Cancer Genome Anatomy Project|
|CMAP|Cancer Molecular Analysis Project|
|CN|Common Name |
|CS|Classification Scheme|
|CSI|Classification Scheme Item|
|CSM|Common Security Module|
|CTEP|Cancer Therapy Evaluation Program|
|CUI|Concept Unique Identifier|
|CVS|Concurrent Versions System|
|DAML|DARPA Agent Markup Language|
|DAO|Data Access Objects|
|DARPA|Defense Advanced Research Projects Agency|
|DAS|Distributed Annotation System|
|DL|Description Logic|
|EA|Enterprise Architect|
|EBI|European Bioinformatics Institute|
|EVS|Enterprise Vocabulary Services|
|GAI|CGAP Genetic Annotation Initiative|
|GEDP|Gene Expression Data Portal|
|GIS|Geographical Information System|
|HIPPA|Health Insurance Portability and Accountability Act|
|HLGT|High Level Group Term|
|HLT|High Level Term|
|HTTP|Hypertext Transfer Protocol|
|ISO|International Organization for Standardization|
|JAAS|Java Authentication and Authorization Service|
|JAR|Java Archive|
|Javadoc|Tool for generating API documentation in HTML format from doc comments in source code ([http://java.sun.com/j2se/javadoc/|http://java.sun.com/j2se/javadoc/])|
|JDBC|Java Database Connectivity|
|JET|Java Emitter Templates|
|JMI|Java Metadata Interface|
|JSP|JavaServer Pages|
|JUnit|A simple framework to write repeatable tests ([http://junit.sourceforge.net/|http://junit.sourceforge.net/])|
|JSF|Java Server Faces|
|LDAP|Lightweight Directory Access Protocol |
|LLT|Lowest Level Term|
|LOINC|Logical Observation Identifier Names and Codes|
|MAGE|MicroArray and Gene Expression|
|MAGE-OM|MicroArray Gene Expression - Object Model|
|MedDRA|Medical Dictionary for Regulatory Activities|
|metadata|Definitional data that provides information about or documentation of other data.|
|MGED|Microarray Gene Expression Data|
|MMHCC|Mouse Models of Human Cancers Consortium|
|MO|MGED Ontology|
|multiplicity|Multiplicity of an association end indicates the number of objects of the class on that end may be associated with a single object of the class on the other end|
|NCI|National Cancer Institute|
|NCICB|National Cancer Institute Center for Bioinformatics|
|OIL|Ontology Inference Layer|
|OilEd|Ontology editor allowing you to build ontologies using DAML+OIL|
|OLLT|Obsolete Lower Level Terms|
|OMG|Object Management Group|
|ORM|Object Relational Mapping|
|PT|Preferred Term|
|RDBMS|Relational Database Management System|
|SDK|Software Development Kit|
|Semantic connector|A development kit to link model elements to NCICB EVS concepts.|
|SOC|System Organ Class|
|SPORE|Specialized Programs of Research |
|SQL|Structured Query Language|
|SSC|Special Search Categories|
|UI|User Interface|
|UID|User Identification|
|UML|Unified Modeling Language|
|UMLS|Unified Medical Language System|
|UPT|User Provisioning Tool|
|URL|Uniform Resource Locators|
|VD|Value Domain|
|WAR|Web Application Archive|
|WSDL|Web Services Description Language|
|XML|Extensible Markup Language ([http://www.w3.org/TR/REC-xml/]) - XML is a subset of Standard Generalized Markup Language (SGML). Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML. XML has been designed for ease of implementation and for interoperability with both SGML and HTML|