Page Comparison - Administrators Guide (v.4 vs v.5) - Federated Query Processor 1.4

Version 4 by Clayton Clark
on Mar 29, 2010 14:36.

<< Changes from 3 to 4

compared with

Version 5 by Clayton Clark
on Jun 17, 2010 10:02.

Changes from 5 to 6 >>

Key

This line was removed.

This word was removed. This word was added.

This line was added.

Changes (37)

View Page History

...

----
{cagrid-1.4-docs-nav:name=FQP|guidetype=Administrators}

{cagridtoc:exclude=Federated Query Processor 1.4 Administrators Guide}

{cagridroundpanel}
{pre:class=cagridheaderfont}Contents{pre}
{toc:outline=true|exclude=Federated Query Processor 1.4 Administrators Guide|style=none}
{cagridroundpanel}

h1. Prerequisites
----
The Federated Query Processor service does not require any software or special system configuration beyond the standard caGrid stack.

To make use of the caGrid Transfer infrastructure, the transfer project is required, and the FQP service must be deployed to the *same* Tomcat container as the transfer service.

The performance of the FQP service may benefit from large amounts of RAM and multiple processor cores for handling concurrent query operations, but this is not strictly required to deploy the service.

...

{tip}Once you have installed caGrid, the FQP software can be found in the directory location where you installed caGrid, in the _caGrid/projects/fqp_ directory. This guide will refer to that location as *{_}FQP_HOME{_}*

{tip}

{tip}To install caGrid/FederatedQueryProcessor and set up a container, please complete the following steps:

{include:caGrid14:Install caGrid and Configure a Secure Container Using the caGrid 1.4 Installer}

h1. Configuration

h1. Configuring the Service

----

{tip}To simply deploy the FQP service with the default configuration, all you really need to edit is the service's standard ServiceMetadata, by following [these instructions|#Edit Service Metadata].

{tip}

h2. Edit Service Properties

The Federated Query Processor service may be configured by changing values specified in the service.properties file found in the root directory of the FQP distribution.

...

* *maxTargetServicesPerQuery*
** Default value: 12
** Type: Integer
** Controls the maximum number of target data services which may be included in any single DCQL query. If a client attempts to execute a query which specifies more than this number of target data services, an exception will be thrown and the query will not execute. If this value is set to zero (0), the number of services is unlimited.
* *maxRetryTimeout*
** Default value: 300
** Type: Integer
** Controls the maximum number of seconds a client may request the FQP service to wait between retrying queries to target data services which failed to respond correctly. If the client specifies a value greater than this, an exception will be thrown and the query will not execute. If this value is set to zero (0), the maximum timeout is unlimited.
* *maxRetries*
** Default value: 4
** Type: Integer
** Controls the maximum number of retries a client may request the FQP service to perform when retrying to execute queries to target data services which failed to respond correctly. If the client specifies a value greater than this, an exception will be thrown and the query will not execute. If this value is set to zero (0), the maximum number of retries is unlimited.
* *threadPoolSize*
** Default value: 10
** Type: Integer
** Controls the size of the thread pool used by the FQP service to perform DCQL and perform final query aggregation against target data services. Increasing this value may improve performance and responsiveness of the FQP service at the expense of potentially using more server resources.
* *initialResultLeaseInMinutes*
** Default value: 30
** Type: Integer
** Controls the initial time-to-live (lease time) of FederatedQueryResults resources. The value is specified in minutes. Unless the client explicitly requests a termination time for their results resource more distant in the future, after this time has elapsed, the resource will be destroyed. When the resource is destroyed, any remaining query execution tasks are terminated and any DCQL query results are lost.

These properties may be configured at deployment time by the Introduce service deployment GUI, or by directly editing the service.properties file before deploying it.

h2. Edit Service Metadata

The FQP service provides a Resource Property, which acts as metadata for its clients and describes the service's capabilities and information on where it is being hosted. This Resource Property is the caGrid standard _ServiceMetadata_, and is loaded from a file on the filesystem (_serviceMetadata.xml_), which is located in the *{_}FQP_HOME{_}*/etc directory, and is deployed with the service upon [deployment|#Deployment]. This file is fully populated, except for the information about where the service is being hosted. Before deploying the service, you must edit this file and provide the information which describes your organization.

FQP provides service metadata to clients and other services that describes information about the service, operations supported by the service, and information on the organization hosting the service.

{tip}If you aren't comfortable manually editing XML, you can use Introduce's graphical editor instead when you deploy the service.
{tip}Below is the relevant section of the file which you should edit:

Edit the service metadata to reflect your organization as follows:

{code:xml} <ns1:hostingResearchCenter>
<ns18:ResearchCenter displayName="" shortName="" xmlns:ns18="gme://caGrid.caBIG/1.0/gov.nih.nci.cagrid.metadata.common">
<ns18:Address country="" locality="" postalCode="" stateProvince="" street1="" street2=""/>
<ns18:ResearchCenterDescription description="" homepageURL="" imageURL="" rssNewsURL=""/>

# Open the FQP service metadata file, *{_}FQP_HOME/etc/serviceMetadata.xml{_}*.
# In the *{_}hostingResearchCenter{_}* element near the bottom of the file, do the following.
## Supply your *ResearchCenter* infomation.
## Supply your *Address*. This is the address that is used when mapping your service on the caGrid Portal.
## Supply the *PointOfContact*. This is the person responsible for maintaining the service.
A completed example:
{code:XML}<ns1:hostingResearchCenter>
<ns53:ResearchCenter displayName="Ohio State University" shortName="OSU" xmlns:ns53="gme://caGrid.caBIG/1.0/gov.nih.nci.cagrid.metadata.common">
<ns53:Address country="US" locality="Columbus" postalCode="43210" stateProvince="OH" street1="3190 Graves Hall" street2="333 W. 10th Ave."/>

<ns18:pointOfContactCollection> <ns53:pointOfContactCollection>

<ns18:PointOfContact affiliation="" email="" firstName="" lastName="" phoneNumber="" role=""/>

<ns53:PointOfContact affiliation="OSU" email="John.Doe@osumc.edu" firstName="John" lastName="Doe" phoneNumber="(555) 555-5555" role="Maintainer"/>

</ns18:pointOfContactCollection> </ns53:pointOfContactCollection>

</ns18:ResearchCenter> </ns53:ResearchCenter>

</ns1:hostingResearchCenter>
{code}

h1. Deployment
----
{tip}The Federated Query Processor service requires the [caGrid Transfer Service|transfer:Home] be deployed to the same Tomcat or JBoss container the FQP service will be deployed to. The Transfer service _must be deployed first_.
{tip}The Federated Query Processor is an Introduce-created service, and as such, supports all the standard Introduce deployment processes, which are described in the [Introduce Administrator's Guide|introduce14:Administrators Guide##AdministratorsGuide-Deployment Undeployment]

{info:title=Note}By default, FQP registers with and publishes its service metadata to the Index Service. The default Index Service is configured as the Index Service of the target grid you selected when you installed Dorian. You can find configuration details on registering and publishing to the Index Service, including disabling registration and changing which Index Service to register with, on the [Registration and Discovery] page.{info}

For example, to deploy the service to Tomcat from the command line, you can type the following command from the *{_}FQP_HOME{_}* directory:

Once the FQP service has been deployed, start the service container and verify that no error messages are printed to the console.

{terminal}> ant deployTomcat
{terminal}You will then want to start up Tomcat so your service is available. For Tomcat, you can run this command:

h1. Starting FQP
----
{tip}The Federated Query Processor service requires that the [caGrid Transfer Service|transfer:Home] be deployed to the same Tomcat or JBoss container the FQP service will be deployed to. The Transfer service _must be deployed first_.
{tip}
The Federated Query Processor is an Introduce-created service, so supports all the standard Introduce deployment processes, which are described in the [Introduce Administrator's Guide|introduce14:Administrators Guide##AdministratorsGuide-Deployment Undeployment].

*{_}Linux / Unix{_}*
{terminal}> $CATALINA_HOME/bin/startup.sh
{terminal}{*}{_}Windows{_}*
{terminal}> %CATALINA_HOME%\bin\startup.bat
{terminal}You must supply portions of the the [service metadata|#Service Metadata] the FQP service instance will use to register with caGrid index services, specifically the Point of Contact information and the Research Center information. This information can be set via the Introduce service deployment GUI, or by [manually editing $CAGRID_LOCATION/projects/fqp/etc/serviceMetadata.xml|#Service Metadata] and deploying via the command line.

For example, to deploy the service to Tomcat from the command line, type the following command from the *{_}FQP_HOME{_}* directory:

Once the FQP service has been deployed, start the service container and verify that no error messages are printed to the console.

{terminal}> ant deployTomcat{terminal}
Start up Tomcat so that your service is available. For Tomcat, run this command:

h1. Validation

*Linux / Unix*
{terminal}> $CATALINA_HOME/bin/startup.sh{terminal}
*Windows*
{terminal}> %CATALINA_HOME%\bin\startup.bat{terminal}
h1. Validating FQP

----
To validate the service has been deployed and is functioning correctly, use this simple client code to invoke a DCQL query and verify its results:

...

}
}

{code}

{code}The simple main method takes two parameters. The first is the URL of the Federated Query Processor service, and the second is the filename of a DCQL query to execute. Try running the following query, which simply queries the caArray data service for Publications with an ID less than or equal to 10.

{code:xml}<ns1:DCQLQuery xmlns:ns1="http://caGrid.caBIG/1.0/gov.nih.nci.cagrid.dcql">

...

<ns1:targetServiceURL>http://array.nci.nih.gov:80/wsrf/services/cagrid/CaArraySvc</ns1:targetServiceURL>
</ns1:DCQLQuery>

{code}

{code}Executing this query should produce results similar to the following:

{code:xml}<ns1:DCQLQueryResultsCollection xmlns:ns1="http://caGrid.caBIG/1.0/gov.nih.nci.cagrid.dcqlresult">

...

<ns1:DCQLResult targetServiceURL="http://array.nci.nih.gov:80/wsrf/services/cagrid/CaArraySvc">
<ns2:CQLQueryResultCollection targetClassname="gov.nih.nci.caarray.domain.publication.Publication" xmlns:ns2="http://CQL.caBIG/1/gov.nih.nci.cagrid.CQLResultSet">
<ns2:ObjectResult>
<ns2:Publication id="1" authors="Calvo A, Xiao N, Kang J, Best CJ, Leiva I, Emmert-Buck MR, Jorcyk C, Green JE" pages="5325-35" publication="Cancer Research" pubMedId="12235003" title="Alterations in gene expression profiles during prostate cancer progression" uri="http://cancerres.aacrjournals.org/cgi/content/full/62/18/5325" volume="62" year="2002" xmlns:ns2="gme://caArray.caBIG/2.1/gov.nih.nci.caarray.domain.publication">
<ns2:status>
<ns3:Term id="680" value="Published" xmlns:ns3="gme://caArray.caBIG/2.1/gov.nih.nci.caarray.domain.vocabulary">
<ns3:categories/>
</ns3:Term>
</ns2:status>
</ns2:Publication>
</ns2:ObjectResult>
<ns2:ObjectResult>
<ns4:Publication id="10" authors="Rygaard K, Sorenson GD, Pettengill OS, Cate CC, Spang-Thomsen M." pages="5312-5317" publication="Cancer Research" pubMedId="2167152" title="Abnormalities in structure and expression of the retinoblastoma gene in small cell lung cancer cell lines and xenografts in nude mice" volume="50" year="1990" xmlns:ns4="gme://caArray.caBIG/2.1/gov.nih.nci.caarray.domain.publication">
<ns4:status>
<ns5:Term id="680" value="Published" xmlns:ns5="gme://caArray.caBIG/2.1/gov.nih.nci.caarray.domain.vocabulary">
<ns5:categories/>
</ns5:Term>
</ns4:status>
<ns4:type>
<ns6:Term id="47" accession="MO_430" url="http://mged.sourceforge.net/ontologies/MGEDontology.php#journal_article" value="journal_article" xmlns:ns6="gme://caArray.caBIG/2.1/gov.nih.nci.caarray.domain.vocabulary">
<ns6:categories/>
</ns6:Term>
</ns4:type>
</ns4:Publication>
</ns2:ObjectResult>
</ns2:CQLQueryResultCollection>
</ns1:DCQLResult>
</ns1:DCQLQueryResultsCollection>
{code}