----
h1. Federated Query Processor 1.4 Developers Guide
----
{cagrid-1.4-docs-nav:name=FQP|guidetype=Developers}This document is intended to provide information to developers who wish to make use of the Federated Query Processor grid service and local Federated Query Engine APIs.

{cagridtoc:exclude=Federated Query Processor 1.4 Developers Guide}

h1. Prerequisites
----
To get started developing against the FQP APIs, your project will require the Java libraries found in the FQP project's ext/dependencies/jars directory, and those in its build/lib directory.

Developers using Ivy to integrate with the caGrid build artifacts may use the following line in their dependencies:
{code}<dependency rev="latest.integration" org="caGrid" name="fqp" conf="myconfiguration->client"/>
{code}

h1. Federated Query Engine API
----
The Federated Query Engine is the core component of the Federated Query Processor, and can be used as either a standalone API, or within the context of the Federated Query Processor grid service.

h2. Constructing an Instance

There are three constructors for the Federated Query Engine:

* public FederatedQueryEngine(GlobusCredential credential, QueryExecutionParameters executionParameters)
* public FederatedQueryEngine(GlobusCredential credential, QueryExecutionParameters executionParameters, ExecutorService workExecutor)
* public FederatedQueryEngine(GlobusCredential credential, QueryExecutionParameters executionParameters, ExecutorService workExecutor, DomainModelLocator modelLocator)

In all of the above cases, all parameters are optional, and the first two constructors are simply a convenience methods which pass *null* for the additional parameters to the third one, and is in place to maintain backwards compatibility with the Federated Query Engine from caGrid 1.3 and earlier.

The four parameters can be used as follows:

* *credential*
** A Globus client credential can be passed along to the Federared Query Engine, and will be used to query secure data services involved in any DCQL queries issued to the engine.
* *executionParameters*
** Query Execution Parameters (described later) allow the user to define how they'd like the engine to behave with respect to things like target data service failures, retries, and timeouts.
* *workExecutor*
** The Federated Query Engine will perform query related tasks in threads. This has the benefit of potentially greatly speeding up the final stage of query processing, which involves broadcasting the final CQL query generated by the engine to all target data services specified by the DCQL query. The Executor Service passed in through this parameter allows users to control the way those threads are allocated and managed.
* *modelLocator*
** The domain model locator which will be used in operations where query processing requires information from a service's domain model. The domain model may be required when converting CQL 2 to CQL 1 for communicating with caGrid 1.3 or earlier data services and making decisions regarding data type conversion. The FQP tools provide a simple concrete implementation of the domain model locator interface in _gov.nih.nci.cagrid.fqp.common.DefaultDomainModelLocator_.

h2. Query Execution Parameters

The Federated Query Engine allows the caller to specify parameters which control various aspects of query execution. This _QueryExecutionParameters_ data type has been designed to be flexible and extensible for future versions of the Federated Query Processor. For this version, it contains a child data type, _TargetDataServiceQueryBehavior_, which controls how the query engine handles various failure conditions when submitting CQL to the target data services specified in the DCQL query. This type contains three properties:

** If set to *true*, the engine will terminate query processing and throw an exception when querying against any target data service fails for any reason. No query results will be returned.

h2. API Methods

The Federated Query Engine exposes two methods for executing a [DCQL 1|fqp:DCQL], and two methods for executing a [DCQL 2|fqp:DCQL 2] query:

h3. Simple Query Execution

Two *execute* methods exist in the Federated Query Engine; one for each version of DCQL. They both take a single DCQL query parameter and return a single DCQLQueryResultsCollection instance. This method may throw a FederatedQueryProcessingException

{code}
public DCQLQueryResultsCollection execute(DCQLQuery dcqlQuery)
throws FederatedQueryProcessingException
{code}

The version of this method which takes DCQL 1 first converts the query to DCQL 2 then invokes the method which natively takes DCQL 2. The DCQL 2 execution method processes the DCQL 2 query by breaking it down into parts according to foreign join conditions and generating further CQL 2 queries until it has produced a single CQL 2 query which is then distributed to all target data services specified by the query. For data services which do not support CQL 2, the query is first converted to CQL 1, then the results are converted back to CQL 2 results. The results of this final query are placed in the DCQLQueryResultsCollection according to which target data service returned them, and then returned to the caller.

h3. Execute and Aggregate Results

Two *executeAndAggregateResults* methods exist; one for each version of DCQL. Both take a single DCQL or DCQL 2 query parameter and returns a single CQLQueryResults instance. This method may also throw a FederatedQueryProcessingException.

{code}
public CQLQueryResults executeAndAggregateResults(DCQLQuery dcqlQuery)
throws FederatedQueryProcessingException
{code}

The version of this method which takes DCQL 1 first converts the query to DCQL 2 then invokes the method which natively takes DCQL 2. The DCQL 2 execution method processes the DCQL 2 query by breaking it down into parts according to foreign join conditions and generating further CQL 2 queries until it has produced a single CQL 2 query which is then distributed to all target data services specified by the query. For data services which do not support CQL 2, the query is first converted to CQL 1, then the results are converted back to CQL 2 results. The results of this final query are aggregated into a single CQL query results instance, which allows it to be manipulated by existing data service infrastructure tooling (iterators, enumerators, etc), while loosing the context of which target data service produced a given result.

h3. Federated Query Processing Exceptions

These exceptions may be thrown from either public API method when something goes wrong in the course of processing a DCQL query. Several common causes of this exception are:
* Failure of a data service involved in the DCQL query
** Failure handling behavior for target data services is controllable by the Query Execution Parameters used to construct the Federated Query Engine
* Invalid CQL is passed along to a data service (typically due to invalid DCQL originally)
* Bad / unrecognized user certificate

h2. Query Processing Status Listeners

The Federated Query Engine provides an API by which updates to the current status of query processing may be received and handled, much like the callback APIs found in Java Swing components. Leveraging this API requires implementing an interface, and passing an instance of that implementation in to the Federated Query Engine. The engine supports any number of listeners to this API, however just as in user interface callbacks, the implementation should be reasonably fast so as not to slow down the work of the query engine.

The Federated Query Engine provides three public methods for adding, removing, and listing processing status listeners:

* *addStatusListener*
** Parameter: A processing status listener to be added to the engine's list of listeners
** Returns: *none*
** Adds a status listener instance to the list of listeners which will be notified of various query processing events
* *getStatusListeners*
** Parameter: *none*
** Returns: An array of status listeners which are registered to the engine
* *removeStatusListener*
** Parameter: A processing status listener to be removed from the engine's list of listeners
** Returns: boolean true if the listener was found and removed, false otherwise

[Details of the FQP Processing Status Listener interface|fqp14:FQPProcessingStatusListener]


h1. Federated Query Processor Client
----
The Federated Query Processor Client is the client-side API for communicating with the caGrid Federated Query Processor Service.

h2. Constructing an Instance

The Federated Query Processor Client has four constructors, most of which are simply convenience accessors to a final constructor. The various constructors are as follows:

* public FederatedQueryProcessorClient(String url) throws MalformedURIException, RemoteException
* public FederatedQueryProcessorClient(String url, GlobusCredential proxy) throws MalformedURIException, RemoteException
* public FederatedQueryProcessorClient(EndpointReferenceType epr) throws MalformedURIException, RemoteException
* public FederatedQueryProcessorClient(EndpointReferenceType epr, GlobusCredential proxy) throws MalformedURIException, RemoteException

The *url* parameter passed in the first two constructors is the URL of the Federated Query Processor Service you wish to connect to. The *epr* parameter in the last two constructors is an Endpoint Reference which resolves to the FQP service you wish to connect to. The *proxy* parameter is a Globus Credential Proxy which you may use to authenticate to and communicate securely with the FQP service. These constructors should look familiar to users of other [Introduce|introduce14:Home]\-generated caGrid services, since they are the standard client constructors.

h3. Connecting to a Secure FQP Service

{import:fqp14:Connecting to Secure FQP Services}

h2. API Methods

The Federated Query Processor Client offers seven public methods for executing DCQL and DCQL 2 queries in different ways:

h3. Simple Query Execution

The _execute_ method takes a single DCQL 1 query parameter and returns a single DCQLQueryResultsCollection instance. This method may throw a FederatedQueryProcessingException

{code:java}
public DCQLQueryResultsCollection execute(DCQLQuery dcqlQuery)
throws RemoteException, FederatedQueryProcessingFault
{code}

This method sends a DCQL 1 query to the service, which then uses the Federated Query Engine to processes the DCQL 1 query as described above.

The _executeQuery_ method takes a single DCQL 2 query parameter and returns a single DCQLQueryResultsCollection instance. This method may throw a FederatedQueryProcessingException

{code:java}
public DCQLQueryResultsCollection executeQuery(DCQLQuery dcqlQuery)
throws RemoteException, FederatedQueryProcessingFault
{code}

This method sends a DCQL 2 query to the service which then uses the Federated Query Engine to process the query as described above.

h3. Execute and Aggregate Results

The _executeAndAggregateResults_ method takes a single DCQL 1 query parameter and returns a single CQLQueryResults instance. This method may also throw a FederatedQueryProcessingException.

{code:java}
public CQLQueryResults executeAndAggregateResults(DCQLQuery dcqlQuery)
throws RemoteException, FederatedQueryProcessingFault
{code}

This method sends a DCQL query to the service, which then uses the Federated Query Engine to processes the DCQL as described above.

The _executeQueryAndAggregate_ method takes a single DCQL 2 query parameter and returns a single CQLQueryResults instance. This method may also throw a FederatedQueryProcessingException.

{code:java}
public CQLQueryResults executeQueryAndAggregate(DCQLQuery dcqlQuery)
throws RemoteException, FederatedQueryProcessingFault
{code}

This method sends a DCQL 2 query to the service, which then uses the Federated Query Engine to processes the DCQL 2 as described above.

h3. Asynchronous Query Execution

The Federated Query Processor Client offers APIs to perform a DCQL or DCQL 2 query asynchronously. With this functionality, a client can issue a DCQL query, immediately receive a Federated Query Results Client, and use that new client to retrieve results at a later time, potentially using WS-Notification functionality to determine when the query has completed processing on the service and results are available.

The _executeAsynchronously_ method takes a single DCQL query parameter and returns a single Federated Query Results Client instance. This method may also throw a Malformed URI Exception, and a Remote Exception.

{code:java}
public FederatedQueryResultsClient executeAsynchronously(DCQLQuery query)
throws RemoteException, org.apache.axis.types.URI.MalformedURIException
{code}

The Federated Query Processor Client offers an API to perform specialized DCQL queries in an asynchronous fashion.

{code:java}
public FederatedQueryResultsClient query(DCQLQuery query,
DelegatedCredentialReference delegatedCredentialReference,
QueryExecutionParameters queryExecutionParameters)
throws RemoteException, org.apache.axis.types.URI.MalformedURIException,
FederatedQueryProcessingFault, InternalErrorFault
{code}

The Federated Query Processor Client offers an API to perform DCQL 2 in an asynchronous fashion, and optionally with specialized properties.

{code:java}
public FederatedQueryResultsRetrievalClient queryAsynchronously(DCQLQuery query,
DelegatedCredentialReference delegatedCredentialReference,
QueryExecutionParameters queryExecutionParameters)
throws RemoteException, org.apache.axis.types.URI.MalformedURIException,
gov.nih.nci.cagrid.fqp.results.stubs.types.InternalErrorFault,
gov.nih.nci.cagrid.fqp.stubs.types.FederatedQueryProcessingFault
{code}

{include:fqp14:FederatedQueryResultsClient}

{include:fqp14:FederatedQueryResultsRetrievalClient}