WS-Enumeration Developers Guide 1.1

	Table of Contents 1 Client API 1.1 Utilities 1.2 Examples 1.2.1 Basic caGrid WS-Enumeration 1.2.2 Using BDT 2 Server API 2.1 Enumeration Implementations 2.2 Utilities 2.3 Examples 2.3.1 All data available at once

Client API

The Globus-provided org.globus.ws.enumeration.ClientEnumIterator API provides java.util.Iterator abstraction for retrieving enumeration data and supports automatic data deserialization. The caGrid WS-Enumeration implementation provides a simplified factory interface to create a new instance of a Client Enum Iterator from an Enumeration Response Container.

For a more detailed discussion of the WS-Enumeration client tools, please see the developer's wiki regarding the WS Core WS-Enumeration.

Utilities

The caGrid-provided class gov.nih.nci.cagrid.wsenum.utils.EnumerationResponseHelper contains static methods which can take an Enumeration Response Container and return a client enum iterator instance. The method createClientIterator takes only the response container, while another implementation of this method takes both the container and a java.io.InputStream to the client-config.wsdd file. The information contained in this file will be used to deserialize results from the enumeration.

Examples

Basic caGrid WS-Enumeration

Given a service (here an enumeration-enabled Data Service) which supports enumeration, the following pattern may be used by a client to enumerate over data results.

TestEnumerationDataServiceClient client = new TestEnumerationDataServiceClient(args[1]);

EnumerationResponseContainer response = client.enumerationQuery(null);

ClientEnumIterator iter = EnumerationResponseHelper.createClientIterator(
     response, TestEnumerationDataServiceClient.class.getResourceAsStream("client-config.wsdd"));

// optional use of iteration constraints to fetch 10 elements w/o regard
// to number of characters or maximum time to wait
IterationConstraints cons = new IterationConstraints(10, -1, null);
iter.setIterationConstraints(cons);

while (iter.hasNext()) {
     SOAPElement elem = (SOAPElement) iter.next();
     if (elem != null) {
          // handle the data element
     }
     // optionally change constraints to fetch 5 elements, max of
     // 50000 characters, and a max wait time of 1 hr, 30 minutes
     cons = new IterationConstraints(5, 50000,
          new Duration(false, 0, 0, 0, 1, 30, 0));
}

The use of IterationConstraints to express how data should be fetched from the service is optional, and the default behavior is to retrieve one element at a time, with no regard for max characters or time constraints. Some server side implementations to not respect all iteration constraints. These constraints may change at any time during enumeration.

While iterating, it is possible that the call to 'hasNext' may return true, yet the call to 'next' return null. This is often the case when results are being dynamically populated on the server side at a rate slower than the client is capable of consuming them.

Using BDT

The case of using the caGrid Bulk Data Transportinfrastructure slightly complicates the client code to access data via enumeration by adding a level of indirection as the BDT resource is created first, and then used to create an enumeration resource.

TestBDTDataServiceClient client = new TestBDTDataServiceClient(args[1]);

// query with BDT
BulkDataHandlerReference bdtRef = client.bdtQuery(null);

// use the reference to create a generic BDT client
BulkDataHandlerClient bdtClient = new BulkDataHandlerClient(bdtRef.getEndpointReference());

// start the enumeration on the server
EnumerationResponseContainer response = bdtClient.createEnumeration();

// rest of code the same as basic WS-Enumeration case

Server API

The caGrid WS-Enumeration support provides a great deal of tooling to simplify using WS-Enumeration in a grid service. This support includes mechanisms for creating services incorporating a service context for enumeration and various implementations of the server side enumeration interface.

Enumeration Implementations

WS-Enumeration support in Globus requires the enumeration support be provided by an implementation of the interface *org.globus.ws.enumeration.EnumIterator.*In addition to the two basic implementations provided by Globus, caGrid supplies three enumeration implementations, as well as a factory to create an instance of any one of these implementations given a list of data objects to be enumerated.

The Java enum gov.nih.nci.cagrid.wsenum.utils.IterImplType defines the five implementations and giving a brief description of each:

• GLOBUS_SIMPLE
  • The Globus-provided simple enum iterator
• GLOBUS_INDEXED_FILE
  • The Globus-provided indexed file enum iterator
• CAGRID_SIMPLE
  • A simple iterator which persists objects to disk
• CAGRID_THREADED_COMPLETE
  • An implementation using threads to respect maxTime constraints
  • This iterator uses threads to respect maxTime constraints as well as respecting maxCharacters. Elements overflowing either of these constraints, however, are lost, and waits for threads are not optimized.
• CAGRID_CONCURRENT_COMPLETE
  • A complete implementation using Java 5's concurrent package
  • This iterator uses the Java 5 java.util.concurrent package to fully support the WS-Enumeration specification for an EnumIterator implementation. All iteration constraints are respected, and elements which cause maxCharacters to be exceded are queued for later retrieval.

For most purposes, the CAGRID_CONCURRENT_COMPLETE implementation should be used as provides full support for the server side WS-Enumeration spec. This is also the default implementation selected by the caGrid data services infrastructure. The other implementations are less complete, and may be useful in emulating the behavior of other enumeration-enabled systems.

Utilities

The class gov.nih.nci.cagrid.wsenum.utils.EnumIteratorFactory is the primary means of obtaining an implementation of the Globus EnumIteratorinterface in caGrid enumeration services. This class provides static methods which allow varying degrees of control over the iterator returned. The most detailed of these methods is the following:

public static EnumIterator createIterator(
     gov.nih.nci.cagrid.wsenum.utils.IterImplType iterType,
     java.util.Iterator objectIter,
     javax.xml.namespace.QName objectQName,
     java.io.InputStream wsddInput,
     java.lang.String tempFilename)

The parameters provide required information to the iterator:

• objectIter
  • An iterator to a collection of caCORE SDK objects to be enumerated
  • In some methods, this is replaced by a java.util.List of the objects to be enumerated.
• objectQName
  • The QName of the objects
  • This is required to indicate the XML type of the data which will be serialized
• wsddInput
  • An input stream of the WSDD configuration file
  • This WSDD will be used to obtain serialization information for the enumerated objects
• tempFilename
  • The name of the file to serialize objects into.
  • NOTE: For security reasons, access to this location must be controlled in a production data environment.

The class gov.nih.nci.cagrid.wsenum.utils.EnumerateResponseFactoryhides the complexity of obtaining the Enumeration resource home, creation of a new resource, setting its visibility, and producing an Endpoint Reference for it. It provides a single static method called createEnumerationResponse takes an EnumIterator instance, and accomplishes all of those tasks and returns an EnumerationResultContainer.

Examples

All data available at once

A common pattern for WS-Enumeration is to have all data available at the start of enumeration, but perhaps it is too large to return all at once. In this case, Enumeration may be used to allow the client to consume the data as it is able. This process may take the following form:

1. Acquire the list of data objects to be enumerated
2. Determine the XML QName of the data type
3. Select a server side enumeration implementation
4. Store the enumeration in an enumeration resource
5. Return the Enumeration Response Container to the client

While the first three steps of this process are frequently specific to each service implementation, caGrid provides common tooling for storing an enumeration resource and creating the enumeration response container.

Several classes must be imported to support this process:

import gov.nih.nci.cagrid.enumeration.stubs.response.EnumerationResponseContainer;
import gov.nih.nci.cagrid.wsenum.common.WsEnumConstants;

import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.List;

import javax.xml.namespace.QName;

import org.apache.axis.message.MessageElement;
import org.apache.axis.message.addressing.EndpointReferenceType;
import org.globus.ws.enumeration.*;
import org.globus.wsrf.*
import org.xmlsoap.schemas.ws._2004._09.enumeration.EnumerateResponse;
import org.xmlsoap.schemas.ws._2004._09.enumeration.EnumerationContextType;
import org.xmlsoap.schemas.ws._2004._09.enumeration.ExpirationType;

The code to produce an enumeration response begins by locating the enum resource home. Then it is used to create an enum resource with visibility set so the caGrid enumeration provider may access it. The server side enum provider is used to generate the enumeration context, and Axis tools provide an endpoint reference to the new resource and enumeration context. Finally, the EPR and enumeration context are combined in an enumeration response container to be returned.

EnumResourceHome resourceHome = EnumResourceHome.getEnumResourceHome();
VisibilityProperties visibility = new VisibilityProperties("cagrid/" + 
     WsEnumConstants.CAGRID_ENUMERATION_SERVICE_NAME, null);

EnumResource resource = resourceHome.createEnumeration(
     enumIter, visibility, false);
ResourceKey key = resourceHome.getKey(resource);

EnumerationContextType enumContext = 
     EnumProvider.createEnumerationContextType(key);

URL baseURL = ServiceHost.getBaseURL();
String serviceURI = baseURL.toString() 
     + "cagrid/" + WsEnumConstants.CAGRID_ENUMERATION_SERVICE_NAME;

EndpointReferenceType epr = 
     AddressingUtils.createEndpointReference(serviceURI, key);

EnumerationResponseContainer container = new EnumerationResponseContainer();
container.setContext(enumContext);
container.setEPR(epr);
return container;

If your implementation does not need such a high level of control over the resulting Enumeration Response Container, caGrid provides a utility which performs the same operations as the above code:

EnumIterator iter = // create an enum iterator instance
EnumerationResponseContainer conteiner = 
     gov.nih.nci.cagrid.wsenum.utils.EnumerateResponseHelper.createEnumerationResponse(iter);