caGrid Lift/Scala Tutorial Phase 3

	Contents Organization of this Tutorial Determining if a Service Supports CQL2 Querying Data Services Modifying the Service Page to Support Query Adding the Query Page to the SiteMap Appearance and Operation of the Query Page HTML for the Query Page The Query Snippet Adding Query Support to the CaGrid Object

You can support:download the completed phase 3 web client.

Organization of this Tutorial

In the previous phase of this tutorial, we constructed a web-based caGrid client powered by the Scala language and using the Lift framework. The functionality of the client was limited to logging in to the grid, requesting a caGrid user id and discovering available services on the grid.

In this phase of the tutorial, we add these functions to the client:

• Determine if a data service supports CQL2 or the original CQL.
• Submit a query to a data service.
• Download the results of a query.

Determining if a Service Supports CQL2

In the previous phase of this tutorial, we added a "Service" page to the grid client that shows some information about a selected service. It looked like

We now make the page look like this:

This new version of the service page show us whether or not a service supports CQL2, the new version of the CQL language that was released with CaGrid 1.4. If a service is a data service, then the value of its Supports CQL2 property will be displayed with as "Yes" or "No". If a service is not a data service, then the value of this property will be "N/A" (not applicable).

To support this, we added a few lines (highlighted in yellow) to the HTML template lift/src/main/webapp/service.html:

To accommodate the new feature, we also need to make some additions to the snippet class Service. Firstly, there is an additional import needed:

import gov.nih.nci.cagrid.data.utilities.DataServiceFeatureDiscoveryUtil

This class contains the logic needed to discover if a service supports CQL2.

We add the logic (highlighted in yellow) to use this class to the end of the Service.render method:

Querying Data Services

We want the client to be able to query data services. We will add a new query page to support this. We also need to modify the service page to allow users to navigate to the new query page. There are a number of steps in making this happen. We will begin by adding some additional dependencies on external .jar files to the build.

We add the dependencies highlighted in yellow to lift/project/build/LiftProject.scala:

Modifying the Service Page to Support Query

To allow users to navigate from the service page to the query page, we will add a button to the service page that the user can click on for this purpose. Of course, we only want the button to be visible if the service we are looking at is a data service.

Our first step is to add a placeholder button to the service page. We will put the new button next to the "OK" button. Here is what it will look like:

Here is what this change looks like with the new HTML for the service page highlighted in yellow:

We want to modify the corresponding snippet method, Service.render, to replace the placeholder with a live button if the service is a data service. Otherwise, the placeholder should be replaced with nothing. Here is what the necessary additions to the code look like:

Adding the Query Page to the SiteMap

To add the query page to the grid client, we will first add it to the site map that is defined in lift/src/main/scala/bootstrap/liftweb/Boot.scala. The new code is highlighted in yellow:

Before we leave this code, we notice that there are six places that create identical objects to control checking whether a user is logged in. Let's make this more concise by creating one object and sharing it:

    val requireUserToBeLoggedIn = If(() => CaGrid.credentials.hasValidUserCertificate, "You must be logged in.")

    // Build SiteMap
    def sitemap = SiteMap(
      Menu("Login") / "index" >> If(() => !CaGrid.credentials.hasValidUserCertificate, "Logging in is for people who have not yet logged in."),
      Menu("Home") / "home" >> requireUserToBeLoggedIn,
      Menu("Logout") / "logout" >> requireUserToBeLoggedIn,
      Menu("Request a User ID") / "request_uid" >> If(() => !CaGrid.credentials.hasValidUserCertificate, "You are already logged in."),
      Menu("Discover Services") / "discover" >> requireUserToBeLoggedIn,
      Menu("Select a Service") / "service_selection" >> Hidden >> requireUserToBeLoggedIn,
      Menu("View Service Metadata") / "service" >> Hidden >> requireUserToBeLoggedIn,
      Menu("Query Data Service") / "query" >> Hidden >> requireUserToBeLoggedIn
    )

Appearance and Operation of the Query Page

Before getting into the details of implementing the Query page we take a look at its appearance and operation. Initially the Query page looks like this:

To use this page, a user first clicks on the Choose File button. The user is then prompted to select a file that contains a CQL query to be submitted to the data service.

After choosing a CQL file, the user clicks on the Run Query button. This causes the query in the file to be submitted to the data service. The results of the service are shown in a read-only text area at the bottom of the page. With results showing, the page looks like this:

HTML for the Query Page

Here is the HTML for the Query page from the file /src/main/webapp/query.html:

<!DOCTYPE html>
<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="content-type" />
    <title>Query</title>
  </head>
  <body class="lift:content_id=main">
    <div id="main" class="lift:surround?with=default;at=content">
      <form class="lift:Query?form=post;multipart=true">
        <table>
          <tbody>
            <tr >
              <td style="background-color: white;" align="center">
		<b>Upload a file containing a CQL query to be submitted.</b>
	      </td>
            </tr>
            <tr >
              <td style="background-color: white;" align="center">
		<input type="file" name="upload" id="upload"/>
	      </td>
            </tr>
            <tr>
              <td style="background-color: white;" align="center">
		<input name="query" type="submit" value="Run Query" />
	      </td>
            </tr>
          </tbody>
	 </table>
	 <p/>
	 <table>
	   <tbody>
	     <tr id="result">
	       <td>No results to display yet.</td>
	     </tr>
	   </tbody>
	 </table>
       </form>        
    </div>
  </body>
</html>

At first glance, it appears that little in this HTML is different from what we have seen before. We have an input element with the type file. This looks like just another placeholder that happens to be a different type of input element that has appeared previously in this tutorial. However, there is one additional detail present that is required to correctly process the file upload.

File uploads are treated as an attachment to the request that is sent when the form is submitted. For Lift to be able to process such an attachment, the form needs to be marked as a multi-part form. For this reason, the class attribute of the form element has some additional information in its class attribute. In addition to specifying form=post it also specifies multipart=true.

The Query Snippet

The Query snippet needs to handle the two new things that we have not yet seen in a snippet. It must handle a file upload and it must handle a textarea. Here is the Query snippet class:

package edu.emory.cci.caGrid.liftClient.snippet

import scala.xml._
import net.liftweb.util._
import net.liftweb.common._
import net.liftweb.http._
import code.lib._
import Helpers._

import java.io._

import edu.emory.cci.caGrid.liftClient.model._

import gov.nih.nci.cagrid.common.Utils
import gov.nih.nci.cagrid.cqlresultset.CQLQueryResults
import gov.nih.nci.cagrid.data.CqlSchemaConstants

import org.globus.wsrf.encoding.ObjectSerializer

class Query extends Logger {

The source for the Query class begins with the usual imports followed by some CaGrid-related imports that are needed for handling the CQL used for queries and query results.

  /**
   * Allow the query file to be remembered across requests.
   */
  object fileHolderBoxVar extends RequestVar[Box[FileParamHolder]](Empty)

  val noResultsYet = Text("No results to Display Yet")

  object results extends RequestVar[Node](noResultsYet)

  /**
   ** Return the results as an HTML table.
   */
  private def serializeResultToHTMLTableRows(cqlResult:CQLQueryResults):NodeSeq = {
    val stringWriter = new StringWriter()
    ObjectSerializer.serialize(stringWriter, cqlResult,
                               CqlSchemaConstants.CQL_RESULT_COLLECTION_QNAME)
    val xmlResult = XML.loadString(stringWriter.toString())
    val topElementName = xmlResult.label
    val topAttributesString = xmlResult.attributes.toString()
    val firstTableRow = <tr><td>&lt;{topElementName} {topAttributesString}&gt;</td></tr>
    val middleTableRows = xmlResult.child.map((node:Node)=> <tr><td>{node.toString}</td></tr>)
    val lastTableRow = <tr><td>&lt;/{new Text(topElementName)}&gt;</td></tr>

    firstTableRow +: middleTableRows :+ lastTableRow
  }

  /**
   * snippet for managing the query form.
   */
  def render = {
    debug("Begin rendering query page.")

The processFile method is called to process the uploaded file. The file's contents and metadata are encapsulated in the FileParamHolder object that is passed to the processFile method. The processFile method takes the FileParamHolder object and set it to be the value of the RequestVar named fileHolderBoxVar

    def processFile(fileParamHolder:FileParamHolder) {
      debug("processing upload file")
      fileHolderBoxVar.set(Full(fileParamHolder))
      results.set(noResultsYet)
      S.notice("File Uploaded")
    }

    def processForm():Unit = {
      debug("Processing Query form")
      fileHolderBoxVar.is match {
        case Full(fileHolder) =>
	  S.notice("Query using file: " + fileHolder.fileName)
          Service.selectedEndpoint.is match {
            case Full(endPoint) =>
	      val resultBox = CaGrid.submitQuery(fileHolder.fileStream, endPoint)
	      resultBox match {
	        case Full(result) =>
                  val resultString = results.set(serializeResultToHTMLTableRows(result))
                  S.notice("Successfully received query results")

                case Failure(msg, exception, chain) =>
		  warn(msg, exception)
		  S.error(msg)
		  results.set(noResultsYet)

		case Empty =>
                  S.warning("Query produced no reasult")
		  results.set(noResultsYet)
              }

            case _ => S.warning("No service selected")
          }

        case _ =>
          S.warning("No query file uploaded")
      }
    }

    "#upload" #> SHtml.fileUpload(processFile) &
    "@query" #> SHtml.submit("Run Query", processForm) &
    "#result" #> results
  }
}

Adding Query Support to the CaGrid Object

The grid client is organized so that all interactions with CaGrid are done through the CaGrid object. We will add a method named submitQuery to the CaGrid object to submit a query to a data service. We will pass it an InputStream that will provide it with the characters of the CQL query. We will also pass it an EndpointReferenceType object that will be used to communicate with the data service.

First we need to add an import to the CaGrid object:

import gov.nih.nci.cagrid.common.Utils;
import gov.nih.nci.cagrid.cqlresultset.CQLQuery
import gov.nih.nci.cagrid.cqlresultset.CQLQueryResults
import gov.nih.nci.cagrid.data.client.DataServiceClient;

Below is the submitQuery method. Notice that it returns its result in a Box, which allows failure cases to be handled without having to worry about what type of exceptions might be thrown.

  def submitQuery(cqlInputStream:InputStream,
                  endPoint:EndpointReferenceType):Box[CQLQueryResults] = {
    val cqlQuery = Utils.deserializeObject(new InputStreamReader(cqlInputStream),
                                           classOf[CQLQuery])
    submitQuery(cqlQuery, endPoint)
  }

We are breaking the submitQuery method into two parts. The first part just constructs a CQLQuery object from the input stream and then calls the second part. The reason for breaking it up this way is that later in this tutorial, we will be adding another from of submitQuery that constructs a CQLQuery object in a different way.

Notice that the following code decides whether it is being asked to query a secure service by whether or not the protocol part of its URL is http or not. For the secure case it must pass the grid credentials into the constructor for the service client.

  def submitQuery(cqlQuery:CQLQuery,
                  endPoint:EndpointReferenceType):Box[CQLQueryResults] = {
    val client =
      if (endPoint.getAddress().getScheme() == "http") {
        new DataServiceClient(endPoint)
      } else {
        credentials.credential match {
	  case Full(globusCredential) =>
            new DataServiceClient(endPoint, globusCredential)

	  case _ =>
	    val msg = "No Valid Grid Credential Available.  Try logging in again"
	    return Failure(msg, Empty, Empty)
        }
      }
    try {
      Full(client.query(cqlQuery))
    } catch {
      case e:Exception =>
        val endPointAddressString = endPoint.getAddress().toString()
        val msg = "Error occurred while quering data service" + endPointAddressString
        Failure(msg, Full(e), Empty)
    }
  }

Go to Phase 4 of This Tutorial