[seek-dev] RE: resultset question
Shawn Bowers
bowers at sdsc.edu
Wed Apr 28 10:38:56 PDT 2004
Peter (and all),
One of the things that we have been doing, or working towards, is
exploiting EML descriptions for datasets to automatically populate part
of the ontology annotation of the dataset (the "semantic registration").
Rich Williams has written Java code that takes part of the
description, e.g., for location coverage, and outputs the corresponding
OWL instances.
I have looked through the EML documentation on line, and like Bertram,
would like to see something that shows the "power" of EML.
In sifting through the XML schema for EML, I undestand there are
multiple types of resources EML can handle. However, in practice, I
don't see these being used. In fact, from the LTER sites that I have
been pointed to, it looks like the majority of the EML files don't
include much metadata at all -- I have only seen EML files for datasets,
and most of these don't even describe the schema of the resource. They
typically have some other format that they use to describe the fields,
e.g. It would be great to see some examples, whether real or
"cooked-up" that show EML for describing web-services (i.e.,
eml-software), spatial raster, spatial vector, literature, and so on.
In general, I agree with your comment below that we need to align
Ecogrid and SMS (assuming there is a desire to add ontology-based
resource discovery, integration, and transformation to Kepler).
Maybe I am missing something, but your email below seems to suggest that
we should think of EML as the "interface" between Ecogrid and SMS. My
concern, however, with focusing (and relying) on EML as the *only*
metadata language, and building Ecogrid around EML, is that it severly
limits the ability of data providers to add existing resources to the
Ecogrid, and clients for accessing resources. For example, in the
current BEAM mammals use case, the data they wish to use doesn't have
EML metadata (data from IPCC) -- and it would be a major effort to
create the EML metadata for these files. Note also that SMS doesn't
require or rely on EML, although, if present, we want to exploit it as
well as any other metadata associated with a resource.
My other concern is whether Ecogrid provides the most basic operations
required to access resources -- which are necessary for SMS, and this
concern is what I was trying to describe in my previous emails. Simply
getting resources by id (in a uniform format), determining what type of
resource it is, and getting all of the associated metadata files for a
resource all seem to be fundamental operations that the Ecogrid should
support. And, providing these operations would enable "naive" clients,
e.g., that don't understand EML, to still leverage the Ecogrid and be
able to generically access stored resources.
What are your thoughts on this?
Shawn
Bertram Ludaescher wrote:
> Peter:
>
> Good points!
>
> Is there a tutorial or reference paper that describes the fundamental
> principles in EML?
>
> Bertram
>
>
>>>>>>"PM" == Peter McCartney <peter.mccartney at asu.edu> writes:
>
> PM>
> PM> I think it is clear that we need a joint ecogrid SMS/KR meeting. On the
> PM> SMS side im hearing still some unfamiliarity with what information has
> PM> been designed into EML (ie - the low level class distinctions between
> PM> resource types is built into the extensible resource module) on the
> PM> ecogrid side, im seeing a need to get some input on where to couple our
> PM> existing data with the new semantic ontologies. Hopefully you will all
> PM> get to talk about some of this in scotland, which I unfortunately will
> PM> miss.
> PM>
> PM> -----Original Message-----
> PM> From: Bing Zhu [mailto:bzhu at sdsc.edu]
> PM> Sent: Tuesday, April 27, 2004 11:52 PM
> PM> To: Shawn Bowers; Rod Spears
> PM> Cc: Chad Berkley; Peter McCartney; Saritha Bhandarkar; seek-dev; Jing
> PM> Tao; Matt Jones; Bertram Ludaescher
> PM> Subject: RE: [seek-dev] RE: resultset question
> PM>
> PM>
> PM> Ecogrid builds low level tools for SEEK users to export, import and
> PM> query datasets and metadata. For storing datasets or building registry
> PM> for "web service", "shape file", "source code", "PDF document",
> PM> "ontology", etc, we need some data modeling work. e.g. in SRB, you can
> PM> create and organize datasets in different collections. With this
> PM> approach, you can have a web service registry collection in which we can
> PM> create and store datasets serving as our web service registry.
> PM>
> PM> It seems to me that a perfect design for our Ecogrid resultset needs to
> PM> use some knowledge from ontologies. I am not sure if it is appropriate
> PM> to mix the ontology layer within Ecogrid software layer.
> PM>
> PM> Bing
> PM>
> PM>
> PM>
> PM> -----Original Message-----
> PM> From: seek-dev-admin at ecoinformatics.org
> PM> [mailto:seek-dev-admin at ecoinformatics.org]On Behalf Of Shawn Bowers
> PM> Sent: Monday, April 26, 2004 9:40 AM
> PM> To: Rod Spears
> PM> Cc: Chad Berkley; Peter McCartney; Saritha Bhandarkar; seek-dev; Jing
> PM> Tao; Matt Jones; Bertram Ludaescher
> PM> Subject: Re: [seek-dev] RE: resultset question
> PM>
> PM>
> PM>
> PM>
> PM> Rod Spears wrote:
>
>>>See comments below.... and please comment on my comments.
>
> PM>
> PM> Comments below on your comments... (Thanks for responding to my orginal
> PM> mail so quickly)
> PM>
> PM> Also, you should comment on my comments on your comments ;-)
> PM>
>
>>>Shawn Bowers wrote:
>>>
>>>
>>>>On Fri, 23 Apr 2004, Rod Spears wrote:
>>>>
>>>>[snip ...]
>>>>
>>>>
>>>>
>>>>
>>>>>What can be done to help generic consumers and SMS?
>>>>>
>>>>>
>>>>
>>>>I have some opinions/observations about what Ecogrid can provide for
>>>>SMS (assuming you mean Semantic Mediation System). No one actually
>>>>asked for my opinion, but the door is opened by the question, and I
>>>>thought I'd barge in :-)
>>>>
>>>>Here is what I see SMS needing. (Note that this might be a lot
>>>>different than what ecogrid actually intends to provide -- these items
>
> PM>
>
>>>>are more aligned with the architecture of traditional integration
>>>>systems and systems being developed like for GEON.)
>>>>
>>>>1). Every resource registered in the Ecogrid should have a persistent,
>
> PM>
>
>>>>Ecogrid-relative unique identifier.
>>>>
>>>
>>>Each does today. It has a unique name.
>
> PM>
> PM>
> PM> I thought that it did. This list of operations and data structures is
> PM> just to say what SMS needs from Ecogrid -- I assumed much of this was
> PM> implemented by Ecogrid already.
> PM>
> PM>
>
>>>>2). Every resource registered in the Ecogrid should fill-in two
>>>>Ecogrid metadata tags (dublin-core style). The first is the type of
>>>>resource registered, e.g., the type could be "dataset", "web service",
>
> PM>
>
>>>>"shape file", "source code", "PDF document", "ontology", etc. (These
>>>>should be controlled values, i.e., come from a predefined list.)
>>>>
>>>
>>>Dave and I were just talking about this. We hoped we could get by
>>>without an extra identifier. Meaning the "type" could be derived from
>>>the service's location (or the interfaces it implements). But maybe we
>
> PM>
>
>>>will need a simple field for easier indentification.
>
> PM>
> PM>
> PM> I think the assumption that the location determines the resource type is
> PM> not general enough (and also not extensible). For example, if we have
> PM> an SRB repository used within Ecogrid for storing datasets as well as
> PM> PDF documents and ontologies, then a namespace would have to capture all
> PM> three of these types. I believe that with many of these underlying
> PM> systems, like SRB and Metacat, there is no requirement that all
> PM> resources stored must be of the same type.
> PM>
> PM> I stated above there should be a metadata tag for storing the type
> PM> information, but it could just as easily be an operation (or query). For
> PM> example, getResourceType : ResourceID -> ResourceType is a partial
> PM> function, where ResourceID is the set of all possible Ecogrid resources
> PM> identifiers and ResourceType is the set of all resource types known by
> PM> Ecogrid ("dataset", "web service", and so on). So, for a given
> PM> resource-id r, getResourceType(r) returns the associated resource type
> PM> of r. If Ecogrid calculates this op based on where r is stored, and
> PM> that is really a valid assumption, that seems fine. Note that the
> PM> operation could also be expressed as a query, as opposed to a function
> PM> or a metadata tag.
> PM>
>
>>>>The other tag
>>>>states the available (and Ecogrid accessible) standards-based metadata
>
> PM>
>
>>>>for the resource, e.g., for a dataset this might include "FGDC",
>>>>"EML", "XML Schema" (for datasets stored in XML), "SQL DDL"; and for a
>
> PM>
>
>>>>web service, "WSDL"; and so on. (Again, these should be controlled
>>>>values.) Other tags that might be useful (but not required by SMS) are
>
> PM>
>
>>>>quality of resource (who registered it, whether it has been deemed
>>>>"accepted", and so on) and whether it is curated (stored by some
>>>>Ecogrid db) or stored externally (e.g., in the PNW database).
>>>>
>>>
>>>Would a namespace be enough to be able to specify "how" the metadata
>>>was stored?
>
> PM>
> PM> In this case, I don't think a namespace is enough. Any given resource
> PM> may have multiple metadata specifications. For example, if a given
> PM> resource-id r happens to be a dataset, then there very easily could be
> PM> both an FGDC and an EML metadata file for r. So, what SMS needs is a
> PM> (partial) function getMetadataType : ResourceID -> MetadataType^2, which
> PM> takes a resource id and returns a set (^2 means powerset) of metadata
> PM> types (e.g., "SQL DDL", "EML", and so on).
> PM>
> PM> One question I have about the Ecogrid, and probably a misconception I
> PM> have, is that it seems like what is searched *for* is metadata (like EML
> PM> files), and not the actual resource. This was what prompted my earlier
> PM> post on how to get all resources from the Ecogrid... do I have to first
> PM> query for all the metadata associated with the resource, then look in
> PM> these files to see where each resource is actually being stored? Like I
> PM> said, this might be a misconception I have -- it seems like this
> PM> metadata-centric view represents the only examples I've seen for
> PM> Ecogrid. I would like for SMS to have resource-centric access for
> PM> datasets; the resource is what is of interest (I give an example in my
> PM> next comment below). The same should be true for Kepler -- datasets can
> PM> be processed in a workflow, not the EML files of the datasets (there is
> PM> a caveat to this; both Chad's EML ingestor and in some ways, Iklay's web
> PM> service actor, take metadata files, but their purpose is to get from the
> PM> metadata to the actual resource, I believe). Of course, for web
> PM> services (as an example), SMS doesn't need the actual resource, and only
> PM> needs the WSDL description (which happens to be all that is needed to
> PM> execute the web service). However, conceptually, it is still the
> PM> web-service that is the resource -- the web-service implementation is
> PM> what is of interest, and the WSDL could be viewed as just a by-product
> PM> of the implementation. In fact, there could be many WSDL descriptions of
> PM> the same implementation. There may be some disagreement about this
> PM> notion of Ecogrid being resource-centric, but I would argue it is the
> PM> more general semantics.
> PM>
> PM> Does that make sense?
> PM>
> PM>
>
>>>>3). Ecogrid should support an operation to retrieve the metadata
>>>>definition for a resource. For example, if a dataset is stored through
>
> PM>
>
>>>>the Ecogrid, and the resource has an EML description (which we know
>>>>from 2), then the operation would return the corresponding EML file
>>>>(of course, although not likely, there is nothing that would prevent a
>
> PM>
>
>>>>resource from having multiple EML files).
>>>>
>>>
>>>Seems reasonable.
>>>
>>>
>>>>4). Ecogrid should support an operation to retrieve the actual
>>>>resource (the thing managed by the ecogrid; either a dataset, a web
>>>>service, a "code", or whatever). Also, datasets should be returned
>>>>using a standard representation. For example, the canonical XML
>>>>representation for relational data or CSV. I believe EML-tools
>>>>already provide some support for this for relational data. Thus, at
>>>>least for datasets, the Ecogrid should serve as a standard wrapper
>>>>service as used in distributed dbs and in information-integration
>>>>architectures. This service I see as useful for both SMS and for
>>>>Kepler in general.
>>>>
>>>
>>>It's either doing this, or I don't quite understand the question.
>
> PM>
> PM> Here is an example. I am a scientist, and I have a dataset (a single
> PM> relation) stored in an Access database. I also have an FGDC file that I
> PM> created to describe my dataset. They are both living on my laptop. I
> PM> want to store my dataset on the Ecogrid. I create an Ecogrid resource-id
> PM> for the dataset, ecogrid:042604, and I register the resource-id for the
> PM> dataset. That is, I upload the Access database to some Ecogrid
> PM> repository as well as the FGDC file, and I tell Ecogrid that the FGDC
> PM> file should be used as the metadata file for the dataset.
> PM>
> PM> Later, SMS needs to integrate the dataset with some other dataset. SMS
> PM> knows the resource-id for both datasets. To do the integration, SMS
> PM> needs access to both datasets. To get access to the datasets, SMS calls
> PM> the Ecogrid function getResource(ecogrid:042604, "CSV"), which returns
> PM> the dataset as a comma-separated-value text-file representation.
> PM> Alternatively (and preferred), SMS could call
> PM> getResource(ecogrid:042604, "RelationalXML"), which returns the same
> PM> exact dataset using the standard relational to XML mapping.
> PM>
> PM> Does Ecogrid already provide something like getResource? (If so that
> PM> would be awesome!)
> PM>
> PM>
> PM> Thanks,
> PM> Shawn
> PM>
>
>>>>5). Optionally (at least for SMS, these aren't required), Ecogrid can
>>>>offer a query-routing/execution service and/or web service invocation.
>
> PM>
>
>>>>The purpose of offering query or invocation services would be for
>>>>optimization (in some cases) and to enable such operations for clients
>
> PM>
>
>>>>that cannot perform these locally.
>>>>
>>>
>>>I think this functionality is one of the benefits of using Globus.
>>>
>>>
>>>>I believe that items 1-4 are the only things really needed by SMS from
>
> PM>
>
>>>>the Ecogrid. In particular, for SMS, it doesn't really matter how or
>>>>where the resource is stored (metacat, src, digir, etc.), and it
>>>>doesn't need services to query the catalog entries of those systems.
>>>>If people bypass the SMS system, then I guess these types of things
>>>>are needed.
>>>>
>>>>Items 1-3 seem relatively straightforward. Item 4 seems harder,
>>>>although EML-tools exist for much of this I guess -- I am not really
>>>>sure.
>>>>
>>>>
>>>>Shawn
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>The issue at the moment is that the contents of the <record> element
>>>>>is basically a blob and anything goes. For example:
>>>>>1) Metacat return a bunch of param elements contain the data
>>>>>2) DiGIR contents a bunuch of namespace qualified elements containing
>
> PM>
>
>>>>>the data.
>>>>>3) The SRB doesn't even have any data in the record, the identifier
>>>>>attr is meaningful.
>>>>>
>>>>>We need to provide a mechanism for the contents to be interpreted, to
>
> PM>
>
>>>>>do this we will add four things to the existing resultset schema:
>>>>>1) One or more <namespace> elements the metadata - this will be the
>>>>>namespace for the new <returnfield> element
>>>>>2) Add a new element <returnfield>
>>>>>3) A "name" attribute for the returnfield element (basically the same
>
> PM>
>
>>>>>as Peter 'xpath' att) which is a unique name within the record and
>>>>>may be meaning for whereever the data came from.
>>>>>4) A "type" attribute for the returnfield element that describe the
>>>>>type of data contained in the returnfield
>>>>>
>>>>>The most important and powerful part of the new additions is the
>>>>>"type" attr. This enables the value to be interpreted. Most of the
>>>>>time it can be described by a schema defintion type, for example
>>>>>"xsi:string" etc. Or it could be an url that points to a schema
>>>>>definition document. This means the value of the returnfield element
>>>>>could be anything from a string or integer to an entire XML document.
>>>>>
>>>>>(Note that the namespace attr has been removed from the record
>>>>>element)
>>>>>
>>>>>The new namespace attrs in the metadata provide a way for the value
>>>>>of the name attr and the type attr to be interpreted.
>>>>>
>>>>>Here is an example of the a metacat resultset that is returned today:
>
> PM>
>
>>>>><rs:resultset system="http://knb.ecoinformatics.org"
>
> PM> resultsetId="eml.001"
>
>>>>>xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>>>>>
>>>>>xsi:schemaLocation="ecogrid://ecoinformatics.org/ecogrid-resultset-1.
>>>>>0.0b
>
> PM> eta1
>
>>>>>../../src/xsd/resultset.xsd">
>>>>><resultsetMetadata>
>>>>><sendTime>2004-03-10T13:47:26-0600</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>14</endRecord>
>>>>><recordCount>14</recordCount>
>>>>></resultsetMetadata>
>>>>><record number="1"
>>>>>system="http://dev.nceas.ucsb.edu"
>>>>>identifier="obfs2.379.1"
>>>>>namespace="eml://ecoinformatics.org/eml-2.0.0"
>>>>>lastModifiedDate="2003-11-02T11:07:43-0600"
>>>>>creationDate="2003-11-02T11:07:43-0600">
>>>>><param
>
> PM> name="/eml/dataset/keywordSet/keyword">seasonality</param>
>
>>>>><param name="/eml/dataset/keywordSet/keyword">macroalgal
>>>>>bloom</param>
>>>>><param name="/eml/dataset/keywordSet/keyword">green
>
> PM> tide</param>
>
>>>>><param name="/eml/dataset/keywordSet/keyword">Ulva</param>
>>>>><param
>>>>>name="/eml/dataset/creator/individualName/surName">Nelson</param>
>>>>><param name="/eml/dataset/keywordSet/keyword">biomass</param>
>>>>><param name="/eml/dataset/keywordSet/keyword">algal
>
> PM> blooms</param>
>
>>>>><param name="/eml/dataset/title">Armitage Bay Ulvoid Algal
>>>>>Biomass and Species Composition</param>
>>>>><param
>
> PM> name="/eml/dataset/keywordSet/keyword">Enteromorpha</param>
>
>>>>><param name="/eml/dataset/keywordSet/keyword">Ulvaria</param>
>>>>></record>
>>>>>
>>>>>Here is an example of the same resultset as described by the new
>
> PM> approach:
>
>>>>><rs:resultset system="http://knb.ecoinformatics.org"
>
> PM> resultsetId="eml.001"
>
>>>>>xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>>>>>
>>>>>xsi:schemaLocation="ecogrid://ecoinformatics.org/ecogrid-resultset-1.
>>>>>0.0b
>
> PM> eta1
>
>>>>>../../src/xsd/resultset.xsd">
>>>>><resultsetMetadata>
>>>>><sendTime>2004-03-10T13:47:26-0600</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>14</endRecord>
>>>>><recordCount>14</recordCount>
>>>>><namespace>eml://ecoinformatics.org/eml-2.0.0</namespace>
>>>>><namespace
>>>>>prefix="xsi">http://www.w3.org/2001/XMLSchema-instance</namespace>
>>>>></resultsetMetadata>
>>>>><record number="1"
>>>>>system="http://dev.nceas.ucsb.edu"
>>>>>identifier="obfs2.379.1"
>>>>>lastModifiedDate="2003-11-02T11:07:43-0600"
>>>>>creationDate="2003-11-02T11:07:43-0600">
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">seasonality</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">macroalgal bloom</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">green tide</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">Ulva</returnfield>
>>>>><returnfield name="/eml/dataset/creator/individualName/surName"
>>>>>type="xsi:string">Nelson</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">biomass</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">algal blooms</returnfield>
>>>>><returnfield name="/eml/dataset/title"
>>>>>type="xsi:string">Armitage Bay Ulvoid Algal Biomass and Species
>
> PM> Composition</returnfield>
>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">Enteromorpha</returnfield>
>>>>><returnfield name="/eml/dataset/keywordSet/keyword"
>>>>>type="xsi:string">Ulvaria</returnfield>
>>>>></record>
>>>>>
>>>>>Note how we now can interpret the resultset in a much more meaningful
>
> PM>
>
>>>>>way. Also, note that there are two new namespace elements, one
>>>>>contains a "prefix" attr the other does not. The one without becaomes
>
> PM>
>
>>>>>the default namespace for unqualified values in the name and type
>>>>>attrs.
>>>>>
>>>>>Here is the before and after for the DiGIR query:
>>>>>Before:
>>>>><rs:resultset resultsetId="foo.1.1"
>>>>>system="urn:not://sure/what/to/put/here"
>>>>>
>
> PM> xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>
>>>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>
>>>>>xsi:schemaLocation="ecogrid://ecoinformatics.org/ecogrid-resultset-1.
>>>>>0.0b
>
> PM> eta1
>
>>>>>../../src/xsd/resultset.xsd">
>>>>><resultsetMetadata>
>>>>><sendTime>2003-05-02T16:45:50-09:00</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>2</endRecord>
>>>>><recordCount>2</recordCount>
>>>>></resultsetMetadata>
>>>>><record number="1"
>>>>>
>>>>>system="http://speciesanalyst.net/digir/DiGIR.php?resource=MammalsDwC
>
> PM> 2"
>
>>>>>identifier="mvz1"
>>>>>
>
> PM> namespace="http://digir.net/schema/conceptual/darwin/2003/1.0"
>
>>>>>lastModifiedDate="2003-03-03T10:42:13"
>>>>>creationDate="2003-03-03T10:42:13">
>>>>><darwin:ScientificName>PEROMYSCUS LEUCOPUS
>>>>>NOVEBORACENSIS</darwin:ScientificName>
>>>>><darwin:Longitude>121</darwin:Longitude>
>>>>><darwin:Latitude>33</darwin:Latitude>
>>>>></record>
>>>>>
>>>>>After:
>>>>><rs:resultset resultsetId="foo.1.1"
>>>>>system="urn:not://sure/what/to/put/here"
>>>>>
>
> PM> xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>
>>>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>
>>>>>xsi:schemaLocation="ecogrid://ecoinformatics.org/ecogrid-resultset-1.
>>>>>0.0b
>
> PM> eta1
>
>>>>>../../src/xsd/resultset.xsd">
>>>>>
>>>>><resultsetMetadata>
>>>>><sendTime>2003-05-02T16:45:50-09:00</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>2</endRecord>
>>>>><recordCount>2</recordCount>
>>>>>
>>>>><namespace>http://digir.net/schema/conceptual/darwin/2003/1.0</namesp
>
> ace>
>
>>>>><namespace
>>>>>prefix="xsi">http://www.w3.org/2001/XMLSchema-instance</namespace>
>>>>></resultsetMetadata>
>>>>>
>>>>><record number="1"
>>>>>
>>>>>system="http://speciesanalyst.net/digir/DiGIR.php?resource=MammalsDwC
>
> PM> 2"
>
>>>>>identifier="mvz1"
>>>>>lastModifiedDate="2003-03-03T10:42:13"
>>>>>creationDate="2003-03-03T10:42:13">
>>>>><returnfield path="ScientificName"
>>>>>type="xsi:string">PEROMYSCUS LEUCOPUS NOVEBORACENSIS</returnfield>
>>>>><returnfield path="Longitude"
>
> PM> type="xsi:int">121</returnfield>
>
>>>>><returnfield path="Latitude" type="xsi:int">33</returnfield>
>>>>></record>
>>>>>
>>>>>Here is the SRB's before and after:
>>>>>Before:
>>>>><rs:resultset system="http://knb.ecoinformatics.org"
>>>>>resultsetId="SeekSRB_001"
>>>>>xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>
> PM>
>
>>>>>><resultsetMetadata>
>>>>>
>>>>><sendTime>2004-04-16T11:02:12-0500</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>2</endRecord>
>>>>><recordCount>2</recordCount>
>>>>></resultsetMetadata>
>>>>><record number="1"
>>>>>system="http://srb.sdsc.edu"
>>>>>identifier="/home/testuser.sdsc/SeekTestArea/Lesli Model::0"
>>>>>namespace="srb://srb.sdsc.edu"
>>>>>lastModifiedDate="2003-11-30T13:04:59-0600"
>>>>>creationDate="2003-11-30T13:04:58-0600">
>>>>></record>
>>>>>
>>>>>After:
>>>>><rs:resultset system="http://knb.ecoinformatics.org"
>>>>>resultsetId="SeekSRB_001"
>>>>>xmlns:rs="ecogrid://ecoinformatics.org/ecogrid-resultset-1.0.0beta1"
>
> PM>
>
>>>>>><resultsetMetadata>
>>>>>
>>>>><sendTime>2004-04-16T11:02:12-0500</sendTime>
>>>>><startRecord>1</startRecord>
>>>>><endRecord>2</endRecord>
>>>>><recordCount>2</recordCount>
>>>>><namespace>eml://ecoinformatics.org/eml-2.0.0</namespace>
>>>>></resultsetMetadata>
>>>>><record number="1"
>>>>>system="http://srb.sdsc.edu"
>>>>>identifier="/home/testuser.sdsc/SeekTestArea/Lesli Model::0"
>>>>>lastModifiedDate="2003-11-30T13:04:59-0600"
>>>>>creationDate="2003-11-30T13:04:58-0600">
>>>>><returnfield name="location"
>>>>>type="xsi:string">/home/testuser.sdsc/SeekTestArea/Lesli
>>>>>Model::0</returnfield>
>>>>></record>
>>>>>---------------------------------------------------------------------
>>>>>---
>>>>>The Query
>>>>>About the only difference between the old query and the new is that
>
> PM> is
>
>>>>>the returnfield value can concept attr values do not have a namespace
>>>>>then the prefix should be dropped from the namespace element , or
>
> PM> they
>
>>>>>should have a namespace if there is a prefix in the element. For
>
> PM> example:
>
>>>>><?xml version="1.0" encoding="UTF-8"?>
>>>>><egq:query queryId="test.1.1" system="http://knb.ecoinformatics.org"
>>>>>xmlns:egq="ecogrid://ecoinformatics.org/ecogrid-query-1.0.0beta1"
>>>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>
>>>>>xsi:schemaLocation="ecogrid://ecoinformatics.org/ecogrid-query-1.0.0b
>>>>>eta1
>>>>>../../src/xsd/query.xsd">
>>>>><namespace>eml://ecoinformatics.org/eml-2.0.0</namespace>
>>>>><returnfield>/eml/dataset/title</returnfield>
>>>>>
>>>>>
> PM> <returnfield>/eml/dataset/creator/individualName/surName</returnfield>
>
>>>>><returnfield>/eml/dataset/pubDate</returnfield>
>>>>><returnfield>/eml/dataset/keywordSet/keyword</returnfield>
>>>>><title>Soils metadata query</title>
>>>>><AND>
>>>>><OR>
>>>>><condition operator="LIKE"
>
> PM> concept="title">%soil%</condition>
>
>>>>><condition operator="NOT LIKE"
>>>>>concept="title">%dirt%</condition>
>>>>></OR>
>>>>><OR>
>>>>><condition operator="LIKE"
>
> PM> concept="surName">%Jones%</condition>
>
>>>>><condition operator="LIKE"
>>>>>concept="surName">%Vieglais%</condition>
>>>>></OR>
>>>>></AND>
>>>>></egq:query>
>>>>>---------------------------------------------------------------------
>>>>>---
>>>>>
>>>>>We can either discuss this via email, or think about it and discuss
>>>>>it further during our phone meeting.
>>>>>
>>>>>Rod
>>>>>
>>>>>
>>>>>Chad Berkley wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Hi,
>>>>>>
>>>>>>Sorry for my late reply...we've been busy with a morpho release.
>>>>>>thanks for getting me in gear, Rod.
>>>>>>
>>>>>>In metacat, we only return leaf nodes (i.e. the text node child of a
>
> PM>
>
>>>>>>CDATA element like in response 4 below). The returnfield
>>>>>>functionality was originally meant as a convenient way to return
>>>>>>enough information for a meaningful resultset to display, say, on a
>>>>>>web page. It was not meant to return whole document chunks for
>>>>>>further processing. I can see how this would be useful, but it
>>>>>>would require returning a namespace defined chunk so that a parser
>>>>>>would know what to do with it. Metacat currently uses the
>>>>>>returnfields to build the resultset table, then a request must be
>>>>>>made for the whole document in order to do further processing.
>>>>>>
>>>>>>Looking at the responses 1-3 below, to me, they are all invalid and
>>>>>>potentially problematic. without a namespace to parse those xml
>>>>>>chunks off of, the parser is left to just do well-formedness
>>>>>>checking and any query into these document chunks may fail because
>>>>>>we don't know what to expect to get back before doing the processing
>
> PM>
>
>>>>>>(e.g. an xpath query).
>>>>>>
>>>>>>So I guess to make a short answer long, I agree with Peter's
>>>>>>assessment of sticking with response 4 (which is basically what
>>>>>>metacat has done all along).
>>>>>>
>>>>>>chad
>>>>>>
>>>>>>
>>>>>>Rod Spears wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Is anyone better qualified than me, going to address Peter's
>>>>>>>questions?
>>>>>>>
>>>>>>>Please someone respond, thanks.
>>>>>>>
>>>>>>>Rod
>>>>>>>
>>>>>>>
>>>>>>>Peter McCartney wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>it has to be well formed no matter what. so the question is really
>
> PM>
>
>>>>>>>>how can we identify a namespace for the result set when the
>>>>>>>>content we stick in there has no hope of being valid? further, how
>
> PM>
>
>>>>>>>>can we define a set of rules for how the results are to be
>>>>>>>>evaluated against that namespace yet not be valid? request 1:
>>>>>>>>'*/creator/individualName/surname', '/eml/dataset
>>>>>>>>
>>>>>>>>Rule1: "content must appear in minimal xml tree needed to
>>>>>>>>accomodate the informaton"
>>>>>>>>
>>>>>>>>Rule2: "content must appear in a potentially valid xml tree that
>>>>>>>>invalidates only due other required elements missing.
>>>>>>>>
>>>>>>>>rule 3 "conent must appear in a tree that placed in in correct
>>>>>>>>node ancestry for the declared namespace.
>>>>>>>>
>>>>>>>>
>>>>>>>>response 1: meets 1 and 3 and is well formed. Requires just
>>>>>>>>knowledge of parent ancestry to build. <eml>
>>>>>>>><dataset>
>>>>>>>><creator>
>>>>>>>><individualName>
>>>>>>>><surname>mccartney</surname>
>>>>>>>><surname>jones</surname>
>>>>>>>></individualname>
>>>>>>>></creator>
>>>>>>>></dataset>
>>>>>>>><eml>
>>>>>>>>
>>>>>>>>response 2: meets 1, 2 and 3 and is well formed. Requires
>>>>>>>>knowledge of ancestry and index (ie jones is in creator[2] of
>>>>>>>>dataset[1] ) <eml>
>>>>>>>><dataset>
>>>>>>>><creator>
>>>>>>>><individualName>
>>>>>>>><surname>mccartney</surname>
>>>>>>>></individualname>
>>>>>>>></creator>
>>>>>>>><creator>
>>>>>>>><individualName>
>>>>>>>><surname>jones</surname>
>>>>>>>></individualname>
>>>>>>>></creator>
>>>>>>>></dataset>
>>>>>>>><eml>
>>>>>>>>
>>>>>>>>
>>>>>>>>response 3: meets 3 and is not well formed. rquires knowledge of
>>>>>>>>ancestry.
>>>>>>>>
>>>>>>>><eml>
>>>>>>>><dataset>
>>>>>>>><creator>
>>>>>>>><individualName>
>>>>>>>><surname>mccartney</surname>
>>>>>>>></individualname>
>>>>>>>></creator>
>>>>>>>></dataset>
>>>>>>>><eml>
>>>>>>>><dataset>
>>>>>>>><creator>
>>>>>>>><individualName>
>>>>>>>><surname>jones</surname>
>>>>>>>></individualname>
>>>>>>>></creator>
>>>>>>>></dataset>
>>>>>>>></eml>
>>>>>>>>
>>>>>>>>and just a reminder of where we originally started from
>>>>>>>>(approximately)
>>>>>>>>reponse 4: meets no rule, cannot validated, but conveys all the
>>>>>>>>information to generate format 1 or 3 above using a string
>>>>>>>>tokenizer and a jDOM. but not option 2. <resultset
>>>>>>>>namespace=eml......>
>>>>>>>><returnfield
>>>>>>>>xpath="dataset/creator/individualname/surname">mccartney</returnfi
>
> eld>
>
>>>>>>>><returnfield
>>>>>>>>xpath="dataset/creator/individualname/surname">jones</returnfield>
>>>>>>>></resultset>
>>>>>>>>
>>>>>>>>I think we should really ask whether we are making ourselves deal
>>>>>>>>with some very complicated rules for really no gain in
>>>>>>>>functionality. None of the results will be valid according to the
>>>>>>>>name space. All of them are valid if i make up my own namespace
>>>>>>>>for the result set. Unless we can hold our selves to the standard
>
> PM>
>
>>>>>>>>where any code or xsl written for the schema will successfuly
>>>>>>>>process the result set (#2 is the closest to that, but depending
>>>>>>>>on how loose the code is, all three could work or none could
>>>>>>>>work), why shouldnt we opt for the easiest rule to comply with?
>>>>>>>>
>>>>>>>>
>>>>>>>>Peter McCartney (peter.mccartney at asu.edu
>>>>>>>><mailto:peter.mccartney at asu.edu>)
>>>>>>>>Center for Environmental-Studies
>>>>>>>>Arizona State University
>>>>>>>>
>>>>>>>>
>>>>>>>>-----Original Message-----
>>>>>>>>*From:* Saritha Bhandarkar
>>>>>>>>*Sent:* Friday, April 09, 2004 10:28 AM
>>>>>>>>*To:* 'seek-dev'
>>>>>>>>*Cc:* Jing Tao; Peter McCartney; Saritha Bhandarkar
>>>>>>>>*Subject:* resultset question
>>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>I had a question about the resultset to be returned by
>>>>>>>>Xanthoria.
>>>>>>>>
>>>>>>>>The schema of the resultset specifies that a record is of type
>>>>>>>>?AnyRecordType? and optionally it may have some element
>
> PM> content
>
>>>>>>>>from the record. Now, my question here is, if I am to return
>
> PM> the
>
>>>>>>>>elements specified in the <returnfields> of the query, for the
>
> PM>
>
>>>>>>>>matching records (that is from the matching
>>>>>>>>eml file), do I need to send it in eml format, with only
>
> PM> relevant
>
>>>>>>>>values for requested fields and no values for the fields which
>
> PM> are
>
>>>>>>>>not requested? Or is it enough to return only the requested
>
> PM> fields
>
>>>>>>>>with their values, as well-formed xml? Can someone please
>
> PM> brief me
>
>>>>>>>>on the contents of a record in resultsetType?
>>>>>>>>
>>>>>>>>Thanks,
>>>>>>>>
>>>>>>>>Saritha
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>Saritha Bhandarkar
>>>>>>>>
>>>>>>>>Research Assistant
>>>>>>>>
>>>>>>>>Center for Environmental Studies
>>>>>>>>
>>>>>>>>ASU-Tempe AZ
>>>>>>>>
>>>>>>>>saritha.bhandarkar at asu.edu <mailto:saritha.bhandarkar at asu.edu>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>--
>>>>>>>Rod Spears
>>>>>>>Biodiversity Research Center
>>>>>>>University of Kansas
>>>>>>>1345 Jayhawk Boulevard
>>>>>>>Lawrence, KS 66045, USA
>>>>>>>Tel: 785 864-4082, Fax: 785 864-5335
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
> PM>
> PM> _______________________________________________
> PM> seek-dev mailing list
> PM> seek-dev at ecoinformatics.org
> PM> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
More information about the Seek-dev
mailing list