[seek-dev] Re: EcoGrid

[Rod Spears]

I must have been completely jinxed for 2003. I came in on Jan 4th and 
got a simple globus client communicating with a simple globus service. 
This was after comparing and copying several files from the globus 
install on "dev" and I am afraid I can't point my finger to exactly one 
thing I did to get it working. Although, I am fairly confident it had to 
do with some conf files for Tomcat.

Now, that I have something working I am making progress with Digir. Dave 
thought the best approach was to create a  EcoGrid Service that would 
then communicate to a Digir Portal. It is less efficient but cleaner. 
Later if necessary we can create a version of the Digir Portal that 
works specifically for the EcoGrid (but even this step makes that work 
easier)

So for last week and this week, I have an EcoGrid Digir Client that can 
communicate with a "generic" Digir Portal and receive the results back. 
Now I have to write some "generic" code to transform Digir queries into 
EcoGrid queries and then a transform for converting the results back.

First, I will be writing a mini-test harness to test just the query 
transforms and the result set transforms. This way we can test a lot of 
different things more quickly and outside of globus. Also, that way when 
folks start an effort for their own DB they can make sure they have that 
right before moving onto globus.

The second part...Now that the queries and the results sets have been 
proven to be correct, I will write the test harness for proving that the 
read, write and delete works.

Comments on the Big Picture
What I have been really concerned about as of late is: risk reduction. 
The biggest risk I have been concerned about (beyond getting globus to 
work :-) ) has been proving that our pipeline works end-to-end. It 
doesn't have to do much, it just needs to work.

For example:
If we could take a simple data set (mocked up for testing) and divide it 
into metacat and the SRB and then query it and do a little SM (maybe 
that is something as simple as Celsius and Far., whatever) and do all 
this from Kepler, as a "proof of concept" that would be very cool and 
something we could demo.

The items and responses below I think takes us most, if not, all the way 
there.

It seems to me there should be some setting that can be "flipped" to 
turn off the security in globus. I have been through the Globus Java 
code and read most of the docs they have had and seems like it should be 
doable (from looking at the code), but I couldn't anywhere in the docs 
on how to disable it (I saw some things that seem to hint at it).

I think we will want a small "test site" that didn't require any 
security for scientists to at least "try out" SEEK. I think this may be 
what Matt mentioned as an "anonymous" accessibility. If not, I think it 
is still needed. I guess I am thinking that we need a brain dead simple 
way for folks to try this out without stumbling all over the security 
issues. Although, I found it nearly impossible to get globus working 
stand-alone, I still think it is nontrivial for those wanting to be a 
client, to work through some of the security issues. Meaning creating 
the secure key request etc.

What I am really saying is I would like any first user's experience to 
be based solely on what SEEK can do, not on how hard it is to jump 
through the security hoops just to get started. Once they have a taste 
of of it and it is something they know they can really make use of, then 
I think they will have more patience with getting an "official" set up.


Beyond providing a way of limiting access to SEEK, I am at a loss as to 
why we need all this security built into our grid? RSA encryption of 
biological data? I don't think the results of a Digir query is something 
of a national security interest :-D . Plus, it does have an impact on 
performance, but maybe performance isn't an issue.

Rod

Matt Jones wrote:

> Hi Raja,
>
> Thanks for the note.  I more or less agree with your assessment. 
> Unfortunately, I haven't tracked the progress to date very 
> effectively, so I'm not sure if we're ready to package stuff up.  I 
> think we should have another conference call to firm up the progress 
> and decide on a release date and its contents.  The call should 
> include you, me, jing, bing, dave, and rod at a minimum.  I propose
>    Thursday January 15th at 1pm PST
> for a conference call.  Can everyone do it at 1pm PST?  Please let me 
> know if you can not.  When we finalize the time, I'll send out 
> conference call information to everyone.
>
> Some additional notes below...
>
> Arcot Rajasekar wrote:
>
>>
>> Hi Matt
>>     Bertram, Bing and I had a long conversation about what is done and
>> what is needed in the next phase of the project.  I looked at what we 
>> had
>> laid out in our Seattle meeting and later confirmed at the  Santa 
>> Barbara
>> meeting. I think we should package the Ecogrid up and
>> provide it with an easy use client sp that our SEEK scientists canm 
>> start
>> playing with it and give feedback to this.
>> I suggest that we quickly make the following available:
>>     1. Server
>>     Metacat wrapper  (Jing)
>>     SRB Wrapper   (Bing)
>>     Digir wrapper (?)
>>         Xanthoria wrapper (?)
>
> Maybe we can get DiGIR out, but I doubt Xanthoria at this point.  I 
> think Rod has been struggling with getting the basic GT3 setup in 
> place, which indicates to me that GT3 will likely have some serious 
> deployment issues.  But no harm in our releasing what we have working 
> in Metacat/SRB as an alpha of some sort.
>
>>
>>      providing the following functionalities:
>>     query
>>     get
>>     put
>
>
> We haven't implemented "put" AFAIK, unless we got it for free by 
> switching to using GridFTP.
>
>>    2. set up a SEEK registry (already done by Matt)
>
>
> Actually, not complete.  I figured out that we could use the globus 
> tools here, but we still have some work to define the metadata for 
> services that will be registered, install a production GT3 instance to 
> act as a registry node, and write up documentation about how to do it 
> (register a node).
>
>>    3. get a client package
>>          for java based developer users
>>          for web-based users with a simple interface into SEEK ecogrid.
>
>
> Agreed.  Jing has a initial client available.  We can and should 
> package this as a library for distribution and make a web interface.  
> But...we need to do a code review to make sure it is designed 
> properly.  At this point there has been very little design going into 
> the code that's been written, so its all rather ad-hoc.  This is 
> mostly because we've been experimenting with GT3 which has been 
> troublesome at the least.
>
> I also think we should make it a high priority to make Kepler 
> communicate over the EcoGrid to retrieve data.  This has been the 
> limiting factor to demonstrating many of the Kepler tools.  I am 
> hoping Jing can incorporate his current client in Kepler by the BEAM 
> meeting in ABQ coming up in a few weeks.
>
>>
>> This will give the portal into SEEK with some EcoGrid funbctionality 
>> which
>>     we can add more functionality through WSDL for AMS/SMS and such
>
>
> Agreed.  One other major component needs to be finalized before we can 
> proceed: a production CA and a testing framework.
>
> We decided on an architecture for it at the last meeting, but haven't 
> made any real progress towards setting that up.  Until this is set up, 
> we won't be able to share data at all unless we decide to only expose 
> anonymously accessible data over EcoGrid for now.
>
> In Santa Barbara we agreed we needed a testing framework to use to 
> test various EcoGrid implementations.  Rod agreed to code this, 
> probably in JUnit, but I haven't heard anything from him.  This would 
> allow us to insert some standard data in a EcoGrid node, query it, and 
> retrieve it, making sure we're getting the exact same behavior from 
> Metacat, SRB and other EcoGrid wrapped services.  I think we need some 
> progress on this before we can release in good conscience.
>
>> Once this is done, we can redirect the EcoGrid team to AMS work until we
>> have gathered more user input  at which time we can proceed to
>> level2 and level3 or EcoGrid implementation
>
>
> OK.  I think some of the other EcoGrid interfaces are important, such 
> as aggregation nodes, but we can discuss that relative to other 
> priorities during the conf call.
>
> Matt
>
>>
>> What do you think?
>>
>>     thanks
>>         raja
>>
>>
>

-- 
Rod Spears
Biodiversity Research Center
University of Kansas
1345 Jayhawk Boulevard
Lawrence, KS 66045, USA
Tel: 785 864-4082, Fax: 785 864-5335

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mercury.nceas.ucsb.edu/ecoinformatics/pipermail/seek-dev/attachments/20040114/ba5336b3/attachment.htm