[seek-kr-sms] Re: [kepler-dev] introduction & distributed ptolemy

Tue Jun 15 03:31:36 PDT 2004

>>>>> "FV" == Ferdinando Villa <ferdinando.villa at uvm.edu> writes:
FV> 
FV> On Mon, 2004-06-14 at 11:01, Bertram Ludaescher wrote:
>> Ideally the ports of an actor should (or at least could)
>> have multiple types:
>> - the data type (including say XML Schema type),
>> - the semantic type (e.g. a concept expression describing more formally
>> what else might be known about the data flowing through the port)
>> [[aside for Ferdinando: our "reductionist/separatist approach" does not 
>> preclude forever an integrated modeling solution - it's just bottom up 
>> to get sth useful soon/in finite time ;-]]
FV> 
FV> Hi Bertram! one point - if you read "reductionistic" (which I probably
FV> wrote somewhere) as "reductive" you're misinterpreting me - "we"
FV> ecologists mostly see reduct. vs. holistic as complementary (with
FV> hierarchical thinking as a possible integrating framework) so when we
FV> say reductionistic, we mean exactly what you also mean... one GOOD way
FV> to look at the problem, usually the most practical, easier to study,
FV> while often the least conducive to synthetic understanding... but NOT
FV> separatist!!!!

oh, that's good news (that we probably mean the same thing). BTW: by
separatist I didn't mean it in the sense of say, "Pais Basque" but
rather in the sense of "sorting things out". 

FV> Philosophy aside, here's a more SEEK-specific provocation: don't you
FV> think that the "conceptual/holistic/top-down" approach is the one that
FV> needs the semantic types, while the "workflow/reductionist/ptolemy"
FV> approach would be just fine with just the "storage/machine" types of
FV> whatever virtual machine AMS will be? 

If you view the latter as just the low level implementation platform,
then yes. But I think it can be more than that and hopefully also
serve us throguh all the levels from conceptual workflows/models to
a  machine-level dataflow engine (or grid workflow if nothing else) 

FV> Also: where do the "transformations" belong (scaling, units)? I'd argue
FV> they belong "mechanically" to the reductionistic side - just like all
FV> other actors, created to calculate a concept - and if the user don't
FV> need to see them, it's not because we hide them up in the conceptual
FV> description, but because they're actors in the workflow, and the
FV> conceptual description that users work with is not the workflow. 

I see. Indeed, workflows (or dataflows for that matter) might "smell"
a bit more procedural than what one would like. But don't you think it
is a suitable formalism to describe analytical processes and
procedures? What else would one use? Sometimes (often) a scientist
might know exactly the algorithm, dataflow network, procedure she
wants to execute. Why not allow the scientist to express the desired
flow as such? 

FV> 
FV> Maybe we're mixing the sides up somewhat, and if so, is this ok... or is
FV> it going to postpone the beautiful "moment of clarity" when we all
FV> realize that we've all been thinking the same thing all along?

probably that moment of clarity and realization will come along
sometime soon ... if not, we need further research .. that's also good 
=B-)

cheers

Bertram

FV> 
FV> Cheers,
FV> ferdinand
FV> 
FV> 
>> - the event consumption/production type (useful for scheduling a la
>> SDF)
>> - the communication type (through the Ptolemy/Kepler client, directly
>> via say FTP or HTTP) etc
>> 
>> At some levels of modeling ones does explicitely hide such detail from 
>> the modeler/user but at other levels this might be a good way of
>> overcoming some scalability issues (if you have terabyte data streams
>> you want them to go directly where they need to)
>> 
>> A related problem of web servies (as actors) is that they send results 
>> back to the caller (Kepler) and don't forward them to the subsequent
>> actor making large data transfers virtually  impossible..
>> 
>> A simple extension to the web service model (anyone knows whether
>> that's already done???) would allow for data to include *references*
>> so that a process would be able return to Kepler just a reference to
>> the result data and that reference would be passed on to the consuming 
>> actor who then understands how to derefernce it. This simple
>> extension seems to be an easy solution to what we called before the
>> 3rd party transfer problem:
>> 
--> [Actor A] ---> [ Actor B] --> ...
>> 
>> To stream large data set D from A to B w/o going through
>> Ptolemy/Kepler one can simply send instead a handle &D and then B,
>> upon receiving &D, understands and dereferenes it by calling the
>> appropriate protocol (FTP/gridFTP, HTTP, SRB,...)
>> 
>> Note that there are already explicit Kepler actors (SRBread/SRBwrite,
>> gridFTP) for large data transfer. More elegant would it be to just
>> send handles in the form, e.g., dereference(http://...<ref-to-D>..)
>> Note that the special tag 'derefence' is needed since not every URL
>> should be dereferenced (a URL can be perfectly valid data all by
>> itself)
>> 
>> It would be good if we could (a) define our extensions in line with
>> web services extensions that deal with dereferencing message parts (if 
>> such exists) and (b) can work on a joint
>> Kepler/Ptolemy/Roadnet/SEEK/SDM etc approach (in fact, Kepler is such
>> a joint forum for co-designing this together..)
>> 
>> Bertram
>> 
>> PS Tobin: I recently met Kent and heard good news about ORB access in
>> Kepler already. You can also check with Efrat at SDSC on 3rd party
>> transfer issues while you're at SDSC..
>> 
>> >>>>> "EAL" == Edward A Lee <eal at eecs.berkeley.edu> writes:
EAL> 
EAL> At 05:48 PM 6/11/2004 -0700, Tobin Fricke wrote:
>> >> A basic question I have is, is there a defined network transport for
>> >> Ptolemy relations?  I expect that this question isn't really well-formed
>> >> as I still have some reading to do on how relations actually work.
>> >> Nonetheless, there is the question of, if we have different instances of
>> >> Ptolemy talking to each other across the network, how are the data streams
>> >> transmitted?  In our case one option is to use the ORB as the stream
>> >> transport, equipping each sub-model with ORB source and ORB sink
>> >> components; and perhaps this could be done implicitly to automatically
>> >> distribute a model across the network.  But this line of thinking is
>> >> strongly tied to the idea of data streams and may not be appropriate for
>> >> the more general notion of relations in Ptolemy.
EAL> 
EAL> We have done quite a bit of experimentation with distributed
EAL> Ptolemy II models, but haven't completely settled on any one
EAL> approach... Most of the recent work in this area has been
EAL> done by Yang Zhao, whom I've cc'd for additional comments...
EAL> Here are some notes:
EAL> 
EAL> - A model can contain a component that is defined elsewhere
EAL> on the network, referenced at a URL.  There is a demo
EAL> in the quick tour that runs a submodel that sits on our
EAL> web server.
EAL> 
EAL> - The Corba library provides a mechanism for transporting
EAL> tokens from one model to another using either push or
EAL> pull style interactions.  The software is in the
EAL> ptolemy.actor.corba package, but there are currently
EAL> no good (easily run) demos, and documentation is sparse.
EAL> 
EAL> - The MobileModel actor accepts a model definition on an
EAL> input port and then executes that model.  Yang has used
EAL> this with the Corba actors to build models where one
EAL> model constructs another model and sends it to another
EAL> machine on the network to execute.
EAL> 
EAL> - The JXTA library (ptolemy.actor.lib.jxta) uses Sun's
EAL> XML-based P2P mechanism.  Yang has used this to construct
EAL> a distributed chat room application.
EAL> 
EAL> - The ptolemy.actor.lib.net has two actors DatagramReader
EAL> and DatagramWriter that provide low-level mechanisms for
EAL> models to communicate over the net.  Three or four years
EAL> ago Win Williams used this to created a distributed model
EAL> where two computers on the net were connected to
EAL> motor controllers and users could "arm wrestle" over
EAL> the network ... when one of the users turned his motor,
EAL> the other motor would turn, and they could fight each
EAL> other, trying to turn the motors in opposite directions.
EAL> 
EAL> - Some years ago we also did some experimentation with
EAL> Sun's JINI P2P mechanism, but this has been largely
EAL> supplanted by JXTA.
EAL> 
EAL> - The security library (ptolemy.actor.lib.security)
EAL> provides encryption and decryption and authentication
EAL> based on digital signatures.
EAL> 
EAL> Most of these mechanisms have not been well packaged,
EAL> and we haven't worked out the "lifecycle management" issues
EAL> (how to start up a distributed model systematically, how
EAL> to manage network failures).
EAL> 
EAL> In my view, working out these issues is a top priority...
EAL> I would be delighted to work with you or anyone else on this...
EAL> 
EAL> Edward
EAL> 
EAL> 
EAL> 
EAL> 
EAL> 
EAL> ------------
EAL> Edward A. Lee, Professor
EAL> 518 Cory Hall, UC Berkeley, Berkeley, CA 94720
EAL> phone: 510-642-0455, fax: 510-642-2739
EAL> eal at eecs.Berkeley.EDU, http://ptolemy.eecs.berkeley.edu/~eal
EAL> 
EAL> _______________________________________________
EAL> kepler-dev mailing list
EAL> kepler-dev at ecoinformatics.org
EAL> http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
>> _______________________________________________
>> kepler-dev mailing list
>> kepler-dev at ecoinformatics.org
>> http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
FV> -- 