[kepler-users] Kepler Question

Thu Oct 2 17:47:27 PDT 2008

On Fri, Oct 3, 2008 at 10:31 AM, Jianwu Wang <jianwu at sdsc.edu> wrote:
> Hi Mark,
>
>     We haven't update the work on distributed execution to the web site. The
> forum is still empty. That's why you got the message. Sorry about it. We
> will update this part ASAP.
>
>     About the distributed actors on top of a distributedresource management
> (DRM) systems, we haven't done much on it. Some collaborative organizations
> have implemented some specific distributed actors. We plant to test and
> import them.
>

Yes, I was relieved to find GridWay - trying to support from all these
DRM's within Kepler seems a development-effort blackhole/nightmare.
However, without the UniCluster stack using Gridway, for any
individual (who is a non-IT professional outside multi-million dollar
lab) is jut a pipe-dream due to the presence of Globus.

>     Your information seems very interesting, I will study it and give you
> feedback later. I would appreciate if you can provide much more detailed
> requirements.
>

My ideal use case would be to several workflows managed as a DAG.
GridWay seems to support DAG.  Alternatively perhaps actors/components
of a workflow could be DAG nodes?  This is a little vague since I've
not yet got my hands dirty with Kepler.

Regards

Mark

> Best wishes
>
> Sincerely yours
>
> Jianwu Wang
> jianwu at sdsc.edu
>
> Post-Doctor
> Scientific Workflow Automation Technologies (SWAT) Laboratory
> San Diego Supercomputer Center
> University of California, San Diego
> San Diego, U.S.A.
>
> Mark V wrote:
>
> Hi Mathew,
>
> On Fri, Oct 3, 2008 at 2:18 AM, Matt Jones <jones at nceas.ucsb.edu> wrote:
>
>
> Yes, in addition, we've built what we call a Master/Slave controller system
> in Kepler that creates a distributed execution system that allows a master
> workflow to spawn subworkflows on slave nodes that are running a Kepler
> instance behind an RMI-based controller.  This system has a GUI in Kepler
> for users to choose which nodes to use based on the ones listed in a node
> registry, and a DistributedComposite actor that workflow designers use to
> indicate which parts of a workflow should be executed remotely. There is a
> functioning prototype of this system in the Kepler trunk, and continuing
> work to refine it for shipment withthe next version of Kepler. The Kepler
> Distributed Execution Interest Group is pursuiing this functionality.
>
> https://dev.kepler-project.org/developers/interest-groups/distributed
>
>
>
> When I got to that page and select 'forum' I receive the "We're sorry,
> but that page doesn't exist�" page.
>
> I'm currently following Kepler's development, and was wondering if you
> are implementing Kepler's distributed actor on top of a distributed
> resource management (DRM) systems such as Condor, SGE, etc?
>
> More to the point: GridWay (www.gridway.org) is a meta-scheduler that
> seems to support all those DRM's and more - permitting Kepler's
> distributed actor to be quite agnostic.
>
> A valid objection to using anything that builds on top of Globus is
> that Globus installation/configuration/management is next to
> impossible outside of a fulltime position.  In that respect I'd point
> to the UniCluster grid/cluster software stack (www.grid.org), which
> gives an individual a working Globus stack.
>
> Aprreciate any insight to your thoughts and plans.
>
> Mark
>
>
>
> Matt
>
> On Thu, Oct 2, 2008 at 2:15 AM, Edward A. Lee <eal at eecs.berkeley.edu> wrote:
>
>
> We have done some experimenting with having multiple workflows running
> and communicating in the (underlying) Ptolemy II system.  Yang Zhao made
> some Ptolemy II models a few years ago that implemented a distributed
> chat application as a concept demo.  But we've never really packaged
> up this work...  It would be well worth doing...
>
> Edward
>
>
> Bertram Ludaescher wrote:
>
>
> On Thu, Oct 2, 2008 at 1:27 AM, ted leslie <tleslie at tcn.net
> <mailto:tleslie at tcn.net>> wrote:
>
>
>    I just stumpled upon Kepler and am absolutly blown away!
>
>    I think it will be very useful for me,
>    but one requirement that I would have is,
>
>    Can I have workflows interact?
>    Can one trigger another, or start it up?
>    Can they communicate to one another?
>    (maybe even a WF run within a WF?, import a WF as a data provider to
>    another WF?)
>
>
> There are several answers to this: First, a workflow running within a
> workflow is called a "composite actor" or "subworkflow" in our terminology.
> When one nests a wf inside another one, the question arises: how should this
> be executed? What's the model of computation. Thanks to Ptolemy's various
> underlying models of computation, we can use in Kepler different ways of
> nesting workflows inside one another, all with clear semantics (there are
> papers and documentation describing how this works).
>
> Having said this, we normally don't think of multiple instances of
> workflows as running indepdently and then somehow communicating. Although
> such a thing is possible (e.g. this *might* be related somewhat to Ptolemy's
> life cycle models, where one model (aka workflow in Kepler lingo) can start
> another one), we rather think of workflows typically as data-driven analysis
> pipelines (dataflow process networks).
>
>    Also, I use Linux exclusively, but to avoid Java installs and other
>    prereq.
>    I just put it into a Windows OS on vmware, to give it a quick look.
>    Is the Linux version as full featured as the Windows? (or fuller, or
>    lessor?)
>    (If not what are diffs?)
>
>
> I'm not aware of a difference. And you can always get the latest version
> via the source code repository.
>
>
>    These might be very noob questions, and I don't mind  "RTFM",
>    but these are fundimental Q's I would appreacate brief feed back on,
>    (or reference pointers)
>    before I take a much deeper dive in.
>
>
> Here is a good starting point:
> http://www.kepler-project.org/Wiki.jsp?page=Documentation
>
> Bertram
>
>
>    Thanks,
>
>
>    --
>    ted leslie <tleslie at tcn.net <mailto:tleslie at tcn.net>>
>    _______________________________________________
>    Kepler-users mailing list
>    Kepler-users at ecoinformatics.org
> <mailto:Kepler-users at ecoinformatics.org>
>
>  http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at ecoinformatics.org
>
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Matthew B. Jones
> Director of Informatics Research and Development
> National Center for Ecological Analysis and Synthesis (NCEAS)
> UC Santa Barbara
> jones at nceas.ucsb.edu                       Ph: 1-907-523-1960
> http://www.nceas.ucsb.edu/ecoinfo
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>
>
>
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>