[kepler-users] Greetings/newbie questions about Kepler
Edward A. Lee
eal at eecs.berkeley.edu
Mon Oct 19 20:43:39 PDT 2009
One thing perhaps worth emphasizing is that asking about the
difference between a building and its foundation is fraught
with difficulty. The building doesn't stand without its
foundation, but the foundation is pointless without the building...
Edward
Matt Jones wrote:
> Edward and Christopher covered most of the differences between Kepler
> and Ptolemy, especially with respect to the 1.0 release. The current
> development version of Kepler that is leading to our 2.0 release
> includes a number of additional major differences, some of which include:
>
> 1) Module-based extension
> - a system for extending Kepler through a well-defined module API
> that allows people and projects to extend and package changes to the
> underlying system, the GUI, the actors, etc. and the published modules
> can be loaded at runtime from within the new (and evolving) Module Manager.
>
> 2) A searchable remote repository that can store new actors,
> workflows, and their associated metadata in order for people to publish
> and share their work
>
> 3) A new module for provenance store that allows tracking of the
> state of the workflow executions and the data that passes among actors
> during a run, and a workflow run manager that uses the provenance store
> to create a searchable database of prior executions.
>
> 4) A GUI report designer module that allows construction of nicely
> formatted reports that organize the outputs and other information about
> a workflow run. After execution, these reports can then be viewed in
> Kepler and can create PDF versions of the report (which we use for a
> web-based reporting system).
>
> 5) Extensions to the remote repository and the packaging format (KAR)
> to allow workflow execution runs from provenance to be exported into an
> archive format, uploaded to the repository, and shared with colleagues.
>
> 6) A smenatic tagging module that extends and generalizes the
> existing ontology system in Kepler to provide a more intuitive GUI for
> tagging actors, workflows, reports, and runs with semantic tags drawn
> from OWL ontologies.
>
> Kepler benefits tremendously from the powerful execution engine and
> infrastructure that is provided by ptolemy, but over time we've been
> improving Kepler by developing features and actors that help streamline
> the process of scientific analysis. Hope this helps in your quest for
> information. When Kepler 2.0 is updated we'll update the FAQ entry
> describing the additional differences between Kepler and Ptolemy.
>
> Matt
>
> On Mon, Oct 19, 2009 at 6:42 AM, Christopher Brooks
> <cxh at eecs.berkeley.edu <mailto:cxh at eecs.berkeley.edu>> wrote:
>
> Hi Tom,
>
> See the Kepler/CORE mission statement at
> https://kepler-project.org/users/projects-using-kepler-1/kepler-core-vision-and-mission
>
> For information about the differences between Kepler and Ptolemy,
> http://ptolemy.berkeley.edu/ptolemyII/ptIIfaq.htm#kepler says
> --start--
>
> Ptolemy (http://ptolemy.eecs.berkeley.edu) and The Kepler Project
> (http://www.kepler-project.org/)
> are two separate projects. The Kepler Project FAQ says
>
> What is the difference between Kepler and Ptolemy?
> Roughly speaking, Ptolemy aims at modeling concurrent systems,
> studying system models, various models of computation, etc. as
> explained above.
> In constrast, Kepler aims at execution of scientific workflows (by
> end users
> and/or workflow engineers), inheriting modeling and design
> capabilities including
> the Vergil GUI and workflow scheduling and execution capabilities
> from Ptolemy.
>
> How does Kepler extend Ptolemy?
> Kepler extensions to Ptolemy include an ever increasing number of
> components
> (called actors in Ptolemy terminology) aimed particularly at
> scientific applications,
> e.g., for remote data and metadata access, data transformations,
> data analysis,
> interfacing with legacy applications, web service invocation and
> deployment,
> provenance tracking, etc. Target application areas include
> bioinformatics, cheminformatics,
> ecoinformatics, and geoinformatics workflows among others.
>
> Kepler also inherits from Ptolemy the actor-oriented modeling
> paradigm that
> separates workflow components from the overall workflow orchestration
> (via so-called directors), making components more easily reusable.
> Through actor-oriented
> and hierarchical modeling features built into Ptolemy, Kepler
> scientific workflows can
> operate at very different level of granularity, from low-level
> "plumbing workflows"
> that explictely move data around, start and monitor remote jobs,
> etc. to high-level
> "conceptual workflows" that interlink complex, domain specific data
> analysis steps.
>
> Concerning using Ptolemy actors within Kepler, Norbert Podhorszki
> writes:
>
> If you find a Ptolemy actor useful, just remember its class name
> (e.g. ptolemy.actor.lib.Ramp). In Kepler, "Tools/Instantiate
> Component" menu allows
> you to type in the class name, which will put an actor instance
> on your canvas just as
> dragging one from the actor tree.
>
> The ptolemy jar is stored in the kepler.jar, so you have access
> to any Ptolemy
> actors and directors.
>
> If you want to use a director not in Kepler tree, you have to use the
> "Tools/Instantiate Attribute" menu. I use it to get a DDF
> director frequently
> (class ptolemy.domains.ddf.kernel.DDFDirector).
>
> I suppose, you can create a kar file for any ptolemy actor just
> as for
> kepler actors, if you want to add some of them into the Kepler
> actor tree.
> --end--
>
>
> 2. In my experimentation, the system appears stable and
> reasonably efficient.
>
> > But my testing has been on the trivial side. Does the system
> scale well to
> > large sophisticated image processing workflows with many levels
> of composite
> > actors and high volume data streams?
>
>
> There are applications with tens of thousands of actors.
> They require machines with quite a bit of memory, however.
>
>
> Ptolemy has JAI and JMF actors. There are also actors that pass
> around Java Image
> types and standard matrices of doubles or integers. Using the
> correct token
> type is the key to good performance. Better performance can be had
> by avoiding
> lots of model based conversions between data formats and instead
> sticking with one token
> type and writing a few Java actors. Using a performance analysis
> tool can help.
>
> We are working on various performance improvements that would
> generate Java code
> for hierarchical sub-components, but this work is still in progress.
>
>
>
> 6. I note you have an actor that provides access to MATLAB.
>
> >> Can the same actor be used for Octave? (see
> http://www.gnu.org/software/octave/).
>
>
> I don't think there is anything standard about the API to MATLAB, so
> I doubt it... But I'm sure one could create an actor for Octave.
>
>
> The Matlab actor will not work with Octave. Last time I tried to
> use Octave
> (probably 10+ years ago), getting Octave to work required Fortran?.
> If there
> is an C API to Octave, then the Matlab actor could be used as an
> template.
>
> There are several ways to interface to external programs:
> 1) Use an Exec actor to open a pipe to the external program and pass
> data.
> This has problems in that it might start up the program each time
> the actor
> fires. Also, passing data can be tricky
> 2) If there is a C interface, then write a custom actor that sends
> and receives
> data
> 3) Micheal Wetter of LBNL has created a system that uses sockets to
> connect
> to external programs. This requires writing a little C code, but
> seems to
> be easier than #2 above. However, I'm not sure if the interface in
> this method
> is as rich as #2.
>
> To me, the biggest issues with interfacing to other programs are:
> - Handling different datatypes can be tricky.
> - Handling error conditions is tricky.
>
>
> _Christopher
>
>
> Edward A. Lee wrote:
>
>
> In addition to the packaging and actor library differences,
> Kepler also has significant UI extensions and infrastructure
> for searching actor libraries...
>
> In some cases, there is functionality that overlaps... E.g.,
> Kepler has an ontology mechanism, but we have built a somewhat
> different ontology mechanism into Ptolemy II. This is part of
> a research project on scalable model engineering. Eventually
> I hope these two mechanisms will merge, but I can't say with
> confidence that that will ever happen...
>
> Anything else I'm missing?
>
> Edward
>
> Thomas M. Parris wrote:
>
> Edward,
>
> Thanks for your prompt reply (over the weekend!). I'll take
> a closer look
> at the FIR actor and the source for the R and MATLAB
> wrappers. I've also
> been focussing on the Kepler documentation instead of the
> Ptolemy II
> doumentation. It looks like I need to switch focus and look
> more
> specifically at Ptolemy II docs. Is it fair to say the
> major difference
> between Kepler and Ptolemy II is packaging and the actor
> library?
>
> -- Tom
>
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org
> <mailto:Kepler-users at kepler-project.org>
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org
> <mailto:Kepler-users at kepler-project.org>
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>
>
> --
> Christopher Brooks (cxh at eecs berkeley edu) University of California
> CHESS Executive Director US Mail: 337 Cory Hall
> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA 94720-1774
> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org <mailto:Kepler-users at kepler-project.org>
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eal.vcf
Type: text/x-vcard
Size: 351 bytes
Desc: not available
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20091019/eb15e81d/attachment.vcf>
More information about the Kepler-users
mailing list