[kepler-users] Greetings/newbie questions about Kepler

Christopher Brooks cxh at eecs.berkeley.edu
Mon Oct 19 07:42:21 PDT 2009 

Hi Tom,

See the Kepler/CORE mission statement at
https://kepler-project.org/users/projects-using-kepler-1/kepler-core-vision-and-mission

For information about the differences between Kepler and Ptolemy,
http://ptolemy.berkeley.edu/ptolemyII/ptIIfaq.htm#kepler says
--start--

Ptolemy (http://ptolemy.eecs.berkeley.edu) and The Kepler Project (http://www.kepler-project.org/)
are two separate projects. The Kepler Project FAQ says

     What is the difference between Kepler and Ptolemy?
  Roughly speaking, Ptolemy aims at modeling concurrent systems,
  studying system models, various models of computation, etc. as explained above.
  In constrast, Kepler aims at execution of scientific workflows (by end users
  and/or workflow engineers), inheriting modeling and design capabilities including
  the Vergil GUI and workflow scheduling and execution capabilities from Ptolemy.

     How does Kepler extend Ptolemy?
     Kepler extensions to Ptolemy include an ever increasing number of components
  (called actors in Ptolemy terminology) aimed particularly at scientific applications,
  e.g., for remote data and metadata access, data transformations, data analysis,
  interfacing with legacy applications, web service invocation and deployment,
  provenance tracking, etc. Target application areas include bioinformatics, cheminformatics,
  ecoinformatics, and geoinformatics workflows among others.

     Kepler also inherits from Ptolemy the actor-oriented modeling paradigm that
  separates workflow components from the overall workflow orchestration
  (via so-called directors), making components more easily reusable. Through actor-oriented
  and hierarchical modeling features built into Ptolemy, Kepler scientific workflows can
  operate at very different level of granularity, from low-level "plumbing workflows"
  that explictely move data around, start and monitor remote jobs, etc. to high-level
  "conceptual workflows" that interlink complex, domain specific data analysis steps.

Concerning using Ptolemy actors within Kepler, Norbert Podhorszki writes:

     If you find a Ptolemy actor useful, just remember its class name
     (e.g. ptolemy.actor.lib.Ramp). In Kepler, "Tools/Instantiate Component" menu allows
     you to type in the class name, which will put an actor instance on your canvas just as
     dragging one from the actor tree.

     The ptolemy jar is stored in the kepler.jar, so you have access to any Ptolemy
     actors and directors.

     If you want to use a director not in Kepler tree, you have to use the
     "Tools/Instantiate Attribute" menu. I use it to get a DDF director frequently
     (class ptolemy.domains.ddf.kernel.DDFDirector).

     I suppose, you can create a kar file for any ptolemy actor just as for
     kepler actors, if you want to add some of them into the Kepler actor tree.
--end--

>> 2. In my experimentation, the system appears stable and reasonably efficient.
 >  But my testing has been on the trivial side.  Does the system scale well to
 > large sophisticated image processing workflows with many levels of composite
 > actors and high volume data streams?
> 
> There are applications with tens of thousands of actors.
> They require machines with quite a bit of memory, however.

Ptolemy has JAI and JMF actors. There are also actors that pass around Java Image
types and standard matrices of doubles or integers.  Using the correct token
type is the key to good performance.  Better performance can be had by avoiding
lots of model based conversions between data formats and instead sticking with one token
type and writing a few Java actors.  Using a performance analysis tool can help.

We are working on various performance improvements that would generate Java code
for hierarchical sub-components, but this work is still in progress.


>> 6. I note you have an actor that provides access to MATLAB.  
 >> Can the same actor be used for Octave? (see http://www.gnu.org/software/octave/).
> 
> I don't think there is anything standard about the API to MATLAB, so
> I doubt it... But I'm sure one could create an actor for Octave.

The Matlab actor will not work with Octave.  Last time I tried to use Octave
(probably 10+ years ago), getting Octave to work required Fortran?.  If there
is an C API to Octave, then the Matlab actor could be used as an template.

There are several ways to interface to external programs:
1) Use an Exec actor to open a pipe to the external program and pass data.
This has problems in that it might start up the program each time the actor
fires.  Also, passing data can be tricky
2) If there is a C interface, then write a custom actor that sends and receives
data
3) Micheal Wetter of LBNL has created a system that uses sockets to connect
to external programs.  This requires writing a little C code, but seems to
be easier than #2 above.  However, I'm not sure if the interface in this method
is as rich as #2.

To me, the biggest issues with interfacing to other programs are:
- Handling different datatypes can be tricky.
- Handling error conditions is tricky.


_Christopher

Edward A. Lee wrote:
> 
> In addition to the packaging and actor library differences,
> Kepler also has significant UI extensions and infrastructure
> for searching actor libraries...
> 
> In some cases, there is functionality that overlaps... E.g.,
> Kepler has an ontology mechanism, but we have built a somewhat
> different ontology mechanism into Ptolemy II.  This is part of
> a research project on scalable model engineering. Eventually
> I hope these two mechanisms will merge, but I can't say with
> confidence that that will ever happen...
> 
> Anything else I'm missing?
> 
> Edward
> 
> Thomas M. Parris wrote:
>> Edward,
>>
>> Thanks for your prompt reply (over the weekend!).  I'll take a closer 
>> look
>> at the FIR actor and the source for the R and MATLAB wrappers.  I've also
>> been focussing on the Kepler documentation instead of the Ptolemy II
>> doumentation.  It looks like I need to switch focus and look more
>> specifically at Ptolemy II docs.  Is it fair to say the major difference
>> between Kepler and Ptolemy II is packaging and the actor library?
>>
>> -- Tom
>>
>>
>> _______________________________________________
>> Kepler-users mailing list
>> Kepler-users at kepler-project.org
>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users

-- 
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director                      US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480

More information about the Kepler-users mailing list