[kepler-dev] question about kepler

Sun Jul 20 18:40:13 PDT 2008

Hi Dawid,
The purpose of the replacement actors is to capture output that is usually
just displayed in a swing gui window and store it on the server. All the
replacement actors are in:
http://www.hpc.jcu.edu.au/git/?p=jc124742/hydrant.git;a=tree;f=src/main/java/au/edu/jcu/kepler/hydrant;h=d31cebde81cf645abdb9251bc5188ddb7d75ec5e;hb=master

The file ReplacementManager.java contains the class which the replacement
actors use to send their data to to get stored. An instance of this class is
added to the kepler workspace before execution.

Currently it is built to work specifically with Hydrant (i.e. it translates
the java hashmap into a python dictionary), but can easily be modified to
work in pure java, and i know Jianwu has already done this for his work
(although i don't think his code is available online yet).

The file ReplacementUtils.java contains a static function which replacement
actors use to find the ReplacementManager in the workspace.

in the file:
http://www.hpc.jcu.edu.au/git/?p=jc124742/hydrant.git;a=blob;f=src/main/jython/hydrant/job/helpers.py;h=9e29499a32412e7a1b5f353a5f3f6ed7f6697972;hb=masterthe
class DefaultReplacementManager (starting line 36) extends the
ReplacementManager class (jython allows python code to interact with java
code). The __init__ function (line 37) is the equivalent of a java
constructor. line 38 is equivalent to the java "super(container, name)".
Lines 40-44 set up a directory on the server to store all the received data
in. The abstract function writePythonData is implemented starting line 45.
It simply takes the data that a replacement actor sends to it, writes the
data to a file, and makes a database entry to keep track of the file.

the file
http://www.hpc.jcu.edu.au/git/?p=jc124742/hydrant.git;a=blob;f=src/main/jython/hydrant/job/__init__.py;h=9c06eb4d64fcdfdddec62763ac309c5addf0d76b;hb=masterhas
the function queue_new_job. At line 26 is where the
DefaultReplacementManager is created and added to the workspace.

I hope this helps. I know my use of jython can make things hard to
understand, but it made building the system much simpler (Hydrant took
approximately 6 months, with one person developing and no prior knowledge of
python)!

If you want a more detailed description of the functions in any part of
hydrant feel free to ask me. I'm very happy to help.

Another note about my replacement actors method. Each replacement actor is
almost an exact copy of the actor it is replacing. This means that there is
a lot of code duplication, and if the original actors code changes, the
replacement actor needs to be changed as well. Thus this isn't very
scalable. I've been trying to push for an MVC approach to building actors in
kepler which would allow different execution platforms to handle the view of
actors (i have built a prototype using spring but it hasn't sparked any
interest yet), but until then, apart from building new actors for your web
portal this is the only solution.

Cheers
-Tristan

2008/7/18 dejw <dejw at man.poznan.pl>:

>  Hi Tristan,
>
> thanks for replay, we are now in the phase of requirements gathering and
> later we will work on desired architecture. I can't say at the moment if we
> are going to use web services at all. We have now quite nice frame work with
> all kind of services (http://vinetoolkit.org/) and we will think add
> another service to it.
>
> Our target is possibility to run quite complicated kepler workflows on
> different grids below and use different data storage management systems to
> store all generated data, we need also very detailed info about every step
> of execution of it.
>
> If we have such service (we use spring for our services) then it is no
> problem to make a web service from it later. So it would be actually
> straight way to do it :) At the moment we have to get known the kepler much
> better and gain some experience in it. So I think we will be in touch then.
>
> PS,
>
> I saw the code of your replacement classes for some actors in python. Would
> it be possible for you to write shortly about them in few sentences? What
> every of it changes vs the original actors? I would be grateful. (it's
> always good to hear such thing from the author of code ;)
>
> Cheers,
> Dawid
>
>
> Tristan King wrote:
>
> Hi Dawid,
>  Hydrant was developed with python purely for prototyping reasons. It
> would be maybe 2 weeks work to translate all the python backend code into
> java, and I would be happy to collaborate with you to do this. The only
> problem would be the interface. You would have to write a new one using your
> java web-framework of choice.
>
>  However, I don't plan to build any more on top of Hydrant in it's current
> state. What I want to do is separate the backend execution out from Hydrant
> and have it run as it's own service. This service could then be accessed
> using some form of web services (i.e. SOAP) by Hydrant or any other
> front-end portal. I hope to get the community involved in building something
> like this, as there seems to be a lot of people interested in it, and there
> has already been a lot of work done in separate instances towards such
> technology, it would be great if we can work together to build something
> that everyone can use rather then a whole number of different solutions.
> Unfortunately there has not been a lot of response to my initial attempts to
> get such a project going, and i've since been distracted working on the
> other various things i want to see put into ptolemy/kepler.
>
>  I will try and get back onto this next week and perhaps get some
> requirements and such up on the wiki.
>
>  Cheers
> -Tristan
>
>
>
> 2008/7/17 dejw <dejw at man.poznan.pl>:
>
>> Hi Jianwu again,
>>
>> I want to ask more about this below:
>>
>>
>> 3. Actor repository - I understand that it is possible to have one common
>> actor repository for some organization? Kepler uses LDAP for it? Are there
>> some access rights for actors or something like this? Is it possible to
>> grant some roles to users when LDAP authentication is used? Is it possible
>> to require login while starting Kepler?
>>
>>      You can find the current actor repository at
>> http://library.kepler-project.org/kepler/. Yet it is not widely used
>> currently. We are trying implement more comprehensive actor repository
>> mechanism.
>>
>>
>>   1. So you use LDAP server to store actors? or something else?
>>
>> 2. How about access rights and user roles as regard access to actors in
>> such repository?
>>
>> 3. I understand I can deploy such repository inside some organization and
>> configure kepler to use it?
>>
>> 4. Can I set kepler to require login before start editor? to recognize
>> user somehow?
>>
>> Dawid
>>
>>
>>
>> _______________________________________________
>> Kepler-users mailing list
>> Kepler-users at ecoinformatics.org
>> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-users
>>
>>
>
>
> --
> Tristan King
> Research Officer,
> eResearch Centre
> James Cook University, Townsville Qld 4811
> Australia
>
> Phone: +61747816902
> E-mail: tristan.king at jcu.edu.au www: http://eresearch.jcu.edu.au
>
>
>
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
>

-- 
Tristan King
Research Officer,
eResearch Centre
James Cook University, Townsville Qld 4811
Australia

Phone: +61747816902
E-mail: tristan.king at jcu.edu.au www: http://eresearch.jcu.edu.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mercury.nceas.ucsb.edu/ecoinformatics/pipermail/kepler-dev/attachments/20080721/0cb8460d/attachment.html>