[kepler-users] [kepler-dev] SRB account needed ?
Jianwu Wang
jianwu at sdsc.edu
Mon Oct 5 13:08:07 PDT 2009
Hi Subhav,
Master-Slave do not need EcoGrid actor, only need Distributed
Composite actor.
There is a sample workflow at
https://code.kepler-project.org/code/kepler/trunk/workflows/distributed/testmodel-works.xml.
You need Kepler 1.x version to run it. use 'ant change-to
-Dsuite=master-slave' at $Kepler.modules/build-area to check out
master-slave module. Full instructions on using Kepler 1.x can be found
at
https://kepler-project.org/developers/teams/build/documentation/build-system-instructions.
Best wishes
Sincerely yours
Jianwu Wang
jianwu at sdsc.edu
http://users.sdsc.edu/~jianwu/
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A.
subhav mital wrote:
> Hi Jianwu,
>
> Thanks for your reply.
>
> 1) I am still contemplating whether installing Open SSH on the server
> would resolve the problem and will have to try this and let you know.
>
> 2) As the repository scales, I would have to use the power of Grid or
> something similar since its not just one simple excel file I would be
> working with.
>
> 3) Master- Slave option sounds interesting. But, do EcoGrid actors
> necessarily be involved in this?
> I am aware that it uses the Distributed Composite Actor. Is there any
> example that I can look at so that I could configure my workflow
> accordingly and send it across?
>
> Thanks!
>
>
>
> On Sun, Oct 4, 2009 at 7:01 PM, Jianwu Wang <jianwu at sdsc.edu
> <mailto:jianwu at sdsc.edu>> wrote:
>
> Hi Subhav,
>
> One solution to use SSH on windows environment is to install
> SSH server on the server.
> http://www.rasyid.net/2007/05/18/install-ssh-server-in-windows-xp/,
> http://www.topshareware.com/MobaSSH-download-59394.htm.
>
> How do you modify/add content are for eg: Excel Files locally?
>
> It looks you do not need huge data transfer and powerful
> remote computing resources. So I think Grid related actors do not
> fit your requirements very well.
>
> Another solution is to use Kepler master-slave architecture
> https://kepler-project.org/developers/incubation/distributed-ms.
> If you send me the local workflow for your purpose, I can try to
> help you transform it using master-slave.
>
> Best wishes
>
> Sincerely yours
>
> Jianwu Wang
> jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>
> http://users.sdsc.edu/~jianwu/ <http://users.sdsc.edu/%7Ejianwu/>
>
> Scientific Workflow Automation Technologies (SWAT) Laboratory
> San Diego Supercomputer Center
> University of California, San Diego
> San Diego, CA, U.S.A.
>
>
>
> subhav mital wrote:
>> Thanks Jianwu.
>>
>> I have tried and implemented the exact same scenario that you are
>> talking about. However, it does not serve my purpose completely,
>> because the files I need to be able to modify/add content are for
>> eg: Excel Files. Hence, the server (remote server) needs to be
>> running on a Windows environment.
>>
>> (Moreover, the control of modifying them on the server needs to
>> be automated and initiated from the Client using Kepler. Though
>> this would be implemented once my primary concern gets resolved).
>>
>> Since this is not possible using the SSH scenario that I tried
>> implementing, I thought the Grid would help in some way.
>>
>> Can you think of any other alternative to get this done? How do
>> other projects collaboratively share/store files across multiple
>> servers? Also, I understand that the Data Grid File Transfer is
>> not working. So, which other Grid related File Support
>> functionality can I use, which can help me better accomplish the
>> aforementioned scenario?
>>
>> Thanks!
>>
>>
>>
>> On Mon, Sep 28, 2009 at 2:29 AM, Jianwu Wang <jianwu at sdsc.edu
>> <mailto:jianwu at sdsc.edu>> wrote:
>>
>> Hi Subhav:
>>
>> There is a test workflow for Data Grid File Transfer
>> actor at
>> https://code.kepler-project.org/code/kepler/trunk/workflows/srb/datagridtest.xml.
>> Yet the server failed to response when I just test it. :(
>>
>> About the scenario, the possible workflows depend on many
>> conditions, such as the server/client operation system, the
>> connection protocols supported between server and clients,
>> and which kind of execution you want to execute on the
>> server. Do you mean client or server which has windows
>> environment? The easiest workflow I can imagine is using SSH
>> File Copier (org.kepler.actor.ssh.FileCopier) or SCP
>> (org.geon.Scp) actor to copy file and use SSH to Execute
>> (org.kepler.actor.ssh.ExecuteCmd) or SSH Execute cmd
>> (org.sdm.spa.Ssh2Exec) actor to execute on the server. Please
>> try.
>>
>> BTW: you do not need to send emails to both kepler-users
>> and kepler-dev, since most Kepler developers are in both
>> mailing lists.
>>
>> Best wishes
>>
>> Sincerely yours
>>
>> Jianwu Wang
>> jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>
>> http://users.sdsc.edu/~jianwu/ <http://users.sdsc.edu/%7Ejianwu/>
>>
>> Scientific Workflow Automation Technologies (SWAT) Laboratory
>> San Diego Supercomputer Center
>> University of California, San Diego
>> San Diego, CA, U.S.A.
>>
>>
>>
>> subhav mital wrote:
>>> Thanks Jianwu.
>>> So how do I go about using the Data Grid File Transfer actor?
>>>
>>> Do I need an SDSC account? Are there any sample workflows
>>> which I can see to understand and learn? (The Globus actors
>>> don't work and the SRB actors require me to have an account)
>>>
>>> My primary interest would be regarding the scenario
>>> mentioned in the User Manual in a Windows environment. These
>>> files may be excel files, for eg, which are being
>>> transferred to the server, some modifications being made
>>> there, and copied back to the client.
>>>
>>> Just for reference, I am pasting the scenario again
>>> mentioned in the manual: "A very common scenario is
>>> the following: a user needs to copy (or stage) a set of
>>> files from one resource (e.g., the local environment) to a
>>> remote resource, run a computational experiment on that remote
>>> resource, and then fetch the results back to the local
>>> environment or copy them to another
>>> resource/database."
>>>
>>> ...Including Kepler-Dev in the loop too..
>>>
>>> Thanks.
>>>
>>>
>>>
>>>
>>> On Sun, Sep 27, 2009 at 1:20 PM, Jianwu Wang
>>> <jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>> wrote:
>>>
>>> Hi Matt,
>>>
>>> DICE group
>>> (http://diceresearch.org/DICE_Site/Home/Home.html) has
>>> iRODS service now. Yet the previous SRB service is still
>>> there. My tests two months ago for SRB actors still
>>> work. For iRODS service, they implemented a new actor
>>> called DataGridTransfer. I checked it in Kepler 1.x.
>>> It's url at kepler trunk is
>>> https://code.kepler-project.org/code/kepler/trunk/modules/actors/src/org/srb/DataGridTransfer.java.
>>>
>>>
>>> The DataGridFileTransfer actor has functionality
>>> similar to the SRB/IRODS commands, namely Sget, Sput,
>>> iget, and iput. DataGridFileTransfer allows users to
>>> copy one or more objects from one local/remote file
>>> system to another local/remote file system. The
>>> following actor expects as input a reference to local or
>>> remote file systems support by the Jargon API . This
>>> reference connection is created from the source and
>>> destination URL values, Currently available file system
>>> URLs are, file:///myDir/myfile.txt,
>>> irods://username:password@myhost.org:1247/myDir/myfile.txt
>>> <mailto:irods://username:password@myhost.org:1247/myDir/myfile.txt>,
>>> srb://username.domain:password@myhost.org:5544/myDir/myfile.txt
>>> <mailto:srb://username.domain:password@myhost.org:5544/myDir/myfile.txt>,
>>> or ftp and http urls.
>>>
>>> Best wishes
>>>
>>> Sincerely yours
>>>
>>> Jianwu Wang
>>> jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>
>>> http://users.sdsc.edu/~jianwu/ <http://users.sdsc.edu/%7Ejianwu/>
>>>
>>> Scientific Workflow Automation Technologies (SWAT) Laboratory
>>> San Diego Supercomputer Center
>>> University of California, San Diego
>>> San Diego, CA, U.S.A.
>>>
>>>
>>>
>>> Matt Jones wrote:
>>>> I've learned that SDSC seems to be migrating away from
>>>> the SRB and instead setting up a new iRODS service.
>>>> Certainly that may affect the accounts that people use,
>>>> and it may also have implications for whether the SRB
>>>> actors continue to function properly -- I'm not sure if
>>>> the iRODS server uses the same API as the SRB server,
>>>> so its not clear to me if they will be backwards
>>>> compatible. Ilkay -- do you know who might be able to
>>>> clarify compatibility plans with the new service at SDSC?
>>>>
>>>> Matt
>>>>
>>>> On Sat, Sep 26, 2009 at 8:55 PM, subhav mital
>>>> <mital.subhav at gmail.com
>>>> <mailto:mital.subhav at gmail.com>> wrote:
>>>>
>>>> Thanks a lot, Jianwu. I was able to download these
>>>> workflows but unable to run them due to the
>>>> certificate error I guess. I have emailed the admin
>>>> and am waiting for a reply to get my account set up.
>>>>
>>>> Thanks!!
>>>>
>>>>
>>>> On Sat, Sep 26, 2009 at 11:29 PM, Jianwu Wang
>>>> <jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>> wrote:
>>>>
>>>> Hi Subhav,
>>>>
>>>> Please try the workflows at
>>>> https://code.kepler-project.org/code/kepler/trunk/workflows/srb/,
>>>> which has test account for SRB. If you want to
>>>> have your own account, you can apply through
>>>> SRB mailing list: srb at sdsc.edu
>>>> <mailto:srb at sdsc.edu>.
>>>>
>>>> Best wishes
>>>>
>>>> Sincerely yours
>>>>
>>>> Jianwu Wang
>>>> jianwu at sdsc.edu <mailto:jianwu at sdsc.edu>
>>>> http://users.sdsc.edu/~jianwu/ <http://users.sdsc.edu/%7Ejianwu/>
>>>>
>>>> Scientific Workflow Automation Technologies (SWAT) Laboratory
>>>> San Diego Supercomputer Center
>>>> University of California, San Diego
>>>> San Diego, CA, U.S.A.
>>>>
>>>>
>>>>
>>>> subhav mital wrote:
>>>>> Hello,
>>>>>
>>>>> I was trying to use SRB actors and believe
>>>>> that I need to set up an account to get a
>>>>> certificate on the website?
>>>>>
>>>>> So in this case, do we use the account already
>>>>> set up for Kepler Users since I do not have a
>>>>> data or compute allocation and wish to have
>>>>> one to run
>>>>> certain workflows...
>>>>>
>>>>> Thanks.
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Kepler-dev mailing list
>>>>> Kepler-dev at kepler-project.org <mailto:Kepler-dev at kepler-project.org>
>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-dev
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Kepler-users mailing list
>>>> Kepler-users at kepler-project.org
>>>> <mailto:Kepler-users at kepler-project.org>
>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20091005/44bb76a2/attachment.html>
More information about the Kepler-users
mailing list