<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Eduardo,<br>
<br>
I agree with you. To make it working, Kepler either need to
share in the externally established SSH connection or establish
the master connection by itself. Because of the limitation of the
underlying JSCH library, I don't think SSH Session actor support
the first option. The second option has to answer two passwords
prompted by different screen questions. The current SSH Session
actor only popup a dialogue for user password. I think Kepler code
can be extended to support two password interaction. Related
classes are org.kepler.ssh.SshSession (especially MyUserInfo inner
class).<br>
<br>
But I don't have access to resources that need two password
interaction. So it's hard for me to update the code. Are you
interested in getting your hands dirty on updating Kepler code? :)<br>
<pre class="moz-signature" cols="72">Best wishes
Sincerely yours
Jianwu Wang, Ph.D.
<a class="moz-txt-link-abbreviated" href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a>
<a class="moz-txt-link-freetext" href="http://users.sdsc.edu/~jianwu/">http://users.sdsc.edu/~jianwu/</a>
Assistant Project Scientist
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A. </pre>
On 11/5/12 2:18 PM, Valente, Eduardo G. (GSFC-610.3)[GLOBAL
SCIENCE & TECHNOLOGY INC] wrote:<br>
</div>
<blockquote cite="mid:CCBDA2F2.D41E%25eduardo.g.valente@nasa.gov"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div>Hi Jianwu,</div>
<div><br>
</div>
<div> The ssh connection is first set up by answering an RSA
token password followed by a system password. The RSA token
based password changes every few seconds. Since there are many
steps (commands) to the remote workflow execution we can't have
every connection prompt for these dynamic passwords. Therefore
it must use the preexisting/authenticated connection. With ssh
the first connection becomes the master and subsequent ones go
straight to the prompt. You can try that yourself. But with
Kepler it would have to either establish the master, or use the
existing system one. If it were to use the system connection
there is no password to provide. If it were to establish the
master connection it would have to answer two passwords prompted
by different screen questions. </div>
<div> The bottom line is that Kepler does not seem to share in
the externally established SSH connection. I will have to
research the JSCH library for more information on this mode of
operation as well.</div>
<div> </div>
<div>Thanks,</div>
<div>Eduardo </div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM: medium none;
BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT:
0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;
BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span
style="font-weight:bold">From: </span> Jianwu Wang <<a
moz-do-not-send="true" href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a>><br>
<span style="font-weight:bold">Date: </span> Thursday,
November 1, 2012 8:12 PM<br>
<span style="font-weight:bold">To: </span> "Valente, Eduardo
G. (GSFC-610.3)[GLOBAL SCIENCE & TECHNOLOGY INC]" <<a
moz-do-not-send="true"
href="mailto:eduardo.g.valente@nasa.gov">eduardo.g.valente@nasa.gov</a>><br>
<span style="font-weight:bold">Cc: </span> Norbert Podhorszki
<<a moz-do-not-send="true" href="mailto:pnorbert@ornl.gov">pnorbert@ornl.gov</a>>,
"<a moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>"
<<a moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>><br>
<span style="font-weight:bold">Subject: </span> Re:
[kepler-users] Sample workflow using JobSubmission/JobManager<br>
</div>
<div><br>
</div>
<div>
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Eduardo,<br>
<br>
If you start kepler from kepler.sh/kepler.bat, not
double-clicking kepler icon, a console will show up and it
normally has more information on errors.<br>
<br>
I checked the '-M' option of ssh and the
implementation of ssh related code in Kepler. I don't
think Kepler supports control master. I also don't see the
Jsch library we used for ssh actors support it. So it's
hard for us to update our code to do it. <br>
<br>
But I think there is still one way workaround. How do
you set up the ssh connection at the first place? Type
your two passwords in command line? Does it work if you
try it in Kepler using 'SSH Session' actor? The same ssh
session actor generated by the 'SSH Session' actor can be
postponed and shared by other workflow executions within
the same JVM. To do it, the parameter 'postpone' of the
actor has to be selected (true) and the 'closeAtEnd' has
to be false. So if you split the workflow into two
workflows. The first workflow only has 'Host' and 'SSH
Session'. The second one has other parts. If you can
generate correct ssh session using the first workflow. You
can run the second workflow many times without creating
new ssh sessions. My tests work here.<br>
<br>
I'm ccing the email to Norbert. He is the original
developer of the actors and workflows. He might have
better solutions for you.<br>
<pre class="moz-signature" cols="72">Best wishes
Sincerely yours
Jianwu Wang, Ph.D.
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a><a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://users.sdsc.edu/%7Ejianwu/">http://users.sdsc.edu/~jianwu/</a>
Assistant Project Scientist
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A. </pre>
On 11/1/12 2:08 PM, Valente, Eduardo G.
(GSFC-610.3)[GLOBAL SCIENCE & TECHNOLOGY INC] wrote:<br>
</div>
<blockquote
cite="mid:CCB85B0E.D2CE%25eduardo.g.valente@nasa.gov"
type="cite">
<div>The failure is graceful in that the session return
true for the failed port. If there is a way to view
logs for that actor please let me know.</div>
<div><br>
</div>
<div>Two factor authentication is the use of two passwords
in the admission process one of which utilizes an RSA
token (changes every 30 seconds or so).</div>
<div><br>
</div>
<div>This means that it becomes impractical to automate
processes if every time we ssh the password is
different. And this two factor authentication cannot be
bypassed with public keys (otherwise known as
passwordless ssh). The only option left is to use an
existing ssh connection enabled as the control master.
Generally the –M switch of ssh clients. But it would
appear the java ssh client and the system ssh client do
not "see" each other. If that is the case than the java
version would need such a mode as well. Currently I see
two modes: interactive with password request and
passwordless with identity file.</div>
<div><br>
</div>
<div>I cannot otherwise provide you a means to recreate
the environment we have. But look into the concept of
control master and you will be able to investigate this
possibility with the ssh session actor.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Eduardo</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM: medium
none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in;
PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP:
#b5c4df 1pt solid; BORDER-RIGHT: medium none;
PADDING-TOP: 3pt"><span style="font-weight:bold">From:
</span> Jianwu Wang <<a moz-do-not-send="true"
href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a>><br>
<span style="font-weight:bold">Date: </span>
Thursday, November 1, 2012 4:52 PM<br>
<span style="font-weight:bold">To: </span> "Valente,
Eduardo G. (GSFC-610.3)[GLOBAL SCIENCE &
TECHNOLOGY INC]" <<a moz-do-not-send="true"
href="mailto:eduardo.g.valente@nasa.gov">eduardo.g.valente@nasa.gov</a>><br>
<span style="font-weight:bold">Cc: </span> "<a
moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>"
<<a moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>><br>
<span style="font-weight:bold">Subject: </span> Re:
[kepler-users] Sample workflow using
JobSubmission/JobManager<br>
</div>
<div><br>
</div>
<div>
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Eduardo,<br>
<br>
So you failed even just using 'SSH Session'
actor? Did you get any error or exception message?
It's new to me for the two-factor authentication.
If you tell me how to reproduce it, I can dig into
it and check what went wrong. <br>
<pre class="moz-signature" cols="72">Best wishes
Sincerely yours
Jianwu Wang, Ph.D.
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a><a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://users.sdsc.edu/%7Ejianwu/">http://users.sdsc.edu/~jianwu/</a>
Assistant Project Scientist
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A. </pre>
On 11/1/12 1:45 PM, Valente, Eduardo G.
(GSFC-610.3)[GLOBAL SCIENCE & TECHNOLOGY INC]
wrote:<br>
</div>
<blockquote
cite="mid:CCB8565D.D2B7%25eduardo.g.valente@nasa.gov"
type="cite">
<div>Thanks. The workflow crashes kepler at a
"type detection" step. So I am trying just the
SSH session step for starters. I am running in
an environment with two-factor authentication.
So I authenticate ssh with control master
enabled hoping that subsequent ssh attempts by
kepler use the existing open connection. At
the kepler workflow I set up the ssh session
with an identity file hoping it would use the
control master session, but it is failing to do
so. Any thoughts on this mode of operation?</div>
<div>Eduardo</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM:
medium none; BORDER-LEFT: medium none;
PADDING-BOTTOM: 0in; PADDING-LEFT: 0in;
PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt
solid; BORDER-RIGHT: medium none; PADDING-TOP:
3pt"><span style="font-weight:bold">From: </span>
Jianwu Wang <<a moz-do-not-send="true"
href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a>><br>
<span style="font-weight:bold">Date: </span>
Wednesday, October 31, 2012 5:50 PM<br>
<span style="font-weight:bold">To: </span>
"Valente, Eduardo G. (GSFC-610.3)[GLOBAL
SCIENCE & TECHNOLOGY INC]" <<a
moz-do-not-send="true"
href="mailto:eduardo.g.valente@nasa.gov">eduardo.g.valente@nasa.gov</a>><br>
<span style="font-weight:bold">Cc: </span> "<a
moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>"
<<a moz-do-not-send="true"
href="mailto:kepler-users@kepler-project.org">kepler-users@kepler-project.org</a>><br>
<span style="font-weight:bold">Subject: </span>
Re: [kepler-users] Sample workflow using
JobSubmission/JobManager<br>
</div>
<div><br>
</div>
<div>
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Eduardo,<br>
<br>
A sample workflow using
JobSubmission/JobManager can be found at <a
moz-do-not-send="true"
class="moz-txt-link-freetext"
href="https://code.kepler-project.org/code/kepler/trunk/workflows/SC06-Tutorial/JobSubmission.xml">https://code.kepler-project.org/code/kepler/trunk/workflows/SC06-Tutorial/JobSubmission.xml</a>.
To use it in PBS environment, you just
need to edit the 'JobManager' parameter to
be 'PBS'. Other parameters such as
'SimTarget' and 'JobScript' also need to
be configured to fit your information.<br>
<pre class="moz-signature" cols="72">Best wishes
Sincerely yours
Jianwu Wang, Ph.D.
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:jianwu@sdsc.edu">jianwu@sdsc.edu</a><a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://users.sdsc.edu/%7Ejianwu/">http://users.sdsc.edu/~jianwu/</a>
Assistant Project Scientist
Scientific Workflow Automation Technologies (SWAT) Laboratory
San Diego Supercomputer Center
University of California, San Diego
San Diego, CA, U.S.A. </pre>
On 10/31/12 2:22 PM, Valente, Eduardo G.
(GSFC-610.3)[GLOBAL SCIENCE &
TECHNOLOGY INC] wrote:<br>
</div>
<blockquote
cite="mid:CCB70EE0.D0C9%25eduardo.g.valente@nasa.gov"
type="cite">
<div>Does anyone have a sample workflow
that exercises a PBS based HPC
environment that they would like to
share? </div>
<div>Thanks.</div>
<div><br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Kepler-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Kepler-users@kepler-project.org">Kepler-users@kepler-project.org</a><a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users">http://lists.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users</a></pre>
</blockquote>
<br>
</div>
</div>
</span> </blockquote>
<br>
</div>
</div>
</span> </blockquote>
<br>
</div>
</div>
</span>
</blockquote>
<br>
</body>
</html>