[kepler-dev] Can "stop" interrupt "initialize()"

[Bertram Ludaescher]

Kevin et al:

One more clarification on the data loading issue:
What I've been suggestion (also before) was that we need additional
phases in Kepler e.g. for data staging.
But there is of course a simpler solution -- at least in the short
term: do a different workflow design! 

(a) For example, whatever actor currently is doing the data loading
during the actor initialization, could do so just as part of the
regular firing round!?

(b) Or if that is difficult, create a new actor upstream from the
current one. That actor then does (essentially) nothing during its
init phase, but during it's run phase it does the actual data loading.
Downstream everything will be the same.

The solution (b) has the advantage that it makes the data loading a
separate step. When there is a problem with data loading it will also
be obvious (e.g. using the 'animate' feature!) where things get stuck
(that is, if we don't use PN.. for which we need a progress bar it
seems).

In the long run, it seems desirable however to build in "first-class"
support for data staging etc. so that the workflow designs can become
simpler (while in the proposed solution (b) above, there are now lots
of explicit data loading actors..)

Bertram



>>> On Mon, 30 Jan 2006 07:41:59 -0800
>>> "Bertram Ludaescher" <ludaesch at ucdavis.edu> wrote: 
BL> 
>>> On Mon, 30 Jan 2006 08:41:50 -0600
>>> Kevin Ruland <kruland at ku.edu> wrote: 
KR> 
KR> Bertram,
KR> 
KR> Things like static type checking may be optional (to the workflow) in 
KR> the sense that one could still execute a workflow without having done 
KR> the static type checking step or perhaps the workflow could be executed 
KR> if static type checking fails (user override?).  In this sense, data 
KR> loading is different because it is necessary to have successfully 
KR> completed data loading prior to execution.
BL> 
BL> I agree that e.g. static type checking is (typically) optional Also it
BL> is different from data loading.  On the other hand, there are
BL> similarities between a long-running job / actor and the data
BL> loading. Whenever things take a very long time, a different "mode of
BL> operation" seems necessary. This what I was arguing for (different
BL> stages or phases) in the light of long-running data loading operations
BL> on start-up. For example, wouldn't it be useful to have the
BL> possibility that the user chooses which dataset is being loaded before
BL> it is actually loaded at startup? 
BL> 
KR> Also, if some actor somewhere does do something extremely time consuming 
KR> in it's initialize method, there is currently no pleasant way to 
KR> interrupt it.  The user can click on the stop button, and it does appear 
BL> 
BL> Maybe we should discourage such practice, i.e., having time consuming
BL> initializes. A rule of thumb could be: if it takes longer than a few
BL> seconds, it should be NOT part of the initialization phase, but a
BL> separate phase!? (obviously this needs some more thought)
BL> 
KR> to have some effect -- that is, it's indentation changes when pressed -- 
KR> however, it does not interrupt the processing.  This behavior could be 
KR> considered a but itself.  Either the stop button needs to be disabled 
KR> until it can do something useful, or it should stop the workflow 
KR> regardless of its execution state.
BL> 
BL> I guess in Ptolemy, hitting 'stop' implies that the system goes into a
BL> "defined state" so that when hitting 'resume' (currently the same as
BL> 'play' ;-) things are consistent. A "hard interrupt" would probably
BL> causes problems with restarting later!? 
BL> 
KR> I have been given no requirements for the data loading process - other 
KR> than to see if we can make ptexecute work correctly.
BL> 
BL> For the particular workflow at hand, could the dataloading be moved
BL> from the init phase to a later phase (when 'play' has been hit)? Or is
BL> the data loading already happing at 'play' time!? 
BL> 
BL> Bertram
BL> 
KR> Kevin
KR> 
KR> Bertram Ludaescher wrote:
>>> Kevin:
>>> 
>>> Your suggestion about changing things in the "guts" of Ptolemy scares
>>> me a bit. Moreover, I think it's probably not needed.
>>> 
>>> We had recently an exchange (on kepler-dev) about different workflow
>>> stages or phases, among them something that could be called "data
>>> staging". I think it's time we recognize that we need to state the
>>> requirements data staging, then come up with a design that extend
>>> Ptolemy/Kepler without further overloading the native Ptolemy
>>> capabilities.
>>> 
>>> In particular, I think the "VCR buttons" for "Play", "Pause", and
>>> "Stop" need to be augmented with other buttons for data staging (if
>>> necessary), static type checking (when desired) etc. Doing things
>>> under the (Ptolemy) hood doesn't seem like a good idea.
>>> 
>>> Is there a requirements or design document for the features you've
>>> described and that we need? If not, we should probably put this on the
>>> agenda of the upcoming Kepler meeting..
>>> 
>>> What do you think?
>>> 
>>> cheers
>>> 
>>> Bertram
>>> 
>>> 
>>> 
>>> 
>>> 
>>>>>> On Sun, 29 Jan 2006 17:51:11 -0600
>>>>>> Kevin Ruland <kruland at ku.edu> wrote: 
>>>>>> 
KR> 
KR> Hi.
KR> In Kepler we have some actors which download large volume of data in the 
KR> background.  They do this without any control from the director.  In 
KR> essence this is done when the actor is instantiated and should complete 
KR> prior to the user running the workflow (ie, before initialize() is called).
KR> 
KR> I was looking at the problem we're having with ptexecute where these 
KR> workflows bomb because ptexecute does not know that it needs to wait 
KR> until the data is loaded before executing the workflow.
KR> 
KR> I was thinking of two different alternatives here.  The first is to have 
KR> initialize() verify that the data download is complete and raise an 
KR> IllegalActionException if it is not complete.  This provides feedback to 
KR> a real user when they press "go" too soon, but does not fix the problem 
KR> we have with ptexecute because it will just bomb when initialize() fails.
KR> 
KR> The other alternative is to have initialize() block until the data is 
KR> downloaded.  This solves the ptexecute problem.  The problem now is with 
KR> the UI.  After a real user presses "go", the "stop" button does not 
KR> appear to do anything.  I believe that stop does not attempt to 
KR> interrupt actors stuck in initialize().
KR> 
KR> I think the best solution overall, is to have the stop button actually 
KR> interrupt the workflow thread.  The InterruptedException could be 
KR> trapped and converted into an IllegalActionException.  Unfortunately, 
KR> such a change would very likely be deep in the guts of Ptolemy, and 
KR> could potentially have other unintended consequences.  What do you think?
KR> 
KR> Kevin
KR> 
KR> ----------------------------------------------------------------------------
KR> Posted to the ptolemy-hackers mailing list.  Please send administrative
KR> mail for this list to: ptolemy-hackers-request at ptolemy.eecs.berkeley.edu
>>> 
>>>