[kepler-dev] Combining the CommandLine and Exec actors

[Christopher Brooks]

As I've said in the past, I'm not that happy with the Exec actor as I
wrote it.  When doing the initial implementation, I really wanted
to create an actor that would connect to a long running process that
had state so that I could implement an interactive shell actor.

It turns out this is tricky, especially in SDF, because the output
is somewhat non-deterministic.   For example, if you run a process
in the background, and it generates output asynchronously some random
time later, how do you handle it?  How do you differentiate between
stderr and stdout?  How do you handle processes that hang?

Edward and I decided there were some good PN research opportunities
here.  In the interests of time, Edward suggested I implement 
a fairly basic Exec actor that would work for most people and
that we consider having separate actors that had more complex
interfaces.

So, I'm while I'm all for merging Exec and CommandLine so that we 
avoid code duplication, I also think there is a real need for
a really simple Exec actor.  

Another issue is backward compatibility, it would be nice if models
that are using the Exec actor would continue to work with the new
combined actor.  We can use the MoMLFilter to do a certain amount
of translation, but there are limits as to what we can do.

One common way of handling these problems is to abstract the common
functionality to a base class and provide simple actors that expose
the functionality in various ways.

Anyways, those are my thoughts, feel free to have at the Exec actor
code and modify it.

-Christopher






--------

    Ilkay,
        On Friday on IRC, you mentioned the possibility of combining the 
    CommandLine and Exec actors. I thought I would just write down some 
    thoughts on that idea.
    
    1) The two actors are similar enough that combining them is probably a 
    good idea. There are some subtle differences, however, that I am not 
    sure I fully understand.
    
    2) The CommandLine actor uses the OS-specific commandline shell program 
    for launching files ('cmd.exe' on WindowsXP, 'sh' on Unix machines) to 
    launch a process. This allows the use of all features that one can put 
    on a command line, including the use of file based io redirection (e.g. 
    'program < infile >outfile'). Currently there is not an input stream 
    port for data being sent to the process being launched (but there can be 
    an input file), but there is an output port.
    
    3) The Exec actor does not use the the commandline executable, but 
    rather launches a subprocess directly with support for input/streams 
    being directly connected. Command line redirection to/from files cannot 
    be done with this approach, although the whole thing is simpler because 
    it doesn't involve an intermediate program (the shell) to run the 
    subprocess. The Exec actor also has special threads for grabbing io 
    streams (and I believe these are needed, at times, on Windows).
    
    4) Both the CommandLine and Exec actors wait for the subprocess to 
    complete. The InteractiveExec actor is an attempt to let the underlying 
    subprocess continue operating and thus avoid repeated code startup 
    delays. It still has some problems with when to shut down the subprocess.
    
    5) Simply adding an input stream to the CommandLine actor would make it 
    quite similar to the Exec actor. There would be some confusion about 
    what the presence of  both inputFileHandle and input stream parameters 
    might mean. [I am not sure what it means to tell the commandline shell 
    to use standard input AND redirect the input from a file ? --- And I 
    assume the actor would send info from an input to the commandline shell 
    which would forward it to the program that it launched?] I would guess 
    that if the stream io connections were added to CommandLine, we should 
    also use the stream grabber threads that seem to be needed for Windows.
    
    6) Should we add a parameter to allow the subprocess that is launched to 
    return without completing? (i.e. continue to run) This could be quite 
    useful with some processes (like R) to avoid reloading code and for 
    showing displays created by the subprocess. The process would be created 
    on the first 'fire' event, and we would have to figure out how to 
    reliably shut it down later.
    
    7) Also note that the Exec actor has some 'expert' parameters that allow 
    for setting execution directories and environment variables for the 
    subprocess being launched. These could be useful at times and should 
    probably be included.
    
    Questions and Comments appreciated.
    
    Dan Higgins
    NCEAS
    _______________________________________________
    kepler-dev mailing list
    kepler-dev at ecoinformatics.org
    http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
--------