[kepler-dev] Running Shell Scripts inside Kepler

Robin Schroeder Robin.Schroeder at asu.edu
Thu Apr 7 11:14:54 PDT 2005


Interesting! I'll have to look into that. Thanks, Dan! r

> Robin Tori Schroeder (formerly Schoeninger)
> International Institute for Sustainability 
> P.O. Box 873211
> Arizona State University
> Tempe, Arizona 85287-3211
> Phone: (480) 727-7290
> 
> 


-----Original Message-----
From: Dan Higgins [mailto:higgins at nceas.ucsb.edu]
Sent: Thursday, April 07, 2005 11:12 AM
To: Robin Schroeder
Cc: Kepler-Dev; Peter McCartney; Corinna Gries
Subject: Re: [kepler-dev] Running Shell Scripts inside Kepler


Robin,
    Thanks for the suggestion. I had experimented with running R in the 
Batch mode as you suggested, but I wanted to avoid having to dynamically 
create input batch files (due to numerous problem in keeping absolute 
paths straight.) Thus I wanted to stream inputs and outputs rather than 
using files.
    You are correct in stating that you cannot run an R workflow in 
unrelated threads (exception - use RServe rather than exec()), but you 
can run more that one R command using runtime.exec(). Simply input a 
string with several commands separated by newlines. (Or there is an 
interactive R example in Kepler). Multiple R commands are all sent to 
the same process (with a q() command at the end).

Dan

---

Robin Schroeder wrote:

>Hi Kepler-Dev, 
>Have you considered running R in Batch mode?
>I have run it both in Windows and Linux from Java's runtime.exec without issue under Java SDK 1.5 in kepler and as a webservice.
>
>Something like this:
>
>//Make the batch command
>String[] batchCmd = new String[5];
>batchCmd[0] = "/usr/local/R-2.0.0/bin/R";
>batchCmd[1] = "CMD";
>batchCmd[2] = "BATCH";
>batchCmd[3] = "/usr/local/batchFiles/batch1.txt";
>batchCmd[4] = "/usr/local/batchFiles/batch1.out";
>
>Process listener = Runtime.getRuntime().exec(batchCmd, envars);
>
>etc...
>
>R is a shell script. 
>I have attached a copy of an example batch.txt and batch.out, if you are interested.
>
>Each time you run java runtime.exec(), each instance begins a new unrelated thread. You can't really treat it like a command line interface. You can't run an R workflow in unrelated threads. R stores variables in memory during the execution of multiple commands. In order to run more than one command in R using the runtime.exec(), you need to use a batch file. I tried this a thousand different ways. I do a very similar thing with grass as well and it works great. :-)
>
>hope this helps, robin
>
>
>  
>
>>Robin Tori Schroeder (formerly Schoeninger)
>>International Institute for Sustainability 
>>P.O. Box 873211
>>Arizona State University
>>Tempe, Arizona 85287-3211
>>Phone: (480) 727-7290
>>
>>
>>    
>>
>
>
>-----Original Message-----
>From: kepler-dev-bounces at ecoinformatics.org
>[mailto:kepler-dev-bounces at ecoinformatics.org]On Behalf Of Tobin Fricke
>Sent: Wednesday, April 06, 2005 9:50 AM
>Cc: Kepler-Dev
>Subject: Re: [kepler-dev] Running Shell Scripts inside Kepler
>
>
>On Wed, 6 Apr 2005, Dan Higgins wrote:
>
>  
>
>>Now here is where I ran into trouble!!! On Linux and the Mac, 'R' is a
>>shell script, not a simple executable! And apparently, the Java
>>runtime.exec() method WILL NOT START A SUBPROCESS SET UP THIS WAY WITH A
>>SHELL SCRIPT!!!
>>    
>>
>
>The JavaDoc for the 'Process' class contains the following note:
>
>  The Runtime.exec methods may not work well for special processes on
>  certain native platforms, such as native windowing processes, daemon
>  processes, Win16/DOS processes on Microsoft Windows, or shell scripts.
>  The created subprocess does not have its own terminal or console. All
>  its standard io (i.e. stdin, stdout, stderr) operations will be
>  redirected to the parent process through three streams
>  (Process.getOutputStream(), Process.getInputStream(),
>  Process.getErrorStream()). The parent process uses these streams to feed
>  input to and get output from the subprocess.  Because some native
>  platforms only provide limited buffer size for standard input and output
>  streams, failure to promptly write the input stream or read the output
>  stream of the subprocess may cause the subprocess to block, and even
>  deadlock.
>
>In my test, executing a shell script worked without problem:
>
>[tobin at robin ~/proving]$ ls
>foo.sh  RunShellScript.class  RunShellScript.java
>
>[tobin at robin ~/proving]$ cat foo.sh
>#!/bin/sh
>echo Hello, World!
>
>[tobin at robin ~/proving]$ cat RunShellScript.java
>
>/* Tobin Fricke
>   2005-04-06
>*/
>
>class RunShellScript {
>  public static void main(String args[]) {
>    try {
>      Process p = Runtime.getRuntime().exec("./foo.sh");
>      /* Note that 'input' and 'output' are from the perspective of the
>         controlling Java class.  This 'input stream' refers to the process's
>         stdout! */
>      java.io.InputStream o = p.getInputStream();
>      /* Wait for the process to exit. */
>      p.waitFor();
>      /* Collect its output. */
>      int l = o.available();
>      byte[] b = new byte[l];
>      o.read(b, 0, l);
>      System.out.print(new String(b));
>    } catch (Exception e) {
>      System.out.println("Woops! " + e);
>    }
>  }
>}
>[tobin at robin ~/proving]$ javac RunShellScript.java
>[tobin at robin ~/proving]$ java RunShellScript
>Hello, World!
>[tobin at robin ~/proving]$
>
>I'd make sure that the R shell script starts with the proper "#!" magic
>cookie & interpreter signture.  Also, does anyone know how to simply dump
>the 'InputStream' obtained from the process to the Java stdout without
>doing the silly byte[] --> String business I did?
>
>Tobin
>
>http://web.pas.rochester.edu/~tobin/
>
>
>_______________________________________________
>Kepler-dev mailing list
>Kepler-dev at ecoinformatics.org
>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>  
>
>------------------------------------------------------------------------
>
>tractWaterScan <- scan('C:/ITRSampleData/ABMFirstRun/AgentStates1.txt', what=list(Tick_Count=0,STFID=0,x=0,y=0,Water_Usage=0.0,Water_Price=0.0),skip=1,sep=','); 
>tractWater <- data.frame(Tick_Count=tractWaterScan$Tick_Count,STFID=tractWaterScan$STFID,Water_Usage=tractWaterScan$Water_Usage)
>summary(tractWater)
>is.data.frame(tractWater)
>tick0 <- tractWater[tractWater[,1]==1300,] 
>getSums <- do.call('rbind', by(tick0$Water_Usage, tick0$STFID, function(x) c(sum=sum(x),mean=mean(x))))
> sumsBySTFID <- data.frame(STFID=rownames(getSums),getSums) 
>library('foreign', character.only=TRUE)
>write.dbf(sumsBySTFID, 'C://ITRSampleData/fromR/sumsBySTFID_tick_1300.dbf')
>  
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Kepler-dev mailing list
>Kepler-dev at ecoinformatics.org
>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>  
>


-- 
*******************************************************************
Dan Higgins                                  higgins at nceas.ucsb.edu
http://www.nceas.ucsb.edu/    Ph: 805-893-5127
National Center for Ecological Analysis and Synthesis (NCEAS) 
Marine Science Building - Room 3405
Santa Barbara, CA 93195
*******************************************************************



More information about the Kepler-dev mailing list