[kepler-dev] Running Shell Scripts inside Kepler
Robin Schroeder
Robin.Schroeder at asu.edu
Thu Apr 7 11:14:54 PDT 2005
Interesting! I'll have to look into that. Thanks, Dan! r
> Robin Tori Schroeder (formerly Schoeninger)
> International Institute for Sustainability
> P.O. Box 873211
> Arizona State University
> Tempe, Arizona 85287-3211
> Phone: (480) 727-7290
>
>
-----Original Message-----
From: Dan Higgins [mailto:higgins at nceas.ucsb.edu]
Sent: Thursday, April 07, 2005 11:12 AM
To: Robin Schroeder
Cc: Kepler-Dev; Peter McCartney; Corinna Gries
Subject: Re: [kepler-dev] Running Shell Scripts inside Kepler
Robin,
Thanks for the suggestion. I had experimented with running R in the
Batch mode as you suggested, but I wanted to avoid having to dynamically
create input batch files (due to numerous problem in keeping absolute
paths straight.) Thus I wanted to stream inputs and outputs rather than
using files.
You are correct in stating that you cannot run an R workflow in
unrelated threads (exception - use RServe rather than exec()), but you
can run more that one R command using runtime.exec(). Simply input a
string with several commands separated by newlines. (Or there is an
interactive R example in Kepler). Multiple R commands are all sent to
the same process (with a q() command at the end).
Dan
---
Robin Schroeder wrote:
>Hi Kepler-Dev,
>Have you considered running R in Batch mode?
>I have run it both in Windows and Linux from Java's runtime.exec without issue under Java SDK 1.5 in kepler and as a webservice.
>
>Something like this:
>
>//Make the batch command
>String[] batchCmd = new String[5];
>batchCmd[0] = "/usr/local/R-2.0.0/bin/R";
>batchCmd[1] = "CMD";
>batchCmd[2] = "BATCH";
>batchCmd[3] = "/usr/local/batchFiles/batch1.txt";
>batchCmd[4] = "/usr/local/batchFiles/batch1.out";
>
>Process listener = Runtime.getRuntime().exec(batchCmd, envars);
>
>etc...
>
>R is a shell script.
>I have attached a copy of an example batch.txt and batch.out, if you are interested.
>
>Each time you run java runtime.exec(), each instance begins a new unrelated thread. You can't really treat it like a command line interface. You can't run an R workflow in unrelated threads. R stores variables in memory during the execution of multiple commands. In order to run more than one command in R using the runtime.exec(), you need to use a batch file. I tried this a thousand different ways. I do a very similar thing with grass as well and it works great. :-)
>
>hope this helps, robin
>
>
>
>
>>Robin Tori Schroeder (formerly Schoeninger)
>>International Institute for Sustainability
>>P.O. Box 873211
>>Arizona State University
>>Tempe, Arizona 85287-3211
>>Phone: (480) 727-7290
>>
>>
>>
>>
>
>
>-----Original Message-----
>From: kepler-dev-bounces at ecoinformatics.org
>[mailto:kepler-dev-bounces at ecoinformatics.org]On Behalf Of Tobin Fricke
>Sent: Wednesday, April 06, 2005 9:50 AM
>Cc: Kepler-Dev
>Subject: Re: [kepler-dev] Running Shell Scripts inside Kepler
>
>
>On Wed, 6 Apr 2005, Dan Higgins wrote:
>
>
>
>>Now here is where I ran into trouble!!! On Linux and the Mac, 'R' is a
>>shell script, not a simple executable! And apparently, the Java
>>runtime.exec() method WILL NOT START A SUBPROCESS SET UP THIS WAY WITH A
>>SHELL SCRIPT!!!
>>
>>
>
>The JavaDoc for the 'Process' class contains the following note:
>
> The Runtime.exec methods may not work well for special processes on
> certain native platforms, such as native windowing processes, daemon
> processes, Win16/DOS processes on Microsoft Windows, or shell scripts.
> The created subprocess does not have its own terminal or console. All
> its standard io (i.e. stdin, stdout, stderr) operations will be
> redirected to the parent process through three streams
> (Process.getOutputStream(), Process.getInputStream(),
> Process.getErrorStream()). The parent process uses these streams to feed
> input to and get output from the subprocess. Because some native
> platforms only provide limited buffer size for standard input and output
> streams, failure to promptly write the input stream or read the output
> stream of the subprocess may cause the subprocess to block, and even
> deadlock.
>
>In my test, executing a shell script worked without problem:
>
>[tobin at robin ~/proving]$ ls
>foo.sh RunShellScript.class RunShellScript.java
>
>[tobin at robin ~/proving]$ cat foo.sh
>#!/bin/sh
>echo Hello, World!
>
>[tobin at robin ~/proving]$ cat RunShellScript.java
>
>/* Tobin Fricke
> 2005-04-06
>*/
>
>class RunShellScript {
> public static void main(String args[]) {
> try {
> Process p = Runtime.getRuntime().exec("./foo.sh");
> /* Note that 'input' and 'output' are from the perspective of the
> controlling Java class. This 'input stream' refers to the process's
> stdout! */
> java.io.InputStream o = p.getInputStream();
> /* Wait for the process to exit. */
> p.waitFor();
> /* Collect its output. */
> int l = o.available();
> byte[] b = new byte[l];
> o.read(b, 0, l);
> System.out.print(new String(b));
> } catch (Exception e) {
> System.out.println("Woops! " + e);
> }
> }
>}
>[tobin at robin ~/proving]$ javac RunShellScript.java
>[tobin at robin ~/proving]$ java RunShellScript
>Hello, World!
>[tobin at robin ~/proving]$
>
>I'd make sure that the R shell script starts with the proper "#!" magic
>cookie & interpreter signture. Also, does anyone know how to simply dump
>the 'InputStream' obtained from the process to the Java stdout without
>doing the silly byte[] --> String business I did?
>
>Tobin
>
>http://web.pas.rochester.edu/~tobin/
>
>
>_______________________________________________
>Kepler-dev mailing list
>Kepler-dev at ecoinformatics.org
>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
>
>------------------------------------------------------------------------
>
>tractWaterScan <- scan('C:/ITRSampleData/ABMFirstRun/AgentStates1.txt', what=list(Tick_Count=0,STFID=0,x=0,y=0,Water_Usage=0.0,Water_Price=0.0),skip=1,sep=',');
>tractWater <- data.frame(Tick_Count=tractWaterScan$Tick_Count,STFID=tractWaterScan$STFID,Water_Usage=tractWaterScan$Water_Usage)
>summary(tractWater)
>is.data.frame(tractWater)
>tick0 <- tractWater[tractWater[,1]==1300,]
>getSums <- do.call('rbind', by(tick0$Water_Usage, tick0$STFID, function(x) c(sum=sum(x),mean=mean(x))))
> sumsBySTFID <- data.frame(STFID=rownames(getSums),getSums)
>library('foreign', character.only=TRUE)
>write.dbf(sumsBySTFID, 'C://ITRSampleData/fromR/sumsBySTFID_tick_1300.dbf')
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Kepler-dev mailing list
>Kepler-dev at ecoinformatics.org
>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
>
--
*******************************************************************
Dan Higgins higgins at nceas.ucsb.edu
http://www.nceas.ucsb.edu/ Ph: 805-893-5127
National Center for Ecological Analysis and Synthesis (NCEAS)
Marine Science Building - Room 3405
Santa Barbara, CA 93195
*******************************************************************
More information about the Kepler-dev
mailing list