[kepler-dev] Kepler can only be invoked once

Fri May 11 08:41:15 PDT 2007

Matt et al:

Can't one get the best of both worlds:
(1) fix the "~/.kepler bug" to avoid shooting yourself in the foot with
multiple running instances
(2) do data caching 

It seems that it's a consequence of the current implementation that
they are interdependent.

The workaround seems like a good short-term solution (I mean
workaround). Also, maybe this whole caching thing needs to be
rethought/re-engineered. 

Bertram

>>> On Fri, 11 May 2007 08:27:18 -0700
>>> Matthew Jones <jones at nceas.ucsb.edu> wrote: 
MJ> 
MJ> We've known about this issue for a while, and recognize that we need a 
MJ> better solution.
MJ> 
MJ> The issue is that Kepler uses a backend relational db for caching and 
MJ> several data processing activities.  That relational db (hsql) stores 
MJ> its files in a subdir of the ~/.kepler directory.  If more than one 
MJ> instance of Kepler tries to start up, you get the exception that 
MJ> Christopher listed.
MJ> 
MJ> One potential workaround is to have the script that runs the multiple 
MJ> instances run from different accounts, and therefore each instance would 
MJ> use a different .kepler folder and therefore different .hsql db.  The 
MJ> limitation is not really on running more than one instance of kepler on 
MJ> a machine -- its running more than one instance of Kepler in a single 
MJ> user account.  Any workaround that allows each process to have its own 
MJ> .kepler dir will work.  Of course, the downside is that these processes 
MJ> will not share the cache, and therefore large data downloads, etc, will 
MJ> have to be repeated for each instance.
MJ> 
MJ> Matt
MJ> 
MJ> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MJ> Matthew B. Jones
MJ> jones at nceas.ucsb.edu                Ph: 541-888-2581 ext. 287
MJ> National Center for Ecological Analysis and Synthesis (NCEAS)
MJ> UC Santa Barbara
MJ> http://www.nceas.ucsb.edu/ecoinformatics
MJ> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MJ> 
MJ> 
MJ> Christopher Brooks wrote:
>> Hi Norbert,
>> 
>> Your solution will probably run Ptolemy models that do not use the UI.
>> I'm not sure if it will run Kepler models that do anything fancy with
>> ontologies etc.
>> 
>> Also, I had to set the classpath.  
>> I got:
>> --start--
>> #! /bin/bash
>> java -Xmx512m -Xss5m -DKEPLER="$KEPLER" \
>> -classpath "${PTII};${KEPLER}" \
>> -Djava.endorsed.dirs=$KEPLER/lib/jar/apache \
>> -Djava.library.path=$KEPLER/lib
>> ptolemy.actor.gui.MoMLSimpleApplication $*
>> --end--
>> 
>> For info about invoking Ptolemy II, see
>> http://ptolemy.berkeley.edu/ptolemyII/ptIIfaq.htm#invoking%20Ptolemy%20II?
>> 
>> 
>> However, invoking Kepler twice on the same machine via "ant run-dev"
>> does not work.  I get:
>> 
>> [java] Caused by: java.sql.SQLException: The database is already
>> in use by another process: org.hsqldb.NIOLockFile at f0f563da[file =C:\Documents
>> and Settings\cxh\.kepler\cache\cachedata\hsqldb.lck, exists=true, locked=false,
>> valid=false,fl =null]: java.lang.Exception: The process cannot access the file
>> because another process has locked a portion of the file :
>> C:\Documents and Settings\cxh\.kepler\cache\cachedata\hsqldb.lck
>> [java]     at org.hsqldb.jdbc.jdbcUtil.sqlException(Unknown Source)
>> [java]     at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
>> [java]     at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
>> [java]     at org.hsqldb.jdbcDriver.connect(Unknown Source)
>> [java]     at java.sql.DriverManager.getConnection(DriverManager.java:525)
>> [java]     at java.sql.DriverManager.getConnection(DriverManager.java:171)
>> [java]     at org.ecoinformatics.util.DBConnectionFactory.getDBConnection(DBConnectionFactory.java:91)
>> [java]     at org.ecoinformatics.util.DBConnectionFactory.getDBConnection(DBConnectionFactory.java:73)
>> [java]     at org.kepler.gui.KeplerInitializer.initializeSystem(KeplerInitializer.java:133)
>> [java]     at org.kepler.gui.KeplerInitializer.<init>(KeplerInitializer.java:80)
>> [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> [java]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> [java]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>> [java]     at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
>> [java]     at java.lang.Class.newInstance0(Class.java:350)
>> [java]     at java.lang.Class.newInstance(Class.java:303)
>> [java]     at ptolemy.actor.gui.MoMLApplication.readConfiguration(MoMLApplication.java:396)
>> [java]     ... 14 more
>> 
>> BUILD SUCCESSFUL
>> Total time: 9 seconds
>> 
>> 
>> 
>> See
>> http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2315
>> This bug was marked as WONTFIX by Chad, who wrote:
>> 
>>> The only way we could fix this is to have multiple .kepler
>>> directories on one machine. This would require a profiling system
>>> like morpho has.  Since I don't think this is on our list of things
>>> to do, i'm going to mark this WONTFIX.  If at some point we add a
>>> profile system, possibly when we add authentication, this problem
>>> should be fixed.
>> 
>> _Christopher
>> 
>> 
>> --------
>> 
>> Hi Lisong,
>> 
>> If you want to execute workflows from command-line, just create a script 
>> similar to kepler.sh and replace the java command for the following:
>> 
>> java -Xmx512m -Xss5m -DKEPLER="$KEPLER" 
>> -Djava.endorsed.dirs=$KEPLER/lib/jar/apache 
>> -Djava.library.path=$KEPLER/lib ptolemy.actor.gui.MoMLSimpleApplication $*
>> 
>> (note, KEPLER should be defined for the above script)
>> 
>> This can be executed several times at once, giving the workflow name as 
>> parameter.
>> 
>> Best regards
>> Norbert
>> 
>> Norbert Podhorszki
>> ------------------------------------
>> University of California, Davis
>> Department of Computer Science
>> 1 Shields Ave, 2236 Kemper Hall
>> Davis, CA 95616
>> (530) 752-5076
>> pnorbert at cs.ucdavis.edu
>> ----------------------------------
>> 
>> On Thu, 10 May 2007, Sun, Lisong wrote:
>> 
>> > Dear Kepler developers,
>> >
>> > Our group is developing a process integration framework which use Kepler 
>> to
>> > build and run workflows to do scientific computations. We built an organi
>> zer
>> > application that will manage the workflow files on the WebDAV server,
>> > input/output for each run and other provenance information.
>> >
>> > So far we are very impressed by the capabilities of Kepler. Meanwhile we 
>> do
>> > run into a few problems that we are not sure how to deal with.
>> >
>> > One of the problem is that we found Kepler can only be invoked once. And 
>> if
>> > there is a running Kepler, the second invoke command will fail. This does
>> > get in our way of using the Kepler since we manage all the workflow files
>> > using our organizer and need to be able to invoke multiple workflows thro
>> ugh
>> > the organizer. We do noticed that it is possible to open multiple workflo
>> ws
>> > windows through the menu options. Wonder if there is any special reason w
>> hy
>> > it can't run with second invoke command.
>> >
>> > Another goal we want to achieve is to deal with long running workflows. I
>> > wonder if there is anyway we can use the Kepler as an execution engine an
>> d
>> > run workflows without open the GUI window.
>> >
>> > Thanks for your help.
>> >
>> > Best regards,
>> >
>> > Lisong Sun
>> > Staff Scientist
>> > Pacific Northwest National Laboratory
>> > _______________________________________________
>> > Kepler-dev mailing list
>> > Kepler-dev at ecoinformatics.org
>> > http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>> >
>> 
>> _______________________________________________
>> Kepler-dev mailing list
>> Kepler-dev at ecoinformatics.org
>> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>> --------
>> _______________________________________________
>> Kepler-dev mailing list
>> Kepler-dev at ecoinformatics.org
>> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
MJ> _______________________________________________
MJ> Kepler-dev mailing list
MJ> Kepler-dev at ecoinformatics.org
MJ> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev