[kepler-dev] Kepler can only be invoked once
Lisong Sun
lisong.sun at pnl.gov
Fri May 11 13:59:24 PDT 2007
Thank your all for the helpful info. About the multiple running problem,
we probably won't care much about data caching. So what does it take in
order to make it working. For example, if the data cache directory is
controlled by an environment variable, we can generate unique temp dir
for each running instance and pass in the path for kepler to use. Or
simply turn off the data caching totally if that is possible.
Lisong
Bertram Ludaescher wrote:
>
> Matt et al:
>
> Can't one get the best of both worlds:
> (1) fix the "~/.kepler bug" to avoid shooting yourself in the foot with
> multiple running instances
> (2) do data caching
>
> It seems that it's a consequence of the current implementation that
> they are interdependent.
>
> The workaround seems like a good short-term solution (I mean
> workaround). Also, maybe this whole caching thing needs to be
> rethought/re-engineered.
>
> Bertram
>
> >>> On Fri, 11 May 2007 08:27:18 -0700
> >>> Matthew Jones <jones at nceas.ucsb.edu> wrote:
> MJ>
> MJ> We've known about this issue for a while, and recognize that we need a
> MJ> better solution.
> MJ>
> MJ> The issue is that Kepler uses a backend relational db for caching and
> MJ> several data processing activities. That relational db (hsql) stores
> MJ> its files in a subdir of the ~/.kepler directory. If more than one
> MJ> instance of Kepler tries to start up, you get the exception that
> MJ> Christopher listed.
> MJ>
> MJ> One potential workaround is to have the script that runs the multiple
> MJ> instances run from different accounts, and therefore each instance
> would
> MJ> use a different .kepler folder and therefore different .hsql db. The
> MJ> limitation is not really on running more than one instance of
> kepler on
> MJ> a machine -- its running more than one instance of Kepler in a single
> MJ> user account. Any workaround that allows each process to have its own
> MJ> .kepler dir will work. Of course, the downside is that these
> processes
> MJ> will not share the cache, and therefore large data downloads, etc,
> will
> MJ> have to be repeated for each instance.
> MJ>
> MJ> Matt
> MJ>
> MJ> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> MJ> Matthew B. Jones
> MJ> jones at nceas.ucsb.edu Ph: 541-888-2581 ext. 287
> MJ> National Center for Ecological Analysis and Synthesis (NCEAS)
> MJ> UC Santa Barbara
> MJ> http://www.nceas.ucsb.edu/ecoinformatics
> MJ> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> MJ>
> MJ>
> MJ> Christopher Brooks wrote:
> >> Hi Norbert,
> >>
> >> Your solution will probably run Ptolemy models that do not use the UI.
> >> I'm not sure if it will run Kepler models that do anything fancy with
> >> ontologies etc.
> >>
> >> Also, I had to set the classpath.
> >> I got:
> >> --start--
> >> #! /bin/bash
> >> java -Xmx512m -Xss5m -DKEPLER="$KEPLER" \
> >> -classpath "${PTII};${KEPLER}" \
> >> -Djava.endorsed.dirs=$KEPLER/lib/jar/apache \
> >> -Djava.library.path=$KEPLER/lib
> >> ptolemy.actor.gui.MoMLSimpleApplication $*
> >> --end--
> >>
> >> For info about invoking Ptolemy II, see
> >>
> http://ptolemy.berkeley.edu/ptolemyII/ptIIfaq.htm#invoking%20Ptolemy%20II?
> >>
> >>
> >> However, invoking Kepler twice on the same machine via "ant run-dev"
> >> does not work. I get:
> >>
> >> [java] Caused by: java.sql.SQLException: The database is already
> >> in use by another process: org.hsqldb.NIOLockFile at f0f563da[file
> =C:\Documents
> >> and Settings\cxh\.kepler\cache\cachedata\hsqldb.lck, exists=true,
> locked=false,
> >> valid=false,fl =null]: java.lang.Exception: The process cannot
> access the file
> >> because another process has locked a portion of the file :
> >> C:\Documents and Settings\cxh\.kepler\cache\cachedata\hsqldb.lck
> >> [java] at org.hsqldb.jdbc.jdbcUtil.sqlException(Unknown Source)
> >> [java] at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
> >> [java] at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
> >> [java] at org.hsqldb.jdbcDriver.connect(Unknown Source)
> >> [java] at
> java.sql.DriverManager.getConnection(DriverManager.java:525)
> >> [java] at
> java.sql.DriverManager.getConnection(DriverManager.java:171)
> >> [java] at
> org.ecoinformatics.util.DBConnectionFactory.getDBConnection(DBConnectionFactory.java:91)
> >> [java] at
> org.ecoinformatics.util.DBConnectionFactory.getDBConnection(DBConnectionFactory.java:73)
> >> [java] at
> org.kepler.gui.KeplerInitializer.initializeSystem(KeplerInitializer.java:133)
> >> [java] at
> org.kepler.gui.KeplerInitializer.<init>(KeplerInitializer.java:80)
> >> [java] at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> >> [java] at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >> [java] at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >> [java] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> >> [java] at java.lang.Class.newInstance0(Class.java:350)
> >> [java] at java.lang.Class.newInstance(Class.java:303)
> >> [java] at
> ptolemy.actor.gui.MoMLApplication.readConfiguration(MoMLApplication.java:396)
> >> [java] ... 14 more
> >>
> >> BUILD SUCCESSFUL
> >> Total time: 9 seconds
> >>
> >>
> >>
> >> See
> >> http://bugzilla.ecoinformatics.org/show_bug.cgi?id=2315
> >> This bug was marked as WONTFIX by Chad, who wrote:
> >>
> >>> The only way we could fix this is to have multiple .kepler
> >>> directories on one machine. This would require a profiling system
> >>> like morpho has. Since I don't think this is on our list of things
> >>> to do, i'm going to mark this WONTFIX. If at some point we add a
> >>> profile system, possibly when we add authentication, this problem
> >>> should be fixed.
> >>
> >> _Christopher
> >>
> >>
> >> --------
> >>
> >> Hi Lisong,
> >>
> >> If you want to execute workflows from command-line, just create a
> script
> >> similar to kepler.sh and replace the java command for the following:
> >>
> >> java -Xmx512m -Xss5m -DKEPLER="$KEPLER"
> >> -Djava.endorsed.dirs=$KEPLER/lib/jar/apache
> >> -Djava.library.path=$KEPLER/lib
> ptolemy.actor.gui.MoMLSimpleApplication $*
> >>
> >> (note, KEPLER should be defined for the above script)
> >>
> >> This can be executed several times at once, giving the workflow name as
> >> parameter.
> >>
> >> Best regards
> >> Norbert
> >>
> >> Norbert Podhorszki
> >> ------------------------------------
> >> University of California, Davis
> >> Department of Computer Science
> >> 1 Shields Ave, 2236 Kemper Hall
> >> Davis, CA 95616
> >> (530) 752-5076
> >> pnorbert at cs.ucdavis.edu
> >> ----------------------------------
> >>
> >> On Thu, 10 May 2007, Sun, Lisong wrote:
> >>
> >> > Dear Kepler developers,
> >> >
> >> > Our group is developing a process integration framework which use
> Kepler
> >> to
> >> > build and run workflows to do scientific computations. We built
> an organi
> >> zer
> >> > application that will manage the workflow files on the WebDAV server,
> >> > input/output for each run and other provenance information.
> >> >
> >> > So far we are very impressed by the capabilities of Kepler.
> Meanwhile we
> >> do
> >> > run into a few problems that we are not sure how to deal with.
> >> >
> >> > One of the problem is that we found Kepler can only be invoked
> once. And
> >> if
> >> > there is a running Kepler, the second invoke command will fail.
> This does
> >> > get in our way of using the Kepler since we manage all the
> workflow files
> >> > using our organizer and need to be able to invoke multiple
> workflows thro
> >> ugh
> >> > the organizer. We do noticed that it is possible to open multiple
> workflo
> >> ws
> >> > windows through the menu options. Wonder if there is any special
> reason w
> >> hy
> >> > it can't run with second invoke command.
> >> >
> >> > Another goal we want to achieve is to deal with long running
> workflows. I
> >> > wonder if there is anyway we can use the Kepler as an execution
> engine an
> >> d
> >> > run workflows without open the GUI window.
> >> >
> >> > Thanks for your help.
> >> >
> >> > Best regards,
> >> >
> >> > Lisong Sun
> >> > Staff Scientist
> >> > Pacific Northwest National Laboratory
> >> > _______________________________________________
> >> > Kepler-dev mailing list
> >> > Kepler-dev at ecoinformatics.org
> >> >
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
> >> >
> >>
> >> _______________________________________________
> >> Kepler-dev mailing list
> >> Kepler-dev at ecoinformatics.org
> >>
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
> >> --------
> >> _______________________________________________
> >> Kepler-dev mailing list
> >> Kepler-dev at ecoinformatics.org
> >>
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
> MJ> _______________________________________________
> MJ> Kepler-dev mailing list
> MJ> Kepler-dev at ecoinformatics.org
> MJ>
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
More information about the Kepler-dev
mailing list