[kepler-dev] kepler start failure on concurrent runs
Emanuel Dima
emanuel.dima at uni-tuebingen.de
Wed Nov 28 06:25:28 PST 2012
Hi,
I was testing kepler performance for a specific use case by
concurrently starting multiple instances and running the same .kar
workflow. I noticed that from time to time kepler refuses to start and
prints an exception stack.
After digging through the available docs I hypothesized that this must
be caused by the kepler cache being used concurrently and that the
-nocache option should solve the problem, which it did (I also had to
change the workflow file format from .kar to .xml).
The question I have is if this behavior is considered normal or is a
bug that I should report.
For the record, an example of exception log is:
ssh -p 9875 root at localhost time
/data/install-kepler/kepler-2.3/kepler.sh -runwf -nogui
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar
The base dir is /data/install-kepler/kepler-2.3
Kepler.run going to run.setMain(org.kepler.Kepler)
JVM Memory = 5m 512m
RunClasspath:kepler-2.3.0 /data/install-kepler/kepler-2.3/kepler-2.3.0/src
RunClasspath:outreach-2.2.0 /data/install-kepler/kepler-2.3/outreach-2.2.0/src
RunClasspath:r-2.3.0 /data/install-kepler/kepler-2.3/r-2.3.0/src
RunClasspath:loader-2.2.0 /data/install-kepler/kepler-2.3/loader-2.2.0/src
RunClasspath:actors-2.3.0 /data/install-kepler/kepler-2.3/actors-2.3.0/src
RunClasspath:directors-2.0.0 /data/install-kepler/kepler-2.3/directors-2.0.0/src
RunClasspath:opendap-2.0.0 /data/install-kepler/kepler-2.3/opendap-2.0.0/src
RunClasspath:dataturbine-2.2.0
/data/install-kepler/kepler-2.3/dataturbine-2.2.0/src
RunClasspath:ecogrid-2.3.0 /data/install-kepler/kepler-2.3/ecogrid-2.3.0/src
RunClasspath:authentication-gui-2.2.0
/data/install-kepler/kepler-2.3/authentication-gui-2.2.0/src
RunClasspath:gui-2.3.0 /data/install-kepler/kepler-2.3/gui-2.3.0/src
RunClasspath:module-manager-gui-2.3.0
/data/install-kepler/kepler-2.3/module-manager-gui-2.3.0/src
RunClasspath:authentication-2.3.0
/data/install-kepler/kepler-2.3/authentication-2.3.0/src
RunClasspath:repository-2.3.0
/data/install-kepler/kepler-2.3/repository-2.3.0/src
RunClasspath:job-2.1.0 /data/install-kepler/kepler-2.3/job-2.1.0/src
RunClasspath:io-2.0.0 /data/install-kepler/kepler-2.3/io-2.0.0/src
RunClasspath:ssh-2.1.0 /data/install-kepler/kepler-2.3/ssh-2.1.0/src
RunClasspath:data-handling-2.1.0
/data/install-kepler/kepler-2.3/data-handling-2.1.0/src
RunClasspath:sms-2.2.0 /data/install-kepler/kepler-2.3/sms-2.2.0/src
RunClasspath:component-library-2.2.0
/data/install-kepler/kepler-2.3/component-library-2.2.0/src
RunClasspath:util-2.1.0 /data/install-kepler/kepler-2.3/util-2.1.0/src
RunClasspath:event-state-2.1.0
/data/install-kepler/kepler-2.3/event-state-2.1.0/src
RunClasspath:core-2.3.0 /data/install-kepler/kepler-2.3/core-2.3.0/src
RunClasspath:common-2.3.0 /data/install-kepler/kepler-2.3/common-2.3.0/src
RunClasspath:module-manager-2.3.0
/data/install-kepler/kepler-2.3/module-manager-2.3.0/src
RunClasspath:configuration-manager-2.3.0
/data/install-kepler/kepler-2.3/configuration-manager-2.3.0/src
RunClasspath:kepler-tasks-2.3.0
/data/install-kepler/kepler-2.3/kepler-tasks-2.3.0/src
RunClasspath:ptolemy-kepler-2.3.0
/data/install-kepler/kepler-2.3/ptolemy-kepler-2.3.0/src
[null] Checking for patches...
[null] Ran additional initialization for module core from class
org.kepler.module.core.Initialize
[null] Ran additional initialization for module gui from class
org.kepler.module.gui.Initialize
[null] loading: org.kepler.KeplerConfigurationApplication args:
ptolemy/configs/kepler/ConfigNoGUIWithCache.xml -runThenExit
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar
[null] invoking: org.kepler.KeplerConfigurationApplication.main
[null] Kepler Initializing...
[null] Unsupported file type or connection not available:
file:/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar
[null] Extracting workflow from KAR
[null] WARN (org.kepler.kar.KARFile:cacheKARContents:573) KAR
should be in a Local Repository Folder to be inserted in the cache:
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar
[null] Cached KAR contents
[null] Processing entry,
LSID=urn:lsid:kepler-project.org/ns/:17606:39:111,
type=ptolemy.actor.TypedCompositeActor
[null] Running workflow /tmp/moml6081054809666236546.xml
extracted from kar file
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar
[null] 1053 ms. Memory: 89664K Free: 26607K (30%)
[null] Ran additional cleanup for module core from class
org.kepler.module.core.Shutdown
it>(KeplerConfigurationApplication.java:191)
[null] at org.kepler.KeplerConfigurationApplication.main(KeplerConfigurationApplication.java:639)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[null] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[null] at java.lang.reflect.Method.invoke(Method.java:597)
[null] at org.kepler.Kepler.load(Kepler.java:440)
[null] at org.kepler.Kepler.load(Kepler.java:465)
[null] at org.kepler.Kepler.parseArgsAndRun(Kepler.java:379)
[null] at org.kepler.Kepler.main(Kepler.java:260)
[null] Cached KAR contents
[null] Processing entry,
LSID=urn:lsid:kepler-project.org/ns/:17606:39:111,
type=ptolemy.actor.TypedCompositeActor
[null] java.sql.SQLException: Connection is broken: java.io.EOFException
[null] at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
[null] at org.kepler.objectmanager.cache.CacheManager.getObject(CacheManager.java:508)
[null] at org.kepler.KeplerConfigurationApplication.<init>(KeplerConfigurationApplication.java:207)
[null] at org.kepler.KeplerConfigurationApplication.main(KeplerConfigurationApplication.java:639)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[null] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[null] at java.lang.reflect.Method.invoke(Method.java:597)
[null] at org.kepler.Kepler.load(Kepler.java:440)
[null] at org.kepler.Kepler.load(Kepler.java:465)
[null] at org.kepler.Kepler.parseArgsAndRun(Kepler.java:379)
[null] at org.kepler.Kepler.main(Kepler.java:260)
[null] Command failed
[null] java.lang.Exception: Failed to parse
"ptolemy/configs/kepler/ConfigNoGUIWithCache.xml -runThenExit
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar"
[null] at ptolemy.actor.gui.ConfigurationApplication.throwArgsException(ConfigurationApplication.java:791)
[null] at org.kepler.KeplerConfigurationApplication.<init>(KeplerConfigurationApplication.java:339)
[null] at org.kepler.KeplerConfigurationApplication.main(KeplerConfigurationApplication.java:639)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[null] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[null] at java.lang.reflect.Method.invoke(Method.java:597)
[null] at org.kepler.Kepler.load(Kepler.java:440)
[null] at org.kepler.Kepler.load(Kepler.java:465)
[null] at org.kepler.Kepler.parseArgsAndRun(Kepler.java:379)
[null] at org.kepler.Kepler.main(Kepler.java:260)
[null] Caused by: org.kepler.objectmanager.cache.CacheException:
SQL exception when getting object
[null] at org.kepler.objectmanager.cache.CacheManager.getObject(CacheManager.java:533)
[null] at org.kepler.KeplerConfigurationApplication.<init>(KeplerConfigurationApplication.java:207)
[null] ... 9 more
[null] Caused by: java.sql.SQLException: Connection is broken:
java.io.EOFException
[null] at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
[null] at org.kepler.objectmanager.cache.CacheManager.getObject(CacheManager.java:508)
[null] ... 10 more
[null] java.lang.Exception: Failed to parse
"ptolemy/configs/kepler/ConfigNoGUIWithCache.xml -runThenExit
/data/WorkflowEngine/engine/kepler/execute_external_sleep_command.kar"
[null] at ptolemy.actor.gui.ConfigurationApplication.throwArgsException(ConfigurationApplication.java:791)
[null] at org.kepler.KeplerConfigurationApplication.<init>(KeplerConfigurationApplication.java:339)
[null] at org.kepler.KeplerConfigurationApplication.main(KeplerConfigurationApplication.java:639)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[null] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[null] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[null] at java.lang.reflect.Method.invoke(Method.java:597)
[null] at org.kepler.Kepler.load(Kepler.java:440)
[null] at org.kepler.Kepler.load(Kepler.java:465)
[null] at org.kepler.Kepler.parseArgsAndRun(Kepler.java:379)
[null] at org.kepler.Kepler.main(Kepler.java:260)
[null] Caused by: org.kepler.objectmanager.cache.CacheException:
SQL exception when getting object
[null] at org.kepler.objectmanager.cache.CacheManager.getObject(CacheManager.java:533)
[null] at org.kepler.KeplerConfigurationApplication.<init>(KeplerConfigurationApplication.java:207)
[null] ... 9 more
[null] Caused by: java.sql.SQLException: Connection is broken:
java.io.EOFException
[null] at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
[null] at org.hsqldb.jdbc.jdbcStatement.executeQuery(Unknown Source)
[null] at org.kepler.objectmanager.cache.CacheManager.getObject(CacheManager.java:508)
[null] ... 10 more
Thanks,
Emanuel Dima
More information about the Kepler-dev
mailing list