[kepler-dev] [Bug 5429] improve default provenance store performance

[bugzilla-daemon at ecoinformatics.org]

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5429

--- Comment #1 from Derik Barseghian <barseghian at nceas.ucsb.edu> 2011-06-24 18:05:01 PDT ---
We're not explicitly doing a SHUTDOWN of the provenance DB when we quit Kepler,
and "the log is never reused unless there is an abnormal termination, i.e. the
database process is terminated without SHUTDOWN, or it was terminated using
SHUTDOWN IMMEDIATELY." Doing a SHUTDOWN COMPACT of a large provenance store
(~500mb) takes a long time, but then startup is near instant. A plain SHUTDOWN
is significantly faster, and startup seemed to be just as fast. I didn't time
how long SHUTDOWN took, but it seemed a lot faster than our current startup
time, so I believe this would be an improvement. Dan's going to try to add a
SHUTDOWN call to kepler on quit.

Two tangential notes:
1) At the end of the provenanceDB.script file is:
SET WRITE_DELAY 10
even though we're now simply setting WRITE_DELAY true, which should use the
default of 20s.

2) We have cache_file_scale=1. The hsql 1.8 guide makes it sound like this
limits the data file to 1gb, and yet I've had provenance stores bigger. I'm
trying to see if hsql will truncate my DB...in which case we should probably
raise this to the max of 8gb, I guess.

-- 
Configure bugmail: http://bugzilla.ecoinformatics.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.