[kepler-dev] RFC new directory structure

Thu Mar 18 08:50:14 PST 2004

Chad:

Thanks for the quick turn-around!

Seems we're converging fast here!

Bertram

PS are you still on travel? (we gotta chat some in Scotia ;-)

>>>>> "CB" == Chad Berkley <berkley at nceas.ucsb.edu> writes:
CB> 
CB> Hello,
CB> See my comments below:
CB> 
CB> Xiaowen Xin wrote:
>> Hi,
>> 
>> David, Ilkay, Zhengang, Dan and I have discussed on the phone and over
>> email the last couple of days about the directory hierarchy
>> reorganization and have come to a rough consensus.  This will be a long
>> email, so please bear with me =)
>> 
>> The problem basically is that there is currently no clear organization
>> of the files in the CVS repository.  Workflows are scattered around the
>> lib/ directory for example, and it's not clear, looking at the
>> repository, which files relate to SPA and which to one of the other
>> projects.
>> 
>> Here's a pictorial view of how we would like to reorganize the
>> repository.  I will be talking mostly about SPA, but the concepts should
>> carry over to the other Kepler projects as well.
>> 
>> - copyright.txt
>> - README
>> - build.xml
>> - bin
>> - runVergil.bat
>> - runVergil.sh
>> - build (directory used for compiling the sources)
>> - docs
>> - lib
>> - jar (directory for all the jars)
>> - dll (directory for all the dlls)
>> - src
>> - org
>> - ecoinformatics
>> - geon
>> - sdm
>> - spa (directory for all the spa-related actors)
>> - util
>> - test
>> - workflows
>> - spa (directory for spa workflows)
>> - seek
>> - geon
>> 
CB> 
CB> this seems good to me, but it might be better to move the workflows 
CB> directory under lib.  it's just nicer to have fewer top level directories.
CB> 
>> 
>> So all the SPA related actors will be in the org.sdm.spa package. 
>> Currently there are some SPA actors in edu.ncsu.sdm, but this doesn't
>> make much conceptual sense.  There's no reason to divide up SPA source
>> files according to the organization that developed it, since all of SPA
>> will be working closely together to create _one_ set of interrelated
>> actors.  Zhengang will work on moving the actors from edu.ncsu.sdm to
>> org.sdm.spa.  It's better to divide up the source files by project
>> rather than by organization because the boundary between organizations
>> is artificial and only serves to confuse things.
>> 
>> Currently, there's a util/ directory in src/.  This isn't really
>> consistent with our naming convention so far (i.e. putting source files
>> in packages that reflect which project made them).  The argument for
>> having a util/ directory has been that we'd like to put useful classes
>> in there that can be shared between the projects.  However, this
>> argument doesn't make much sense because theoretically, all of our
>> actors could be shared between the projects.  So we'd like to move the
>> files in util/ to org/sdm/spa/util.  If another project requires the
>> functionality of those two classes, then it would have to include the
>> org.sdm.spa.util package instead of simply the util package.
>> 
>> After making these changes, all the SPA source files will be in
>> org/sdm/spa, thus making it much easier to distinguish SPA and its
>> contribution to Kepler.
>> 
>> Currently, all the workflows are in lib/ and there are some that are not
>> in CVS at all.  We would like create a top-level workflows/ directory to
>> store all of the workflows.  spa, seek, and geon would be subdirectories
>> under there.  Thus all SPA workflows will be put in workflows/spa/. 
>> Similarly, GEON and SEEK should probably do the same with their
>> workflows.
>> 
>> With this directory structure, it would be easy to tell which workflows
>> are designed for which project, but we must also remember to check all
>> of our workflows into CVS, and update them when/if they break.  Having
>> PIW-full.xml, PIW-full_new_matt.xml, PIW-int-ex0.xml, and
>> PIW-full_new.xml, as we do right now is just plain confusing!
>> 
>> Currently the lib/ directory is a mess.  It appears to be the garbage
>> bin, where everything is dumped if the author can't find a better
>> container for it.  So we propose a series of steps to clean this up.
>> 
>> 1. There are two dll's in lib/.  There should probably be a subdirectory
>> called dll/ under lib/ that contains these dll's.  The person who put
>> these there should probably move them ...
>> 
CB> 
CB> yeah, i'll do that.
CB> 
>> 2. demos.htm and ptolemy-index.html should probably be moved out of lib/
>> and into a more appropriate folder, probably into src/.
CB> 
CB> ptolemy-index.html isn't a demo.  it's the startup view for kepler.  it 
CB> gets copied into the ptolemy directory as kepler-index.html.  I think 
CB> it's fine in lib.
CB> 
>> 
>> 3. Ilkay will dispose of lib/forBerkeley/ and lib/forSB/ folders or move
>> them into the top-level test/ folder since they contain testing material
>> and so don't belong in lib/.  Whoever's responsible for
>> lib/ecoPipelines/ should probably do the same because that's testing
>> material also as I understand it.
CB> 
CB> the ecoPipelines directory just contains some random seek stuff i was 
CB> working on.  i can put them under the workflows directory, whereever it 
CB> ends up.
CB> 
>> 
>> 4. Is everyone ok with our deleting makefile and makefile.lib from
>> lib/?  These are also not library files, and we're not using makefiles
>> any more.
CB> 
CB> I have no problem with that.  ilkay or efrat?
CB> 
>> 
>> 5. We should move runVergil.bat and runVergil.sh into a top-level bin/
>> directory.
CB> 
CB> they aren't really binary, but sure.  Efrat and Ilkay, are you still 
CB> using these?
CB> 
>> 
>> 6. Does anyone know what lib/sample.dat and lib/scew-0.3.1.tar.gz are? 
>> Can we delete them?
CB> 
CB> please do not delete these.  they go with the garp stuff i've been 
CB> working on.  i can move sample.dat into testdata/garp.  scew is in there 
CB> so i had access to the version that I know compiles with the garp cpp code.
CB> 
>> 
>> 7. We will delete the lib/soap directory because it's empty, and there's
>> already a lib/jar/soap that contains jar files.
>> 
>> 8. We need to do something about lib/testdata/ because it's not a
>> library file.  I personally think it should be moved into the global
>> test/ directory.
CB> 
CB> i personally think it's fine where it is, though it would also be fine 
CB> in a test directory.  the test directory was supposed to be for junit 
CB> tests, but more structure could be created under that as not many junit 
CB> tests have been created yet.
CB> 
>> 
>> 9. We will move lib/workflow/ into the top-level workflows/ directory.
>> 
>> The existence of src/exp/ seems a bit questionable.  It seems to stand
>> for "experimental".  Maybe it's time to either make it stable and
>> incorporate it into an existing project, or delete it ...
CB> 
CB> the stuff in exp was was some stuff we were messing around with to 
CB> extend the type system.  it hasn't been used yet but may be in the 
CB> future.  please leave it there.
CB> 
>> 
>> Please comment!  If nobody objects to the proposed restructuring here, I
>> can do nothing but assume everybody loves it =)  We'd like to get this
>> finalized as soon as possible, which would make it easier to create a
>> distribution.  Matt will be back next week from travel I believe, and it
>> would be wonderful to have some kind of rudimentary installer for him.
CB> 
CB> Thanks a lot for looking into this.  the messiness of it's been bugging 
CB> me as well.
CB> 
CB> chad
CB> 
CB> 
>> 
>> Xiaowen
>> 
>> _______________________________________________
>> kepler-dev mailing list
>> kepler-dev at ecoinformatics.org
>> http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
CB> 
CB> 
CB> _______________________________________________
CB> kepler-dev mailing list
CB> kepler-dev at ecoinformatics.org
CB> http://www.ecoinformatics.org/mailman/listinfo/kepler-dev