[kepler-dev] Re: [SDM-SPA] RFC new directory structure

[Christopher Hylands Brooks]

I like the CVS structure you propose below.

Ptolemy development follows a similar structure where releases get
branches, the main line is the development tree.

In Ptolemy, experimental code goes in ptolemy/apps.  ptolemy/apps is
not released, and not usually built as part of the nightly build.
Some code is eventually moved from ptolemy/apps into released part of
the tree.

In general, merging code back from a branch in to the main line can be
slightly tricky, which is why I prefer not to have experimental work
in a separate branch.

I can see situations where having experimental work in a separate
branch and folding it back in would be a good idea, but this has not
come up for us.

A couple of other points:

* We follow a "don't break the build" policy, where the tree should
always compile from scratch.  Work in progress that does not compile
is either in apps/ or in a directory that is not listed in another
makefile and thus not compiled.  It is ok to have new features that
compile but do not work as long as preexisting features continue to
work.

* With regard to dlls etc., if there are dlls, then there are probably
lib*.so files for Linux etc.  In Ptolemy classic, each platform
had a lib.$PTARCH and bin.$PTARCH directory for platform dependent
variables.   PTARCH was usually set to a platform, such as sol2.8, or
linux, but could also be set to include a specific platform and or
debugging level (sol2.8.cfront.debug).  This allows the developer to
use multiple compilers and debugging levels.

I'm not sure if this would applies to your situation.  Ptolemy II
does not use this system (it probably should).

I'm slightly opposed to a directory called dll, if only because there
is probably a need for a place for shared libraries under Linux or
Solaris as well.  I would rather see the directory called w32.
I don't feel strongly about this, it is just a slight preference.

* Another issue is that Ptolemy II has a vendors/ directory that
contains software from other vendors that we are not shipping, but
portions of Ptolemy II rely on it.  Often the configure script will
look in vendors/ for optional packages and if the code is not found,
print a message about where to get the software and where to install
it.

Having a vendors/ directory might eventually be useful.

* I prefer to see the tests adjacent to the code that is tested.
  - It makes it easier for the developer to run unit tests, in
  Ptolemy, the developer can run 'make test' in many directories
  and something useful will happen.
  - If only a portion of a project is shipped, it is easier to include
  the tests.  This is especially true if a end user decides that they
  only want to use a few directories in a project of their own.
  If the tests are adjacent or under the code, they are more likely
  to copy the tests.

* How are nightly builds and tests handled right now?  Forgive my
ignorance on this, it might be apparent from the code, I just have
not looked.

-Christopher

--------

    Some more build comments, follwoing on my earlier comments:
    
    Xiaowen Xin wrote:
    > Hi Mladen,
    > 
    > Actually, right now, new development goes into src/ just like all other
    > source code.  As long as it compiles and it doesn't disrupt other
    > people's classes, it's allowed in src/.  But of course this wouldn't
    > work for the few days/weeks leading up to the release.  We should
    > probably have a code freeze for a period of time before the release so
    > that we can all test it and make sure it works.
    > 
    > I think the best way to deal with this is to have branches in CVS.  I
    > believe this is how most major projects do it.  So we could have a
    > stable branch, from which we would make a release.  In addition, we
    > would have a development branch where untested code will go.
    Agreed. This reiterates my emails from earlier.
    
    > 
    > So we don't need development directories.  We just need a development
    > branch in CVS.
    I prefer to model the tree as a semi-stable development version, with 
    branches added for 1) highly experimental work, and 2) releases as they 
    are stabilized, and 3) patches to those releases.  This allows 
    development to continue while the release is debugged.  Here's an 
    example tree CVS revision tree for one file:
    
                               KEPLER-1_0_0_BRANCH             KEPLER-1_0_0
                            -- 1.3.0.1 -- 1.3.0.2 -- 1.3.0.3 -- 1.3.0.4
                           /
    1.0 -- 1.1 -- 1.2 -- 1.3 -- 1.4 -- 1.5 -- 1.6 -- 1.7 --1.8
                                  \
                                  EXP_FEATURE1_BRANCH
                                   -- 1.4.0.1 -- 1.4.0.2 -- 1.4.0.3
    
    Of course, this model requires that both the release bug fixes (e.g., 
    from 1.3.0.4) and experimental features (from 1.4.0.3) must be merged 
    into the HEAD when they are finished, else they be lost from the tree. 
    The alternative (of having separate development branches) requires many 
    more merge operations, which are hard.  So, I think this is the most 
    stable model for developers, because it keeps the development tree on 
    the trunk.  There are a number of treatises on the web about how to use 
    CVS in a team environment, and this is a common (maybe even preferred) 
    model for shared code repositories to minimize merging while maximizing 
    stability.
    
    
    > 
    > While we're on the topic, Zhengang has suggested that we outline a
    > release schedule and goals for our next release.  I think that's a great
    > idea because I personally would really like to see a release soon to
    > give to Matt.  So any suggestions/proposals? =)
    
    Me too.  I already proposed a schedule in one of my earlier emails, but 
    got no responses from anyone.  My target milestones are listed in 
    Bugzilla for Kepler.  Any problems with that schedule?
    
    > 
    > Regarding jar files in lib/, yes you're right, we didn't discuss
    > dividing that up by subproject.  Do you see a need to?
    No.
    
    > 
    > This is also true for bin/, which at the moment isn't a problem because
    > we have only two files to put in there: runVergil.sh and runVergil.bat.
    No need for bin.
    
    Matt
    
    > 
    > Xiaowen
    > 
    > On Wed, 2004-03-17 at 15:37, Vouk wrote:
    > 
    >>Xiaowen,
    >>Where does new development go, and stuff in progress?
    >>Programmer X works on item Z. Does not
    >>finish it that day, goes home.
    >>
    >>Normally, I would say, put it into a development
    >>archive (shared nevertheless and put a lock on it),
    >>but do not leave it on your machine, the latter may
    >>not exist in the morning.
    >>
    >>Do you plan to have development directories
    >>(or are we just discussing integrated release level codes?)
    >>
    >>It also looks like the jar and lib files will be a mixture
    >>of project-specific and shared items (according to
    >>the sctructure below). Is that true or am I misreading it?
    >>Same for bins etc.
    >>?
    >>Thanks
    >>Mladen
    > 
    > 
    > 
    > _______________________________________________
    > kepler-dev mailing list
    > kepler-dev at ecoinformatics.org
    > http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
    
    -- 
    -------------------------------------------------------------------
    Matt Jones                                     jones at nceas.ucsb.edu
    http://www.nceas.ucsb.edu/    Fax: 425-920-2439    Ph: 907-789-0496
    National Center for Ecological Analysis and Synthesis (NCEAS)
    University of California Santa Barbara
    Interested in ecological informatics? http://www.ecoinformatics.org
    -------------------------------------------------------------------
    _______________________________________________
    kepler-dev mailing list
    kepler-dev at ecoinformatics.org
    http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
--------