[kepler-dev] Re: Nightly Builds

Matt Jones jones at nceas.ucsb.edu
Thu Mar 18 11:06:54 PST 2004


So, even using "make fast" the build is far slower using "make" than 
"ant".  On my P4 3.06Ghz PII with 1GB ram, these are the stats I get:

[jones at snow ptII3.0.2]$ time make clean fast
real    6m48.208s
user    2m0.610s
sys     0m8.580s

[jones at snow kepler]$ time ant -f build-ptolemy.xml clean install
real    0m23.757s
user    0m16.060s
sys     0m1.160s

Granted, this isn't a totally fair comparison, because the make does 
more stuff than the ant build.  But the ant build does everything 
*needed* in a normal edit/compile/debug cycle, and in this context the 
difference between 23 seconds and over six minutes is huge.  My hunch is 
that make is lauching javac separately for every java file, which incurs 
huge per-file overhead, while ant does its compilation of all java files 
within a single javac invocation.  But that's just a hunch on my part as 
I haven;t looked into it in detail.  And its not just your makefile -- 
my experience has been that compiling java files using make is slow 
under all projects where I've seen it used.

Thanks for your insights into why this might be,

Christopher Hylands Brooks wrote:
> [I changed the Subject line]  
> Matt wrote:
>>[I (Christopher) wrote: ]
>>>* How are nightly builds and tests handled right now?  Forgive my
>>>ignorance on this, it might be apparent from the code, I just have
>>>not looked.
>>We don't have a nightly build.  We should get one :)  I know your 
>>existing makefile permits this.  Somtime I would like to discuss the 
>>Makefile with you and try to figure out why it builds so much more 
>>slowly than the ant build that I wrote.  Then it would make more sense 
>>to use it directly for the nightly build.
> BTW - "cd $PTII; make fast" should be much faster than "cd $PTII; make".
> The $PTII/adm/gen-3.1/makefile is what I use to run the nightly build.
> It is a very gross file, I don't recommend looking at it.
> The reason it is gross is that it works under Cygwin and Solaris, and
> it does multiple builds on remote machines.
> However, what it does is conceptually simple:
> 1. In a directory that has a preexisting Ptolemy II CVS tree, do
>       cd $PTII
>       cvs update -Pd
>       make clean fast
>    and run the tests.
>    In the nightly build, the tests are listed in a strange order to
>    ferret out package dependencies, but in principle
>    (cd $PTII; make tests) should work.
> 2. Build a release by excluding things we don't ship, make a few
>    modifications to configurations (exclude references to things
>    that do not ship)
> 3. Untar the release elsewhere, build (with a different java and make)
>    and run tests.
> 4. Run the very long codegen tests and send email about it to 
>    only a few developers.
> So, no matter how you build, you can still do step 1 above.
> BTW - If you want to quickly build Ptolemy as part of that, you can
> do 'make clean fast'
> The other steps are also not particularly make-centric.
> I strongly encourage you to set up a nightly build, it is a big win.
> Our nightly build is available at
> http://chess.eecs.berkeley.edu/ptexternal/nightly/index.htm
> http://chess.eecs.berkeley.edu/ptexternal/nightly/coverage.html
> is a large table that shows the code coverage for the Java classes.
> BTW - With regard to JUnit vs. Jacl, JUnit is much more of an industry
> wide standard, so going with it is appropriate.
> The one advantage of using Jacl, a 100% Java implementation of a
> subset of Tcl) is that Jacl is interpreted, which makes writing tests
> a little easier.  I find it is easier to create small test fragments
> by typing at a command line prompt and then piece together a larger
> test file.
> There are many more cool JUnit tools out there than there are Jacl
> tools. 
> -Christopher

Matt Jones                                     jones at nceas.ucsb.edu
http://www.nceas.ucsb.edu/    Fax: 425-920-2439    Ph: 907-789-0496
National Center for Ecological Analysis and Synthesis (NCEAS)
University of California Santa Barbara
Interested in ecological informatics? http://www.ecoinformatics.org

More information about the Kepler-dev mailing list