[kepler-dev] Re: Nightly Builds

[Stephen Andrew Neuendorffer]

An interesting epistle on the downsides of recursive make:
http://aegis.sourceforge.net/auug97.pdf

This paper claims that you can get the same behavior as recursive make 
using includes and local build rules.
I think the javac thing is still a problem though, because you can overrun 
the command line doing javac *.java

Steve

At 11:11 AM 3/18/2004 -0800, Stephen Andrew Neuendorffer wrote:

>make fast basically runs one javac for each directory...  But you still 
>have a javac process and a recursive make process for every directory.
>Given a large number of directories, that's still alot of processes...
>
>At 10:06 AM 3/18/2004 -0900, Matt Jones wrote:
>>Christopher,
>>
>>So, even using "make fast" the build is far slower using "make" than 
>>"ant".  On my P4 3.06Ghz PII with 1GB ram, these are the stats I get:
>>
>>[jones at snow ptII3.0.2]$ time make clean fast
>>real    6m48.208s
>>user    2m0.610s
>>sys     0m8.580s
>>
>>[jones at snow kepler]$ time ant -f build-ptolemy.xml clean install
>>real    0m23.757s
>>user    0m16.060s
>>sys     0m1.160s
>>
>>Granted, this isn't a totally fair comparison, because the make does more 
>>stuff than the ant build.  But the ant build does everything *needed* in 
>>a normal edit/compile/debug cycle, and in this context the difference 
>>between 23 seconds and over six minutes is huge.  My hunch is that make 
>>is lauching javac separately for every java file, which incurs huge 
>>per-file overhead, while ant does its compilation of all java files 
>>within a single javac invocation.  But that's just a hunch on my part as 
>>I haven;t looked into it in detail.  And its not just your makefile -- my 
>>experience has been that compiling java files using make is slow under 
>>all projects where I've seen it used.
>>
>>Thanks for your insights into why this might be,
>>Matt
>>
>>Christopher Hylands Brooks wrote:
>>>[I changed the Subject line]
>>>Matt wrote:
>>>
>>>>[I (Christopher) wrote: ]
>>>>
>>>>>* How are nightly builds and tests handled right now?  Forgive my
>>>>>ignorance on this, it might be apparent from the code, I just have
>>>>>not looked.
>>>>
>>>>We don't have a nightly build.  We should get one :)  I know your 
>>>>existing makefile permits this.  Somtime I would like to discuss the 
>>>>Makefile with you and try to figure out why it builds so much more 
>>>>slowly than the ant build that I wrote.  Then it would make more sense 
>>>>to use it directly for the nightly build.
>>>
>>>BTW - "cd $PTII; make fast" should be much faster than "cd $PTII; make".
>>>The $PTII/adm/gen-3.1/makefile is what I use to run the nightly build.
>>>It is a very gross file, I don't recommend looking at it.
>>>The reason it is gross is that it works under Cygwin and Solaris, and
>>>it does multiple builds on remote machines.
>>>However, what it does is conceptually simple:
>>>1. In a directory that has a preexisting Ptolemy II CVS tree, do
>>>       cd $PTII
>>>       cvs update -Pd
>>>       make clean fast
>>>    and run the tests.
>>>    In the nightly build, the tests are listed in a strange order to
>>>    ferret out package dependencies, but in principle
>>>    (cd $PTII; make tests) should work.
>>>2. Build a release by excluding things we don't ship, make a few
>>>    modifications to configurations (exclude references to things
>>>    that do not ship)
>>>3. Untar the release elsewhere, build (with a different java and make)
>>>    and run tests.
>>>4. Run the very long codegen tests and send email about it to    only a 
>>>few developers.
>>>
>>>So, no matter how you build, you can still do step 1 above.
>>>BTW - If you want to quickly build Ptolemy as part of that, you can
>>>do 'make clean fast'
>>>The other steps are also not particularly make-centric.
>>>I strongly encourage you to set up a nightly build, it is a big win.
>>>Our nightly build is available at
>>>http://chess.eecs.berkeley.edu/ptexternal/nightly/index.htm
>>>http://chess.eecs.berkeley.edu/ptexternal/nightly/coverage.html
>>>is a large table that shows the code coverage for the Java classes.
>>>
>>>BTW - With regard to JUnit vs. Jacl, JUnit is much more of an industry
>>>wide standard, so going with it is appropriate.
>>>The one advantage of using Jacl, a 100% Java implementation of a
>>>subset of Tcl) is that Jacl is interpreted, which makes writing tests
>>>a little easier.  I find it is easier to create small test fragments
>>>by typing at a command line prompt and then piece together a larger
>>>test file.
>>>There are many more cool JUnit tools out there than there are Jacl
>>>tools.
>>>-Christopher
>>
>>--
>>-------------------------------------------------------------------
>>Matt Jones                                     jones at nceas.ucsb.edu
>>http://www.nceas.ucsb.edu/    Fax: 425-920-2439    Ph: 907-789-0496
>>National Center for Ecological Analysis and Synthesis (NCEAS)
>>University of California Santa Barbara
>>Interested in ecological informatics? http://www.ecoinformatics.org
>>-------------------------------------------------------------------
>>_______________________________________________
>>kepler-dev mailing list
>>kepler-dev at ecoinformatics.org
>>http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
>
>
>_______________________________________________
>kepler-dev mailing list
>kepler-dev at ecoinformatics.org
>http://www.ecoinformatics.org/mailman/listinfo/kepler-dev