[kepler-dev] using svn:externals for linking repositories

[David Welker]

Hi Matt,

I think that this is a very interesting proposal which should be  
contrasted to another approach.

If we decide to do Maven-style dependency management (which seems very  
likely, given the advantages) that is supported by Maven, Ant via Ivy,  
and Buildr, it may be a good idea to just make Ptolemy another  
dependency. That is, we could have versions of the Ptolemy that we  
depend on packaged as jars and stored in our Maven repository. Then,  
our dependency on Ptolemy would be specified just like any other  
dependency.

This has the following advantages:

(1) Treating Ptolemy just like any other dependency is simpler to  
understand and maintain. Taking a unified approach is better than  
taking heterogeneous approaches, each of which will have to be studied  
by those developers who need to understand what is going.

(2) It does not couple our Kepler svn repository to the Ptolemy svn  
repository. We would not need to require or even care whether Ptolemy  
uses a single URL or multiple URLs for read and write access, for  
example.

(3) There would be less configuration and maintenance trying to  
synchronize our Kepler tree with the Ptolemy tree using svn:externals.

(4) It is much less extra work, assuming that we are going to do Maven- 
style dependency management anyway.

(5) It is easy enough for Ptolemy to publish multiple versions of the  
code we require into our Maven repository. All of the major existing  
build technologies that are under consideration, including Ant + Ivy,  
Maven, and Buildr support making releases to our Maven repository  
where it will be accessible to anyone who needs it quite easily. In  
Buildr, uploading a new version of Ptolemy would be as easy as  
changing the version number, and typing the command "buildr upload."

(6) This makes it quite easy for a developer to depend on precisely  
whatever version of Ptolemy they like. If they want to use a different  
version of Ptolemy, it is as simple as changing the version number in  
whatever build tool we decide to go with. As long as that version of  
Ptolemy is in our local Maven repository, it will be available and be  
used when it is specified. It does not get much easier to change the  
version of Ptolemy one is using than to change 1 number in 1 file.

(7) Ptolemy does not need to be downloaded if it is already on a local  
machine.

(8) If we ever decided to migrate to another SCM system (it could  
happen, someday), we would not need to reconfigure the relationship  
between Kepler and Ptolemy in our SCM. If, for some reason, Ptolemy  
decided to go back to SCCS (its not going to happen, this is purely  
hypothetical), we could live with that as long as they continued to  
publish new versions of Ptolemy into our Maven repository.

My tentative view is that the advantages of integrating Ptolemy into  
our Maven repository are far greater than the advantages of doing this  
dependency management with svn:externals, which is both more  
complicated and more fragile.

-David


On Jun 12, 2008, at 2:35 AM, Matthew Jones wrote:

> Hi,
>
> I wanted to mention one of the advantages that we might utilize to  
> make
> it easier for people to build Kepler along with its ptolemy
> dependencies.  In SVN, one can use the svn:externals property to  
> define
> external dependencies that should be retrieved in a given directory  
> when
> checking out a repository.  This means we can define an external
> dependency within the Kepler directory on the ptolemy svn tree, and it
> will be automatically updated when one updates Kepler.  For more
> information on svn:externals, see the SVN Book:
>   http://svnbook.red-bean.com/en/1.4/svn-book.html#svn.advanced.externals
>
>
> Here's an example of how its done:
>
> $ pwd
> /Users/jones/development/kepler/trunk
> $ svn propset svn:externals "ptII/docs/uml
> svn://source.eecs.berkeley.edu/chess/ptII/trunk/doc/uml" .
> property 'svn:externals' set on '.'
> $ svn propget svn:externals .ptII/docs/uml
> svn://source.eecs.berkeley.edu/chess/ptII/trunk/doc/uml
>
> $ svn update
>
> Fetching external item into 'ptII/docs/uml'
> External at revision 49784.
>
> At revision 7309.
> $
>
> In the previous example, I created an external dependency on the
> ptII/trunk/doc/uml directory, and told svn to retrieve its contents  
> and
> place them in the relative directory "ptII/docs/uml".  Then I  
> displayed
> the property using the propget command, and then I updated showing the
> external checkout.  Something analogous could be done for the whole
> ptolemy tree, and then the kepler build.xml file could be modified  
> to be
> aware of the presence of the ptII tree inside the kepler checkout.
>
> If we were to include such an external dependency, then nobody would
> have to check out ptolemy separately from kepler, which would of  
> course
> make the default build simpler.  It would also simplify the current
> setup in Eclipse (which currently requires creating a link to  
> ptolemy in
> a separate checkout), because the ptolemy directory would be a
> subdirectory of kepler.  Thus, the default .classpath and .project  
> files
> could be shipped to work immediately after the kepler checkout in  
> Eclipse.
>
> There are some disadvantages to consider:
>
> 1) Because ptolemy is using a different URL for anonymous access (svn)
> and write access (svn+ssh), the svn:externals link will not work for
> people who want to write into the ptolemy tree if we default to the
> anonymous access.  Probably not a big deal (the few people with write
> access can have a separate checkout for ptII for writing), but also
> easily solved if ptolemy used a single access url for its repository
> (the kepler SVN url can be checked out anonymously, but also can be  
> used
> for writing, in which case a valid username and password are solicited
> required over an SSL connection).
>
> 2) It becomes somewhat more difficult to build against multiple
> different versions of ptolemy, as 1) we'd no longer use the PTII
> environment variable (which we've been trying to eliminate for a long
> time anyway), and 2) the svn:externals is always either linked to the
> HEAD of the directory or to a specific version, so its hard for a user
> to dynamically choose a different revision. Presumably we could modify
> the build to use the PTII environment variable if it is defined, and  
> if
> not use the svn:external link, which might give us the best of both  
> worlds.
>
> 3) There's no way to selectively exclude the svn:externals link, so
> everyone who checks out kepler will get a copy of ptolemy, even if  
> they
> already have one elsewhere on disk.  We'd be trading some storage
> redundancy for people who might already have ptII against ease of use
> for newcomers to the project who need to know less to get started  
> building.
>
> That's about it.  I'll be contemplating these build changes some more
> and then making a concrete proposal -- this is just a heads up for
> background. Comments appreciated.
>
> Cheers,
> Matt
> -- 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Matthew B. Jones
> Director of Informatics Research and Development
> National Center for Ecological Analysis and Synthesis (NCEAS)
> UC Santa Barbara
> jones at nceas.ucsb.edu                       Ph: 1-907-523-1960
> http://www.nceas.ucsb.edu/ecoinfo
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev