[kepler-dev] Duplicated jar contents.

Dan Higgins higgins at nceas.ucsb.edu
Wed Dec 14 11:02:09 PST 2005


Hi All,
    Some more comments for the discussion:

    We have a real mess with versioning here that has broken the CVS 
build several times. Deveopers add jars needed for their part of Kepler 
and get their actors, etc.. working without knowing that this 'breaks' 
something else. This is possible because their new jars happen to get 
loaded into the class path in the right order for their use (but no 
necessarily for others).
    But simply removing older jars can be tricky also, especially if the 
jars do not have identical classes. If there are additional classes in a 
jar  further along the classpath, then those will be loaded even when 
other classes (that are duplicated) are loaded from a jar ahead of it in 
the classpath.
    In my opinion, a primary reason for the change to the KAR packages 
was to solve this problem by not loading everything with the default 
classloader. Unfortunately, the custom classloader part of the KAR 
system has been pushed off to after the 1.0 release! To solve this 
problem, we might want to reconsider that decision. The other 
possibility is to let the actor developer take responsibility for 
creatieng a custom classloader. If your actor requires certain versions 
of classes, then load them dynamically using a custom classloader that 
avoids classes in the default classpath!

Dan

---

"Jenny" Guilian Wang wrote:

>Hi,
>
>I agree with Shawn. I know that at least Jena uses a newer version of
>xerces that is not compatible with the old xerces that Kweelt
>requires. So I said, I am going to try the overlapping jars. If
>another jar works for SCIA, I will get rid of the corresponding jar in
>SCIA jar dir. I don't expect it is safe to simply remove them now,
>instead I assume Kevin would just exclude the SCIA jars from build.
>
>Thanks,
>
>Jenny
>
>On 12/14/05, Shawn Bowers <sbowers at ucdavis.edu> wrote:
>  
>
>>I think this is a bad idea ...
>>
>>It seems like there is some underlying assumption that
>>sms, scia, or any other kepler package is sticking in
>>isolated jars, and using them independently.  However,
>>the reality is that most Java packages (e.g., jena and
>>many of the other packages that do xml processing) rely
>>on many other jars, and specific versions of these.
>>
>>You can't just start removing "old" jars because not all of
>>these are backwards compatible!  My fear is that using
>>this motivation is going to cause many problems.
>>
>>
>>-shawn
>>
>>Matt Jones wrote:
>>    
>>
>>>Hi Kevin,
>>>
>>>Yeah, just use your judgement on the xml jars (as long as part of your
>>>judgement values the "everything still works" principle).  Changing this
>>>stuff after alpha8 this Friday is probably best.
>>>
>>>I will move the jxta stuff to a branch to reduce the cruft.
>>>
>>>Matt
>>>
>>>Kevin Ruland wrote:
>>>      
>>>
>>>>Sorry that came originally from my sunflower account.  Thunderbird
>>>>doesn't always read my mind correctly and use the right return addr --
>>>>and I'm too lazy to look all the time.
>>>>
>>>>Matt:
>>>>
>>>>I won't make any changes until after the release this week.  In fact, I
>>>>shouldn't even have removed the outdated log4j or removed the dead code
>>>>from Util.java.  I have a little too much 'Alpha' zeal right now.
>>>>
>>>>So after the release,  should I just use my judgement for the xml
>>>>related jars and the obvious duplicates?
>>>>
>>>>We will also have a headache when it comes to Zhijie's new jars which I
>>>>think will also wait.
>>>>
>>>>Kevin
>>>>
>>>>Matt Jones wrote:
>>>>
>>>>        
>>>>
>>>>>Thanks Kevin.
>>>>>
>>>>>Contact Jenny Wang about the scia jar issues.  Scia probably won't
>>>>>ship in the 1.0.0 release unless a lot of work is done before the
>>>>>feature freeze (incuding integration with Shawn's SMS work), so we
>>>>>might consider excluding the SCIA jar and src in the build.xml targets
>>>>>until it is ready for primetime.  We want it in the HEAD so Jenny can
>>>>>keep working on it, but maybe it should have its own build targets.
>>>>>
>>>>>Contact Shawn Bowers about the SMS jars and any conflicts there.  Chad
>>>>>might know a little about it too.
>>>>>
>>>>>Matt
>>>>>
>>>>>Kevin Ruland wrote:
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>>>I started this task again.  I used Dan's new autobat ant task to
>>>>>>generate the classpath to analyze.  This report is only those jars on
>>>>>>the classpath which contain the same classes.  I've attempted to group
>>>>>>them by higher level packages to make it easier to look at.
>>>>>>
>>>>>>The plus sign ('+') in front of a particular jar indicates which one is
>>>>>>used to load the classes in question.  So, if the system appears to work
>>>>>>when you use run-dev, then we can probably just drop the extras.
>>>>>>
>>>>>>I did not attempt to accomodate the "endorsed jars" thing.  I think we
>>>>>>should create a directory kepler/lib/jars/endorsed/ and put there the
>>>>>>jars which need endorsing.  These can then be used for building and
>>>>>>provide a convient place to point to.
>>>>>>
>>>>>>There are two broad categories of conflicts:
>>>>>>
>>>>>>1)  XML and related processing.
>>>>>>
>>>>>>There are some intracies here.  We need to have consistent versions of
>>>>>>axis, xerces, xalan, and xml-apis.   In addition, 1, 2, or 3 of these
>>>>>>will need to be endorsed.  axis uses saaj.jar, jaxrpc.jar, etc.
>>>>>>
>>>>>>I suggest we grab a known versions of axis, xalan, xerces and their
>>>>>>supporting jars and remove all the current jars.  We should in addition
>>>>>>document the versions of each we are using.
>>>>>>
>>>>>>2) lib/jars/scia/* and lib/jars/sms/*
>>>>>>
>>>>>>There appears to be considerable overlap among the jars in these
>>>>>>directories.  There are two jars which are particularly troubling.
>>>>>>
>>>>>>- scia/sf_org.jar appears to be a conglomeration of a number of things
>>>>>>including:  org.xml.sax (which should be in xml-apis.jar, is in rt.jar
>>>>>>and must be endorsed), org.w3c.rdf, and org.brownell.jar.
>>>>>>
>>>>>>- sms/rdf-api-2001-01-19.jar has similar problems since it also has
>>>>>>org.xml.sax, and overlaps with sf_edu.jar and sf_org.jar from scia
>>>>>>
>>>>>>Kevin
>>>>>>
>>>>>>
>>>>>>javax.xml.namespace.QName
>>>>>>
>>>>>> lib/jar/jax-qname.jar
>>>>>> + lib/jar/jaxrpc.jar
>>>>>> lib/jar/jwsdp/jsr173_api.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>javax.xml.namespaceNamespaceContext
>>>>>>javax.xml.XMLConstants
>>>>>>
>>>>>> lib/jar/namespace.jar
>>>>>> + lib/jar/jwsdp/jsr173_api.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>javax.xml.transform.*
>>>>>>
>>>>>> lib/jar/jaxrpc.jar
>>>>>> lib/jar/apache/xml-apis.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>javax.xml.soap.*
>>>>>>
>>>>>> lib/jar/saaj.jar
>>>>>> + lib/jar/jwsdp/saaj-api.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>org.jdom.*
>>>>>>
>>>>>> lib/jar/scia/jdom.jar
>>>>>> + lib/jar/jdom.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>com.mysql.jdbc.*
>>>>>>org.gjt.mm.mysql.Driver
>>>>>>
>>>>>> lib/jar/scia/mysql-connector-java-3.0.16-ga-bin.jar
>>>>>> + lib/jar/dbdrivers/mysql-connector-java-3.0.15-ga.bin.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>org.xml.sax.*
>>>>>>
>>>>>> lib/jar/scia/sf_org.jar
>>>>>> + lib/jar/apache/xml-apis.jar
>>>>>> lib/jar/sms/rdf-api-2001-01-19.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>javax.xml.parsers.*
>>>>>>javax.xml.transform.*
>>>>>>
>>>>>> lib/jar/scia/xml-apis.jar
>>>>>> + lib/jar/apache/xml-apis.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>org.apache.commons.logging.*
>>>>>>
>>>>>> lib/jar/sms/commons-logging.jar
>>>>>> + lib/jar/commons-logging.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>edu.stanford.db.*
>>>>>>
>>>>>> lib/jar/sms/rdf-api-2001-01-19.jar
>>>>>> + lib/jar/scia/sf_edu.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>org.w3c.rdf.*
>>>>>>org.brownell.xml.*
>>>>>>
>>>>>> lib/jar/sms/rdf-api-2001-01-19.jar
>>>>>> + lib/jar/scia/sf_org.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>org.apache.html.dom.*
>>>>>>org.apache.wml.dom.*
>>>>>>org.apache.xerces.*
>>>>>>org.w3c.dom.*
>>>>>>
>>>>>> lib/jar/sms/xercesImpl.jar
>>>>>> + lib/jar/scia/xercesImpl.jar
>>>>>>
>>>>>>=================================================
>>>>>>
>>>>>>Special category:  jars which have classes also contained
>>>>>>in j2sdk1.4.2_08 rt.jar
>>>>>>
>>>>>> lib/jar/jce-jdk13-117.jar (includes
>>>>>>java.security.spec.PSSParameterSpec)
>>>>>> lib/jar/dbdrivers/oracleJdbc.jar (includes javax.sql.*)
>>>>>> lib/jar/apache/xml-apis.jar (includes javax.xml.*, org.w3c.dom.*)
>>>>>> lib/jar/apache/xalan.jar (includes org.apache.xml.*,
>>>>>>org.apache.xalan.*, org.apache.xpath.*)
>>>>>> lib/jar/jgss.jar (includes org.ietf.jgss.*
>>>>>>
>>>>>>_______________________________________________
>>>>>>Kepler-dev mailing list
>>>>>>Kepler-dev at ecoinformatics.org
>>>>>>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>>>>>>
>>>>>>            
>>>>>>
>>>>>
>>>>>          
>>>>>
>>_______________________________________________
>>Kepler-dev mailing list
>>Kepler-dev at ecoinformatics.org
>>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>>
>>    
>>
>_______________________________________________
>Kepler-dev mailing list
>Kepler-dev at ecoinformatics.org
>http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>  
>




More information about the Kepler-dev mailing list