[kepler-users] [Fwd: Re: problem with the URL To Local File actor and a symbol]

Christopher Brooks cxh at eecs.berkeley.edu
Mon Jun 15 12:49:23 PDT 2009 

FYI to Kepler-users.

-------- Original Message --------
Message-ID: <4A36A447.8070206 at eecs.berkeley.edu>
Date: Mon, 15 Jun 2009 12:43:03 -0700
From: Christopher Brooks <cxh at eecs.berkeley.edu>
User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302)
MIME-Version: 1.0
To: Trina Myers <t.myers at bigpond.net.au>
Subject: Re: [kepler-users] problem with the URL To Local File  actor and a symbol
References: <000001c9ec85$51847310$f48d5930$%myers at bigpond.net.au> <4A353096.4010100 at eecs.berkeley.edu> <002601c9ed4b$33187220$99495660$%myers at bigpond.net.au>
In-Reply-To: <002601c9ed4b$33187220$99495660$%myers at bigpond.net.au>
Content-Type: multipart/mixed; boundary="------------040105060009070608070504"

[Apologies about the corrupted message, I'm not sure what happened]

Hi Trina,
Yep, the model I sent fails for me in Kepler-1.0.0.
Yesterday, URLToLocalFile2.xml worked for me in the devel version
of Kepler, today it does not.  I'll look into this.

The devel version can be built with Eclipse, see

https://kepler-project.org/developers/reference/kepler-and-eclipse


Another idea would be to use the StringReplace actor to replace
the | with &7C

There is a description of what encodings are used in URLs at
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm

The attached model uses StringReplace to replace \| with &7C.

_Christopher

Trina Myers wrote:
> Hi Chris,
> 
> Thankyou for you fast reply.
> I am using version 1.0.0 on Windows with java.runtime.version = 1.5.0_12-b04
> 
> I tried the example you sent and there were multiple errors in opening the
> file (below), when it did open I used the URL with | and received the same
> error message as I did with my workflow:
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjIdReferralList">
> org.kepler.moml.NamedObjReferralList
> Error encountered in:<property name="timeResolution"
> class="ptolemy.actor.parameters.SharedParameter" value ="1E-10">
> Error encountered in:<property name=""
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:director:1:1">
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjIdReferralList"
> value="urn:lsid:kepler-project.org:director:1:1">
> Error encountered in:<property name=""
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:actor:252:1">
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:actor:252:1">
> Error encountered in:<property name="title"
> class="ptolemy.data.expr.StringParameter" value=""> Attempt to insert object
> named "title" into a container that already contains an object with that
> name
> 
> java.lang.ClassNotFoundException: org.kepler.moml.NamedObjIdReferralList
> 	at java.net.URLClassLoader$1.run(Unknown Source)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(Unknown Source)
> 	at java.lang.ClassLoader.loadClass(Unknown Source)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> 	at java.lang.ClassLoader.loadClass(Unknown Source)
> 	at java.lang.ClassLoader.loadClassInternal(Unknown Source)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Unknown Source)
> 	at
> ptolemy.moml.MoMLParser._handlePropertyElement(MoMLParser.java:4976)
> 	at ptolemy.moml.MoMLParser.startElement(MoMLParser.java:2784)
> 	at com.microstar.xml.XmlParser.parseElement(XmlParser.java:921)
> 	at com.microstar.xml.XmlParser.parseContent(XmlParser.java:1104)
> 	at com.microstar.xml.XmlParser.parseElement(XmlParser.java:924)
> 	at com.microstar.xml.XmlParser.parseDocument(XmlParser.java:481)
> 	at com.microstar.xml.XmlParser.doParse(XmlParser.java:159)
> 	at com.microstar.xml.XmlParser.parse(XmlParser.java:132)
> 	at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1334)
> 	at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1292)
> 	at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1265)
> 	at
> ptolemy.actor.gui.PtolemyEffigy$Factory.createEffigy(PtolemyEffigy.java:412)
> 	at
> ptolemy.actor.gui.EffigyFactory.createEffigy(EffigyFactory.java:209)
> 	at ptolemy.actor.gui.Configuration.openModel(Configuration.java:595)
> 	at ptolemy.actor.gui.Configuration.openModel(Configuration.java:555)
> 	at ptolemy.actor.gui.TableauFrame._read(TableauFrame.java:893)
> 	at ptolemy.gui.Top._open(Top.java:704)
> 	at ptolemy.gui.Top$FileMenuListener.actionPerformed(Top.java:1045)
> 	at
> ptolemy.vergil.basic.MenuMapper$1.actionPerformed(MenuMapper.java:326)
> 	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
> 	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown
> Source)
> 	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown
> Source)
> 	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
> 	at javax.swing.AbstractButton.doClick(Unknown Source)
> 	at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
> 	at
> javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
> 	at java.awt.Component.processMouseEvent(Unknown Source)
> 	at javax.swing.JComponent.processMouseEvent(Unknown Source)
> 	at java.awt.Component.processEvent(Unknown Source)
> 	at java.awt.Container.processEvent(Unknown Source)
> 	at java.awt.Component.dispatchEventImpl(Unknown Source)
> 	at java.awt.Container.dispatchEventImpl(Unknown Source)
> 	at java.awt.Component.dispatchEvent(Unknown Source)
> 	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
> 	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
> 	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
> 	at java.awt.Container.dispatchEventImpl(Unknown Source)
> 	at java.awt.Window.dispatchEventImpl(Unknown Source)
> 	at java.awt.Component.dispatchEvent(Unknown Source)
> 	at java.awt.EventQueue.dispatchEvent(Unknown Source)
> 	at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown
> Source)
> 	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown
> Source)
> 	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
> 	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
> 	at java.awt.EventDispatchThread.run(Unknown Source)
> 
> I am not a programmer although I understand what you say (I am in a
> different area of expertise). I need to use Kepler for my research but would
> prefer to stay away from the developers version as it would be quite a large
> learning hurdle at this time.  
> 
> I am familiar with the URI, it appears the "|" symbol is "unsafe" in both
> the URL and URI RFC's, is there some way to work around this within the
> URL-to-LocalFile actor (AIMS is not the only organisation that seems to use
> it in their url's)?
> Sincerely
> Trina
> 
> 
> -----Original Message-----
> From: Christopher Brooks [mailto:cxh at eecs.berkeley.edu] 
> Sent: Monday, 15 June 2009 3:17 AM
> To: Trina Myers
> Cc: Kepler-users at ecoinformatics.org
> Subject: Re: [kepler-users] problem with the URL To Local File actor and a
> symbol
> 
> Hi Trina,
> 
> I can't get this to fail for me, what version of Kepler are you using
> (Kepler 1.0.0?  Kepler 1.x from the svn tree?) and what version of Java?
> To get the version of Java: start Kepler and go to Tools -> Check System
> Settings and find the the java.runtime.version property.
> For example, I have:
> java.runtime.version = 1.5.0_16-b06-284
> 
> I've attached a model that works for me in the development version of
> Kepler.  I did not test it in Kepler-1.0.0.
> 
> Does the attached model fail for you?  If not, can you send a test case that
> does fail?  The URI issue is fairly tricky, I have no doubt that there are
> bugs to be found.
> 
> I think the reason the error would occur is because | is probably not
> permitted in URIs.  The relationship between URIs and URLs is tricky.
> http://java.sun.com/j2se/1.5.0/docs/api/java/net/URI.html says:
> "A URI is a uniform resource identifier while a URL is a uniform resource
> locator. Hence every URL is a URI, abstractly speaking, but not
>   every URI is a URL. This is because there is another subcategory of
>   URIs, uniform resource names (URNs), which name resources but do not
>   specify how to locate them. The mailto, news, and isbn URIs shown above
>   are examples of URNs."
> 
> In Java, java.net.URI was introduced after java.net.URL.  The URI class
> gives us certain methods not present in the URL class.
> 
> The reason you would see the error is because | is not permitted in Java's
> URI class.  To replicate this, note that creating a URI fails:
> 
> bash-3.2$ java -classpath $PTII:${PTII}/lib/ptjacl.jar tcl.lang.Shell %
> java::new java.net.URI
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4h
> &dataLevel=level0&bleach=0&vers=2
> java.net.URISyntaxException: Illegal character in query at index 140: 
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4h
> &dataLevel=level0&bleach=0&vers=2
> %
> 
> 
> However, for me, running nameToURL works:
> % set a [java::call ptolemy.util.FileUtilities nameToURL 
> "http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
> h&dataLevel=level0&bleach=0&vers=2" 
> [java::null] [java::null]]
> java0x3
> % $a toString
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4h
> &dataLevel=level0&bleach=0&vers=2
> %
> 
> I tried the above command under Ptolemy II 7.0.1, which is the version
> of Ptolemy used by Kepler 1.0.0.  I also tried it under Java 1.5.0_16
> and Java 1.6.0_07.
> 
> _Christopher
> 
> Trina Myers wrote:
>> Hi All,
>>
>> I have a workflow that extracts xml files from a servlet and transforms
> the
>> data within using the XPATH actor. It has been working very well until I
>> have tried to import 2 elements from the site. The only difference between
>> the URL in question and the working URL is the use of "|" (separating
>> max|avg). 
>> If I download the xml file with both elements and use the absolute address
>> to the file on the local machine the workflow works fine (extracting both
>> the avg and max data as required), so I am sure the problem is at the URL
> to
>> local file actor.
>>
>> I would appreciate any guidance
>> Cheers
>> Trina
>>
>> The error message:
>>
>> ptolemy.kernel.util.IllegalActionException: Cannot read file
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2'
>>   in .test_url.URL To Local File.fileOrURL
>> Because:
>> Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hypothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> 	at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:216)
>> 	at util.URLToLocalFile._openAndReadBytes(URLToLocalFile.java:226)
>> 	at util.URLToLocalFile.fire(URLToLocalFile.java:166)
>> 	at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:398)
>> 	at
>>
> ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDirector.j
>> ava:170)
>> 	at ptolemy.actor.CompositeActor.fire(CompositeActor.java:400)
>> 	at ptolemy.actor.Manager.iterate(Manager.java:688)
>> 	at ptolemy.actor.Manager.execute(Manager.java:332)
>> 	at ptolemy.actor.Manager.run(Manager.java:1071)
>> 	at ptolemy.actor.Manager$3.run(Manager.java:1112)
>> Caused by: java.io.IOException: Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hypothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> 	at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:356)
>> 	at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:213)
>> 	... 9 more
>> Caused by: java.lang.IllegalArgumentException
>> 	at java.net.URI.create(Unknown Source)
>> 	at java.net.URI.resolve(Unknown Source)
>> 	at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:353)
>> 	... 10 more
>> Caused by: java.net.URISyntaxException: Illegal character in query at
> index
>> 140:
>>
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4h
>> &dataLevel=level0&bleach=0&vers=2
>> 	at java.net.URI$Parser.fail(Unknown Source)
>> 	at java.net.URI$Parser.checkChars(Unknown Source)
>> 	at java.net.URI$Parser.parseHierarchical(Unknown Source)
>> 	at java.net.URI$Parser.parse(Unknown Source)
>> 	at java.net.URI.<init>(Unknown Source)
>> 	... 13 more
>> Caused by: java.io.IOException: Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hypothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> 	at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:356)
>> 	at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:213)
>> 	at util.URLToLocalFile._openAndReadBytes(URLToLocalFile.java:226)
>> 	at util.URLToLocalFile.fire(URLToLocalFile.java:166)
>> 	at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:398)
>> 	at
>>
> ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDirector.j
>> ava:170)
>> 	at ptolemy.actor.CompositeActor.fire(CompositeActor.java:400)
>> 	at ptolemy.actor.Manager.iterate(Manager.java:688)
>> 	at ptolemy.actor.Manager.execute(Manager.java:332)
>> 	at ptolemy.actor.Manager.run(Manager.java:1071)
>> 	at ptolemy.actor.Manager$3.run(Manager.java:1112)
>> Caused by: java.lang.IllegalArgumentException
>> 	at java.net.URI.create(Unknown Source)
>> 	at java.net.URI.resolve(Unknown Source)
>> 	at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:353)
>> 	... 10 more
>> Caused by: java.net.URISyntaxException: Illegal character in query at
> index
>> 140:
>>
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&vars=4h
>> &dataLevel=level0&bleach=0&vers=2
>> 	at java.net.URI$Parser.fail(Unknown Source)
>> 	at java.net.URI$Parser.checkChars(Unknown Source)
>> 	at java.net.URI$Parser.parseHierarchical(Unknown Source)
>> 	at java.net.URI$Parser.parse(Unknown Source)
>> 	at java.net.URI.<init>(Unknown Source)
>> 	... 13 more
>>
>> _______________________________________________
>> Kepler-users mailing list
>> Kepler-users at kepler-project.org
>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
> 

-- 
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director                      US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480


-- 
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director                      US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
-------------- next part --------------
A non-text attachment was scrubbed...
Name: URLToLocalFileTest3.xml
Type: text/xml
Size: 36909 bytes
Desc: not available
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20090615/02906e9e/attachment.xml>

More information about the Kepler-users mailing list