[kepler-users] problem with the URL To Local File actor and a symbol
Trina Myers
t.myers at bigpond.net.au
Mon Jun 15 20:13:10 PDT 2009
Thank you for your help with this Chris, the links are very handy!
I had been thinking of using the String replace actor but was not sure what
to replace with and whether it would interfere with the URL interpretation.
It appears at first to work, however, when extracting the elements required,
the URLtoLocal actor only sees the text before the newly replaced 'unsafe'
symbol. I have attached a modification of your example to illustrate. I wish
to extract all temperature elements (average and max) from the xml document
(the URL link) using the XPath actor. Once running the string replace, the
URL to Local actor only downloads the 'max' temperature.
Regards
Trina
-----Original Message-----
From: kepler-users-bounces at kepler-project.org
[mailto:kepler-users-bounces at kepler-project.org] On Behalf Of Christopher
Brooks
Sent: Tuesday, 16 June 2009 5:49 AM
To: KEPLER USERS
Subject: [kepler-users] [Fwd: Re: problem with the URL To Local File actor
and a symbol]
FYI to Kepler-users.
-------- Original Message --------
Message-ID: <4A36A447.8070206 at eecs.berkeley.edu>
Date: Mon, 15 Jun 2009 12:43:03 -0700
From: Christopher Brooks <cxh at eecs.berkeley.edu>
User-Agent: Thunderbird 2.0.0.21 (Macintosh/20090302)
MIME-Version: 1.0
To: Trina Myers <t.myers at bigpond.net.au>
Subject: Re: [kepler-users] problem with the URL To Local File actor and a
symbol
References: <000001c9ec85$51847310$f48d5930$%myers at bigpond.net.au>
<4A353096.4010100 at eecs.berkeley.edu>
<002601c9ed4b$33187220$99495660$%myers at bigpond.net.au>
In-Reply-To: <002601c9ed4b$33187220$99495660$%myers at bigpond.net.au>
Content-Type: multipart/mixed;
boundary="------------040105060009070608070504"
[Apologies about the corrupted message, I'm not sure what happened]
Hi Trina,
Yep, the model I sent fails for me in Kepler-1.0.0.
Yesterday, URLToLocalFile2.xml worked for me in the devel version of Kepler,
today it does not. I'll look into this.
The devel version can be built with Eclipse, see
https://kepler-project.org/developers/reference/kepler-and-eclipse
Another idea would be to use the StringReplace actor to replace the | with
&7C
There is a description of what encodings are used in URLs at
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
The attached model uses StringReplace to replace \| with &7C.
_Christopher
Trina Myers wrote:
> Hi Chris,
>
> Thankyou for you fast reply.
> I am using version 1.0.0 on Windows with java.runtime.version =
> 1.5.0_12-b04
>
> I tried the example you sent and there were multiple errors in opening
> the file (below), when it did open I used the URL with | and received
> the same error message as I did with my workflow:
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjIdReferralList">
> org.kepler.moml.NamedObjReferralList
> Error encountered in:<property name="timeResolution"
> class="ptolemy.actor.parameters.SharedParameter" value ="1E-10"> Error
> encountered in:<property name=""
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:director:1:1">
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjIdReferralList"
> value="urn:lsid:kepler-project.org:director:1:1">
> Error encountered in:<property name=""
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:actor:252:1">
> Error encountered in:<property name="derivedFrom"
> class="org.kepler.moml.NamedObjReferralList"
> value="urn:lsid:kepler-project.org:actor:252:1">
> Error encountered in:<property name="title"
> class="ptolemy.data.expr.StringParameter" value=""> Attempt to insert
> object named "title" into a container that already contains an object
> with that name
>
> java.lang.ClassNotFoundException: org.kepler.moml.NamedObjIdReferralList
> at java.net.URLClassLoader$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
> at java.lang.ClassLoader.loadClass(Unknown Source)
> at java.lang.ClassLoader.loadClassInternal(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Unknown Source)
> at
> ptolemy.moml.MoMLParser._handlePropertyElement(MoMLParser.java:4976)
> at ptolemy.moml.MoMLParser.startElement(MoMLParser.java:2784)
> at com.microstar.xml.XmlParser.parseElement(XmlParser.java:921)
> at com.microstar.xml.XmlParser.parseContent(XmlParser.java:1104)
> at com.microstar.xml.XmlParser.parseElement(XmlParser.java:924)
> at com.microstar.xml.XmlParser.parseDocument(XmlParser.java:481)
> at com.microstar.xml.XmlParser.doParse(XmlParser.java:159)
> at com.microstar.xml.XmlParser.parse(XmlParser.java:132)
> at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1334)
> at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1292)
> at ptolemy.moml.MoMLParser.parse(MoMLParser.java:1265)
> at
>
ptolemy.actor.gui.PtolemyEffigy$Factory.createEffigy(PtolemyEffigy.java:412)
> at
> ptolemy.actor.gui.EffigyFactory.createEffigy(EffigyFactory.java:209)
> at ptolemy.actor.gui.Configuration.openModel(Configuration.java:595)
> at ptolemy.actor.gui.Configuration.openModel(Configuration.java:555)
> at ptolemy.actor.gui.TableauFrame._read(TableauFrame.java:893)
> at ptolemy.gui.Top._open(Top.java:704)
> at ptolemy.gui.Top$FileMenuListener.actionPerformed(Top.java:1045)
> at
> ptolemy.vergil.basic.MenuMapper$1.actionPerformed(MenuMapper.java:326)
> at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
> at javax.swing.AbstractButton$Handler.actionPerformed(Unknown
> Source)
> at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown
> Source)
> at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
> at javax.swing.AbstractButton.doClick(Unknown Source)
> at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
> at
> javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown
Source)
> at java.awt.Component.processMouseEvent(Unknown Source)
> at javax.swing.JComponent.processMouseEvent(Unknown Source)
> at java.awt.Component.processEvent(Unknown Source)
> at java.awt.Container.processEvent(Unknown Source)
> at java.awt.Component.dispatchEventImpl(Unknown Source)
> at java.awt.Container.dispatchEventImpl(Unknown Source)
> at java.awt.Component.dispatchEvent(Unknown Source)
> at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
> at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
> at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
> at java.awt.Container.dispatchEventImpl(Unknown Source)
> at java.awt.Window.dispatchEventImpl(Unknown Source)
> at java.awt.Component.dispatchEvent(Unknown Source)
> at java.awt.EventQueue.dispatchEvent(Unknown Source)
> at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown
> Source)
> at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown
> Source)
> at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
> at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
> at java.awt.EventDispatchThread.run(Unknown Source)
>
> I am not a programmer although I understand what you say (I am in a
> different area of expertise). I need to use Kepler for my research but
> would prefer to stay away from the developers version as it would be
> quite a large learning hurdle at this time.
>
> I am familiar with the URI, it appears the "|" symbol is "unsafe" in
> both the URL and URI RFC's, is there some way to work around this
> within the URL-to-LocalFile actor (AIMS is not the only organisation
> that seems to use it in their url's)?
> Sincerely
> Trina
>
>
> -----Original Message-----
> From: Christopher Brooks [mailto:cxh at eecs.berkeley.edu]
> Sent: Monday, 15 June 2009 3:17 AM
> To: Trina Myers
> Cc: Kepler-users at ecoinformatics.org
> Subject: Re: [kepler-users] problem with the URL To Local File actor
> and a symbol
>
> Hi Trina,
>
> I can't get this to fail for me, what version of Kepler are you using
> (Kepler 1.0.0? Kepler 1.x from the svn tree?) and what version of Java?
> To get the version of Java: start Kepler and go to Tools -> Check
> System Settings and find the the java.runtime.version property.
> For example, I have:
> java.runtime.version = 1.5.0_16-b06-284
>
> I've attached a model that works for me in the development version of
> Kepler. I did not test it in Kepler-1.0.0.
>
> Does the attached model fail for you? If not, can you send a test
> case that does fail? The URI issue is fairly tricky, I have no doubt
> that there are bugs to be found.
>
> I think the reason the error would occur is because | is probably not
> permitted in URIs. The relationship between URIs and URLs is tricky.
> http://java.sun.com/j2se/1.5.0/docs/api/java/net/URI.html says:
> "A URI is a uniform resource identifier while a URL is a uniform
> resource locator. Hence every URL is a URI, abstractly speaking, but not
> every URI is a URL. This is because there is another subcategory of
> URIs, uniform resource names (URNs), which name resources but do not
> specify how to locate them. The mailto, news, and isbn URIs shown above
> are examples of URNs."
>
> In Java, java.net.URI was introduced after java.net.URL. The URI
> class gives us certain methods not present in the URL class.
>
> The reason you would see the error is because | is not permitted in
> Java's URI class. To replicate this, note that creating a URI fails:
>
> bash-3.2$ java -classpath $PTII:${PTII}/lib/ptjacl.jar tcl.lang.Shell
> % java::new java.net.URI
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:
> 00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&v
> ars=4h
> &dataLevel=level0&bleach=0&vers=2
> java.net.URISyntaxException: Illegal character in query at index 140:
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:
> 00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&v
> ars=4h
> &dataLevel=level0&bleach=0&vers=2
> %
>
>
> However, for me, running nameToURL works:
> % set a [java::call ptolemy.util.FileUtilities nameToURL
> "http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
> h&dataLevel=level0&bleach=0&vers=2"
> [java::null] [java::null]]
> java0x3
> % $a toString
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:
> 00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&v
> ars=4h
> &dataLevel=level0&bleach=0&vers=2
> %
>
> I tried the above command under Ptolemy II 7.0.1, which is the version
> of Ptolemy used by Kepler 1.0.0. I also tried it under Java 1.5.0_16
> and Java 1.6.0_07.
>
> _Christopher
>
> Trina Myers wrote:
>> Hi All,
>>
>> I have a workflow that extracts xml files from a servlet and
>> transforms
> the
>> data within using the XPATH actor. It has been working very well
>> until I have tried to import 2 elements from the site. The only
>> difference between the URL in question and the working URL is the use
>> of "|" (separating
>> max|avg).
>> If I download the xml file with both elements and use the absolute
>> address to the file on the local machine the workflow works fine
>> (extracting both the avg and max data as required), so I am sure the
>> problem is at the URL
> to
>> local file actor.
>>
>> I would appreciate any guidance
>> Cheers
>> Trina
>>
>> The error message:
>>
>> ptolemy.kernel.util.IllegalActionException: Cannot read file
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2'
>> in .test_url.URL To Local File.fileOrURL
>> Because:
>> Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name
>> is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hy
> pothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:216)
>> at util.URLToLocalFile._openAndReadBytes(URLToLocalFile.java:226)
>> at util.URLToLocalFile.fire(URLToLocalFile.java:166)
>> at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:398)
>> at
>>
> ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDire
> ctor.j
>> ava:170)
>> at ptolemy.actor.CompositeActor.fire(CompositeActor.java:400)
>> at ptolemy.actor.Manager.iterate(Manager.java:688)
>> at ptolemy.actor.Manager.execute(Manager.java:332)
>> at ptolemy.actor.Manager.run(Manager.java:1071)
>> at ptolemy.actor.Manager$3.run(Manager.java:1112)
>> Caused by: java.io.IOException: Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name
>> is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hy
> pothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:356)
>> at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:213)
>> ... 9 more
>> Caused by: java.lang.IllegalArgumentException
>> at java.net.URI.create(Unknown Source)
>> at java.net.URI.resolve(Unknown Source)
>> at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:353)
>> ... 10 more
>> Caused by: java.net.URISyntaxException: Illegal character in query at
> index
>> 140:
>>
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:
> 00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&v
> ars=4h
>> &dataLevel=level0&bleach=0&vers=2
>> at java.net.URI$Parser.fail(Unknown Source)
>> at java.net.URI$Parser.checkChars(Unknown Source)
>> at java.net.URI$Parser.parseHierarchical(Unknown Source)
>> at java.net.URI$Parser.parse(Unknown Source)
>> at java.net.URI.<init>(Unknown Source)
>> ... 13 more
>> Caused by: java.io.IOException: Problem with URI format in
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2'. and
>>
> 'http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00
> :00:00
> &thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&
> vars=4
>> h&dataLevel=level0&bleach=0&vers=2' This can happen if the file name
>> is
> not
>> absoluteand is not present relative to the directory in which the
> specified
>> model was read (which was
>>
> 'file:/C:/Documents%20and%20Settings/Trina/Desktop/THESIS_wORKFLOWS/Hy
> pothes
>> isTest_Reef/AIMS_REALTIME/test_url.xml')
>> at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:356)
>> at ptolemy.data.expr.FileParameter.asURL(FileParameter.java:213)
>> at util.URLToLocalFile._openAndReadBytes(URLToLocalFile.java:226)
>> at util.URLToLocalFile.fire(URLToLocalFile.java:166)
>> at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:398)
>> at
>>
> ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDire
> ctor.j
>> ava:170)
>> at ptolemy.actor.CompositeActor.fire(CompositeActor.java:400)
>> at ptolemy.actor.Manager.iterate(Manager.java:688)
>> at ptolemy.actor.Manager.execute(Manager.java:332)
>> at ptolemy.actor.Manager.run(Manager.java:1071)
>> at ptolemy.actor.Manager$3.run(Manager.java:1112)
>> Caused by: java.lang.IllegalArgumentException
>> at java.net.URI.create(Unknown Source)
>> at java.net.URI.resolve(Unknown Source)
>> at ptolemy.util.FileUtilities.nameToURL(FileUtilities.java:353)
>> ... 10 more
>> Caused by: java.net.URISyntaxException: Illegal character in query at
> index
>> 140:
>>
> http://data.aims.gov.au/awsqaqc/XmlDataServlet?fromDate=2008-02-01T00:
> 00:00&
> thruDate=2008-02-10T00:00:00&stations=4&agperiod=day&agFunct=max|avg&v
> ars=4h
>> &dataLevel=level0&bleach=0&vers=2
>> at java.net.URI$Parser.fail(Unknown Source)
>> at java.net.URI$Parser.checkChars(Unknown Source)
>> at java.net.URI$Parser.parseHierarchical(Unknown Source)
>> at java.net.URI$Parser.parse(Unknown Source)
>> at java.net.URI.<init>(Unknown Source)
>> ... 13 more
>>
>> _______________________________________________
>> Kepler-users mailing list
>> Kepler-users at kepler-project.org
>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>
--
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
--
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
-------------- next part --------------
A non-text attachment was scrubbed...
Name: URLToLocalFileTest4.xml
Type: text/xml
Size: 49373 bytes
Desc: not available
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20090616/06b6c25f/attachment.xml>
More information about the Kepler-users
mailing list