[kepler-dev] [Ptolemy] Re: OrderedRecordToken change[s]
Christopher Brooks
cxh at eecs.berkeley.edu
Fri Aug 21 09:53:49 PDT 2009
Hi Ben,
[I'm cc'ing ptdevel and kepler-dev because these are good
questions about the test harness.]
To run all the tests in the directory:
cd $PTII/ptolemy/data/expr/test
make
Or
cd $PTII/ptolemy/data/export
$PTII/bin/ptjacl PtParser.tcl
On PtParser-18.6, the fix would be to change the error message
that is returned. What is happening is that you are changing the
syntax, so the list of accepted tokens changed.
See also
http://ptolemy.eecs.berkeley.edu/ptolemyII/ptIIlatest/ptII/doc/coding/testing.htm
Edward correctly points out that we would need to update the expression chapter
with this change. I'll send you the latex file separately, if you could update
the section with text about ordered record tokens, then that would be a win.
You need not worry about getting the LaTeX right, just getting the text in
is key.
_Christopher
ben leinfelder wrote:
> Is there a way to just run a single test (say on the PtParser)?
> The entire test quite takes a long time (hour or more) for me to complete.
> I'm still trying to figure out what I broke and whether I can fix it.
> -ben
>
> On Aug 20, 2009, at 1:00 PM, Stephen Neuendorffer wrote:
>
>> Remember that 'lossless' isn't necessarily the goal...
>>
>> My guess that in most cases it will be much more useful to strip an ORT
>> of the order to, for instance, use RecordDisassembler than to apply an
>> implicit order to RecordTokens...
>>
>> Steve
>>
>>> -----Original Message-----
>>> From: ptolemy-admin at chess.eecs.berkeley.edu
>> [mailto:ptolemy-admin at chess.eecs.berkeley.edu] On Behalf
>>> Of ben leinfelder
>>> Sent: Thursday, August 20, 2009 12:51 PM
>>> To: Christopher Brooks
>>> Cc: ptdevel; kepler-dev
>>> Subject: Re: [Ptolemy] Re: OrderedRecordToken change[s]
>>>
>>> Christopher,
>>> Great instructions. I did make a test fail after my change (and oddly,
>>> three tests stopped failing after my change).
>>> The one that failed was: PtParser-18.6 (Test bad expression) -- the
>>> tcl snippet is below.
>>> I'm not entirely sure what is being tested by this let alone how my
>>> changes may have impacted it.
>>> Best guess: I've introduced an expression that used to be considered a
>>> "bad expression" but is now valid because we can parse
>>> OrderedRecordTokens that are notated with [brackets].
>>> I'm not sure where to go from here... Suggestions would be most
>>> appreciated.
>>>
>>> In terms of the type lattice: All OrderedRecordTokens are
>>> RecordTokens, but they will loose their original order if represented
>>> as a RecordToken only. RecordTokens can be converted to
>>> OrderedRecordTokens losslessly since they will be naturally ordered
>>> when creating the OrderedRecordToken and that order will be preserved.
>>> -ben
>>>
>>> ------------------------
>>> test PtParser-18.6 {Test bad expression.} {
>>> set p [java::new ptolemy.data.expr.PtParser]
>>> catch {$p {generateParseTree String} "\\0"} errmsg
>>> regsub -all [java::call System getProperty "line.separator"] \
>>> $errmsg "\n" output
>>> set lines [split $output "\n"]
>>> list $lines
>>> } {{{ptolemy.kernel.util.IllegalActionException: Error parsing
>>> expression "\0"} Because: {Encountered "\\" at line 1, column 1.} {Was
>>> expecting one of:} { "function" ...} { "!" ...} { "~" ...}
>>> { <COMPLEX> ...} { <DOUBLE> ...} { "-" ...}
>>> { <INTEGER> ...} { <STRING> ...} { <BOOLEAN> ...}
>>> { "(" ...} { "[" ...} \ \ \ \ \"\{\"\ ... \ \ \ \ \"\{|\"\ ...
>>> { <ID> ...} { <SMID> ...} { <SMIDBRACE> ...}
>>> { <SMIDPAREN> ...} { }}}
>>>
>>>
>>> On Aug 19, 2009, at 12:43 PM, Christopher Brooks wrote:
>>>
>>>> Hi Ben,
>>>> Steve nailed it, the thing to do is update the visitors.
>>>>
>>>> One issue is that it might make sense to discuss adding
>>>> OrderedRecordTokens
>>>> to the type lattice. I'm not sure how this would work.
>>>>
>>>> Can an OrderedRecordToken be lossly converted to a RecordToken?
>>>> What about the other way around?
>>>>
>>>> Anyway, to try your changes, what I would do is:
>>>> 1) Check out a new ptII tree
>>>> svn co svn+ssh://source.eecs.berkeley.edu/chess/ptII ptII
>>>>
>>>> You have an account on source.eecs.berkeley.edu so you should
>>>> have ssh access
>>>>
>>>> 2) Build using configure
>>>> cd ptII
>>>> export PTII=`pwd`
>>>>
>>>> Note that if you are on Windows, then don't use `pwd`, use
>>>> export PTII=c:/yourdirectory/ptII
>>>>
>>>> ./configure
>>>> make >& make.out
>>>>
>>>> Look in make.out for compilation failures
>>>>
>>>> 3) Then run the tests, and save the output
>>>> make install tests >& tests.out
>>>>
>>>> The install target is not strictly necessary for the tests,
>>>> but some will fail if jar files are not present.
>>>>
>>>> 4) Look at the failed tests
>>>> egrep "^Failed: [1-9]" tests.out
>>>> Some tests will fail for you, no big deal, all we want
>>>> to do is see if your changes cause new failures.
>>>>
>>>> 5) Then, make your changes.
>>>> Your changes should follow the Ptolemy style guide, I've
>>>> included a recent version. In particular, please follow
>>>> our naming convention for variables. The formatting can
>>>> get cleaned up automatically.
>>>>
>>>> 6) Rerun:
>>>> make clean fast install >& make.out
>>>> Look in make.out for lines that have *** in them.
>>>>
>>>> make tests >& tests2.out
>>>> egrep "^Failed: [1-9]" tests2.out
>>>>
>>>> 7) Compare the output of step 4 (tests before your changes)
>>>> and step 5 (tests after your changes).
>>>>
>>>> 8) Check in your changes
>>>>
>>>> 9) Profit!
>>>>
>>>> _Christopher
>>>>
>>>>
>>>> Stephen Neuendorffer wrote:
>>>>> Assuming that you modified the AST, probably the main thing is to
>>>>> make
>>>>> sure that all the AST visitors get updated.
>>>>> Steve
>>>>>> -----Original Message-----
>>>>>> From: ptolemy-admin at chess.eecs.berkeley.edu
>>>>> [mailto:ptolemy-admin at chess.eecs.berkeley.edu] On Behalf
>>>>>> Of ben leinfelder
>>>>>> Sent: Wednesday, August 19, 2009 11:38 AM
>>>>>> To: Christopher Brooks
>>>>>> Cc: ptdevel
>>>>>> Subject: [Ptolemy] Re: OrderedRecordToken change[s]
>>>>>>
>>>>>> Christopher -
>>>>>> JavaCC, while mysterious, is pretty amazing. After a few
>>>>>> modifications, a little compilation, and some rat spit (MacGyver
>>>>>> reference) I have OrderedRecordTokens being created by PtParser.
>> I'm
>>>>>> not sure how far I need to cascade some of these changes.
>>>>>>
>>>>>> The bulk of the changes to support bracket-notation happens in:
>>>>>> ptolemy/data/expr
>>>>>>
>>>>>> but there are other references (based on how the original
>>>>>> RecordToken
>>>>>> is being parsed) in other packages:
>>>>>>
>>>>>> ptolemy.cg.kernel.generic.program.procedural.c
>>>>>> ptolemy.cg.kernel.generic.program.procedural.java
>>>>>> ptolemy.codegen.c.kernel
>>>>>> ptolemy.codegen.java.kernel
>>>>>> ptolemy.codegen.vhdl.kernel
>>>>>> ptolemy.verification.kernel.maude
>>>>>>
>>>>>> Are there comprehensive tests that I should run before even
>> thinking
>>>>>> about committing this?
>>>>>>
>>>>>> Thanks for the pointers - got me right where I needed to be
>>>>>> -ben
>>>>>>
>>>>>> On Aug 18, 2009, at 4:58 PM, Christopher Brooks wrote:
>>>>>>
>>>>>>> ben leinfelder wrote:
>>>>>>>> Hi Christopher,
>>>>>>>> I want to revisit OrderedRecordToken support in the ptolemy
>> "type
>>>>>>>> lattice" (is that the correct way to say it?).
>>>>>>>> The actual class (OrderRecordToken) is working great - and I
>> like
>>>>>>>> that it is a subclass of RecordToken since they share so much.
>>>>>>>> As a refresher, the problem with OrderedRecordToken comes up
>> when
>>>>>>>> we represent it as a string.
>>>>>>>> For example:
>>>>>>>> OrderedRecordToken is represented as:
>>>>>>>> {b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>>>>> but when we create a new Token object from this string we get:
>>>>>>>> {a={1,2,3}, b={4,5,6}, c={7,8,9}}
>>>>>>>> How might we "denote" that the given order be maintained?
>>>>>>>> Could we use a square bracket:
>>>>>>>> [b={4,5,6}, a={1,2,3}, c={7,8,9}]
>>>>>>>> or prepend an "o":
>>>>>>>> o{b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>>>>> Other suggestions?
>>>>>>> Hi Ben,
>>>>>>>
>>>>>>> In the Ptolemy expression language:
>>>>>>> {1,2} is an array of two ints
>>>>>>> [1, 2, 3; 4, 5, 6] is a 2x3 matrix of ints
>>>>>>> {a=1, b="foo"} is a record
>>>>>>>
>>>>>>> So, having
>>>>>>> [b="foo", a=1] be an ordered record makes a certain amount of
>>>>>>> sense.
>>>>>>> Do you want to try the mysteries of JavaCC and ptolemy/data/expr/
>>>>>>> PtParser.jj?
>>>>>>> What I would do is look at the current record code and work from
>>>>>>> there.
>>>>>>>
>>>>>>> Note that JavaCC 4.1 and 4.2 don't currently work, you will need
>> to
>>>>>>> use
>>>>>>> JavaCC 4.0, see
>>>>>>> https://chess.eecs.berkeley.edu/bugzilla/show_bug.cgi?id=286
>>>>>>> JavaCC may be found at
>>>>>>> https://javacc.dev.java.net/
>>>>>>>
>>>>>>> The way JavaCC works is that if PtParser.jjt is changed, then
>> this
>>>>>>> command is run:
>>>>>>> jjtree PtParser.jjt
>>>>>>> which produces PtParser.jj. Then, this command is run:
>>>>>>> javacc PtParser.jj
>>>>>>> which produces PtParser.java.
>>>>>>>
>>>>>>> The devil is in the details here. There could be conflicts in
>> the
>>>>>>> syntax
>>>>>>> that will prevent us from using [] to denote an ordered record
>>>>> token.
>>>>>>> An alternative would be to add a function to the expression
>>>>>>> language
>>>>>>> that
>>>>>>> creates an ordered token from its argument, but this is kind of
>>>>> gross.
>>>>>>> Your suggestion of "o{" would mean that we need to look at two
>>>>>>> characters
>>>>>>> of lookahead, I'm not sure if this is such a good thing. If
>> using
>>>>> "["
>>>>>>> worked, then that would be great.
>>>>>>>
>>>>>>> Does anyone else have any comments?
>>>>>>>
>>>>>>> _Christopher
>>>>>>>> Thanks,
>>>>>>>> -ben
>>>>>>>> On Jun 30, 2009, at 8:35 AM, Christopher Brooks wrote:
>>>>>>>>> Hi Ben,
>>>>>>>>> You would need to introduce something into the type system that
>>>>>>>>> would denote that a token is an OrderedRecordToken. That would
>>>>>>>>> not be that easy, especially at this late date.
>>>>>>>>>
>>>>>>>>> An alternative would be to create a converter:
>>>>>>>>> RecordTokenToOrderedRecordToken.java
>>>>>>>>>
>>>>>>>>> We would need to insert a type as well though. I think this is
>>>>> best
>>>>>>>>> left until after the release.
>>>>>>>>>
>>>>>>>>> _Christopher
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> An alternative would be to create a
>>>>>>>>> ben leinfelder wrote:
>>>>>>>>>> Okay, but now I'm stuck on the PtParser code that's used for
>>>>>>>>>> turning the string representation of a RecordToken into the
>>>>>>>>>> actual object. There's definitely stuff in there that is not
>>>>>>>>>> immediately obvious to me, but it seems like there won't be a
>>>>>>>>>> way
>>>>>>>>>> to determine from the string alone whether it should be parsed
>>>>>>>>>> into a normal RecordToken or an OrderedRecordToken.
>>>>>>>>>> The constructor RecordToken(String) kicks it off - and it
>> looks
>>>>>>>>>> like changing those node/parser/visitor/whatnot constructs
>> could
>>>>>>>>>> be invasive (unless there's a hook I've missed).
>>>>>>>>>> If there is a way to include OrderedRecordTokens in the
>> PtParser
>>>>>>>>>> stuff I'm happy to try (with hints). Otherwise I might just
>>>>>>>>>> override (gasp!) the RecordToken class in our Kepler 'wrp'
>>>>>>>>>> module
>>>>>>>>>> to use the ordered map implementation and call it a day (for
>> the
>>>>>>>>>> 2.0alpha1/South Africa demo).
>>>>>>>>>> Thanks for your feedback,
>>>>>>>>>> -ben
>>>>>>>>>> On Jun 29, 2009, at 3:47 PM, Christopher Brooks wrote:
>>>>>>>>>>> Hi Ben,
>>>>>>>>>>> Sure!
>>>>>>>>>>> One issue is I'm leaving for vacation tomorrow at 4 or 5.
>>>>>>>>>>>
>>>>>>>>>>> _Christopher
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ben leinfelder wrote:
>>>>>>>>>>>> Hi Christopher,
>>>>>>>>>>>> I know we're getting down to the wire with this 2.0alpha1
>>>>>>>>>>>> release - but I've found a need to override RecordToken's
>>>>>>>>>>>> toString() method in the subclass OrderedRecordToken (that I
>>>>>>>>>>>> wrote). The current implementation sorts the labels - and
>>>>>>>>>>>> that's not what we (or anyone) want it to do.
>>>>>>>>>>>> Can I commit that change and slip it in Ptolemy even as you
>>>>>>>>>>>> "clean"?
>>>>>>>>>>>> On a related note, I'm also using
>>>>>>>>>>>> ptolemy.actor.lib.RecordAssembler but finding that I (of
>>>>>>>>>>>> course) don't get my nice OrderedRecordTokens from it. Can I
>>>>>>>>>>>> slip in a similar actor (OrderedRecordAssembler) that will
>> use
>>>>>>>>>>>> the ordered implementation of RecordToken? I can do it in
>>>>>>>>>>>> Kepler and leave the Ptolemy tree alone, but it seems like
>>>>>>>>>>>> it'd
>>>>>>>>>>>> be good to have the two options next to each other.
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -ben
>>>>>>>>>>> --
>>>>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>>>>>> California
>>>>>>>>>>> CHESS Executive Director US Mail: 337
>> Cory
>>>>>>>>>>> Hall
>>>>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>>>>>>>> 94720-1774
>>>>>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q
>> Cory)
>>>>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>>>> California
>>>>>>>>> CHESS Executive Director US Mail: 337 Cory
>>>>> Hall
>>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>>>>>> 94720-1774
>>>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
>>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>> --
>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>> California
>>>>>>> CHESS Executive Director US Mail: 337 Cory
>>>>>>> Hall
>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>> 94720-1774
>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q
>> Cory)
>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>> _______________________________________________
>>>>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>>>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>>>> This email and any attachments are intended for the sole use of the
>>>>> named recipient(s) and contain(s) confidential information that may
>>>>> be proprietary, privileged or copyrighted under applicable law. If
>>>>> you are not the intended recipient, do not read, copy, or forward
>>>>> this email message or any attachments. Delete this email message
>>>>> and any attachments immediately.
>>>>> _______________________________________________
>>>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>>>
>>>> --
>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>> California
>>>> CHESS Executive Director US Mail: 337 Cory Hall
>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>> 94720-1774
>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>> <codingStyle.tex>
>>>
>>> _______________________________________________
>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>
>>
>> This email and any attachments are intended for the sole use of the
>> named recipient(s) and contain(s) confidential information that may be
>> proprietary, privileged or copyrighted under applicable law. If you
>> are not the intended recipient, do not read, copy, or forward this
>> email message or any attachments. Delete this email message and any
>> attachments immediately.
>>
>>
>
--
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
More information about the Kepler-dev
mailing list