[kepler-dev] [Ptolemy] Re: OrderedRecordToken change[s]
Edward A. Lee
eal at eecs.berkeley.edu
Fri Aug 21 13:18:01 PDT 2009
Ben,
equality checks need to be commutative, so 3a and 3b won't work.
I.e., if x == y, then it must be true that y == x.
Also, with RecordToken, there is no meaning to "the order matches."
So I think that if x is a RecordToken and y is an OrderedRecordToken,
it needs to always be true that x != y and y != x.
Edward
ben leinfelder wrote:
> Christopher -
> I've resolved the PtParser test - actually just rearranged how I had
> rules defined in the .jjt file. I'm re-running the test suites to verify
> that nothing is broken. Thanks for pointing out those resources and
> keeping the appropriate lists involved in this discussion - a learning
> experience for me.
>
> I'm happy to add a section to the expressions chapter, but I think there
> are some questions remaining - I had a few after reading the chapter:
>
> 1. OrderedRecordToken.equals() should return false if the order is
> different but all the labels and fields still match. This is different
> from RecordToken (where the order is not taken into consideration). I've
> implemented that now.
> 2. In terms of expressions and equality tests, I see reference to
> 'equals()' and also '==' where the former takes 'type' into
> consideration and the latter does not. I'm guessing that the
> implementation of '_isEqualTo()' is used when '==' is encountered in the
> expression. Yes?
> 3. How should we handle RecordToken-to-OrderedRecordToken equality
> checks (and vice versa)? I believe this is appropriate:
> 3a. OrderedRecordToken.equals(RecordToken) will require the order to match.
> 3b. RecordToken.equals(OrderedRecordToken) will not care about the order.
>
> Related note: I've added an additional protected method
> _createRecordToken() that allows us to reuse RecordToken methods like
> _add() without overriding them almost verbatim in OrderedRecordToken class.
>
> Thanks!
> -ben
>
> On Aug 21, 2009, at 9:53 AM, Christopher Brooks wrote:
>
>> Hi Ben,
>> [I'm cc'ing ptdevel and kepler-dev because these are good
>> questions about the test harness.]
>>
>> To run all the tests in the directory:
>> cd $PTII/ptolemy/data/expr/test
>> make
>>
>> Or
>> cd $PTII/ptolemy/data/export
>> $PTII/bin/ptjacl PtParser.tcl
>>
>> On PtParser-18.6, the fix would be to change the error message
>> that is returned. What is happening is that you are changing the
>> syntax, so the list of accepted tokens changed.
>>
>> See also
>> http://ptolemy.eecs.berkeley.edu/ptolemyII/ptIIlatest/ptII/doc/coding/testing.htm
>>
>>
>>
>> Edward correctly points out that we would need to update the
>> expression chapter
>> with this change. I'll send you the latex file separately, if you
>> could update
>> the section with text about ordered record tokens, then that would be
>> a win.
>> You need not worry about getting the LaTeX right, just getting the
>> text in
>> is key.
>>
>> _Christopher
>>
>> ben leinfelder wrote:
>>> Is there a way to just run a single test (say on the PtParser)?
>>> The entire test quite takes a long time (hour or more) for me to
>>> complete.
>>> I'm still trying to figure out what I broke and whether I can fix it.
>>> -ben
>>> On Aug 20, 2009, at 1:00 PM, Stephen Neuendorffer wrote:
>>>> Remember that 'lossless' isn't necessarily the goal...
>>>>
>>>> My guess that in most cases it will be much more useful to strip an ORT
>>>> of the order to, for instance, use RecordDisassembler than to apply an
>>>> implicit order to RecordTokens...
>>>>
>>>> Steve
>>>>
>>>>> -----Original Message-----
>>>>> From: ptolemy-admin at chess.eecs.berkeley.edu
>>>> [mailto:ptolemy-admin at chess.eecs.berkeley.edu] On Behalf
>>>>> Of ben leinfelder
>>>>> Sent: Thursday, August 20, 2009 12:51 PM
>>>>> To: Christopher Brooks
>>>>> Cc: ptdevel; kepler-dev
>>>>> Subject: Re: [Ptolemy] Re: OrderedRecordToken change[s]
>>>>>
>>>>> Christopher,
>>>>> Great instructions. I did make a test fail after my change (and oddly,
>>>>> three tests stopped failing after my change).
>>>>> The one that failed was: PtParser-18.6 (Test bad expression) -- the
>>>>> tcl snippet is below.
>>>>> I'm not entirely sure what is being tested by this let alone how my
>>>>> changes may have impacted it.
>>>>> Best guess: I've introduced an expression that used to be considered a
>>>>> "bad expression" but is now valid because we can parse
>>>>> OrderedRecordTokens that are notated with [brackets].
>>>>> I'm not sure where to go from here... Suggestions would be most
>>>>> appreciated.
>>>>>
>>>>> In terms of the type lattice: All OrderedRecordTokens are
>>>>> RecordTokens, but they will loose their original order if represented
>>>>> as a RecordToken only. RecordTokens can be converted to
>>>>> OrderedRecordTokens losslessly since they will be naturally ordered
>>>>> when creating the OrderedRecordToken and that order will be preserved.
>>>>> -ben
>>>>>
>>>>> ------------------------
>>>>> test PtParser-18.6 {Test bad expression.} {
>>>>> set p [java::new ptolemy.data.expr.PtParser]
>>>>> catch {$p {generateParseTree String} "\\0"} errmsg
>>>>> regsub -all [java::call System getProperty "line.separator"] \
>>>>> $errmsg "\n" output
>>>>> set lines [split $output "\n"]
>>>>> list $lines
>>>>> } {{{ptolemy.kernel.util.IllegalActionException: Error parsing
>>>>> expression "\0"} Because: {Encountered "\\" at line 1, column 1.} {Was
>>>>> expecting one of:} { "function" ...} { "!" ...} { "~" ...}
>>>>> { <COMPLEX> ...} { <DOUBLE> ...} { "-" ...}
>>>>> { <INTEGER> ...} { <STRING> ...} { <BOOLEAN> ...}
>>>>> { "(" ...} { "[" ...} \ \ \ \ \"\{\"\ ... \ \ \ \ \"\{|\"\ ...
>>>>> { <ID> ...} { <SMID> ...} { <SMIDBRACE> ...}
>>>>> { <SMIDPAREN> ...} { }}}
>>>>>
>>>>>
>>>>> On Aug 19, 2009, at 12:43 PM, Christopher Brooks wrote:
>>>>>
>>>>>> Hi Ben,
>>>>>> Steve nailed it, the thing to do is update the visitors.
>>>>>>
>>>>>> One issue is that it might make sense to discuss adding
>>>>>> OrderedRecordTokens
>>>>>> to the type lattice. I'm not sure how this would work.
>>>>>>
>>>>>> Can an OrderedRecordToken be lossly converted to a RecordToken?
>>>>>> What about the other way around?
>>>>>>
>>>>>> Anyway, to try your changes, what I would do is:
>>>>>> 1) Check out a new ptII tree
>>>>>> svn co svn+ssh://source.eecs.berkeley.edu/chess/ptII ptII
>>>>>>
>>>>>> You have an account on source.eecs.berkeley.edu so you should
>>>>>> have ssh access
>>>>>>
>>>>>> 2) Build using configure
>>>>>> cd ptII
>>>>>> export PTII=`pwd`
>>>>>>
>>>>>> Note that if you are on Windows, then don't use `pwd`, use
>>>>>> export PTII=c:/yourdirectory/ptII
>>>>>>
>>>>>> ./configure
>>>>>> make >& make.out
>>>>>>
>>>>>> Look in make.out for compilation failures
>>>>>>
>>>>>> 3) Then run the tests, and save the output
>>>>>> make install tests >& tests.out
>>>>>>
>>>>>> The install target is not strictly necessary for the tests,
>>>>>> but some will fail if jar files are not present.
>>>>>>
>>>>>> 4) Look at the failed tests
>>>>>> egrep "^Failed: [1-9]" tests.out
>>>>>> Some tests will fail for you, no big deal, all we want
>>>>>> to do is see if your changes cause new failures.
>>>>>>
>>>>>> 5) Then, make your changes.
>>>>>> Your changes should follow the Ptolemy style guide, I've
>>>>>> included a recent version. In particular, please follow
>>>>>> our naming convention for variables. The formatting can
>>>>>> get cleaned up automatically.
>>>>>>
>>>>>> 6) Rerun:
>>>>>> make clean fast install >& make.out
>>>>>> Look in make.out for lines that have *** in them.
>>>>>>
>>>>>> make tests >& tests2.out
>>>>>> egrep "^Failed: [1-9]" tests2.out
>>>>>>
>>>>>> 7) Compare the output of step 4 (tests before your changes)
>>>>>> and step 5 (tests after your changes).
>>>>>>
>>>>>> 8) Check in your changes
>>>>>>
>>>>>> 9) Profit!
>>>>>>
>>>>>> _Christopher
>>>>>>
>>>>>>
>>>>>> Stephen Neuendorffer wrote:
>>>>>>> Assuming that you modified the AST, probably the main thing is to
>>>>>>> make
>>>>>>> sure that all the AST visitors get updated.
>>>>>>> Steve
>>>>>>>> -----Original Message-----
>>>>>>>> From: ptolemy-admin at chess.eecs.berkeley.edu
>>>>>>> [mailto:ptolemy-admin at chess.eecs.berkeley.edu] On Behalf
>>>>>>>> Of ben leinfelder
>>>>>>>> Sent: Wednesday, August 19, 2009 11:38 AM
>>>>>>>> To: Christopher Brooks
>>>>>>>> Cc: ptdevel
>>>>>>>> Subject: [Ptolemy] Re: OrderedRecordToken change[s]
>>>>>>>>
>>>>>>>> Christopher -
>>>>>>>> JavaCC, while mysterious, is pretty amazing. After a few
>>>>>>>> modifications, a little compilation, and some rat spit (MacGyver
>>>>>>>> reference) I have OrderedRecordTokens being created by PtParser.
>>>> I'm
>>>>>>>> not sure how far I need to cascade some of these changes.
>>>>>>>>
>>>>>>>> The bulk of the changes to support bracket-notation happens in:
>>>>>>>> ptolemy/data/expr
>>>>>>>>
>>>>>>>> but there are other references (based on how the original
>>>>>>>> RecordToken
>>>>>>>> is being parsed) in other packages:
>>>>>>>>
>>>>>>>> ptolemy.cg.kernel.generic.program.procedural.c
>>>>>>>> ptolemy.cg.kernel.generic.program.procedural.java
>>>>>>>> ptolemy.codegen.c.kernel
>>>>>>>> ptolemy.codegen.java.kernel
>>>>>>>> ptolemy.codegen.vhdl.kernel
>>>>>>>> ptolemy.verification.kernel.maude
>>>>>>>>
>>>>>>>> Are there comprehensive tests that I should run before even
>>>> thinking
>>>>>>>> about committing this?
>>>>>>>>
>>>>>>>> Thanks for the pointers - got me right where I needed to be
>>>>>>>> -ben
>>>>>>>>
>>>>>>>> On Aug 18, 2009, at 4:58 PM, Christopher Brooks wrote:
>>>>>>>>
>>>>>>>>> ben leinfelder wrote:
>>>>>>>>>> Hi Christopher,
>>>>>>>>>> I want to revisit OrderedRecordToken support in the ptolemy
>>>> "type
>>>>>>>>>> lattice" (is that the correct way to say it?).
>>>>>>>>>> The actual class (OrderRecordToken) is working great - and I
>>>> like
>>>>>>>>>> that it is a subclass of RecordToken since they share so much.
>>>>>>>>>> As a refresher, the problem with OrderedRecordToken comes up
>>>> when
>>>>>>>>>> we represent it as a string.
>>>>>>>>>> For example:
>>>>>>>>>> OrderedRecordToken is represented as:
>>>>>>>>>> {b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>>>>>>> but when we create a new Token object from this string we get:
>>>>>>>>>> {a={1,2,3}, b={4,5,6}, c={7,8,9}}
>>>>>>>>>> How might we "denote" that the given order be maintained?
>>>>>>>>>> Could we use a square bracket:
>>>>>>>>>> [b={4,5,6}, a={1,2,3}, c={7,8,9}]
>>>>>>>>>> or prepend an "o":
>>>>>>>>>> o{b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>>>>>>> Other suggestions?
>>>>>>>>> Hi Ben,
>>>>>>>>>
>>>>>>>>> In the Ptolemy expression language:
>>>>>>>>> {1,2} is an array of two ints
>>>>>>>>> [1, 2, 3; 4, 5, 6] is a 2x3 matrix of ints
>>>>>>>>> {a=1, b="foo"} is a record
>>>>>>>>>
>>>>>>>>> So, having
>>>>>>>>> [b="foo", a=1] be an ordered record makes a certain amount of
>>>>>>>>> sense.
>>>>>>>>> Do you want to try the mysteries of JavaCC and ptolemy/data/expr/
>>>>>>>>> PtParser.jj?
>>>>>>>>> What I would do is look at the current record code and work from
>>>>>>>>> there.
>>>>>>>>>
>>>>>>>>> Note that JavaCC 4.1 and 4.2 don't currently work, you will need
>>>> to
>>>>>>>>> use
>>>>>>>>> JavaCC 4.0, see
>>>>>>>>> https://chess.eecs.berkeley.edu/bugzilla/show_bug.cgi?id=286
>>>>>>>>> JavaCC may be found at
>>>>>>>>> https://javacc.dev.java.net/
>>>>>>>>>
>>>>>>>>> The way JavaCC works is that if PtParser.jjt is changed, then
>>>> this
>>>>>>>>> command is run:
>>>>>>>>> jjtree PtParser.jjt
>>>>>>>>> which produces PtParser.jj. Then, this command is run:
>>>>>>>>> javacc PtParser.jj
>>>>>>>>> which produces PtParser.java.
>>>>>>>>>
>>>>>>>>> The devil is in the details here. There could be conflicts in
>>>> the
>>>>>>>>> syntax
>>>>>>>>> that will prevent us from using [] to denote an ordered record
>>>>>>> token.
>>>>>>>>> An alternative would be to add a function to the expression
>>>>>>>>> language
>>>>>>>>> that
>>>>>>>>> creates an ordered token from its argument, but this is kind of
>>>>>>> gross.
>>>>>>>>> Your suggestion of "o{" would mean that we need to look at two
>>>>>>>>> characters
>>>>>>>>> of lookahead, I'm not sure if this is such a good thing. If
>>>> using
>>>>>>> "["
>>>>>>>>> worked, then that would be great.
>>>>>>>>>
>>>>>>>>> Does anyone else have any comments?
>>>>>>>>>
>>>>>>>>> _Christopher
>>>>>>>>>> Thanks,
>>>>>>>>>> -ben
>>>>>>>>>> On Jun 30, 2009, at 8:35 AM, Christopher Brooks wrote:
>>>>>>>>>>> Hi Ben,
>>>>>>>>>>> You would need to introduce something into the type system that
>>>>>>>>>>> would denote that a token is an OrderedRecordToken. That would
>>>>>>>>>>> not be that easy, especially at this late date.
>>>>>>>>>>>
>>>>>>>>>>> An alternative would be to create a converter:
>>>>>>>>>>> RecordTokenToOrderedRecordToken.java
>>>>>>>>>>>
>>>>>>>>>>> We would need to insert a type as well though. I think this is
>>>>>>> best
>>>>>>>>>>> left until after the release.
>>>>>>>>>>>
>>>>>>>>>>> _Christopher
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> An alternative would be to create a
>>>>>>>>>>> ben leinfelder wrote:
>>>>>>>>>>>> Okay, but now I'm stuck on the PtParser code that's used for
>>>>>>>>>>>> turning the string representation of a RecordToken into the
>>>>>>>>>>>> actual object. There's definitely stuff in there that is not
>>>>>>>>>>>> immediately obvious to me, but it seems like there won't be a
>>>>>>>>>>>> way
>>>>>>>>>>>> to determine from the string alone whether it should be parsed
>>>>>>>>>>>> into a normal RecordToken or an OrderedRecordToken.
>>>>>>>>>>>> The constructor RecordToken(String) kicks it off - and it
>>>> looks
>>>>>>>>>>>> like changing those node/parser/visitor/whatnot constructs
>>>> could
>>>>>>>>>>>> be invasive (unless there's a hook I've missed).
>>>>>>>>>>>> If there is a way to include OrderedRecordTokens in the
>>>> PtParser
>>>>>>>>>>>> stuff I'm happy to try (with hints). Otherwise I might just
>>>>>>>>>>>> override (gasp!) the RecordToken class in our Kepler 'wrp'
>>>>>>>>>>>> module
>>>>>>>>>>>> to use the ordered map implementation and call it a day (for
>>>> the
>>>>>>>>>>>> 2.0alpha1/South Africa demo).
>>>>>>>>>>>> Thanks for your feedback,
>>>>>>>>>>>> -ben
>>>>>>>>>>>> On Jun 29, 2009, at 3:47 PM, Christopher Brooks wrote:
>>>>>>>>>>>>> Hi Ben,
>>>>>>>>>>>>> Sure!
>>>>>>>>>>>>> One issue is I'm leaving for vacation tomorrow at 4 or 5.
>>>>>>>>>>>>>
>>>>>>>>>>>>> _Christopher
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ben leinfelder wrote:
>>>>>>>>>>>>>> Hi Christopher,
>>>>>>>>>>>>>> I know we're getting down to the wire with this 2.0alpha1
>>>>>>>>>>>>>> release - but I've found a need to override RecordToken's
>>>>>>>>>>>>>> toString() method in the subclass OrderedRecordToken (that I
>>>>>>>>>>>>>> wrote). The current implementation sorts the labels - and
>>>>>>>>>>>>>> that's not what we (or anyone) want it to do.
>>>>>>>>>>>>>> Can I commit that change and slip it in Ptolemy even as you
>>>>>>>>>>>>>> "clean"?
>>>>>>>>>>>>>> On a related note, I'm also using
>>>>>>>>>>>>>> ptolemy.actor.lib.RecordAssembler but finding that I (of
>>>>>>>>>>>>>> course) don't get my nice OrderedRecordTokens from it. Can I
>>>>>>>>>>>>>> slip in a similar actor (OrderedRecordAssembler) that will
>>>> use
>>>>>>>>>>>>>> the ordered implementation of RecordToken? I can do it in
>>>>>>>>>>>>>> Kepler and leave the Ptolemy tree alone, but it seems like
>>>>>>>>>>>>>> it'd
>>>>>>>>>>>>>> be good to have the two options next to each other.
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -ben
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>>>>>>>> California
>>>>>>>>>>>>> CHESS Executive Director US Mail: 337
>>>> Cory
>>>>>>>>>>>>> Hall
>>>>>>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>>>>>>>>>> 94720-1774
>>>>>>>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q
>>>> Cory)
>>>>>>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>>>>>> California
>>>>>>>>>>> CHESS Executive Director US Mail: 337 Cory
>>>>>>> Hall
>>>>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>>>>>>>> 94720-1774
>>>>>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
>>>>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>>>> --
>>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>> California
>>>>>>>>> CHESS Executive Director US Mail: 337 Cory
>>>>>>>>> Hall
>>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>>>>> 94720-1774
>>>>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q
>>>> Cory)
>>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>>> _______________________________________________
>>>>>>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>>>>>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>>>>>> This email and any attachments are intended for the sole use of the
>>>>>>> named recipient(s) and contain(s) confidential information that may
>>>>>>> be proprietary, privileged or copyrighted under applicable law. If
>>>>>>> you are not the intended recipient, do not read, copy, or forward
>>>>>>> this email message or any attachments. Delete this email message
>>>>>>> and any attachments immediately.
>>>>>>> _______________________________________________
>>>>>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>>>>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>>>>>
>>>>>> --
>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>> California
>>>>>> CHESS Executive Director US Mail: 337 Cory Hall
>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA
>>>> 94720-1774
>>>>>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>> <codingStyle.tex>
>>>>>
>>>>> _______________________________________________
>>>>> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
>>>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>>>>
>>>>
>>>> This email and any attachments are intended for the sole use of the
>>>> named recipient(s) and contain(s) confidential information that may
>>>> be proprietary, privileged or copyrighted under applicable law. If
>>>> you are not the intended recipient, do not read, copy, or forward
>>>> this email message or any attachments. Delete this email message and
>>>> any attachments immediately.
>>>>
>>>>
>>
>> --
>> Christopher Brooks (cxh at eecs berkeley edu) University of California
>> CHESS Executive Director US Mail: 337 Cory Hall
>> Programmer/Analyst CHESS/Ptolemy/Trust Berkeley, CA 94720-1774
>> ph: 510.643.9841 fax:510.642.2718 (Office: 545Q Cory)
>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>
> _______________________________________________
> Ptolemy maillist - Ptolemy at chess.eecs.berkeley.edu
> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eal.vcf
Type: text/x-vcard
Size: 351 bytes
Desc: not available
URL: <http://mercury.nceas.ucsb.edu/kepler/pipermail/kepler-dev/attachments/20090821/6495f0d0/attachment.vcf>
More information about the Kepler-dev
mailing list