[kepler-dev] [Ptolemy] Re: OrderedRecordToken change[s]

[ben leinfelder]

Christopher,
Great instructions. I did make a test fail after my change (and oddly,  
three tests stopped failing after my change).
The one that failed was: PtParser-18.6 (Test bad expression) -- the  
tcl snippet is below.
I'm not entirely sure what is being tested by this let alone how my  
changes may have impacted it.
Best guess: I've introduced an expression that used to be considered a  
"bad expression" but is now valid because we can parse  
OrderedRecordTokens that are notated with [brackets].
I'm not sure where to go from here... Suggestions would be most  
appreciated.

In terms of the type lattice: All OrderedRecordTokens are  
RecordTokens, but they will loose their original order if represented  
as a RecordToken only. RecordTokens can be converted to  
OrderedRecordTokens losslessly since they will be naturally ordered  
when creating the OrderedRecordToken and that order will be preserved.
-ben

------------------------
test PtParser-18.6 {Test bad expression.} {
     set p [java::new ptolemy.data.expr.PtParser]
     catch {$p {generateParseTree String} "\\0"} errmsg
     regsub -all [java::call System getProperty "line.separator"] \
	        $errmsg "\n" output
     set lines [split $output "\n"]
     list $lines
} {{{ptolemy.kernel.util.IllegalActionException: Error parsing  
expression "\0"} Because: {Encountered "\\" at line 1, column 1.} {Was  
expecting one of:} {    "function" ...} {    "!" ...} {    "~" ...}  
{    <COMPLEX> ...} {    <DOUBLE> ...} {    "-" ...}  
{    <INTEGER> ...} {    <STRING> ...} {    <BOOLEAN> ...}  
{    "(" ...} {    "[" ...} \ \ \ \ \"\{\"\ ... \ \ \ \ \"\{|\"\ ...  
{    <ID> ...} {    <SMID> ...} {    <SMIDBRACE> ...}  
{    <SMIDPAREN> ...} {    }}}


On Aug 19, 2009, at 12:43 PM, Christopher Brooks wrote:

> Hi Ben,
> Steve nailed it, the thing to do is update the visitors.
>
> One issue is that it might make sense to discuss adding  
> OrderedRecordTokens
> to the type lattice.  I'm not sure how this would work.
>
> Can an OrderedRecordToken be lossly converted to a RecordToken?
> What about the other way around?
>
> Anyway, to try your changes, what I would do is:
> 1) Check out a new ptII tree
> svn co svn+ssh://source.eecs.berkeley.edu/chess/ptII ptII
>
> You have an account on source.eecs.berkeley.edu so you should
> have ssh access
>
> 2) Build using configure
>  cd ptII
>  export PTII=`pwd`
>
> Note that if you are on Windows, then don't use `pwd`, use
>  export PTII=c:/yourdirectory/ptII
>
>  ./configure
>  make >& make.out
>
>  Look in make.out for compilation failures
>
> 3) Then run the tests, and save the output
>  make install tests >& tests.out
>
> The install target is not strictly necessary for the tests,
> but some will fail if jar files are not present.
>
> 4) Look at the failed tests
>   egrep "^Failed: [1-9]" tests.out
>   Some tests will fail for you, no big deal, all we want
>   to do is see if your changes cause new failures.
>
> 5) Then, make your changes.
> Your changes should follow the Ptolemy style guide, I've
> included a recent version.  In particular, please follow
> our naming convention for variables.  The formatting can
> get cleaned up automatically.
>
> 6) Rerun:
>   make clean fast install >& make.out
> Look in make.out for lines that have *** in them.
>
>   make tests >& tests2.out
>   egrep "^Failed: [1-9]" tests2.out
>
> 7) Compare the output of step 4 (tests before your changes)
> and step 5 (tests after your changes).
>
> 8) Check in your changes
>
> 9) Profit!
>
> _Christopher
>
>
> Stephen Neuendorffer wrote:
>> Assuming that you modified the AST, probably the main thing is to  
>> make
>> sure that all the AST visitors get updated.
>> Steve
>>> -----Original Message-----
>>> From: ptolemy-admin at chess.eecs.berkeley.edu
>> [mailto:ptolemy-admin at chess.eecs.berkeley.edu] On Behalf
>>> Of ben leinfelder
>>> Sent: Wednesday, August 19, 2009 11:38 AM
>>> To: Christopher Brooks
>>> Cc: ptdevel
>>> Subject: [Ptolemy] Re: OrderedRecordToken change[s]
>>>
>>> Christopher -
>>> JavaCC, while mysterious, is pretty amazing. After a few
>>> modifications, a little compilation, and some rat spit (MacGyver
>>> reference) I have OrderedRecordTokens being created by PtParser. I'm
>>> not sure how far I need to cascade some of these changes.
>>>
>>> The bulk of the changes to support bracket-notation happens in:
>>> ptolemy/data/expr
>>>
>>> but there are other references (based on how the original  
>>> RecordToken
>>> is being parsed) in other packages:
>>>
>>> ptolemy.cg.kernel.generic.program.procedural.c
>>> ptolemy.cg.kernel.generic.program.procedural.java
>>> ptolemy.codegen.c.kernel
>>> ptolemy.codegen.java.kernel
>>> ptolemy.codegen.vhdl.kernel
>>> ptolemy.verification.kernel.maude
>>>
>>> Are there comprehensive tests that I should run before even thinking
>>> about committing this?
>>>
>>> Thanks for the pointers - got me right where I needed to be
>>> -ben
>>>
>>> On Aug 18, 2009, at 4:58 PM, Christopher Brooks wrote:
>>>
>>>> ben leinfelder wrote:
>>>>> Hi Christopher,
>>>>> I want to revisit OrderedRecordToken support in the ptolemy "type
>>>>> lattice" (is that the correct way to say it?).
>>>>> The actual class (OrderRecordToken) is working great - and I like
>>>>> that it is a subclass of RecordToken since they share so much.
>>>>> As a refresher, the problem with OrderedRecordToken comes up when
>>>>> we represent it as a string.
>>>>> For example:
>>>>> OrderedRecordToken is represented as:
>>>>> {b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>> but when we create a new Token object from this string we get:
>>>>> {a={1,2,3}, b={4,5,6}, c={7,8,9}}
>>>>> How might we "denote" that the given order be maintained?
>>>>> Could we use a square bracket:
>>>>> [b={4,5,6}, a={1,2,3}, c={7,8,9}]
>>>>> or prepend an "o":
>>>>> o{b={4,5,6}, a={1,2,3}, c={7,8,9}}
>>>>> Other suggestions?
>>>> Hi Ben,
>>>>
>>>> In the Ptolemy expression language:
>>>> {1,2} is an array of two ints
>>>> [1, 2, 3; 4, 5, 6] is a 2x3 matrix of ints
>>>> {a=1, b="foo"} is a record
>>>>
>>>> So, having
>>>> [b="foo", a=1] be an ordered record makes a certain amount of  
>>>> sense.
>>>> Do you want to try the mysteries of JavaCC and ptolemy/data/expr/
>>>> PtParser.jj?
>>>> What I would do is look at the current record code and work from
>>>> there.
>>>>
>>>> Note that JavaCC 4.1 and 4.2 don't currently work, you will need to
>>>> use
>>>> JavaCC 4.0, see
>>>> https://chess.eecs.berkeley.edu/bugzilla/show_bug.cgi?id=286
>>>> JavaCC may be found at
>>>> https://javacc.dev.java.net/
>>>>
>>>> The way JavaCC works is that if PtParser.jjt is changed, then this
>>>> command is run:
>>>> jjtree PtParser.jjt
>>>> which produces PtParser.jj.  Then, this command is run:
>>>> javacc PtParser.jj
>>>> which produces PtParser.java.
>>>>
>>>> The devil is in the details here.  There could be conflicts in the
>>>> syntax
>>>> that will prevent us from using [] to denote an ordered record
>> token.
>>>> An alternative would be to add a function to the expression  
>>>> language
>>>> that
>>>> creates an ordered token from its argument, but this is kind of
>> gross.
>>>> Your suggestion of "o{" would mean that we need to look at two
>>>> characters
>>>> of lookahead, I'm not sure if this is such a good thing.  If using
>> "["
>>>> worked, then that would be great.
>>>>
>>>> Does anyone else have any comments?
>>>>
>>>> _Christopher
>>>>> Thanks,
>>>>> -ben
>>>>> On Jun 30, 2009, at 8:35 AM, Christopher Brooks wrote:
>>>>>> Hi Ben,
>>>>>> You would need to introduce something into the type system that
>>>>>> would denote that a token is an OrderedRecordToken.  That would
>>>>>> not be that easy, especially at this late date.
>>>>>>
>>>>>> An alternative would be to create a converter:
>>>>>> RecordTokenToOrderedRecordToken.java
>>>>>>
>>>>>> We would need to insert a type as well though.  I think this is
>> best
>>>>>> left until after the release.
>>>>>>
>>>>>> _Christopher
>>>>>>
>>>>>>
>>>>>> An alternative would be to create a
>>>>>> ben leinfelder wrote:
>>>>>>> Okay, but now I'm stuck on the PtParser code that's used for
>>>>>>> turning the string representation of a RecordToken into the
>>>>>>> actual object. There's definitely stuff in there that is not
>>>>>>> immediately obvious to me, but it seems like there won't be a  
>>>>>>> way
>>>>>>> to determine from the string alone whether it should be parsed
>>>>>>> into a normal RecordToken or an OrderedRecordToken.
>>>>>>> The constructor RecordToken(String) kicks it off - and it looks
>>>>>>> like changing those node/parser/visitor/whatnot constructs could
>>>>>>> be invasive (unless there's a hook I've missed).
>>>>>>> If there is a way to include OrderedRecordTokens in the PtParser
>>>>>>> stuff I'm happy to try (with hints). Otherwise I might just
>>>>>>> override (gasp!) the RecordToken class in our Kepler 'wrp'  
>>>>>>> module
>>>>>>> to use the ordered map implementation and call it a day (for the
>>>>>>> 2.0alpha1/South Africa demo).
>>>>>>> Thanks for your feedback,
>>>>>>> -ben
>>>>>>> On Jun 29, 2009, at 3:47 PM, Christopher Brooks wrote:
>>>>>>>> Hi Ben,
>>>>>>>> Sure!
>>>>>>>> One issue is I'm leaving for vacation tomorrow at 4 or 5.
>>>>>>>>
>>>>>>>> _Christopher
>>>>>>>>
>>>>>>>>
>>>>>>>> ben leinfelder wrote:
>>>>>>>>> Hi Christopher,
>>>>>>>>> I know we're getting down to the wire with this 2.0alpha1
>>>>>>>>> release - but I've found a need to override RecordToken's
>>>>>>>>> toString() method in the subclass OrderedRecordToken (that I
>>>>>>>>> wrote). The current implementation sorts the labels  - and
>>>>>>>>> that's not what we (or anyone) want it to do.
>>>>>>>>> Can I commit that change and slip it in Ptolemy even as you
>>>>>>>>> "clean"?
>>>>>>>>> On a related note, I'm also using
>>>>>>>>> ptolemy.actor.lib.RecordAssembler but finding that I (of
>>>>>>>>> course) don't get my nice OrderedRecordTokens from it. Can I
>>>>>>>>> slip in a similar actor (OrderedRecordAssembler) that will use
>>>>>>>>> the ordered implementation of RecordToken? I can do it in
>>>>>>>>> Kepler and leave the Ptolemy tree alone, but it seems like  
>>>>>>>>> it'd
>>>>>>>>> be good to have the two options next to each other.
>>>>>>>>> Thanks,
>>>>>>>>> -ben
>>>>>>>> --
>>>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>>>> California
>>>>>>>> CHESS Executive Director                      US Mail: 337 Cory
>>>>>>>> Hall
>>>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA
>>>>>>>> 94720-1774
>>>>>>>> ph: 510.643.9841 fax:510.642.2718          (Office: 545Q Cory)
>>>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>>>>>>
>>>>>> --
>>>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>>>>>> California
>>>>>> CHESS Executive Director                      US Mail: 337 Cory
>> Hall
>>>>>> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA
>>>>>> 94720-1774
>>>>>> ph: 510.643.9841 fax:510.642.2718          (Office: 545Q Cory)
>>>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>>> --
>>>> Christopher Brooks (cxh at eecs berkeley edu) University of
>> California
>>>> CHESS Executive Director                      US Mail: 337 Cory  
>>>> Hall
>>>> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA
>> 94720-1774
>>>> ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
>>>> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
>>> _______________________________________________
>>> Ptolemy maillist  -  Ptolemy at chess.eecs.berkeley.edu
>>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>> This email and any attachments are intended for the sole use of the  
>> named recipient(s) and contain(s) confidential information that may  
>> be proprietary, privileged or copyrighted under applicable law. If  
>> you are not the intended recipient, do not read, copy, or forward  
>> this email message or any attachments. Delete this email message  
>> and any attachments immediately.
>> _______________________________________________
>> Ptolemy maillist  -  Ptolemy at chess.eecs.berkeley.edu
>> http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
>
> -- 
> Christopher Brooks (cxh at eecs berkeley edu) University of California
> CHESS Executive Director                      US Mail: 337 Cory Hall
> Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
> ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
> home: (F-Tu) 707.665.0131 (W-F) 510.655.5480
> <codingStyle.tex>