[kepler-dev] [Bug 5722] Check for problems with sanitized RecordToken labels

[bugzilla-daemon at ecoinformatics.org]

http://bugzilla.ecoinformatics.org/show_bug.cgi?id=5722

--- Comment #7 from Derik Barseghian <barseghian at nceas.ucsb.edu> ---
I believe that by reverting these files as follows:
*    64638   ptolemy/actor/lib/OrderedRecordAssembler.java
*    64638   ptolemy/actor/lib/io/test/auto/ReadCSV2.xml
*    64638   ptolemy/actor/lib/RecordDisassembler.java
*    64632   ptolemy/data/type/RecordType.java
*    64632   ptolemy/data/RecordToken.java
*    64638   ptolemy/data/OrderedRecordToken.java

gets us back to Kepler 2.3 behavior -- e.g.: 
1) An Ordered Record Assembler can produce tokens with pre and postfix numbers,
spaces, and parens in label:
[1Day of Week1 = {"Mon", "Tue"}, 2Rain Fall (in)2 = {3, 7}]

1b) You can also use the Record Disassembler to disassemble that token without
any character changes.

2) Trying to use the same token in an Expression will fail for three reasons:
the included prefix numbers, spaces, and parens in the labels.


After reverting the above files, I made a change to PtParser.jjt (attached)
that allows prefix number and spaces (I haven't gotten parens to work). I
initially tried many things to specifically not change the rule for ID and
instead make specific rules for the two types of recordToken, but I wasn't able
to do this.

While the change gives us Kepler 2.3 behavior plus better handling in the
expression language, it's not ideal -- it doesn't support arbitrary strings,
and tests failing in data/expr are now 4 instead of 3. The new failure:

==== $PTII/ptolemy/data/expr/test/PtParser.tcl: PtParser-15.0 Test parsing to
end of expression.
==== Contents of test case:

    set p [java::new ptolemy.data.expr.PtParser]
    catch {$p generateParseTree "1 + 2 foo"} errmsg

    # Error message is machine dependent, so we look at the first
    # two lines

    # This hack is necessary because of problems with crnl under windows
    regsub -all [java::call System getProperty "line.separator"]  $errmsg "\n"
output
    set lines [split $output "\n"]
    list [lindex $lines 0] [lindex $lines 1] [lindex $lines 2]

==== Result was:
java0x3545 {} {}
---- Result should have been:
{ptolemy.kernel.util.IllegalActionException: Error parsing expression "1 + 2
foo"} Because: {Encountered " <ID> "foo "" at line 1, column 7.}
---- PtParser-15.0 FAILED


I don't understand this yet. Typing "1 + 2 foo" without quotes into the
Expression actor fails in 2.3 and trunk without my changes, and yet the test
passes on trunk without my changes.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nceas.ucsb.edu/kepler/pipermail/kepler-dev/attachments/20121221/90eb1b2c/attachment.html>