[kepler-dev] Token.toString method(s)

Christopher Brooks cxh at eecs.berkeley.edu
Fri Feb 24 15:27:43 PST 2006

Hi Dan and Jing,
Rainy day here in Spain (The rain/in Spain/falls mainly/in Spain).
So, I've pushed further with nil tokens.
The Token.nil() method has been removed, it violated immutability.
This might break code, sorry about that, but it was baaad.

I've added constructors that take Token objects to DoubleToken,
IntToken and a few other places. In principle, we could have take
an Object instead of a Token, but that seems overly general and
breaks type safety.  One issue is that DoubleToken(Token token)
is much like a copy constructor if Token is a DoubleToken.  This
is a little strange because Tokens are immutable, so there is not
much point in having a copy constructor.  This rings some bells, but
I don't have access to my logs, so I'm not sure if this is a problem.
Maybe these constructors should be further restricted to be
DoubleToken(DoubleToken token) instead of what we have now,
which is DoubleToken(Token  token).  Comments?

I started with only IntToken and DoubleToken because to update
all the types would  take quite a bit of work and I want to be sure
I have it right.  For IntToken and DoubleToken, in addition to the
constructor that takes a Token, I modified the constructor that takes
a String to properly handle "nil".  The toString() method also prints nil.

I modified ScalarToken so that most of the operations on Int
and Double tokens now properly handle nil values.  For example
anything + a nil token equals a nil token.  There is also a test that
exercises these operations.   Most operations on a nil token return
a nil token of the same type.

I modified ArrayToken so it does a reasonable job creating arrays
from strings like {nil, 1.0} or {1.0, nil, 2.0}.   Adding two
arrays where one has a nil element seems to work, though more
tests are necessary.

I started looking at DoubleMatrixToken, but one issue is that
the underlying data type is a Java double[], so I'd have to
keep a shadow list of nil tokens to make toString() print
nil instead of NaN.  Right now, it is possible to create
a DoubleMatrix with [1.0, 2.0 ; 3.0,nil], but this is printed as
[1.0, 2.0; 3.0, NaN].

About the types of nil tokens, nil tokens have a universal type,
where operations any type and a nil token result in a nil token.
Right now,  we can create a DoubleToken that is nil
and use it in operations.  We could try to force everything to
be a Token that is nil, but this would  requiring hacking the type
system, and I don't think this is what we want.  So, what we have

% set p6 [java::new ptolemy.data.DoubleToken [java::null]]
% $p6 toString
% [$p6 getType] toString

I modified SequencePlotter so if it gets a nil token, then the line for that
dataset will have a break in it (it won't be continuous).  I also modified
NonStrictTest so that if known good results has a nil token in it, then
anything will match.  More thought needs to go in to how to handle
NonStrictTest and Test.  In general, I think that a nil token is not equal
to any other token, including a nil token.  Perhaps an nil token should be
equal to all tokens?  I dunno.


Jing wrote:
> I vote 2 too. Now in order to create nil token (Except DoubleToken), I 
> need create a XXToken with arbitrary value, then use nil() method (which 
> come from Token class) to set this token to nil. This is not very 
> convenience. But DoubleToken has a constructor to set nil value, it is 
> very helpful to me to create nil DoubleToken.

----- Original Message ----- 
From: "Dan Higgins" <higgins at nceas.ucsb.edu>
To: "Christopher Brooks" <cxh at eecs.berkeley.edu>
Cc: <Kepler-dev at ecoinformatics.org>
Sent: Friday, February 17, 2006 1:13 PM
Subject: Re: Token.toString method(s)

> Hi Christopher,
>    I forget you were going on vacation. Are you staying in the Barcelona 
> area? We were there several years ago and greatly enjoyed it.  Have fun! 
> (and don't waste vacation time worrying about 'Tokens'!)  ;-)
>    I agree on you option #2.
>   I might add that I find the expression 'nil token' a bit misleading. I 
> prefer to express it as a 'token with the nil property' (or something like 
> that) since I think that makes it clearer that any type of token object 
> can be 'nil'. (i.e. nil is NOT a type like Boolean, Double, Int, etc.)
> Dan
> -----
> Christopher Brooks wrote:
>> Hi Dan,
>> I'm on vacation, but I meditated on nil tokens while in the Barcelona 
>> airport.
>> The relationship between nil tokens and type is a little fuzzy to me.  It 
>> seems that nil
>> tokens can really be of any type.  We need a way to create a nil token 
>> and use that token
>> in an expression and have the expression type check properly.  In 
>> particular, if we
>> have an array of doubles and one element is nil, we want to have a nil 
>> token that will properly
>> type check.  Tokens are immutable so I think we also need a way to create 
>> nil tokens by
>> construction.  This can get a little tricky.  I see two possible ways of 
>> creating nil tokens:
>> 1) Have a XXXToken(boolean) constructor that would set whether the token 
>> was nil
>> or not.  This is not so great for BooleanToken() though.
>> 2) Have a XXXToken(Object) constructor where if the Object parameter is 
>> null, then the
>> token is nil.  If the Object parameter is non-null, then some tokens 
>> (AWTImage) could do
>> instanceof and try to use the parameter if it was the right type. 
>> Perhaps Some tokens would just
>> throw an exception if the perameter was non-null though.
>> I'm leaning towards #2.  What do you think?
>> Also, yes, I agree, we need to extend the toString() methods to handle 
>> nil tokens.  Edward
>> was not that opposed to modifying the Token classes to properly handle 
>> nil tokens (Edward,
>> correct me if I've been drinking to much Rioja and heard you wrong.)  We 
>> should try to
>> make the checks for nil tokens very efficient though.
>> Working on the nil token is probably at the top of my todo list, so I'll 
>> be spending some time on it.
>> _Christopher
>> ----- Original Message ----- From: "Dan Higgins" <higgins at nceas.ucsb.edu>
>> To: "Christopher Brooks" <cxh at eecs.berkeley.edu>; 
>> <Kepler-dev at ecoinformatics.org>
>> Sent: Friday, February 17, 2006 9:58 AM
>> Subject: Token.toString method(s)
>>> Hi Christopher,
>>>    Jing and I were looking over the Ptolemy code for the Token class. 
>>> The 'toString' method does indeed return "nil" when there is a missing 
>>> value. However, when you look at the code for derived token classes like 
>>> IntToken or DoubleToken, each has its own version of 'toString' that 
>>> overrides the Token base class. Don't we need to change the 'toString' 
>>> method in all the derived classes to return 'nil' when the token is set 
>>> to that? [If not, it makes the RExpression actor conversion much more 
>>> difficult, especially for arrays of tokens.]
>>> Dan
>>> -- 
>>> *******************************************************************
>>> Dan Higgins                                  higgins at nceas.ucsb.edu
>>> http://www.nceas.ucsb.edu/    Ph: 805-893-5127
>>> National Center for Ecological Analysis and Synthesis (NCEAS) Marine 
>>> Science Building - Room 3405
>>> Santa Barbara, CA 93195
>>> *******************************************************************
> -- 
> *******************************************************************
> Dan Higgins                                  higgins at nceas.ucsb.edu
> http://www.nceas.ucsb.edu/    Ph: 805-893-5127
> National Center for Ecological Analysis and Synthesis (NCEAS) Marine 
> Science Building - Room 3405
> Santa Barbara, CA 93195
> *******************************************************************

More information about the Kepler-dev mailing list