[kepler-dev] NIL token

Tristan King tristan.king at jcu.edu.au
Tue Apr 4 23:27:08 PDT 2006


Hi Christopher,

After a lot of messing around to get kepler to compile over the latest 
ptolemy cvs (in the end i had to remove the ClusterW stuff and the 
Director.java file from $KEPLER/src/exp/ directory, as well as add a few 
methods to FSMActor.java in ptolemy)... anyways, my test workflow worked 
exactly how i wanted it to work. so it looks like you've solved the 0 
length array problem for now :).

I am on the same level of confusion as you with the difference between 
an empty arraytoken and a nil arraytoken. From my understanding of the 
definition of a nil, a nil arraytoken would specify an array should (or 
could) exist, but is currently missing. But it may not be true to say 
that a zero size array is missing, since it's not really missing, it 
just has a size of zero.

Currently when i check for a zero size arraytoken i check the length() 
of the array rather than it's nil-ness, since that's what i'm looking 
for, and that works perfectly for me. I'm still concerned about the 
token being considered nil, but it wont keep me up at night :).

I think it comes into the same league as the stringtoken making nulls 
into an empty string, the difference is currently so subtle that it will 
probably never effect anyone.

It may though, in the future, so maybe it is worth the time to try and 
specifically define the differences before the code is so deeply 
embedded that changing it would cause major issues.

You mentioned in a previous email that you used to have nil-ness as a 
property of the token rather than a state like the current 
implementation. What were the problems with this system?

Thanks
--Tristan

Christopher Brooks wrote:
> Hi Tristan,
> I just checked in a change to ptolemy.data.type.TypeLattice so that
> the niltype is convertible to an UnsignedByte.  Yesterday I checked
> in changes that support nil in UnsignedByte and empty ArrayTokens.
> 
> I'm not so sure I understand the difference between and empty
> ArrayToken and a nil ArrayToken, so there could be problems there.
> Perhaps it is a little like the difference betweeen an empty
> StringToken a nil StringToken.
> 
> $PTII/ptolemy/actor/lib/test/auto/NilTokenTypeTest.xml
> contains the simplistic type tests that I'm using to verify these type
> changes.  The model consists of Const actors connected to Test actors,
> where the Consts have the value nil and the type of the port is set
> to int, double, long, or unsignedByte.  The unsignedByte test was 
> failing until my most recent changes to TypeLattice.  I also added
> tests that test empty arrays in a similar fashion.
> 
> If, after my most recent change to TypeLattice, empty arrays are still
> not working for you, please send me a really simple model or code
> fragment so I can construct a test and fix it.  I suspect that
> my most recent change will do the trick, but it could be I missed the
> bug that is affecting your model and fixed a different bug.
> 
> I spent some time yesterday adding tests to the testsuite for nil
> tokens and discovered some bugs, like that we need an UnsignedByte nil
> token.  However, more tests are needed.  This work will probably not
> happen until tomorrow.
> 
> _Christopher
> 
> 
> 
> 
> --------
> 
>     Hi Christoper,
>     
>     I understand the need for the nil token now, thanks for clearing that up.
>     
>     I take it you're still in the middle of getting the ArrayToken nils to 
>     work? I just want to check to make sure I'm not doing something wrong or 
>     there's a bug.
>     
>     I am still getting a conversion error when trying to send a 0 element array
>    .
>     
>     ptolemy.kernel.util.IllegalActionException: Run-time type checking 
>     failed. Token {} with type {nil} is incompatible with port type: 
>     {unsignedByte}
>        in .<Unnamed Object>.Binary File Reader (dart).output
>     	at ptolemy.actor.TypedIOPort._checkType(TypedIOPort.java:741)
>     	at ptolemy.actor.TypedIOPort.send(TypedIOPort.java:463)
>     	at org.dart.BinaryFileReader.fire(BinaryFileReader.java:117)
>     	at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:309)
>     	at 
>     ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDirector.
>    java:170)
>     	at ptolemy.actor.CompositeActor.fire(CompositeActor.java:331)
>     	at ptolemy.actor.Manager.iterate(Manager.java:613)
>     	at ptolemy.actor.Manager.execute(Manager.java:322)
>     	at ptolemy.actor.Manager.run(Manager.java:987)
>     	at ptolemy.actor.Manager$3.run(Manager.java:1028)
>     
>     the port is assigned a type via the following code:
>     output.setTypeEquals(new ArrayType(BaseType.UNSIGNED_BYTE));
>     
>     Thanks
>     --Tristan
>     
>     Christopher Brooks wrote:
>     > Hi Tristan,
>     > I think I have most of the implementation of an empty array working.
>     > 
>     > I agree that having new StringToken(null) create an empty string is
>     > probably wrong.  We could change it, but I'm not sure if it would buy
>     > us much, and it might break things.
>     > 
>     > The way I'm thinking of the nil token is that it represents missing
>     > data that could have any value.  Certain operations work on nil
>     > values, for example any value added to a nil returns a nil.  However,
>     > operations like average should return regular numeric data, even if
>     > one or more elements of the average are nil.  However, we do get
>     > increased uncertainty in the average, and we don't have any way
>     > of showing that.
>     > 
>     > I took on the nil token task because it was at the top of the bug list
>     > for Kepler and it seemed like it needed to be implemented by someone
>     > who would make changes to the Ptolemy tree.  I somewhat understand why
>     > we need to represent missing data, but I make no claims as to
>     > whether this is the right implementation.  I'm sure that some people
>     > wish Java had better support for missing data.
>     > 
>     > The relationship between nil and null is subtle.
>     > 
>     > Anyway, thanks for your comments, I may have more to say later.
>     > 
>     > _Christopher
>     > 
>     > 
>     > 
>     > --------
>     > 
>     >     Ok this is a bit of a rant, and since i probably don't know even 5% o
>    f 
>     >     the internal workings of ptolemy/kepler i may be misguided in a lot o
>    f 
>     >     this rant so please correct me anywhere i need correction :).
>     >     
>     >     Maybe we need to take a step back and look at what a Token actually i
>    s, 
>     >     and what we're actually trying to achieve from a nil token. From the 
>     >     javadoc in the Token class a Token is simply a container for standard
>     
>     >     java objects and native types.
>     >     
>     >     So, What are we trying to achieve with a nil token? There must be a v
>    ery 
>     >     good rationale to why we need such a mechanism, but i can't for the l
>    ife 
>     >     of me figure out what it is. Are we trying to signify the lack of a 
>     >     variable, or a variable that has a null value?
>     >     
>     >     If we are talking about the lack of a variable, i.e an empty containe
>    r 
>     >     then for one, I think nil is the wrong syntax for this.
>     >     
>     >     Now, should an empty container be allowed? Sure. But only for use as 
>    a 
>     >     trigger (i.e. the EventToken). I don't think it should be used to 
>     >     signify the lack of a value which may or may not exist.
>     >     
>     >     If we are trying to say the value in the container is null. It just 
>     >     doesn't make sense.
>     >     
>     >     If we look at the native types like ints doubles booleans etc, they 
>     >     cannot be null. They either exist, or they don't. So if the expressio
>    n 
>     >     "int i = null;" is invalid in java, how can we say "IntToken i = 
>     >     Token.NIL;" (or IntToken.NIL)? I think forcing a specific condition t
>    o 
>     >     be handled as null can limit the programmer. Especially if they need 
>    to 
>     >     use the value Integer.MAX_VALUE and not have other actors view it as 
>    null.
>     >     
>     >     So what about Objects that can be assigned to null?
>     >     
>     >     What's wrong with something like:
>     >     Object o = null;
>     >     ObjectToken ot = new ObjectToken(o);
>     >     
>     >     or
>     >     Object[] o = null;
>     >     ArrayToken at = new ArrayToken(o);
>     >     
>     >     Sure, the null values stored in the Tokens may cause exceptions, but 
>    i 
>     >     think those exceptions should be handled by the programmer rather tha
>    n 
>     >     avoided. I don't see why the container itself needs to know anything 
>     >     about the object it contains (except for the type).
>     >     
>     >     I feel the current direction is moving more away from the Tokens bein
>    g 
>     >     transport containers to actually being the variables themselves.
>     >     
>     >     For example the StringToken container converts Strings which are null
>     to 
>     >       Strings which have a value "". This doesn't fit with the definition
>     of 
>     >     a container. It's not just transporting the objects, it's manipulatin
>    g 
>     >     them as well. There is a very distinct difference between a String se
>    t 
>     >     to null and "", this conversion adds ambiguity to the String object.
>     >     
>     >     Another example is the ArrayToken container. When the Token is though
>    t 
>     >     of as just being a container for arrays it makes no sense that empty 
>     >     arrays cannot be contained inside. Any array that is valid in java 
>     >     should be containable inside the token.
>     >     
>     >     As with my problem, I'm not trying to alert the other actors of the l
>    ack 
>     >     of a token, I want to send them an array that has no values. In java 
>     >     there is a distinct difference between: "Object[] o = null;" and 
>     >     "Object[] o = Object[0];". The ArrayToken container should just 
>     >     transport my array object and not be concerned about how long it is e
>    tc.
>     >     
>     >     What about the type of the array? maybe it should be specified by the
>     
>     >     programmer (the same way that ports have to be set), rather than bein
>    g 
>     >     interpreted from the elements in the array, and have the base class a
>    s 
>     >     BaseType.OBJECT incase no type is specifically assigned. i've not put
>     a 
>     >     lot of thought into this just yet, i'm sure i'll be doing a lot over 
>    the 
>     >     weekend :)
>     >     
>     >     In conclusion, 1. Scrap the nil token idea, and think more of how you
>     
>     >     would represent the missing values in a normal java program, and how 
>    you 
>     >     could contain that representation in a Token. And 2. the ArrayToken (
>    and 
>     >     possibly some of the other tokens) need to be reworked so they act mo
>    re 
>     >     like containers.
>     >     
>     >     Anyone else have any thoughts (anything to add, or critisisms) about 
>    the 
>     >     things i've brought up?
>     >     
>     >     --Tristan
>     >     
>     >     Edward A. Lee wrote:
>     >     > At 11:53 PM 3/29/2006, Christopher Brooks wrote:
>     >     >> I just checked in some changes so that some of the Tokens have the
>    ir 
>     >     >> own NIL
>     >     >> token. Token.NIL, DoubleToken.NIL, IntToken.NIL, LongToken.NIL and
>     all
>     >     >> of these NILs are of a new type, "NilType".  NilType is losslessly
>     >     >> convertible to (from?) DoubleType, IntType, LongType etc.
>     >     >>
>     >     >> I think we need these Token specific NILs so that the convert()
>     >     >> method will work.
>     >     > 
>     >     > This looks like the right solution to me...
>     >     > 
>     >     > An awkwardness is that every token class has to have a NIL
>     >     > and an appropriate convert method, but this awkwardness is
>     >     > only visible to token class developers.
>     >     > 
>     >     > I think that having an empty array be ArrayToken.NIL
>     >     > might be a very good solution too... We could then augment
>     >     > the expression language so that the string "{}" returns
>     >     > an instance of this.  It's element type is NilType.
>     >     > Since this is losslessly convertible to every type
>     >     > (including ArrayToken, by the way), then I think it should
>     >     > just work!
>     >     > 
>     >     > Amazing...  There may be a type system paper in this...
>     >     > 
>     >     > Edward
>     >     > 
>     >     > 
>     >     > 
>     >     > ------------
>     >     > Edward A. Lee
>     >     > Professor, Chair of the EE Division, Associate Chair of EECS
>     >     > 231 Cory Hall, UC Berkeley, Berkeley, CA 94720
>     >     > phone: 510-642-0253 or 510-642-0455, fax: 510-642-2845
>     >     > eal at eecs.Berkeley.EDU, http://ptolemy.eecs.berkeley.edu/~eal 
>     >     > 
>     >     
>     >     -- 
>     >     Tristan King                            | Ph: (07) 4781 6911
>     >     DART project team                       | Email: Tristan.King at jcu.edu
>    .au
>     >     James Cook University                   | Web: http://dart.edu.au
>     >     Townsville QLD 4814                     | http://plone.jcu.edu.au/dar
>    t/
>     >     Australia                               |
>     > --------
>     > 
>     
>     -- 
>     Tristan King                            | Ph: (07) 4781 6911
>     DART project team                       | Email: Tristan.King at jcu.edu.au
>     James Cook University                   | Web: http://dart.edu.au
>     Townsville QLD 4814                     | http://plone.jcu.edu.au/dart/
>     Australia                               |
> --------
> 

-- 
Tristan King                            | Ph: (07) 4781 6911
DART project team                       | Email: Tristan.King at jcu.edu.au
James Cook University                   | Web: http://dart.edu.au
Townsville QLD 4814                     | http://plone.jcu.edu.au/dart/
Australia                               |


More information about the Kepler-dev mailing list