[kepler-dev] NIL token

Jing Tao tao at nceas.ucsb.edu
Thu Apr 6 16:55:53 PDT 2006


Hi, Christopher:

I draft this email couple days ago:

Thanks for modification. Now the nil token works as same as before(i.g. 
kepler/workflows/test/test-eml-missing-value-field.xml works).

But it still doesn't work for the ArrayToken which has nil element (e.g. 
{nil, 1, 4}), the non-strict-test actor couldn't configure it out. The 
example is in 
kepler/workflows/test/test-eml-missing-value-column-vector.xml

Thanks.

Jing

Jing Tao
National Center for Ecological
Analysis and Synthesis (NCEAS)
735 State St. Suite 204
Santa Barbara, CA 93101

On Wed, 29 Mar 2006, Christopher Brooks wrote:

> Date: Wed, 29 Mar 2006 23:53:18 -0800
> From: Christopher Brooks <cxh at eecs.berkeley.edu>
> To: Tristan King <tristan.king at jcu.edu.au>
> Cc: kepler-dev at ecoinformatics.org
> Subject: Re: [kepler-dev] NIL token
> 
> Hi Tristan,
> About empty arrays, what's the type of the elements of an empty array?
> Hint: it is the sound of one hand clapping. :-)
>
> (Sorry, it is late here in California)
>
> You could try disabling the check in ArrayToken for empty arrays
> and see what happens when you create an empty array and try
> to perform an operation on it.  My guess is that the type
> system will not be able to resolve the type of elements.
>
> I think the issue is that the way the type system works depends on
> having some element of the array have a type.  If there are no
> elements, then it is unclear what the type of the elements would be.
>
> I think we need to differentiate between ArrayTokens that have
> one or more nil elements, but also have non-nil elements and
> ArrayTokens that are themselves nil, that is, they have no non-nil
> elements.  Another case is where an array has one or more nil
> elements and non non-nil elements
>
> In other words:
> {1, nil} or {nil, 1}   An array of 2 IntTokens, one of which is nil
> {nil}		       Is this an empty array or an array of size 1?
> {nil, nil}	       This would be an array of size 2.
>
>
> I'm starting to see nil tokens as the universal token.
>
> I just checked in some changes so that some of the Tokens have their own NIL
> token. Token.NIL, DoubleToken.NIL, IntToken.NIL, LongToken.NIL and all
> of these NILs are of a new type, "NilType".  NilType is losslessly
> convertible to (from?) DoubleType, IntType, LongType etc.
>
> I think we need these Token specific NILs so that the convert()
> method will work.
>
> For example, IntToken.convert() looks like:
>
>    public static IntToken convert(Token token) throws IllegalActionException {
>
> If the Token argument is Token.NIL, we can't just do:
>        if (token == null || token.isNil()) {
>            return (IntToken)Token.NIL;
>        }
> because Token.NIL is not an IntToken and we will get a cast exception
> at runtime.
>
> Instead, we must do:
>        if (token == null || token.isNil()) {
>            return Intoken.NIL;
>        }
>
> Anyway, if we figure out how the basetypes work, then we can
> figure out how to better handle empty arrays.
> It could be that an ArrayToken with no elements is a nil
> ArrayToken {nil}
>
> Anyway, I checked in a modified version of Jing's example as
> $PTII/ptolemy/actor/lib/test/auto/NilTokenTypeTest.xml
> This test has a Const with value nil connected to a Test.
> However, the output port of the Const is set to type int by
> using the configure ports dialog
>
> With my most recent changes, this now test passes.
>
> [There seems to be a bug in the dialog where setting the port back from
> int to nothing does not work, the error message still says int]
>
> I'll need to do some cleanup tomorrow on my most recent changes.
>
>
> _Christopher
>
> --------
>
>    How about ArrayTokens? I've got a problem where my workflow goes into
>    deadlock if an ArrayToken with no values (which of course can't exist)
>    needs to be sent. And as you should know (or if i'm doing something
>    wrong), a Token can't be cast to an ArrayToken.
>
>    Currently I just have a 2nd output port which sends a boolean indicating
>    if there are no tokens being sent on the 1st port and have my other
>    actors check this port before trying to get values from the 1st port.
>    This works, but I find the extra relations needed can make the workflow
>    look quite messy, and i don't feel that having programming logic like
>    this in the workflow is good practice.
>
>    I've not been able to find a good explanation for why ArrayTokens can't
>    be constructed with empty arrays, if someone knows the reason for this
>    can I get an explanation please?
>
>    Christopher Brooks wrote:
>    > Hi Jing,
>    > Thanks for the stack trace.  I am tied up with getting Ptolemy II
>    > ready shipping, so time is short.  Also, I wanted Edward to see the
>    > stack trace.  If I had a small test, I could check it in to the tree.
>    > Yep, I should start up kepler . . .
>    >
>    > The problem is that TypeLattice.compare() returning
>    > either CPO.HIGHER or CPO.INCOMPARABLE.
>    >
>    >     protected void _checkType(Token token) throws IllegalActionException
>   {
>    >         int compare = TypeLattice.compare(token.getType(), _resolvedType)
>   ;
>    >
>    >         if ((compare == CPO.HIGHER) || (compare == CPO.INCOMPARABLE)) {
>    >             throw new IllegalActionException(this,
>    > Line 741:             "Run-time type checking failed. Token " + token
>    >                             + " with type " + token.getType()
>    >                             + " is incompatible with port type: "
>    >                             + getType().toString());
>    >         }
>    >     }
>    >
>    > One hack would be to modify the code so that we check to see
>    > if the Token is nil before calling TypeLattice.compare().
>    > This seems a bit brute force and would require changing at least
>    > 112 calls to TypeLattice.compare() in the Ptolemy tree.
>    >
>    > Another idea would be for each Token to have its own NIL token.
>    > This seems inelegant.
>    >
>    > Perhaps the NIL token needs to have a getType() method that returns
>    > something that the type lattice will properly type check.  A NIL
>    > token is the univesal token, it can be used anywhere.
>    >
>    > Modifying ptolemy.data.Token so it redefines getType might work:
>    >
>    >     /** A token that represents a missing value.
>    >      *  Null or missing tokens are common in analytical systems
>    >      *  like R and SAS where they are used to handle sparsely populated d
>   ata
>    >      *  sources.  In database parlance, missing tokens are sometimes call
>   ed
>    >      *  null tokens.  Since null is a Java keyword, we use the term "nil"
>   .
>    >      *  The toString() method on a nil token returns the string "nil".
>    >      */
>    >     public static final Token NIL = new Token() {
>    >             /** Return the type of this token.
>    >              *  @return BaseType.UNKNOWN;
>    >              */
>    >             public Type getType() {
>    >                 return BaseType.UNKNOWN;
>    >             }
>    >         };
>    >
>    > However, I think we need to instead add a new type, which is the
>    > type that is specific to NIL.  This type is in some ways in
>    > comparable, but operations such as add can be performed on it.
>    > TypeLattice.compare() would need to be modified so if one
>    > of the arguments is the newly defind NIL type, then it returns
>    > ptolemy.graph.CPO.SAME.
>    > This seems like a scary change that could affect performance.
>    >
>    > If you can send me a small test case that I can add to the Ptolemy
>    > tree, I can see about hacking something in.
>    >
>    > Comments?
>    >
>    > _Christopher
>    >
>    > --------
>    >
>    >     Hi, Christopher:
>    >
>    >     The test workflow is alreay in
>    >     kepler/workflows/test/test-eml-missing-value-field.xml. You may run i
>   t and
>    >     to see what will happen. The stack trace is:
>    >
>    >     ptolemy.kernel.util.IllegalActionException: Run-time type checking fa
>   iled.
>    >     Token nil with type general is incompatible with port type: double
>    >        in .test-eml-missing-value-field.Missing Value.Column2
>    >      	at ptolemy.actor.TypedIOPort._checkType(TypedIOPort.java:741)
>    >      	at ptolemy.actor.TypedIOPort.send(TypedIOPort.java:463)
>    >      	at
>    >     org.ecoinformatics.seek.datasource.eml.eml2.Eml200OutputTypeField.fir
>   e(Eml2
>    >    00OutputTypeField.java:103)
>    >      	at
>    >     org.ecoinformatics.seek.datasource.eml.eml2.Eml200DataSource.fire(Eml
>   200Dat
>    >    aSource.java:552)
>    >      	at ptolemy.actor.AtomicActor.iterate(AtomicActor.java:309)
>    >      	at
>    >     ptolemy.actor.sched.StaticSchedulingDirector.fire(StaticSchedulingDir
>   ector.
>    >    java:170)
>    >      	at ptolemy.actor.CompositeActor.fire(CompositeActor.java:331)
>    >      	at ptolemy.actor.Manager.iterate(Manager.java:613)
>    >      	at ptolemy.actor.Manager.execute(Manager.java:322)
>    >      	at ptolemy.actor.Manager.run(Manager.java:985)
>    >      	at ptolemy.actor.Manager$3.run(Manager.java:1026)
>    >
>    >
>    >     Thanks,
>    >
>    >     Jing
>    >
>    >     Jing Tao
>    >     National Center for Ecological
>    >     Analysis and Synthesis (NCEAS)
>    >     735 State St. Suite 204
>    >     Santa Barbara, CA 93101
>    >
>    >     On Wed, 29 Mar 2006, Christopher Brooks wrote:
>    >
>    >     > Date: Wed, 29 Mar 2006 11:04:25 -0800
>    >     > From: Christopher Brooks <cxh at eecs.berkeley.edu>
>    >     > To: Jing Tao <tao at nceas.ucsb.edu>
>    >     > Cc: kepler-dev at ecoinformatics.org
>    >     > Subject: Re: [kepler-dev] NIL token
>    >     >
>    >     > Hi Jing,
>    >     >
>    >     > Good question, I was afraid there might be issues surrounding
>    >     > Nil tokens and types.  My previous design had nil-ness being a
>    >     > property of the token, so we had a DoubleToken that was of
>    >     > type double and was nil.  Now we have just one token which is nil.
>    >     >
>    >     > Can you construct a small model that has a similar failure?
>    >     >
>    >     > Also, what's the stack trace of the current failure?
>    >     > Perhaps where the types are being checked needs to be updated to
>    >     > handle nilness
>    >     >
>    >     > _Christopher
>    >     >
>    >     > --------
>    >     >
>    >     >    hi, Christopher:
>    >     >
>    >     >    According to the changes you made in ptolemy about nil token, I
>   made t
>    >    he
>    >     >    changes in kepler for missing value too.
>    >     >
>    >     >    Since currently we only have one public static variable NIL in T
>   oken c
>    >    lass
>    >     >    (NIL is a generic type and there is no any other nil in sub toke
>   n
>    >     >    classes), I think every missing value element should be replaced
>    by th
>    >    e
>    >     >    NIL token no matter what the data type it is. But when I run the
>    workf
>    >    low
>    >     >    test, I got some error:
>    >     >
>    >     >    Workflow workflows/test/test-ecogrid-eml-gce-data.xml FAILED:
>    >     >    [java] ptolemy.kernel.util.IllegalActionException: Run-time type
>    check
>    >    ing
>    >     >    failed. Token nil with type general is
>    >     >    incompatible with port type: int
>    >     >
>    >     >    So it seems the NIL token is not compatible to int port type. Di
>   d I mi
>    >    ss
>    >     >    something? How can I make the NIL token to compatible int port t
>   ype?
>    >     >
>    >     >    Thanks,
>    >     >
>    >     >    Jing
>    >     >
>    >     >    Jing Tao
>    >     >    National Center for Ecological
>    >     >    Analysis and Synthesis (NCEAS)
>    >     >    735 State St. Suite 204
>    >     >    Santa Barbara, CA 93101
>    >     >    _______________________________________________
>    >     >    Kepler-dev mailing list
>    >     >    Kepler-dev at ecoinformatics.org
>    >     >    http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/ke
>   pler-d
>    >    ev
>    >     > --------
>    >     >
>    >     >
>    > --------
>    > _______________________________________________
>    > Kepler-dev mailing list
>    > Kepler-dev at ecoinformatics.org
>    > http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>    >
>
>    --
>    Tristan King                            | Ph: (07) 4781 6911
>    DART project team                       | Email: Tristan.King at jcu.edu.au
>    James Cook University                   | Web: http://dart.edu.au
>    Townsville QLD 4814                     | http://plone.jcu.edu.au/dart/
>    Australia                               |
>    _______________________________________________
>    Kepler-dev mailing list
>    Kepler-dev at ecoinformatics.org
>    http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
> --------
> _______________________________________________
> Kepler-dev mailing list
> Kepler-dev at ecoinformatics.org
> http://mercury.nceas.ucsb.edu/ecoinformatics/mailman/listinfo/kepler-dev
>
>
>


More information about the Kepler-dev mailing list