[kepler-dev] [Ptolemy] Re: Bug 2240: add support for null values to data

[Christopher Brooks]

Hi Adam,
Thanks for taking a look at my fix.

The fix does go fairly deep in to the heart of Ptolemy, but it
does not extensively modify the current functionality.
Token gets a boolean field, a setter method and a getter method.
Constant gets a couple of lines. 

The actor I propose strips out missing data tokens so that 
downstream actors don't have to deal with missing data.
However, this corrupts the data because now I have a set
that had missing elements but now the set "seems" to be complete,
but is in fact missing points.  A good example is that
if we are measuring temperature in Death Valley and we have
only one data point for the month, it is not really fair to say
the average monthly temperature is the value of that point.

I think that in this case, if we really wanted to know the monthly
average temperature, we should use a timed domain, like DE. 

I'm not sure I see why we would want to have an interface here.
I suppose we could try to use the type system to include types
that have the possibility of a nil token and then make
intelligent choices about casting from one type to another.

If the type system understood nil compatible types then it could help
with other strategies for handling missing types.

So, at this point I think I need feedback from the Kepler group
about the model I created and about how we should handle nil
tokens.  A very small model of a real world example with a
written description of what the output data would look like would
help. 

_Christopher

--------

    My comment is that this fix seems to go too deep in the code hierarchy. 
      I don't see any fundamental problem with using nil tokens for this 
    problem, but it seems to me that not all token types should support nil 
    tokens.  If you don't want to deal with nil tokens in your design, why 
    should some upstream actor, which someone else designed, be able to 
    produce nil tokens?   Also, I think there's probably a good argument to 
    use DE for situations where a sensor may not have a value at a 
    particular time.
    
    Nil tokens may still be useful in such a situation.  Within the DE 
    model, we could create a "listener" which listens for values from a 
    sensor.  If it gets a value by time t, it outputs (t, v); otherwise, it 
    outputs (t, nil).  (Although, I think DE already supports an absent 
    value, so this may be redundant.)
    
    If we do decide on the nil token solution, perhaps these methods should 
    be in a Java interface somewhere.  Then you can easily create token 
    types that allow for nil by extending a predefined token type and 
    implementing the interface.
    
    Cheers,
    Adam
    
    Christopher Brooks wrote:
    > Ok, I hacked up the following:
    > 
    > * Token.java now has nil() and isNil() methods
    >   I went with nil because null is a Java keyword.
    > 
    >   The term "missing" is rather appealing since the Token.toString()
    >   method usually prints out "present".  So, it could be modified to
    >   print out "missing".  However, I feel that someone is likely to have
    >   a parameter named "missing" somewhere.  nil seems safer.
    > 
    >   This is not cast in stone, comments are welcome.
    > 
    >   These methods have tests. 
    >   
    > * data/expr/Constants.java now defines a "nil" constant which
    >   is a Token that has the nil() method:
    >         ptolemy.data.Token nil = new ptolemy.data.Token();
    >         nil.nil();
    >         _table.put("nil", nil);
    >   Thus, one can now create expressions that have nil in them
    > 
    >   The "nil" constant has a test in data/expr/test/PtParser.tcl
    > 
    > * actor/lib/RemoveNilTokens.java:
    >   A new actor that reads its input and discards any nil tokens
    >   in the fire() method.  It might be better to do this in prefire()
    >   This actor is available in the "More Libraries" -> "Esoteric" section.
    > 
    >   Note that this actor should not be used in SDF.
    >   
    >   No tests yet.
    > 
    > * domains/pn/demo/RemoveNilTokens/RemoveNilTokens.xml
    >   A model that uses RemoveNilTokens.
    >   I think I'm terminating the PN process poorly.  I get 7 outputs
    >   instead of 5.  I could use some help here.
    >   Also, I have to explicitly set the type of output of the RemoveNilToken
   s
    >   actor.
    > 
    > 
    > The model looks like:
    > 
    >           Bool Switch
    > Ramp----> | 
    >           |----------------> RemoveNilTokens ---------> Display
    > Const     |                                      |
    > that ---> |                                       --> "Code that stops PN
   "
    > produces  _
    > nil       ^
    >           |
    >           |
    > Bernoulli -
    > 
    > 
    > So, now we have a straw man in PN to try out.  
    > 
    > Questions:
    > 1) Would something like this meet the needs of the Kepler group?
    > 2) Should RemoveNilTokens do something in prefire()?
    >    I'm not sure if I can get access to the Token and call Token.isNil()
    >    in prefire().
    > 3) Is "nil" an ok name?
    > 4) How do I get PN to terminate properly after I see 5 non-nil tokens?
    > 
    > _Christopher
    > 
    > Edward wrote:
    > --------
    > 
    > 
    >     At 08:16 AM 12/13/2005 -0800, Shawn Bowers wrote:
    >     >One option for SDF would be to literally "propagate" the missing-val
   ued
    >     >tokens, instead of "dropping" them (as in PN).   Then, it seems the
    >     >consumption/production rates at least wouldn't be violated.  Doing t
   his
    >     >correctly might be a bit tricky, but could probably perform this bas
   ed on
    >     >the token production/consumption rates --
    >     
    >     In SDF, before any actor is fired, its prefire() method is called.
    >     For all our actors that require input data, if prefire() will
    >     return false if there is no input token, and the actor will not
    >     be fired. Consequently, its outputs will also have no token...
    >     
    >     So SDF already does this propagation...
    >     
    >     Edward
    >     
    >     
    >     
    >     ------------
    >     Edward A. Lee
    >     Professor, Chair of the EE Division, Associate Chair of EECS
    >     231 Cory Hall, UC Berkeley, Berkeley, CA 94720
    >     phone: 510-642-0253 or 510-642-0455, fax: 510-642-2845
    >     eal at eecs.Berkeley.EDU, http://ptolemy.eecs.berkeley.edu/~eal  
    >     
    >     _______________________________________________
    >     Ptolemy maillist  -  Ptolemy at chess.eecs.berkeley.edu
    >     http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
    > --------
    > _______________________________________________
    > Ptolemy maillist  -  Ptolemy at chess.eecs.berkeley.edu
    > http://chess.eecs.berkeley.edu/ptolemy/listinfo/ptolemy
--------