[kepler-dev] Reading files in Kepler

[Christopher Brooks]

Hi Michal,
You can create a Token and a type that represents a stream and
pass that around.  See ptolemy.actor.lib.security.KeyToken for an example.
See also
http://ptolemy.eecs.berkeley.edu/ptolemyII/ptIIfaq.htm#AddAType

The primary issue I see is that it is not clear how much data to
read from the stream.  If your data consists of multiple lines
or fields, you could read it and generate a RecordToken and
process that.

_Christopher

Michal Owsiak wrote:
> Hello Christopher,
> 
> thanks for the answer
> 
>> Hi Michal,
>> Try LineReader or ExpressionReader, the will avoid
>> the problem of reading in the entire file.
> 
> Ok, I will try these.
> 
>> See also:
>> http://ptolemy.eecs.berkeley.edu/ptolemyII/ptIIfaq.htm#reading%20data%20from%20files?
> 
> Thanks, I will take a look at that as well.
> 
>> FileReader reads the entire file and sends the contents to its
>> output.  A BufferedFileReader would be an interesting actor, but
>> the semantics might be a little unusual because it would not be clear
>> how much data would be read and sent.  The buffer size could be
>> specified as a parameter and the BufferedFileReader could send a string
>> of at most the buffer size.
> 
> That's why we thought that passing InputStream/OutputStream objects
> between actors might be a solution in here. This would allow us to
> retrieve the stream object inside one actor (A) (the one responsible for
> all low level  data access computations), while actor (B) that consumes
> data can simply read/write the stream - it doesn't have to think about
> low level data access.
> 
> data source location --> (A) --- InputStream ---> (B) ---- transformed
> data ----> (C)
> 
> But I am not sure whether this solution is compatible with workflow
> paradigm. The case is, that we manipulate very large data sets.
> 
> Cheers
> 
> Michal
> 

-- 
Christopher Brooks (cxh at eecs berkeley edu) University of California
CHESS Executive Director                      US Mail: 337 Cory Hall
Programmer/Analyst CHESS/Ptolemy/Trust        Berkeley, CA 94720-1774
ph: 510.643.9841 fax:510.642.2718	      (Office: 545Q Cory)
home: (F-Tu) 707.665.0131 (W-F) 510.655.5480