[kepler-dev] BinaryFileReader

Efrat Jaeger efrat at sdsc.edu
Tue Oct 12 13:21:57 PDT 2004


The actor can output at each fire a byteArray of size 20000 and the EOF port
can be used to specify read completion.

Efrat

----- Original Message ----- 
From: "Dan Higgins" <higgins at nceas.ucsb.edu>
To: "Efrat Jaeger" <efrat at sdsc.edu>; "Kepler-Dev"
<kepler-dev at ecoinformatics.org>
Sent: Tuesday, October 12, 2004 1:15 PM
Subject: Re: [kepler-dev] BinaryFileReader


> Efrat,
>     Hmm - I hadn't thought much about the size of the byte array. I just
> assumed that it would be the same size as the file. But as I consider
> it, I was thinking about moving large filesa (>5MB). Maybe I need a
> special actor that streams the data directly to the file system rather
> than tries to store it in memory as a byte array that would use large
> amounts of RAM ?
>     So just do whatever chages you need. If I need some soecial
> capabilities I can always add on or creater a new actor.
>
> Dan
>
> Efrat Jaeger wrote:
>
> >Yeah, I noticed that. I couldn't use the openForReading method, as it
> >returns a bufferReader. I'm fixing it now.
> >
> >Currently, the actor returns byteArr of size 20000, I am going to make
the
> >array size variable. Are you interested in outputing the whole file
content
> >at once as well?
> >
> >Thanks,
> >
> >Efrat
> >
> >----- Original Message ----- 
> >From: "Dan Higgins" <higgins at nceas.ucsb.edu>
> >To: "Efrat Jaeger" <efrat at sdsc.edu>
> >Sent: Tuesday, October 12, 2004 11:38 AM
> >Subject: Re: [kepler-dev] BinaryFileReader
> >
> >
> >
> >
> >>Efrat,
> >>    I think that the problem may be in line 130
> >>
> >>_reader = new FileInputStream(fileOrURL.asFile());
> >>
> >>where 'asFile' assumes that the URL is a File
> >>
> >>Dan
> >>
> >>----
> >>
> >>
> >>Efrat Jaeger wrote:
> >>
> >>
> >>
> >>>Hi Dan,
> >>>
> >>>Thanks for pointing out. The current version of BinaryFileReader cannot
> >>>
> >>>
> >open
> >
> >
> >>>URLs. I'll take care of it ASAP.
> >>>
> >>>Efrat
> >>>
> >>>----- Original Message ----- 
> >>>From: "Dan Higgins" <higgins at nceas.ucsb.edu>
> >>>To: "Efrat Jaeger" <efrat at sdsc.edu>; "Kepler-Dev"
> >>><kepler-dev at ecoinformatics.org>
> >>>Sent: Tuesday, October 12, 2004 11:02 AM
> >>>Subject: [kepler-dev] BinaryFileReader
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>Hi Efrat,
> >>>>   I was looking into using your BinaryFileReader actor to read a
> >>>>binary URL (i.e. something like
> >>>>"http://www.nceas.uscb.edu/~higgins/garp/world.dxl"). Reading a remote
> >>>>http URL works fine with the PTII FileReader actor but it won't handle
> >>>>binary files. Are you aware of this?
> >>>>
> >>>>   I am interested because I was looking into using a workflow to load
> >>>>additional data for kepler; e.g. GARP example data. If we could simply
> >>>>load binary files from http servers in a workflow, we might be able to
> >>>>make the kepler installer smaller and more robust.
> >>>>
> >>>>Dan
> >>>>
> >>>>-- 
> >>>>*******************************************************************
> >>>>Dan Higgins                                  higgins at nceas.ucsb.edu
> >>>>http://www.nceas.ucsb.edu/    Ph: 805-892-2531
> >>>>National Center for Ecological Analysis and Synthesis (NCEAS)
> >>>>735 State Street - Room 205
> >>>>Santa Barbara, CA 93195
> >>>>*******************************************************************
> >>>>
> >>>>_______________________________________________
> >>>>kepler-dev mailing list
> >>>>kepler-dev at ecoinformatics.org
> >>>>http://www.ecoinformatics.org/mailman/listinfo/kepler-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>-- 
> >>*******************************************************************
> >>Dan Higgins                                  higgins at nceas.ucsb.edu
> >>http://www.nceas.ucsb.edu/    Ph: 805-892-2531
> >>National Center for Ecological Analysis and Synthesis (NCEAS)
> >>735 State Street - Room 205
> >>Santa Barbara, CA 93195
> >>*******************************************************************
> >>
> >>
> >>
>
>
> -- 
> *******************************************************************
> Dan Higgins                                  higgins at nceas.ucsb.edu
> http://www.nceas.ucsb.edu/    Ph: 805-892-2531
> National Center for Ecological Analysis and Synthesis (NCEAS)
> 735 State Street - Room 205
> Santa Barbara, CA 93195
> *******************************************************************
>




More information about the Kepler-dev mailing list