[kepler-users] Help understanding Kepler paradigm for batch processing images

[Ustun Yildiz]

Just a quick question, are you using PN director ?



On Sun, Jul 12, 2009 at 10:41 PM, Jeremy
Douglass<jeremydouglass at gmail.com> wrote:
> Ustun,
>
> Thanks. Because I'm just trying to batch process the files, I need to
> produce a series of whatever Image Converter will consume -- which I
> assume (?) based on the 03 example that ships with Kepler is just a
> plain file path e.g. "/User/test/file1.PNG". Essentially, I want Image
> Converter receive 5 file path strings and run 5 times.
>
> What OS are you running Kepler on? If the Directory Listing returns a
> different data structure depending on the OS, perhaps I should be file
> this as a bug?
>
> Right now I'm trying to adding a "name" column to the array before
> feeding it to Record Disassembler by using the Concatenate Arrays
> actor, but no luck so far....
>
> -- Jeremy
>
>
> On Sun, Jul 12, 2009 at 10:07 PM, Ustun Yildiz<yildiz at cs.ucdavis.edu> wrote:
>> Hi:
>>
>> I am sorry but I guess I gave you a wrong advise in my previous message.
>> I have the impression that Directory Listing-->Array to Sequence give
>> you the elements as separate tokens. Your number 1) guess is correct
>> below.
>>
>> I tried to put Display after Array to Sequence, I have a name tag to
>> filter with Record Dissasemler. I guess different computers generate
>> different listings. Looking at your output, you just need the name of
>> the files right? and not complete directory/file_name elements ?
>>
>> -Ustun.
>>
>> On Sun, Jul 12, 2009 at 9:52 PM, Jeremy
>> Douglass<jeremydouglass at gmail.com> wrote:
>>> Thank you Ustun -- I'm trying a Record Dissasembler with an added
>>> "name" output port now, but it isn't working.
>>>
>>> When I add just a Record Disassembler where you suggested (Directory
>>> Listing ---> Array to Sequence ---> Record Disassembler), I'm getting
>>> "Exception: ptolemy.data.stringToken." Once I use the GUI to add an
>>> output port named "name" to the Disassembler I get the more verbose
>>> error "Type resolution failed because of an error during type
>>> inference in .test, because: Invalid type for input port in
>>> .test.Record Disassembler." Adding the Disassembler directly to the
>>> Directory Listing gives me similar errors, only the exception is
>>> arrayToken instead of stringToken.
>>>
>>> Right now I'm wildly guessing that the problem is either:
>>> 1) perhaps my directory listing results don't actually come with a
>>> "name" element column, and so there is no name element for the
>>> Disassembler to filter on -- as this apparently works for you, perhaps
>>> returning a single column array is OS X specific behavior of Directory
>>> listing, and perhaps there is a way to add this column myself through
>>> an array operation? ...or
>>> 2) Perhaps you didn't mean I should use Array to Sequence, but rather
>>> Array to Elements in some fashion?
>>>
>>> Any advice greatly appreciated. For your ref, here is what I'm seeing
>>> if I run Directory Listing -> Display:
>>>
>>> {"/Users/test/Picture 1.PNG", "/Users/test/Picture 2.PNG",
>>> "/Users/test/Picture 3.PNG", "/Users/test/Picture 4.PNG",
>>> "/Users/test/Picture 5.PNG"}
>>>
>>> ...and if I run Directory Listing -> Array to Sequence -> Display:
>>>
>>> /Users/test/Picture 1.PNG
>>> /Users/test/Picture 2.PNG
>>> /Users/test/Picture 3.PNG
>>> /Users/test/Picture 4.PNG
>>> /Users/test/Picture 5.PNG
>>>
>>> -- Jeremy
>>>
>>>
>>> On Sat, Jul 11, 2009 at 1:06 PM, Ustun Yildiz<yildiz at cs.ucdavis.edu> wrote:
>>>> I have the impression that you try to apply an operation (actor) to
>>>> the elements in a directory. In order to capture the names of such
>>>> elements and pipeline them into subsequent actors, I use
>>>>
>>>> Directory Listing ---> Array to Sequence ---> Record Dissasembler
>>>>
>>>> The first takes a directory name and the last outputs single names (as
>>>> many as the elements). The purpose of the Record Dissasembler is to
>>>> capture "name" element in the output of Array to Sequence actor. You
>>>> have to change the output port of Record Dissasembler to "name".
>>>>
>>>> Ustun.
>>>>
>>>>
>>>>
>>>>
>>>> On Sat, Jul 11, 2009 at 12:51 PM, Jeremy
>>>> Douglass<jeremydouglass at gmail.com> wrote:
>>>>> I'm a new Kepler and struggling to apply a single actor (e.g. Image
>>>>> Converter) to a batch of objects (e.g. a directory of images, as
>>>>> represented by the array generated by Directory Listing).
>>>>>
>>>>> I've read the documentation and searched the forums, and tried
>>>>> modifying examples, but I'm not understanding something paradigmatic,
>>>>> and would appreciate a tip: how do you pipe an array of files through
>>>>> single-file actors?
>>>>>
>>>>> EXAMPLE:
>>>>>
>>>>> In Kepler 1.0.0, 03-ImageDisplay.xml uses an SDC director, and specifies:
>>>>>   Constant Image Filename --> Image Converter --> ImageJ
>>>>>
>>>>> In an attempt to converter a *directory* of images, I tried adding a
>>>>> Directory Listing actor:
>>>>>   Constant Dirname --> Directory Listing --> Image Converter --> ImageJ
>>>>>
>>>>> ...however this generates an error, as Image Converter will only act
>>>>> on a string, not an array. So I tried
>>>>>   Constant Dirname --> Directory Listing --> Array to Sequence -->
>>>>> Image Converter --> ImageJ
>>>>>
>>>>> ...which almost works if I hand-specified the number of Array elements
>>>>> in the directory ahead of time), but still only processed the first
>>>>> file listed in the string (Picture 1.png) -- I'm assuming it just
>>>>> throws the rest of the string away.
>>>>>
>>>>> I'm not understanding something about how one processes a list in a
>>>>> Kepler workflow -- I've looked through Array Operation and Iterative
>>>>> Operation actors, and none of them seem to be analogous to an
>>>>> imperative programming loop. Am I using the wrong kind of director?
>>>>> Something else?
>>>>>
>>>>> Thanks for your help.
>>>>>
>>>>> BACKGROUND: My larger goal is to adapt a frame based image processing
>>>>> workflow into Kepler (it is currently a combination of python,
>>>>> javascript, and bash scripts). To illustrate, one part of the workflow
>>>>> takes a heterogeneous collection of images (all file types and sizes)
>>>>> and copies them to a new directory as a collection of standardized
>>>>> collection jpegs. Another part of the workflow dumps a movie to a
>>>>> frame directory of jpegs, then runs multiple image measurement and
>>>>> image processing operations (e.g. ImageJ, MATLAB) on each frame and
>>>>> saves the results to a text file.
>>>>> _______________________________________________
>>>>> Kepler-users mailing list
>>>>> Kepler-users at kepler-project.org
>>>>> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>>>>>
>>>>
>>>
>>
> _______________________________________________
> Kepler-users mailing list
> Kepler-users at kepler-project.org
> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users
>