Hi Jeremy:<br><br>Can you send the part oft he workflow that creates the problem.<br>Seems to be a couple of actors only.<br>Maybe looking at that will help us, help you.<br><br>best<br><br>Bertram<br><br>PS Re. COMAD release plans, I'm cc-ing Tim who should have more info.<br>
<br><div class="gmail_quote">On Mon, Jul 13, 2009 at 9:39 AM, Jeremy Douglass <span dir="ltr"><<a href="mailto:jeremydouglass@gmail.com">jeremydouglass@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Bertram, thanks for the suggestion, and for your paper on COMAD. Yes,<br>
"SDC" was my typo of SDF. I've since tried changing the director to<br>
DDF and PN for all my tests in the situation above, although that had<br>
no effect on the type mismatch problem -- Directory Listing produces<br>
an array, and I can't get that into a form consumable by an operation<br>
such as Image Converter.<br>
<br>
I read the paper on COMAD with interest. COMAD isn't part of 1.0 right<br>
now, but I look forward to a version that includes it.<br>
<font color="#888888"><br>
-- Jeremy<br>
</font><div><div></div><div class="h5"><br>
<br>
On Sat, Jul 11, 2009 at 1:09 PM, Bertram Ludaescher<<a href="mailto:ludaesch@ucdavis.edu">ludaesch@ucdavis.edu</a>> wrote:<br>
><br>
><br>
> On Sat, Jul 11, 2009 at 12:51 PM, Jeremy Douglass <<a href="mailto:jeremydouglass@gmail.com">jeremydouglass@gmail.com</a>><br>
> wrote:<br>
>><br>
>> I'm a new Kepler and struggling to apply a single actor (e.g. Image<br>
>> Converter) to a batch of objects (e.g. a directory of images, as<br>
>> represented by the array generated by Directory Listing).<br>
>><br>
>> I've read the documentation and searched the forums, and tried<br>
>> modifying examples, but I'm not understanding something paradigmatic,<br>
>> and would appreciate a tip: how do you pipe an array of files through<br>
>> single-file actors?<br>
>><br>
>> EXAMPLE:<br>
>><br>
>> In Kepler 1.0.0, 03-ImageDisplay.xml uses an SDC director, and specifies:<br>
>> Constant Image Filename --> Image Converter --> ImageJ<br>
><br>
> I think you mean SDF here..<br>
><br>
>><br>
>> In an attempt to converter a *directory* of images, I tried adding a<br>
>> Directory Listing actor:<br>
>> Constant Dirname --> Directory Listing --> Image Converter --> ImageJ<br>
>><br>
>> ...however this generates an error, as Image Converter will only act<br>
>> on a string, not an array. So I tried<br>
>> Constant Dirname --> Directory Listing --> Array to Sequence --><br>
>> Image Converter --> ImageJ<br>
>><br>
>> ...which almost works if I hand-specified the number of Array elements<br>
>> in the directory ahead of time), but still only processed the first<br>
>> file listed in the string (Picture 1.png) -- I'm assuming it just<br>
>> throws the rest of the string away.<br>
><br>
><br>
> SDF has the restriction that the director needs to know a-priori (ie, before<br>
> running the wf) how many tokens an actor consumes and produces on each port<br>
> for each firing. For cases where this is known, SDF is usually a good<br>
> choice.<br>
> However, when processing arrays / lists of varying lengths, you usually<br>
> don't have this information.<br>
> Try DDF instead (or PN if you want actors to execute as independent<br>
> threads).<br>
>><br>
>> I'm not understanding something about how one processes a list in a<br>
>> Kepler workflow -- I've looked through Array Operation and Iterative<br>
>> Operation actors, and none of them seem to be analogous to an<br>
>> imperative programming loop. Am I using the wrong kind of director?<br>
>> Something else?<br>
><br>
> See above: DDF might solve your problem.<br>
> There is also a different way to work with nested data collections (of which<br>
> lists are a simple example): using the COMAD director. I'll let the COMAD<br>
> experts and developes say more about how to use this feature (might not be<br>
> part of Kepler 1.0, but in the current developers' version; a new COMAD<br>
> version is also under active development)<br>
>><br>
>><br>
>> Thanks for your help.<br>
><br>
> Also, since you mentioned the paradigma / how-to deal with collections, you<br>
> might want to check out this overview paper:<br>
><br>
> Scientific workflow design for mere mortals. Timothy McPhillips and Shawn<br>
> Bowers and Daniel Zinn and Bertram Ludaescher (2008) Future Generation<br>
> Computer Systems. Volume 25. Pages: 541-551. (Download)<br>
><br>
> from the Kepler pubs page:<br>
> <a href="https://kepler-project.org/publications?tags=keplerworkflow" target="_blank">https://kepler-project.org/publications?tags=keplerworkflow</a><br>
><br>
> Bertram<br>
><br>
>><br>
>> BACKGROUND: My larger goal is to adapt a frame based image processing<br>
>> workflow into Kepler (it is currently a combination of python,<br>
>> javascript, and bash scripts). To illustrate, one part of the workflow<br>
>> takes a heterogeneous collection of images (all file types and sizes)<br>
>> and copies them to a new directory as a collection of standardized<br>
>> collection jpegs. Another part of the workflow dumps a movie to a<br>
>> frame directory of jpegs, then runs multiple image measurement and<br>
>> image processing operations (e.g. ImageJ, MATLAB) on each frame and<br>
>> saves the results to a text file.<br>
>> _______________________________________________<br>
>> Kepler-users mailing list<br>
>> <a href="mailto:Kepler-users@kepler-project.org">Kepler-users@kepler-project.org</a><br>
>> <a href="http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users" target="_blank">http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users</a><br>
><br>
><br>
_______________________________________________<br>
Kepler-users mailing list<br>
<a href="mailto:Kepler-users@kepler-project.org">Kepler-users@kepler-project.org</a><br>
<a href="http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users" target="_blank">http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users</a><br>
</div></div></blockquote></div><br>