[seek-dev] RE: pgm_cut program
Ferdinando Villa
ferdinando.villa at uvm.edu
Tue Jul 13 11:03:52 PDT 2004
Note that GDAL contains OGR - functions to handle shapes as well as
rasters, with ARC import - and Frank Warmerdam is in the process of
integrating the whole package with GEOS (geos.refractions.net) in order
to provide the shapes with robust topological operators, so in the end
it will be a pretty powerful package that SEEK could do wonders with. I
do think that GDAL does not offer as much as GRASS in terms of raster
processing, so there may be some interventions required. In case we can
define our needs and GDAL does not meet them all, I'm sure Frank will be
more than happy to collaborate...
ferdinando
On Tue, 2004-07-13 at 13:57, Matt Jones wrote:
> Hi,
>
> Also, Chad has implemented several important GIS functions by wrapping
> GRASS libraries behind web services rather than using GDAL directly.
> These services can be called by Kepler and other clients. The idea is
> that a workflow could be constructed that would first query to locate
> data, then do the various clipping and reprojections needed directly at
> the source, then transfer the clipped data to whatever analytical
> service needs it in the workflow. This use of what we've been calling
> 'third party data transfer' and sometimes 'passing data by reference' is
> going to be an important component of data handling. Ultimately, by
> exposing the actual data query and manipulation plan to the workflow
> engine, we will be able to design a scheduler and optimizer that can
> figure out what the most efficient way to execute the workflow is (where
> to do what). The workflow engine becomes a controller for a variety of
> distributed data access, data manipulation, and analysis tasks.
>
> So, I think we should be careful to plan out an end-to-end strategy for
> all data manipulation, rather than doing it piece by piece. Dave's
> suggestion to use GDAL I think is the right approach in that it is more
> generic and more broadly useful than just adding on a specific SRB
> extension. Dave -- could Chad's web-service wrappers around GRASS serve
> in the same function?
>
> Matt
>
> David Stockwell wrote:
> > Hi,
> > pgm_cut.c is a client program that just implements the srb library
> > fread and fseek routines. I thought about implementing more
> > functions on the server side but that is a much bigger project.
> >
> > There is an argument of just having a cut function on the
> > server side, as a cut (or crop) must reduce the data to be
> > transferred. Some other operations like rescaling may
> > increse the size of the data to be transferred, and besides
> > may put a lot of load on the server.
> >
> > GDAL seems useful, but it may make sense to modify
> > it so it could be used on the client side to make simple
> > SRB (or ecogrid) calls. I have been hoping for something
> > like this on the server side but I am not in a position to
> > modify the SRB.
> >
> > Cheers
> >
> >
> > Dave Vieglais wrote:
> >
> >> Hi Bing and Dave,
> >> I was wondering if you had considered using tools more appropriate for
> >> handling geospatial data? The pgm libraries handle image data ok, but
> >> spatial raster data sets have a number of nuances that require
> >> additional processing on top of merely extracting pixel values. One
> >> library in particular that is applicable to a wide variety of raster
> >> data formats is GDAL (geospatial data abstraction library).
> >> Integrating this with the SRB and as an EcoGrid service would provide
> >> access to a huge variety and volume of spatial data available from
> >> various sources, as well as the pgm images. That library also has
> >> numerous methods and examples for windowing, scaling and reprojecting
> >> raster data, and so should be fairly easy to integrate in the same
> >> manner as you have described below. It would also provide the
> >> significant advantage of providing the foundation of a more
> >> standardized mechanism for accessing raster spatial data through the
> >> EcoGrid- something that will be useful to a much larger audience
> >> compared with the very limited data accessible for the "Why Where"
> >> system.
> >>
> >> cheers,
> >> Dave V.
> >>
> >> Bing Zhu wrote:
> >>
> >>> David,
> >>>
> >>> I am glad that the SRB implementation works fine in getting partial
> >>> images
> >>> for Niche
> >>> Modeling pgm data.
> >>>
> >>> This is a good step. I just had a meeting with Raja discussing about
> >>> moving
> >>> this approach to our Ecogrid. Raja suggested to modify the SRB code in
> >>> Ecogrid
> >>> node to handle the parameters currently implemented in pgm_cut.
> >>>
> >>> With this approach, the URL used by Niche Modeling in calling our
> >>> Ecogrid's
> >>> 'get' function
> >>> to retrieve partial pgm image will have the syntax similar to a cgi
> >>> request.
> >>> e.g.
> >>> $ java org.ecoinformatics.ecogrid.client.EcogridQueryClient get \
> >>>
> >>> srb://../home/whywhere/.../n00an1.10.pgm?left=10&top=10&width=25&height=30
> >>> \
> >>>
> >>> http://orion.sdsc.edu:8080/ogsa/services/org/ecoinformatics/ecogrid/EcoGridL
> >>>
> >>> evelI
> >>>
> >>> One of my next tasks in SEEK will be modify the SEEK SRB code to
> >>> handle this
> >>> for Niche Modeling data. And this approach is also applicable for other
> >>> applications which
> >>> request partial data transfer.
> >>>
> >>> Bing
> >>>
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: hwliu [mailto:hwliu at sdsc.edu]
> >>> Sent: Monday, July 12, 2004 12:59 PM
> >>> To: bzhu at sdsc.edu
> >>> Cc: davids at sdsc.edu
> >>> Subject: RE: pgm_cut program
> >>>
> >>>
> >>> Hi, Bing,
> >>> Thanks for your help. I've just tried the program and it worked well.
> >>> But, I don't understand the need to use S-command. I just copied
> >>> pgm_cut to
> >>> my local directory on landscape and it worked even if I don't copy the
> >>> S-command directory. By the way, if I want to rebuild the program on
> >>> Windows, do I need to download the whole directory of SRB1.2.1?
> >>>
> >>> Thanks,
> >>>
> >>> Haowei
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Bing Zhu [mailto:bzhu at sdsc.edu]
> >>> Sent: Monday, July 12, 2004 10:56 AM
> >>> To: hwliu
> >>> Cc: davids at sdsc.edu
> >>> Subject: RE: pgm_cut program
> >>>
> >>> Hi Haowei,
> >>>
> >>> I changed the permission for some directories and files.
> >>> Please try it again and let me know if the permission problem still
> >>> exists.
> >>>
> >>> The files are in the directory, /export/home/bzhu/pgm_cut.
> >>>
> >>> You might also need to use SRB software which is installed in
> >>> /export/home/bzhu/SRB2.1.2. Basically, you just need to use
> >>> SRB client programs, S-commands, which can be found
> >>> in /export/home/bzhu/SRB2.1.2/utilities/bin. To use them, you need
> >>> an SRB account. Currently we can use the user, 'whywhere'.
> >>> I will be glad to show you how to use SRB software so that
> >>> you can test the software with different pgm files stored
> >>> under 'whywhere'.
> >>>
> >>> Sincerely,
> >>> Bing
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: hwliu [mailto:hwliu at sdsc.edu]
> >>> Sent: Monday, July 12, 2004 10:05 AM
> >>> To: bzhu at sdsc.edu
> >>> Cc: davids at sdsc.edu
> >>> Subject: RE: pgm_cut program
> >>>
> >>>
> >>> Hi, Bing:
> >>> I will test the new version of pgm_cut, but I don't have the
> >>> permission
> >>> to access the directory. Is it possible for you to mail me pgm_cut?
> >>>
> >>> Thanks,
> >>>
> >>> Haowei
> >>>
> >>> -----Original Message-----
> >>> From: Bing Zhu [mailto:bzhu at sdsc.edu]
> >>> Sent: Friday, July 09, 2004 5:48 PM
> >>> To: davids at sdsc.edu
> >>> Cc: Seek-Dev; Arcot Rajasekar; Matt Jones
> >>> Subject: pgm_cut program
> >>>
> >>> David,
> >>>
> >>> I modified the pgm_cut.c code to read pgm files stored in SRB storage
> >>> place.
> >>>
> >>> The modified pgm_cut.c can be found in the directory,
> >>> /export/home/bzhu/pgm_cut,
> >>> in landscape machine.
> >>>
> >>> Here is an example of running 'pgm_cut' to get partial image from a
> >>> pgm file
> >>> in SRB.
> >>>
> >>> $ pgm_cut 0 0 20 20
> >>> /home/whywhere.seek/ei/Data/Terrestrial/n00an1.10.pgm
> >>>
> >>> Note that I have re-uploaded whywhere data using a new SRB account,
> >>> 'whywhere',
> >>> as we discussed last time. The new uploaded data, along with their
> >>> metadata,
> >>> is in the collection, /home/whywhere.seek/ei/Data/Terrestrial.
> >>>
> >>> Here is the whywhere account info.
> >>> user name: whywhere
> >>> password: I will send you in another mail.
> >>> SRB server: srb.sdsc.edu
> >>> SRB domain: seek
> >>> SRB port: 6613
> >>>
> >>> I'd like to test it next Monday. Will you be in office next Monday?
> >>>
> >>> Sincerely,
> >>> Bing
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> seek-dev mailing list
> >>> seek-dev at ecoinformatics.org
> >>> http://www.ecoinformatics.org/mailman/listinfo/seek-dev
> >>>
> >
--
Ferdinando Villa, Ph.D., Associate Research Professor, Ecoinformatics
Gund Institute for Ecological Economics and Dept of Botany, Univ. of Vermont
http://ecoinformatics.uvm.edu
More information about the Seek-dev
mailing list