Just wanted to clarify and summarize some things I typed while sitting in my car from my blackberry, not the easiest to be eloquent.<br><br>Basically what I was saying is that I see two major components to this.<br><br>1. The actual data ingestion/cataloguing/sourcing/management. This could include coordinate transformation, image cleanup, etc.. depending on scope. Realistically I'd hope that everything submitted here is orthorectified/etc imagery that's basically ready to go, but could potentially need some subsetting/metadata cleanup/etc to help catalogue it better. You can automate some of this, but you need (realistically) some content managers in here helping to push data around. In this step you put good imagery in and you get it into "some sort of database" aka imagery catalog where it's accessible by...<br>
<br>2. The actual tiling & dissemination/content delivery portion. This is much more automated, and while initially could be something as simple as a few servers rsynching or whatever, eventually would be an "automagical" self-balancing/managing/etc content delivery platform. Realistically a lot of the tech for this part is probably already written. I know we have a lot of tiling stuff out there right now, and I'm sure there is some open source content federation work (though I haven't looked). This is where your big bandwidth and platters come into play, and is of course crucial to the whole process - but I do not see it as being especially "geospatial" in the sense that this is a rather generic (aka license-plate making) step of the process.<br>
<br>So my advice is to focus on the first part, built with the second part in mind. If you can get step 1 rolling, then I think step 2 will sort of naturally progress due to existing availability of a lot of this software. I do not think the reverse is quite so true though - I.e. just focusing on the tiling/dissemination piece is pretty moot until you get a good set of data partners and a framework of business processes/rules for managing the workflow of data. I don't mean to say that we need some massive unwieldy bureacracy or anything of that sort. Not at all! I detest such overhead. I'm merely saying we need some kind of plan for managing the imagery catalogue, otherwise it will very rapidly become unwieldy.<br>
<br>"Most" people doing this sort of stuff are dealing with a few providers that are publishing in relatively standardized formats, e.g. USGS, NGA, NASA, Space Imaging, etc.. OAM by its nature will benefit from some of these (e.g. NAIP), however my guess is we'd end up with a lot more "indie" imagery, i.e. kite photography, uav, random county ortho shots.. etc.. This is all fine, but if you're not just dealing with a few standard products, I think you really want a plan to deal with all that imagery and how you'll stitch it together, handle quality vs timeliness, etc.. Otherwise it will just be a big cluster of unusable imagery (IMO). So while I know it's tempting to just start stuffing NAIP into tilecache, I'd caution against too hasty of an approach. <br>
<br>regards,<br> - bri<br><br><div class="gmail_quote">On Wed, Nov 4, 2009 at 5:07 PM, Brian Russo <span dir="ltr"><<a href="mailto:brian@beruna.org">brian@beruna.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Depends which scale you cache, down to around 1:100k or so isn't a big<br>
deal space-wise, below that you probably won't initially have much<br>
coverage outside of the US, even then you can start with clusters<br>
around points of interest. Even google doesn't have truly global high<br>
res coverage. You can always tile on demand or run metrics to<br>
determine areas of demand and prioritize accordingly. In my mind this<br>
aspect is really a generic content staging/delivery problem akin to<br>
federated staging like google/akamai/etc do. Not to dismiss it, but I<br>
do not see it as particularly unique to imagery.<br>
<br>
I am a bit troubled by the focus on tiles, certainly that's probably<br>
the best way to technically disseminate large skins but will impact<br>
certain applications that want the actual imagery. I think its<br>
important to maintain a link to the source imagery - a simple polygon<br>
vector catalog would probably work.<br>
<br>
I admit I haven't read every message in these threads so I apologize<br>
if I missed it, but there's some separation between the actual<br>
sourcing and image cataloging and the primary means of dissemination<br>
(tiles). Building tiles isn't really that difficult - its mostly just<br>
resource-intensive (cpu and disk/io mostly). My suggestion is to focus<br>
on the management/maintenance side of it and then worry less about<br>
the actual tile generation. Because building tiles faster just means<br>
more disk space and cpu - but managing a large imagery catalog of<br>
disparately sourced imagery is quite challenging - I see this as the<br>
hard part. Most people have relatively few sources of imagery. I<br>
could go on but hopefully you get the idea, blackberry makes brevity!<br>
<br>
I'm interested on the HADR support side of it, as I am the GIS<br>
manager/architect for marforpac (executive agent for pacom hadr).<br>
<br>
Regards,<br>
- bri<br>
<div><div></div><div class="h5"><br>
<br>
On 11/4/09, Schuyler Erle <<a href="mailto:schuyler@nocat.net">schuyler@nocat.net</a>> wrote:<br>
> On Wed, 2009-11-04 at 15:06 -0800, Christopher Lippitt wrote:<br>
>> I created a wiki page here (<a href="http://www.openaerialmap.org/RELEIF_Nov09" target="_blank">http://www.openaerialmap.org/RELEIF_Nov09</a>)<br>
>> to propose alternative "phase 1" approaches with Schuyler's as the the<br>
>> first. [snip]<br>
>><br>
>> This approach will allow us to flush out and compare a few different<br>
>> architecture approaches directly, and then come out of the meeting<br>
>> with a clearer picture of how to start building the larger system and<br>
>> hopefully with a small code base to start from.<br>
><br>
> Chris, thanks for your work in getting OAM started, and for taking the<br>
> time to outline these big-picture options on the wiki. I edited the page<br>
> slightly to reflect a few more specifics on these options.<br>
><br>
> I want to suggest in the strongest possible terms that a fully<br>
> centralized solution, although simpler in some ways, and more comforting<br>
> to organizations with strong hierarchical authority, I think such an<br>
> approach is going to prove to be unworkable.<br>
><br>
> First, no organization has yet volunteered to provide 100s of TB of<br>
> hosting, with full backups, and gigabits of bandwidth, both of which we<br>
> will eventually need if this takes off. Don has graciously put in for<br>
> funding for 10 TB of hosting, but if we tile the 15m NGA data he has<br>
> (been?) offered, we will use up all of that space instantly, between the<br>
> tiles and the source imagery.<br>
><br>
> Second, a centralized data warehouse is a single point of failure. OAM<br>
> has already had problems with this in the most recent iteration. I will<br>
> repeat myself here by saying that I think that an unreliable OAM is<br>
> worse than no OAM at all, because it will only serve to discredit the<br>
> concept and discourage the community.<br>
><br>
> Also, while each of the three options you describe have distinctive<br>
> merit, they all depend on a metadata catalog to make the whole thing<br>
> work. I would like to suggest that we discuss seriously making the<br>
> further development of the metadata service the first priority for<br>
> renewed technical work on OAM. I'm hoping to wrangle together folks who<br>
> are interested in this at Random Hacks of Kindness next week, and<br>
> perhaps we'll have something pulled together for discussion at Camp<br>
> Roberts the week after?<br>
><br>
> SDE<br>
><br>
><br>
> _______________________________________________<br>
> talk mailing list<br>
> <a href="mailto:talk@openaerialmap.org">talk@openaerialmap.org</a><br>
> <a href="http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org" target="_blank">http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org</a><br>
><br>
<br>
</div></div><font color="#888888">--<br>
Sent from my mobile device<br>
</font></blockquote></div><br>