I am making progress on my Maine topo map but all the data files are too big to load in Global Mapper (ran out of memory). I could easily split the map North/South but what is the best way to do it? Should I overlap the two parts? Any other comments/tricks I could use.
Is this occurring when you try to load all the files or just the dem files?
My new trick is to use my program and then it processes each quad separately so i don't have all the arcgrid/tiff files loaded at once. If you are gonna split it I recommend splitting it along a degree line and then you won't need overlap.
I am having issues with california for this reason (trying to create a better version than what is offered now).
First release of the Maine topo map is available here (http://www.gameandfishnetwork.com/Content/pid=5.html)
First release of the map. Includes:
* Contour lines (50' interval)
* Street names, rail roads
* Lakes, ponds and streams
* Federal lands
* State and counties border
* Apalachian trail
How did you get over the ran out of memory thing? I keep running into it with New Mexico and thus far exported the segments with roads, water, land coverage, etc as one file than the contours as another file then combine each one individually. Processing is taking 4+hrs a piece with cgpsmapper though so that stinks.
I'll download your map soon.
I think the problems were with the contours. I switched to the TIFF files for the South part of the map and was able to do it with all the problems I reported in other messages. ME was hard to do. I think it is easier to do it in multiple parts and combine at the end of the process.
I realize I did not clearly answer your question.
What I did is to generate 2 sets of contours, one for the North and one for the South.
I did not change anything for the other data (water, transportation, gnis, etc).
I generated the .MP files in 2 runs. One run with the North elevations and one run with the South elevations.
Everything else is the same.
The cut between North and South is at 45 degree latitude. For the North I used Global Mapper for the contours but I use your tool for the South side. For some reasons Global Mapper could not process the South side (internal error of some sort).
Aha, what was the internal error message?
I also am now running into RGN table limits with cgpsmapper; same thing I was having with Hawaii (and have yet to resolve) is happening with New Mexico.
There was no specific error message, just internal error or something similar.
Hi, first I want to say thanks for making the maps available. I live in Massachusetts and I've loaded the MA one into my GPS and as far as I'm concerned, with roads and topo data, it's giving me everything I bought the GPS unit for.
I took a quick look at the maps as shown on the computer (I used GPSMapEdit). Being a railroad buff, I looked at the railroads and one thing I saw was that the Massachusetts map included the Watertown Branch of the Boston & Maine--that line closed in 1960! Clearly the database isn't very well updated.
Then I took a look at the Maine map, specifically 94537126.img, because it covers the area around Chesuncook Lake, where I went canoeing this summer. The landscape all looks right, but the name on Chesuncook Lake is "Foster Field", and because it's a large lake, the name shows up over and over. There is a "Foster Field" on the same map, or at least there should be: but when I went to see what was there, the name wasn't shown (I found it on the topo, as seen on USAPhotomaps.com). It must have wandered off to Chesuncook! This was a bit strange, but if it's just one error I wouldn't complain, considering the price. I did check a few other place names and they were all OK. But maybe this one error could be demonstrating something systematic that affects more places? I don't know if you want to hear about errors of this kind.
Edited a little while later to say that it seems that a "lake" can have only one name on the map, whereas in Maine it's common for connected bodies of water at the same level to have more than one name. Often this is a result of water levels being artificially raised by dams, which causes lakes to merge, but you'll still find the original names on the topo maps. Example is Telos Lake and Chamberlain Lake. On map 94537126.img it's all "Telos Lake", which is actually the smaller of the two (and there are other parts of the same lake with separate name, also now part of "Telos Lake"). Likewise, Eagle Lake has disappeared into "Heron-Churchill Lakes".
All the data comes from government sites. It would be interesting to see if the Garmin maps or other topo maps have the same bugs. Maybe they use other source of information to produce their map.
I have only use the MA maps around where I live and so far it is good. I will travel to ME soon and I will see how it goes.
Thanks for the Maine and Massachusetts maps, they look great.
My one issue is the elevation and contours on the Maine map - the numbers seem to be way off. Is anybody else having this problem? I don't know if it's my setup or something with the map itself.
Could you give an example where you think the map is wrong?
If you look in the western part of the state, at the Mahoosuc Range, the elevations are all over the place. Goose Eye Mountain is listed at 11,483 feet and all the surrounding elevations are similarly inflated. I check around the map and it occurs over the largest area there, but I found pockets in other spots. When you move the cursor over a mountain, it actually gives two elevations - one correct, and one very high, but when you look at the contour lines they also seem to have two or even three figures. Here's a spot: N44.33.621 W70.58.804
Don't know! I will investigate that in the next few days. I used the map in the northern part of the state and everything looked reasonably good but there might be something wrong in the process.
Quote from: smr260 on November 30, 2008, 04:28:58 PM
If you look in the western part of the state, at the Mahoosuc Range, the elevations are all over the place. Goose Eye Mountain is listed at 11,483 feet and all the surrounding elevations are similarly inflated. I check around the map and it occurs over the largest area there, but I found pockets in other spots. When you move the cursor over a mountain, it actually gives two elevations - one correct, and one very high, but when you look at the contour lines they also seem to have two or even three figures. Here's a spot: N44.33.621 W70.58.804
I have not looked at your map, but this sounds very strange. The 11,483 is close to a correct elevation in feet being converted from meters to feet again!? Sounds like data from GNIS might have been duplicated - once with a meter to feet conversion, and again with a second conversion applied. I have not seen the 2 or 3 contour line label issue before. Are you close to using the max
memory on your machine?
No, I have a new computer with 4GB of memory, so I don't think that's it. I figured it was something with the unit conversion, but I'm not sure if it's something on my end or in the map as created.
I downloaded and looked at the ME map. In MapSource when you zoom in enough you will see a double set of contours for some 'tiles'; one with the proper numbers for the elevation in feet and another with slightly different nodes and numbers 3.2xxx higher (the ratio being meters converted to feet). I would guess somehow there are two sets of contours and GNIS points in some of the 'tiles' - one with proper elevations and one with a second conversion of meters to feet (the original NED 10m and recent GNIS are in meters - some builds of GM know when to do the conversion, in others it was left out for some reason). Two files may have been merged, or different contouring options were tried and the unwanted one(s) not removed before the contours were saved or .mp file created.
I had this problem in Arizona. Since I don't use Global Mapper to make my contours anymore I thought nothing of it.
You need to make sure when it comes in its creating everything in the right unit.
I don't really remember perfectly what I did but I had to remake a bunch of contours and make sure in global mapper the units were set to english (or metric can't remember which it was trial and error).
I had a lot of problem with the contours and Global Mapper. In fact I could not use GM to finish the map, it would hang in some area so I switch to another tool.
I will double check my contours and try to regenerate them. The Northern part of the state should be OK because it is where I use GM.
Did you use gdal_contour?
If not what did you end up using?
I do not remember. I think I used dem2topo. I know I had all sorts of problems in the southern part of the state.
I am now finishing up NH and as soon as I release NH I will redo the contour on ME.
sweet
Yeah looking forward to it.
There is a new version of the Maine topo map available here: http://www.gameandfishnetwork.com/News/article/sid=14.html
And this one was not an easy one to do. Let me know if there are bugs.
savage, would you be interested in code to cut the shapefiles to the state border exactly?
Quote
Fantastic work, 3 down! Do you have plans to do other states in the NE?
I do not know which one I will do next. Maybe New York state but that is not NE
Quote from: -Oz- on January 05, 2009, 08:38:19 PM
savage, would you be interested in code to cut the shapefiles to the state border exactly?
I do not know. I was thinking merging several NE maps into a larger one instead?
Quote from: savage on January 06, 2009, 02:25:52 AM
Quote from: -Oz- on January 05, 2009, 08:38:19 PM
savage, would you be interested in code to cut the shapefiles to the state border exactly?
I do not know. I was thinking merging several NE maps into a larger one instead?
That would actually probably be even better to do all of that area at once, Maine, Mass, Vermont, New Hampshire, Rhode Island (thats gotta be quick), Connecticut and maybe even New York.
ah ... show some consideration for little rhodie
My current runs (in progress) for 30x60 areas using Census hydro and transportation plus GNIS result in .img files of size:
Providence - 5.92Mb
and for 765 'quads' west of 102 degrees longitude:
Los Angeles - 9.7Mb
Santa Ana - 8.6Mb
San Bernardino - 6.7Mb
Long Beach - 6.1Mb
Seattle -5.89Mb
Tacoma -5.7Mb
Mesa AZ -5.4Mb
Boston is 9.2Mb and others may be as high as 11Mb
We all need to remember that due to more precip (hydro) and people (transportation), the average file size in the eastern US will be larger.
Very interesting.
What computer are you using to do all this? You'll cream my topos soon.
Gateway M-series laptop with 3Gb RAM, 13 months old. Try to run it with 100+Gb free space. Data stored on external USB hard drive(s). Only GM and some programs for GPS map creation on it. Never have found the processor speed, but similiar laptops are now under $500.
I was not intending to put any files for the western states on a website. You produce excellent maps at reasonable file sizes. I intended to fill the need for transparent contours and those not having the need and/or download capacity for the Ibycus 800+Mb US file.
The planimetic runs using Censu & GNIS data are 'finished'. Of 40 30x60 quads with an .img file size above 5Mb, 8 are west of 102W longitude. The Houston (10.5) and Chicago (10.2) top the list.
After resolving the website (current one seams to have gone belly-up) and some other data issues, I may do at set of contours at half my current interval - about equivalent to the interval you are using. Then tackle hydro with new NHD data. Perhaps a full topo with 24K equivalent contours, NHD hydro, GNIS, and Census transportation, etc.
The USGS totally goes down way too often.
Yea, automation is good. For nevada that I'm uploading now the longest part was actually the processing to .img files; took about 4 days which is insane.