Messages

On GPSmapedit I have found easier to convert all the .shp files to .mp and then only load the data needed to complete one 30x60 quad. GPSmapedit for some reason can load larger files in polish format. Also try closing the program and then reopening it if you run into a memory issue, for some reason this seems to free up "room" for larger files. Closing the file in mapedit doesn't help. I think it may have a memory leak.   

   In my rememberance and understanding - The version of cgpsmapper I first used in Jan 2007 'processed' at a more uniform rate.  The version of mid-2007 has a lag in the 'processing' at the start but completes the total processing faster.  I am guessing that it is reading the entire file and setting up indexes which speed up later processing at the higher 'zoom' bit levels. 

   What I have not been able to understand is the recent number of people trying to use cgpsmapper to process monster sized .mp files.  Could someone please explain the reason (besides less files to cover an area) for using these large sized .mp files? 

   Has mapwel actually decifered the Garmin coding, or does it just invoke cgpsmapper to produce the .img files?

I'm actually not sure why I have monster .mp files. I really think that the topoprocess beta program created larger files for the DEM than was needed. The other issue I'm having with it is when I do use it to convert to img that it overwrites the output file making every file 0000000.img, so I only process one at a time in a folder then rename it and transfer into my completed directory. My monster files are only covering one 30x60 quad (1x.5 degrees) and are compressing down to the right size range 10-12mb each, perhaps when it is using POSTgis to produce the contours it is doubling up on some data. I always tell gpsMapedit to remove duplicate objects before I save my final polish format. It has to be related that cgpsmapper takes so long to run but reaches the right finished size and that topoprocess is exporting extra large contour data.
-Erik   

I have heard form a guy producing maps with mapwel using scanned raster images of actual 7.5min quad maps and he reported it worked really well and could produce a quad in about an hour. The major issue he had was that it worked too well and was difficult to render on the gps units requiring 20-30 sec to draw. The other issue is it is tied to a specific GPS unit and I think even their offering of unlimited gps program (which naturally costs more) allows you to upload to any GPS but still requires the individual units to be logged in with the program....

 

It finally got the quad done, it took 27 hours but it compressed that 300mb behemoth down to a 10.2 mb file. Now processing the next quad. One down 39 to go....

The map looks great!!!!
Thanks Oz and Savage for all the help Can't wait to finish the set so I can upload it all.

-Erik

Well I just split off a 7.5 min section without changing anything and it ran in about 15min. I guess I will let it run at least 48 hours to see if it finishes the quad.

Most of my contours are from 150 to 600 mb each, depending on the relief of the area, mainly the NC/TN border ones are large. It should be noted that the actual printed 7.5 min series is in 20ft contours for this area so I selected it to build 20' contours

My methdology was as follows

Downloaded data, used Topoprocess to build DEM data (40 quads, all seperate) and converted to.mp, used topoprocess to build NHD data in 4 sets. Combined NHD data after trimming to fit my area, combined trimmed NHD data into one .mp with Gpsmapedit. Used topoprocess to retrieve other datas. Combined and trimmed with gpsmapedit. Added trail data by converting gpx trails to polylines and labeling as 0x16 in gpsmapedit. Combined all data except DEM and NHD into one trimmed .mp file. Loaded large NHD file, trimmed out a section to fit the commerce 60x30 quad, then loaded DEM for commerce then the other combined data. Trimmed to fit that quad. Saved as commerce.mp (and fixed the zoom levels to  0=0, etc). Ran topoprocess to convert the commerce.mp to a img file. Still running...

I had to use this method to get around the small load limit of gpsmapedit. For some reason it can load larger polish files than shapefiles and strangly enough sometimes restarting the program seems to free up more "room" to load up files. Weird, I think it has a memory leak....
I also need more RAM as I only have 1gb of DDR2.

Thanks for all the help, any other suggestions would be appreciated.
-Erik

Is the 300mb file size for a finished polish file quad okay? It looks great in GPSmapedit all the data lines up with where it is supposed to be. How small will that be in .img format?

BTW I am using XP if it matters

How did you get the files down that small?
I am using the topo process Beta program and following the tutorial directions as they apply to that program as best I can. I processed the files with 20 foot contours and added the transportation, POI, borders, NHD data, land use and some hiking trails. The damn NHD file was over 1.7GB for the 40 30x60 quads I processed. The DEM data produced .mp files ranging from 150-660 mb per quad. The run I am doing now was only for one 30x60 quad so I could see how it works thats only .5X1 degree. It should be a much smaller file I think but with all the layers saved as one .mp file by GPSmapedit that quad is 300mb.  It has been running like 16 hours now and is doing SOMETHING but I know not what...
Please tell me how to pare these files down some, I can't fit all 40 DEMs plus the NHD data in GPSmapedit I followed the tutorial directions to a T for downloading but clearly I have larger than needed files.....
-Erik

I'm running CGPmapper to convert my first .mp to an .img file to test it out, it is the 30x60 min section around my house, Commerce GA quad to be exact. The mp file was around 303 mb I set it to run last night and has been running for the last 14 hours.... Is this normal?

....Says it has 160433 elements to process....

One other small thing I embeded the dictionary file in the borders before I trimmed it to fit the small quad and I had to change the zoom settings on the header to 0=0 1=1 2=2 etc b/c cgpsmapper wouldn't start converting saying that the zooms had to be ascending. They are ascending now 1-5 and everything looked good in gpsmapedit. Any thoughts on this would be appreciated.

-Erik 

Okay thanks for the info, I can get the DEM's to work fine most are pretty small but the ones in the upper crest of the blueridge and smokeys are larger due to the extreme height over base level. The  one that has clingman's dome is huge. Is there a way for me to split up this already compiled NHD flowline shapefile into four smaller files so I can trim it, once trimmed it should be much closer to a GB as the subbasins extended far out from my intended area. I can't use GPSmapedit b/c of the memory limitation and I don't want to re run the NHD data in segements again unless I have to. Can I do this with the demo of global mapper? What about saving the flowline data as a .mp file and then going ahead and compressing it down into a .img file, then compiling the map from there. Everything fits on gpsmapedit for splitting it into 30x60 quads except the NHD flowlines....
-Erik

Okay I upped my pagefile to reflect the larger files, now all of the elevation data will load, as well as all of my other data except for NHD_flowline.shp, the file is about 1.7GB for the area untrimmed and it will not load in GPSmapedit. It is reporting that it will not fit into virtual memory, but I have over 4GB allocated for the minimum. Because my map crosses so many watersheds I have an insane amount of hydrology data. Any suggestions would be helpful as I am ready to combine and split if I can get this to load up.

I'm using GPSmapedit. On the scaned 7.5 min USGS quads for the entire area I'm mapping right now is in 20' contour intervals. I'm mapping the southern Appalachian mountains for those into hiking primarily (and so I set the contour interval to 20' in rough terrain that can make a difference). I have included all of the major long distance hiking trails I could find in my state as well as all of the data in the tutorial. My map encompasses part of 6 states, the major upthrust and some of the piedmont or plateau. It is 40 100000:1 quads in its entirety.

Good to know there is some compression, I have a 60csx and while I didn't want to load it all at once, at least 8 quads at a time would sure be nice....

Thanks for the 411!!! 

-Erik

My dem mp files are very large. I don't see how these maps are going to be broken down into anything close to a 8mb file. Does converting them to Img's compress them somehow? My elevation data alone for a 30x90 quad is sometimes as large as 600mb. Did I get to detailed of files off of the  seemless server?

-Erik

okay thanks a bunch the program is making shapefiles now. For some reason I also had to to edit the other connection line to allow it to trust and opened up a port on the firewall as well. I used the one click postgres installer  and I guess that gave me a little hell. That topo process program is nice and was needed to since the dem2topo runtime files are nearly impossible to get.

BTW I requested the hydrology data for the quads on Sunday and still haven't gotten any of the emails back from the server. How long does it take usually before they send them? When should I resubmit my request?

Thanks,
Erik

I am assuming that it should be leaving an .mp file for every geotiff as long as it isn't checked to produce one file. I restarted the service and went to pgadminIII and found that all of the databases were x'd out. I typed in my password in the admin tool and it brought them up and allowed me to access them through admin. I am rerunning the contour function today while I am at work. It should be noted that I am seeing it build several files while the process is running but when it concludes there are only .prj files left.

Still not working properly, I am going to wipe both programs and start with a completely new install of postGIS and see if that corrects the issue.