GPSFileDepot.com
 

News:

Welcome to GPSFileDepot!

Main Menu

cGPSMapper Taking a long time

Started by erik.the.awful, January 01, 2009, 05:34:48 AM

Previous topic - Next topic

erik.the.awful

I'm running CGPmapper to convert my first .mp to an .img file to test it out, it is the 30x60 min section around my house, Commerce GA quad to be exact. The mp file was around 303 mb I set it to run last night and has been running for the last 14 hours.... Is this normal?

....Says it has 160433 elements to process....

One other small thing I embeded the dictionary file in the borders before I trimmed it to fit the small quad and I had to change the zoom settings on the header to 0=0 1=1 2=2 etc b/c cgpsmapper wouldn't start converting saying that the zooms had to be ascending. They are ascending now 1-5 and everything looked good in gpsmapedit. Any thoughts on this would be appreciated.

-Erik 

maps4gps

I do not know the limit for an .mp file size, but most of mine have been well under 30Mb.  With that size you must have contoured in 10 or even 5 foot intervals.  I have had over 200,000 elements.  The compile/convert time is related to the types of elements and how complex the area is.  It also seems Vista sometimes cleans house during the processing.  A number of smaller files takes much less time to process than one very large file.  The gmapsupp.img file in limited to 2048? or 2052? files and the Garmin units only support a 2Gb memory card, so there does not seem to be much reason to create super large files.  The larger files might also slow the drawing speed on the GPSr.

erik.the.awful

How did you get the files down that small? ???
I am using the topo process Beta program and following the tutorial directions as they apply to that program as best I can. I processed the files with 20 foot contours and added the transportation, POI, borders, NHD data, land use and some hiking trails. The damn NHD file was over 1.7GB for the 40 30x60 quads I processed. The DEM data produced .mp files ranging from 150-660 mb per quad. The run I am doing now was only for one 30x60 quad so I could see how it works thats only .5X1 degree. It should be a much smaller file I think but with all the layers saved as one .mp file by GPSmapedit that quad is 300mb.  It has been running like 16 hours now and is doing SOMETHING but I know not what...
Please tell me how to pare these files down some, I can't fit all 40 DEMs plus the NHD data in GPSmapedit I followed the tutorial directions to a T for downloading but clearly I have larger than needed files.....
-Erik

erik.the.awful


savage

On average cGPSMapper produces 1 .img per hour on my machine using the same parameters suggested in the tutorial. But I have also seen .img taking more than 5 hours!

erik.the.awful

Is the 300mb file size for a finished polish file quad okay? It looks great in GPSmapedit all the data lines up with where it is supposed to be. How small will that be in .img format?

maps4gps

  Perhaps I have said too much already since I use GlobalMapper and not Oz's topo process Beta nor tutorial directions (developed my methods for GM before Oz published his methods. 
  I have GM making .mp files of smaller areas, therefore the .mp file size is smaller.  Not sure how you could get 300mb with 20 ft contours for your area.  These file sizes are way larger than  anything I have produced, so Oz or someone using his methods could advise you better.
  Polish size to .img size depends on a lot of factors; without contours: I have seen only 30% (10Mb > 3+Mb) for metro areas (all those 2 point roads with attributes) to less than 4% for areas with few roads and hydro (1Mb > 40Kb).  Contours are usually around 5-8%.  Also depends on the bit-levels you are using and how many features are defined on how many levels.  Time to process depends on your computer speed - and I think becomes much slower if cgpsmapper needs to use hard disk space for temp files rather than RAM.

-Oz-

You'll have to wait for it to finish the entire 300mb to know how big it'll be.  Sometimes I have 80mb files that end up at 3mb img files and sometimes i have 70mb files that end up at 6mb files; depends on the number of elements I believe.

My NHD file for nevada is around 2gb in total so that is pretty normal.

And I would say for a file that large it is very normal to run for 14hrs.  Expect that many elements to take somet ime.

What I will say though is I have never had a file over 150mb.

Which method have you used to build the other files; mainly, which file is huge?  I'm assuming the contours data; did you use the topo process beta; gpsmapedit; or global mapper?
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

erik.the.awful

Well I just split off a 7.5 min section without changing anything and it ran in about 15min. I guess I will let it run at least 48 hours to see if it finishes the quad.

Most of my contours are from 150 to 600 mb each, depending on the relief of the area, mainly the NC/TN border ones are large. It should be noted that the actual printed 7.5 min series is in 20ft contours for this area so I selected it to build 20' contours

My methdology was as follows

Downloaded data, used Topoprocess to build DEM data (40 quads, all seperate) and converted to.mp, used topoprocess to build NHD data in 4 sets. Combined NHD data after trimming to fit my area, combined trimmed NHD data into one .mp with Gpsmapedit. Used topoprocess to retrieve other datas. Combined and trimmed with gpsmapedit. Added trail data by converting gpx trails to polylines and labeling as 0x16 in gpsmapedit. Combined all data except DEM and NHD into one trimmed .mp file. Loaded large NHD file, trimmed out a section to fit the commerce 60x30 quad, then loaded DEM for commerce then the other combined data. Trimmed to fit that quad. Saved as commerce.mp (and fixed the zoom levels to  0=0, etc). Ran topoprocess to convert the commerce.mp to a img file. Still running...

I had to use this method to get around the small load limit of gpsmapedit. For some reason it can load larger polish files than shapefiles and strangly enough sometimes restarting the program seems to free up more "room" to load up files. Weird, I think it has a memory leak....
I also need more RAM as I only have 1gb of DDR2.

Thanks for all the help, any other suggestions would be appreciated.
-Erik

-Oz-

Are you sure that the 300mb only covers 1 degree X 1/2 degree?  Like during this loading unloading process did it grow larger than that?

Another option you have is next time to grid to the same area always then use dos to combine the .mp files (since they are just text); just have one have a header and the rest don't and it should work.

I started doing the above that way I could keep the contours seperate from everything else since they caused me the most memory/crash issues.

Your steps don't sound bad though.  That file is just very very large; i've been processing Arizona for almost a day straight now.  Since cgpsmapper doesn't take advantage of dual core cpu I just run it twice so that both CPUs are in use; something to know if you have a dual core.
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

savage

I have an idea (probably a bad one but we are amongst friends here) why cgpsmapper is so slow.
This morning I looked at my computer and cgpsmapper was running but not making any progress for more than 8 hours. I wanted to kill it. I went to the DOS CMD window to check it out and suddenly the % of completion that was stuck at 2 is now incrementing nicely, currently at 10%.

Is it possible something could be blocking the DOS output? Maybe when I checked it out last night before going to bed?

-Oz-

If you're using vista it is definitely possible that something is blocking it.

On the other hand; I've noticed that its really slow in the beginning and then just cruises after a certain point (a varying point mind you).
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

erik.the.awful

 ;D ;D ;D ;D ;D ;D ;D ;D ;D ;D

It finally got the quad done, it took 27 hours :o but it compressed that 300mb behemoth down to a 10.2 mb file. Now processing the next quad. One down 39 to go....

The map looks great!!!!
Thanks Oz and Savage for all the help Can't wait to finish the set so I can upload it all.

-Erik

-Oz-

Can't wait to see it.

Quite laughable but I now have multiple 300+mb segments for Nevada.  Hopefully it won't take quite as long.
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

savage

Quote from: erik.the.awful on January 03, 2009, 01:05:53 PM
;D ;D ;D ;D ;D ;D ;D ;D ;D ;D

It finally got the quad done, it took 27 hours :o but it compressed that 300mb behemoth down to a 10.2 mb file. Now processing the next quad. One down 39 to go....

The map looks great!!!!
Thanks Oz and Savage for all the help Can't wait to finish the set so I can upload it all.

-Erik

Good job and I have Maine still running. Hopefully it should be ready this week-end unless something unexpected happens.