Messages

I have encountered something similiar using GlobalMapper.  Happens when I have somewhat over 1Gb of .shp files open (with 3Gb of RAM) to create the .mp files (usually under 20Mb in size).  The template file with the IMG ID section information does not get placed in front of the data and therefore neither cgpsmapper nor mapedit can use the file.  I have never used mapedit to create the .mp file (nor created one over 80Mb in size).  You may want to check to see if all the 'header' info that is supposed to be in front of the data is actually there in that large of a .mp file.

Not including CT ??

I also get those with Global Mapper.  It appears USGS's process to 'edge match' data from individual 7 1/2' quads does not work very well at times.  Also that a mask for sea level (nor large lakes/reservoirs) was not used, so the DEM can have numerous areas slightly above and below the water level which then get contoured.  There are also bad situations were they patched in data at a different resolution along the US-Canada border in the eastern portion of the US.

   I never used the verify map option, however intersecting polygons between natural and 'manmade'  objects are quite common, ie. any stream wide enough to be shown with opposite shorelines or waterbody in two or more counties/states or inside and outside of a park, etc will have intersecting boundary lines.
   

Gateway M-series laptop with 3Gb RAM, 13 months old.  Try to run it with 100+Gb free space.  Data stored on external USB hard drive(s).  Only GM and some programs for GPS map creation on it.  Never have found the processor speed, but similiar laptops are now under $500.

I was not intending to put any files for the western states on a website.  You produce excellent maps at reasonable file sizes.  I intended to fill the need for transparent contours and those not having the need and/or download capacity for the Ibycus 800+Mb US file.

The planimetic runs using Censu & GNIS data are 'finished'.  Of 40 30x60 quads with an .img file size above 5Mb, 8 are west of 102W longitude.  The Houston (10.5) and Chicago (10.2) top the list.

After resolving the website (current one seams to have gone belly-up) and some other data issues, I may do at set of contours at half my current interval - about equivalent to the interval you are using.  Then tackle hydro with new NHD data.  Perhaps a full topo with 24K equivalent contours, NHD hydro, GNIS, and Census transportation, etc.

ah ... show some consideration for little rhodie

My current runs (in progress) for 30x60 areas using Census hydro and transportation plus GNIS result in .img files of size:

Providence - 5.92Mb

and for 765 'quads' west of 102 degrees longitude:
Los Angeles      - 9.7Mb
Santa Ana        - 8.6Mb
San Bernardino - 6.7Mb
Long Beach       - 6.1Mb

Seattle              -5.89Mb
Tacoma             -5.7Mb
Mesa  AZ           -5.4Mb

Boston is 9.2Mb and others may be as high as 11Mb

We all need to remember that due to more precip (hydro) and people (transportation), the average file size in the eastern US will be larger.

A program which would check a master file containing info on area extent, file names, etc. then build the GM script file to process the data files.  Somewhat of an extension of the program I created in late Dec and then deleted to process NED files at different contour intervals and area sizes.  It was becoming too involved to keep up with the new NED data being released by USGS every two months.  File names might be a problem as I now include the corner lon-lat in the name.  Also what might be an appropriate division for the planimetric data (roads etc) probably would not be for the contours.  The biggest issues I see would be the very large files in metro areas and the very small files were only a small portion of the quad area was US land.

I was wondering when you would get to Alaska.  NHD can be very large in areas with huge numbers of waterbodies(NW).  I think AK was 2 or 2&1/2 Gb out of 20Gb for the entire NHD.
AK large? - Only slightly over 3 times the size of TX; and most of it you need an airplane to get to.

I have not included Hawaii nor Alaska yet because of: 1. the nice map you have with contours of Hawaii, 2. the number of 'slivers' of land in so many 30x60's, 3. the NED data at 2 arc sec (60m) in Alaska and how this affects the appropriate contour interval and what, if any, consideration should  be given to the 2:1 at 60N to 3:1 for northern Alaska that convergence of the meridians has on the grid spacing.

I have been working with standard areas too.  A number of the 30x60 quads in the east with large metro areas have been in the 8-11Mb - for just the planimetric data using the hydro in Census (which on-average is about 60% of what is in the NHD).  Then add contours at '24K scale'.

At the other end is if anything special should be done with quads that have little data because much of their area is ocean/Great Lakes and/or non-US land area.  In the 12x12' test quads I did last summer in areas like that, some of the .img files were larger than the source .mp files.

Anyone try using different sized 'quads' to narrow the range of .img file sizes?

If maps in the GPS world are like paper maps, your area of interest is usually in the corner of 4 quads, therefore <2Mb file sizes would be more appropriate for GPSr units with 8Mb of memory.  Yes/No  ?  ?? 

I will have to take some time to run a test on the performance of a 10Mb 30x60 file versus 8 15x15 for the same area.

I stand with foot in mouth - sort of.  I was going to process a 30x60 along the NC-TN border, but seem to have deleted the modified processing routines I made just before the holidays.  However, in looking at the detailed level files I made last summer and stored on an external hard drive, 250-300Mb of .mp files in that area is about right.  I used 12x12 minute 'quads' because some test I ran showed that for a 1x1 degree area, 25 12x12 .mp 'compiled' to .img files in about 40% of the time needed for 16 15x15 files.  A 1x2 degree quad in that area took 4 hrs for the 50 12x12' files to compile - about 1 hr per 30x60 area.   Perhaps 2.5 ?? hrs total for 16 15x15' sized files.  If you can specify output file size in topoprocess, run a test at 15x15 or 15x30 and tell us the results.

Again I will ask for info on why these large file sizes are of such interest??

BTW - an hour in mapwel for per 7&1/2 is 32+- hrs for the 32 7&1/2's in a 30x60 quad.  I doubt if any of the features in the pseudo vector output had attribute tags without a lot of extra manual work.

I have not used topoprocess as I have GM and my own procedures for processing, so maybe I suggest will not work.

I run cgpsmapper from a .bat file in a command window - the output .img file has the same name as the input .mp file.  If you have a final .mp file (with everything in one file, contour, hydro, etc),
try running cgpsmapper on it in a command window.

I remember OZ? mentioning adding .mp files together.  cgpsmapper processes points, then lines, then polygons for each bit level.  If an .mp file has these mixed, perhaps cgpsmapper has to spend a lot of time rearranging the data before/during the processing.  I observe that 'on average' a file twice the size takes 2.5/3/3+ the time to process.

Again, please, someone tell me why a 10-12Mb .img is the right size?    I have not heard anything about an optimum size for an .img file, however someone recently posted about Garmin files being in the 400-500K ? size (perhaps because earlier GPSr only had 24Mb internal memory).  I would expect large files to slow performance of the GPSr.  If Garmin only supports a 2Gb gmapsupp.img file with slightly over 2000 component files, about 1Mb per component .img file, why 10Mb file?
I just made some 30x60's using the new Census data which were this size for metro areas - I may test mixing quad/tile sizes to even out the variation in .img file sizes.  I would appreciate it if someone could tell my why large .img sizes are desirable (if they are?)

Give me some specifics and I will create and process one of your areas and let you know what I get.    30x60 area is?   10m NED?  Contour interval?  Other data included as NHD hydro or Census hydro, Census roads,etc, GNIS, etc.?

   In my rememberance and understanding - The version of cgpsmapper I first used in Jan 2007 'processed' at a more uniform rate.  The version of mid-2007 has a lag in the 'processing' at the start but completes the total processing faster.  I am guessing that it is reading the entire file and setting up indexes which speed up later processing at the higher 'zoom' bit levels. 

   What I have not been able to understand is the recent number of people trying to use cgpsmapper to process monster sized .mp files.  Could someone please explain the reason (besides less files to cover an area) for using these large sized .mp files? 

   Has mapwel actually decifered the Garmin coding, or does it just invoke cgpsmapper to produce the .img files?

  Perhaps I have said too much already since I use GlobalMapper and not Oz's topo process Beta nor tutorial directions (developed my methods for GM before Oz published his methods. 
  I have GM making .mp files of smaller areas, therefore the .mp file size is smaller.  Not sure how you could get 300mb with 20 ft contours for your area.  These file sizes are way larger than  anything I have produced, so Oz or someone using his methods could advise you better.
  Polish size to .img size depends on a lot of factors; without contours: I have seen only 30% (10Mb > 3+Mb) for metro areas (all those 2 point roads with attributes) to less than 4% for areas with few roads and hydro (1Mb > 40Kb).  Contours are usually around 5-8%.  Also depends on the bit-levels you are using and how many features are defined on how many levels.  Time to process depends on your computer speed - and I think becomes much slower if cgpsmapper needs to use hard disk space for temp files rather than RAM.

I do not know the limit for an .mp file size, but most of mine have been well under 30Mb.  With that size you must have contoured in 10 or even 5 foot intervals.  I have had over 200,000 elements.  The compile/convert time is related to the types of elements and how complex the area is.  It also seems Vista sometimes cleans house during the processing.  A number of smaller files takes much less time to process than one very large file.  The gmapsupp.img file in limited to 2048? or 2052? files and the Garmin units only support a 2Gb memory card, so there does not seem to be much reason to create super large files.  The larger files might also slow the drawing speed on the GPSr.

The cgpsmapper manual is a little unclear on this (at least to me) - I think it means you have a feature (or the entire file area) which is too large in extent for the bit-level you are using.  What is the geographic size you covering in the .mp/.img files and what are the bit levels you are defining the features at?