Welcome, Guest. Please login or register.

Login with username, password and session length
Forums Search:  


Author Topic: IMG efficiency question  (Read 4901 times)

glendeni

  • Mapper
  • ***
  • Posts: 80
  • Karma: 0
    • View Profile
IMG efficiency question
« on: July 31, 2012, 03:50:58 PM »
I have 10 .img files containing topo data for different regions, 2 .img files containing steam data, and 1 .img file containing trail data.  I expect to be frequently updating the trail data whereas the topo and stream data will be static.  Currently I create an overall IMG file for MapSouce using all 13 files as input to MapsetToolkit but it would be more convenient to combine the static topo and stream data into a single .img file so that I only have to worry about two .img files when using MapsetToolkit.

I can combine the topo, for example, by generating a .mp file from each of the regional .img files, combining these into a single .mp file and generating a single .img topo file. I've already tried this - the compilation took 3 hours on my i5 processor but it did work.  But I've started wondering about the useage efficiency of using such a large single .img file instead of 10 smaller files.  Does anyone know whether using such a .img file will result in less efficient use either in MapSource or the GPS?

(As an alternative I've considered using the "Join" functionality of GmapTool to combine the 10 regional topo file, but I don't know exactly what that does.  Perhaps using GmapTool's "Join" would create a single .img file but within that file the 10 regional files would still be separate??)

Seldom

  • Expert Advisor
  • *****
  • Posts: 1852
  • Karma: 19
    • View Profile
Re: IMG efficiency question
« Reply #1 on: July 31, 2012, 04:03:09 PM »
Your problem is easy to handle with GPSmapedit.  I keep a set of topo, hydro, woods,and boundaries in one MP file, I keep the roads separate.  When I want to update a trail, I add the trail to the roads map and then "ADD" the roads map to my topo/hydro map, saving the combined map as something else.  This is especially handy with routable maps, because you can verify your routable map a lot faster if it doesn't keep all the non-routable data in the same file.

3 hours on an i5?  How big a map are you compiling.  My DesertSouthwest tiles are each 1 degree square with 40 foot contours, but the tile of Bright Angel Trail at the Grand Canyon only took about 10 minutes to compile (and it will route you on foot across the canyon from Kanab, UT to Tusayan, AZ).  Of course I'm using an i7, but that seems like a long time.

maps4gps

  • Expert Advisor
  • *****
  • Posts: 1524
  • Karma: 17
    • View Profile
Re: IMG efficiency question
« Reply #2 on: July 31, 2012, 05:21:19 PM »
Quote
Currently I create an overall IMG file for MapSouce using all 13 files as input to MapsetToolkit but it would be more convenient to combine the static topo and stream data into a single .img file so that I only have to worry about two .img files when using MapsetToolkit.
Should not be that much of an issue if you have all the files in a single folder.

Quote
...combine the static topo and stream data ...
Depending on the area, this is not correct.  USGS does a bimontlyl release of new/updated NED data which many/most of us use to create the contour lines.  The NHD hydro data is still in daily revision/updating. 

Quote
I can combine the topo, for example, by generating a .mp file from each of the regional .img files, ...
  You should have the .mp files if you created the topo data yourself.  If using the data from a mapset you did not create yourself from original source data, it may violate the use/license agreement (and copyright) of that mapset .

Quote
Does anyone know whether using such a .img file will result in less efficient use either in MapSource or the GPS?
I have wonder that too, but never had the time to prepare mapsets to do a comparison.  You seam to have done so, please test it and let us know your findings
« Last Edit: July 31, 2012, 05:49:02 PM by maps4gps »

glendeni

  • Mapper
  • ***
  • Posts: 80
  • Karma: 0
    • View Profile
Re: IMG efficiency question
« Reply #3 on: August 02, 2012, 10:07:23 AM »
I've created two IMG files using MapsetToolkit, one from 10 separate topo .img files plus 2 stream data .img files and the another from a single (large!) topo ,img plus a single stream data .img file.  I loaded each onto a GPSMAP60csx (I wanted to use an older device with a likely slower CPU) and so far can see no differences regarding screen drawing speed.  Don't know if there is anything else I can test.

One thing I like about use of the single topo .img file is that I don't have CGPSMAPPER labels all over the map.  It did take over 3 hours to complile on cgpsmapper (the .mp file is 545 MB, the resulting .img file is 41 MB) - but I only have to do it once.


Boyd

  • Expert Advisor
  • *****
  • Posts: 3849
  • Karma: 43
    • View Profile
Re: IMG efficiency question
« Reply #4 on: August 02, 2012, 10:57:30 AM »
Well, if you understand custom types (.typ files), you might find another solution for removing the cgpsmapper watermarks on your .img tiles.  ;)

In making this extremely complex map I found that smaller tiles are much more efficient at rendering on the gps. http://www.gpsfiledepot.com/maps/view/276/ And they also compile very quickly - most of the tiles in that mapset took under a minute to compile. But most people don't make maps with this density of data. I think the issue comes when you are travelling an area near the corner of the tiles, so the device may have to read 4 tiles to render the map on the screen.

Also, when I updated this complex topo map I went from 35 tiles in the previous version to about 200 tiles: http://www.gpsfiledepot.com/maps/view/294/ That resulted in much better performance on the device as well. Again, there's a pretty extreme amount of detail here including complex LIDAR derived contours and landcover polygons.

If your map is simple enough to fit completely in a single .img file, I suppose that would eliminate the issue of reading multiple tiles in the corners. But that isn't going to work well for a big complex map that covers a whole state. :)

maps4gps

  • Expert Advisor
  • *****
  • Posts: 1524
  • Karma: 17
    • View Profile
Re: IMG efficiency question
« Reply #5 on: August 03, 2012, 05:27:16 AM »
Quote
I loaded each onto a GPSMAP60csx (I wanted to use an older device with a likely slower CPU) and so far can see no differences regarding screen drawing speed.

The screen draw while moving and also whied panning around on the GPSr was primarly what I was thinking of.  Another would be if the startup time would be effected.  I have an 8Gb (2,000 segments +-) mapset and it takes my OR300 about 80 seconds for startup.

If you are going to share that mapset with others, consider there are many users with GPSrs without card slots and only 24Mb of internal memory.  I had a PM from a user this Spring who had a unit with only 1.4Mb usable for map data.

glendeni

  • Mapper
  • ***
  • Posts: 80
  • Karma: 0
    • View Profile
Re: IMG efficiency question
« Reply #6 on: August 03, 2012, 09:41:29 AM »
For some perspective here, the largest file in my "combined-region" .img file is 39 MB, which may not be that large by other standards.

> The screen draw while moving and also whied panning around on the
> GPSr was primarly what I was thinking of.  Another would be if the
> startup time would be effected.

The former was what I previously tested.  I cannot see any notable startup time difference. (I have two GPSMAP60csx units, one loaded with a combined-region topo/stream/trail .img file and the other with separate-region topo/stream/trail .img file, so can test them simultaneously.)

Currently the "separate-region files" version is on my website, but I'm considering replacing it with the "combined-region" version so first testing it.

My main reason for the combined-region version is to get rid of the multiple CGPSMAPPER lines which are all over the map due to the multiple overlapping regions of topo, streams, and trail data.  I do not want to do away with ALL those lines (as I myself put a copyright on my map which I expect others to respect) but think that a single one suffices for copyright purposes.  I have attempted to remove the multiple lines by using extended text in a TYP file, but so far setting "small cities" (0x0b00) to "invisible" has not the desired effect - likely I am doing some thing wrong, but so far have not discovered what.  If I can get that working I will stay with the "separate-region" version.


Seldom

  • Expert Advisor
  • *****
  • Posts: 1852
  • Karma: 19
    • View Profile
Re: IMG efficiency question
« Reply #7 on: August 03, 2012, 09:55:32 AM »
Do you have access to MP source files for all your data?  If so, and you combined all your data into a single map you'd only have one cgpsmapper watermark.

Also, if you edit the MP file in GPSmapedit you'd have access to lots of useful tools like "verify map" (for routable) and "generalize", "remove object duplicates", and "slice all objects".  Slicing all objects greatly reduces compile times for non-routable objects like contours and hydro.

Re. 39MB, how big an area is that?  My 1 degree routable Grand Canyon tile with 40 foot contours, woods, and land use is only about 15MB.
« Last Edit: August 03, 2012, 10:01:27 AM by Seldom »

glendeni

  • Mapper
  • ***
  • Posts: 80
  • Karma: 0
    • View Profile
Re: IMG efficiency question
« Reply #8 on: August 03, 2012, 10:25:24 AM »
> Do you have access to MP source files for all your data? 
> If so, and you combined all your data into a single map
> you'd only have one cgpsmapper watermark.

Perhaps my previous posts were not clear, but that is exactly what I am doing, for that very reason.

My understanding is that GpsMapEdit creates an .mp file to be compiled by cGPSmapper and does not do any compilation itself, leaving that to cGPSmapper.  Since I create my own .mp files, I already have complete control over the .mp file sent to cGPSmapper.

The area is the (loosely defined) Big Sur region of California.  The topo is complex and moreover I am using contours lines with 20 ft spacing, so have quite a few of them.

Seldom

  • Expert Advisor
  • *****
  • Posts: 1852
  • Karma: 19
    • View Profile
Re: IMG efficiency question
« Reply #9 on: August 03, 2012, 10:51:06 AM »
Suggest you look at GPSmapedit if you haven't.  It's a graphic editor with extensive utilities as I mentioned above.  Frankly, I never use it to call cgpsmapper.  I do that with a batch file.

How are you generating your MP files yourself, with a text editor?


Boyd

  • Expert Advisor
  • *****
  • Posts: 3849
  • Karma: 43
    • View Profile
Re: IMG efficiency question
« Reply #10 on: August 03, 2012, 11:15:45 AM »
but so far setting "small cities" (0x0b00) to "invisible" has not the desired effect - likely I am doing some thing wrong

I think you are, because it works for me.  ;)

Personally, I think 39MB for a single map tile is way off the charts. My target for best performance is about 1.5MB per tile. But it also depends on how large an area the tile covers. If you are offering this as a free map, then people can't really complain, but you are going to be out of reach for a significant number of users.

I realize that this isn't a "big" file by today's standards, but this is Garmin's world. Aside from older models with only 8MB of memory, Garmin still has 4 current models with no slot and only 24MB memory:

https://buy.garmin.com/shop/compare.do?cID=145&compareProduct=30120&compareProduct=8709&compareProduct=8707&compareProduct=30122

I wouldn't buy one of these, but lots of people do because they shop solely on the basis of price.

glendeni

  • Mapper
  • ***
  • Posts: 80
  • Karma: 0
    • View Profile
Re: IMG efficiency question
« Reply #11 on: August 03, 2012, 11:58:45 AM »
> How are you generating your MP files yourself, with a text editor?

With a perl script (in Linux) which reads data from different databases, so everything is automated - alterations are easy to make and I get fine-scale control over the .mp file created.