GPSFileDepot Forums

General Category => Map Making Support => Topic started by: leszekp on October 20, 2009, 09:40:23 PM

Title: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on October 20, 2009, 09:40:23 PM
Been plugging away the last week, and now have the first version ready of G-Raster, a program that will load georeferenced imagery, tile it automatically, and create a Garmin-compatible KMZ file.

Don't install KMZ files created with G-Raster into your Garmin's internal memory - put them into an external memory card in an external reader. Just a precaution for use with the Garmin beta software, and version 1.0 of G-Raster. That way, if there's a problem with the overlay, you can always just remove the card from the GPS unit.

Download link to most recent version can be found on this page:

http://moagu.com/?page_id=155

Interface is fairly straightforward:

1. Open your graphic file; supported formats are:

a. Files with embedded geodata (GeoTiff, NOAA BSB, MRSID, IMG)
b. USAPhotoMaps Big JPEG files (UTM, you'll need the UTM zone)
c. Generic graphic files with UTM worldfiles (you'll need UTM zone, and specify datum and hemisphere)
d. Generic graphic files with worldfile in any coordinate system (you'll need to specify the EPSG code)

2. Process the raster graphic file

3. Set basic options (overlay name, JPG compression, transparency in Google Earth). You can also rescale the image by a factor >= 1.0 to reduce its size (reduces resolution, but also reduces the number of tiles).

4. Create the overlay.

You should definitely check it in Google Earth to make sure it's in the right position; just remember that Google Earth imagery can be off from 0 to 100m depending on where you are. If it looks good, put it in the \Garmin\Custom Maps directory on the external data card and test it out.

Limitations:

- Maximum file pixel size is 100 million; larger than that, and you have the option of cropping it down to a smaller size

- Tile size is set to be the largest compatible with the image dimensions and a max tilesize of 1024 pixels.

Mostly functional for free (though I do ask for a small registration fee if you find it useful)
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: -Oz- on November 08, 2009, 01:58:21 AM
Holy crap; this looks amazing.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 08, 2009, 02:37:23 PM
Thanks! Version 1.3 should be out shortly, maybe even tomorrow. It adds:

- Image preview, with a slider that changes JPG compression and shows those effects on the fly

- User-selectable image rescaling algorithm

- For images that are too large, you now have the option of either cropping or rescaling them before processing
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 09, 2009, 08:58:50 PM
Version 1.3 now available from the download page:

http://moagu.com/?page_id=155
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 21, 2009, 09:56:52 PM
I can get it to work for topo maps, but this is what I get for aerial photos:

(http://imghst.gpsfiledepot.com/1258869090.jpg)

I get this if I get them from state sites or the USGS server.

If I cut and paste something from GoogleEarth, it works.  But I cannot get this to work.  The files open in GE, but not on my CO.

I am using default settings.  I tried scaling it to 2 and 3 but nothing works.

I just installed 1.4


Any suggestios?

Thanks
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 21, 2009, 11:08:38 PM
Odd. No one else has seen this (or at least, no one else has reported it). And lots of people have reported success with aerial GeoTiffs. The program treats all TIFFs the same, regardless of whether they're topos or aerials. I'd need more information to figure this out.

The only difference between 1.3 and 1.4 is that 1.4 drops the pop-up at the beginning about it still being beta firmware.

You might try dropping back to an older version of the firmware, then re-installing the new firmware that supports custom maps.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 22, 2009, 06:57:12 AM
Thanks for the help (and program!)

I never had the beta firmware installed.

I tried 1.4 of your program to see if it solved it.

I did get one to work last night.

Here is an example of the data I used:

http://extract.cr.usgs.gov/Website/distreq/RequestSummary.jsp?PR=0&CU=Native&ZX=-1.0&ZY=-1.0&ML=COM&MD=DL&AL=37.16289,37.1026,-89.30824,-89.37813&CS=250&PL=QDB02HZ,QBD02HZ,QCD02HZ,n1602HZ,p1602HZ,vd902HZ

The top two caused the problem.  I made a map for each of them and it did not work.

The third one down (whch had the entire area) works.

Makes no sense to me.

Am I correct all I need to do is use default settings?

I am getting ready to leave town for a few days so I will not be able to give any more info for a while.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 22, 2009, 07:34:23 AM
Default settings should work fine. The fact that you got one of the images to work suggests some peculiarity with the imagery you had problems with; I'll take a look at the image formats to see if there's anything obvious. In the meantime, I'd recommend using the 1-meter color imagery; I've found that color imagery is actually more revealing of ground detail than higher-resolution black-and-white imagery.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 22, 2009, 08:09:41 AM
Thanks.  I THINK everything that I had the problem with was higher than 1 meter resolution. 
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: maps4gps on November 22, 2009, 08:28:10 AM
Used G-Raster (1.3 from Friday morning download) yesterday on new Oregon 300. 
Worked OK.
Tested USGS 0.3m high res city imagery, NAIP 1 m, 15m ETM, 30M NLCD, and 24K DRG.

Is there any way to set the zoom limits at which the custom map displays on the GPSr?

Are there any general guidelines for which zoom levels the custom maps display at versus the overall image size or the pixel size?
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 22, 2009, 09:05:47 AM
AFAIK, there's currently no way to set the zoom limits. G-Raster tells you roughly the maximum zoom level you can get before pixelization sets in (based on GPSFix's chart), and adjusts this based on your rescaling factor.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 22, 2009, 09:20:44 AM
People have reported no problems with resolutions better than 1 meter per pixel, so I don't think that's it. Thought for a second that it might be a problem with grayscale imagery, in that Garmin units might not be able to display grayscale JPGs. But if you had success with the 1-meter black-and-white image, then that's not it.

Don't know what the problem is, then, and without a compatible Garmin unit handy, it'll be hard to debug it. I'm waiting for enough people to register G-Raster at $5 to cover the cost of a compatible unit; at the current registration rate, that may take a while ;-).
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Zaphir on November 25, 2009, 09:51:30 AM
Question:  USA Photo Maps shows my area in black and white.  Google Earth shows my area in color.  I would prefer color.  Will G-Raster work with jpgs saved from Google Earth?

Thanks
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 26, 2009, 11:03:00 PM
Messed with it some more and still cannot get it to work.  I upgraded to the new firmware and it still does not work.  Has anyone been able to get it to work on higher resoluiotn pictures?  Are there some sucessful ones posted somewhere I could try to see if they work in my GPS.  Thanks
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Boyd on November 27, 2009, 06:40:54 AM
Here area bunch that I've created. I used Globalmapper to make these however. Most of them push the limits of Garmin's maximum file size and coverage area: http://www.gpsfiledepot.com/maps/byuser/282/
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 27, 2009, 08:38:46 AM
You also might try posting a link to one of your overlays that doesn't work, and see if others have problems viewing them in their units. Don't have a compatible unit myself yet, so can't help you there.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Boyd on November 27, 2009, 11:48:27 AM
Here's a "gotcha" that you may not have considered. Do NOT use anything but letters and numbers in your filenames! I just spent the better part of an evening going nuts, re-doing a map over and over but had no luck getting it recognized by my Oregon. Then I saw a thread over on the Garmin support forum about this.

My file was named plains-2007.kmz and it is not recognized. I re-did the map with the exact same data and called it plains2007.kmz and it works! Now I know better than to call it something like plains&2007, but geez... you can't even use a minus sign? Also - it isn't just the name of the file itself. All the jpeg images inside your .kmz must also respect this convention.

But here's the strange thing... when I use globalmapper to make .kmz files, it inserts an underscore character into the name (such as plains2007_kmzA3.jpg). That doesn't seem to be a problem. Bug, or feature?  :-\

Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: maps4gps on November 27, 2009, 01:21:19 PM
Quote
http://extract.cr.usgs.gov/Website/distreq/RequestSummary.jsp?PR=0&CU=Native&ZX=-1.0&ZY=-1.0&ML=COM&MD=DL&AL=37.16289,37.1026,-89.30824,-89.37813&CS=250&PL=QDB02HZ,QBD02HZ,QCD02HZ,n1602HZ,p1602HZ,vd902HZ

The top two caused the problem.  I made a map for each of them and it did not work.

The third one down (whch had the entire area) works.

One of the test I ran was on a USGS DOQQ.  Worked fine, but a 80Mb it was at the limit of 100 tiles in the .kmz file.  Hires city data at 1' resolution also worked.

Looks like something is wrong in the request summary. 1m is 42Mb, however for the 0.5m there are two 211Mb files - would expect it to be 4 times 42, or about 168Mb.

Have you tried viewing the files in GlobalMapper or something similiar?
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 29, 2009, 11:37:36 AM
I tried Boyd's Philadephia map that he made with globalmapper and it worked.  So it sounds like graster is doing something wrong. 

I was using "-" and spaces in file names.  So I tried it without them and it still did not work. 
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Boyd on November 29, 2009, 05:07:43 PM
One of the test I ran was on a USGS DOQQ.  Worked fine, but a 80Mb it was at the limit of 100 tiles in the .kmz file.  Hires city data at 1' resolution also worked.

That's interesting. Try though I might, I have never made a custom map with globalmapper larger than about 36MB. This is with ~100 tiles at 1024x1024, both for USGS 24k topo's and NJ 2007 orthophotography.

So I wonder what's going on there? 80 MB sounds really large, perhaps the jpeq quality is 100%. You cannot adjust it using my globalmapper workflow, but my understanding is that it defaults to 80% quality.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 29, 2009, 07:03:54 PM
I noticed another problem with g-raster.  I was loading a file and it said it had too many files and it would  reduce the files to 100.  But it did not.  I get a warning message when start up the GPS if I have the file loaded.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 30, 2009, 12:05:59 AM
A few comments:

- It may be something odd about the exact format of jbensman's imagery that may be causing the problem. If you (jbensman) could send one of your bad KMZ files to somebody else to try on their Garmin, and concurrently get a high-res file created by G-Raster and known to work to try out on your system, that might help in figuring out what's going on.

- Didn't know about the problem with non-alphanumeric characters and spaces in filenames. Just released 1.5, and pulled it back to fix that issue. Oddly enough, like GlobalMapper, I'd been using "_" in the jpg filenames without any problems, but I've removed them from those files too just in case. Non-alphanumeric characters are now automatically pulled from all filenames. If that's the problem, the latest version should fix that.

- The problem with not resizing when the number of tiles is greater than 100 is driving me nuts - sometimes it shows up, other times it doesn't, and with the same image file. I'm working on it ...

- The default JPG quality factor for G-Raster is 80, which should result in a 100-JPG-tile filesize of roughly 35-40 MB or so. But I think maps4gps may have been talking about the original GeoTiff filesize, not the KMZ filesize.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 30, 2009, 12:10:55 AM
Oh, there is a new feature in G-Raster 1.5: it now accepts KMZ image overlay files as input. So you can calibrate a map larger than 1024 x 1024 in Google Earth as an image overlay, save it in KMZ format, then open it in G-Raster and have that large image chopped down into smaller tiles.

Couple of limitations:

- Image overlay can't be rotated, only stretched and translated. So it only works for images where up is North. If you have an image where up isn't north, you'll need to open it in a graphics editor, rotate it to fix that issue, then load it into Google Earth to create the overlay. Fortunately, most maps that will work as overlays do have "up" as North.

- For the unregistered version, images are cropped down to 1500 x 1500 pixels; the registered version allows up to 10,000 by 10,000 pixels.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 30, 2009, 12:03:44 PM
Thanks

I tired 1.5 and still does not work.  I have used data from multiple places so I do not think I just got some bad data.  I have tried IL and MO servers and multiple data types from the USGS server.

Is there someplace I can download and try a high resolution map made with G-raster that works?

I don't really want to upload a map on this site that does not work.  I do not see a way to remove a map once it has been posted.  Are there other places where I could upload a map I made that is causing problems. 
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota mode
Post by: Boyd on November 30, 2009, 12:34:44 PM
On page one of this thread you indicated that you made working G-raster maps with USGS topo's.

I wonder if it could have something to do with your projection settings for the aerial imagery?
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on November 30, 2009, 01:37:34 PM
Whenever I use G-raster, I have never been asked for the projection.

I have got graster to work with topo maps and 1 meter color photos.  I also got it to work on a landcover file from IL's website. 
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on November 30, 2009, 04:22:10 PM
For GeoTiffs with full metadata, G-Raster doesn't need you to specify projection/datum. Only imagery with worldfiles needs that info.

I'm stumped. G-Raster works for lots of people, so I don't know why you're having these problems with high-res imagery.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on December 11, 2009, 10:43:19 AM
Version 1.6 out; now supports KMZ overlays with rotation (version 1.5 required that the overlay not be rotated, just moved or stretched).

http://moagu.freegeographytools.com/?page_id=155
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on December 16, 2009, 10:22:14 AM
I tried the new version and got the same result: garbled map display.

The first time I ran it, the program crashed with an error message of Run time error 62 Input past end of file.  I started the program again and ran it on te same file.  It worked the second time (other than not working on my GPS).

Has anyone been able to get the program to work with a high resoluiton photo?  If so, can you upload a map somewhere so I can try it?
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on December 16, 2009, 10:28:49 AM
Can you post the problematic file somewhere? I'd like to take a look at it.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: jbensman on December 27, 2009, 11:48:21 AM
Anyone know where you can upload a file like this? 
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on January 06, 2010, 11:05:03 AM
Problem with scrambled/bad imagery has been resolved. The 8-bit aerial black and white imagery from the Seamless Server was being converted into 8-bit JPG files, which the Garmin units apparently don't support. Version 2.1 now converts all imagery into 24-bit format before further processing, and aerial imagery that had the problem before shows that it works. I hope. Download it here:

http://moagu.com/?page_id=155
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Andreas on January 07, 2010, 03:46:04 AM
This KMZ file appears to have multiple overlay tiles ; G-Raster only supports KMZ files with a single image overlay.

Can you tell me what am I doing wrong? The KMZ file contains a single 1500x1500px jpeg file  (saved with Google Map Saver, calibrated in Google Earth), but I get this message from G-Raster 2.1 when I try to load the KMZ file.

doc.kml:
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2" xmlns:kml="http://www.opengis.net/kml/2.2" xmlns:atom="http://www.w3.org/2005/Atom">
<GroundOverlay>
<name>test</name>
<color>c2ffffff</color>
<Icon>
<href>files/test.jpg</href>
<viewBoundScale>0.75</viewBoundScale>
</Icon>
<LatLonBox>
<north>47.52170629024896</north>
<south>47.47738018154748</south>
<east>19.13332202564915</east>
<west>19.06798411091299</west>
</LatLonBox>
</GroundOverlay>
</kml>
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on January 07, 2010, 08:43:31 AM
As usual, you are doing nothing wrong; I introduced a bug into the last version that causes this problem. Just uploaded a fixed version of the program (version 2.2); download and install it, and it should work fine:

http://moagu.com/?page_id=155
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Andreas on January 07, 2010, 12:59:41 PM
Thanks for the quick fix.
The first 4096 x 2048 px map looks amazing on the Oregon :)
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: fleetwalker on January 10, 2010, 04:06:10 PM
I have created a KMZ file in Google Earth.  When I try to run G-Raster on it on the file, the program just hangs.  The underlying map was a PDF that I converted to a JPG using Photoshop Elements.  I think I read somewhere that this can cause problems.  Anyone else experience this or know a solution?

Thanks
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on January 10, 2010, 04:14:14 PM
Check to make sure you have the latest version (2.3). If you still have the problem with 2.3, email me the KMZ file at leszekmp at gmail dot com, and I'll take a look at it. It shouldn't have anything to do with it starting life as a PDF; once it's converted to a JPG, it should work fine.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: Andreas on January 11, 2010, 07:30:15 AM
I have created a KMZ file in Google Earth.  When I try to run G-Raster on it on the file, the program just hangs.  The underlying map was a PDF that I converted to a JPG using Photoshop Elements.  I think I read somewhere that this can cause problems.  Anyone else experience this or know a solution?

Thanks

Don't use accentuated letters in the name of KMZ file.
Title: Re: Free utility to create tiled raster overlays for Colorado/Oregon/Dakota models
Post by: leszekp on January 11, 2010, 08:41:56 AM
Don't use accentuated letters in the name of KMZ file.

I don't think this is the problem - G-Raster generates its own names for both the embedded JPG tiles and the final output tiles, and those don't include any accents, punctuation, underscores, anything that people have said might be a problem. And it has no problem reading files that include those characters.