Converting raster landcover data to shapefiles

[Boyd]

I posted elsewhere about using Filemaker Pro to process New Jersey landcover data for a project I'm working on. I want to expand this map to neighboring states, but there just isn't any high resolution landcover data available for Pennsylvania or New York. The only thing I could find was 250k historical USGS LULC from the 70's and 80's.

But the National Landcover Database has relatively high res data (30m per pixel) from 2001 in raster format where the colors correspond to codes (see: http://www.mrlc.gov/). The problem is, how to get the raster data into shapefiles without a really tedious workflow or purchasing expensive software? I found a way to do this using Mapwel, and I thought I'd post here in case it helps someone else. If you know a better way to do this, please share!

First, I downloaded the NLCD imagery from the link on the site above. I then opened the file and exported small tiles (~1500x1500 pixels) as GeoTiff files. These can be imported into Mapwel with one click.



I then used the magic wand (trace) tool and clicked on each of the different colors to vectorize them. There are several settings to play with in the magic wand control panel, and some trial and error is needed to get the best results. But after that, it's pretty straightforward - I just clicked on each of the colors in the image (there are about 16), gave them names and let the software trace the outlines. I named the areas with the hex codes of the custom types I would use later in Globalmapper.



The results were then exported as a .mp file and opened in Globalmapper to massage the data. In Globalmapper I searched for each of the names given to the polygons and assigned the correct MP_TYPE values to them. Seems like there's probably an easier way to do this, but I didn't find it.



After getting all the correct MP_TYPE's assigned, I combined the landcover with the other layers of my map, exported as a .mp file and compiled with cgpsmapper. My custom type file is the secret sauce that makes it all work in Mapsource.



This technique should work with any kind of raster data that uses color codes to represent values. If you just want to produce simple, green forest shading, that would be easy to do from the NLCD  since you'd only need to vector three different colors (for Deciduous, Evergreen and Mixed forest).

Mapwel also has a one-click auto-vectorizer and I played with that also. It creates a lot of polygons, and I didn't like the pixellation so much. The Magic Wand tool has a setting to create curved polygons.

I have the advanced version of Mapwel, but I believe the free download may do all of this as well. The free version won't send a fully processed map to a GPS, but it will let you save files on your computer. I'm not certain whether it allows the export of .mp files however. Regardless, you can experiment with the magic wand tool in the free version and see if it looks useful.

Here are a couple screenshots from the Nuvi 3790 along with the North America DEM file.

[Seldom]

First, I downloaded the NLCD imagery from the link on the site above. I then opened the file and exported small tiles (~1500x1500 pixels) as GeoTiff files. These can be imported into Mapwel with one click.

Boyd, what program did you export small tiles, Global Mapper?  Why the 1500 pixel file size? Did you have to pick each island color with the magic wand, or could you select all islands of a given color?

I did this with the National Landcover database and Photoshop and Illustrator.  In doing the Photoshop work, I lost the georeferencing of the TIF, which I then had to re-reference using GM.
Since Mapwell is a lot cheaper than the Adobe products and maintains the georeference, it looks like a nice option. 

[Boyd]

Yes, the tiles are part of my Globalmapper source files that I have built layer-by-layer over the past few years. It was a pretty arbitrary decision originally - I drew a rectangle around the state of New Jersey with a little extra space, then exported it as a 5x7 grid which made tiles that are about 20 miles wide x 25 miles high. The NLCD tiff's for these tiles are 1270x1212 pixels. Some mental arithmetic made me think this was about right for 30 meter/pixel data.

I hadn't used Mapwel in a long time, so I upgraded to the newest version (which is free to registered users). It's pretty impressive how fast it can process raster images. Download the free trial and give it a try.

There are a number of settings and I find I have to play with them for each image to get the best results. Tolerance determines how precisely it will match colors (0 would be an exact match). Simplicity determines how many sides the polygon will have (0 is the max). You can choose to draw either curves or straight lines through the pixels. You can select tracing just the outline or outline with islands. Using the "Add Similar" setting, it will select all pixels in of the desired color. So you have to do that for each color in the image's pallette. Like I said, that is only about 16 for the NLCD (a few of those are specific to alaska too )

I found a way to automate things. After creating the .mp file in Mapwel, I open it in Globalmapper then export it as a shapefile. I import the .dbf file into Filemaker, and assign the Area Name I gave each polygon in Mapwel to the MP_TYPE attribute, discarding the value that Mapwel had chosen. I also use a relationship in my database to create some other fields. The modified .dbf and shapefile is opened again in Globalmapper and exported to an .mp file for compilation by cgpsmapper.

[Boyd]

Have been working on this some more, and it definitely takes a lot of trial and error to get the best results. At .7 miles zoom in mapsource, it generally looks pretty good now





.5 miles zoom is also pretty good







At .2 miles, it starts gettig pretty rough.





There may be better ways to do this, but eventually you run into the quality limits of the landsat data. Anyway, it's been a good project for me and I hope to post a map soon now.

[maps4gps]

Very impressive.

[Boyd]

Thanks. For one of my 20x25 mile tiles, this technique generated 140,000 polygons! Still seems to work fine though.

[Boyd]

Wow, looks like the new version of GlobalMapper (released on 1/13) can now do this as well. This looks like it would do the same thing that I was doing with Mapwel. Damn, now I am going to be tempted to upgrade. 

Global Mapper Software LLC is excited to announce the availability of our next release, Global Mapper v12.01

Added option to create area features from equal colors/elevations in any loaded raster/elevation layer by right-clicking on it in the Control Center and selecting the new option to do so. This is useful for things like extracting vector coverages from something like a land use/cover raster image, like NLCD data.

[Seldom]

Glad somebody out there reads the fine print.  I'll have to try this one out.

[Boyd]

I gave in (no self control  ) and upgraded. BTW, if you do, look at the banner ad that runs on the GlobalMapper forums. It offers a 10% discount on the upgrade cost, saving $14.

This new feature is very cool but requires some experimentation to understand the settings. I wish they gave you a bit more control, but it's still pretty impressive. I used about a 2 mile x 2 mile section of 1/9 arc second DEM to experiment with. Using the default detail setting, it generated something like 1.9 million polygons - basically one square for every pixel!

But playing around with a smaller test file I was able to get a better feel for it. You could easily use this feature to create pseudo-DEM for a Garmin map. Have you ever seen this article? http://www.cgpsmapper.com/download/Creating%20custom%20types%20to%20represent%20elevation%20data.pdf

That article uses a time consuming technique to generate polygons for different elevations. You could use this new Globalmapper feature to do the same thing with just a few mouse clicks. I used these settings for example




And I created this. I have selected one of the polygons to show what they look like




Now I used a setting that wouldn't make too many polygons - this created only 348 for an area measuring 1.1 miles x 1.4 miles. Of course, this is a waste of the good 1/9 arc second DEM, but I was just playing around. Using a smaller value for "maximum match distance", such as 0.10, would capture more detail and creates a lot more polygons - over 23,000 in this case!




Here's the lower resolution version with the addition of contour lines. All you would need to do is convert each ELEVATION value to a custom type to turn this into a Garmin map - another job for FileMaker




Cool stuff, I can think of lots of things to try with this.

[Seldom]

Biggest tile I've been able to vectorize.  All "wooded areas" converted to the same semi-transparent green. 

One thing I've noticed was the selection by COLOR attribute is pretty fussy.  GM drops leading "0"s and replaces them with spaces so a text search by COLOR for RGB(028,102,051) needs to be expressed as "RGB(space space 28, space 102, space space 51 space)"

[Boyd]

Yeah, I don't like the COLOR attribute either. They need to beef up that dialog box a bit, and let you create some custom attributes. Looks like the ELEVATION attribute will be easier to manipulate, since it's just a height in meters.

What was the source file in your example? NLCD? I need to play with this some more, still not sure but I think I may like Mapwel more for vectorizing NLCD data. Globalmapper can vectorize color images that look just like the originals, but at the expense of creating huge numbers of polygons. From my quick tests, it looked like it would create a square for every pixel in the source image when you set the match distance to zero. That gets out of hand really quickly.

To make it look best in Globalmapper, you can edit all the polygons and set their borders to Null.

[Seldom]

Source data is NLCD.  Pale green color is my default color for all wooded areas in GM.  When GM does the conversion it sets the color of the polygon to match the COLOR attribute so the only way you can differentiate the base raster from the output vector is the black lines indicating polygon edges.

I don't think the individual pixels issue is a concern with NLCD, because they get combined to make large polygons.  The workspace file for the image above was about 35 Meg.  When I added 2.5 more tiles it bumped up to 95 Meg.  My standard tile is about 30 miles square with pixels at 30m grid.

FWIW, when I did my attribute edit for the COLOR, I set the GM type to "wooded area", set the display type to default for wooded area, and added a "Covertype" attribute which I set to "coniferous", "deciduous", or "mixed", based on the NLCD color chart.

 

[Hillbilly61]

It's not really a raster file if it can be converted to a vector graphic. Raster data are pure bitmaps, much like what comes out of a digital camera. Vector data is all about lines, points and areas - with area fill properties, etc.

Many image (raster) programs claim to be able to trace raster graphics, few do it very well, especially for high resolution bit mapped graphics that contain data on the meter or less resolution level.

[Boyd]

It's not really a raster file if it can be converted to a vector graphic.

Nonsense. Read this whole thread. There are various ways to convert raster data to shapefiles. The results will vary in quality. This new feature of Globalmapper can do it quite accurately but produces mountains of data. Basically, you can create a square for every pixel in the source image.

The method I described using Mapwel earlier in the thread is less faithful as it creates shapes that are more generalized.

Yes, clearly there are trade-offs when converting a raster image to vector format, and it works better for certain kinds of data. But your quote above is simply not true.

[Boyd]

I have attached a small example of how this Globalmapper feature works. In this case, the vectorized version looks exactly the same as the original .tif file. The attachment is a ZIPped folder containing the original source raster image (test.tif) and the shapfile that I exported (test.shp, test.dbf, test.shx, test.prj).

The source image is very small - only 117 x 112 pixels, but it should be enough to give you an idea. You can do the same thing with much larger images, but the file size will grow exponentially into millions of polygons.

At 117 x 112, the source image contains about 13,000 pixels. The exported shapefile has about 7,000 polygons in it, so Globalmapper was able to achieve some savings by grouping adjacent pixels of the same color together. I used the highest quality setting to create this example.

The source .tif file was cropped from NLCD data in the vicinity of Wilmington, DE.