Help requested with Colorado Public Map data

[modernhistorian]

Colorado State publishes a set of vector data showing land ownership. The directories consist of gdb files and layer files. I'd like help in getting this data into my etrex 20. I run Mac OS 10.6.8 and have base camp and GPSBabel. Thanks for any assistance.

[Boyd]

.gdb is an ambiguous file extension. Garmin uses it ("Garmin Data Base") to store user data for Mapsource and Basecamp. But most likely, what you have is an ESRI Geo Data Base file: http://www.esri.com/software/arcgis/geodatabase

These are proprietary and I'm not sure that you will find any free software to use them on the Mac. They have been a problem for Windows users as well until Globalmapper recently started supporting them. Globalmapper is a Windows-only program that costs about $400. ESRI's own software costs $$$$$.

Generally speaking, you are going to have a hard time making maps on the Mac. And I am a Mac user myself dating all the way back to 1985 (typing this on my MacBook Air right now). In fact, I didn't get a Windows machine myself until I started making my own maps about 5 years ago. I personally gave up on trying to do this on the Mac.

Indrid Cold, one of the moderators here, is also a Mac users so maybe he has some other ideas? For starters, I would see if these are really ESRI files and if Colorado offers the data in another format. Some government agencies have been criticized for offering public data in .gdb format because it require expensive proprietary software to access.

If, perhaps, the .gdb files are actually Garmin files, then it should be much easier to make your own map from them.

[Seldom]

Have you read these tutorials?  If so you know you need to get your data into MP format and compile it with cgpsmapper.
Another, more Mac friendly, but technical process would be to convert your data into OSM format and compile it with mkgmap, which is Java based.  Global Mapper will convert from GDB to OSM, but it will only run on Windows.  You can find more about mkgmap, and OSM data at the OpenStreetmap site.

To add to what Boyd said, the GDB files supported by Global Mapper version 13 are only Geodatabase version 10 or later.  ESRI hasn't published the standard for GDB 9 or earlier.  GPSbabel will probably convert Garmin GDBs to GPX, but I doubt it will touch ESRI GDBs.  I you can get the data you want in a shapefile format, and you have a PC, you can import it into GPSmapedit (shareware) directly. 

[maps4gps]

Could you give the url for these.  A quick web search only showed a 'state trust lands' from a co state website.

[modernhistorian]

Thanks for the prompt reply. I do have a windows netbook that I can use if necessary. Below I've copied the methodology section of their tech doc (sorry if too long). I see a reference to ArcGIS in it. Does this mean I'm out of luck?


The first step in this ambitious project was to collect all the spatial data available from federal and state agencies. Datasets from the federal agencies were often maintained in separate databases or agency departments. Once these data were collected they were merged together to make one complete feature class for the respective agency. All input datasets only contained features for that specific agency. For example, datasets from the twelve individual National Park Service (NPS) lands were gathered and merged to make one NPS feature class. This was also done for the US Forest Service (USFS), US Fish and Wildlife Service (USFWS), Bureau of Land Management (BLM) and State Land Board (SLB) datasets. (For a list of all datasets used in COMaP v9, see Table 5).

While the initial processing steps have remained constant for each version, methods used to integrate all of the datasets into one have evolved. The first two versions made use of Topology feature classes and editing tools in ArcGIS 8.3 to address the numerous overlaps and gaps among the various input datasets. Though these methods worked quite well, we found that manual editing and updating the dataset with more current input datasets was too time consuming. Additionally it was difficult to manually address topological errors in a consistent manner.

For the next three versions, COMaP v3, COMaP v4 and COMaP v5, we assembled the input data in a more systematic and automated manner. We used Model Builder and Python scripting in ArcGIS 9.0 to systematically merge the various input datasets. By building a program we were able to control the order in which datasets were compiled. The datasets were ranked according to their quality, vintage, spatial extent, and our level of confidence in the data (Table 1).

Table 1: Compiled dataset rankings for COMaP v5
Rank   Dataset
1   Land Trusts
2   Non-governmental organization datasets
3   City datasets
4   County datasets
5   Colorado State Land Board
6   Colorado Division of Wildlife
7   Colorado State Parks
8   US Fish and Wildlife Service
9   National Park Service
10   US Forest Service
11   Bureau of Land Management

The Python script outputs two layers: the "overlap layer" (comap_v5_overlap) and the "unedited layer" (comap_v5_unedited). As the different input layers are intersected, those areas that overlap are written to the "overlap layer" and the overlapping area from the higher ranked input dataset is left in the "unedited layer". The "overlap layer" contains attributes that show the differing management codes and the source data that contained the overlaps.

After removing overlapping areas from input data sets, the remaining datasets were unioned to create the initial COMaP layer. Finally, any lands in the state that had not been identified in any of the input source datasets were assigned as private lands due to the fact that there is no reliable statewide layer of private lands.

After the "unedited layer" and the "overlap layer" were created, editing processes were performed on a copy of the "unedited layer". Initially, a length/area ratio was calculated and all features with a ratio greater than 0.8 and an area of less than 0.1 acres were eliminated into the polygon with the largest shared border. Next, county-by-county manual edits were performed consisting of minor spatial adjustments, cuts and merges. All of these edits were based on assumptions concerning the quality of the input datasets and were meant to maintain proper adjacency throughout the final dataset. For example, if there was a 20 meter gap between a NPS feature and a BLM feature, the gap was filled with the lower ranked BLM feature relying on the assumption that the spatial accuracy of the BLM input was not as high as the accuracy of the NPS feature. In addition, most edits were performed with the guidance of the BLM's Geographic Coordinate Data Base (GCDB) in an attempt to distinguish which features were properly aligned. Other features inspected during the editing process included features smaller than 10 acres having a length/area ratio of greater than 0.5 and all features that had an angle less than 1° between any 3 vertices.

After completing the edits to the entire dataset, the final COMaP layer was created, comap_v5_final. Unioning this "final layer" and the "unedited layer", and then selecting areas that had differing attributes between the two, resulted in the "edit layer" (comap_v5_edits). This edit layer tracks all of the manual edits that were performed on the original data to create the final COMaP. It contains attributes that show the original (unedited) state and what area was edited to represent in the final layer.

In the interest of condensing the dataset, the "unedited layer" was removed from the geodatabase. Therefore, each COMaP v5 geodatabase (public and private) contains three feature classes: comap_v5_final, comap_v5_edits, and comap_v5_overlap

The final COMaP layer (comap_v5_final) could be used as a stand-alone feature class for land stewardship in Colorado. The other three layers illustrate the major processing steps in the project. The "edit layer" provides a record of where we manually edited the dataset. The "overlap layer" shows where the input datasets overlapped and which datasets were involved. Of primary interest to GIS data creators is the "overlap layer" which should assist in identifying discrepancies with other agencies. Symbolized layer files for all layers are included with COMaP v5.

For COMaP v6 – v9, we started with the previous version as a base and simply added easements and open space as they became known to us. Many easements were identified by traveling to various County Clerk's offices around the state and gathering legal documents of those easements. Changes to v6 included updates to BLM, CDOW, and State Land Board data. V7 also has updates to BLM, State Land Board, and CDOW data as well as the inclusion of many new easements from organizations such as Colorado Open Lands, Aspen Valley Land Trust, Black Canyon Regional Land Trust, The Nature Conservancy and many others across the state. In the interest of easier downloads, we have eliminated the edit layer and overlap layers from the v6 –v9 databases.

COMaP v8 included updates to the following federal and state agencies: BLM, National Park Service, U.S. Forest Service, U.S. Fish & Wildlife Service, CO Division of Wildlife, CO State Parks, and State Land Board. Additional data were incorporated from Great Outdoors Colorado (GOCO), land trusts, counties, cities and other non-profit organizations. However, some data that we received from agencies within the past year were not incorporated due to our inability to verify the accuracy of the data or because of a lack of attribute information provided.

COMaP v9 includes updates from the following: Colorado Division of Parks and Wildlife, Colorado State Land Board, Denver Parks, Larimer County, City of Westminster, Town of Parker, City of Aurora, City of Louisville, Denver Water Board, GOCO-funded easements, Rocky Mountain Elk Foundation, San Isabel Land Protection Trust, Palmer Land Trust, Sand Miguel Conservation Foundation, Mountain Area Land Trust, La Plata Open Space Conservancy, Black Canyon Land Trust, Aspen Valley Land Trust, Palmer Land Trust, and Legacy Land Trust.

[modernhistorian]

Could you give the url for these.  A quick web search only showed a 'state trust lands' from a co state website.

The address is nrel.colostate.edu/projects/comap/ (not allowed to post the link, sorry). I didn't notice before that there are kml files, and have tried those, but with other problems.

The kmz files (eg, Public_Access) are kmz, and load and view fine in Google Earth. But Basecamp insists they are not valid. I saved one as a kml from within Google Earth, with the same result. I tried using GPSBabel to convert this kml file, and BC seemed to load that, but with no display. I noticed that the 165 mb kml file became a 4 kb gdb (also tried mps) file, so I assume most of the info was lost.

My questions now are: why won't BC import kml files when the manual says it will? How could I use Babel to convert these kml files to something BC will import (file type and options)? Thanks again.

[Seldom]

I'm pretty sure that BC only supports KML/KMZ raster graphics for "CustomMaps", but GPSbabel will convert vector KMZ to GPX which you should be able to import into BC.

[modernhistorian]

Would that be GPX XML as the output file? I got the same empty file in BC with that. There are scores of file types listed in Babel. I am assuming I should use "Google Keyhole Markup" for the input, but what file type and options should I use for output (please, the actual name that Babel lists)?

[Seldom]

GPX XML is the correct output, and Google Keyhole Markup Language is the correct input.  Mind that if you have a KMZ file you need to unzip it using 7-zip or Mac equivalent to get KML that GPSbabel will read.  Also in BaseCamp you'll need to "Import" rather than Open the GPX.

[Boyd]

Zip files are natively supported under MacOSX and double-clicking them will unzip. But you might need to change the file extension to .zip instead of .kmz. It would be a good idea to look inside the .kmz file and see what's there since it could be a variety of different things.

If you find raster imagery (.jpg or .png) files, then I doubt that GPSBabel will help much.

[dbperry]

Looks like the KMZ files have a single doc.kml file with multiple vector polygons coded in it...

[Boyd]

AFAIK, polygons are not supported in .gpx files so that may be the problem. But you may be able to convert them to polylines.

[modernhistorian]

GPX XML is the correct output, and Google Keyhole Markup Language is the correct input.  Mind that if you have a KMZ file you need to unzip it using 7-zip or Mac equivalent to get KML that GPSbabel will read.  Also in BaseCamp you'll need to "Import" rather than Open the GPX.

If I try to unzip these kmz files (changing extension to .zip) then I get a smaller file, ie archive is compressing them. To clarify, here are the steps I take-

1. I use Google Earth to open Public_Access.kmz, obtained from the ColoState site. It opens and displays properly.
2. I use GE to save that place as Public Access.kml. It is much larger than the .kmz file.
3. I delete Public Access from GE, then open Public Access.kml. It displays in GE the same as the .kmz file. This leads me to conclude that there is nothing wrong with either the .kmz or .kml file.
4. I use GPSBabel to convert the working .kml file to something Basecamp will understand. I select Keyhole as the input format with default as input file option. I click waypoints, routes, and tracks. I select GPX XML as the ouput. Here is the command line statement in Babel-

gpsbabel -w -r -t -i kml,deficon=Public,lines=1,points=1,line_width=6,line_color=99ffac59,floating=0,extrude=0,track=1,trackdata=1,trackdirection=0,labels=1 -f /Users/timothy/Documents/Google map/Public Access.kml -o gpx,suppresswhite=0,logpoint=0,humminbirdextensions=0,garminextensions=0 -F /Users/timothy/Downloads/Public.gpx

Translation successful

Although it seems to work, the output file is only 4 kb. I can import it into BC, but the contents are empty, as shown by the detailed list brought up with a right click on the "recently imported" data in BC.

I conclude that the problem lies with Babel. It seems to output an understandable (by BC) file but not fill it with any of the contents. Perhaps I am setting the wrong format or options in Babel. For anyone who understands the command line options listed above, are any wrong?

[Boyd]

If I try to unzip these kmz files (changing extension to .zip) then I get a smaller file, ie archive is compressing them. To clarify, here are the steps I take

Heh, I got the same problem of archive wanting to compress instead of decompress. But you can do it from a unix shell using the unzip command:

Code Select Expand

% cd /Users/ostroff/Downloads
% unzip /Users/ostroff/Downloads/BLM_Owned_Lands.zip
Archive:  /Users/ostroff/Downloads/BLM_Owned_Lands.zip
  inflating: doc.kml

Don't know that there is really any value to that exercise however, since the only thing inside the .kmz is vector data in a single file named doc.kml.           

[Seldom]

I downloaded BLM_Owned_Lands.kmz from the site modernhistorian referred to.  Doc.kml loads into GE as a set of polygons, all west of Denver.  When GPSBabel converts to GPX it the result is this:

Code Select Expand

<?xml version="1.0" encoding="UTF-8"?>
<gpx
  version="1.0"
  creator="GPSBabel - http://www.gpsbabel.org"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://www.topografix.com/GPX/1/0"
  xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
<time>2012-08-02T21:49:08Z</time>
</gpx>
No data.

I haven't been able to find a polygon>polyline conversion, and the output options for GPX are waypoints and tracks.