GPSFileDepot.com
 

News:

Welcome to GPSFileDepot!

Main Menu
Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - bacall213

#1
Hi all,

I've made it most of the way through the [awesome] tutorial on making custom Garmin Maps, but I'm caught up on the USGS NHD data. I submitted my orders for the NHD data 3 or 4 days ago but never received the download links.

I emailed the helpdesk at USGS and received a reply later in the day indicating that my requests are queued up behind 80 large requests and it may take 3-4 weeks for my requests to be generated.

As an alternative, the gentleman from the The National Map helpdesk suggested that staged subregion data in File GDB or Personal GDB formats would be much quicker to obtain.

I looked into the differences in the file formats and found that there's an overwhelming preference  for GDB files over Shapefiles. However, since I'm not all that familiar with what's going on in the background to produce the final map nor familiar with these formats (my knowledge of cartography is very limited), I don't know if I can actually substitute the Shapefiles with File/Personal GDB files. To use the same "word for word" process, I understand I would have to convert the files first (GBD > Shapefile). However, since I don't know much of these formats, I'd hate to end up with the Pacific Ocean on top Mount Olympus.

Can anyone offer advice on how to proceed? Would it be better to wait for the Shapefiles to process, or can I work with the File GDB or Personal GDB data instead?

Thanks,

Brian
#2
Thanks for the info, -Oz-.
#3
Hi All,

I've been going through the amazing tutorial on making custom Garmin Maps. I got to the section for GNIS data, but ran into some problems.

The first problem is that GNIS Processing requests an old version of the features file from geonames.usgs.gov (at least for Washington State) and continues despite receiving a "404 Not Found" from the webserver.

The second problem is that GNIS Processing calls pgsql2shp.exe with a "-p<password>" flag. In version 9.2, the password flag is a capital "P" ("-P") and a lowercase "p" ("-p") is used to indicate a specific server port.

Here's a timeline of what I've observed and my workaround. I identified the following through the use of packet captures and Sysinternals Process Monitor.

GNIS Processing Workflow:

1) Start GNIS Processing with Internet
2) Select state and output directory (I'll use Washington State for my example)
3) Click "All At Once"
4) GNIS Processing requests "geonames.usgs.gov/doc/stategaz/WA_Features_20090401.zip"
5) geonames.usgs.gov returns "404 Not Found"
6) GNIS Processing seemingly continues without error. The only noticeable error (aside from the lack of output) is in error.log where it's indicated that "gnis.shp" couldn't be found.

Workaround:

1) Download the latest features file from geonames.usgs.gov (WA_Features_20130210.zip)
2) Rename the zip file to "WA_Features_20090401.zip"
3) Unpack the zip file and rename the .TXT file to "WA_Features_20090401.txt" and place it in the output directory.
4) Stand up an Apache server and serve "/doc/stategaz/WA_Features_20090401.zip"
5) Add line to Windows hosts file pointing to Apache server as "geonames.usgs.gov" (I used a VM that had a separate IP)
6) Confirm the webserver is accessible and the URL is valid
7) Execute GNIS processing as usual. The correct .SHP files are created... if you don't have a password set in GNIS Processing...

GNIS processing appears to pull the .ZIP file off the server, but fails to unpack it in the correct location. I found that I had to put an unpacked version of the file in the output directory for it to read. Since I had a screwy workaround, this may be have been an unintended consequence. I couldn't identify what was going wrong through the use of Process Monitor.

And now for the password issue...

- GNIS Processing allows for the entry of a database password
- The command called is:
>> "C:\Program Files\PostgreSQL\9.2\bin\pgsql2shp.exe" -f "C:\GIS\washington_topo\5 gnis\gnis.shp" -h localhost -u postgres -p<password> postgis gnis
>> postgis is the name of the database
>> gnis is the name of the table
>> <password> is the password I have on the database, without brackets and removed for security reasons
- In PostgresSQL 9.2, the "-p" flag is for port, whereas "-P" is the actual flag that should be used.
- I followed the instructions early on and enabled no password authentication when coming from localhost, so I was able to remove the password from the config to make it work near-flawlessly with my above workaround. Leaving the password in place results in pgsql2shp.exe throwing an error that isn't handled nor exposed.

And finally, the .SHP file generated...

- The column "MP_TYPE" did not exist in several tests I ran. Instead, there was a "TYPE" column.
- I haven't completed the map yet (waiting on water-features data), so I don't know if it will work with the column named "TYPE," but it does diverge from the instruction set which indicates it should be "MP_TYPE."

To summarize:

- GNIS Processing requests an outdated version of the features file from geonames.usgs.gov and doesn't handle the error properly
- GNIS Processing calls pgsql2shp.exe with an invalid flag on PostgresSQL 9.2
- The GNIS data .MP file generated in Part 5 may be invalid once it's all put together

Is the author of GNIS Processing still around on these forums? Could the program be updated? Or, if the author can't, could it be open sourced so I the community could maintain the code base and update it for changes like these?

Thanks,

Brian