GPSFileDepot Forums

General Category => Map Making Support => Topic started by: -Oz- on January 19, 2016, 09:20:40 PM

Title: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: -Oz- on January 19, 2016, 09:20:40 PM
All,

I have completely rewritten the How To Create Garmin Topo Maps (http://www.gpsfiledepot.com/tutorials/how-to-create-garmin-topo-maps/) tutorial. This was a major rewrite and was done as I updated the Arizona Topo map.

Biggest changes:
Anyway, I post this here for two reasons:
1. If you see any issues/errors please let me know
2. If you are using some of the processes from the previous tutorials (bat files especially) have huge updates that will speed up what you do.
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: Longostino on February 09, 2016, 03:57:19 PM
Thank you for all your work.  I am working through this tutorial, the latest version.  Most of it is working for me just fine, even though I am using Ubuntu Studio 14.04 LTS.  This means I have to accept slightly older versions of some of the software.  That is usually okay because I can still find my way around to the parts used in the tutorial.  One problem I had was the .BAT file.  I successfully reworked it into a BASH shell script (mergecontours.bat > mergedcontours.sh).  Now I am having trouble with the script that is supposed to be inserted in QGIS, mp_type, Expression field.  I am using QGIS 2.01-Dufour.  The script as written is invalid.  In fact, based on all the QGIS documentation, it is nothing like the syntax the Field Calculator suggests.  Undaunted, I thought I might rework the script using the suggested CASE WHEN condition THEN result END form.  That lead me to an even more frustrated state, because I am unfamiliar with your written syntax.  I imagine that the if (.... ,....,...) form suggests "if (this condition, return that result, otherwise...)."  If that is the case, it would seem the first case says, "if "FCODE" = '10101' OR "FCODE" = '10104', then return 0x20"  Following the rest of the code in a similar manner, the other conditions return, 0x21, 0x22 or NULL.  Now I am really confused.  Why do we need 15 characters for those results?  And are these supposed to be SPACE,  !, ", and NULL, respectively, or the decimal values, 32, 33, 34.  What seems to be written as conditions would make more sense, translating the result of dividing CONTOURELE by CONTOURINT, making it an integer by the power of 1, 10, or 100, depending on how big the result is.  So, what is the point of the conversion expression?  What is the result supposed to be?
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: Longostino on February 10, 2016, 02:24:28 PM
i found by reading ahead in the tutorial that the values to be created in MP_TYPE are literally '0x20', '0x21' and '0x22', not their ASCII or control equivalents.  Knowing that here is the code I created based on the original in the tutorial, but using the syntax the QGIS Field Calculator Expression Parser would accept as valid.  I believe I interpreted the logic correctly, and I have results that seem to reflect that interpretation:

CASE
WHEN "FCODE" = '10101' OR "FCODE" = '10104'  THEN '0x20'
WHEN  (  "FCODE"  = '10102' OR  "FCODE"  = '10105' ) AND ( "CONTOURELE" / ( "CONTOURINT" *10 )>100) AND regexp_match(  tostring(  abs(  (  "CONTOURELE"  /  (  "CONTOURINT"  * 10 ) )  / 100 -  toint( ( "CONTOURELE"  /  (  "CONTOURINT"  * 10 )  )  / 100 )  )  ) , '5' ) THEN '0x21'
WHEN  (  "FCODE"  = '10102' OR  "FCODE"  = '10105' ) AND ( "CONTOURELE" / ( "CONTOURINT" *10 )>10 ) AND  regexp_match(  tostring(  abs(  (  "CONTOURELE"  /  (  "CONTOURINT"  * 10 ) )  / 10 -  toint( ( "CONTOURELE"  /  (  "CONTOURINT"  * 10 )  )  / 10 )  )  ) , '5' ) THEN '0x21'
WHEN  (  "FCODE"  = '10102' OR  "FCODE"  = '10105' ) AND  regexp_match(  tostring(  abs(   "CONTOURELE"  /  (  "CONTOURINT"  * 10 )  -  toint( "CONTOURELE"  /  (  "CONTOURINT"  * 10  )  )  )  ) , '5' ) THEN '0x21'
WHEN  (  "FCODE"  = '10102' OR  "FCODE"  = '10105' ) THEN '0x22'
ELSE NULL END
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: MikeDk on February 20, 2016, 01:11:43 AM
I have tried the tutorial. When I get to step 8 - GridSplitter using QGIS - it it VERY slow. My lines.shp is approx. 1,7 GB and after 8 hours less than 1% is splittet. Is there any way to speed up the process?
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: -Oz- on February 21, 2016, 09:08:22 AM
Does your gridplit look similar to this?:
(https://imghst.gpsfiledepot.com/1456074377.jpg)

When I do that it tells me that I will make 36 tiles. How many tiles did it tell you it was going to do?

I just did 3.4gb of lines in ~3 hours. I assume you've gotten rid of all the excess columns at this point (as per the tutorial). Because excess data will slow it down.
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: GeoH on July 15, 2016, 04:36:33 AM
In Part 8 - Combining Data, instructions 30, 35 and 40 specify the file prefixes of Lines_, Polygons_, and Points_ respectively.

The batch file ConvertSplit2mp downloaded in Step 1 of subsection Convert to .MP specifies Line, Point and Polygon in the filename fetch and won't run with that discrepancy.

You may also want to note in instructions 30,35 and 40 that the subsequent batch file is case sensitive (e.g. the prefix of "line" will not work while "Line" will). Depending on how you choose the change the batch file.

Thanks for your work on this.  :)
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: glendeni on January 06, 2018, 12:58:17 AM
Want to thank you for all the time you put into creating this updated tutorial.  Seeing it motivated me to try comparing contours form the latest one-third arcsec data to the ones I've been using from the old CATOPO datafile for my area which has very complex topography (Big Sur, CA).  Following your directions, it was quite painless, using QGIS 2.18 (admittedly I already had some familiarity with that software).

So I only went through the "Part 2 - Elevation Data - Option 2" section.  For that, my only suggestions would be:

Using The National Map Viewer to Download the Data
() instead of "state" should use "area of interest, e.g. a state"

Processing the data - QGIS
() would help understanding at each step to give a description of what will be created following a "click OK'
 eg 12. "Click "OK" -> Click "OK to create a contour .shp shapefile
() after 8. need to insert: Click "OK"

Also, in QGIS I found that the specified filenames sometimes did not appear in the directory I expected, so had to hunt them down.  That may have been due to directories used in my last project being used instead of the the ones I expected.  Anyway, I think its good practice to click on the  "Select" button to make sure the file will be written to the directory you want.

Thanks again.

Jack
Title: Re: Huge Update to Topo Tutorial: QGIS Support/Faster BAT Files/National Map
Post by: Aodhfin on November 03, 2018, 12:41:42 PM
I am stuck on Elevation: Combine All The Contours Together
I downloaded contours for Mississippi (24 tiles)
Tried to merge using FWTools, hit the 4gb limit.
   (Note: Currently FWTools will crash if the file gets over 4gb; if this occurs you must split your contours into two separate folders and combine each half of the state separately.  For reference: for Arizona the file was only 3gb.)
Did in two chunks, succeeded.
Tutorial unclean on how to  combine each half of the state separately.  Tried to do with both QGIS & ArcMap 10.5 by combining shapefiles.  Neither worked.  Then both programs started giving errors on drawing each half.  Size limit on shapefiles is 2gb?

Help!