Welcome, Guest. Please login or register.

Login with username, password and session length
Forums Search:  


Author Topic: How big are these files supposed to be?  (Read 4870 times)

erik.the.awful

  • Jr. Member
  • **
  • Posts: 34
  • Karma: 0
    • View Profile
How big are these files supposed to be?
« on: December 28, 2008, 09:08:46 AM »
My dem mp files are very large. I don't see how these maps are going to be broken down into anything close to a 8mb file. Does converting them to Img's compress them somehow? My elevation data alone for a 30x90 quad is sometimes as large as 600mb. Did I get to detailed of files off of the  seemless server?

-Erik

maps4gps

  • Expert Advisor
  • *****
  • Posts: 1532
  • Karma: 18
    • View Profile
Re: How big are these files supposed to be?
« Reply #1 on: December 28, 2008, 10:40:41 AM »
In my experience for contours the .img file sizes are about 6% of the .mp file sizes.

Do you mean a 30x60 minute quad? 

I am using GM for contouring, etc., however, I would say your spatial dimension (30x60) needs to reflect the contour interval you are using.  If the resulting .mp files are as large as 600Mb, you need to go to smaller sized .mp files (15x15 minute) and/or use a larger contour interval (40 versus 10 feet, etc.).

What geographic area are you contouring and at what contour interval?

Using 30m, 10m, or 3m from the Seamless Server should not make much of a difference in the .mp file sizes.  However, the grid cell spacing has an effect on what the smallest appropriate
contour interval is for any given slope angle.

erik.the.awful

  • Jr. Member
  • **
  • Posts: 34
  • Karma: 0
    • View Profile
Re: How big are these files supposed to be?
« Reply #2 on: December 28, 2008, 11:52:00 AM »
I'm using GPSmapedit. On the scaned 7.5 min USGS quads for the entire area I'm mapping right now is in 20' contour intervals. I'm mapping the southern Appalachian mountains for those into hiking primarily (and so I set the contour interval to 20' in rough terrain that can make a difference). I have included all of the major long distance hiking trails I could find in my state as well as all of the data in the tutorial. My map encompasses part of 6 states, the major upthrust and some of the piedmont or plateau. It is 40 100000:1 quads in its entirety.

Good to know there is some compression, I have a 60csx and while I didn't want to load it all at once, at least 8 quads at a time would sure be nice....

Thanks for the 411!!!  ;D

-Erik

erik.the.awful

  • Jr. Member
  • **
  • Posts: 34
  • Karma: 0
    • View Profile
Re: How big are these files supposed to be?
« Reply #3 on: December 29, 2008, 06:04:38 AM »
Okay I upped my pagefile to reflect the larger files, now all of the elevation data will load, as well as all of my other data except for NHD_flowline.shp, the file is about 1.7GB for the area untrimmed and it will not load in GPSmapedit. It is reporting that it will not fit into virtual memory, but I have over 4GB allocated for the minimum. Because my map crosses so many watersheds I have an insane amount of hydrology data. Any suggestions would be helpful as I am ready to combine and split if I can get this to load up.

maps4gps

  • Expert Advisor
  • *****
  • Posts: 1532
  • Karma: 18
    • View Profile
Re: How big are these files supposed to be?
« Reply #4 on: December 29, 2008, 10:11:02 AM »
I would say you need to divide your data\area into smaller sized 'subarea'; probably 15x15 minute quads for 20 foot contours in the Southern Appalachians.  My file sizes for 20ft contours in that area were over 20% bigger than anywhere in the Western mountains.  You do not have as much hydro (or contours) as I processed for the 50 States; the key to handling it is to divide it into smaller manageable 'quads'.  I would say your area is way to large for a single .img file, but could be easily handled in 200-300 smaller (15x15) quads.  It would just be another mapset in MapSource where you could combine part of all of it with other mapsets to be sent to the GPSr in a single gmapsupp.img file. 

-Oz-

  • Map Maker!
  • Administrator
  • Expert Advisor
  • *****
  • Posts: 1580
  • Karma: 28
    • View Profile
    • GPSFileDepot
Re: How big are these files supposed to be?
« Reply #5 on: December 29, 2008, 12:53:40 PM »
gpsmapedit has very limited memory it can use; that is the issue you're running into.

definitely need to shrink the flowline too; try to keep it below a gb.

I had many issues when I went to 20ft; I generally stick to 30-40ft out West; Mississippi was 10-20ft though because it was so flat.
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

erik.the.awful

  • Jr. Member
  • **
  • Posts: 34
  • Karma: 0
    • View Profile
Re: How big are these files supposed to be?
« Reply #6 on: December 29, 2008, 01:50:46 PM »
Okay thanks for the info, I can get the DEM's to work fine most are pretty small but the ones in the upper crest of the blueridge and smokeys are larger due to the extreme height over base level. The  one that has clingman's dome is huge. Is there a way for me to split up this already compiled NHD flowline shapefile into four smaller files so I can trim it, once trimmed it should be much closer to a GB as the subbasins extended far out from my intended area. I can't use GPSmapedit b/c of the memory limitation and I don't want to re run the NHD data in segements again unless I have to. Can I do this with the demo of global mapper? What about saving the flowline data as a .mp file and then going ahead and compressing it down into a .img file, then compiling the map from there. Everything fits on gpsmapedit for splitting it into 30x60 quads except the NHD flowlines....
-Erik

-Oz-

  • Map Maker!
  • Administrator
  • Expert Advisor
  • *****
  • Posts: 1580
  • Karma: 28
    • View Profile
    • GPSFileDepot
Re: How big are these files supposed to be?
« Reply #7 on: December 29, 2008, 03:51:52 PM »
You can compress it; i'm just not sure how you'd create the .mp file if you can't open it in gpsmapedit.  Global Mapper could probably open it but I think the demo blocks a lot of the exporting and that is where I normally grid it.

Now, I will tell you that I've been using postgis to cut all my files down to just the area I want (which is just inside the state boundary in my case).  If you are interested in doing that you'll need a solid object that represents your borders.  Make sure it is in the same projection as the flowlines.

Then, load the flowline data into postgis and the state into postgis; the command is similar to this:
Code: [Select]
SELECT i.gid, i.comid, i.fdate, i.resolution, i.gnis_id, i.name, i.lengthkm, i.reachcode, i.flowdir, i.wbareacomi, i.ftype, i.fcode, i.shape_leng, i.enabled, i.mp_type, ST_Intersection(state.the_geom, i.the_geom)  As the_geom
INTO cut_nhdflowline
FROM state INNER JOIN nhd_flowline i ON ST_Intersects(state.the_geom,
i.the_geom)
Where your table is called nhd_flowline and the state is loaded into a table called state (or whatever your border is).  Then just run that command and then extract it back out.  This command WILL take quite some time to run; I have left it to run overnight before for a whole state but it did cut all my files down to just the area I'm working in.
Dan Blomberg
Administrator - GPSFileDepot
GPS Units: Garmin Dakota 20, Garmin GPSMap 60csx, Nuvi 255W, Nuvi 250W, ForeRunner 110, Fenix 2, Tactix Bravo, Foretrex 401
See/Download My Maps!

 

anything