Experiments with LIDAR

[Boyd]

Am planning to expand the map to a much larger area later this summer.

The new, expanded map is now available! This is the new coverage area - over twice the size of the original map, more than 600 gb with almost 22 million map tiles! 




Here's Shenandoah, Virginia in 3d

https://boydsmaps.com/#11.71/38.454567/-78.602919/mbx3dmidatl/40.20/73.20



3d view of the Catskills

https://boydsmaps.com/#14.00/41.998428/-74.386694/mbx3dmidatl/-174.00/72.00

This is how it looks on a phone.



You can view the 3d map in 2d North-up mode if you prefer - just click the compass to toggle between 2d/3d. But there's a separate 2d version of the same map without the elevation data if you aren't interested in 3d views, it loads much faster (note that 3d maps appear in blue text in the map menu)

Here's the 2d map of Martha's Vineyard, for example.

https://boydsmaps.com/#11.71/41.398941/-70.647387/midatlidar/0.00/0.00



I would like to continue expanding this map to cover an even larger area in the future. I still have over 200gb free disk space on the server but need to find the best use for it. Ultimately, I'll need to lease additional disk space to support future expansion.

It's not practical to create a map this large in one piece, so I divide it into smaller blocks of 100gb to 150gb of uncompressed 32-bit floating point DEM data. After processing this, I end up with blocks of a few hundred thousand .png map tiles which are ~1/4 the size of the source files. Then there is a companion set of .png tiles that contain the elevation data in Mapbox RGB format. The elevation tiles only use about half the space of the map tiles.

Before creating the RGB tiles, the geoTIFF DEM source file must be processed with Mapbox rasterio, which is a rather complex command-line program

https://blog.mapbox.com/rasterio-gdal-especially-for-python-programmers-553dabf7dd71

This encodes the DEM data as a .tif raster image which can be exported as .png map tiles. It took awhile to figure out exactly how to do all of this, Mapbox does not really support creating your own RGB DEM, they expect you to use their own pre-packaged DEM. But their DEM is about 5-meter resolution and I wanted 1-meter DEM. Using their own content can also result in usage charges but it's free to use their API with your own data. But you're pretty much on your own, I could find very little information about how to implement all of this locally but eventually figured it out through trial and error.

I create all the the tiles with Globalmapper using the option to only export fully-covered tiles, this prevents gaps between different blocks of data when I merge them with the full map. Finally, I zip the tiles, upload them to the server and unzip.

There are so many steps to the process and so many different files, I had to create a spreadsheet to track my progress!