README.md

October 8, 2023 · View on GitHub

tinygpkg-data

Small, TWKB compressed, and simplified GeoPackage datasets to be used with tinygpkg.

Report Bug · Request Feature

Table of Contents

About
Datasets
Featured Variants
Parameters
Variants
Contributing
License
Acknowledgements

About

This repository contains a set of scripts and tools for generating Tiny Well-known Binary (TWKB) compressed GeoPackage files from various open data sources. Check out Featured Variants or releases to download the data or keep reading for more details and comparisons.

Datasets

Name	Contents	Features	Source	License
ne_110m_admin_0_countries	Country borders, 1:110m scale	177	Natural Earth	Public Domain
ne_10m_admin_0_countries	Country borders, 1:10m scale	258	Natural Earth	Public Domain
ne_10m_urban_areas_landscan	Big cities only, 1:10m scale	6018	Natural Earth	Public Domain
geoBoundariesCGAZ_ADM0	Country-level administrative boundaries	200	geoBoundaries	Attribution required
geoBoundariesCGAZ_ADM2	City-level administrative boundaries	49689	geoBoundaries	Attribution required

Featured Variants

These are some selected useful variants of the datasets. See Variants for the full list.

File	Description	Size (vs. source)	Gzipped size (vs. source)
Tiny (TWKB)
📥 ne_110m_admin_0_countries_s5_twkb_p3	117 country borders	393 KB 46.9% of 838 KB	116 KB 56.0% of 208 KB
📥 ne_10m_admin_0_countries_s4_twkb_p3	258 country borders	1249 KB 9.4% of 13287 KB	782 KB 17.0% of 4614 KB
📥 ne_10m_urban_areas_landscan_s5_twkb_p3	6018 city borders	4333 KB 7.5% of 57682 KB	1780 KB 22.0% of 8078 KB
📥 geoBoundariesCGAZ_ADM2_s5_twkb_p3	49689 city borders	16420 KB 6.0% of 272859 KB	10782 KB 6.4% of 168993 KB
Compatible (WKB)
📥 ne_110m_admin_0_countries_s5_wkb	117 country borders	557 KB 66.4% of 838 KB	215 KB 103.8% of 208 KB
📥 ne_10m_admin_0_countries_s4_wkb	258 country borders	4997 KB 37.6% of 13287 KB	2566 KB 55.6% of 4614 KB
📥 ne_10m_urban_areas_landscan_s5_wkb	6018 city borders	21704 KB 37.6% of 57682 KB	4663 KB 57.7% of 8078 KB
📥 geoBoundariesCGAZ_ADM2_s5_wkb	49689 city borders	52469 KB 19.2% of 272859 KB	34726 KB 20.5% of 168993 KB

Parameters

The makevalid variant is the original source file converted to GeoPackage format using ogr2ogr with the -makevalid flag. This ensures a consistent intermediary format with valid geometry that is then further simplified and compressed.

Source datasets are minified using two methods, simplification and Tiny Well-known Binary (TWKB) compression.

Simplification is performed using the Ramer-Douglas-Peucker Simplify method on the polygons. If the simplification fails (creates an invalid polygon), less and less simplification is used until the polygon remains valid. If the polygon has less than "Min. Points", it is not simplified.

Precision is the maximum number of decimal places used to store the coordinates using TWKB. From empirical testing, less than 3 decimal places does not save a lot of space and more than 3 decimal places does not gain a lot in precision for these datasets.

The wkb variants are only simplified and not compressed, keeping the original Well-known Binary (WKB) format. This increases compatibility with other tools at the cost of larger file size.

Name	Simplify	Min. Points	Precision
makevalid	none	n/a	full
s3_twkb_p3	1	20	3
s4_twkb_p3	0.1	20	3
s5_twkb_p3	0.01	20	3
s6_twkb_p3	0.001	20	3
s7_twkb_p3	0.0001	20	3
s8_twkb_p3	0.00001	20	3
s3_wkb	1	20	full
s4_wkb	0.1	20	full
s5_wkb	0.01	20	full
s6_wkb	0.001	20	full
s7_wkb	0.0001	20	full
s8_wkb	0.00001	20	full

Variants

These are the variants of the datasets available. Each variant is a combination of the parameters described above.

ne_110m_admin_0_countries

Country borders, 1:110m scale sourced from Natural Earth (Public Domain).

See Parameters for what each variant means and Datasets for details on the dataset itself.

Variant	Size	world	europe	africa	usa	japan
📥 makevalid	569 KB
📥 s3_twkb_p3	352 KB
📥 s4_twkb_p3	393 KB
📥 s5_twkb_p3	393 KB
📥 s6_twkb_p3	393 KB
📥 s7_twkb_p3	393 KB
📥 s8_twkb_p3	393 KB
📥 s3_wkb	409 KB
📥 s4_wkb	520 KB
📥 s5_wkb	557 KB
📥 s6_wkb	557 KB
📥 s7_wkb	557 KB
📥 s8_wkb	557 KB

ne_10m_admin_0_countries

Country borders, 1:10m scale sourced from Natural Earth (Public Domain).

See Parameters for what each variant means and Datasets for details on the dataset itself.

Variant	Size	world	europe	africa	usa	japan
📥 makevalid	9457 KB
📥 s3_twkb_p3	921 KB
📥 s4_twkb_p3	1249 KB
📥 s5_twkb_p3	1441 KB
📥 s6_twkb_p3	1810 KB
📥 s7_twkb_p3	2146 KB
📥 s8_twkb_p3	2158 KB
📥 s3_wkb	3002 KB
📥 s4_wkb	4997 KB
📥 s5_wkb	5984 KB
📥 s6_wkb	8462 KB
📥 s7_wkb	9179 KB
📥 s8_wkb	9297 KB

ne_10m_urban_areas_landscan

Big cities only, 1:10m scale sourced from Natural Earth (Public Domain).

See Parameters for what each variant means and Datasets for details on the dataset itself.

Variant	Size	world	berlin	nyc	tokyo	ljubljana
📥 makevalid	48156 KB
📥 s3_twkb_p3	5599 KB
📥 s4_twkb_p3	3788 KB
📥 s5_twkb_p3	4333 KB
📥 s6_twkb_p3	6119 KB
📥 s7_twkb_p3	6131 KB
📥 s8_twkb_p3	6127 KB
📥 s3_wkb	30052 KB
📥 s4_wkb	18726 KB
📥 s5_wkb	21704 KB
📥 s6_wkb	34611 KB
📥 s7_wkb	34639 KB
📥 s8_wkb	34643 KB

geoBoundariesCGAZ_ADM0

Country-level administrative boundaries sourced from geoBoundaries (Attribution required).

See Parameters for what each variant means and Datasets for details on the dataset itself.

Variant	Size	world	europe	africa	usa	japan
📥 makevalid	157384 KB
📥 s3_twkb_p3	11522 KB
📥 s4_twkb_p3	14368 KB
📥 s5_twkb_p3	17473 KB
📥 s6_twkb_p3	18358 KB
📥 s7_twkb_p3	20705 KB
📥 s8_twkb_p3	28446 KB
📥 s3_wkb	29437 KB
📥 s4_wkb	49922 KB
📥 s5_wkb	67690 KB
📥 s6_wkb	72953 KB
📥 s7_wkb	89964 KB
📥 s8_wkb	135143 KB

geoBoundariesCGAZ_ADM2

City-level administrative boundaries sourced from geoBoundaries (Attribution required).

See Parameters for what each variant means and Datasets for details on the dataset itself.

Variant	Size	world	europe	africa	usa	japan
📥 makevalid	266186 KB
📥 s3_twkb_p3	42856 KB
📥 s4_twkb_p3	15691 KB
📥 s5_twkb_p3	16420 KB
📥 s6_twkb_p3	25751 KB
📥 s7_twkb_p3	37814 KB
📥 s8_twkb_p3	44625 KB
📥 s3_wkb	232214 KB
📥 s4_wkb	54140 KB
📥 s5_wkb	52469 KB
📥 s6_wkb	128806 KB
📥 s7_wkb	208105 KB
📥 s8_wkb	246050 KB