Everything about OpenStreetMap.

Data Structure

  1. Entity ID;
  2. Entity types:
    1. Node: coordinates representing a point.
    2. Way: a list of constituent node IDs representing a line connecting multiple nodes. (e.g. street segments, curved path)
    3. area: a way whose ends are connected.
    4. Relation: a list of reference entity IDs that recursively groups elements.
  3. Tags: key-value pairs describing attributes of an entity, e.g oneway=yes.

Metadata: author, timestamp, etc.

Tags

OSM Cheat Sheet: tags and keyboard shortcuts.

highway: the main tag used for identifying any kind of road, street or path.

  • restricted access road: motorway
  • standard road network: trunk, primary, secondary, tertiary, unclassified
  • smaller road: residential, service
  • special types: road, living_street, pedestrian
  • paths: footway

File Format

Format consideration: compression ratio, processing speed.

  • .osm, an XML format.
  • .pbf, Protocol buffer Binary Format, the primary binary format that uses Google Protocol Buffers as low-level storage. (<50% the size of XML format)
  • .o5m, a flat binary format. Nobody uses it.

Characteristics of OSM PBF:

  1. multi-level structure: compressed Blobs - PrimitiveBlocks - PrimitiveGroups - OSM entities
  2. mapping of one Protobuf type to each OSM entity type is difficult due to the “dense nodes” data type and parallel arrays in OSM data.
  3. String tables replacing repetitive strings are redundant as Gzip will do an equivalent job.
  4. Separate Protobuf specifications are used for file block structure and OSM data, which requires implementers write auxiliary code.
  5. The most effective Protobuf technique applied in PBF is varints for delta-coded fixed-precision coordinates.

Delta coding and variable-byte coding are applied throughout.

Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. (https://developers.google.com/protocol-buffers)

.vex, Vanilla Extract exchange format, a binary format invented by Conveyal. Format specification

Tools

Converting map data between OSM and external formats:

  • GeoJSON: osm-and-geojson (import); osmtogeojson, OSM2GEO, ogr2ogr (export).
  • GML: gml2osm.
  • CSV: csv2osm.
  • Open Data (a JOSM plugin) supports import from Shapefile, KML, GML, CSV.

Web Services

iD is the default online map editor for openstreetmap.org.

The OSM Editing API (API v0.6) is a RESTful HTTP API accessible via http://api.openstreetmap.org/.

Overpass API is a read-only API for querying an OpenStreetMap database and extracting features over the web. Overpass Turbo is a Web frontend of Overpass API.

OSM turn restrictions

Openstreetmap Navigation Data Map

Standalone Executables

osmosis is a command line Java application for processing OSM data, which can extract data inside a bounding box or polygon.

osmconvert can convert and process OSM files faster than osmosis but has less functionality; it has a few special functions.

osmium also has a command line tool to convert file format, apply changeset, and extract historical data.

JOSM, Java OpenStreetMap, is the offline editor of OSM data, with many plugins. Features: validator feature can check and fix invalid data.

KeepRight, data consistency checks (quality assurance) for OSM.

PostgreSQL Loader: osm2pgsql; osm2pgrouting, imports OSM topology to PostgreSQL database; other java programs.

SQLite/SpatiaLite for OSM

Libraries & Frameworks

Programming Frameworks for accessing, processing, map rendering (static, interactive), geocoding, and navigation with OpenStreetMap data. Debian GIS Blend meta-package for OpenStreetMap already include many of these tools and libraries.

Accessing:

  • osm-common (Java): accessing, processing, geocoding; supports Overpass.
  • osmapi (Python): Python wrapper for the OpenStreetMap API.
  • osmaR (R): access OpenStreetMap data from file or API, and convert to other classes.
  • Data access APIs for Ruby and PHP are also available.

Data processing / parsing:

  • C/C++: pbf2osm, osmpbf; LibOsm (SQLlite); osmium;
  • GO: Gosmparse, osmpbf;
  • Java: osmosis, osm4j, BasicOSMParser, OSMemory;
  • Python: imposm.parser, osmread;

Navigation:

  • OSRM (C++), Open Source Routing Machine, used by Mapbox Directions.
  • Valhalla (C++), Mapzen Turn-by-Turn backend.
  • GraphHopper (Java), used by OpenStreetMap.org for bike and pedestrian routing.

Geocoding: Gisgraphy (Java)

Map rendering:

  • Web map interface: Leaflet, OpenLayers
  • WebGL: Mapbox GL JS, Tangram
  • HTML5 Canvas: Kothic JS, Cartagen
  • SVG: Kartograph (Python, JavaScript)

🏷 Category=Geographic Information System