Everything about OpenStreetMap.

Table of Contents

Data Structure

  1. Entity ID;
  2. Entity types:
    1. Node: coordinates representing a point.
    2. Way: a list of constituent node IDs representing a line connecting multiple nodes, such as street segment and curved path.
      • Area: a way whose ends are connected.
    3. Relation: a list of reference entity IDs that recursively groups elements.
  3. Tag: key-value pairs describing attributes of an entity.

Keys can have suffixes, e.g lanes:bus:forward=1; multiple values can be delimited by semicolon.

Metadata: author, timestamp, etc.

Tags

OSM Cheat Sheet: tags and keyboard shortcuts.

Names:

  • name, the primary tag used for common names;
  • ref, reference numbers or codes.
  • alt_name, less common names.

highway is the main tag (confusingly) used to classify any kind of road, street or path:

  • Restricted access road: motorway (default to one-directional);
    • ramps or motorway junctions: motorway_link (speed depends on curvature).
  • Standard road network: trunk, primary, secondary, tertiary, unclassified;
    • link roads: trunk_link, primary_link, secondary_link, tertiary_link.
  • Smaller road: residential (in residential area), service (in private property or parking lots);
  • Special type: living_street (slow traffic has absolute right-of-way), pedestrian (normally forbidden for motor vehicles), road (classification unknown);
  • Tagged on node: traffic_signals, mini_roundabout, stop (stop signs), bus_stop;
  • May be tagged on areas, i.e. area=yes or type=multipolygon.

Other tags that can classify ways as part of a road network:

  • lanes (lanes:forward, lanes:backward) indicate number of lanes (in a given direction);
  • turn:lanes (left, through, right) and parking:lane indicate turning lanes and parking lanes;
  • bridge, tunnel for bridges and tunnels;
  • junction=roundabout, a single lane roundabout (one-directional);

Restrictions:

  • access additionally describes legal access to an entity via all or particular forms of transport.
  • maxspeed, maximum legal speed limit (default in kilometers per hour).
  • oneway (yes, no, -1)

Spatial relation:

  • tagged as is_in;
  • inside an area tagged as place (continent, country, state, region, county, city, town, village, hamlet, suburb, island);
  • inside an area tagged as boundary=administrative and admin_level=8;

barrier, a physical structure that blocks or impedes movement (toll_booth, etc).

Relation

Turn restriction: tag type=restriction

  • tag restriction
    • prohibitory: no_right_turn, no_left_turn, no_u_turn, no_straight_on;
    • mandatory: only_right_turn, only_left_turn, only_straight_on;
    • dead end: no_entry, no_exit;
  • tag except, exception by vehicle type.
  • tags for conditional restriction: day_on, day_off, hour_on, hour_off.
  • member roles: from, to, via.

File Format

Format consideration: compression ratio, processing speed.

  • .osm, an XML format.
  • .pbf, Protocol buffer Binary Format, the primary binary format that uses Google Protocol Buffers as low-level storage. (<50% the size of XML format)
  • .o5m, a flat binary format. Nobody uses it.

Protocol buffers is a language-neutral, platform-neutral extensible mechanism for serializing structured data. Delta coding and variable-byte coding are applied throughout.

Characteristics of OSM PBF:

  1. multi-level structure: compressed Blobs - PrimitiveBlocks - PrimitiveGroups - OSM entities
  2. mapping of one Protobuf type to each OSM entity type is difficult due to the “dense nodes” data type and parallel arrays in OSM data.
  3. String tables replacing repetitive strings are redundant as Gzip will do an equivalent job.
  4. Separate Protobuf specifications are used for file block structure and OSM data, which requires implementers write auxiliary code.
  5. The most effective Protobuf technique applied in PBF is varints for delta-coded fixed-precision coordinates.

For better compression of OSM data, Conveyal invented a binary format called Vanilla Extract eXchange format (.vex).

Tools

Converting map data between OSM and external formats:

  • GeoJSON: osm-and-geojson (import); osmtogeojson, OSM2GEO, ogr2ogr (export).
  • GML: gml2osm.
  • CSV: csv2osm.
  • Open Data (a JOSM plugin) supports import from Shapefile, KML, GML, CSV.

Web Services

iD is the default online map editor for openstreetmap.org.

The OSM Editing API (API v0.6) is a RESTful HTTP API accessible via http://api.openstreetmap.org/.

Overpass API is a read-only API for querying an OpenStreetMap database and extracting features over the web. Overpass Turbo is a Web frontend of Overpass API.

OSM turn restrictions:

Standalone Executables

osmosis is a command line Java application for processing OSM data, which can extract data inside a bounding box or polygon.

osmconvert can convert and process OSM files faster than osmosis but has less functionality; it has a few special functions.

osmium also has a command line tool to convert file format, apply changeset, and extract historical data.

JOSM, Java OpenStreetMap, is the offline editor of OSM data, with many plugins. Features: validator feature can check and fix invalid data. JOSM plugins for fixing missing turn restrictions: ScoutSigns, ImproveOsm.

KeepRight, data consistency checks (quality assurance) for OSM.

PostgreSQL Loader: osm2pgsql; osm2pgrouting, imports OSM topology to PostgreSQL database; other java programs.

SQLite/SpatiaLite for OSM

Libraries & Frameworks

Programming Frameworks for accessing, processing, map rendering (static, interactive), geocoding, and navigation with OpenStreetMap data. Debian GIS Blend meta-package for OpenStreetMap already include many of these tools and libraries.

Accessing:

  • osm-common (Java): accessing, processing, geocoding; supports Overpass.
  • osmapi (Python): Python wrapper for the OpenStreetMap API.
  • osmaR (R): access OpenStreetMap data from file or API, and convert to other classes.
  • Data access APIs for Ruby and PHP are also available.

Data processing / parsing:

  • C/C++: pbf2osm, osmpbf; LibOsm (SQLlite); osmium;
  • GO: Gosmparse, osmpbf;
  • Java: osmosis, osm4j, BasicOSMParser, OSMemory;
  • Python: imposm.parser, osmread;

Navigation:

  • OSRM (C++), Open Source Routing Machine, used by Mapbox Directions.
  • Valhalla (C++), Mapzen Turn-by-Turn backend.
  • GraphHopper (Java), used by OpenStreetMap.org for bike and pedestrian routing.

Geocoding: Gisgraphy (Java)

Map rendering:

  • Web map interface: Leaflet, OpenLayers
  • WebGL: Mapbox GL JS, Tangram
  • HTML5 Canvas: Kothic JS, Cartagen
  • SVG: Kartograph (Python, JavaScript)

🏷 Category=Geographic Information System