OpenStreetMap

Everything about OpenStreetMap.

Data Format

Format consideration: compression ratio, processing speed.

• .osm, an XML format.
• .pbf, Protocol buffer Binary Format, the primary binary format that uses Google Protocol Buffers as low-level storage. (<50% the size of XML format)
• .o5m, a flat binary format. Nobody uses it.

Characteristics of OSM PBF:

1. multi-level structure: compressed Blobs - PrimitiveBlocks - PrimitiveGroups - OSM entities
2. mapping of one Protobuf type to each OSM entity type is difficult due to the “dense nodes” data type and parallel arrays in OSM data.
3. String tables replacing repetitive strings are redundant as Gzip will do an equivalent job.
4. Separate Protobuf specifications are used for file block structure and OSM data, which requires implementers write auxiliary code.
5. The most effective Protobuf technique applied in PBF is varints for delta-coded fixed-precision coordinates.

Delta coding and variable-byte coding are applied throughout.

Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. (https://developers.google.com/protocol-buffers)

.vex, Vanilla Extract exchange format, a binary format invented by Conveyal. Format specification

Data Schema

1. entity ID;
2. entity types:
   1. node: coordinates representing a point.
   2. way: a list of constituent node IDs representing a line connecting multiple nodes. (e.g. street segments, curved path)
      • area: a way whose ends are connected.
   3. relation: a list of reference entity IDs that recursively groups elements.
3. tags: key-value pairs describing attributes of an entity. (e.g oneway=yes)

author and timestamp metadata.

Tags

OSM Cheat Sheet

highway: the main tag used for identifying any kind of road, street or path.

• restricted access road: motorway
• standard road network: trunk, primary, secondary, tertiary, unclassified
• smaller road: residential, service
• special types: road, living_street, pedestrian
• paths: footway

Tools

• Extract
  • osmosis, extract data inside a bounding box or polygon.
  • Overpass Turbo API: query OpenStreetMap data set and extract specific features.
• Editor
  • iD, the default online map editor.
  • JOSM, Java OpenStreetMap (offline, extensible) editor of OSM data, background imagery, and GPX tracks.
• Quality Assurance
  • KeepRight, data consistency checks for OSM. GitHub Repo
  • The Validator feature of JSOM can check and fix invalid data.
• PostgreSQL Loader
  • osm2pgsql
  • osm2pgrouting, imports OSM XML file topology to PostgreSQL database.
  • Converting OSM-Files into PostgreSQL-Scripts
• Routing engine
  • Graphserver, an open source multimodal trip planner developed by Brandon Martin-Anderson.
  • Routing services using OpenStreetMap data
  • RealReach™ algorithm in skobbler/Scout
• Renderer
  • Mapnik: Cartographic Rendering for OSM
  • GMT (Generic Mapping Tools): Cartographic Rendering

Programming Frameworks:

• RESTful APIs: Java osmapi; Python osmapi; R osmar.
• Data processing / parsing libraries: Python osmread, imposm.parser, osm4routing, PyrouteLib2; Java se.kodapan.osm, com.gisgraphy.servlet.
• Frameworks (suites of modules): Java osmosis, OSMNavigation, osm4j.
• Widgets (integrate OSM into your software): Marble.
• Web maps: Leaflet, OpenLayers, Google Maps API.

Debian GIS Blend meta-packages:

• OpenStreetMap: OpenStreetMap parser, editor, viewer.
• Map server, Present geographic information via web map server