Skip to content
/ graphium Public

Store, manage and version transport graphs from different sources, use a REST-API for data access and deploy in a distributed system.

License

Apache-2.0 license
14 stars 3 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

graphium-project/graphium

BranchesTags

Repository files navigation

Graphium

Graphium is an Open Source project to store, manage and version transport graphs. Graph data can be stored from several data providers. With the REST-API imports and management is easy. Graphium is designed for deployment in distributed systems for centralized management and publishing to several servers.

Features

Import transport graphs from different sources

There are several providers of transport graph data. Each of them uses its own data format. Using those transport graph data is easy with Graphium. Either use an existing converter or implement an custom one to convert transport graph data into a JSON file. Currently there are converters for OpenStreetMap and GIP (Graph Integration Platform - a joint, nationwide transport graph of Austria) available. The transport graph data is imported into the Graphium server using the JSON file. The Extended Information data model allows to import additional information to the basic transport graph.

Uniform and expandable data model to fulfill specific use cases

After you have imported the transport graphs, you can access and process the data through its uniform data model. You can also enhance transport graphs and handle additional information.

Manage time restricted versions

Transport networks and their abstract mapping into digital transport graphs evolve over time. Graphium can import and manage several time restricted versions of one graph. This feature makes it possible to process data for real-time tasks as well as historic data processing.

Define Views

One major design decision was to keep one (or more) graph version(s) including all kind of data needed and set up various views for data only needed for special use cases. This approach is similar to database views. Hence it is possible to import the whole data of a graph and extract only way segment used by car or by bike using views.

Publish transport graphs to other servers

Graphium is designed for use in distributed systems. If data processing based on graph data has to be parallalized using a number of machines, graph data has to be distributed and persisted on them. With Graphium it is possible to define one central Graphium server for administration of used graph versions. On worker machines clients - so called satellite Graphium servers - can deployed and subscribe on the central Graphium server for one or more specific views on graph versions. The publishing of graph versions will be done on central Graphium server's side and guarantees a consistent state over all subscribers of one view.

Concepts

Data Management & Expandability

For data management transport graph data has to be persisted. By default, Graphium uses PostgreSQL, but because of the interchangable database tier other DBMS could be used, too.

The base network model represents a directed graph consisting of waysegments and their connections. In PostgreSQL the entity waysegments stores data of graph segments (the linestring between two crossings) in a directed way, the segment entries itself are not directed. The entity waysegment_connections stores directed data for each connection between two joined segments. Both entities store an array of access types (in case of waysegments for each direction) representing the allowed usage for different vehicle types.

The base model is easy to expand if storage of additional data is needed. The entity waysegments is able to store tags as key-value-pairs. Alternatively, the entity xinfo (short for extended information) can be used. This entity defines the basic structure to reference waysegments and can be inherited to model your specific needs.

PostgreSQL DB Scheme:

Views

The concept of views in a DBMS means to filter only relevant data needed for further processing. Graphium uses the same concept to allow different views on transport graphs. For example a fine granulated transport graph is stored in database but for one specific analysis only cycle ways are necessary. Then you have to define a custom view returning only waysegments having the access type "BIKE".

To access those views over the API the graph name can be replaced with each view name. So graph name and view name will be treated similarly at the API.

Find out how to define custom views.

States

A graph version is time restricted version of a single graph. Each graph version could assume a type of state. Those types of state are shown here:

state description
INITIAL graph version is imported into the database; graph versions having this state are meant to be a pre-version, non productive-ready
PUBLISH graph version is being published to subscribed satellite servers
SYNCHRONIZED graph version is published to subscribed satellite servers (publish completed)
PUBLISH_FAILED publishing of graph version failed on one or more subscribed satellite servers
ACTIVATING graph version is being activated on all subscribed satellite servers and the central server itself
ACTIVE graph version is activated on the central server and all subscribed satellite servers; graph version can be used for productive use cases; activating a graph version could trigger post processes and restricts the previous active version of the same graph with the current timestamp;
DELETED graph version is deleted

Publish / Subscribe

Graphium is designed for use as a server in standalone mode or for deployment in distributed systems. The main use case in distributed systems is to deploy one Graphium central server storing all transport graph data needed within the whole system. One or more worker servers, so called Graphium satellite servers, could be deployed to process data based on specific transport graph data. This architecture supports horizontal scaling of Graphium satellite servers. All Graphium satellite servers will be automatically informed by the Graphium central server as soon as a new graph version is available to always process consistently on the newest transport graph version.

Detailed information about the publish / subscribe process can be found here.

API

API Reference Documentation

Quickstart

  1. Install PostgreSQL: https://www.postgresql.org/download/

  2. Install PostGIS: https://postgis.net/install/

  3. Create Database (using psql)

    CREATE ROLE graphium LOGIN ENCRYPTED PASSWORD 'md5e85d3e0c9e3a933a0c9103b21ed017df'
   VALID UNTIL 'infinity';

CREATE DATABASE graphium WITH OWNER = graphium;

\i /path/to/graphium-postgis/db/composite/create_graphium_database.sql

  4. Install Apache Tomcat: http://tomcat.apache.org/

  5. Building

    mvn clean install

  6. Deploy tutorial's Graphium central server (graphium.war) on Apache Tomcat and start

  7. Download OSM File, convert it into Graphium's JSON format and import it into Graphium central server *) (more details in graph data conversion):

    java -jar osm2graphium.one-jar.jar -i http://download.geofabrik.de/europe/andorra-latest.osm.pbf -o /path/to/output -n osm_andorra -fd true -v 200603 -q 20000 -t 5 --highwayTypes "motorway, motorway_link, primary, primary_link" -u "http://localhost:8080/graphium/api/segments/graphs/osm_andorra/versions/200603?overrideIfExists=true"

  8. Activate imported graph version

    curl -X PUT "http://localhost:8080/graphium/api/metadata/graphs/osm_andorra/versions/200603/state/ACTIVE"

  9. Check server state

    curl -X GET "http://localhost:8080/graphium/api/status"

Docker

  1. Start Docker setup

    docker-compose up -d

Application and database logs can be obtained via docker-compose logs.

If any of the following steps crashes because of a Java heap exception you have configure memory definition of Docker.

  1. Download and convert OSM File into Graphium's JSON format, import into Graphium central server *):

    docker exec -it graphium-server java -jar /osm2graphium.one-jar.jar -i http://download.geofabrik.de/europe/andorra-latest.osm.pbf -o / -n osm_andorra -fd true -v 200603 -q 20000 -t 5 --highwayTypes "motorway, motorway_link, primary, primary_link" -u "http://localhost:8080/graphium/api/segments/graphs/osm_andorra/versions/200603?overrideIfExists=true"

  2. Activate imported graph version

    docker exec -it graphium-server curl -X PUT "http://localhost:8080/graphium/api/metadata/graphs/osm_andorra/versions/200603/state/ACTIVE"

  3. Check server state

    docker exec -it graphium-server curl -X GET "http://localhost:8080/graphium/api/status"

*) Notice parameter highwayTypes: only high level streets will be imported; adapt if needed (see OSM highway types).

Tutorials

How to convert graph data to Graphium format

Look at graph data conversion.

How to define custom views

Look at custom views.

How to extend Graphium

Look at the tutorial package.

Dependencies

About

Store, manage and version transport graphs from different sources, use a REST-API for data access and deploy in a distributed system.

Topics

spring server graphs osm traffic transport gip transport-graphs graph-versions transport-graph-data

Resources

Readme

License

Apache-2.0 license
Activity
Custom properties

Stars

14 stars

Watchers

6 watching

Forks

3 forks
Report repository

Releases

6 tags

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages