Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Middle: Get dependencies with single queries #1996

Merged
merged 3 commits into from
Jul 14, 2023
Merged

Middle: Get dependencies with single queries #1996

merged 3 commits into from
Jul 14, 2023

Commits on Jul 13, 2023

  1. Find node dependencies faster by doing a single lookup 

    For each changed node osm2pgsql has to find all ways and all relations
referencing that node and make sure they are updated as well. This was
done by doing one SQL query per node which is quite inefficient. Instead
the new code collects all node ids and does only one query for the
parent ways and one for the parent relations after all nodes have been
read.

This implementation uses temporary tables. The changed nodes are written
to a temp table and from that temp tables are filled with the parent
nodes and ways. Strictly speaking the latter tables are not necessary,
we could read the data directly from the query. But maybe those tables
could be useful in the future for some additional processing in the
database. (They'd need to be converted to non-temp tables then, though.)
    @joto
    joto committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    ea7858a View commit details
    Browse the repository at this point in the history

  2. Find way dependencies faster by doing a single lookup 

    For each changed way osm2pgsql has to find all relations referencing
that way and make sure they are updated as well. This was done by doing
one SQL query per way which is quite inefficient. Instead the new code
collects all way ids and does only one query for the parent relations
after all ways have been read.

This commit also makes sure that ways which have been in the input file
are removed from the pending way tracker so that they are not processed
again later.
    @joto
    joto committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    fa8336d View commit details
    Browse the repository at this point in the history

  3. Make sure not to process relations twice in append mode 

    In append mode relations could be processed twice if they are in the
input and also the parent of a node or way in the input. With this
change we are removing all ids of changed relations from the list of
pending relations before processing them.

Fixes osm2pgsql-dev#423.
    @joto
    joto committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    36b4fb8 View commit details
    Browse the repository at this point in the history