-
-
Notifications
You must be signed in to change notification settings - Fork 474
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
WIP: Add code to generalize data using different strategies
This commit adds the code to generalize various types of data using different strategies. The following strategies work on a tile-by-tile basis and operate on polygons: The "vector-union" strategy buffers and unionizes polygons using vector operations. The "raster-union" strategy does a similar thing but does it in raster space which is much faster. First the polygons are rendered into a raster, an open/close operation is called (which basically does the same thing as the buffering in vector space) and finally the resulting raster is vectorized again. The "builtup" strategy is intended to derive a layer of builtup areas from landuse=residential/industrial etc. as well as building cover and dense road networks. This still needs some work... Also a new "discrete-isolation" strategy which rates places based on some importance metric. (This is not tile-based.) The new "rivers" strategy finds important rivers, this is still very much work in progress. For the raster support this adds two new library dependency: CImg and potrace. The functionality is accessed through a new command line program called osm2pgsql-gen. Call it with -h to get some usage information. This program is for testing only, eventually the functionality should be accessible from osm2pgsql itself (using the Lua config file for configuration). Some additional information is in README-gen.md . See also https://osm2pgsql.org/generalization/ . wip rivers
- Loading branch information
Showing
33 changed files
with
2,811 additions
and
22 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
|
|
||
| # Generalization | ||
|
|
||
| There is an experimental new program `osm2pgsql-gen`. It is only built if the | ||
| CImg and potrace libraries are available. | ||
|
|
||
| This is part of a project to add generalization support to osm2pgsql. See | ||
| https://osm2pgsql.org/generalization/ for more information. | ||
|
|
||
| Generalization is currently only supported for Web Mercator (EPSG 3857). This | ||
| is by far the most common use case, we can look at extending this later if | ||
| needed. | ||
|
|
||
| ## Usage | ||
|
|
||
| Run osm2pgsql as usual. This example assumes you have a table called | ||
| `landcover` with a polygon geometry column called `geom` and a text column | ||
| called `type` with the type of the landcover (forest, grass, etc.). | ||
|
|
||
| Create a table `landcover_z10` with (at least) columns `geom` and `type` as | ||
| well as integer columns `x`, and `y`. Then call this: | ||
|
|
||
| ``` | ||
| PGDATABASE=mydb ./osm2pgsql-gen -t landcover -T landcover_z10 -g geom -G type -z 10 -m 0.125 -s raster-union | ||
| ``` | ||
|
|
||
| The `landcover_z10` table will be filled with generalized polygons. | ||
|
|
||
| Database connection parameters have to be specified using [environment | ||
| variables](https://www.postgresql.org/docs/current/libpq-envars.html). | ||
|
|
||
| Call `osm2pgsql-gen -h` for more command line options. | ||
|
|
||
| ## Extent | ||
|
|
||
| You can process a single tile by setting `-x`, and `-y`. If `-X` and `-Y` are | ||
| also set, all tiles between the arguments of `-x` and `-X`, and `-y` and `-Y` | ||
| are processed. Without any of these, the program gets the extent from the data | ||
| in the source table. | ||
|
|
||
| You can also use the option `-e, --expire-list=TABLE` to read the list of tiles | ||
| from a database table. | ||
|
|
||
| In any case `-z` or `--zoom` sets the zoom level. | ||
|
|
||
| ## Parameters | ||
|
|
||
| For a full list of parameters see `osm2pgsql-gen -h`. | ||
|
|
||
| * `-s, --strategy=STRATEGY`: Set the strategy used for generalization. See | ||
| below for available strategies. | ||
| * `-m, --margin=MARGIN`: This sets the margin around the tile as a fraction of | ||
| the tile size. So a value of 0.1 sets a 10% boundary on each side of the tile, | ||
| so as a result the tiles will overlap by 20% of their size. When using the | ||
| `raster-union` strategy the margin will be rounded up to the nearest multiple | ||
| of 64 pixels. | ||
| * `-w, --width=WIDTH`: Size of the imaged rendered when using the `raster-union` | ||
| strategy, not used in the `vector-union` strategy. | ||
| * `-b, --buffer=BUFFER`: The amount by which the polygons will be buffered. For | ||
| the `raster-union` strategy this is in pixels, for the `vector-union` strategy | ||
| this is in Mercator map units. | ||
| * `-g, --group-by-column=COL`: Set this to the column describing the type of | ||
| polygon if any, the program will group the data by this column before | ||
| generalization. If this is not specified, no grouping is performed and | ||
| the destination table also doesn't need a column for this. | ||
| * `--img-path=PATH`: Used to dump PNGs of the "before" and "after" images | ||
| generated with the `raster-union` strategy. Use something like this: | ||
| `--img-path=some/dir/path/img`. Resulting images will be in the | ||
| directory `some/dir/path` and are named `img-X-Y-TYPE-[io].png` for | ||
| input (`i`) or output (`o`) images. The `TYPE` is the value from the | ||
| column specified with the `-G` option. | ||
| * `--img-table=TABLE`: Used to dump "before" and "after" raster images to the | ||
| database. The table will be created if it doesn't exist already. | ||
| * `-e, --expire-list=TABLE`: Get list of tiles to expire from the specified | ||
| table. If this is set the options `-x`, `-X`, `-y`, and `-Y` are ignored. The | ||
| content of the table is not removed after processing! | ||
| * `-p, --param=KEY=VALUE`: Set any parameter. This allows for easier | ||
| experimentation with new parameters. | ||
|
|
||
| ## Strategies | ||
|
|
||
| Some strategies work on a tile-by-tile basis (`vector-union`, `raster-union`, | ||
| `builtup`), some work for all data at once (`discrete-isolation`). | ||
|
|
||
| * `vector-union`: Buffer and union polygons together in vector space to | ||
| form generalized polygons for landcover and similar use. | ||
| * `raster-union`: Buffer and union polygons together in raster space to | ||
| form generalized polygons for landcover and similar use. | ||
| See https://blog.jochentopf.com/2022-11-21-generalizing-polygons.html for | ||
| details. | ||
| * `builtup`: Aggregate data from several layers (such as landcover, buildings, | ||
| and roads) to derive built-up areas. | ||
| * `discrete-isolation`: Classify point geometries by importance to render | ||
| only the most important places but still don't leave areas too empty. | ||
| See https://blog.jochentopf.com/2022-12-19-selecting-settlements-to-display.html | ||
| for some background. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| /** | ||
| * SPDX-License-Identifier: GPL-2.0-or-later | ||
| * | ||
| * This file is part of osm2pgsql (https://osm2pgsql.org/). | ||
| * | ||
| * Copyright (C) 2006-2023 by the osm2pgsql developer community. | ||
| * For a full list of authors see the git log. | ||
| */ | ||
|
|
||
| #include "canvas.hpp" | ||
| #include "raster.hpp" | ||
|
|
||
| cimg_library::CImg<int> canvas_t::create_pointlist(geom::point_list_t const &pl, | ||
| tile_t const &tile) const | ||
| { | ||
| cimg_library::CImg<int> points{static_cast<unsigned int>(pl.size()), 2}; | ||
|
|
||
| int n = 0; | ||
| for (auto const point : pl) { | ||
| auto const tp = tile.to_tile_coords(point, m_extent); | ||
| points(n, 0) = static_cast<double>(m_buffer) + tp.x(); | ||
| points(n, 1) = static_cast<double>(m_buffer + m_extent) - tp.y(); | ||
| ++n; | ||
| } | ||
|
|
||
| return points; | ||
| } | ||
|
|
||
| std::size_t canvas_t::draw_polygon(geom::polygon_t const &polygon, | ||
| tile_t const &tile) | ||
| { | ||
| if (polygon.inners().empty()) { | ||
| m_rast.draw_polygon(create_pointlist(polygon.outer(), tile), &White); | ||
| return polygon.outer().size(); | ||
| } | ||
|
|
||
| std::size_t num_points = polygon.outer().size(); | ||
| m_temp.draw_polygon(create_pointlist(polygon.outer(), tile), &White); | ||
| for (auto const &inner : polygon.inners()) { | ||
| num_points += inner.size(); | ||
| m_temp.draw_polygon(create_pointlist(inner, tile), &Black); | ||
| } | ||
| m_rast |= m_temp; | ||
|
|
||
| return num_points; | ||
| } | ||
|
|
||
| std::size_t canvas_t::draw_linestring(geom::linestring_t const &linestring, | ||
| tile_t const &tile) | ||
| { | ||
| m_rast.draw_line(create_pointlist(linestring, tile), &White); | ||
| return linestring.size(); | ||
| } | ||
|
|
||
| std::size_t canvas_t::draw(geom::geometry_t const &geometry, tile_t const &tile) | ||
| { | ||
| if (geometry.is_linestring()) { | ||
| auto const &linestring = geometry.get<geom::linestring_t>(); | ||
| return draw_linestring(linestring, tile); | ||
| } | ||
|
|
||
| if (geometry.is_polygon()) { | ||
| auto const &polygon = geometry.get<geom::polygon_t>(); | ||
| return draw_polygon(polygon, tile); | ||
| } | ||
|
|
||
| if (geometry.is_multipolygon()) { | ||
| auto const &mp = geometry.get<geom::multipolygon_t>(); | ||
| std::size_t num_points = 0; | ||
| for (auto const &p : mp) { | ||
| num_points += draw_polygon(p, tile); | ||
| } | ||
| return num_points; | ||
| } | ||
|
|
||
| // XXX other geometry types? | ||
|
|
||
| return 0; | ||
| } | ||
|
|
||
| void canvas_t::save(std::string const &filename) const | ||
| { | ||
| m_rast.save(filename.c_str()); | ||
| } | ||
|
|
||
| std::string canvas_t::to_wkb(tile_t const &tile, double margin) const | ||
| { | ||
| std::string wkb; | ||
| wkb.reserve(61 + 2 + m_rast.size()); | ||
|
|
||
| // header | ||
| wkb_raster_header header{}; | ||
| header.nBands = 1; | ||
| header.scaleX = tile.extent() / m_extent; | ||
| header.scaleY = -header.scaleX; | ||
| header.ipX = tile.xmin(margin); | ||
| header.ipY = tile.ymax(margin); | ||
| header.width = m_extent + 2 * m_buffer; | ||
| header.height = header.width; | ||
| add_raster_header(&wkb, header); | ||
|
|
||
| // band | ||
| wkb_raster_band band{}; | ||
| band.bits = 4; | ||
| add_raster_band(&wkb, band); | ||
|
|
||
| // rasterdata | ||
| wkb.append(reinterpret_cast<char const *>(m_rast.data()), m_rast.size()); | ||
|
|
||
| assert(wkb.size() == 61 + 2 + m_rast.size()); | ||
|
|
||
| return wkb; | ||
| } | ||
|
|
||
| void canvas_t::merge(canvas_t const &other) { m_rast |= other.m_rast; } | ||
|
|
||
| std::string to_hex(std::string const &in) | ||
| { | ||
| std::string result; | ||
| char const *const lookup_hex = "0123456789ABCDEF"; | ||
|
|
||
| for (const auto c : in) { | ||
| unsigned int const num = static_cast<unsigned char>(c); | ||
| result += lookup_hex[(num >> 4U) & 0xfU]; | ||
| result += lookup_hex[num & 0xfU]; | ||
| } | ||
|
|
||
| return result; | ||
| } |
Oops, something went wrong.