Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PBF writing #16

Open
nyurik opened this issue Feb 17, 2022 · 4 comments
Open

Support PBF writing #16

nyurik opened this issue Feb 17, 2022 · 4 comments

Comments

@nyurik
Copy link
Contributor

nyurik commented Feb 17, 2022

I would like to implement PBF write support. My initial thoughts are to have two structs:

  • BlockBuilder struct equivalent to PrimitiveBlock, contains shared state and a list of GroupBuilders
  • GroupBuilder struct contains values that will go into PrimitiveGroup

The BlockBuilder owns zero or more GroupBuilder instances, possibly of different types, and has methods to add node/way/rels to them. When adding a single feature the block builder would also track the bounding box of all features, as well as the shared tags table. Once the block is full, it self-compresses into a stream. The actual interfaces are TBD.

@b-r-u
Copy link
Owner

b-r-u commented Feb 19, 2022

I like that idea!

And builder structs seem like a good way to avoid having to allocate a node with all of its tags, only to be destroyed again so that it fits the PBF memory model. So I guess there should be a NodeBuilder too with an add_tag method that mutates the current stringtable.

First I thought that GroupBuilder can be abstracted away and a BlockBuilder would just have add_node, add_way, etc. and take care of putting it into the right group. But then the order of the elements in reading and writing could be different. So for a low-level interface I think it's good to have an explicit GroupBuilder.

@b-r-u
Copy link
Owner

b-r-u commented Jul 2, 2022

Hey nyurik,

I wanted to let you know, that I started pushing commits to this branch:

https://github.com/b-r-u/osmpbf/tree/writer

Writing blobs and blocks is implemented, but there is still a lot of functionality (and ergonomics) missing.

@nyurik
Copy link
Contributor Author

nyurik commented Jul 2, 2022

Thanks!! I have been hacking locally on it for a long time on and off, and will push my work in progress to my fork. What do you think about the delta crate? I think it will make the code simpler to read, and might help with writing too

@nyurik
Copy link
Contributor Author

nyurik commented Sep 7, 2022

@b-r-u what's the status of the rewrite? I have pushed my own work in that direction, but not sure if that's how you want to proceed. See https://github.com/nyurik/osmpbf/tree/write

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants