Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delta-state CRDTs #2749

Merged
merged 12 commits into from
Jun 25, 2017
Merged

Delta-state CRDTs #2749

merged 12 commits into from
Jun 25, 2017

Conversation

Horusiath
Copy link
Contributor

This PR introduces CRDTs upgrade, allowing us to optimize replication process by using deltas instead of sending full state changes across the network.

Code will need further upgrade (branch was originally targetting C# 6). Also currently known problem is the state of Hyperion (CRDTs are using custom Hyperion-based serializer), which seems to have some bugs on .NET Core implementation.

@sean-gilliam
Copy link
Member

very nice 👍

@Horusiath Horusiath force-pushed the v1.3-ddata branch 4 times, most recently from b738b1c to b44b5ef Compare June 16, 2017 17:56
@Horusiath
Copy link
Contributor Author

Horusiath commented Jun 16, 2017

Ok, I think this PR is ready for review. This doesn't mean, that work here is done:

  • We need to think about CRDT serialization (especially in context of persistence). Hyperion is not good choice for this, as it's binary format is not stable. The best candidates here would be protobuf or bond. Problem with Bond is that we already have a direct dependency on protobuf and adding the next serializer doesn't seem to be a good idea. Problem with protobuf is lack of support for generics, and CRDTs heavily rely on that.
  • I needed to provide a generic-less versions of interfaces in few places (i.e. IReplicatedData, IDeltaReplicatedData) to have a common way of calling them from Replicator. Tbh I think, we could rework the methods a bit and try to use dynamic there - first call will be expensive, but subsequent calls are proven to be fast. We already use similar technique in Hyperion.
  • I didn't ported all specs for serialization (those regarding payload size assertions) and didn't fixed MNTK specs for cluster, as last time those were problematic due to some strange behavior in akka cluster itself.
  • We still need performance tests, both for new data structures and replication protocol itself.

@Aaronontheweb Aaronontheweb merged commit 47f0761 into akkadotnet:v1.3 Jun 25, 2017
@Aaronontheweb Aaronontheweb modified the milestones: 1.2.2, 1.3.0 Jun 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants