feat: add Features + datastore scoping #188
Conversation
guseggert
added
the
kind/enhancement
guseggert
force-pushed
the
feat/scoped
branch
5 times, most recently
from
8805340
to
ed2042e
Compare
|
@aschmahmann pointed out yesterday that we could also use this, along with a new method on mount datastores to return a scoped datastore for a given key/key prefix/query, which effectively solves the problem with mixing multiple datastore types together with a mount datastore. |
guseggert
force-pushed
the
feat/scoped
branch
8 times, most recently
from
f1aa84f
to
8d7bb60
Compare
| return featureSet | ||
| } | ||
|
|
||
| // Wrap returns a datastore based on the source, whose concrete type is scoped down to only the features supported by the target. |
There was a problem hiding this comment.
This sentence is wrong, as far as I can tell, this actually return the intersection of features.
I think the intersection is less useful, and I would like it to return an error instead of trying to silently continue.
There was a problem hiding this comment.
The elements of set A that are also in set B is the definition of set intersection, the two statements are equivalent.
The reason this doesn't return an error if B is not a subset of A is to allow for both datastores to evolve independently, without breaking consumers of this.
There was a problem hiding this comment.
For me this sentence assume that set A always is >= than set B. It's not really important.
|
Minor changes should be needed. I don't like routines that try to make up with degraded state silently. If something is not happening as it should, fail loudly, so I (the programmer) can learn about it and fix it. |
The motivation for this is to enable "dispatching" datastores that dynamically implement the type of the datastore they are dispatching to, so that type assertions behave equivalently on the dispatcher as on the dispatchee. We also want this to be backwards-compatible with existing code using type assertions. At a high level, this works by generating a concrete implementation of every possible combination of "features", and then picking the right implementation at runtime. This is necessary due to language constraints in Go--it is currently impossible to create a concrete type dynamically with reflection that implements an interface. "Features" are introduced here, which are supplemental, optional interfaces that datastores may implement. These are backwards-compatible with existing "features", which are: * Batching * CheckedDatastore * GCDatastore * PersistentDatastore * ScrubbedDatastore * TTLDatastore * TxnDatastore New features can also be added in a backwards-compatible way. E.g. if datastore A is scoped down to datastore B, a new feature F is added, and then implemented on B, then A will continue to implement the same set of features since it hasn't implemented F yet (and vice versa if F is implemented on A but not B). Examples of things this enables: * Allow us to deprecate ErrBatchUnsupported * Allow existing dispatching datastores to support all features (keytransform, retrystore, MutexDatastore, autobatch, etc.) * Features supported by a Mount datastore could be scoped down to the intersection of all children * Communication with data about what functionality a datastore supports (e.g. for cross-language/RPC support) Some related issues: * #160 * #88
| return featureSet | ||
| } | ||
|
|
||
| // Wrap returns a datastore based on the source, whose concrete type is scoped down to only the features supported by the target. |
There was a problem hiding this comment.
For me this sentence assume that set A always is >= than set B. It's not really important.
This is ripped out of https://github.com/guseggert/go-ds-grpc, as I realized it solves a more general problem.
Commit
The motivation for this is to enable "dispatching" datastores that
dynamically implement the type of the datastore they are dispatching
to, so that type assertions behave equivalently on the dispatcher
as on the dispatchee. We also want this to be backwards-compatible
with existing code using type assertions.
At a high level, this works by generating a concrete implementation of
every possible combination of "features", and then picking the right
implementation at runtime. This is necessary due to language
constraints in Go--it is currently impossible to create a concrete
type dynamically with reflection that implements an interface.
"Features" are introduced here, which are supplemental, optional
interfaces that datastores may implement. These are
backwards-compatible with existing "features", which are:
New features can also be added in a backwards-compatible way. E.g. if
datastore A is scoped down to datastore B, a new feature F is added, and
then implemented on B, then A will continue to implement the same set
of features since it hasn't implemented F yet (and vice versa if F is
implemented on A but not B).
Examples of things this enables:
features (keytransform, retrystore, MutexDatastore, autobatch, etc.)
intersection of all children
supports (e.g. for cross-language/RPC support)
Some related issues: