scylla: lazily close excess connections once equilibrium is reached. #14
Conversation
| @@ -80,12 +80,14 @@ func isScyllaConn(conn *Conn) bool { | |||
|
|
|||
| // scyllaConnPicker is a specialised ConnPicker that selects connections based | |||
| // on token trying to get connection to a shard containing the given token. | |||
| // A list of excess connections is maintained to allow for lazy garbage collection | |||
| type scyllaConnPicker struct { | |||
There was a problem hiding this comment.
Yes it's a bit sparse, I will amend the doc and commit message.
There was a problem hiding this comment.
Updated both in the code and the commit message.
dahankzter
force-pushed
the
lazy_purge_excess_conns
branch
from
7a755ca
to
80b732b
Compare
| @@ -189,12 +194,28 @@ func (p *scyllaConnPicker) Put(conn *Conn) { | |||
| copy(p.conns, conns) | |||
| } | |||
| if c := p.conns[s.shard]; c != nil { | |||
| conn.Close() | |||
| p.excessConns = append(p.excessConns, conn) | |||
There was a problem hiding this comment.
Could we put a limit to how big p.excessConns could be, so that it does not grow unbounded if something goes wrong? Not sure how many connections we need to hit all shards, so I guess if we reach 10*p.nrShards something is off and maybe we should close the connections anyway.
There was a problem hiding this comment.
I am not sure it helps. It can even make it worse. The idea is that the chance to get a new shard increases significantly with the number of connections so equilibrium should be reached faster.
There was a problem hiding this comment.
You could have multiple clients connecting to the same host though, which would cause it to race, so theoretically you could keep opening connections forever (even if the chance drops with time), so I think it's still good to have some kind of limit.
dahankzter
force-pushed
the
lazy_purge_excess_conns
branch
4 times, most recently
from
ddb8628
to
c917adb
Compare
A list of excess connections is maintained to allow for lazy removal of. excess connections. Keeping excess connections open helps reaching equilibrium faster since the likelihood of hitting the same shard decreases with the number of connections to the shard. The excess connections list is currently capped to 10 times the number of shards. This magic number has no backing statistics but sounds reasonable.
When the pool is not yet fully materialized we give the caller a random connection to allow the session to start working.
dahankzter
force-pushed
the
lazy_purge_excess_conns
branch
from
c917adb
to
cdf6dc8
Compare
Keep a list of excess connections and only close these when we have connections to all the shards.