Integration tests fail using Scylla 5.2.x #126

vponomaryov · 2023-06-08T09:53:59Z

Please answer these questions before submitting your issue. Thanks!

What version of Scylla or Cassandra are you using?

5.2.1 and 5.2.0

What version of Gocql are you using?

v1.7.3

What version of Go are you using?

1.20.4

What did you do?

I explicitly defined the Scylla version:

SCYLLA_IMAGE=scylladb/scylla:5.2.1

Then ran tests:

./integration.sh integration scylla

And got following result:

==> Running integration scylla tests with args: -gocql.timeout=60s -proto=4 -rf=1 -clusterSize=1 -autowait=2000ms -compressor=snappy -gocql.cversion=3.11.4 -cluster=192.168.100.11
2023/06/08 12:42:51 logger.go:35: error: failed to connect to "[HostInfo hostname=\"192.168.100.11\" connectAddress=\"192.168.100.11\" peer=\"<nil>\" rpc_address=\"192.168.100.11\" broadcast_address=\"192.168.100.11\" preferred_ip=\"<nil>\" connect_addr=\"192.168.100.11\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"38da3ff8-79e5-4515-96d6-9edbe943df37\" version=\"v3.0.8\" state=UP num_tokens=1]" due to error: EOF
2023/06/08 12:42:51 logger.go:35: error: failed to connect to "[HostInfo hostname=\"192.168.100.11\" connectAddress=\"192.168.100.11\" peer=\"<nil>\" rpc_address=\"192.168.100.11\" broadcast_address=\"192.168.100.11\" preferred_ip=\"<nil>\" connect_addr=\"192.168.100.11\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"38da3ff8-79e5-4515-96d6-9edbe943df37\" version=\"v3.0.8\" state=UP num_tokens=1]" due to error: Keyspace 'invalidKeyspace' does not exist
2023/06/08 12:42:56 logger.go:35: gocql: unable to dial control conn 127.0.0.1:1: dial tcp :0->127.0.0.1:1: connect: connection refused
panic: test timed out after 5m0s
running tests:
        TestBatchObserve (4m38s)

goroutine 54910 [running]:
testing.(*M).startAlarm.func1()
        /usr/local/go/src/testing/testing.go:2241 +0x219
created by time.goFunc
        /usr/local/go/src/time/sleep.go:176 +0x48

goroutine 1 [chan receive, 4 minutes]:
testing.(*T).Run(0xc000103380, {0xbb06fc, 0x10}, 0xbea210)
        /usr/local/go/src/testing/testing.go:1630 +0x82e
testing.runTests.func1(0x0?)
        /usr/local/go/src/testing/testing.go:2036 +0x8e
testing.tRunner(0xc000103380, 0xc00016fb48)
        /usr/local/go/src/testing/testing.go:1576 +0x217
testing.runTests(0xc000140dc0?, {0xf95a40, 0x94, 0x94}, {0x1c?, 0x4a8b19?, 0xf99780?})
        /usr/local/go/src/testing/testing.go:2034 +0x87d
testing.(*M).Run(0xc000140dc0)
        /usr/local/go/src/testing/testing.go:1906 +0xb45
main.main()
        _testmain.go:345 +0x2ea

goroutine 125 [select, 1 minutes]:
github.com/gocql/gocql.(*Session).reconnectDownedHosts(0xc000472000, 0xdf8475800?)
        /home/kiparis/projects/gocql/session.go:374 +0x177
created by github.com/gocql/gocql.(*Session).init
        /home/kiparis/projects/gocql/session.go:321 +0x108c

goroutine 1360 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0x47943a?, {0xc9b758, 0xc0003614f0})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 1381 [select]:
github.com/gocql/gocql.(*writeCoalescer).writeFlusher(0xc000591c80, 0xbea210?)
        /home/kiparis/projects/gocql/conn.go:880 +0x245
created by github.com/gocql/gocql.newWriteCoalescer
        /home/kiparis/projects/gocql/conn.go:785 +0x305

goroutine 1374 [select]:
github.com/gocql/gocql.(*Conn).prepareStatement(0xc0003a8900, {0xc9b790, 0xc000126018}, {0xc0000a9040, 0x38}, {0x0?, 0x0})
        /home/kiparis/projects/gocql/conn.go:1091 +0x44c
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1349 +0x7bf
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
github.com/gocql/gocql.(*Conn).executeBatch(0xc0003a8900, {0xc9b790, 0xc000126018}, 0xc0000d0000)
        /home/kiparis/projects/gocql/conn.go:1425 +0x18ea
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1629 +0x806

goroutine 194 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0xc00033e300?, {0xc9b758, 0xc0001b4c30})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 75 [select]:
github.com/gocql/gocql.(*writeCoalescer).writeFlusher(0xc000386780, 0x30d40?)
        /home/kiparis/projects/gocql/conn.go:880 +0x245
created by github.com/gocql/gocql.newWriteCoalescer
        /home/kiparis/projects/gocql/conn.go:785 +0x305

goroutine 181 [select, 4 minutes]:
github.com/gocql/gocql.(*eventDebouncer).flusher(0xc0000e6b40)
        /home/kiparis/projects/gocql/events.go:42 +0x126
created by github.com/gocql/gocql.newEventDebouncer
        /home/kiparis/projects/gocql/events.go:30 +0x2b7

goroutine 1345 [select]:
github.com/gocql/gocql.(*controlConn).heartBeat(0xc000498940)
        /home/kiparis/projects/gocql/control.go:76 +0x1a9
created by github.com/gocql/gocql.(*controlConn).connect
        /home/kiparis/projects/gocql/control.go:265 +0x2cf

goroutine 161 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867b070, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001e6a98, 0xc000346000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001e6a80, {0xc000346000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc0001e6a80, {0xc000346000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000192060, {0xc000346000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc0001b0e40, {0xc00033e348, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc0001b0e40}, {0xc00033e348, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc0001b0e40?}, {0xc00033e348?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc00033e300, {0xc9b758, 0xc0001b4c30})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0xc0000e6ba0?, {0xc9b758, 0xc0001b4c30})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 1375 [select, 4 minutes]:
github.com/gocql/gocql.(*eventDebouncer).flusher(0xc0005918c0)
        /home/kiparis/projects/gocql/events.go:42 +0x126
created by github.com/gocql/gocql.newEventDebouncer
        /home/kiparis/projects/gocql/events.go:30 +0x2b7

goroutine 182 [select, 4 minutes]:
github.com/gocql/gocql.(*eventDebouncer).flusher(0xc0000e6c00)
        /home/kiparis/projects/gocql/events.go:42 +0x126
created by github.com/gocql/gocql.newEventDebouncer
        /home/kiparis/projects/gocql/events.go:30 +0x2b7

goroutine 128 [select]:
github.com/gocql/gocql.(*writeCoalescer).writeFlusher(0xc0001b1320, 0x30d40?)
        /home/kiparis/projects/gocql/conn.go:880 +0x245
created by github.com/gocql/gocql.newWriteCoalescer
        /home/kiparis/projects/gocql/conn.go:785 +0x305

goroutine 1351 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867b160, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc00045c598, 0xc00030c000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00045c580, {0xc00030c000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc00045c580, {0xc00030c000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc0000140b8, {0xc00030c000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc0005919e0, {0xc0000c0348, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc0005919e0}, {0xc0000c0348, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc0005919e0?}, {0xc0000c0348?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc0000c0300, {0xc9b758, 0xc00040af50})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0x47943a?, {0xc9b758, 0xc00040af50})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 77 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0xc0003a8180?, {0xc9b758, 0xc000388500})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 76 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867af80, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0003a0198, 0xc0003d2000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003a0180, {0xc0003d2000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc0003a0180, {0xc0003d2000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000380040, {0xc0003d2000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc0003866c0, {0xc0003a81c8, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc0003866c0}, {0xc0003a81c8, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc0003866c0?}, {0xc0003a81c8?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc0003a8180, {0xc9b758, 0xc000388500})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0xc00038e4e0?, {0xc9b758, 0xc000388500})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 70 [select]:
github.com/gocql/gocql.(*controlConn).heartBeat(0xc0001e1100)
        /home/kiparis/projects/gocql/control.go:76 +0x1a9
created by github.com/gocql/gocql.(*controlConn).connect
        /home/kiparis/projects/gocql/control.go:265 +0x2cf

goroutine 1352 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0x47943a?, {0xc9b758, 0xc00040af50})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 129 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867ae90, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0003a0518, 0xc0003e4000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003a0500, {0xc0003e4000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc0003a0500, {0xc0003e4000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000380048, {0xc0003e4000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc0003867e0, {0xc0003a8348, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc0003867e0}, {0xc0003a8348, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc0003867e0?}, {0xc0003a8348?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc0003a8300, {0xc9b758, 0xc000388b90})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0x4aa657?, {0xc9b758, 0xc000388b90})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 210 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0xc0003a8300?, {0xc9b758, 0xc000388b90})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 1376 [select, 4 minutes]:
github.com/gocql/gocql.(*eventDebouncer).flusher(0xc000591980)
        /home/kiparis/projects/gocql/events.go:42 +0x126
created by github.com/gocql/gocql.newEventDebouncer
        /home/kiparis/projects/gocql/events.go:30 +0x2b7

goroutine 1383 [select]:
github.com/gocql/gocql.(*Conn).heartBeat(0x47943a?, {0xc9b758, 0xc0004de7d0})
        /home/kiparis/projects/gocql/conn.go:616 +0x192
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:326 +0x78c

goroutine 1359 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867b250, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0002d0a18, 0xc0002d2000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0002d0a00, {0xc0002d2000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc0002d0a00, {0xc0002d2000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc000192068, {0xc0002d2000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc000317b60, {0xc0003a8948, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc000317b60}, {0xc0003a8948, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc000317b60?}, {0xc0003a8948?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc0003a8900, {0xc9b758, 0xc0003614f0})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0x47943a?, {0xc9b758, 0xc0003614f0})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 1357 [select]:
github.com/gocql/gocql.(*Session).reconnectDownedHosts(0xc000600800, 0x0?)
        /home/kiparis/projects/gocql/session.go:374 +0x177
created by github.com/gocql/gocql.(*Session).init
        /home/kiparis/projects/gocql/session.go:321 +0x108c

goroutine 1358 [select]:
github.com/gocql/gocql.(*writeCoalescer).writeFlusher(0xc0004f8b40, 0x0?)
        /home/kiparis/projects/gocql/conn.go:888 +0x310
created by github.com/gocql/gocql.newWriteCoalescer
        /home/kiparis/projects/gocql/conn.go:785 +0x305

goroutine 1382 [IO wait]:
internal/poll.runtime_pollWait(0x7f343867ada0, 0x72)
        /usr/local/go/src/runtime/netpoll.go:306 +0x89
internal/poll.(*pollDesc).wait(0xc0001e7698, 0xc0000b8000?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001e7680, {0xc0000b8000, 0x1000, 0x1000})
        /usr/local/go/src/internal/poll/fd_unix.go:167 +0x405
net.(*netFD).Read(0xc0001e7680, {0xc0000b8000, 0x1000, 0x1000})
        /usr/local/go/src/net/fd_posix.go:55 +0x51
net.(*conn).Read(0xc00037a008, {0xc0000b8000, 0x1000, 0x1000})
        /usr/local/go/src/net/net.go:183 +0xb1
bufio.(*Reader).Read(0xc0004f8840, {0xc0000ba048, 0x1, 0x9})
        /usr/local/go/src/bufio/bufio.go:237 +0x4f3
io.ReadAtLeast({0xc97860, 0xc0004f8840}, {0xc0000ba048, 0x1, 0x9}, 0x1)
        /usr/local/go/src/io/io.go:332 +0xde
io.ReadFull(...)
        /usr/local/go/src/io/io.go:351
github.com/gocql/gocql.readHeader({0xc97860?, 0xc0004f8840?}, {0xc0000ba048?, 0x9?, 0x9?})
        /home/kiparis/projects/gocql/frame.go:436 +0xb6
github.com/gocql/gocql.(*Conn).recv(0xc0000ba000, {0xc9b758, 0xc0004de7d0})
        /home/kiparis/projects/gocql/conn.go:659 +0x130
github.com/gocql/gocql.(*Conn).serve(0x47943a?, {0xc9b758, 0xc0004de7d0})
        /home/kiparis/projects/gocql/conn.go:576 +0x47
created by github.com/gocql/gocql.(*Conn).init
        /home/kiparis/projects/gocql/conn.go:325 +0x6d4

goroutine 54909 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc0001e0a50, 0x1a19c)
        /usr/local/go/src/runtime/sema.go:527 +0x14c
sync.(*Cond).Wait(0xc0001e0a40)
        /usr/local/go/src/sync/cond.go:70 +0xa5
github.com/gocql/gocql.(*writeCoalescer).Write(0xc0004f8b40, {0xc001c68900, 0x47, 0x80})
        /home/kiparis/projects/gocql/conn.go:857 +0x39f
github.com/gocql/gocql.(*Conn).Write(0xc0003a8900, {0xc001c68900, 0x47, 0x80})
        /home/kiparis/projects/gocql/conn.go:332 +0x6e
github.com/gocql/gocql.(*framer).finishWrite(0xc001c489c0)
        /home/kiparis/projects/gocql/frame.go:779 +0x63c
github.com/gocql/gocql.(*writePrepareFrame).writeFrame(0xc001d8ede0, 0xc001c489c0, 0x6680)
        /home/kiparis/projects/gocql/frame.go:861 +0xe16
github.com/gocql/gocql.(*Conn).exec(0xc0003a8900, {0xc9b758, 0xc0003614f0}, {0xc981e0, 0xc001d8ede0}, {0x0, 0x0?})
        /home/kiparis/projects/gocql/conn.go:934 +0x4d9
github.com/gocql/gocql.(*Conn).prepareStatement.func2()
        /home/kiparis/projects/gocql/conn.go:1048 +0x293
created by github.com/gocql/gocql.(*Conn).prepareStatement
        /home/kiparis/projects/gocql/conn.go:1035 +0x3a5
exit status 2
FAIL    github.com/gocql/gocql  300.076s

What did you expect to see?

I expect it to work ok like with all other more older Scylla versions 5.1.x, 5.0.x and 4.6.x which provide following result:

==> Running integration scylla tests with args: -gocql.timeout=60s -proto=4 -rf=1 -clusterSize=1 -autowait=2000ms -compressor=snappy -gocql.cversion=3.11.4 -cluster=192.168.100.11
2023/06/08 11:46:57 logger.go:35: error: failed to connect to "[HostInfo hostname=\"192.168.100.11\" connectAddress=\"192.168.100.11\" peer=\"<nil>\" rpc_address=\"192.168.100.11\" broadcast_address=\"192.168.100.11\" preferred_ip=\"<nil>\" connect_addr=\"192.168.100.11\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"09183270-1045-4cb8-adde-5aed8578c76b\" version=\"v3.0.8\" state=UP num_tokens=1]" due to error: context canceled
2023/06/08 11:46:57 logger.go:35: error: failed to connect to "[HostInfo hostname=\"192.168.100.11\" connectAddress=\"192.168.100.11\" peer=\"<nil>\" rpc_address=\"192.168.100.11\" broadcast_address=\"192.168.100.11\" preferred_ip=\"<nil>\" connect_addr=\"192.168.100.11\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"09183270-1045-4cb8-adde-5aed8578c76b\" version=\"v3.0.8\" state=UP num_tokens=1]" due to error: Keyspace 'invalidKeyspace' does not exist
2023/06/08 11:47:01 logger.go:35: gocql: unable to dial control conn 127.0.0.1:1: dial tcp :0->127.0.0.1:1: connect: connection refused
2023/06/08 11:47:18 logger.go:35: error: failed to connect to "[HostInfo hostname=\"192.168.100.11\" connectAddress=\"192.168.100.11\" peer=\"<nil>\" rpc_address=\"192.168.100.11\" broadcast_address=\"192.168.100.11\" preferred_ip=\"<nil>\" connect_addr=\"192.168.100.11\" connect_addr_source=\"connect_address\" port=9042 data_centre=\"datacenter1\" rack=\"rack1\" host_id=\"09183270-1045-4cb8-adde-5aed8578c76b\" version=\"v3.0.8\" state=UP num_tokens=1]" due to error: context canceled
2023/06/08 11:47:37 logger.go:35: scylla: 192.168.100.11:9042 connection to shard-aware address 192.168.100.11:19042 resulted in wrong shard being assigned; please check that you are not behind a NAT or AddressTranslater which changes source ports; falling back to non-shard-aware port for 5m0s
PASS
ok      github.com/gocql/gocql  43.496s

What did you see instead?

I see constant connection 300s timeouts.

If you are having connectivity related issues please share the following additional information

Describe your Cassandra cluster

please provide the following information

output of 'nodetool status'
output of 'SELECT peer, rpc_address FROM system.peers'
rebuild your application with the 'gocql_debug' tag and post the output

nodetool status:

root@d0e0d92882dd:/# nodetool status
nodetool: Unable to connect to Scylla API server: java.net.ConnectException: Connection refused (Connection refused)
See 'nodetool help' or 'nodetool help <command>'.

CQLSH:

root@d0e0d92882dd:/# cqlsh
Connected to  at 192.168.100.11:9042.
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> SELECT peer, rpc_address FROM system.peers;

 peer | rpc_address
------+-------------

(0 rows)

With gocql_debug:
gocql_debug_for_issue_126.log

The text was updated successfully, but these errors were encountered:

mykaul · 2023-06-08T10:07:45Z

127.0.0.1:1 looks wrong to me and can explain the connection refused.

vponomaryov · 2023-06-08T10:40:13Z

127.0.0.1:1 looks wrong to me and can explain the connection refused.

Only Scylla docker image version was changed.
So, it can be said now that the gocql integration tests are not compatible with the latest Scylla releases.

fruch · 2023-06-08T11:27:48Z

running the same with master, and in 300s it got killed by my kernel:

Jun 08 14:20:13 fruch-laptop kernel: Out of memory: Killed process 3160814 (gocql.test) total-vm:13869872kB, anon-rss:11142664kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:22996

roydahan · 2023-06-14T13:11:08Z

@avelanarius / @zimnx can someone please take a look at this?

avelanarius · 2023-06-14T13:24:15Z

I will look into it.

dkropachev · 2023-06-20T17:00:27Z

This test failure due to the unabling execute prepared statement due to the getting 0x2500 error from the scylla server:
https://github.com/apache/cassandra/blob/7337fc0/doc/native_protocol_v5.spec#L1414-L1417

Same problem from java-driver scylladb/java-driver#124

fruch · 2023-06-20T17:17:12Z

This test failure due to the unabling execute prepared statement due to the getting 0x2500 error from the scylla server: https://github.com/apache/cassandra/blob/7337fc0/doc/native_protocol_v5.spec#L1414-L1417

Same problem from java-driver scylladb/java-driver#124

that issue in java-driver wasn't fixed, and was fixed on scylla end in some way.
so I'm guessing it's not exactly the same.

dkropachev · 2023-06-20T18:33:39Z

I have run this test locally on 5.2.3 and 5.2.1 100 times and it did not fail even once.
Also I run whole suite for 10 times and it also did not fail even once.
Which go version are we using there ?

dkropachev · 2023-06-20T18:48:16Z

I ran it via integration.sh and it did fail, will look at the difference tomorrow

dkropachev · 2023-06-21T12:33:53Z

Most likely it is driver issue, it's logic does not cover the case when prepared statement cache on scylla is under high pressure which could lead to being unable to run big batches as prepared statements, because scylla is expunging statements while driver preparing them.

If this is true, then all our and upstream drivers have exact same issue, which is not regression and only regression there is that scylla tightened prepared statement buffer.

I have opened issue on the scylla repo to confirm it.
Fast fix would be to increase scylla memory limit in docker-compose.yaml

avelanarius · 2023-06-21T13:08:20Z

This test failure due to the unabling execute prepared statement due to the getting 0x2500 error from the scylla server: https://github.com/apache/cassandra/blob/7337fc0/doc/native_protocol_v5.spec#L1414-L1417
Same problem from java-driver scylladb/java-driver#124

that issue in java-driver wasn't fixed, and was fixed on scylla end in some way. so I'm guessing it's not exactly the same.

That particular one was fixed by: scylladb/scylladb#10456 (seehttps://github.com/scylladb/scylladb/issues/10440)

dkropachev · 2023-06-21T13:54:40Z

From scylladb/scylladb#14338 :

Refs #10440 - a similar issue from the past. That issue was fixed by changing the eviction behavior of loading_cache (note that the fix should be improved: #10674).

One possible improvement to prevent infinite spining:

The way it works now (looking at Rust Driver, but others are similar): let's suppose that there is a batch statement of N prepared statements and the driver sends this statement to Scylla. Scylla might respond that some statement in that batch is not prepared (not present in prepared statements cache). The driver will then re-prepare all statements in that batch. However, if the cache is too small, it won't be possible for all statements to be in the cache at the same time.

Improvement:

The driver(s) should not infinite loop, instead impose some limit of how many times it will try to re-prepare (some draft PR from the past: scylladb/java-driver#143)
When Scylla receives a batch of N prepared statements, it should check if the prepared statements cache is large enough and return some error if it's not.

Which confirms my assumption, it is needed to be fixed on gocql side

…lladb#126

dkropachev · 2023-06-21T15:46:56Z

PR to increase memory limit - #128 to avoid hitting the issue

fix(docker-compose.yaml): increase memory limit to avoid hitting #126

…lladb#126

roydahan · 2023-11-09T11:04:38Z

@fruch / @k0machi (or anyone who work on the go-driver-matrix), is this issue still relevant?

vponomaryov · 2023-11-09T11:58:30Z

@fruch / @k0machi (or anyone who work on the go-driver-matrix), is this issue still relevant?

@roy , I reviewed the fix and verified it works. So, this should have been closed with the merge of the fix.

roydahan · 2023-11-09T12:32:40Z

Thanks, closing.

fruch · 2023-11-09T12:33:54Z

@fruch / @k0machi (or anyone who work on the go-driver-matrix), is this issue still relevant?

We didn't have that issue, we are using ccm and not docker with limited memory.

vponomaryov changed the title ~~Tests do work using Scylla 5.2.x~~ Integration tests fail using Scylla 5.2.x Jun 8, 2023

roydahan assigned avelanarius Jun 14, 2023

dkropachev mentioned this issue Jun 21, 2023

Scylla on low memory footpring (512M) returning Unprepared(0x2500) on executing prepared statement after successfully replying to opPrepare(0x9) with resultKindPrepared(0x4) scylladb/scylladb#14338

Open

1 task

dkropachev pushed a commit to dkropachev/gocql that referenced this issue Jun 21, 2023

fix(docker-compose.yaml): increase memory limit to avaoid hitting scy…

f325aa5

…lladb#126

dkropachev mentioned this issue Jun 21, 2023

fix(docker-compose.yaml): increase memory limit to avoid hitting #126 #128

Merged

avelanarius added a commit that referenced this issue Jun 27, 2023

Merge pull request #128 from dkropachev/dk/fix-intergration-sh

8bf4a23

fix(docker-compose.yaml): increase memory limit to avoid hitting #126

sylwiaszunejko pushed a commit to sylwiaszunejko/gocql that referenced this issue Jul 18, 2023

fix(docker-compose.yaml): increase memory limit to avaoid hitting scy…

0ef91ee

…lladb#126

roydahan closed this as completed Nov 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration tests fail using Scylla 5.2.x #126

Integration tests fail using Scylla 5.2.x #126

vponomaryov commented Jun 8, 2023 •

edited

Loading

mykaul commented Jun 8, 2023

vponomaryov commented Jun 8, 2023

fruch commented Jun 8, 2023

roydahan commented Jun 14, 2023

avelanarius commented Jun 14, 2023

dkropachev commented Jun 20, 2023 •

edited

Loading

fruch commented Jun 20, 2023

dkropachev commented Jun 20, 2023

dkropachev commented Jun 20, 2023

dkropachev commented Jun 21, 2023 •

edited

Loading

avelanarius commented Jun 21, 2023

dkropachev commented Jun 21, 2023 •

edited

Loading

dkropachev commented Jun 21, 2023

roydahan commented Nov 9, 2023

vponomaryov commented Nov 9, 2023

roydahan commented Nov 9, 2023

fruch commented Nov 9, 2023

Integration tests fail using Scylla 5.2.x #126

Integration tests fail using Scylla 5.2.x #126

Comments

vponomaryov commented Jun 8, 2023 • edited Loading

What version of Scylla or Cassandra are you using?

What version of Gocql are you using?

What version of Go are you using?

What did you do?

What did you expect to see?

What did you see instead?

Describe your Cassandra cluster

mykaul commented Jun 8, 2023

vponomaryov commented Jun 8, 2023

fruch commented Jun 8, 2023

roydahan commented Jun 14, 2023

avelanarius commented Jun 14, 2023

dkropachev commented Jun 20, 2023 • edited Loading

fruch commented Jun 20, 2023

dkropachev commented Jun 20, 2023

dkropachev commented Jun 20, 2023

dkropachev commented Jun 21, 2023 • edited Loading

avelanarius commented Jun 21, 2023

dkropachev commented Jun 21, 2023 • edited Loading

dkropachev commented Jun 21, 2023

roydahan commented Nov 9, 2023

vponomaryov commented Nov 9, 2023

roydahan commented Nov 9, 2023

fruch commented Nov 9, 2023

vponomaryov commented Jun 8, 2023 •

edited

Loading

dkropachev commented Jun 20, 2023 •

edited

Loading

dkropachev commented Jun 21, 2023 •

edited

Loading

dkropachev commented Jun 21, 2023 •

edited

Loading