postgres: expose database/sql tunables

[srenatus]

This is for #1354.

I've not had a need for MaxIdleConns and ConnMaxLifetime myself, but it felt weird to only expose of the three settings.

[ericchiang]

When would it be appropriate to set these to a higher value? (about how many clients?) What's the default?

[srenatus]

The defaults right now should be zeros, which is the same as the stdlib default.

[srenatus]

Realised I hadn't answered your question at all there, but it was time to go to bed. I'll update the issue #1354 with some details, as I'd think that's what people might find first.

[ericchiang]

Can we include defaults here that we feel are reasonable values?

[srenatus]

👍 sure. I'm not sure what makes for good defaults, but I can try 😅

[ericchiang]

I thought we already added retries in a previous PR.

[srenatus]

(rebased away; but #1356 hasn't gone in yet..)

[srenatus]

The defaults are getting messed up by this right now. Will push another commit.

[srenatus]

✔️ Now, if these are not provided in the config, they'll not be set. So, database/sql's default for MaxIdleConns will stay intact.

I've looked into adding further tests around the connection counts, and came up with a tests here:

diff --git a/storage/sql/postgres_test.go b/storage/sql/postgres_test.go
index a207739..64d6fef 100644
--- a/storage/sql/postgres_test.go
+++ b/storage/sql/postgres_test.go
@@ -3,8 +3,12 @@
 package sql
 
 import (
+	"context"
+	"database/sql"
 	"os"
+	"sync"
 	"testing"
+	"time"
 )
 
 func TestPostgresTunables(t *testing.T) {
@@ -46,4 +50,44 @@ func TestPostgresTunables(t *testing.T) {
 			t.Errorf("expected MaxOpenConnections to be set to 101, got %d", m)
 		}
 	})
+
+	t.Run("attempt to open max+1 connections stalls", func(t *testing.T) {
+		cfg := *baseCfg
+		max := 2
+		cfg.MaxOpenConns = &max
+		c, err := cfg.open(logger, cfg.createDataSourceName())
+		if err != nil {
+			t.Fatalf("error opening connector: %s", err.Error())
+		}
+		defer c.db.Close()
+		done := sync.WaitGroup{}
+		done.Add(1)
+		hogs := sync.WaitGroup{}
+		hogs.Add(max)
+		hogConnection := func(db *sql.DB) {
+			r, err := db.Query("SELECT 1")
+			if err != nil {
+				return
+			}
+			hogs.Done()
+			done.Wait()
+			r.Close() // not closing the rows makes it occupy the connection
+		}
+		for i := 0; i < max; i++ {
+			go hogConnection(c.db)
+		}
+		hogs.Wait()
+		if n := c.db.Stats().OpenConnections; n != max {
+			t.Fatalf("expected %d open connections, found %d", max, n)
+		}
+
+		ctx, cancel := context.WithTimeout(context.Background(), time.Second)
+		defer cancel()
+		_, err = c.db.QueryContext(ctx, "SELECT 1")
+		if err != context.DeadlineExceeded {
+			t.Errorf("expected context deadline exceeded for (max+1)th connection, got %v", err)
+		}
+
+		done.Done() // release it all
+	})
 }

but since this would be testing github.com/lib/pq more than it would test storage/sql, I don't think it's worth including.

[ericchiang]

Differentiating between zero and nil is hairy. Can we use 0 to set a reasonable default?

[ericchiang]

lgtm