feat: Add worker retry mechanic for database connection

[eelkevdbos]

Context
When using the worker process defined in pyproject.toml like below, it sometimes boots before the database is ready.

[tool.plain.dev.services]
postgres = {cmd = "docker run --name app-postgres ..."}

[tool.plain.dev.run]
worker = {cmd = "uv run plain worker run"}

Resulting in an error like:

plain.models.db.OperationalError: connection failed: connection to server at "127.0.0.1", port 54321 failed: FATAL:  the database system is starting up

Proposed solution
Giving the worker a (limited) retry mechanic, for example with tenacity, would greatly improve the reliability of booting.

[davegaeddert]

The comment here is directly related to this one too: #24 (comment)

The current solution is to use plain models db-wait && plain worker run...

In plain.dev, the services and main dev command also use db-wait explicitly (but in the package itself), and I suppose we could consider whether db connection retrying should be happening at a higher level, like in plain.models? That would be a pretty big change though and, without thinking it all the way through, it seems like there's also a lot of situations where an immediate error is a good thing when the database isn't available...

Or, maybe this is just a plain.dev concern for now and what we should really do is start up the [tool.plain.dev.services], then run plain models db-wait and plain migrate --backup once the services are running, and THEN start web server and all the other [tool.plain.dev.run] commands? That's probably the way I'd lean initially — then plain.worker itself doesn't need to do anything differently and would function just like the plain web server currently does.

[davegaeddert]

@eelkevdbos I updated plain.dev to do what I described above — give it a try and let me know! It should remove the need to run plain models db-wait manually for this type of thing.