Load Replit DB URL from a file #139

brenoafb · 2023-04-07T19:10:46Z

Why

The JWT used for authenticating into the Replit DB is placed on the Repl as an environment variable on boot and lasts for 30h or so. This is fine for shorter lasting Repls, but if we want to have long-running Repls we need a mechanism to refresh the DB token.

What changed

Let's make it so that we read the DB URL data from a file, which can be refreshed while a program is running. If the file does not exist or we fail to read from it, fall back to the environment variable. We then run this hourly.

Testing

Fork this Repl
Change the version of the replit dependency in pyproject.toml to the latest commit in this branch. The resulting [tool.poetry.dependencies] section should look something like the following.

[tool.poetry.dependencies]
python = ">=3.10.0,<3.11"
numpy = "^1.22.2"
Flask = "^2.2.0"
urllib3 = "^1.26.12"
replit = { git = "https://github.com/replit/replit-py.git", rev = "71466f774ef1c0d737259d249085dc42606424cd" }

Run poetry update
Run the Repl and make sure the application works
Deploy the Repl and make sure that it still works. Also check that both the plain Repl and the deployment are using the same database by checking that the values shown on the page match.

brenoafb · 2023-04-07T19:11:05Z

Current dependencies on/for this PR:

master
- PR Load Replit DB URL from a file #139 👈

This comment was auto-generated by Graphite.

airportyh · 2023-05-10T15:52:36Z

Probably meant poetry update instead of flask update?

airportyh · 2023-05-10T15:58:32Z

src/replit/database/default_db.py

+    """
+    global db
+    global db_url
+    if path.exists("/tmp/replitdb"):


Didn't find this file. Is this created exclusively in deployment?

airportyh · 2023-05-10T16:00:14Z

tests/test_database.py

@@ -16,6 +16,13 @@ async def asyncSetUp(self) -> None:
        """Grab a JWT for all the tests to share."""
        if "REPLIT_DB_URL" in os.environ:
            self.db = AsyncDatabase(os.environ["REPLIT_DB_URL"])
+        elif "DB_RIDT" in os.environ:


Is this var set exclusively in deployment? Or for testing only?

Just for testing, I found it to be the quickest way to switch between using JWT or Repl Identity for authing into the DB

airportyh

lgtm. Just a couple of questions.

airportyh · 2023-05-11T10:55:51Z

src/replit/database/default_db.py

+        db = Database(db_url)
+    else:
+        # The user will see errors if they try to use the database.
+        db = None


Sorry I missed this. But there's a problem here. The global db being updated will not be updated for callers that are importing it via from default_db import db.

Example counter.py

count = 1 def increment(): global count count += 2

main.py

from counter import count, increment print(count) increment() print(count)

This will print:

1 1

I suggest instead of making updating the db with a new instance, make the database self-reloading. Or have a db that's exported to the users be a proxy object that defers to a read db instance which handles reloading.

Might be doable to periodically update the db_url https://github.com/replit/replit-py/blob/master/src/replit/database/database.py#L419

Nice catch! i think updating the db_url property should work here, let me take a look

Alright, I added a update_db_url method to the Database class. Now we initialize the default database and call the method periodically with an up-to-date URL. I tested on an example Repl and it appears to be working fine.

airportyh

Tested with a 10 second period. Works fine after the db refresh.

PotentialStyx · 2023-06-11T21:11:07Z

Is a timer in a thread really needed? Couldn't the db class just store the time of last url refresh. Then every db operation check if the last refresh was over an hour ago, if so only refresh the url then.

lafkpages · 2023-07-02T21:05:00Z

Is this ever gonna get merged? I've seen a few users complaining about no support for Repl DB in deployed Python Repls.

triviatroy · 2023-07-02T21:11:35Z

What is the timeline on this update? This is a high priority for me given always-on is unstable now.

brenoafb · 2023-07-03T12:08:17Z

Is this ever gonna get merged? I've seen a few users complaining about no support for Repl DB in deployed Python Repls.
What is the timeline on this update? This is a high priority for me given always-on is unstable now.

This fix is already in pypi (version 3.3.0 https://pypi.org/project/replit/). Just run poetry update replit on a Repl and the DB should work on a deployment. Sorry for any inconvenience, if you need further help getting ReplDB to work on deployments just get in touch with support and we'll help you out :)

PotentialStyx · 2023-07-04T02:57:11Z

This fix is already in pypi (version 3.3.0 https://pypi.org/project/replit/).

Just wondering, but shouldn't this PR be merged then?

lafkpages · 2023-07-06T15:07:26Z

Nice

Load Replit DB URL from a file

a6b5e81

Add option to auth into DB using Repl Identity

aaaf2bd

brenoafb marked this pull request as ready for review April 10, 2023 18:43

brenoafb added 3 commits May 10, 2023 13:54

Periodically reload DB

5a93f6a

Add some debug logs

a2ed3d4

Cleanup

71466f7

airportyh reviewed May 10, 2023

View reviewed changes

airportyh approved these changes May 10, 2023

View reviewed changes

airportyh requested changes May 11, 2023

View reviewed changes

Update refresh flow

8954610

brenoafb requested a review from airportyh May 11, 2023 16:53

Remove print

9f9ade6

airportyh approved these changes May 11, 2023

View reviewed changes

Update version

ae33648

install poetry 1.3.2

7857fa3

masad-frost merged commit cee2529 into master Jul 6, 2023

masad-frost deleted the bb04-07-Load_Replit_DB_URL_from_a_file branch July 6, 2023 14:53

blast-hardcheese added the enhancement New feature or request label Dec 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load Replit DB URL from a file #139

Load Replit DB URL from a file #139

brenoafb commented Apr 7, 2023 •

edited

Loading

brenoafb commented Apr 7, 2023

airportyh commented May 10, 2023

airportyh May 10, 2023 •

edited

Loading

brenoafb May 10, 2023

airportyh May 10, 2023 •

edited

Loading

brenoafb May 10, 2023 •

edited

Loading

airportyh left a comment

airportyh May 11, 2023 •

edited

Loading

airportyh May 11, 2023

brenoafb May 11, 2023

brenoafb May 11, 2023 •

edited

Loading

airportyh left a comment

PotentialStyx commented Jun 11, 2023

lafkpages commented Jul 2, 2023

triviatroy commented Jul 2, 2023

brenoafb commented Jul 3, 2023

PotentialStyx commented Jul 4, 2023

lafkpages commented Jul 6, 2023

Load Replit DB URL from a file #139

Load Replit DB URL from a file #139

Conversation

brenoafb commented Apr 7, 2023 • edited Loading

Why

What changed

Testing

brenoafb commented Apr 7, 2023

airportyh commented May 10, 2023

airportyh May 10, 2023 • edited Loading

Choose a reason for hiding this comment

brenoafb May 10, 2023

Choose a reason for hiding this comment

airportyh May 10, 2023 • edited Loading

Choose a reason for hiding this comment

brenoafb May 10, 2023 • edited Loading

Choose a reason for hiding this comment

airportyh left a comment

Choose a reason for hiding this comment

airportyh May 11, 2023 • edited Loading

Choose a reason for hiding this comment

airportyh May 11, 2023

Choose a reason for hiding this comment

brenoafb May 11, 2023

Choose a reason for hiding this comment

brenoafb May 11, 2023 • edited Loading

Choose a reason for hiding this comment

airportyh left a comment

Choose a reason for hiding this comment

PotentialStyx commented Jun 11, 2023

lafkpages commented Jul 2, 2023

triviatroy commented Jul 2, 2023

brenoafb commented Jul 3, 2023

PotentialStyx commented Jul 4, 2023

lafkpages commented Jul 6, 2023

brenoafb commented Apr 7, 2023 •

edited

Loading

airportyh May 10, 2023 •

edited

Loading

airportyh May 10, 2023 •

edited

Loading

brenoafb May 10, 2023 •

edited

Loading

airportyh May 11, 2023 •

edited

Loading

brenoafb May 11, 2023 •

edited

Loading