-
Notifications
You must be signed in to change notification settings - Fork 13.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
convert postgresql date_trunc() to UTC to prevent pandas error #4319
Conversation
From a quick look at the docs, it looks like Sometimes I think about all of the cumulated engineering wasted time in the world spent thinking about time zones and unicode and I sob a little. |
Couldn't agree more. I changed the cast to a timezone conversion and it seems to do the trick as well, maybe a bit more robust even. Though I'm still getting the same incorrect hours on the graphs as before (me and my data are on UTC+2 or +3 DST) but this may not be the PR to fix that one. |
I ran some more tests of my own, and it seems that just converting to UTC will cause invalid times with the type "timestamp without time zone" as it adds the current (session) timezone to the timestamp. It also swaps the return type so "timestamp without time zone" becomes "timestamp with time zone" and vice versa. I found that the most consistent approach would be to use "SET TIMEZONE to 'UTC';" at the start of a connection and then casting to "timestamp without time zone". This way all returned timestamps would be in UTC. Couldn't find a proper place to set a query to run at connection time though, any pointers? Also this would then require the UI to handle TZ conversions when rendering the data. |
…e#4319) * cast postgresql date_trunc() to timestamp without time zone to prevent pandas error * fix formatting for flake8 * change cast to timezone conversion instead
…e#4319) * cast postgresql date_trunc() to timestamp without time zone to prevent pandas error * fix formatting for flake8 * change cast to timezone conversion instead
Postgresql DATE_TRUNC() returns type "timestamp with time zone" when used with a "date" type. This causes an error with pandas library using utc=False and when used with a time grain different from the source column.
The problem is fixed here by casting the function call to "timestamp without time zone" regardless of the source type. This fix may cause wrong times to be shown in the query result though, depending on the data.
Reference to issue #4250