How do you convert a timestamp into a datetime in python with the correct timezone?

Question

You would naively expect this to work:

import pytz
from datetime import datetime
def tz_datetime_from_timestamp(timestamp):
    """Convert a timestamp into a datetime """
    tz = pytz.timezone('Australia/Perth')
    server_time = datetime.utcnow()
    delta = tz.utcoffset(server_time)
    value = datetime.utcfromtimestamp(timestamp).replace(tzinfo=tz)
    return value + delta

print tz_datetime_from_timestamp(1416387060)

and convert the timestamp 1416387060 into Wed Nov 19 16:51:00 2014 GMT+8:00.

...but it does not. It prints:

2014-11-19 16:51:00+07:43

The timezone of Australia/Perth is not GMT+7:43.

It is GMT+8:00. This is unequivocally stated on the Australian government website http://www.australia.gov.au/about-australia/our-country/time:

AWST is equal to Coordinated Universal Time plus 8 hours (UTC +8).

So, where the heck is pytz pulling 7.43 from?

Well, it turns out the pytz pulls its data from the http://www.iana.org/time-zones time zone database, and that database states:

# Western Australia
#
# Rule  NAME  FROM  TO  TYPE  IN  ON  AT  SAVE  LETTER/S
Rule  AW  1974  only  - Oct lastSun 2:00s 1:00  D
Rule  AW  1975  only  - Mar Sun>=1  2:00s 0 S
Rule  AW  1983  only  - Oct lastSun 2:00s 1:00  D
Rule  AW  1984  only  - Mar Sun>=1  2:00s 0 S
Rule  AW  1991  only  - Nov 17  2:00s 1:00  D
Rule  AW  1992  only  - Mar Sun>=1  2:00s 0 S
Rule  AW  2006  only  - Dec  3  2:00s 1:00  D
Rule  AW  2007  2009  - Mar lastSun 2:00s 0 S
Rule  AW  2007  2008  - Oct lastSun 2:00s 1:00  D
Zone Australia/Perth   7:43:24 -  LMT 1895 Dec
       8:00 Aus AW%sT 1943 Jul
       8:00 AW  AW%sT

Well. That's just great.... but it's also incorrect.

...and so is python, as a result, unfortunately.

The olson time database is full of values like like (4 hours and 15 minutes, 5 hours and 20 minutes, etc.). Totally weird.

Technically I don't really care what the exact timezone is, so long as its universally consistent; unfortunately it's not. Making (for example) an API call and passing the timezone across to a remote service that (correctly) uses the GMT+8 timezone results in a slightly discrepancy between values when the timezones are converted to UTC for processing.

ie. To be absolutely clear, these two values are different times when converted to UTC:

• Wed Nov 19 16:51:00 2014 GMT+8:00
• Wed Nov 19 16:51:00 2014 GMT+7:43

Irritatingly:

server_time = datetime.utcnow()
delta = tz.utcoffset(server_time)
print(delta)

Yields:

8:00:00

So clearly pytz internally does know the correct timezone offset. It just generates output with the wrong value attached.

So, finally: How do I generate a datetime with the correct timezone attached to it?

Answer 1

The long story short, pytz defaults to the incorrect timezone, and datetime.replace() doesn't perform conversions:

import pytz
from datetime import datetime

fmt = '%Y-%m-%d %H:%M:%S %Z%z'
tz = pytz.timezone('Australia/Perth')
timestamp = 1416387060
print(tz)

# Correct way
utctime = pytz.utc.localize(datetime.utcfromtimestamp(timestamp))
print(utctime.strftime(fmt))

astimezone = utctime.astimezone(tz)
print(astimezone.strftime(fmt))

localtime = tz.normalize(astimezone)
print(localtime.strftime(fmt))

# Incorrect way
incorrect = datetime.utcfromtimestamp(timestamp).replace(tzinfo=tz)
print(incorrect.strftime(fmt))

You can read more about this in the various bugs in pytz in which it is discussed:

• https://bugs.launchpad.net/pytz/+bug/1394441
• https://bugs.launchpad.net/pytz/+bug/1319939
• https://bugs.launchpad.net/pytz/+bug/1389042