Cannot Import from https://wakapi.dev/
solonovamax opened this issue · comments
Describe the bug
I'm attempting to import the data from https://wakapi.dev/ to my own self-hosted instance, and for some reason it keeps failing.
I configured wakapi to import from https://wakapi.dev/api/compat/wakatime/v1
, with my wakapi token, (tested both the normal importer & legacy importer) and
In the logs, it says the following:
wakapi-1 | 2024-02-18T04:22:20.072738807Z [INFO ] [request] status=200, method=GET, uri=/summary?interval=today, duration=4.198602ms, bytes=17266, addr=174.91.204.250, user=solonovamax
wakapi-1 | 2024-02-18T04:22:37.351212285Z [INFO ] scheduling wakatime import for user 'solonovamax' (interval [0001-01-01 00:00:00 +0000 UTC, 2024-02-18 04:22:37.351202115 +0000 UTC m=+66.577188448])
wakapi-1 | 2024-02-18T04:22:37.351345669Z [INFO ] running wakatime import for user 'solonovamax'
wakapi-1 | 2024-02-18T04:22:37.354936161Z [INFO ] [request] status=202, method=POST, uri=/settings, duration=7.358299ms, bytes=73176, addr=174.91.204.250, user=solonovamax
wakapi-1 | 2024-02-18T04:22:38.247704114Z [ERROR] failed to fetch heartbeats for day '0001-01-01' and user 'solonovamax' - got status 500 from wakatime api
I'm not sure why it keeps failing.
I can send my config.yml
if needed.
To re-attempt the imports, I'm connecting to my database and running
DELETE FROM key_string_values WHERE key LIKE 'last%import%solonovamax';
to remove the key that says when I last imported.
System information
Please provide information on:
- Wakapi version: 2.10.5
- Operating system: Debian Linux 6.1.67-1, Docker 25.0.3, docker image: latest
- Database: PostgreSQL (not running in docker, running on host machine)
Hi @solonovamax, thanks for reporting this. I checked the server logs, here's an excerpt:
...
Feb 18 05:22:38 wakapi.dev wakapi[537064]: 2024-02-18T05:22:38.208600967+01:00 [ERROR] failed to retrieve heartbeats - year is not in the range [1, 9999]: 0
Feb 18 05:22:38 wakapi.dev wakapi[537064]: 2024-02-18T05:22:38.208834952+01:00 [INFO ] [request] status=500, method=GET, uri=/api/compat/wakatime/v1/users/current/heartbeats?date=0001-01-01, duration=1.4894ms, bytes=25, addr=86.48.27.55, user=solonovamax
Feb 18 05:22:38 wakapi.dev wakapi[537064]: 2024-02-18T05:22:38.208948648+01:00 [INFO ] [request] status=200, method=GET, uri=/api/compat/wakatime/v1/users/current/heartbeats?date=0001-01-03, duration=1.450698ms, bytes=101, addr=86.48.27.55, user=solonovamax
...
Looks like a time zone issue to me (whatelse). You're located in Toronto time zone (UTC-5) and request 0001-01-01
(without TZ information). The endpoint will try to interpret the date according to your local timezone (see
I'll fix this (hopefully still today). Actually, while investigating this, I realized the importer is super inefficient, by requesting heartbeats starting at year 1. We could save a couple ten thousand requests by starting in year 2000 or in 2013 rather (which is when the first commit to WakaTime was made).
Also, until the bug is fixed, you can probably work around it by inserting a heartbeat to your database with a recent date and its origin
field set to "wakatime"
, see
Lines 536 to 545 in 7dda996
Should work now. I updated wakapi.dev, it should return a safe import range now. Please give it a try (you won't have to recompile your Wakapi).
I'll fix this (hopefully still today). Actually, while investigating this, I realized the importer is super inefficient, by requesting heartbeats starting at year 1. We could save a couple ten thousand requests by starting in year 2000 or in 2013 rather (which is when the first commit to WakaTime was made).
rather than hardcoding to not fetch prior to a certain date, it might be a good idea to have some way to query "what is the oldest hearbeat you have" and then use that
That's what's already being done. The hard-coded date is only used if no heartbeats exist or if something goes wrong.
That's what's already being done. The hard-coded date is only used if no heartbeats exist or if something goes wrong.
ah, I see