Add Gmail data source
mholt opened this issue · comments
1. What is the data source you want to add?
Gmail - the ability to download emails and attachments, with very heavy filtering options. Having a record of emails would be useful because my family often documents their lives to the rest of us through emails.
2. How are items obtained from the data source?
Gmail API via Google OAuth2
2a. If authentication is required, how does a user create or obtain credentials for Timeliner to access the data?
Google OAuth2 is already documented in the wiki.
2b. If an API is available, what are its rate limits?
250 queries per user per second: https://developers.google.com/gmail/api/v1/reference/quota
2c. If a file is imported, how is the file obtained?
n/a
3. What constitutes an "item" from this data source?
An email or attachment.
4. How can items from this data source be related?
Emails can have attachments. Emails can be replies to other emails.
5. What constitutes a "collection" from this data source?
Perhaps a conversation (a thread of messages); but that might be redundant if emails can be related as replies to other ones.
6. What might not be trivial, obvious, or straightforward when implementing this data source?
A lot of filtering will need to be configurable, since Gmail contains a messy amount of noise; somehow I have Google Voice messages, Hangout chats, and other things that aren't really emails in my Gmail account which also show up in the API.
Will need to filter by label, subject, sender, To, other headers, etc. Optionally include attachments (from some emails only, or files of a certain type or size, etc).
I've started on this.
Be great to see this implemented - perhaps also for other email providers (yahoo/hotmail - lots of history in those old accounts). Perhaps an IMAP/POP3 general access data source would be great. Unless there is this capability already.
IMAP/POP3 would be a separate data source (it does not yet exist); this one is specifically for Gmail via its API.
I have an implementation of .eml and .mbox files working in Timelinize (see #84 and updated README). Gmail API integration can come later, but it's quite compelling that the archive files are the superior import method.