Very slow (~10s) Google Login

[tripleredadam]

I am running gkeepapi on an AWS Lambda function.

After a laborious process to get login to work at all (for reference, maybe to help others)

// Use AmazonLinux EC2 for this
// First install Python 3.9.x (I think I used 3.9.13)

mkdir python && cd python
pip3.9 install -t . gkeepapi
pip3.9 install -t . gpsoauth==0.4.2

// Copy over to s3 and update lambda layer
// Set mobile management to "unmanaged" here https://admin.google.com/ac/devices/settings/general

Now login works (yay!) but it takes upwards of 10s - using the token on future logins its more like 9.5s

Appreciate this is not explicitly a gkeepapi issue - but is there anything that can be done to speed this up, do you think?

[Kai]

If you have a large number of notes, you can speed up login by caching notes: https://gkeepapi.readthedocs.io/en/latest/#caching-notes. Of course, this means you'll need to keep state somewhere.

If you're only interesting in creating notes from this lambda (NOT updating/deleting existing notes), then there are some optimizations you could use. These aren't implemented atm, but I can describe how they'd work.

[tripleredadam]

Thanks for the pointer, but implementing this cache (mine is 2.5mb) actually makes the login part take longer - now 12s!

For reference, anyone coming here from Google - as I am running in a Lambda - S3 is needed for file storage so I've adapted the code at https://gkeepapi.readthedocs.io/en/latest/#caching-notes to suit

My use case is only for updating a single list - any optimisations for this you can suggest would be awesome

    state = self.load_state(AWS_BUCKET_NAME, state_path)
    self.save_state(AWS_BUCKET_NAME, state_path, state)

    def load_state(self, AWS_BUCKET_NAME, state_path):
        s3 = boto3.resource('s3')
        
        try:
            s3object = s3.Object(AWS_BUCKET_NAME, state_path) 
        
            data = s3object.get()['Body'].read().decode('utf-8')
            return json.loads(data)
        except Exception as e:
            return None
        
    def save_state(self, AWS_BUCKET_NAME, state_path, state):
        s3 = boto3.resource('s3')
        s3object = s3.Object(AWS_BUCKET_NAME, state_path)

        s3object.put(
            Body=(bytes(json.dumps(state).encode('UTF-8')))
        )   

[Kai]

That's rather odd. Can you profile the following sequence?

import time

...
start = time.time()
keep.login(user, pass, sync=False)
end_1 = time.time() - start
keep.restore(state)
end_2 = time.time() - end_1
keep.sync()
end_3 = time.time() - end_2
print(end_1, end_2, end_3)

[tripleredadam]

So this isn't exactly how you asked for it, but I've just adapted my existing profiling code - it certainly has revealed that the delay can be explained in the state restoration part too

Got keep --- 0.0005350112915039062 seconds ---
Got secret --- 1.4195172786712646 seconds ---
Loaded state --- 4.941903352737427 seconds ---
Logged in --- 5.919759035110474 seconds ---
Restored state --- 9.440753936767578 seconds ---
Synced --- 10.271468877792358 seconds ---
Saved state --- 12.87191390991211 seconds ---

[Kai]

Hmm, with 2.5 mb that does make sense...

Optimization ideas:

All Google Keep clients keep track of a cursor, which lets the server determine what updates are available. If you remove everything inside the cache file except for the keep_version key, that should let you create new Notes. For Lists, you just need to go a step beyond that - make sure that the List and all ListItems are cached and nothing else. Important: Please test this on a test List first

Try pickle-ing the Keep class and see if that's any faster.

[tripleredadam]

As I only needed a single list; what I did in the end to make this optimal was create a new google account that holds the single keep list and share it with my main account

gkeepapi logs in to the new account and it’s blazing fast, especially when also using the token and caching functionality

I am now finally using Alexa to add things to my google keep list! Thanks for all your hard work to help me with this :)

Once I’ve finished the CloudFormation templates for it I’ll be publishing and I’ll come back and share the repo. But you can probably close this (it wasn’t a real issue anyway 😂)

Cheers

[Kai]

Glad you arrived at a solution. I think that's a great workaround - I'll add a note in the docs. Thanks!