reworkd / tarsier

Vision utilities for web interaction agents πŸ‘€

Home Page:https://reworkd.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

πŸ› Annotations not removed when filling out inputs

rgarcia opened this issue Β· comments

Seeing this a lot, e.g. when I tweak the example in the cookbook to log in to resy :)

image

I've tried adding both await page.locator(x_path).fill('') and await page.locator(x_path).clear() to the type_text tool, but it doesn't seem to work (my playwright experience is very limited).

Heyo, thanks for trying out the tool!

This issue arises because our tagging process will actually insert text directly into the desired elements.

Within the cookbooks, after any action is taken we re-scan the page which will introduce more tags in the elements as shown in your image.

We're thinking of two approaches to solve this:

  1. Functionality to overlay tags on top of elements as opposed to writing in the elements directly (Similar to vimium)
  2. Exposing an API in tarsier to delete all tags within elements. (In this case, you would have to manually clear tags before actions like clicking.

I'll add a first issue ticket for number two. Thanks for the feedback!

@KhoomeiK will be submitting a fix for this shortly