Support for full screen tagging
breandan opened this issue · comments
According @bulenkov, most or all String painting can be traced through edit: Bypass rendering stack altogether, let's capture raw pixels and run OCR, then draw bounding boxes. Then we can simulate a mouse click in the desired region to "select" the tag.Graphics2DLog (other places to look: Graphics2DLog, DrawString/DrawChars). If this is the case, we can search/tag all UI text on the screen (ie. not only editor text, but menus, toolbars, stack traces, etc).
Implementing this feature should also support tagging for multiple editor windows simultaneously.
edit: See this SO Question for implementation details.