gui-automation

There are 1 repository under gui-automation topic.

• crab

  camel-ai / crab

  🦀️ CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents. https://crab.camel-ai.org/

  gui-automationlanguage-model-agentlarge-language-modelsmulti-agent-systemsvisual-language-models
  Language:Python 377

• TransformerOptimus / AutoNode

  AutoNode: A Neuro-Graphic Self-Learnable Engine for Cognitive GUI Automation

  artificial-intelligenceartificial-neural-networksgui-automationknowledge-graphllmmachine-learningrobotic-process-automation
  Language:Python 290

• Integration-Automation / AutomationIDE

  AutomationIDE , Python IDE creare by Python, Include [WEB, API, GUI, Load & Stress] automation.

  api-automationapi-automation-testingautomationguigui-automationgui-testingload-testingpythonseleniumtest-automationtestingweb-automationbinggptchatchatbotchatgptidetext-editorpython-ide
  Language:Python 64

• Integration-Automation / AutoControlGUI

  A framework for GUI automation

  automationtestingpythonwindowsmacoslinuxautomatictest-automationtest-automation-frameworktesting-toolstesting-frameworkauto-controlguigui-automationgui-testingkeyword-driven-testingreport-generatorpython-automation
  Language:Python 43

• imbus / robotframework-robosapiens

  Fully localized Robot Framework library for automating the SAP GUI using text locators

  gui-automationgui-testingrobotframework
  Language:Python 38

• juancarlospaco / nim-xdo

  Nim GUI Automation Linux, simulate user interaction, mouse and keyboard.

  nimnim-langxdotoolui-automationgui-automationautohotkeylinuxkeyboard-inputmouse-movementxdokeyboardmousex11
  Language:Nim 38

• nolantait / deskbot

  Desktop automation library for Ruby

  bottingdesktop-automationgui-automationruby
  Language:Ruby 28

• Dentosal / autogui-rs

  GUI automation toolkit for Rust

  rust-librarygui-automation
  Language:Rust 23

• abdur75648 / V-Zen

  V-Zen: Efficient GUI Understanding and Precise Grounding With A Novel Multimodal LLM Resources

  agichatgptgptgrounding-dinoguigui-automationlarge-language-modelsllamallmmistralmultimodal-large-language-modelssuperagivicuna
  7

• arsho / t_rex_bot

  Google Chrome offline game (T Rex) bot using Python.

  botgui-automationmousepyautoguirex
  Language:Python 7

• michalkasiarz / selenium-webdriver-java

  Selenium WebDriver with Java from LetsKodeIt

  gui-automationjava-automationseleniumselenium-javaselenium-webdriver
  Language:Java 7

• ClimenteA / pyrobogui

  Wrapper around pyautogui - plus some new functions. Ideal for doing robotic process automation.

  python-rpagui-automationrpa-robotsrpa
  Language:Jupyter Notebook 5

• JustoSenka / MouseRobot

  Mouse Robot C#

  automationgui-automationgui-testingmouse-robotrotestingui-automated-tests
  Language:C# 5

• lambsteak / auto-pixel-canvas

  Automatically supply intelligent clicks to targeted kinds of pixel blocks in a dynamic canvas element.

  automationcanvascanvas-gamecanvas-paintergui-automationpixel-blockspyautoguipythonpython3
  Language:Python 5

• sagerpascal / computer-control-agents

  Official Repo of "AI Agents for Computer Use: A Review of Instruction-based Computer Control, GUI Automation, and Operator Assistants"

  agentcomputer-controldesktopgui-automationmobile
  Language:HTML 5

• varunbpatil / OTP_scanner

  Scan one-time-password generated on your mobile phone via webcam for use on your computer.

  python3opencv-pythonopencv3computer-visiondigit-recognitiongui-automationpyautogui
  Language:Python 5

• crispyzingy / gui-automation

  💻⌨🖱 Programmatically control the mouse & keyboard

  pythonautomationgui-automationcontrol-mousecontrol-keyboard
  Language:Python 4

• jithurjacob / Clash-Of-Clans---TrophyDropBot

  A simple Automation bot for dropping trophies in Clash of Clans

  pythonclashofclansclash-of-clanssikuliautomationgui-automationbot
  Language:HTML 4

• Mordekai66 / design2apps

  This project automates the conversion of Figma designs into Python code, supporting frameworks like Tkinter, Kivy, and PyQt5. It streamlines UI development by transforming design files into fully functional, customizable Python code while preserving the original layout, colors, fonts, and components.

  code-generationdesign-to-appdesign-to-codedesign-to-kivydesign-to-pyqt5design-to-tkdesign-to-tkinterdeveloper-toolsfigma-apifigma-pyqt5figma-to-codefigma-to-kivyfigma-to-tkfigma-to-tkintergui-automationui-generator
  Language:Python 4

• EddieTGH / tftBot

  A Python-programmed bot using GUI Automation (PyAutoGUI library) and image recognition (OpenCV) that completely automated the process of queuing into and playing the video game Teamfight Tactics in order to earn tokens for rewards while I was not actively using my computer (ie. during the night).

  opencv-pythonpyautoguileague-of-legendsteamfight-tacticsgui-automation
  Language:Python 3

• pmqtt / IntelliType

  Automate the typing process

  automationgui-automationscriptingtest-automationtext-processing
  Language:Python 3

• veilair / UI-testing-development

  An ongoing curated list of frameworks, books, articles, talks, screencasts, recordings, libraries, learning tutorials and resources about UI Testing & Best Practices Development

  uiui-componentsunit-testingui-testingui-libraryui-frameworkui-designintegrationintegration-testingtestingtesting-toolstest-frameworkui-toolsguigui-frameworkgui-automationgui-toolboxbest-practicesguidelines
  3

• Vibhu-Agarwal / Automated-Draw-In-MS-Paint

  It asks for names. Using GUI Automation, opens MS-paint in Windows 10 and draws the names using brushes or pencils.

  ms-paintgui-automationdrawingpythonpython3automation
  Language:Python 3

• f01c33 / robotgo-macro-recorder

  records keypresses and mouse clicks and exports as a Go program that does the same actions

  automationgui-automationmacrosrobotgo
  Language:Objective-C 2

• lksrz / mcp-desktop-pro

  Advanced MCP server for AI agents, computer use automation, and desktop operator control: Intelligent Window Management 🪟, Multi-Action Chaining ⛓️, AI-Optimized Screenshots 🖼️, macOS and Retina Display Support 🍎. Ideal for testing apps, games, and running desktop tasks locally with AI agents through Model Context Protocol.

  automation-testingdesktop-automationgui-automationkeyboard-automationmcpmcp-servermodel-context-protocolmouse-automationrobotjsscreen-capturescreenshot-automationmulti-action-sequencesai-agentai-operatorcomputer-usemouse-controldesktop-operator
  Language:JavaScript 2

• rafbradey / miscrit-auto-farm-bot-py

  A simple bot for Miscrits that helps you grind EXP by auto-attacking and capturing rare Miscrits using image recognition.

  botimage-recognitionmiscritsautomationgame-botgui-automationpython-bot
  Language:Python 2

• Rafiere / click-helper

  A simple GUI automation tool that uses a declarative YAML-based language. Instead of writing complex code, you can define your automation flows with clear and readable commands. It makes it easy to automate desktop tasks, from simple clicks and movements to opening applications, all within a single configuration file.

  automationclidesktop-automationgui-automationproductivitypythonscriptingyaml
  Language:Python 2

• venomnert / Cell

  Cell the trading bot

  gui-automationpythonstocktrading-bot
  Language:Python 2
• PYAutmation

  batuafk / PYAutmation

  Code executor in python (send key in background, locate image, keyboard, mouse, discord, tor proxy, growtopia)

  growtopiagrowtopia-autofarmgrowtopia-autospamgrowtopia-hackautomationgui-automationpyautoguipython-automationpywinautoui-automationcode-executioncode-executorexecutorguiscript-executorui
  Language:HTML 1

• DineshNeupane / GUI-Automation-Tool-GAT

  Python based GUI automation tool to do tedious repetitive tasks automatically, uses Tkinter and pyautogui.

  automationclickbotgui-automationpython
  Language:Python 1

• EddieTGH / Python-GUI-Bot

  This is Edmond 2.0, a Python-programmed bot using GUI Automation (PyAutoGUI library) and image recognition (OpenCV) that completely automated the entire “trading” process in my favorite video game, Realm of the Mad God.; facilitated game objectives including publishing trade offers, communicating with both sellers and buyers, selecting the items to trade, and confirming that the trade was legit. Recorded videos to continuously catch and fix bugs and improve the bot’s communication methods.

  pyautoguiopencv-pythongui-automationrealm-of-the-mad-godtrading-bot
  Language:Python 1

• FilipRokita / pyClicker

  pyClicker is a Python program that automates mouse clicks. It utilizes the PyAutoGUI library to perform the clicking action and the keyboard library to start and stop the clicking process. Users can customize the start and stop buttons according to their preferences. With pyClicker, repetitive clicking tasks can be completed with ease.

  autoclickerguigui-automationkeyboardmousepyautoguipythonpython3utilitiesutility
  Language:Python 1

• kumarmuthu / PythonGUIAutomation

  PythonGUIAutomation

  guigui-automationpython3youtube-viewer
  Language:Python 1

• mccarrmb / automator-selenium

  A modifiable, Ruby-based Selenium3 project scaffold

  seleniumselenium-webdriverselenium-rubyselenium-testsautomationgui-automationbrowser-automationrubycontinuous-integrationtravis-ciparallel-testsconcurrent-queueselenium-gridminitestrakebundlerrubocoppage-object
  Language:Ruby 1

• melchiorhering / Automating-DS-DE-Workflows

  A modular framework for benchmarking multimodal AI agents in a reproducible, full-OS environment. Using and adaption of the Smolagents's CodeAgent, Docker containers to run the VM in, VM's created using Qemu.

  ai-agentsbenchmarkingdockergui-automationqemusmolagentsmultimodal-agentsspider2-v
  Language:Jupyter Notebook 1
• ColorSearch

  H-D-OWL / ColorSearch

  A WinAPI-based console application for detecting specific pixel colors on your screen and automating interactions.

  autoclickautomationautomation-toolsconsole-appcppgame-automationgui-automationhookspixelscanscreen-capturescreen-scrapingwinapiwindowstriggerbotpet-projectcolor-detection
  Language:C++