This is a desktop automated testing framework based on accessibility api. At the same time, with the help of the open source framework of JNA, the purpose of calling the Mac and Windows system-level API is achieved.
<dependency>
<groupId>io.github.letmeno1</groupId>
<artifactId>aki</artifactId>
<version>1.1.5</version>
</dependency>
mvn install
Requires jdk version >= 1.8.0_295
For Mac
Requires a system running OS X and Xcode installed. Requires system version 10.6 and above
Systemwide accessibility must be enabled. Check the checkbox: System Preferences > Security and privacy
Universal Access > Enable access for assistive devices.
For Windows Requires system version >= Windows 7
PS: If you need to use image recognition positioning Download opencv_java451.dll and move it to your jdk path（C:\Program Files\Java\jdk1.8.0_202\bin）
For Mac
Accessibility Inspector：Xcode -> Open Developer Tools
Using Accessibility Inspector can provide a quick way to find these attributes.
For Windows
download inspect.exe
Using inspect.exe can provide a quick way to find these attributes.
For Mac： UIElementRef app = Operation.initializeAppRefForMac("com.apple.calculator");
For Windows： UIElementRef app = Operation.initializeAppRefForWin("C:\\WINDOWS\\System32\\calc.exe");
App window itself is an UIElementRef object, And every elements are an UIElementRef object yet. You can call the find or click method through UIElementRef
Mouse event：
UIElementRef object support click, double click, long click, hover
e.g. app.findElementsByText("Input phone number",0).click() or app.findElementsByText("Input phone number",0).doubleClick()
Input Event：
UIElementRef object support type and clear
e.g. app.findElementsByText("Input phone number",0).type("188888") or app.findElementsByText("Input phone number",0).clear()
Launch Option：
Before starting the App, you can set the following startup parameters,
LaunchOption launchOption = new LaunchOption();
//Set whether it is a UWP application, only for Window
launchOption.setIsUWPApp(true);
//Set the startup timeout, the default is 20000ms
launchOption.setLaunchTimeoutTimeout(30000);
//Set the search timeout, the default is 20000ms
launchOption.setDefaultTimeout(30000);
//Set whether the App has been started. If it is TRUE, the process of starting the creation process will not be started.
lauchOpthion.setAlreadyLaunch(false)
Pass startup parameters as objects
app = Operation.initializeAppRefForWin("C:\\WINDOWS\\System32\\calc.exe",launchOption);
For Mac：
support text, role(which is element type), identifier, xpath.
For text and roles, the searched elements will be returned in the form of a list, and the index needs to be notified when calling.
e.g. `app.findElementsByText("Input phone number",0)` & `app.findElementsByRole("AXTextField",2)`
1.By text
The text corresponds to the "Title", "Value" and "Help" in the element attribute
2.By Role
The role corresponds to the "Role"
3.By Xpath
This is a way to find through the path of the element
e.g. `app.findElementByXpath("AXStandardWindow/AXButton[1]")`
4.By identifier
The identifier corresponds to the "identifier"
e.g. `app.findElementByIdentifier("JoinButton")`
For Windows：
support text(name), role, automationId, fullDescription.
For text, role and fullDescription, the searched elements will be returned in the form of a list, and the index needs to be notified when calling.
e.g. `app.findElementsByText("Input phone number",0)` & `app.findElementsByRole("AXTextField",2)`
1.By text
The text corresponds to the "Name" in the element attribute
2.By Role
The role corresponds to the "Role"
e.g. `app.findElementsByRole(app.findElementsByText("client",0))
4.By automationId
The automationId corresponds to the "automationId"
e.g. `app.findElementByAutomationId("num3button")`
For Common：
Support image recognition and positioning
No requirement for image resolution size
With the help of LOF outlier algorithm to remove impurities, the accuracy of identification is greatly improved.
1. Single object recognition, this method can be used when the element on the page is unique
`String imageFolderPath = "image path"`
`app.findElementByImage(imageFolderPath + "001.png")`
2. For multi-object recognition, you need to provide the number of objects to be recognized on the target page. A collection of elements will be returned, and the elements you need are indexed by index.
`int k = 3; //Number of images to be recognized`
`int index = 1 //The index of the element you want to get`
`app.findElementsByImage(imageFolderPath + "001.png",k, index)`
3. The above feature recognition algorithm, the default precision is 4f (this value is 0 value. The range value is 1~1, the value is all value 1)
The value is passed through the ratio ThreshValue
`app.findElementByImage(imageFolderPath, ratioThreshValue)` or `app.findElementsByImage(imageFolderPath,ratioThreshValue,cluster, index)`
Support Mouse events, Combination keyboard events, take screenshot and kill app
Aki是一个基于Accessibility Api实现的跨平台(Mac/Windows）桌面端自动化测试框架，借助开源框架JNA实现了对系统底层Api的访问。
<dependency>
<groupId>io.github.letmeno1</groupId>
<artifactId>aki</artifactId>
<version>1.1.1</version>
</dependency>
mvn install
需要 jdk version >= 1.8.0_295
对于 Mac
需要安装Xcode 需要系统版本高于10.6
需要在在隐私权限中开启Xcode对辅助设备访问的权限。系统偏好设置 > 安全性与隐私 > 通用访问 > 启用辅助设备访问。
对于 Windows
需要版本高于Windows 7
PS:如需使用图像识别定位 请下载opencv_java451.dll，并放在你jdk安装目录下（C:\Program Files\Java\jdk1.8.0_202\bin
对于 Mac
Accessibility Inspector：Xcode -> 打开 Developer Tools
使用“Accessibility Inspector”可以查找到App对应的元素属性
对于 Windows
下载 inspect.exe
使用“inspect.exe”可以查找到App对应的元素属性
对于 Mac： UIElementRef app = Operation.initializeAppRefForMac("com.apple.calculator");
对于 Windows： UIElementRef app = Operation.initializeAppRefForWin("C:\\WINDOWS\\System32\\calc.exe");
App 窗口本身就是一个 UIElementRef 对象，而每个元素也都是一个 UIElementRef 对象。你可以通过UIElementRef来调用各种查找或者是点击的方法
鼠标事件：
UIElementRef 对象支持单击、双击、长按、悬停
例如: app.findElementsByText("Input phone number",0).click() 或 app.findElementsByText("Input phone number",0).doubleClick()
输入事件：
UIElementRef 对象支持输入和清除
例如: app.findElementsByText("Input phone number",0).type("188888") 或 app.findElementsByText("Input phone number",0).clear()
启动参数：
在启动App前，可设置以下启动参数，
LaunchOption launchOption = new LaunchOption();
//设置是否是UWP应用，仅限于Window
launchOption.setIsUWPApp(true);
//设置启动超时时间，默认是20000ms
launchOption.setLaunchTimeoutTimeout(30000);
//设置查找超时时间，默认是20000ms
launchOption.setDefaultTimeout(30000);
//设置App是否已启动，若为TRUE，则不走启创建进程启动的流程
lauchOpthion.setAlreadyLaunch(false)
将启动参数以对象的方式传递
app = Operation.initializeAppRefForWin("C:\\WINDOWS\\System32\\calc.exe",launchOption);
对于 Mac：
支持文本、role（元素类型）、identifier、xpath。
对于文本和role，搜索到的元素会以列表的形式返回，调用时需要加入索引。
例如 `app.findElementsByText("Input phone number",0)` 和 `app.findElementsByRole("AXTextField",2)`
1.通过文本
文本对应元素属性中的“Title”、“Value”和“Help”
2.通过Role
Role对应元素属性中的“Role”
3.通过Xpath
这是一种通过元素的路径查找的方法(待改进)
例如 `app.findElementByXpath("AXStandardWindow/AXButton[1]")`
4.通过identifier
identifier对应元素属性中的“identifier”
例如 `app.findElementByIdentifier("JoinButton")`
对于 Windows：
支持 text(name)、role、automationId、fullDescription。
对于text、role和fullDescription，搜索到的元素会以列表的形式返回，调用时需要加入索引。
例如`app.findElementsByText("Input phone number",0)` 和 `app.findElementsByRole("AXTextField",2)`
1.通过文本
文本对应于元素属性中的“Name”
2.通过Role
Role对应于元素属性中的“Role”
例如 `app.findElementsByRole(app.findElementsByText("client",0))
4.通过automationId
automationId对应于元素属性中的"automationId"
例如 `app.findElementByAutomationId ("num3button")`
通用定位方式：
支持 图像识别定位
对图像分辨率大小无要求
借助了LOF离群算法进行了除杂极大提高了识别的准确性。
1.单一对象识别，当页面上元素唯一时可使用该方法
`String imageFolderPath = "图像路径"`
`app.findElementByImage(imageFolderPath)`
2.多对象识别，需提供目标页面上待识别对象个数，将返回一个元素集合，通过index来索引你需要的元素
`String imageFolderPath = "图像路径"`
`int cluster = 3; //待识别的图像个数`
`int index = 1 //想要获取的元素索引`
`app.findElementsByImage(imageFolderPath,cluster, index)`
3.以上定位都采用特征值算法进行图像识别,默认精度对0.4f(此数值为浮点型, 范围值为0.1~1，值越大识别精度越低,值越小识别精度越高)
该值通过通过ratioThreshValue来传递
`app.findElementByImage(imageFolderPath, ratioThreshValue)` or `app.findElementsByImage(imageFolderPath,ratioThreshValue,cluster, index)`
`
支持鼠标事件、组合键盘事件、截图和结束进程