Author:

Bradlee Speice <bspeice@uncc.edu>

Introduction

Welcome to Libcvautomation! Libcvautomation is a GUI automation and testing tool based on image recognition and response. This program was designed as a direct replacement for Sikuli and Xpresser. I was having incredible difficulty getting either of these solutions to work - Sikuli would crash whenever I tried to take a screenshot, and Xpresser was both too new for our Prominent North American Enterprise Linux systems, but also didn't work or import correctly. I really liked the way each of these programs approached GUI automation, but they simply didn't work. Additionally, I wanted to create a simple solution - it does what you want it to, and that's it.

How Libcvautomation Works

Libcvautomation represents two software products coming together - OpenCV and the XTest extension to the X11 server. OpenCV is used for image recognition, and XTest is used to actually drive the X server. You can dig into libcvautomation-xtest.h to get an idea of what all this library is capable of.
Basically what happens is that for whenever you need to do image recognition, OpenCV is used to find the images, and XTest is used to generate any events needed. Libcvautomation is mostly a wrapper to integrate both of these products, but also adds some functions like matchSubImage_X11() that allow you to match an image against the X11 root window in place. This means no more 'xwd | convert "<out_name>"'.

Using Libcvautomation And Writing Application Tests

So how does one go about using libcvautomation?
I'm so glad you asked! I've provided a few reference programs - cva-match and cva-input - that can be used to demonstrate most of libcvautomation's capabilities. I've even provided a BASH wrapper to make it incredibly easy to use BASH with libcvautomation as well (requires that cva-match and cva-input are installed). Python bindings are even included too!
Finally, if you want to know how to write your own application tests, please see Writing Application Tests for more information on that. I've provided code to give you a basic idea of how they work.

Questions? Comments? Concerns?

Please send any feedback to <bspeice@uncc.edu>. Pull requests can be submitted to my github repository.