Building Vision Apps Using Sikuli

SikuliX is a very unusual technology - a scripting/automation technology that relies on pattern matching, and is available for use via Python or Java. Developed at the User Interface Design Group at MIT, is a powerful and easy-to-use technology that uses image recognition to automate just about anything that appears on-screen.

Sikuli is rather hard to slot as a technology - it offers all of the functionality of an automation or scripting tool, but it also offers some powerful and very novel image-matching functionality for truly novel use-cases that revolve around image search. In addition it has an OCR-mode, in which image matches are performed after converting those image patterns to text. This gives rise to some pretty new applications.

The OCR-functionality is powered by Tesseract, an open-source optical character recognition engine whose development is sponsored by Google.