COMP 537 - Intelligent User Interfaces Term Project
Implementation of a UML class diagram sketching tool, which recognizes user sketches.
https://youtu.be/tXRMwprSzBM
Annotations provide information when an action is taken.
Sketching Mode
Default mode.
Deletion Mode
Draw a shape to cover the area where you want to erase.
Handwriting Mode
Enabled by a check gesture while in sketching mode. A second check gesture exits this mode.
Canvas Panning
Moving around the canvas is achieved by gestures:
Chevron Up
Chevron Down
Chevron Left
Chevron Right
Canvas Clearing
Clears everything.
Suggestions Tab
You can correct a misrecognition by clicking on a suggestion. Predicted probabilities decrease from left to right.
Image Export
Save canvas as image.
Sketching Scale
The bigger you sketch, the bigger is the recognized sketch.
There are three classes, each having 50 train and 10 test sketches at a resolution of 1440x2560 pixels.
Class
Implementation Arrow
Inheritance Arrow
HOG features extracted using
VLFeat in MATLAB.
Multi-class SVM with linear kernel trained using
LIBSVM in MATLAB.
Thread running in the background on the Windows Presentation Foundation (WPF) GUI written in C#.
When you complete a sketch, its class is predicted by sending data from C# to MATLAB.
Recognized sketches get replaced by predefined images.
Visual Studio 2015 Community Edition
MATLAB
LIBSVM for MATLAB
Windows 10
[1] C. Chang, C. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology. 2:27:1–27:27. 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm .
[2] A. Vedaldi, B. Fulkerson. VLFeat: An Open and Portable Library of Computer Vision Algorithms. 2008. http://www.vlfeat.org
[9] L. Qiu. SketchUML: The Design of a Sketch-based Tool for UML Class Diagrams. 2007.
[10] T. Hammond, R. Davis. Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. 2002.