eeyrdkl / COMP537

Intelligent User Interfaces Term Project

COMP 537 - Intelligent User Interfaces Term Project

Implementation of a UML class diagram sketching tool, which recognizes user sketches.

Index

• Demonstration Video
• Functionalities
• Dataset
• Implementation
• Requirements
• References

Demonstration Video

https://youtu.be/tXRMwprSzBM

Annotations provide information when an action is taken.

Functionalities

• Sketching Mode

  Default mode.

• Deletion Mode

  Draw a shape to cover the area where you want to erase.

• Handwriting Mode

  Enabled by a check gesture while in sketching mode. A second check gesture exits this mode.

• Canvas Panning

  Moving around the canvas is achieved by gestures:

  Chevron Up

  Chevron Down

  Chevron Left

  Chevron Right

• Canvas Clearing

  Clears everything.

• Suggestions Tab

  You can correct a misrecognition by clicking on a suggestion. Predicted probabilities decrease from left to right.

• Image Export

  Save canvas as image.

• Sketching Scale

  The bigger you sketch, the bigger is the recognized sketch.

Dataset

There are three classes, each having 50 train and 10 test sketches at a resolution of 1440x2560 pixels.

1. Class

   

2. Implementation Arrow

   

3. Inheritance Arrow

   

Implementation

HOG features extracted using VLFeat in MATLAB.

Multi-class SVM with linear kernel trained using LIBSVM in MATLAB.

Thread running in the background on the Windows Presentation Foundation (WPF) GUI written in C#.

When you complete a sketch, its class is predicted by sending data from C# to MATLAB.

Recognized sketches get replaced by predefined images.

Requirements

• Visual Studio 2015 Community Edition
• MATLAB
• LIBSVM for MATLAB
• Windows 10

References

[1]	C. Chang, C. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology. 2:27:1–27:27. 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

[2]	A. Vedaldi, B. Fulkerson. VLFeat: An Open and Portable Library of Computer Vision Algorithms. 2008. http://www.vlfeat.org

[3]	Microsoft Visual Studio Image Library. https://www.microsoft.com/en-us/download/details.aspx?id=35825

[4]	Microsoft Check Gesture Icon. https://i-msdn.sec.s-msft.com/dynimg/IC5936.gif

[5]	Microsoft Chevron Up Gesture Icon. https://i-msdn.sec.s-msft.com/dynimg/IC100787.gif

[6]	Microsoft Chevron Down Gesture Icon. https://i-msdn.sec.s-msft.com/dynimg/IC109874.gif

[7]	Microsoft Chevron Left Gesture Icon. https://i-msdn.sec.s-msft.com/dynimg/IC137959.gif

[8]	Microsoft Chevron Right Gesture Icon. https://i-msdn.sec.s-msft.com/dynimg/IC40744.gif

[9]	L. Qiu. SketchUML: The Design of a Sketch-based Tool for UML Class Diagrams. 2007.

[10]	T. Hammond, R. Davis. Tahuti: A Geometrical Sketch Recognition System for UML Class Diagrams. 2002.