ReDraw: Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Team Members: Kevin Moran, Carlos Bernal-Cárdenas, Michael Curcio, Richard Bonett, & Denys Poshyvanyk

College of William & Mary --- SEMERU

Purpose

This project was created by the Software Engineering Maintenance and Evolution Research Unit (SEMERU) at the College of William & Mary, under the supervision of Dr. Denys Poshyvanyk.  The major goal of the ReDraw project is to leverage recent advances in machine learning and computer vision in order to automate the practice of mobile application prototyping.  

Video Demonstration

Publications

• Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
  ‣ Kevin Moran, Carlos Bernal-Cárdenas, Michael Curcio, Richard Bonett, and Denys Poshyvanyk
  ‣ IEEE Transactions on Software Engineering (TSE), Journal First Paper, accepted
  ‣ To be presented as a Journal First Paper at the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'18)
  ‣ [pdf]

We are working to make ReDraw Open Source!

The code for ReDraw will be coming in Winter 2020. We have re-written the original source to improve the tool and make it easier to use and reproduce results. We plan to publish the tool alongside a Tool Demo paper.

ReDraw

ReDraw Workflow Overview

Description coming soon...

Tools used for ReDraw Implementation

We provide the tools we used in our implementation of the GUI-Component Detection, Data Mining & Dynamic Analysis, CNN Classifier, and App Assembler.

Tools used to implement GUI-Component Detection:

• Coming Soon!

Tools used for Data mining and dynamic analysis:

• Coming Soon!

tools used to Implement the CNN classifier:

• Coming Soon!

tools used to Implement the Application Assembler:

• Coming Soon!

Using ReDraw

We are working to launch a live web-version of ReDraw. Check back soon for more details.

ReDraw Documentation

When we release the web-application we will also release documentation which will be available here.

ReDraw Study Results

Download the ReDraw Dataset

The initial ReDraw Screen and GUI-Component Dataset is now available for Download! We have permanently archived the ReDraw dataset on Zenodo to provide a DOI that will be accessible for the foreseeable future.