Computer Science


The Capstone Experience provides the educational capstone for all students majoring in computer science at Michigan State University. Teams of students build software projects for corporate clients. For information on becoming a project sponsor, see Project Sponsorship or contact Dr. Wayne Dyksen. The following were the project sponsors and projects for Spring 2018:


Accenture: AMAP: Automated Malware Analysis Platform

Accenture is a professional services company that solves their clients’ problems by providing services in strategy, consulting, digital, technology and operations. Accenture’s iDefense provides timely, relevant and actionable cyber threat intelligence to the largest organizations in the world.

Malware is software that is intended to damage or disable computers. Accenture iDefense maintains a large database with more than 260 million samples of malware. Before being added to this database, each new malware sample must be analyzed to determine what it does and how it works.

iDefense uses various software modules to analyze malware. However, the number of malware samples is growing so fast that it cannot be processed manually.

Our Automated Malware Analysis Platform (AMAP) is a web app that combines iDefense’s various malware analysis modules to process malware automatically at the speed and scale required.

For each batch of malware, users choose which iDefense malware analysis modules to apply. Our dashboard displays the status of malware currently being processed and the status of the overall system. Malware is also processed automatically by our system when users are not actively using it.

Our Automated Malware Analysis Platform runs as a web-based application with a backend written in Python. The underlying database holding malware samples and analysis output is mongoDB.

Team Members (L-R): Griffin Metevia, Sam Kling, Julian Ellis, Andrew Mitchell, Teng Xu


Amazon: AMPED: Amazon Marketplace Podcast Earnings Detection

Amazon is an e-commerce and cloud computing company based in Seattle. Accounting for 1 in 3 online shopping transactions in North America, Amazon is the world’s largest online retailer.

In the third quarter of 2017, Nielsen reported that 40% of Americans listened to a podcast. Despite having a large audience, podcast producers have few opportunities to profit from their work.

Our Amazon Marketplace Podcast Earnings Detection system (AMPED) provides podcast producers an additional source of revenue by finding products related to their content. The results are displayed on a dashboard for producers with an audio player integrated with the items.

For example, if a podcast producer is doing a tech review on the latest MacBook Pro, AMPED searches the Amazon marketplace for the latest MacBook Pro, and provides an image and link for the listener to buy it.

AMPED finds relevant products by first transcribing the podcast’s audio content to text. Next AMPED comprehends the meaning of the text. Finally, AMPED searches the Amazon marketplace for the most relevant products.

Products appear in the bottom of the app as the podcaster mentions them. Listeners can choose to purchase the products in Amazon’s app or website by clicking on the picture of the product.

AMPED is built with Python and is hosted on AWS Lambda serverless architecture. AMPED uses Amazon Transcribe for speech to text, Amazon Comprehend for natural language processing, and Product Advertising API to find products.

Team Members (L-R): Zach Whitener, Hansheng Zhao, Dillon Stock, Cam Korzecke, Chess luo


Aptiv: CMS: Cybersecurity Management System Experience

With over 147,000 employees in 45 countries around the world, Aptiv is a global technology company focused on creating the next generation of active safety, autonomous vehicles and smart cities.

Technology-focused mobility solutions introduce an increased need for implementing and testing product cybersecurity standards to ensure that Aptiv’s products are not susceptible to malicious attacks.

Aptiv’s cybersecurity lab conducts cybersecurity consulting and testing on every new product during the entire development process. The team works closely with product engineers to ensure the company’s products are up to the latest security standards and impenetrable from outside attacks.

Our Cybersecurity Management System (CMS) is a web app that automates the entire product cybersecurity process. This five-tiered process includes threat analysis, mitigation remediation, vulnerability assessment, penetration testing and incident response.

Maintaining all cybersecurity-related information for Aptiv products, our CMS includes database trackers to visualize important trends and analyze data for threats, vulnerabilities, risks, incidents and mitigations.

CMS provides the source of communication between product and cybersecurity engineers who utilize our app’s request system and task management service to increase productivity and streamline Aptiv’s entire cybersecurity process.

Our Cybersecurity Management System is an ASP.NET app written in CSS and HTML. Our backend database is implemented with MS SQL.

Team Members (L-R): Dillon Brown, Clayton Peters, Ashtaan Rapanos, Winton Qian, Wei Jiang


Auto-Owners Insurance: IMAGINE: IMAGe INtake Experience

Auto-Owners Insurance is a Fortune 500 company represented by over 4,900 associates nationwide. Founded in 1916, Auto-Owners Insurance provides its claims service to 2.7 million policyholders.

In order to appraise risks, Auto-Owners underwriters must evaluate physical environments such as homes or businesses with as much information as possible.

Our IMAGe INtake Experience system (IMAGINE) automates part of this process by identifying objects of interest in images taken of an environment and enabling Auto-Owners associates to recreate a physical environment at any time in virtual reality (VR).

Auto-Owners associates upload 360-degree images taken by a spherical camera to our web app, which analyzes the images to detect relevant objects such as a door or a piece of furniture. Associates can then display and edit information about the environments in those images.

Associates can create and edit names, descriptions and comments about environments and the objects in them. They can also download inventories of detected objects in various formats.

Our companion VR app creates a virtual version of the physical environment, which is viewed using an Oculus Rift headset. Associates use Oculus Touch controllers to select objects and to view and edit environment information such as item descriptions or comments.

IMAGINE is written in PHP and JavaScript and runs on all modern web browsers. Our virtual reality application is developed in Unity for the Oculus Rift platform and runs on Windows computers. Our object detection suite is written in Python using TensorFlow.

Team Members (L-R): Zack Geizer, Reece Cole, Sean Larabell, Nick Frederick, Xinyun Zhao


The Dow Chemical Company: Virtual Reality Simulation for Railcar Loading

With over 100 years of success and industry-leading innovation, Dow is a global leader in specialty chemicals, advanced materials and plastics. Along with Dow’s ongoing innovation and success is a commitment to its employees’ safety and wellbeing.

Dow manufactures hazardous chemicals that require care and knowledge when handling and transporting. Training new employees is dangerous because a mistake could have serious health implications if employees are exposed to these chemicals.

Our Virtual Reality Simulation for Railcar Loading eliminates this danger by training employees safely in a virtual environment. Our simulator trains employees in the proper way to load a railcar in a virtual railcar facility.

Dow employees wear an HTC Vive Headset and use two controllers to simulate a real working environment. Trainees are taught the steps needed to load a railcar properly and what to do when a spill occurs.

Our training simulator has two difficulty levels, non- certified and certified. The non-certified level gives a trainee hints to teach them the proper procedure. In the certified level, a trainee receives no aid, so they need to know what to do. Our simulator also includes scenarios where a spill occurs, so trainees can learn the proper way to clean a spill.

To help the employees learn, our training simulator gives the employees a score when they complete a session along with feedback on how to improve.

Our Virtual Reality Simulation for Railcar Loading uses Unity Game engine and is played on a Windows computer with the HTC Vive hardware.

Team Members (L-R): Kyle White, Matthew Smith, John Yoo, JD Hayward, David Wang


DRIVEN-4: 2020 Business in a Box

DRIVEN-4 is focused on driving competitive advantage for their customers by providing strategies, insights and proven implementations of integrated process and technology to deliver products and services to market.

Our 2020 Business in a Box system shows how a connected product platform can bridge the gap between product design and consumer use.

Our system comprises both manufacturer and consumer platforms to provide the insight needed to improve product design and consumer relationships while at the same time providing consumers with an enhanced experience for improving their daily lives.

Our web-based manufacturer portal provides for domain-specific configuration to create a dashboard for users in various job roles to visualize data and perform actions specific to their fulfillment of business and consumer needs. The built-in device simulator allows manufacturers to digitally simulate connected products before they are produced.

Our showcase of the platform demonstrates its use through the eyes of a mattress manufacturer and its consumers. When placed in the mattress, our connected device uses accelerometer and temperature sensors to transmit sleep cycle information to the platform.

The end-consumer experiences the platform through a mobile app that provides sleep analysis, suggestions to improve sleep quality and augmented reality visualizations for servicing their device.

Our web portal is written in CSS, HTML and ReactJS. Our device simulator is written in Java. Our mobile app is built in React Native for Apple iOS devices.

Team Members (L-R): Nick Mikelsavage, Brian Pinsky, Charles Carroll, Betty Wu, Sam Coffey


General Motors: Plato: DevBot for Microsoft Teams

Founded in 1908, General Motors designs and manufactures a wide variety of vehicles that meet the needs and expectations of drivers around the globe.

GM’s continued success requires its software developers to integrate an expanding number of software applications into their workflow. To do so, they use several unique applications to design, develop, test, and maintain software. Having to frequently maneuver between these applications hinders the development process.

To enable collaboration and teamwork, GM developers use Microsoft Teams, a chat-based workspace. Developers are members of a channel, which is specific to their team where they can chat, manage files, and access external applications.

Our DevBot for Microsoft Teams system (Plato) extends the functionality of Microsoft Teams through the creation of three separate but cohesive applications.

Our first Plato app is an artificially intelligent chatbot that enables team members to create and manage virtual machines and test cases. A graphical web dashboard embedded into Microsoft Teams in the form of a tab is our second extension. Our third app is a Microsoft Teams connector that enables the team to configure regular notifications about the system’s activity.

GM developers using our apps have full control over their resources and can monitor resource allocation, run tests and update the status of virtual machines all within a familiar Microsoft Teams environment.

Our Plato system is designed with the Microsoft Bot Framework, Microsoft LUIS, C# with .NET, AngularJS, HTML and CSS, and is deployed into Microsoft Teams.

Team Members (L-R): Alex Lepird, Tao Tao, Simeon Goolsby, Matthew Eaton, Colin Coppersmith


Herman Miller: Adjust: Augmented Reality Chair Adjustment

Headquartered in Zeeland, Michigan, Herman Miller is one of the world’s largest producers of high-end office furnishings. The company’s ergonomic office chairs are used in modern workspaces around the globe.

Every Herman Miller chair is highly configurable to match the unique needs of each user, ensuring proper ergonomic support in any office setting.

Adjust is our augmented reality (AR) chair adjustment app for owners of Herman Miller chairs.

Adjust automatically identifies the model of a chair using an iPhone’s camera along with computer vision and machine learning.

Once the chair model is known, Adjust displays an interactive 3D model and guides the user through an ergonomic adjustment session by highlighting the relevant chair parts.

Tapping on any highlighted part presents an instructional video and additional adjustment tips. For returning users, the application menu features a gallery option, which enables users to interact with a variety of Herman Miller chairs.

Our Adjust app reduces the need for in-person training sessions, allows for day-two training and eliminates the need for paper reference texts. Chair owners can locate any adjustment instantly using our built-in search functionality.

Adjust is a native iOS app written in Swift. Object recognition is performed using Microsoft Custom Vision and CoreML. Interactive 3D chairs are rendered using Apple’s SceneKit.

Team Members (L-R): Kevin Gaban, Mike Bremiller, Kyle Kinsey, Jacob Weber, Han Huang


Meijer: Thrifty: Personal Shopping Assistant

Meijer is a large supercenter chain located in the Midwest. With 237 supercenters in six states, Meijer is one of the largest retailers and private companies in the nation.

From shopping carts to self-checkouts, Meijer always leads on the frontline of innovation to ensure Meijer guest satisfaction.

Our Personal Shopping Assistant app (Thrifty) continues that tradition by enabling a guest to ask Thrifty anything they would ask an in-store Meijer team member. In addition, a guest can navigate through coupons and check competitor pricing.

Meijer guests interact with Thrifty using either text or voice. If Thrifty isn’t sure what they mean, it responds with a list of possibilities from which a guest can select.

After an item is selected, Thrifty tells a guest where it’s located in the store, how much it costs, and if there are any coupons available from Meijer’s rewards program, mPerks. Guests can clip all coupons to their mPerks account and display a single barcode at checkout to apply all of their savings.

Thrifty also informs guests of item prices at competitors like Target or Walmart.

If an item is missing from the shelf, Thrifty notifies a Meijer team member to check for more.

Thrifty is built using the Microsoft Bot Framework and is hosted on Microsoft Azure cloud services. It uses Microsoft LUIS for natural language processing and supports both Apple iOS and Google Android devices.

Team Members (L-R): Emerson Chan, Zach Richardson, Megan Lippert, Jacob Bonesteel, Aaron Carlson


Michigan State University: Pulse: Classroom Engagement System

Michigan State University (MSU) is the nation’s pioneer land-grant university and is a world- renowned research university, with top-tier academic and service-learning programs.

Our Classroom Engagement System (Pulse) provides a simple and convenient interface that enables extended engagement between course instructors and students. Pulse is designed around attendance, communication and analytics.

Attendance is made easy. Students simply respond to a notification that is sent to their mobile device as they enter a classroom. The student is then transported into the app to access the class material.

Pulse allows students and instructors to access multiple classroom resources on a compact and intuitive platform. Pulse offers a class overview page with course information, as well as a class forum where students and instructors can hold discussions.

Pulse’s quizzes replace paper quizzes, thereby saving paper and time. The quizzes can be taken during class or at any time. Progress bars and timers are available to enhance the user experience.

Analytics are collected and displayed by our instructor- side web app for evaluation in real-time. Based on student responses to questions, instructors can vary the content in their lectures.

Our Pulse mobile apps are native apps for Apple iOS and Google Android developed with Swift and Kotlin, respectively. Our web apps are created with Vue.js. The backend interface consists of several platforms from Amazon Web Services.

Team Members (L-R): Owen McMahon, Adam Balda, Lauren Malik, Yaqeen Al-Marhoon, David York, Andrew Schafer


Mozilla Corporation: Dark Theme Darkening

Mozilla is a global, not-for-profit organization dedicated to improving the World Wide Web. They have an international community of developers who contribute to open-source software.

Mozilla’s most popular open-source project is Firefox, the second most used web browser globally. Its most recent release, Quantum, includes new styles, features and ways for users to customize the appearance of the browser.

Users are reluctant to switch internet browsers once the browser has been personalized to their liking. Previously, themes only had the ability to alter the appearance of the default toolbars and tabs.

Our Dark Theme Darkening increases the customizability of Quantum to include menus, popups, settings pages, loading indicators, sidebars, icons and added theme transitions. Additionally, Google Chrome themes can now be imported directly into Firefox.

The default Dark Theme is updated to reflect the new additions. Extending theme support in the browser gives theme developers more power to express their creativity, and gives users more surface area on which to enjoy themes.

In the screenshots to the right, one can see the current un-themed sidebar and settings page in the background, while the foreground shows the same content themed.

Our Dark Theme Darkening is implemented using CSS and JavaScript, and impacts hundreds of millions of users.

Team Members (L-R): Vivek Dhingra, Conor Masini, Dyl Stokes, Bogdan Pozderca, Zhengyi Lian


MSU Federal Credit Union: Digital Assistant and Personal Financial Coach

Founded in 1937, Michigan State University Federal Credit Union (MSUFCU) is the largest university- based financial credit union in the world. With over 246,000 members around the globe, customer service is a high priority for MSUFCU.

Our Digital Assistant and Personal Financial Coach expands the capabilities of MSUFCU’s existing apps for Apple and Android devices. In addition, it offers services through a web app, an Amazon Alexa app, Facebook Messenger and iMessage.

Using a chatbot interface, MSUFCU members can get answers to questions to everything from “What’s my account balance?” to “Can I afford to buy a laptop?” Our bot also offers financial planning services. If a member cannot afford a laptop, for example, our Digital Assistant and Personal Financial Coach will suggest a plan for a member on how to save for one.

For MSUFCU members who prefer a hands-free approach to banking, our Alexa skill can respond to the same voice commands as the chatbot interface. Our Apple and Android apps, as well as Facebook Messenger and iMessage, all accept the same voice commands.

Our Digital Assistant and Personal Financial Coach Apple iOS app is written in Swift, and the Google Android app in Java. Our Alexa skill is written in JavaScript. The natural language processing is handled using DialogFlow. All six of the available apps contact a SQLite database hosted on a server.

Team Members (L-R): Patrick Dane, Dallas Nowak, Michael Carter


The Phoenix Group: Customer Service System with Chatbot

The Phoenix Group (TPG), an Ingram Micro company, is one of the largest distributors of credit card terminals. They offer customers a vast selection of devices from all major manufacturers and take pride in having outstanding customer service.

Customer service associates at TPG frequently reference paper device manuals to provide customers with helpful information. With hundreds of manuals to reference, TPG associates need a better way to manage and access them.

Our Customer Service System enables TPG to convert paper manuals to a digital format using optical character recognition (OCR) technology. An associate easily adds new manuals into the system by uploading images of its pages. Once added to our system, a specific manual can be found quickly by searching for it and viewed conveniently as an eBook.

Our Customer Service System enables TPG’s customer service associates to maintain their outstanding responsiveness despite a growing lineup of carried devices.

Additionally, our Customer Service Chatbot provides an informative resource for customers visiting TPG’s website. Customers can converse with the chatbot to ask about TPG and the devices it offers.

Our Customer Service System is a Windows WPF application written in C# that communicates with our custom remote server where the digital manuals are stored. Our Customer Service Chatbot is implemented using Google Dialogflow and is written in AngularJS.

Team Members (L-R): James Finch, Sarah Fillwock, Dan Shumaker, Fatema Alsaleh, Amanuel Goshu


Proofpoint: Next Generation Malware Analysis Platform 

Proofpoint is a leading next generation cybersecurity firm that provides comprehensive cloud-based security solutions to protect organizations from advanced threats and attacks that target email, mobile apps and social media.

Every day Proofpoint blocks threats in more than 600 million emails, 7 million mobile apps and hundreds of thousands of social media accounts. This volume of threats makes it increasingly more difficult to rely solely on human analysis.

Our Next Generation Malware Analysis Platform provides a comprehensive tool for analyzing malware samples automatically and quickly.

Our system first analyzes malware using a basic set of tools. Depending on the results of this initial analysis, our system determines whether or not more in-depth analysis should be done.

Malware is classified and identified using something called a signature. Our system clusters malware based on similarity, thereby enabling Proofpoint analysts to generate signatures more efficiently.

Analysts use our dashboard to visualize the results of malware analysis. Additionally, they can view malware of interest and apply filters.

Our Next Generation Malware Analysis Platform uses Bootstrap. A RESTful API based on NodeJS communicates with the MongoDB Database. Cuckoo, YARA, ClamAV and Suricata are utilized for the malware analysis tools. All of our tools run on virtual machines handled by an ESXi Hypervisor.

Team Members (L-R): Graham Thomas, Yash Patel, Crystal Lewis, Brad Doherty, George Zhao


QuickenLoans: Fundamenta: Trust in New Home Construction 

Quicken Loans is the largest online mortgage lender in the US, and is the nation’s largest FHA lender and premier Veteran Affairs lender.

New home construction is a complex process involving many parties including the home buyer, the builder, subcontractors and inspectors. Dependencies among the steps in the building workflow can cause bottlenecks. Lack of communication can further delay the process.

Our Trust in New Home Construction web app (Fundamenta) facilitates the home building process. Fundamenta is a platform for organization and collaboration among parties involved in a build.

Fundamenta displays a construction workflow comprised of tasks for all parties. It visualizes the workflow with progress bars that show who completes tasks and when, along with all construction deliverables.

In addition, Fundamenta provides a conversation page, which enables messaging between parties.

Blockchain technology manages the workflow, opening new tasks for completion when contingent steps are finished, and stores all data pertaining to a home build.

Our Fundamena app keeps all parties informed on the current state of a home build, thereby bringing transparency to the entire process.

Fundamenta is written in C# with React and the .NET framework. Hyperledger Fabric blockchain is used with the Composer framework. The database is implemented using Microsoft SQL Server.

Team Members (L-R): Vishal Adusumilli, Riley Annis, Erin O’Hara, Turner Anderson, Jaiwant Bhushan


Rook Security: Endpoint Data Monitoring and Analysis Agent

Rook Security, based out of Indianapolis, Indiana, is a leader in the managed detection and response service, providing IT security solutions to clients around the world.

Rook’s Force Platform protects their clients from malicious attacks by analyzing log data about client devices and appliances such as critical application servers. This log data is collected by Rook’s software agents running remotely on client host computers.

Rook’s diverse client base is growing quickly. In order to keep up with this growth, Rook’s log data agents must support various operating systems, and be easy to install and configure on client computers.

Our Endpoint Data Monitoring and Analysis Agent sends the digital defenders at Rook their clients’ system and application logs from client machines running Microsoft Windows, Apple MacOS or Linux operating systems. This enables client computers to continue working seamlessly and securely with our agent running in the background.

In addition to the agent, an extension of the Force Platform enables Rook analysts to deploy and configure agents as well as view agent health metrics. The ability of our agent to be managed remotely by Force Platform administrators ensures that any infringing anomalies are acted upon quickly.

Our Endpoint Data Monitoring and Analysis Agent system is written in Go for Windows, Mac, and Linux. The logs are stored using Amazon Simple Storage Service (S3). Our extension of the Force Platform is written in Python using the Django web framework. The front end utilizes ReactJS and Redux.

Team Members (L-R): Jeremy Specht, Bohao Gao, Vikram Thakur, Jared Clark, Drew Gilbertson


SpartanNash: Volunteer Tracking System

SpartanNash is a wholesale grocery distributor, supplying more than 2,100 independent grocery retail locations throughout the US, as well as a grocery retailer with more than 140 corporate supermarkets in 47 states.

Known as the local grocery store, SpartanNash values its close relationship with the local community. As part of their commitment to these local communities, SpartanNash sets yearly goals of volunteer hours for their associates.

Our Volunteer Tracking System provides a new automated way for SpartanNash to keep track of associates’ volunteer hours, replacing the previous manual “paper and pencil” system.

Our cross-platform app provides a simple and intuitive user experience. SpartanNash volunteers keep track of volunteer hours easily by using our web app. Social media integration and gamification increases overall participation.

When a SpartanNash associate logs into our app, the leaderboard is displayed showing their progress towards reaching their individual goal along with overall company progress. Associates can compare their progress with others.

SpartanNash supervisors manage the system using our companion administrative web portal.

Our Volunteer Tracking System is developed using React Native. Our administrative web portal is written with PHP and HTML. All of the data is stored in a MySQL database with Oauth2 security measures.

Team Members (L-R): Tianyi Li, Aleks Bonev, Abbott Wang, Denis Andreev, Nino Candela


 Spectrum Health: Spectrum Health Go

Spectrum Health is a not-for-profit health system based in Grand Rapids, Michigan that provides high quality healthcare at over 140 sites in West Michigan, including 12 hospitals, 8 urgent care centers and 43 laboratories.

Patients, visitors and volunteers often have trouble navigating the large Spectrum Health facilities. Our Spectrum Health Go mobile app helps these patrons find their destinations with ease.

Our Spectrum Health Go app employs the user’s smartphone camera and waypoints set up throughout the facility. These waypoints are posters that include the Spectrum Health Go icon, a QR code and the name of the waypoint.

After launching the app, a user chooses from a list of destinations and scans the nearest waypoint. Spectrum Health Go then displays an arrow that points to the next waypoint on their route along with a textual description of the directions. As the user turns, the arrow remains pointed towards the next waypoint.

Administrative assistants at Spectrum Health use our companion web app to view data about the routes, alter routes, and add or remove waypoints to improve navigation.

Our Spectrum Health Go mobile app supports both Apple iOS and Google Android devices and is written in Swift and Java, respectively. Our web app is written in C# and HTML. Data is stored using Microsoft SQL Server.

Team Members (L-R): Andrew Astakhov, Colin Skonieski, Rachel Polus, Pierce Neal, Xin Zhao


Symantec: Detecting Security Threats from User Authentications 

Symantec is a global leader in providing security and information management solutions. Their customers range from consumers and small businesses to large global organizations.

Symantec’s flagship authentication solutions include Symantec VIP, a multifactor authentication solution that enables businesses to secure access to networks and applications without affecting productivity.

Our system detects security threats by analyzing user authentication patterns and visualizing the results using Splunk and Elasticsearch, Logstash and Kibana (ELK).

Our custom Splunk and ELK apps enable VIP customers to view and analyze various operational and security trends in near real time. Both apps ingest and visualize VIP authentication log data pulled from Symantec servers using their VIP Reference Client.

A VIP customer initially configures their Splunk or ELK apps with their VIP account certificate. The apps open directly to pre-built dashboards, which show operational data and security trends. Users can add custom charts and panels to their dashboard.

While our dashboards do help to visualize data trends and behavior, security analysts may not always be viewing the dashboards. To prevent an analyst from missing an important security event, both the Splunk and ELK apps alert analysts via alternate methods when threats are detected.

Our Splunk app is written using the Splunk Enterprise software and dashboards are created with Splunk Processing language. Our ELK app runs on an Amazon Machine Image hosted on Amazon Web Services.

Team Members (L-R): Robert Novak, Abby Urbanski, Stephen Alfa, Keerthana Kolisetty, Xiaoyu Wu


TechSmith: Snagit and Camtasia Output Extensibility

TechSmith engineers engaging products which help people share ideas and information around the globe. With a focus on visual content, TechSmith software provides users with intuitive communication solutions.

TechSmith’s flagship products, Snagit and Camtasia, create image and video files for sharing with others. The TechSmith Extensibility Framework (TEF) provides support for the creation of specialized output plugins, which enable users to output their media directly to another product or service.

Imgur is a popular website for hosting images. TechSmith Video Review is a TechSmith service where users can have others review their videos. Wistia is a popular video hosting site used by businesses.

Our Snagit and Camtasia Output Extensibility suite of TEF outputs features three new options. These enable users to send their media directly from Snagit or Camtasia to Imgur, TechSmith Video Review and Wistia.

Users can specify several options specialized to these destinations, such as a review-by date for TechSmith Video Review.

Supplementing the addition of our plugins, our test harness is used to aid in output development and demonstration. The harness, much like Snagit and Camtasia, can dynamically load any TEF output plugin. Consequently, it serves as a tool for developers to help create any future outputs.

Our outputs are written in C# using abstraction provided by TEF. The user interface design for both the outputs and harness utilizes C# and XAML under WPF

Team Members (L-R): Ryan Schiller, Logan Arent, Collin Dillinger, Carter Chamberlain


Union Pacific: “Alexa, what’s my work schedule look like?”

Union Pacific is a leading transportation company headquartered in Omaha, Nebraska. Union Pacific has over 8,500 locomotives running on 32,100 miles of track across 23 states with almost 43,000 employees.

Union Pacific currently provides access to scheduling information for trainmen, yardmen and enginemen (TY&E) employees by phone or web app. Employees are on call 24/7 and are assigned trains to work on through a revolving list of employees. Train schedules are frequently updated, so employees must always stay up to date.

Our custom Google Assistant and Amazon Alexa voice assistants provide Union Pacific TY&E employees hands-free access to their place in the list as well as up-to-date train schedules that they can listen to individually or in groups. Voice assistants provide a fast and flexible solution to access essential scheduling information.

Our system also keeps a detailed log of every employee interaction with the assistants. These logs are viewable through our companion web app that enables managers to monitor the application’s usage statistics and to filter the logs to see individual assistant or user statistics.

Alexa uses the Amazon Developer Console and Google Assistant uses DialogFlow to map dialogs. Each voice assistant sends HTTPS requests to a NodeJS web URL, which calls a Java web service that accesses a MySQL database and returns a response. Our web app is implemented with HTML5, CSS and JavaScript.

Team Members (L-R): Jared McMillan, Daniel Agbay, Doyeon M Kim, David Hubble, Austin McGee


Urban Science: Mobile Maestro

Urban Science is a consulting firm that helps clients identify and improve market share, sales and profitability, and customer loyalty. They work closely with their sister company, Talem Technologies, to build medical devices that empower disabled people.

Talem Technologies has an exoskeleton arm, the X-Ar, that enables wheelchair bound people with Duchenne muscular dystrophy to regain control of their arms and experience a much larger range of motion.

The X-Ar is purely mechanical and must be calibrated to a stable angle of incline, which is adjusted using a push button key fob. Unfortunately, using this key fob presents a significant challenge for some users.

Our Mobile Maestro app enables users to control their X-Ar from their mobile devices, which are often mounted on their wheelchairs. Using touch button or voice controls, users can adjust the mount, lock and unlock the joints, enable auto leveling and voice controls, and put the app into standby mode.

Mobile Maestro also adjusts the incline calibration automatically when the user changes inclines and it provides user profiles to allow quick settings changes.

Our app supports both Apple iOS and Google Android devices. It is fully integrated with Siri and Google Assistant, allowing the user to access all touch button controls through them at any time. Furthermore, the user can add new voice commands to trigger any of the available actions.

Our Mobile Maestro app is written using Ionic for cross platform development and MySQL for data storage. The mobile devices communicate with the X-Ar using Bluetooth Low Energy.

Team Members (L-R): Mustafa Jebara, Dane Rosseter, Samantha Oldenburg, Alex Wuillaume, Shun Ran


USAA: LIMElight: Life Insurance Made Easy

USAA is a Fortune 500 company based in San Antonio, Texas. USAA provides a range of competitive financial products and services to the military community and their families.

Life insurance is one of USAA’s primary services. Applying for life insurance can be a time-consuming and tedious process for customers.

Our Life Insurance Made Easy app (LIMElight) delivers personalized quotes quickly and easily, thereby expediting the underwriting process and converting more life insurance applicants to enrolled USAA members. LIMElight accomplishes this by using machine learning and blockchain technology.

Applicants use a mobile-friendly web app to answer a few questions about their demographic and lifestyle. Our machine learning model then assesses the risk class of the applicant and quickly generates a personalized life insurance quote.

LIMElight also enables USAA and medical providers to exchange applicant medical records via blockchain technology.

Due to the nature of blockchain, information stored on the blockchain is secure, immutable and easily accessible by authorized parties. As a result, USAA underwriters have quicker access to the medical records that are critical in the life insurance underwriting process.

Our LIMElight quote generation web app is built utilizing React with a Python machine learning component. Our medical record interface uses Angular with a Hyperledger blockchain solution.

Team Members (L-R): Xingchi Zhou, Nathaniel Finley, Ibrahim Ahmed, Mike Ronayne, Dong-Hyun Lee


Yello: IVAT: Interview Video Analysis Tool

Yello is a Chicago-based company that provides software for talent acquisition. Their products help recruitment teams hire the right talent at the right time.

When evaluating job applicants, recruitment teams often have conflicting assessments due to differing opinions of applicants. This is challenging for recruiters when selecting candidates with conflicting reviews.

Automated emotion and sentiment analysis provides unbiased assessments to help evaluate candidates. Our Interview Video Analysis Tool (IVAT) automatically evaluates a candidate’s sentiment (speech attitude) and emotion (facial expressions) in a video interview.

IVAT allows recruitment teams to set up video interview questions, schedule live interviews, and invite candidates to an interview. A candidate can pre-record video responses to interview questions or participate in a live interview.

A candidate’s speech is processed and evaluated in terms of the sentiment of their words being negative, positive or neutral. Likewise, a candidate’s facial expressions are analyzed and summarized across a set of emotions such as angry, sad or joyful.

IVAT is built with Ruby on Rails. The video interviews are streamed and archived using TokBox’s OpenTok API. Sentiment analysis is done with Microsoft’s Video Indexer and Emotion analysis uses Kairos Human Analytics API. Videos are stored on Microsoft’s Azure cloud storage.

Team Members (L-R): Jonathon Zeitler, Leo Yao, Jenn Proos, Quan Zhou, Brian Wang