The Capstone Experience provides the educational capstone for all students majoring in computer science at Michigan State University. Teams of students build software projects for corporate clients. For information on becoming a project sponsor, see Project Sponsorship or contact Dr. Wayne Dyksen. The following were the project sponsors and projects for Fall 2018:

Amazon: AVAST: Amazon Video and Shopping Technology

Founded in 1994 as an online bookstore, Amazon is the largest online retailer in the world. In addition to retail, Amazon offers services in cloud infrastructure through Amazon Web Services, and audio and video streaming through Amazon Music and Prime Video.

According to a recent study, 80% of internet usage will be people watching online videos by the year 2020. This presents a significant opportunity for all online retailers.

Our AVAST (Amazon Video And Shopping Technology) platform leverages the growth in online video streaming by providing users with an easy way to purchase products of interest that they see in the videos they are watching.

Using AVAST, an Amazon customer can stream videos from content providers such as YouTube and their favorite TV networks.

While a user is watching a video, AVAST analyzes it to find items of potential interest to the viewer. As the video plays, related Amazon products are displayed alongside the video as illustrated in the examples at the right.

For each item, AVAST displays a product description, pictures and ratings. A viewer can easily purchase any product simply by clicking on the conveniently provided link to Amazon.

The frontend of AVAST (Amazon Video And Shopping Technology) is built using Angular 6, while the backend is implemented using PHP Laravel. In addition, several Amazon Web Services are used including Rekognition to analyze videos, and EC2 to host the AVAST website.

Aptiv: Autonomous Vehicle Fleet Connectivity App

Aptiv is a global technology company that is transforming mobility with its portfolio of safe, green, and connected solutions for its customers.

As a leader in autonomous vehicle development, Aptiv maintains an extensive test fleet of autonomous vehicles, which must be managed and monitored.

Our Autonomous Vehicle Fleet Connectivity App provides connectivity to Aptiv’s autonomous test fleet, which operates across the US, Europe and Asia, and includes various vehicles with software for every level of autonomy.

Among other features, our system provides scheduling of test vehicles. After logging in, Aptiv engineers see a calendar view of the entire fleet from which they can select a particular day to obtain a list of available vehicles.

Once a vehicle is selected, our app displays a complete set of information about it including its past usage, reservations and diagnostic information.

In addition to checking availability of vehicles based on dates, our app provides for advanced search to narrow the scope based on things like type of vehicle, location of vehicle and level of autonomy.

The “My Reservations” tab shows a user’s upcoming vehicle reservations as well as enabling them to make and cancel reservations.

Our Autonomous Vehicle Fleet Connectivity App is written using the Angular web framework, obtaining information from Aptiv’s native servers. Communications are implemented using Microsoft Azure Services.

Auto-Owners Insurance: Jeffrey: Virtual Insurance Claim Advisor

Auto-Owners Insurance is a Fortune 500 company that provides automotive, home, life and commercial insurance. Headquartered in Lansing, Michigan, Auto- Owners is represented by over 44,000 licensed insurance agents across 26 states, and provides insurance to nearly 3 million policyholders.

Every day, hundreds of insurance claims are filed with Auto-Owners through its independent agents. This process can be tedious for both policyholders and agents.

Our Jeffrey Virtual Insurance Claim Advisor system is a virtual claim assistant that automates the entire claim reporting process. Our mobile app, shown at the right, enables both agents and policyholders to file a claim easily and efficiently.

Jeffrey engages in a dialogue with policyholders and agents to gather information required to file their claim through natural conversation. If necessary, Jeffrey prompts users to take photos, record videos or attach documents relevant to a claim.

After completing a dialogue with a user, Jeffrey automatically gathers the appropriate claim information and submits it to Auto-Owners.

Our companion web app enables agents and Auto- Owners associates to find and review claim information that is submitted through the mobile application.

Our Jeffrey Virtual Insurance Claim Advisor system features natural language processing, which is implemented using Google’s Dialogflow. A custom REST API, written in Kotlin, handles interactions between the applications and our MySQL database. Our web application is built using the React JavaScript framework.

Dow Chemical Company: IT Assistant

With over a century of experience, Dow Chemical Company is changing the world through innovation by providing advancements like more drinkable water, more clean and affordable energy, and increasing food production.

Dow employs over 70,000 people worldwide, including some 30,000 of which are contractors. For many of them, information technology (IT) is central to their work. Providing IT support is crucial, but to do so for so many people in so many locations is a challenge.

Our IT Assistant is a chatbot that brings all of Dow’s IT knowledge to one place, providing a one-stop shop for resolving IT issues.

Our chatbot leverages natural language processing to engage with a Dow employee in a natural and intuitive way, handling both text and voice input.

When a user describes their IT problem, IT Assistant either provides a solution by searching Dow’s vast knowledge base of issues and solutions or it asks the user for more information.

IT Assistant is a responsive web app so it can be used with any web browser on a desktop or on any mobile device. And, since it’s web-based, it provides IT support at any time, from anywhere.

Our IT Assistant uses a variety of Microsoft Azure Cloud Services including LUIS and Voice Services. Our chatbot leverages Dow’s extensive IT knowledge base of issues and solutions. Hosted on Azure, IT Assistant is implemented using React.js and C#.

Ford Motor Company: Ford Customer App Review Dashboard

Founded in 1903, Ford Motor Company is the second- largest automaker in the U.S. and fifth-largest in the world, employing 202,000 people worldwide and selling 6.6 million vehicles in 2017.

Currently, Ford offers several mobile apps for connected vehicles, allowing users to take advantage of features such as remote start, parking search, media streaming and more.

With their passion for customer satisfaction, Ford associates continually monitor user feedback for their apps to ensure that they meet the needs of their customers.

Our Customer App Review Dashboard streamlines this process by providing a convenient and automated system to analyze, summarize and display data from many app reviews all at a single glance.

Reviews are gathered from multiple app stores and analyzed using Natural Language Processing (NLP) to determine whether the sentiment of each review is positive, negative or neutral. Our web app visualizes the results of this analysis, enabling Ford associates to see the distribution of positive versus negative reviews at a glance.

Updates to our review analysis are sent through the Slack messaging service, which gives a real-time feed of user feedback. Ford associates use our companion web app to adjust dashboard settings.

Our Customer App Review Dashboard’s frontend is written in Angular 6. Our RESTful API is written in Java using Spring Cloud Function. Our NLP is implemented using Amazon Comprehend. The backend processing is serverless, performed using AWS Lambda.

Herman Miller: FIBRE: Fabric Identification Based Recommendation Engine

Herman Miller, a 100-year-old-plus company, is an industry leader in office and home furniture, which are sold and used in countries all over the world.

Herman Miller furniture is highly customizable, with thousands of combinations for each piece including choices for the color and pattern of the fabric. In addition, customers can not only choose from a catalog of Herman Miller fabrics, they can also request a fabric of their own, making an order even more complex.

When a custom fabric is requested, the process of manually searching for a similar fabric in Herman Miller’s existing catalog of fabrics is tedious, error prone and very time-consuming.

Our Fabric Identification Based Recommendation Engine, FIBRE, leverages computer vision and machine learning to classify fabrics, automatically detecting color and pattern.

FIBRE is first applied to Herman Miller’s existing extensive catalog of fabrics, to tag images with standard, quantifiable measures of both color and pattern.

When a request for a custom fabric is submitted, our system analyzes it, generates its measures of color and pattern, searches the extensive Herman Miller catalog of fabrics, and finds fabrics that most closely match. The four most similar fabrics are displayed for review by a Herman Miller associate or customer, thereby reducing the number of fabrics to consider from thousands to just four.

Our FIBRE utilizes scikit-learn for color detection, and SageMaker and TensorFlow for pattern detection. Flask provides the client side interface to the backend, which is hosted on Amazon Web Services.

Meijer: Shrink Reduction Using Blockchain

Founded in 1934, Meijer is the pioneer of the modern supercenter with 242 stores located throughout the Midwest.

As a large retailer, Meijer ships and tracks thousands of products from supplier to store shelf. Timely and safe arrival of products is key for customer satisfaction and business efficiency.

All retailers experience product loss, often from theft or from product expiration for things like produce and dairy products. This product loss is known as “shrink.”

Our Shrink Reduction Using Blockchain system reduces shrink resulting from product expiration by tracking products along the entire supply chain from supplier to store.

Meijer team members use our web app to record a product’s expiration date when it is received from a supplier. Team members update product information by scanning QR codes each time a product arrives at a new location, which is updated in our blockchain database.

Meijer team members use our shrink reduction alerts to decide which products should be put on the shelf based on their expiration dates. As a result, more products are sold before their expiration dates, thereby reducing shrink.

Our companion web app shows loss statistics on specific products such as product location and estimated cost, which is used to prevent future loss.

Our Shrink Reduction Using Blockchain system uses the Microsoft Azure Framework with Ethereum Private blockchain technology. Our iOS app is built using XCode. Our web app uses HTML, CSS, JavaScript and ASP.NET.

Michigan State University: Navigating MSU’s Campus Using Augmented Reality

Founded in 1855, Michigan State University is the nation’s pioneer land-grant university. Today, MSU is one of the largest universities in the US with over 50,000 students and holds many distinctions, including being one of the top 70 universities in the world.

Each year, thousands of people come to MSU for the first time, including some 8,000 freshmen. MSU’s campus is comprised of hundreds of buildings spread across its large 5,200-acre sprawling campus. Navigating this large campus is a challenge for all newcomers who often rely on paper maps and hard-to-find signs.

Our Navigating MSU’s Campus Using Augmented Reality app leverages augmented reality (AR) to guide people around MSU’s large campus with ease.

Our app, shown at the right, identifies and labels buildings and landmarks simply by using the camera of any supported mobile device. Some landmarks, such as the Spartan Statue and the Rock, have clickable names where a user can learn fun facts about them.

A user can easily navigate MSU’s campus with a 3D AR experience called wayfinding, which gives directions to any destination on campus.

After selecting a destination, directional arrows, as seen at the right, are overlaid on a user’s mobile device showing the way to go. Users simply follow the arrows to find their way to their chosen destination.

Our Navigating MSU’s Campus Using Augmented Reality app supports both Google Android and Apple iOS, which use ARCore and ARKit SDKs respectively. Our app obtains building information and wayfinding directions from MSU’s ArcGIS API.

Microsoft: ITPro Company Portal

Microsoft is a long-time leader in the technology industry, providing enterprises with a comprehensive suite of software solutions created to drive productivity.

More and more, people are using their personal mobile devices to do company work, both inside and outside of the office. This enhances work flexibility and productivity, and also lowers company hardware costs.

However, this new work paradigm presents significant security and reliability risks since personal devices are now connecting to corporate networks and often storing confidential corporate data.

Our ITPro Company Portal is a system that enables information technology (IT) administrators to ensure that all company employees’ personal mobile devices are both secure and reliable.

Before using their personal mobile device for work, an employee installs a simple app, which allows an IT professional to check that it is indeed secure and reliable. ITPro does so with very limited access, which maintains an employee’s personal privacy and complete control over their device’s non-work information.

With ITPro, IT administrators can be confident that all mobile devices company-wide are compliant with corporate security and reliability policies. And, since ITPro itself is a mobile cross-platform app that supports Google Android, Apple iOS and Microsoft Windows devices, administrators can do so using any device, from anywhere in the world, at any time.

Our ITPro Company Portal app is written in C# using the Xamarin framework within Microsoft Visual Studio. It communicates with Intune via the Microsoft Graph API.

Mozilla Corporation: Asynchronize All the (Localization) Things!

Since its establishment in 1998, Mozilla’s mission has been to build a better, more open and accessible internet. As a part of this mission, Mozilla’s Firefox web browser supports 99 languages worldwide.

Currently, non-English users must download a separate version of Firefox or perform a series of complex configuration steps and then restart the browser to use their preferred language. Such customization of software is referred to as “localization.”

Mozilla’s new framework, called Fluent, addresses these issues and improves the localization experience for users, translators and developers. Fluent allows users to change their locale on the fly, and helps localizers and developers by simplifying the work required to convert phrases from English to other languages.

Firefox contains many pages, which enable users to localize Firefox for things like menus and error messages. For each page, the English phrases are now consolidated into new Fluent files, and the Firefox frontend code is updated. These phrases take several forms, from static content to dynamic expressions where the phrasing varies depending on language and context.

To save work for volunteer localizers, our new Python scripts are used to migrate phrases from old files to new Fluent files in every language that is currently supported.

After thoroughly testing our system, our localization patches are now integrated into the Firefox codebase for distribution to hundreds of millions of users worldwide.

Our Asynchronize All the (Localization) Things project is implemented using Fluent, JavaScript, Python, XHTML, XML and XUL.

MSU Federal Credit Union: Transaction Anomaly Detection

Founded in 1937, Michigan State University Federal Credit Union offers financial services to students, faculty and staff of Michigan State University. With over $4.1 billion in assets and 265,000 members, MSUFCU is the largest university-based credit union in the world.

In the past year, U.S. consumers experienced over $8 billion in losses due to payment card fraud. MSUFCU protects its members from fraud by monitoring almost 40 million transactions of its members annually.

Our Transaction Anomaly Detection system leverages the MSUFCU’s existing software platforms to notify its members of unusual account activity.

When anomalous transactions are detected, our system sends push notifications to MSUFCU members and flags the transactions on our companion web app used by MSUFCU associates.

Our system identifies irregularities ranging in severity from a peculiar increase in a monthly cable bill to a suspected account takeover.

In addition to detecting suspicious account activity, our system harnesses a member’s transaction data to provide them with insights into their spending habits.

Within the MSUFCU member web portal, transactions flagged as suspicious can be expanded for more information. Spending habits are summarized by personally tailored reports, which categorize purchases and visualize patterns.

Our Transaction Anomaly Detection iOS, Android and web apps connect to a Django server via a RESTful API. Anomalies are detected using a machine learning TensorFlow module and other scientific Python packages.

Proofpoint: Improved Detonation of Evasive Malware

Headquartered in Sunnyvale, California, Proofpoint provides cybersecurity to many organizations, including Fortune 100 companies and educational institutions such as Michigan State University.

Analyzing malware is challenging. Viruses, spyware, ransomware and other malicious programs come in many complex forms. To protect its customers, Proofpoint uses tools called sandboxes, which are restricted computing environments where potentially harmful malware can be tested and analyzed safely.

Unfortunately, a new class of malware called “evasive malware” is rapidly emerging, thereby presenting a new, more dangerous class of cybersecurity threats.

Evasive malware has the ability to detect the presence of the sandbox environment. After doing so, it changes what it does, thereby evading analysis.

Our Improved Detonation of Evasive Malware system modifies evasive malware to block its ability to detect the sandbox environment, which causes it to execute. When the evasive malware does execute, its behavior is analyzed to determine precisely what it does so that Proofpoint can design countermeasures to protect against it.

Our web app, shown at the right, displays the results of processed malware. Users can check the status of the malware samples being tested as well as see the top evasive techniques being used. Both harmless and harmful evasive results are presented.

Our Improved Detonation of Evasive Malware system is implemented in Python, using the Cuckoo sandboxing framework and Suricata network monitor. Our web app is implemented using Python and Flask with the interface framed in Bootstrap and jQuery.

Quicken Loans: Walter, You Gotta Go

Quicken Loans is the largest online mortgage lender in the US, the nation’s largest FHA lender, and the premier Veteran Affairs lender. Legacy software systems can be tedious and inefficient to use, sometimes requiring employees to spend valuable time manually inputting data from documents. Unfortunately, replacing legacy software systems with modern ones is complex and costly.

Our Walter, You Gotta Go system provides a modern software system that accomplishes the same task as the legacy system without replacing it.

Our system leverages Robotic Process Automation (RPA) to create a specialized program called a “virtual worker,” which watches how a user processes documents in the legacy software and then learns how to replicate it.

Having learned the process, the virtual worker then automates the data-entry process by extracting information from documents and submitting the data to the legacy system directly, saving time and reducing errors.

In addition to our virtual worker, our system includes a web app used to view and edit previously entered data. This web app provides all of the functionality of the legacy system as well as some additional features, all with a modern user experience.

In the spirit of Quicken Loans, each part of our system is anthropomorphized with its own persona. Walter is the tired legacy system that is replaced by Lindsey who is the new cutting-edge princess of the mortgage world.

Our virtual worker is implemented with an RPA tool using UiPath to extract data. Our web app is written with the Angular 6 framework and Quicken Loans’ Spark Design Systems. Our backend is powered with a GraphQL API connected to a Microsoft SQL Server database.

Spectrum Health: Spectrum Health Virtual Reality Experience

Based in Grand Rapids, Michigan, Spectrum Health is a not-for-profit integrated health-care system including 12 hospitals, 8 urgent care facilities and 48 lab centers, making them the largest employer in West Michigan.

For patients and their families, a trip to a hospital or other medical facility is often confusing and stressful. Strange devices, some rather large and others rather intimidating, seem to be everywhere.

Our Spectrum Health Virtual Reality Experience app gives patients and their families the ability to explore Spectrum Health’s many facilities virtually, from the convenience and comfort of home.

Patients enjoy an interactive experience with a full 360° view. As they look around, items in the room are identified and explained. From operating rooms to treatment rooms to patient rooms, users learn what’s what before ever entering a Spectrum Health facility, thereby reducing confusion and stress.

Our Spectrum Health Virtual Reality Experience app runs within any desktop or mobile web browser. In addition, our app supports various virtual reality devices such as an Oculus Rift or a Google Cardboard to provide a completely immersive experience.

Spectrum Health employees use our companion administrative web portal to add new rooms, which includes uploading 360° images and annotating points of interest within a room.

Our Virtual Reality Experience app is written in HTML and JavaScript. A-Frame is used for 360° image browser support. ASP.NET Core and MSSQL provide server integration.

TechSmith: TechSmith Video Review and Slack Integration 

TechSmith provides software that empowers people to communicate more effectively by easily creating visual content such as images and video. Their flagship products, Snagit and Camtasia, are used by more than 30 million people worldwide.

TechSmith Video Review is a web-based system that enables video authors to obtain feedback on their videos before publishing them.

Our TechSmith Video Review and Slack Integration project extends the functionality of Video Review by connecting video authors with video reviews using team messaging systems.

In particular, our system integrates the popular messaging system Slack into Video Review. Users post and review videos, all from within their desired Slack channel. Reviewers post reviews of their own and comment on other users’ reviews. Video authors read reviews and respond to reviewers interactively using Slack.

Reviews can be created from directly within Slack or synchronized from a preexisting review created from within TechSmith’s Video Review.

Once a review is posted on Slack, all reviews and replies are updated both within Slack and within Video Review. This enables users to create and modify reviews on the platform of their choice while having consistent information.

Our TechSmith Video Review and Slack Integration system is implemented using two proxy servers, one for Slack’s API and one serving a REST API that communicates with Video Review. Both servers are written in C#, contained in Docker, and documented using Swagger.

Union Pacific: Augmented Reality Mechanic Training

Headquartered in Omaha, Nebraska, Union Pacific is a leading railroad company with some 42,000 employees, 8,600 locomotives and 64,000 freight cars riding on 32,000 miles of track covering the western two-thirds of the U.S. To maintain this massive infrastructure, Union Pacific must train teams of mechanics to repair and operate complex machinery. This equipment is often large, unwieldy and expensive, making it challenging to provide training that is both safe and cost- effective.

Our Augmented Reality Mechanic Training systems provide an immersive and intuitive virtual training experience using the Microsoft HoloLens and Android. Our system is separated into two educational modules: Learn About Machinery and Build a Train.

Learn About Machinery displays holographic images of equipment for mechanics to study as a replacement for actual, physical machinery. As shown in the lower example at the right, trainees can rotate and interact with these holograms, accessing detailed maintenance information.

Build a Train provides real-time instructions on how to assemble a train, verifying that each step is completed correctly along the way. Labels projected above each train car provide guidance to the trainee as seen at the right.

Our Learn About Machinery and Build a Train systems demonstrate proof that augmented reality environments such as ours provide safe and cost-effective training.

Our Augmented Reality Mechanic Training systems are written in C# using the Unity game engine. They use Vuforia for object recognition and the PiXYZ plugin to process CAD models.

United Airlines: Toolkit Content Verification System

United Airlines is one of the world’s largest airlines, serving over 148 million customers a year to 357 destinations in 48 countries with 4,600 daily departures.

For any airline, aircraft grounded due to mechanical issues may cause cancelled flights, loss of revenue, and dissatisfied customers. United operates the third largest fleet in the world with some 757 planes.

In order to maintain their fleet, United uses over 200,000 tools stored at various airports. Many of these tools are bundled into kits designed for very specific tasks like changing out the engine on a Boeing 787.

When a toolkit arrives at a United destination, it is crucial that the kit be complete so that technicians can address the issue and return the aircraft to service safely.

Our Toolkit Content Verification System ensures that toolkits are complete before they are checked out for use and before they are checked back in for storage. The system also logs every transaction for each toolkit.

To verify the completeness of a toolkit, a United technician simply uses their Apple iPhone or iPad to capture a photo of an opened kit as shown at the right.

Once a photo is captured and sent to our server, computer vision algorithms determine which tools are present and which are missing. On screen, our app then informs the technician whether or not the kit is complete.

Senior technicians use our companion web app to add new toolkits to the system and to track histories of kits.

Our Tooling Kit Content Verification System iOS app is written in Swift, with computer vision processing using the OpenCV library. Our web app uses the Django framework.

Urban Science: VIN-Verse

Headquartered in Detroit, Urban Science is internationally renowned for providing data-driven, science-based solutions to problems in the automotive, health and retail industries.

The history of a vehicle is important to owners, sellers and buyers alike. Keeping an accurate and accessible record of a vehicle is a challenge, which often results in a disorganized group of documents stored in a vehicle’s built-in filing cabinet, its glove box.

Based on a vehicle’s unique Vehicle Identification Number (VIN), our VIN-Verse system leverages Urban Science’s existing data to provide a comprehensive history of a vehicle, including maintenance, both past and upcoming, repairs, recalls, and accidents.

VIN-Verse enables a vehicle’s owner to manage who can view their vehicle’s history. Repair facilities give better service when they are able to review a vehicle’s complete history. Potential buyers are more likely to purchase if they know the complete facts about a vehicle.

To augment Urban Science’s existing vehicle data, VIN-Verse includes a verified self-reporting system with which vehicle owners can enter their own repairs, thereby making the history complete.

For vehicle manufacturers, VIN-Verse provides a dashboard that visualizes trends in the service history of the vehicles that they make and sell.

As a responsive web app, VIN-Verse is accessible using any web browser on desktops or mobile devices.

The frontend of our VIN-Verse system is built with ASP.NET/C# and Angular 6. The backend is implemented using Microsoft SQL Server 2016.

Volkswagen Group of America: VW Car-Net Demo App 

Volkswagen Group of America is the North American operation headquarters and subsidiary of the Volkswagen Group. Volkswagen sold 10.5 million vehicles globally in 2017, the most of any manufacturer.

In 2013, Volkswagen introduced VW Car-Net, a Connected and Mobility Services system that offers convenience and ease of access to Volkswagen owners through a variety of features and tools. With technology evolving quickly, VW is preparing to launch an updated version of VW Car-Net with new features and technologies starting with model year 2020 vehicles. As a result, it is important that dealers and customers be made aware of the benefits of this new version of VW Car-Net.

Our VW Car-Net Demo App provides a platform where users are introduced to and familiarized with VW Car-Net while at the dealership. Our app guides users through several tutorial modules. Each module explains the functionality of a VW Car-Net feature, acquainting users with the advantages of their connected car services while simultaneously engaging with the VW brand.

Our app consists of five tutorial modules that explore VW Car-Net features including Registration, Remote Services, Vehicle Health Reports, Navigation and Guardian Services.

To encourage the greatest number of potential VW customers to learn about VW Car-Net, our app supports both Google Android and Apple iOS mobile devices.

Our VW Car-Net Demo App is written in Kotlin for Android and Swift for iOS. Firebase Analytics is integrated into each app to gather usage statistics.

Whirlpool: IRAV: Image Recognition, Annotation and Validation

Whirlpool Corporation, headquartered in Benton Harbor, Michigan, is the world’s leading major home appliance company with approximately $21 billion in annual sales and 92,000 employees.

Whirlpool is reducing the daily challenge of getting meals on the table by providing their customers with a seamless kitchen experience. To this end, they recently acquired Yummly, the premier digital food platform.

Yummly gives personalized recipe recommendations based in part on the ingredients that a user already has on hand. Yummly determines these ingredients by applying computer vision to images captured with mobile devices.

In order to recognize ingredients accurately, Yummly compares captured images to a large set of existing images in which ingredients are already annotated with labels and validated for accuracy.

Our Image Recognition, Annotation and Validation (IRAV) mobile application leverages crowdsourcing of Whirlpool employees, utilizing users’ captured images and validating them, effectively contributing to the many comparison images required by Yummly.

IRAV users take pictures of ingredient items, annotate them, and submit them. However, before Yummly accepts an image, the annotations must be validated by other users.

An image is considered to be acceptable only after it is validated by enough IRAV users. Only then is it added to Yummly’s set of comparison images.

Our Image Recognition, Annotation and Validation application is written in Swift using Xcode for iOS, and Java using Android Studio for Android. Google Firebase is used for the backend.