CSE Projects – Spring 2021

Amazon: Sentinel

Founded in Bellevue, Washington in 1994, Amazon is a Fortune 500 company that provides a variety of services to customers and is the world’s largest cloud services provider and online retailer.

Amazon’s online marketplace handles millions of orders every day from over a million unique sellers. With an operation of such magnitude, it is inevitable that some buyers and even some sellers engage in fraudulent activities. Amazon’s current fraud detection system requires individual sellers to detect and recognize unusual activity themselves, which can lead to undetected fraud and extra work for sellers.

Our Sentinel system helps to resolve these issues by automatically detecting fraudulent transactions and notifying sellers in real time. Sentinel uses machine learning to detect patterns from historical transaction data, then applies these patterns to all incoming transactions to determine when a fraudulent transaction has occurred.Once Sentinel has detected fraud, the seller is notified immediately.

Using our mobile application, sellers can manage all of their orders, including any fraud detected by Sentinel. Our application gives sellers options to freeze their account, cancel any fraudulent orders, or to explore more transactions before making a decision. Sentinel helps Amazon sellers save time and money by detecting and mitigating fraudulent purchases automatically, giving sellers much needed peace of mind.

Our machine learning models are built using Amazon SageMaker, trained on user data and transactions stored in Amazon S3. AWS Lambda is used to detect instances of fraud in real time and the cloud platform sends a notification to the merchant’s mobile device. The iOS application is written in Swift.

Anthropocene Institute: Siting of Marine Turbines for Power Generation

The Anthropocene Institute is a non governmental organization that drives and facilitates innovation in clean energy to address the urgency of climate change. It supports start-ups and universities to develop emerging and disruptive energy technologies that are clean, safe, and reliable. It is also involved in raising public awareness for ‘one-cent’ electricity as a key goal for the power grid.

The Anthropocene Institute has started to look at aquatic turbines as a promising new type of renewable power generation. However, a major hurdle to overcome is determining the best placement for the turbines that maximizes their power generation and minimizes costs.

When placing a water turbine, many factors have to be considered, including the depth of the water, the velocity of the current, and the distance to the nearest coast. However, current data on oceans and rivers are incomplete, meaning accurate predictions for turbine placement cannot be achieved.

Our Siting of Marine Turbines for Power Generation project solves this issue by using machine learning to fill the gaps of the currently incomplete data. Our models use historical data to predict the water conditions in any area without sufficient data, allowing Anthropocene to give accurate power generation and cost estimates for turbine placement.

Our web application provides an easy-to-use interface designed for policymakers, utilities, and investors who are looking for clean energy solutions. Users simply search for an area, and our application provides them with statistics and recommendations relating to the cost and power generation potential of a water turbine placed in that area.

The machine learning models were developed in Google CoLab, using the scikit-learn library. The user interface is a Flask- based web server with data visualization by the Google Maps API.

Authoritek: Crisis911 Emergency Communication

Authoritek is a software consultant company that provides their clients with purpose driven software that manages their data. They are located in the Grand Rapids area of Michigan. Emergency situations are stressful, both for those involved and their loved ones. During times like these, the ability to stay informed and inform others is key. It is vital that the user experience is quick and simple so precious time is not wasted.

Our software, Crisis911 Emergency, is used in schools when staff encounters an emergency situation that puts them or students in danger. With the help of this application, users can notify loved ones or other faculty in the school of the situation. The software is simple yet effective in notifying individuals who need to be updated about a crisis.

The application has a live feed for crisis updates that can be sent out to whomever the user wants to notify. That feed contains updates of the situation and continues to be updated until the crisis has ended. This allows parents, faculty and guardians to be aware of what is happening in the school without being present.

A key element of Crisis911 Emergency is the ability to have live updates on the crisis at hand, which the user can select when starting the event. This means users in the system receive all information and everyone can be kept safe.

A mobile and web application are available for users to operate, with both being able to indicate the start and end of a crisis. Users can choose who they want to notify by adding contacts through the web application and creating new events to further articulate what type of crisis is happening. Crisis911 Emergency keeps all parties updated about crises in real time.

Our application’s back-end database is managed through Google Firebase. The design and functionality of the application is accomplished with React Native, ReactJS, HTML, CSS and Twilio.

Auto-Owners Insurance: AO Sidekick

Auto-Owners Insurance is a Fortune 500 company headquartered in Lansing, Michigan. With over 48,000 licensed insurance agents in over 26 states, Auto-Owners provides automotive, home, life, and business insurance to nearly 3 million policyholders.

With over 600,000 visits to their website every month by a broad base of both agents and policyholders, Auto-Owners strives to ensure that their products and resources are easily accessible to everyone, including those who have difficulty using a web browser.

The AO Sidekick is an accessibility application offered to users trying to access the Auto-Owners’ website. The Sidekick directs users to pages across the site quickly and simply using only voice input.

When a user arrives at the AO Sidekick homepage, they are greeted with links to common pages on the AO website, as well as a description of what the Sidekick can do. From there, the user can navigate to the Sidekick page where the communication interface is located. Here, the user can see their correspondence with the Sidekick displayed in a text message style format. At the press of the microphone button, Sidekick interprets the user’s query and directs them to where they want to go.

If the user’s intent is unclear or their request too vague, our system asks follow-up questions for clarification.

Information gathered from this phase is stored and displayed in an intuitive manner on the dashboard page. AO employees can use this insight to facilitate further improvements in the future.

AO Sidekick is developed as a web application hosted by Azure. Azure provides the MS SQL database as well as the LUIS natural language processing API. The front end was developed using HTML, CSS and the ReactJS framework.

Bosch: Real Time ADAS Endurance Run Data Validation

Bosch is a German-based international technology and service provider employing approximately 395,000 employees worldwide. Bosch specializes in integrated technology, such as smart devices and automobiles, generating almost 72 billion euros in sales in 2020.

Bosch uses test vehicles to gather data from vehicle systems as they drive along a variety of roads and highways across the world. These vehicles collect data from specific areas where the driving environment generates conditions of special interest to Bosch engineers for use in improving their systems.

Presently, these vehicles drive and gather data for up to a month before the data is delivered en masse to engineers for analysis. This timeline generates a problem when the data contains errors of some form and the test vehicle is distant from where the error occurred.

Our Real-Time Advanced Driver Assistance System (ADAS) Endurance Run Data Validation platform solves this problem by ensuring the data collected is error-free in near real time. Any time errors are present in the data collection, the test vehicle drivers are notified, allowing the driver to repeat the test in a timely manner.

Our software system achieves this goal by validating data upon completion of the data collection process, utilizing a computer embedded within the test vehicle. This verification program then communicates with other infrastructure to allow drivers and engineers to view the status of this verification software in a web application. The website contains capabilities which allow individuals to easily view and update information regarding the verification process.

Our Real-Time ADAS Endurance Run Validation platform uses a Python script to read and validate vehicle data as soon as data collection has been completed. A web application then allows users to view, update, and manage the data validation process.

CSAA Insurance Group: Insurance Coverage Wizard

CSAA Insurance Group (CSAA) is a subsidiary of AAA
Insurance operating out of Walnut Creek, California. CSAA offers auto, homeowners, and other lines of insurance in twenty-three different states and the District of Columbia.

Purchasing insurance is a time-consuming and difficult
process that often leaves customers annoyed and confused. CSAA is always innovating insurance and wants to educate their customers on possible insurance coverage options.

Our Insurance Coverage Wizard is a tool designed to help
customers quickly and easily receive a personalized insurance quote. The wizard consists of a series of questions to help it learn more about the user and their insurance needs. The wizard then guides the user through various types of insurance coverages to better inform the customer of their insurance options.

The Insurance Coverage Wizard includes eight steps, each of which educates the user about a specific coverage and how it pertains to them. Using the knowledge from the wizard, users can confidently build an insurance plan that is right for them.

Our wizard is accessible from any computer or smartphone and is designed to keep the user engaged with the material to provide them the best experience possible.

To achieve the best experience for their customers, CSAA collects data on how users interact with the wizard. This data is available to CSAA employees in a PowerBI dashboard. With this user engagement information, CSAA constantly updates the wizard to improve the insurance buying process for their customers.

Our software uses a ReactJS front end to communicate user input to the .NET Core back end. Data is hosted in a MySQL database. All of this is deployed on an Amazon EC2 Instance.

CSAA Insurance Group: Eye in the Sky: Intelligent Drone Video Processing

CSAA Insurance Group, headquartered in Walnut Creek,
California, is a AAA insurer, one of the top insurance groups. They offer auto, homeowners and other lines of insurance in twenty three states and the District of Columbia.

One major area of CSAA’s business is home insurance. With the increasing number of wildfires in recent years, it is important for both the insurer and homeowners to accurately assess property risk from wildfires.

Our Eye in the Sky: Intelligent Drone Video Processing project
uses drone footage to determine high fire risk areas as well as the risk of specific homes, providing key information for CSAA in determining the necessity of action and risk associated with insurance throughout California.

To determine the fire risk of a home, a drone is sent to collect images of its surrounding area which are then sent to our web application, Eye in the Sky. Our application processes these images using computer vision to detect objects of risk, including things like dry brush, concrete and trees. Based on the different properties of surrounding objects, the property is given an overall fire risk score.

CSAA employees use our web dashboard to assess the overall fire risks of potentially vulnerable areas. A large coverage map allows employees to easily view the fire risk of large areas, as well as the specific fire risks of an individual property.

Based on these fire-risk scores, employees can take appropriate action either by providing recommendations based on the surroundings to mitigate fire risk or investigate further using drones or by an in-person inspection.

The front end of our Eye in the Sky project is written in React, HTML and CSS. The back end is written in Python using Flask to link the two. MongoDB is used as the database and the entire web application is hosted on AWS.

Delta Dental of Michigan, Ohio and Indiana: Smart Dental Benefit Recommendation Engine

Delta Dental is the largest dental system in the United States, which operates two of the nation’s largest networks of participating dentists. Delta Dental provides dental and orthodontic coverage to more than 78 million people in all 50 states, Puerto Rico and other U.S territories, including small businesses, government entities, and professional organizations of all kinds.

Underwriting is the process of how Delta Dental determines the best dental insurance plan for their clients. In addition to crafting insurance plans for new clients, underwriters also spend a significant amount of time recommending benefit changes for current customers. This time-consuming process requires aggregating data from several sources and significant domain knowledge to properly recommend benefit updates.

Our Smart Dental Benefit Recommendation Engine helps automate the underwriting process to improve benefits for customers and save underwriters time.

Our system uses machine learning to look at historical data to help determine the best insurance plan for a given individual or family. Based on this analysis, our system recommends additions or reductions to the customer’s current plan.

Our web dashboard aggregates all of the information needed by underwriters in one convenient location, including the benefit changes recommended by our machine learning model.

Using our system, Delta Dental underwriters can offer customers faster and more efficient service, and it allows them to focus their valuable time on more important issues.

The recommendation and prediction algorithms are written in Python in Jupyter Notebook, the web application is written in ReactJS. Our data is hosted on Snowflake from Delta Dental’s Microsoft Azure databases. Docker is used to containerize our engine and host it on Microsoft Azure.

Delta Dental of Michigan, Ohio and Indiana: Rule Engine Command Line Interface

Delta Dental is the nation’s leading provider of dental insurance, serving more than 80 million Americans, with the core purpose of advancing the oral health of their customers, partners and consumers. Delta Dental of Michigan, headquartered in Okemos, Michigan, is at the forefront of utilizing rule-based solutions within their business operations and intelligence, with over 15 years of excellence in this domain.

When a Delta Dental developer wishes to engineer a rule-based solution, a non-trivial amount of setup must be completed first. Prototyping and testing of rule-based solutions are slowed down significantly by the setup process.

Our Rule Engine Command Line Interface provides an intuitive and interactive interface to Drools, the rule-based computing engine that Delta Dental’s developers use on a daily basis.

Using our software, developers interact with every part of the Drools engine. The interface is organized into various modules, each of which handles a core function of rule-based computing. Our modular design makes it easy to update or even swap out various components when Drools is updated, or new software is needed.

The Rule Engine Command Line Interface streamlines the development process for rule-based computing, saving Delta Dental a significant amount of time and resources.

For the convenience of Delta Dental developers, our software functions as a command-line shell program. Our software is written in core Java, and therefore runs on any computer that has a Java runtime environment. The external libraries, including the core Drools library, and build process are managed by Maven. Version control and collaboration management were provided by Gitlab.

The Dow Chemical Company: Improving the Performance of the Corporate Computer

With over a century of experience, Michigan-based Dow is a global leader in the innovation, creation, and distribution of specialty chemicals, advanced materials, and plastics. As a large company with 54,000 employees worldwide, Dow has a massive collection of computers and devices which employees use. The computers are of varying operating systems, models and generations. Dow collects computer usage and performance data on the computers in order to efficiently use their resources.

Our Improving the Performance of the Corporate Computer system analyzes the data so that events such as application crashes, blue screens and application hang times can be minimized. Data analysis reveals the optimal parameters for computers so that Dow employees can determine which are best to use. Data analysis also reveals applications most susceptible to crashes so that potential workflow improvements can be made.

Using the collected data, our system uses a machine learning model that predicts the performance of a computer given a set of properties.

We developed a web application where employees from Dow can visually input computer specifications and other parameters. Then, the application runs this input through our machine learning algorithm and visually displays the result for the employee to see.

This helps Dow easily determine whether it is worth investing money to upgrade or purchase different computers or components. Employees can use the web application to determine whether they should make changes to their system and what the expected outcome would be. In the long run, this helps Dow be more efficient with costs and its employees be most productive.

Data analysis is conducted using Python in Jupyter notebooks. The machine learning algorithm is developed using Python and Microsoft Azure. The web application is developed in Python Flask.

Ford Motor Company: Ford Team View

Ford Motor Company is a multinational automotive manufacturer headquartered in Dearborn, Michigan, with operations in over 125 countries and a worldwide workforce of 199,000 employees. Ford designs and manufactures a full line of cars, trucks, SUVs and electric vehicles under both the Ford and Lincoln brands.

Ford is committed to promoting a work environment that
supports and benefits from collaboration across multiple teams. However, due to the COVID-19 pandemic, team collaboration has been made more challenging. Instead of working side by side, teams are now working in their own private residences, making it difficult to locate other teams or make contributions to their projects.

Our Ford Team View web application provides a convenient way for users to locate other teams by either searching for a team name, advisory number, or an individual. Once a team is found and selected, their team profile is displayed.

In the team profile, a user can see every team member and their contact information, any incidents and issues the team is facing, the projects the team is working on, and a hierarchical chain of team members to contact when an issue arises.

Our application also allows users to search for a project by either entering an API name, base path, or contributor. When a project is clicked on, a user can view the number of times code was added to the project, the project’s build and deployment history, and the contributors to the project.

Team View makes it easier for teams to collaborate and work together while working remotely, thereby increasing productivity.

The front end of our web application is created with the REACT framework, while our back end uses the Spring framework. Our application uses a MongoDB to store the team profiles and project information, which is retrieved by a REST API.

General Motors: Malware Reverse Engineering Platform

General Motors (GM) is a multinational automotive
manufacturer headquartered in Detroit, Michigan. GM is
ranked #18 on the Fortune 500 for total revenue and is the
largest auto manufacturer headquartered in the United States.

GM is committed to maintaining corporate security and the
security of their customers. For this reason, effectively sharing information about malware and indicators of compromise within their organization is critical.

Our Malware Reverse Engineering Platform is a web application that provides a unified and easy-to-use interface that allows users to submit suspicious samples for analysis.

These samples may be files the user already has, or they may be scraped from web sources. The user can submit local files by either dragging a file and dropping it into an indicated box, or by selecting a file. They can submit a website to scrape by simply entering the URL and selecting the start button.

The analysis returns a report about the sample’s behavior, such as what files it modifies, what it attempts to do on the network, and other potentially malicious activities. The report is visible within the interface after the analysis, and the user may download the report to their local machine. The report is also automatically sent to their organization’s database for storage and for other members to view.

This platform automates and greatly simplifies a currently manual process that requires the user to interact with multiple programs. This gives GM’s security analysts greater flexibility and efficiency in analyzing malware and sharing results.

The malware analysis incorporates Cuckoo. The samples and analyses are stored in a Malware Information Sharing Platform (MISP) instance. For integrating these tools, we are using the PyMISP library. The web interface uses Flask built off of Adobe XD.

Herman Miller: Scout 2.0: Dynamic Data Visualization for Dealers

With over 100 years of experience, Herman Miller is a globally recognized provider of furnishings, related technologies and services headquartered in Zeeland, Michigan.

Herman Miller uses the Atlas Suite, an online suite of applications meant to make their users’ experience as fluid as possible. Users leverage an application called Scout in the Atlas Suite to help potential customers visualize ideas, select commercial furniture and textiles, and build custom proposals.

The underlying data in the application, like clicks and hits, are monitored and recorded. However, substantial analysis of this data has not been done and potential insights were going unrealized.

To overcome this, our Scout 2.0 application dynamically visualizes the data for users through various interactive models.

The customer acquisition tab displays website traffic throughout the month, week, and day. Any user can hover over the diagram to get more specific data on all of the different tabs.

The popular content tab displays the projects and products views. The user can use the pagination feature which allows them to move back and forth between the most and least viewed items.

The customer locations tab provides insight into the geographical data recorded by Scout, ranging from country to region to city. Knowing this information gives dealers a better understanding of the state of their sale.

Our models provide various ways to visualize the data and help users understand different variables such as when and where the product is viewed, and the projects and products views. This aids dealers in selling products to customers, improving productivity.

The front end of Scout 2.0 is built through Visual Studio Code using AngularJS. The back end is implemented using several Amazon Web Services, including Lambda, Simple Storage Service, AppFlow, and Athena.

Learning A-Z: Definition Station Word Matching Game

Founded in 2002, Learning A-Z is a technology company dedicated to expanding literacy through an extensive collection of thoughtfully designed educational tools. Catering to grades PreK-6, Learning A-Z’s resources assist a wide variety of learners.

In the age of COVID-19, Learning A-Z’s devotion to creating engaging online resources has become more crucial than ever. As a forward-thinking company, Learning A-Z is always developing new ways to keep students of many different abilities and interests excited to learn.

Our Definition Station Word Matching Game is a web game that teaches and reinforces vocabulary to children. The game empowers students to further their education in spelling and definition recognition. In addition, our game is customizable to fit the diverse needs of students at many different learning levels.

Upon beginning the game, a deck consisting of ten vocabulary words is created. The student is presented with one definition at a time until they complete all ten words in their deck or until they make three spelling errors.

The student is given a few seconds to process each definition, at which point train cars carrying letters begin rolling down the train track. The student is tasked with using switches to control the train cars’ routes to the train station. A round is won when the student leads the train cars to the station in the correct order, meaning their word matches the definition and is spelled correctly.

Students choose their difficulty level from the home page before they begin playing. Parents, teachers, and older students can use the advanced settings page to further customize the difficulty.

Our game teaches new words and definitions to students in a fun and interactive way.

The software for this game is developed using JavaScript for the front end, which communicates with a MySQL database via PHP.

Lockheed Martin Space: SmartSat^TM Heterogenous Computing in Space

Lockheed Martin, headquartered in Bethesda, Maryland, is the
largest defense contractor in the world and receives $60 billion
in revenue annually. Lockheed Martin Space is a division of
Lockheed Martin which specializes in building and deploying satellites and spacecraft for both commercial and military use.

The SmartSat software infrastructure, designed by Lockheed
Martin Space, facilitates the development and deployment of software for spacecraft, which have historically leveraged all the power of the flight computers running the software that is responsible for operation of the spacecraft with little room to spare. Now that more powerful computers are available for the space domain, there is additional capacity that can be leveraged.

Our SmartSatTM Heterogenous Computing in Space system
enables data to be processed on the satellite, reducing the need to send information over slow network connections. This reduces the bandwidth usage of the satellite by keeping raw data in orbit.

Because a satellite has significantly less computation power than computers on Earth, our system introduces clever hardware allocation schemes that can speed up image processing times on the satellite. One such optimization includes taking advantage of the wide variety of computers that exists onboard the satellite to run multiple applications at the same time.

Our system constantly analyzes the available hardware on the satellite, then assigns different SmartSat image processing applications to the hardware based on the best available option at any given time.

Our system reduces the time it takes to analyze data from satellites and ensures that the hardware onboard is used to its full capacity.

We use SYCL and Vitis to accelerate OpenCV computer vision applications on CPU, GPU and FPGA hardware.

Malleable Minds: Improving Access to PreK-12 Educational Opportunities

Malleable Minds is an emerging startup building the world’s
most extensive collection of PreK-12 programs, from the arts to the sciences, so students can further develop academic, interpersonal, and communication skills. Malleable Minds is breaking barriers to educational opportunities and empowering families to create a tailored educational roadmap for their children, leading them to bright futures.

Malleable Minds continually strives to bring the best user
experience when designing their application. They currently have a review aggregator for educational programs.

Our Improving Access to PreK-12 Educational Opportunities
project builds many features on top of the existing review aggregator website and improves site performance.

The site’s users benefit from several new additions. When a user creates an account, a walkthrough shows key site features. Users are kept up to date through customized notifications based on type of user profile, location, and what areas they are interested in. User surveys capture feedback about the site’s features and potential future improvements.

Administrative users also gain additional functionality. The administrative workflow page allows for authentication of applicants requesting either a program owner or educator account.

Site administrators can view a user dashboard with details about how the site is being used. The dashboard provides data such as when users log in, search for programs, write reviews, and more.

Our enhanced site is more efficient, faster, and includes more features that enhance the user experience and track usage metrics and user feedback, allowing Malleable Minds administrators to continually improve the site.

Our software uses React on the front end and Python on the back end. We host our software on Amazon Web Services.

Mejier: Meijer Store Wayfinding

Meijer is a major supercenter chain headquartered in Grand
Rapids, Michigan, with locations throughout the Midwest. There are a total of 253 Meijer Supercenter stores, making it the 26th largest retailer in the United States. The Supercenter store layout is an idea that was pioneered by the Meijer Corporation, which combines groceries and department store goods all in one store.

Meijer is constantly at the forefront of innovation, especially when it comes to offering customers the optimal shopping experience. Because of the inherent size of Meijer supercenters, customers may find it difficult to locate the specific product they are looking for in an efficient manner. In addition, the layout of Meijer stores can vary greatly by location and are periodically changed, adding to shopper confusion.

Our Meijer Store Wayfinding functionality offers Meijer shoppers a solution to this problem by integrating an intuitive and easy-to-use navigation interface in the Meijer mobile application. Using our mobile application, customers are given a full map of their Meijer store, including a dot showing their current location in the store. This is achieved using MIST wireless access points that Meijer is introducing to their stores.

In addition to this mapping feature, our application offers turn- by turn navigation of Meijer stores. Users input an item or list of items into our application and are shown the optimal path through the store to each item on their list. Directions to each item are shown on the map screen, similar to how Google Maps works while driving a car.

Meijer Store Wayfinding enhances the customer experience of all Meijer shoppers and allows them to feel more in control of their shopping experience.

Our software is written in Swift and is integrated with Mist Access Points to get live location data of shoppers.

Michigan State University Animal Science: Pesticide Management for Sustainable Vineyards

Michigan State University is a public research university boasting approximately 50,000 enrolled students. Originally founded as an agricultural college, the MSU Department of Animal Science remains a pioneer in agricultural research.

Michigan is home to a very large grape-growing industry. Vineyard owners and workers must cooperate to manage, track and control insect infestations and disease outbreaks as they arise.

Our Pesticide Management for Sustainable Vineyards system
provides a solution for workers in both the field and in the office.

When a field worker opens the app on their Android or iOS device, they can immediately view upcoming tasks, such as spraying a new pesticide onto their crop. When performing these actions, pertinent information like weather conditions and GPS location is collected. While in the field, workers can take pictures of new infestations as they arise, reporting them to management.

Vineyard managers can view this information on our system webpage, as well as automatically generate records of pesticide use for the USDA or MDARD. They can view maps of their field with up-todate information about current infestations and outbreaks, and assign new tasks to their workers to mitigate them.

Our system also calculates a risk factor for each pesticide based on the unique circumstances of each vineyard and field. This allows vineyard owners to select pesticides that are effective, sustainable, and safe for both their employees and their consumers.

Our software streamlines the process of identifying, mitigating, and reporting pests and chemical usage for the grape-growing industry, and creates a repository of information for further research into Michigan’s ecology.

The Pesticide Management for Sustainable Vineyards back end is a server utilizing a LAMP stack. The application is written in PHP for web, Java for Android, and Swift for iOS.

Mozilla Corporation: Pushing Picture-in-Picture towards Perfection

Mozilla is the company behind Firefox, the world’s
leading open-source web browser with over 200 million monthly active users. Mozilla is a non-profit organization headquartered in Mountain View, California, that has long been on a mission to keep the internet open and accessible to all people to enrich lives and foster innovation.

Firefox has introduced many tools to help their users better manage their multitasking. At the forefront is their Picture-in-Picture (PiP) feature that allows multiple videos to be “popped” out of the browser into dedicated windows that are always on top of all other windows. PiP is ideal for taking notes during a lecture or even watching multiple YouTube or Netflix videos at once.

Since the release of Picture-in-Picture, many users have requested that the feature be expanded, improved, and pushed “towards perfection.”

Previously, users had limited control over the PiP window when it was “popped” out of the web browser, which could lead to frustrations when trying to interact with the video.

Our Pushing Picture-in-Picture towards Perfection project mitigates these issues by giving the user greater control of their PiP windows and improves the overall user experience of Firefox’s PiP.

Our additions to PiP include allowing captions in the PiP window and allowing for keyboard control of things like timing, volume and size, as opposed to forcing the user to use the mouse in the browser.

These additions reduce the need for the user to interact with the browser and allow for more seamless multitasking while using Firefox.

Running with multiple languages (C++, HTML, JavaScript), PiP lives within the Firefox codebase. However, as a feature, PiP is primarily based on HTML and JavaScript.

MSU Federal Credit Union: Augmented Reality Financial Education

Founded in 1937 in East Lansing, MSU Federal Credit Union provides a variety of financial services to students, faculty and staff at Michigan State University and Oakland University. With 21 branches and over 300,000 members, it is the largest university-based credit union in the world.

MSUFCU provides a variety of financial education resources to its
customers to ensure they are making the best monetary decisions. Our
Augmented Reality Financial Education application serves to gamify
saving money and provide encouragement to their customers throughout the process.

Our mobile app teaches customers about saving money through a
new feature called goal journeys, which includes various savings goals
that a user can choose from, such as saving for a beach vacation, paying off student loans, or buying a house.

After a user selects a goal journey, they are asked to set aside a certain
amount of money for the goal, as well as determine what their overall
savings goal is and how long they want to save. Once this information is gathered, our application offers detailed savings reports and their overall progress towards their goal.

As the user progresses towards their goal, they have the option of using our augmented reality view, which helps them visualize their progress towards their goal. For example, as the user saves money, the augmented reality view for the beach vacation goal journey shows the user with a beach towel, then with sunglasses and a surfboard, until they eventually reach their savings goal, and the augmented reality view shows them on vacation. This visualization of progress helps fully immerse the customer in their savings goal.

Our Augmented Reality Financial Education mobile app is built using Swift for iOS and Java for Android. The web app is built using ReactJS. The back end for all applications is built using PHP and PHPMyAdmin.

Proofpoint: Predicting the Future through Spam Signal Intelligence

Founded in 2002, Proofpoint is a cybersecurity company based in Sunnyvale, California, that safeguards their clients and their clients’ data from online threats. Proofpoint identifies and filters threats from email, the web, the cloud, social media and mobile messaging. They protect banks, research facilities, universities and numerous companies across the world.

To this end, Proofpoint has accumulated terabytes of spam and analyzed it to predict potential risks and provide improved security.

Our Predicting the Future through Spam Signal Intelligence system analyzes this spam email data for patterns to help users predict real-world events.

Our system sorts emails by topic and sentiment with machine learning, analyzes them for underlying patterns, then charts them against real-world outcomes to hone the predictive algorithms. These methods can be used to anticipate wide societal events such as election outcomes and stock market fluctuations.

The resulting predictions and data are viewable from our web dashboard. The dashboard can be configured to view statistics such as spam volume, general sentiment regarding a topic, changes in sentiment over time, and projected future sentiment.

Proofpoint analysts evaluating future risks use our dashboard to forecast potential events and take measures early on to ensure they are well-prepared to protect their clients. This allows Proofpoint to provide improved security.

Our back end runs a PostgreSQL database server to store information gleaned from the spam email files. We use Flair and scikit-learn for machine learning and an email parser to extract information from the spam.

Our web dashboard is built on the Django framework and Apache server. It displays information from the database through a React user interface.

Quicken Loans: Project Relo

Headquartered in the heart of Detroit, Michigan’s financial
district, Quicken Loans is the nation’s largest online mortgage
lender. They have been providing affordable mortgages and award-winning client service for more than 30 years.

Employee relocation can be complex and stressful, especially
the home mortgage acquisition process. Unfortunately, enterprise relocation agents must complete this process manually for multiple employees concurrently. This can be time-consuming and leaves room for data entry errors.

Our Project Relo software is a web-based application that allows agents to easily store employee data and distribute it to partnered mortgage companies. Within our website, mortgage request update messages keep the agent and mortgage bankers aware of the requests’ statuses.

Upon login, relocation agents can view their currently active employee relocations and mortgage requests, shown in the top artwork. They can add new relocations and requests to the website through a simple form.

When selecting an employee relocation, the details are displayed, shown in the bottom artwork. From here, the agent can create new mortgage requests and complete the mortgage process on the employee’s behalf. Agents can select a mortgage request to view progress updates with timestamps. Completed relocations are archived and can be searched for by name or ID.

Our website also supports an admin user. They can add mortgage companies and agents into our software. If either of these users have their access revoked, their information is archived, and the admin can search for it by name or ID.

Our front-end web application is built with ReactJS while the back end utilizes Node.js, GraphQL and DynamoDB. All components of Project Relo are hosted on Amazon Web Services.

Quicken Loans: Rocket DevRel Tracker

Headquartered in the heart of Detroit, Michigan’s financial district,
Quicken Loans is the United States’ largest online mortgage lender. Rocket Mortgage was launched in 2015 to serve as
Quicken Loans’ online mortgage lending platform.

Rocket Mortgage recently created a Developer Relations team that participates, sponsors, and speaks at virtual or in-person conferences and events to help communicate supported products and services. This has resulted in the need to collect and share relevant information and data.

Our Rocket DevRel Tracker is a responsive web-based application
that allows the Developer Relations team to track upcoming conferences, view submitted talk details for discussion, and store related personal media.

Users can view future conference details along with whether Rocket Mortgage has obligations such as operating booths or speaking at any given event. Conferences are stored so that ratings can be given to past events for leadership to evaluate if the team should return the following year.

Within our system, speakers can store, track and submit talk details for acceptance at conferences and events. The Developer Relations team reviews talk details and decides whether to approve them. Other users can then view accepted talks and their information, such as location and time.

Our system automates much of the work currently done manually by the Developer Relations team in order to free their time. This allows them to focus on the conferences and events themselves, improving efficiency and productivity.

Our web application uses Google’s Firebase hosting platform for the API as well as the document-oriented database, Cloud Firestore. In addition, the front end of the application utilizes ReactJS.

TechSmith: TechSmith Answers

Headquartered in Okemos, Michigan, TechSmith provides software that empowers people to communicate more effectively by easily creating visual content. Their flagship products, Snagit and Camtasia, are used by more than 30 million people worldwide.

TechSmith also offers TechSmith Knowmia, a platform where
content creators can upload and share collections of media for other users to watch. One common use case of Knowmia is that of instructor and student within an educational institution.

Instructors often upload large collections of videos for students
to watch, such as a set of videos that covers a module within a course. Because there are such a large number of videos, students often find it difficult to find videos to answer their question, or a video on a particular topic.

Our TechSmith Answers platform provides a solution to this problem through an intuitive web dashboard that allows users to easily search and navigate large collections of media without any additional effort by video creators.

TechSmith Answers uses speech-to-text software to create subtitles for every video uploaded to our system. Using these subtitles, our system uses natural language processing to determine questions that can be answered by the video.

When a user searches for a video on a particular topic, TechSmith Answers directs them to the exact minute of a particular video that helps solve their problem.

Our system improves the learning experience of students by allowing them to spend their time studying instead of searching through videos for hours on end.

TechSmith Answers uses Microsoft Azure for data storage, speechto-text generation, natural language processing, search, and sign in. The front end is built using ReactJS, and the back end is built using .NET Core.

United Airlines: Turn Management Analyzer

United Airlines, Inc. is a leading American airline headquartered in Chicago, Illinois. In 2019, United and United Express operated more than 1.7 million flights carrying more than 162 million customers to their destinations safely.

Safety is United Airlines’ number one priority on every flight. An aircraft turn accounts for the time it takes for a series of safety procedures to be carried out by United Airlines’ employees from when the aircraft arrives at the gate to when it clears for departure.

Auditors conduct live audits of aircraft turns to determine whether the arrival and departure steps are compliant or noncompliant per standard operating procedure. United Airlines is exploring ways to automate the recognition of some of these aircraft turn tasks.

Our Turn Management Analyzer automatically identifies and evaluates the aircraft turn using the video camera feeds at each airport gate.

When an aircraft begins a turn, our system uses computer vision algorithms to track any objects and people working on the aircraft to determine if they are compliant with all safety protocols and complete the proper steps for a safe aircraft turn.

If our system detects a task that is performed incorrectly, out of sequence, or any employee not following safety protocols, it flags this action for review by a United Airlines auditor who can evaluate the situation.

Our software helps auditors ensure that United Airlines always provides their customers with the safest flight possible.

The front end is written in HTML, JavaScript and CSS. The back end consists of a MySQL database and PHP scripts to process data. The computer vision analyzer is developed with PyTorch, TensorFlow, OpenCV and Python.

United Airlines: Training Multimedia Content Management System II

United Airlines is a major international air-carrier
headquartered in Chicago, Illinois. In 2019, United Airlines
operated 4,900 flights a day from 362 airports. Running an
airline requires diligence in all logistical and technical aspects to ensure the best flight experience for “Every customer. Every flight. Every day.”

Within United Airlines, the TechOps training division is
responsible for teaching United’s technicians how to operate and maintain their wide variety of aircrafts. To supply this training, United Airlines currently maintains a vast encyclopedia of training documents and videos.

Our Tech Ops Training Content Management System II builds
off a previous capstone project and provides quick access to this training content through an intuitive, easy-to-use website.

Our work expands an already existing website to give United
Airlines employees and administrators a better user experience and greater control of their training media needs.

When employees use our system for training, they are given many tools to track and assess their training progress. An engagement report is generated for each user, which includes a collection of their recently viewed content, bookmarked videos, comments and likes.

United Airlines administrators are given improved tools for content upload and management, including options for bulk import and export of training content. Additionally, administrators are given an audit log to track any requested changes.

Our system greatly improves content management, saving employees and administrators significant time and effort.

Our system is optimized for both tablet and desktop use and is built using ASP.NET Core 3.1, Angular 10, Node.js, Entity Framework, and an Azure SQL Database. The web app is hosted on a Microsoft IIS server.

Urban Science: Service Flash Mobile

Headquartered in Detroit, Urban Science is internationally
renowned for providing data-driven, science-based solutions
to problems in the automotive industry. With strong industry
knowledge, Urban Science provides meaningful solutions for companies from GM to Ferrari.

Currently, dealership field workers, who go from dealership to
dealership analyzing data, must set up a work environment to access key metrics in every dealership they visit. This manual process takes time that could be used devising sales strategies.

Our Service Flash Mobile application provides an easily digestible interface for field workers to choose what metrics they would like to view in order to help with sales strategy.

When a user first logs in they are met with a customizable home page. Metrics can be added to the home page by selecting the filters that are desired. The filters range from geography, part type, sold or purchased, and time.

Our software is designed to let users access each metric to view more details about that metric. Details about the metric are broken down further on the metrics page. Depending on the filters applied, users can compare the data selected to either a target goal or the previous year’s data. This allows executives at Urban Science to identify areas of opportunity and track progress towards goals quickly, allowing them to act early enough to meet their goals.

Our mobile application allows Urban Science clients to access key metrics intuitively and in a timely manner, which increases communication and productivity in the workplace.

The front end of our application uses Angular, which is a framework that entails HTML, CSS and Typescript, wrapped in Cordova for iOS and Android integration. The back end uses C# and a SQL database hosted on Microsoft Azure. The API is hosted on Microsoft Azure.

Vectorform: Remote Teams AR Training

Vectorform was founded in Detroit in 1999 and has provided a
platform for many leading brands to solve big problems with
inventions emerging at the forefront of technology. They work
with a variety of technologies including hardware engineering, Internet of Things, augmented reality and virtual reality in their solutions.

COVID-19 has caused an unprecedented number of employees
worldwide to work from home. This has created many problems that need to be solved. One such problem is that of providing interactive learning in a remote environment.

Our Remote Teams AR Training platform helps mitigate these
problems with a Microsoft Teams application that gives instructors and students the tools needed to succeed in a remote learning environment.

Our instructor application allows educators to display 3D objects in a Microsoft Teams video stream (shown on the right), and provides a framework to present questions in real time to trainees that are saved for analysis of trainee progress.

When planning a lecture, instructors are given an intuitive interface to select and test various 3D object files on their video feed. Instructors can then present these 3D objects with themselves and other training materials in the background when they hold a live lecture to make their lecture more engaging to the trainees.

Our software also allows instructors to quiz trainees on the topics being discussed, which can then be analyzed to make sure that trainees are learning the material properly.

The 3D object manipulation aspect of our program is implemented as a C# program which interfaces with the camera feed for Microsoft Teams. The quiz portion of our program utilizes JavaScript to make a custom Microsoft Teams module. Both of these sides of the program connect to a SQLite database, which stores the quiz and 3D object information.