Shayan Hosseini
Vedant Malhotra
Joseph Jacobson
Dave Feil-Seifer
Sara Davis
Vinh Le
External Advisors
Sean Montgomery, Ph.D. of Connected Future Labs
Tina Samavat, Ph.D Student at Big Data Analytics and Informatics Lab
Introducing SoulSync
SoulSync is a mental wellbeing app that integrates biometrics from Apple HealthKit, Apple Watch, and the EmotiBit to help users understand and manage stress and anxiety. With over 57 million people in the U.S. affected by anxiety disorders, current virtual tools often lack real-time feedback, requiring extensive manual input. SoulSync addresses these issues by tracking live and historical health data, offering personalized insights and trends, wellness recommendations, and enabling emotion sharing with contacts. SoulSync’s core innovation lies in combining passive data with user input and activities, to provide a deeply personalized mental health experience.
https://www.youtube.com/watch?v=Wqms_-Ol6AE
Introducing SoulSync
6:30 duration
SoulSync is marketed towards Apple users who want to take a data driven, yet personal approach to their wellbeing. Our app will show trends, deliver insights, and connect your emotions to others. The app will be the central point for the user, where they can log in, create a profile, and add friends. They would be able to view their music and share it to loved ones, while viewing their health data. In the app they would be able to actively set their mood that can be shown on their watch, which will also be collecting health data. Wellbeing sessions such as breathing exercises and meditation will be monitored by the EmotiBit, which can store biometric data for and give the user a live feed of how their emotions react to the activity. Integrating the user’s calendar and the weather can add in further information about possible stressors for the user.
We will be developing the app for iOS, with the capability to connect an Apple Watch and an EmotiBit. On the software side we will be working in Swift, while integrating HealthKit, MusicKit, and the Spotify API. Our backend will consist of Apple’s CoreData, and Firebase. To integrate the EmotiBit we will be running a Flask server on a Mac which will be running a Python script with a Wifi module for mobility. The script will be utilizing Brainflow for the data processing.
In terms of dependability, SoulSync is designed to take data points from sources, including hardware and software. EmotiBit is more actively involved, while iPhone and Apple Watch passively measure data. Data from iPhone and Apple Watch is aggregated through HealthKit and is available in SoulSync. This ensures data is reliably available. As for security and safety, all health data is processed locally. SoulSync operates on a minimum needed basis. Firstly, all data is opt-in from the user. All data, algorithms, and processing is done locally and offline. For SoulSync social, data is obscured and simplified before it is passed to firebase. Additionally, SoulSync social is private and will require users to follow their friends. Just like SoulSync, the user can opt-in to share as much, or as little data as they want for SoulSync Social.
Project Related Resources:
Book
R. W. Picard, “Affective Computing”, 1997, https://direct.mit.edu/books/monograph/4296/Affective-Computing
Rosalind W. Picard’s Affective Computing is one of the pioneer books on affective computing. It was published in 1997 and laid the groundwork for how emotions play a role in our thinking, and that to have a natural interaction with a computer they need to understand our emotions.
Articles
S. M. Montgomery, N. Nair, Yi-Ping Phoebe Chen, and S. Dikker, “Introducing EmotiBit, an open-source multi-modal sensor for measuring research-grade physiological signals,” Science Talks, vol. 6, pp. 100181–100181, May 2023, https://www.sciencedirect.com/science/article/pii/S2772569323000567
Introducing EmotiBit, an open-source multi-modal sensor for measuring research-grade physiological signals This article goes over what the EmotiBit is and its usage. The EmotiBit is an open source sensor that collects multiple different data points. It can transmit the data either locally or wirelessly and can be placed on different parts of the body to collect the data. Some of the information it collects are electrodermal activity, PPG, temperature, and others.
J. Gratch, G. Greene, R. Picard, L. Urquhart and M. Valstar, “Guest Editorial: Ethics in Affective Computing,” in IEEE Transactions on Affective Computing, vol. 15, no. 1, pp. 1-3, Jan.-March 2024, doi: 10.1109/TAFFC.2023.3322918.
This paper talks about the ethical implications of Affective computing practice such as privacy, bias, and societal implications. The author expands on potential risks and the need for more responsible technology design. More things to consider while working on affective computing is risk of misuse in areas such as workplace surveillance.
Websites
R. Riyam, “Exploring HealthKit: Working with State of Mind APIs,” Rudrank Riyam, https://www.rudrank.com/exploring-healthkit-working-with-state-of-mind/ (accessed Nov. 1, 2024).
This website post does a deep dive into the new State of Mind APIs in HealthKit that came with the recent iOS 18 update. The API pre-defines things such as emotion and mood, which our app can pull from. Additionally, apps can add to HealthKit and the State of Mind data.
News, Information, Updates
https://developer.apple.com/documentation/healthkit
HealthKit is at the heart of SoulSync and powers data processing, storage, mood, and a lot more.
https://www.apple.com/watch/
The Apple Watch is just one of the devices that brings in data to SoulSync.
https://www.emotibit.com/
The EmotiBit provides accurate data from a variety of sensors to SoulSync.