INTRODUCTION
HeRVY is a proof-of-concept research project regarding tracking heart rate data to determine anxiety levels, based on a mobile application that would allow users to track their anxiety levels over time and tell when the user is anxious. The target audience would be vulnerable persons who have trouble communicating their feelings or needs, or those who want to track their ongoing medical professional help.
Tracking anxiety attacks is hard to remember and is not something that is often remembered to do in the moment. This application would allow for passive data collection and allow users to monitor their data over time in an easy to view format.
There have been many studies that have stated there is a correlation between anxiety and heart rate or heart rate variability. The main paper I used to back my research was Kunal Khanade and Farzan Sasangohar’s review from Proceedings of the Human Factors and Ergonomics Society 2017 Annual Meeting, “Efficacy of Using Heart Rate Measurements as an Indicator to Monitor Anxiety Disorders: A Scoping Literature Review”. This paper states, “Assessment of resting and overall heart rate accelerations can provide insight into intervention measures for individuals with anxiety and panic disorders including the very prevalent PTSD”.
Heart Rate Variability is usually the primary choice for studies monitoring cardiology data.
Heart Rate Variability (HRV) – “… a measure of the variation in time between each heartbeat.” HRV is controlled by the autonomic nervous system which is the part of the brain that controls the fight-or-flight mechanism or relaxation response. Ideally people will have high HRV since this correlates with a relaxed state, however there are many contributing factors.
HRV is best monitored when attached to the chest and wrist wearables have been questioned in their validity for monitoring HRV. I wanted to use heart rate variability due to the majority of studies working with HRV but on a consumer level I found two main obstacles:
Wrist wearable technology is not advanced enough
Companies who have wrist HRV measures do not have an open API
Wrist wearables such as Apple Watches and Whoop promote HRV statistics but do not allow third party testing to test the validity of their product and do not let developers gather the information via open API.
Due to these obstacles, I chose Heart Rate as the main data to analyze.
There are many more options that support heart rate and I briefly looked into using Shimmer to attempt to integrate with the majority of applications but found documentation lacking and the community empty.
The main wearables that I considered as viable options are:
DEVELOPMENT AND RESEARCH
I ended up switching between Fitbit and Garmin several times since Fitbit has an easy open source web API that suited my needs. But Garmin had a “stress” value it tracked which was based on HRV using Firstbeat technology. Garmin requires many extra steps to be done to access their API for security reasons, which I went through with the help of my Professor, Russ. The issue switching between Fitbit and Garmin was that Fitbit used OAuth 2.0 and Garmin uses OAuth 1.0 which are vastly different protocols. I managed to support both OAuth 2.0 and OAuth 1.0 but Garmin is targeted towards more professional projects and required things such as a ping/post URL to notify of new data which would require the server to be constantly up and receiving calls and a signed SSL certificate.
To avoid the costs of deploying the application through a service I went back to Fitbit since I could test it without limitations or hosting costs.
Using Fitbit meant I had to come up with my own application for detecting anxiety.
I had applied to the Research Ethics Board at Conestoga College to get approval for a research study calibrating the application but needed initial data to create the application. To create the base application, I turned to PhysioNet’s Non-EEG Dataset for Assessment of Neurological Status. This data consisted of various physical and mental states for 20 subjects. I attempted to analyze the heart rate wave data using various researching tools but had difficulty since I am not familiar with this field. Most resources were using cardiology data to detect abnormalities in the heart for uses in the medical field.
I ended up parsing the data using a WFDB format reader and Python’s Pandas in five-minute intervals. I looked over the data summaries that I collected and the raw graphs to develop a rudimentary formula for detecting anxiety.
Due to COVID-19 I was unable to properly calibrate the application but believe it proves that it should be possible given the proper resources and training.
Heart rate data is taken in through the request’s body and the data is smoothed using the Savitzky–Golay filter. The heart rate values are read in five-minute increments and information regarding the mean, mode, min, max, and variance is produced. Based on this information for the five-minute period, if the data is consistently above the resting heart rate by a set amount, the user scores an anxiety level of 1. If the data has a large spike in heart rate the user gets an anxiety level of 0.8. The anxiety levels are converted to the times they occurred and is returned with the total level.
I tested many variations and believed this variation was adequate but could still be improved with more time and test subjects.
With the anxiety detecting application finished, the pattern detecting part of the application also needed to be created. The model I created is simple, counting the occurrences of anxiety in the hour daily or weekly. I attempted to use Facebook’s Prophet but found it did not suit my data at this time. The simple pattern detecting I implemented relies on the user to analyze their own data and attempt to find trends in their life that correlate with the data.
TECHNOLOGIES
The final anxiety detecting application was created using mostly CherryPy for the web framework and Pandas for data reading. Since heart rate and heart rate variability vary between individuals based on their health background, gender, and outside factors, I believe that further iterations would need to include changing over time using user response or the accumulation of more data.
Initially for the front end I considered Android or iOS applications but as a single person supporting both operating systems would be an issue. I decided on a website which would be used on both iOS and Android operating systems, as well as the traditional computer browser. I am using Ionic Framework which ideally would use Cordova to present input and output options matching the styling of the mobile operating system being used; due to time constraints I did not get this far.
I wanted to avoid creating my own login authentication and decided to integrate with Okta to gain an easy to use authentication I could use for my system. Since Fitbit only allows the account owner to make intraday heart rate requests (daily heart rate values on a second by second basis), I only technically had to worry about one user at this time, but I thought it would be interesting to look into Okta’s authentication.
With an Okta sign in widget set up, I wanted to work on the front-end display and integration with the back-end to be able to demonstrate what the application would be like. The front-end application has room for improvement and I wish I had more time to work on it, perhaps without the issues with Garmin this might have been achievable. I may have also spent too much time researching Heart Rate and the best way to analyze the data.
The front-end app displays the last recorded heart rate from Fitbit, the time it occurred, and the last time the front end tried to sync. The data is graphed using ChartJS and the data is sent to the server to analyze if the user is anxious and the times they were anxious. This is displayed to the user in a list below the chart with the heart rate data.
The daily and weekly data is displayed in their respective tabs, being built dynamically based on the data returned from the server. An additional page was added to view old heart rate data and heart rate values.
Ideally the front-end application would allow users to select dates to view patterns between, at this time it is hard coded for a month, ending at the current day. Additionally, a tips and tricks page will be added, with users being able to customize their own page with their own tips. Documenting the user’s response to if they are anxious or not at certain times will allow for fitting the data model to their heart rate trends. The application will be able to handle several layers of users who are able to join groups for one person’s wearable, allowing guardians or the user’s support group to see their progress and if they might need help. Additional features include an emergency contact button which would call or text the contact, resource-based permissions for different types of users, localization, and detailed reports.
At this time, false positives for anxiety are shown due to increased heart rate linked to exercise. Fitbit allows for steps to be tracked which could be used to attempt to ignore exercise data. Another option would be to attempt to model the user’s heart rate when they go from resting to going up the stairs or running a short distance to see how their heart rate changes in response to exercise.
Overall, I believe this experience was a great learning experience. I believe I learned a lot from this experience and while there could be improvements, I learned quite a few new technologies and have a solid idea how the app can continue to grow.
In conclusion, I believe that biometrics are the way of the future. Monitoring your own biometrics allows for better control watching genetic conditions or response to outside factors. Technology is getting to the point where people can view information about their body without needing hospital equipment, which means there is a market for technology to help guide people with this information. Technology paired with trained professionals is the way of the future.
LESSONS LEARNED
WHAT WENT RIGHT
Fitbit
Fitbit integration is very easy and there are a lot of different resources for integrating with their different APIs. The band I am using is roughly 4 years old and suited well for this project.
CherryPy and Pandas
CherryPy and Pandas were also very easy to educate myself on. Pandas is a very powerful tool and I appreciate being able to experiment with Python’s data processing and various libraries.
WHAT WENT WRONG
Among the various issues that occur during development such as scope creep and not having enough time, here are some issues I came across.
Cross Origin Resource Sharing
I had many problems with Cross-Origin Resource Sharing (CORS). It is a mechanism using HTTP headers to communicate with browsers to give a web application running at one origin, access to another resource from a different origin. It is triggered whenever the origin is different in terms of domain, protocol, or port.
Since I was testing on localhost, I set up a proxy that allows cors-anywhere so I could freely communicate with Fitbit and Garmin.
Also, my backend python web service was running on a different port, meaning I had to handle pre-flight requests.
Garmin
Garmin only works with researchers and businesses. To really utilize their services, I would need a published domain with signed SSL certificates, which was not in my capstone budget. After struggling for two weeks without real progress, I moved on from working with Garmin. In retrospect I could have used Heroku without a custom domain, but testing would be very difficult with Garmin.
MOVING FORWARD
As I stated in the development section, the app needs to be polished, contain more of my ideal workflow, and some bug fixes.
Localization and time zones are currently an issue, but it was a complicated issue I preferred to leave alone for the prototype.
Coloured tracking for high anxiety/key patterns
User customized heart rate tracking
Some small rendering issues
User/groups/permissions support
User settings
Cloud data storage with separated heart rate data and user information for anonymity and security
Contact importing, calling/texting features
Notifications
Tips & tricks page with medical and custom user information
More research!
A cool mascot, hopefully also cute
ABOUT ME
Thanks for reading about my project! Here is a little more about me:
I am a soon to be graduate from Conestoga College’s Software Engineering Technology Co-op program. I enjoy full-stack development, with a passion for fixing bugs and working on new projects. I worked at Siemens Healthineers as both a Software Developer (8 Months) and Software Verification Specialist (4 Months).
I have loved technology and computers from a young age, building my first computer game at the age of 14 and building my first desktop at 15. In my spare time I enjoy playing video games, building computers, 3d printing, reading, baking/cooking, and watching movies.
Github: https://github.com/egoodwinx
Cambridge, ON