tl;dr check it out at http://craigcarlyle.me

A Quick Intro

Several years ago I started on a weight loss and health journey. I was introduced to the quantified self community with Fitbit and MyFitnessPal. After losing over 50 pounds, I was hooked on the idea of personal data tracking.

Fast-forward to the present and I’m tracking more than ever. Not because I believe in optimizing every aspect of my life, but because it helps me balance it. Haven’t exercised in a few days? Time to hit the gym. I’ve been eating 4,000 calories a day this week? I should eat clean for a while. Haven’t meditated or read in a while? Let me grab my Kindle.

Lots of fantastic tracking tools are available for mobile devices (Gyroscope has been my personal favourite), but I wanted something for the desktop/iPad where I could see more data at once over a longer period of time. I couldn’t find anything that did what I wanted, so I built a solution myself. You can see it here.

I’ve set up my data (since January 2019) to appear in a calendar view. Each date will contain the data from the following sources:

Data tracked and imported from external sources

• ❤️ Heart rate & HRV via Apple Health
• ⚖️ Weight and body fat percentage via Apple Health
• 👟 Steps and distance travelled via Apple Health
• 🧘‍♂️ Meditation sessions via Apple Health
• 🔥 Calories consumed via MyFitnessPal
• 🍱 Macronutrients (carbohydrates, fat, and protein) via MyFitnessPal
• ☕️ Caffeine (coffee, tea, and pre-workout) via MyFitnessPal
• 💊 Supplement intake via MyFitnessPal
• ✅ Number of tasks completed (work and personal) via Todoist
• 🏋 Workouts via MyFitnessPal
• 🧖 Sauna sessions via MyFitnessPal
• 💆‍♂️ RMT sessions via MyFitnessPal
• 👩‍⚕️ Chiropractor sessions via MyFitnessPal
• 🛫 Places I'm travelling to via Nomad List
• 🏊‍♂️ Dives I've completed via ScubaEarth
• 🦉 Duolingo XP via Duolingo
• 👨‍💻 Productivity via RescueTime
• 🐙 GitHub Contributions via GitHub
• 📖 Reading progress via Goodreads

Data entered manually into a Google Sheet

• 🆙 How long I'm using my standing desk
• 🔬 Thyroid-stimulating hormone (TSH) results
• 🏆 Accomplishments
• 🤒 Sickness
• 🤕 Injuries
• 🌴 Vacation
• 📝 Bucket list

Fetching the data (the technical stuff)

From Apple Health

In iOS 12, Apple added Siri Shortcuts which allows for some powerful scripting. I wrote a script that goes through my Health Samples, formats the data as valid JSON, then saves the file to my server via SSH. Unfortunately at the time of writing, Apple doesn’t allow these shortcuts to be triggered automatically, so I usually run them manually when I arrive at work in the morning.

From MyFitnessPal

I could have leveraged the same Siri Shortcuts to upload my nutritional data, but MyFitnessPal is notorious for having missing or duplicated entries when synced with Apple Health.

MyFitnessPal also has an invite-only API. I’ve been on a waiting list for years, but have never heard anything.

I came across this mfp module which loads a user’s printable diary and scrapes the data from there using cheerio. I’ve modified it to scrape exercise, caffeine consumption, supplements, and RMT sessions. I collect this data every 10 minutes with a Node.js task run with Heroku Scheduler and save it on my server as a JSON file.

Leveraging IFTTT

Conveniently, Google Sheets are importable as JSON objects. IFTTT has plenty of applets that add a new row to a spreadsheet when triggered by an action. A new row is created whenever:

• I mark a task as complete on Todoist
• My reading progress is updated on Goodreads
• A daily productivity summary is created by RescueTime

No APIs, No Problem

Recently I’ve fallen in love with scuba diving. I use ScubaEarth to track my dives, but unfortunately there’s no API. Again using Heroku Scheduler, I launch an instance of Puppeteer to log in to my account and collect the page’s data from my logbook. The data is parsed into a JSON object using cheerio and then uploaded to my server.

Duolingo has an API, but it’s undocumented and (as far as I can tell) doesn’t have exactly what I need. Using the same Puppeteer instance, I go to my duome profile and collect my recent XP. The new XP JSON objects are added to the existing array of XP objects and uploaded as a file to my server.

The Easy Stuff

Pieter Levels of Nomad List is awesome and allows anybody to access their data as JSON. All that’s required is a simple fetch of https://nomadlist.com/@craigcarlyle.json.

Pushing Further

I thought it’d be cool to map out my travels and dives, so I added another page that does that. The data is listed and visualized with Mapbox.

I also thought it’d be cool to share my bucket list. This is just a simple Google Sheet that’s imported as a JSON object.

Final Thoughts

Although it’s far from elegant this has been a fun project to hack away on. Not only have I built an app that I needed in my life, but it helped me get over my imposter syndrome (something we all have). One of my goals was to teach myself React as I’ve mostly been working in AngularJS and vanilla JavaScript/TypeScript for the past 6+ years. Although it’s not perfect, I’m now comfortable working with React.

At any point I could have said “oh, this is too hard", or "there’s no API so it can’t be done”. Finding creative solutions (like using Puppeteer to scrape data) has helped boost my confidence as a developer.

If anyone has any questions, please don’t hesitate to reach out. I’d be happy to answer them. Thanks for reading!