SleepLens: Turning Multimodal Sleep Data Into Conversations With Databricks Genie

Chais4140 — Tue, 01 Sep 2026 02:26:45 GMT

The Problem

Sleep trackers collect a lot of information, including heart rate, movement, sleep stages, and wake events. But having more data does not always mean people understand their sleep better. A user might see a Sleep Quality Score of 72.7 and still want to know why it was 72.7. SleepLens was built to explore how Databricks and Genie can make multimodal sleep data easier to understand by letting users ask questions about their sleep in natural language and receive answers grounded in their underlying data.

What Is SleepLens?

SleepLens is an end-to-end multimodal sleep intelligence application built on Databricks. The prototype uses real wearable heart-rate and accelerometer or movement recordings paired with EEG-derived sleep-stage labels. From these signals, SleepLens creates analytics for sleep duration, sleep efficiency, sleep quality, wake events, deep sleep, REM sleep, and physiological and movement-based model features.

How I Built It

SleepLens was designed as a full data-to-AI workflow rather than just a machine-learning model. The overall pipeline moves from raw wearable data into Databricks, then through Bronze, Silver, and Gold layers, followed by machine learning with MLflow, the SleepLens app, and finally a Genie Agent.

Multimodal Sleep Data

The foundation of SleepLens is multimodal physiological data. The prototype combines heart-rate and movement recordings with EEG-derived sleep-stage labels so that multiple signals can be analyzed together. The current prototype also includes simulated contextual variables such as room temperature and sound, and these are not presented as real sensor measurements.

Building the Data Pipeline With Databricks

I organized the data using a Bronze, Silver, and Gold architecture. Bronze stores the raw sleep and wearable data. Silver cleans and transforms those recordings into more useful sleep features. Gold produces application-ready analytics that can be used by the dashboard, machine-learning workflow, and Genie. The final Gold layer includes information about sleep sessions, wake events, and model factors.

Adding Machine Learning

Once the sleep data was structured, I built a machine-learning workflow using physiological and movement features. I used MLflow to track the model and its performance. The resulting model achieved approximately 51.34 percent balanced accuracy. I also incorporated feature importance into the Gold analytics layer so the application could show which signals were contributing to the model's predictions.

The SleepLens Application

I used Databricks Apps to turn the pipeline and model results into an interactive application. The app shows metrics such as Sleep Quality Score, Sleep Duration, and Sleep Efficiency, and it also lets users compare different nights. In the current prototype, one night had a Sleep Quality Score of 76.5 while another had a score of 72.7, which makes it easier to see differences in sleep duration, efficiency, wake events, deep sleep, and REM sleep.

Genie at the Core of SleepLens

The main goal of SleepLens is not just to show sleep data, but to help users understand it. I created the Sleep Analysis and Factors Genie Agent and connected it to the SleepLens application. Genie has access to the Gold analytics tables, including gold_sleep_sessions, gold_wake_events, and gold_model_factors. This lets users ask questions such as which night had better sleep quality and what factors likely contributed to the difference.

Why Genie Made a Difference

Without Genie, SleepLens can tell someone their Sleep Quality Score. With Genie, the application can help them investigate why that score was higher or lower. Instead of requiring users to understand SQL, schemas, tables, feature importance, or model outputs, they can simply ask a question and receive an explanation grounded in the SleepLens data.

Why Databricks Was Useful for SleepLens

Databricks allowed the entire project to live within one ecosystem. I was able to move from raw multimodal wearable data to data engineering, Bronze, Silver, and Gold analytics, machine learning, MLflow tracking, model explainability, an interactive app, and conversational analytics with Genie. This made it possible to build the data pipeline, model, application, and conversational layer as one connected system.

Architecture

The final SleepLens architecture starts with wearable heart-rate and accelerometer data paired with EEG-derived sleep labels. That data moves through the Bronze, Silver, and Gold layers, then into the machine-learning and MLflow workflow, then into the Databricks App, and finally into the SleepLens dashboard and Genie Agent for natural-language sleep insights.

What I Learned

The biggest lesson from building SleepLens was that an AI application is much more than the model itself. The data needs to be cleaned and structured correctly, the model needs to be evaluated, the results need to be understandable, and the application needs to make those results accessible. Building SleepLens showed me how data engineering, machine learning, application development, and conversational AI can work together as one system.

What's Next for SleepLens?

SleepLens is currently a prototype and is not intended to provide medical diagnoses or medical advice. In the future, I would like to explore streaming data from real wearable or edge devices, adding more physiological sensors, replacing simulated environmental context with real temperature and sound sensors, expanding the dataset across more users and nights, improving personalization, exploring on-device inference, and giving Genie access to longer-term sleep trends.

My longer-term vision is for SleepLens to become an intelligent interface between wearable sensor data and human understanding.

Main SleepLens Page

Ingested Raw Data

Medallion Architecture

ML model performance

Genie's Impact

topic SleepLens: Turning Multimodal Sleep Data Into Conversations With Databricks Genie in Community Articles

SleepLens: Turning Multimodal Sleep Data Into Conversations With Databricks Genie