LSM-2: Learning from incomplet... Note

LSM-2: Learning from incomplete wearable sensor data

Wearable devices generate vast amounts of health data, but labeling this data is expensive. Self-supervised learning (SSL) can leverage unlabeled data to learn underlying data structures. Current SSL methods struggle with incomplete data, a common issue in wearable sensor streams due to various reasons. "LSM-2" introduces Adaptive and Inherited Masking (AIM), an SSL framework that learns directly from incomplete wearable sensor data. AIM uses a dual masking approach, treating naturally occurring and artificially masked tokens equivalently. A Large Sensor Model (LSM-2) was developed using AIM, improving upon the previous LSM-1 model. LSM-2 was pre-trained on 40 million hours of wearable data from 60,000 participants. It was evaluated on tasks like activity recognition, hypertension classification, and data reconstruction. LSM-2 outperforms LSM-1 in classification, reconstruction, and predicting health metrics. AIM enables LSM-2 to handle data missingness without imputation, resulting in improved performance and robustness. LSM-2 also exhibits improved scaling across users, data volume, and model size.
CdXz5zHNQW_UBgwlKqhZf.jpeg