References & Citations
Computer Science > Machine Learning
Title: Towards Low-Energy Adaptive Personalization for Resource-Constrained Devices
(Submitted on 23 Mar 2024 (v1), last revised 29 Mar 2024 (this version, v4))
Abstract: The personalization of machine learning (ML) models to address data drift is a significant challenge in the context of Internet of Things (IoT) applications. Presently, most approaches focus on fine-tuning either the full base model or its last few layers to adapt to new data, while often neglecting energy costs. However, various types of data drift exist, and fine-tuning the full base model or the last few layers may not result in optimal performance in certain scenarios. We propose Target Block Fine-Tuning (TBFT), a low-energy adaptive personalization framework designed for resource-constrained devices. We categorize data drift and personalization into three types: input-level, feature-level, and output-level. For each type, we fine-tune different blocks of the model to achieve optimal performance with reduced energy costs. Specifically, input-, feature-, and output-level correspond to fine-tuning the front, middle, and rear blocks of the model. We evaluate TBFT on a ResNet model, three datasets, three different training sizes, and a Raspberry Pi. Compared with the $Block Avg$, where each block is fine-tuned individually and their performance improvements are averaged, TBFT exhibits an improvement in model accuracy by an average of 15.30% whilst saving 41.57% energy consumption on average compared with full fine-tuning.
Submission history
From: Yushan Huang [view email][v1] Sat, 23 Mar 2024 18:19:02 GMT (5298kb,D)
[v2] Tue, 26 Mar 2024 11:11:49 GMT (5298kb,D)
[v3] Thu, 28 Mar 2024 15:00:04 GMT (5298kb,D)
[v4] Fri, 29 Mar 2024 16:53:58 GMT (5298kb,D)
Link back to: arXiv, form interface, contact.