References & Citations
Computer Science > Computation and Language
Title: Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modeling
(Submitted on 16 Apr 2024 (v1), last revised 23 Apr 2024 (this version, v2))
Abstract: Researchers must stay current in their fields by regularly reviewing academic literature, a task complicated by the daily publication of thousands of papers. Traditional multi-label text classification methods often ignore semantic relationships and fail to address the inherent class imbalances. This paper introduces a novel approach using the SciBERT model and CNNs to systematically categorize academic abstracts from the Elsevier OA CC-BY corpus. We use a multi-segment input strategy that processes abstracts, body text, titles, and keywords obtained via BERT topic modeling through SciBERT. Here, the [CLS] token embeddings capture the contextual representation of each segment, concatenated and processed through a CNN. The CNN uses convolution and pooling to enhance feature extraction and reduce dimensionality, optimizing the data for classification. Additionally, we incorporate class weights based on label frequency to address the class imbalance, significantly improving the classification F1 score and enhancing text classification systems and literature review efficiency.
Submission history
From: Hamsini Sankaran [view email][v1] Tue, 16 Apr 2024 05:21:47 GMT (590kb,D)
[v2] Tue, 23 Apr 2024 05:15:18 GMT (589kb,D)
Link back to: arXiv, form interface, contact.