Abstract:
To address the challenges involved in evaluating the societal impact of humanities and social science research, this study develops an AI-enabled dynamic evaluation framework that quantitatively assesses the societal impact of academic papers. The framework incorporates five dimensions: social, cultural, economic, policy, and environmental impacts. By integrating data mining techniques with large language models, the proposed approach enables automated collection and semantic analysis of multi-source data, resulting in the development of the SI (Societal Impact) index. An empirical study is conducted using academic papers as the primary research outputs. The findings demonstrate that the SI index effectively captures signals of societal impact in humanities and social science research and reveals disciplinary differences in the pathways through which societal impact is generated. The cumulative effects of societal impact exhibit significant temporal dependence. Significant positive correlations between 2024SI and 2025SI across disciplines indicate the temporal consistency and stability of the SI index. Furthermore, the SI index shows predictive capability for the subsequent academic impact of research papers. Societal impact and academic impact are positively associated, although the strength of this relationship varies across disciplines. The study suggests that the SI index can serve as a complementary quantitative tool for peer review by providing dynamic evidence for identifying the societal impact of humanities and social science research. Future research should further develop human–AI collaborative evaluation frameworks, promote open and transparent assessment standards, and facilitate interdisciplinary sharing of multimodal data.