Publications
Beyond Benchmarks: A Novel Framework for Domain-Specific LLM Evaluation and Knowledge Mapping
Nitin Sharma, Thomas Wolfers, Çağatay Yıldız
arXiv preprint arXiv:2506.07658, 2025
This paper addresses critical challenges in language model evaluation by introducing a deterministic pipeline that converts raw domain corpora into completion-type benchmarks without relying on LMs or human curation. The approach generates domain-specific keywords using TF and Term TF-IDF methods and constructs prompt-target pairs, providing a direct assessment of domain knowledge with low computational cost. Through mechanistic analysis, the research reveals that initial-to-mid layers handle attribute extraction while later layers focus on next token prediction, and that forgetting during adaptation begins in middle layers and amplifies in later layers.
Investigating Continual Pretraining in Large Language Models: Insights and Implications
Çağatay Yıldız, Nishaanth Kanna Ravichandran, Nitin Sharma, Matthias Bethge, Beyza Ermis
Transactions on Machine Learning Research (TMLR), 2025
This study introduces a new benchmark for continual domain-adaptive pretraining in large language models, examining how models adapt to changing data landscapes while retaining previous knowledge. The research reveals that continual pretraining consistently improves smaller models (<1.5B parameters), with larger models achieving better perplexity but smaller models showing higher sensitivity to both learning and forgetting. The findings demonstrate that semantic similarity in domain sequences enables better specialization, while randomized training domains lead to superior transfer and final performance.
Predicting Mental and Neurological Illnesses Based on Cerebellar Normative Features
Milin Kim, Nitin Sharma, Esten H Leonardsen, Saige Rutherford, Geir Selbæk, Karin Persson, et al.
Biological Psychiatry Global Open Science, 2025
This study used machine learning to test whether individual differences in cerebellar brain structure could help predict various mental and neurological conditions. The research analyzed brain imaging data from over 27,000 participants and found that cerebellar features could moderately predict autism spectrum disorder (ASD) and schizophrenia (SZ), with accuracy rates between 56-64%. The strongest predictive signals came from the posterior regions of the cerebellum - areas more strongly linked to higher cognitive functions than motor control.
Predicting Postoperative Delirium in Older Patients: a multicenter retrospective cohort study
Shun-Chin Jim Wu*, Nitin Sharma*, Anne Bauch, Hao-Chun Yang, Jasmine L Hect, et al.
medRxiv, 2024
This study analyzed data from 1,624 elderly patients (≥70 years) across five medical centers to develop machine learning models for predicting postoperative delirium (POD). Using demographic, clinical, surgical, and neuropsychological features, the models achieved strong predictive performance (AUC 0.79) before surgery, with specific cognitive tests like the Montreal Cognitive Assessment memory subdomain and Trail Making Test Part B emerging as crucial predictors. The findings demonstrate that effective POD risk prediction is possible before surgery, potentially enabling better surgical planning and postoperative care for high-risk patients.
Ketamine-induced pleasant but not unpleasant dissociation is linked to the functional connectivity profile of the posteromedial cortex
Zumrut Duygu Sen, Nitin Sharma, Lena Vera Danyeli, Lejla Colic, et al.
PsyArXiv, 2024
This study reveals that pleasant dissociative experiences during ketamine treatment can be predicted by examining the functional connections of the posteromedial cortex (PMC) in the brain, both before and during ketamine infusion. The research found that pleasant dissociation correlates with specific brain connectivity patterns, particularly between the PMC and control network regions, while unpleasant dissociation shows no such correlation - suggesting different neural mechanisms underlie these two types of ketamine-induced experiences.
Ketamine-induced ego dissolution is related to the functional connectivity reconfiguration of the posteromedial cortex
Meng Li, Nitin Sharma, Lena Danyeli, Lejla Colic, et al.
Biological Psychiatry, 2023
A single dose of intravenous ketamine improves the severity of symptoms in disorders such as depression while also introducing acute and transient ego dissolution during administration. Understanding the neural correlates of ego dissolution may help elucidate its therapeutic effect mechanism. Previous studies suggested that the posteromedial cortex (PMC) activity is related to the altered sense of self and the sense of dissociation, phenomena underlying ego dissolution.
Epileptic seizure detection using STFT based peak mean feature and support vector machine
Nitin Sharma, G Gaurav, RS Anand
2021 8th International Conference on Signal Processing and Integrated Networks (SPIN)
This research presents a novel approach for detecting epileptic seizures using EEG data through a combination of Short-time Fourier transform (STFT) and a new feature called "peak mean". The study decomposed EEG signals into sub-bands and extracted three key features - mean, sample entropy, and peak mean - achieving 100% classification accuracy in distinguishing epileptic ictal EEG signals from healthy subjects using Support Vector Machine with radial basis function (SVM-RBF). The proposed peak mean feature outperformed other commonly used features in epilepsy detection, demonstrating its potential for practical seizure detection applications.