Does Seed Matter?: Investigating the Effect of Random Seeds on LLM Accuracy

Author(s)	Praneeth Vadlapati
Country	United States
Abstract	The recent rapid advancements in Natural Language Processing (NLP) include Large Language Models (LLMs), which have transformed the landscape of NLP. However, a challenge exists in understanding the factors that affect model accuracy and consistency. Randomseed values are frequently used to maintain consistency in the responses. This research investigates the impact of the usage of random seeds on the accuracy of LLM responses to challenging questions. The study investigates the potential for seed-specific hallucination patterns across various types of questions. The findings of the experiment have demonstrated a considerable amount of variability across multiple questions using different seeds. The experiment discovers a new factor that affects the reliability of LLMs. The discovery is crucial for mission-critical applications that use LLMs since accuracy and consistency are of high importance.The source code is available atgithub.com/Pro-GenAI/PromptSeed.
Keywords	Large Language Models (LLMs), hallucinations, prompt engineering, random seed, accuracy, Natural Language Processing (NLP)
Published In	Volume 14, Issue 3, July-September 2023
Published On	2023-08-06
Cite This	Does Seed Matter?: Investigating the Effect of Random Seeds on LLM Accuracy - Praneeth Vadlapati - IJSAT Volume 14, Issue 3, July-September 2023. DOI 10.5281/zenodo.14288248
DOI	https://doi.org/10.5281/zenodo.14288248
Short DOI	https://doi.org/g8txsc

About IJSAT Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us	Message on WhatsApp	+91-9687-182-185	editor@ijsat.org

International Journal on Science and Technology