1 10 1 https://urss.warwick.ac.uk/files/original/8fc2754163c0e086d7050b0d1bb842f5.pdf 952ec5d66fdc4bf6febc6b24ad8bc9d7 Dublin Core The Dublin Core metadata element set is common to all Omeka records, including items, files, and collections. For more information see, http://dublincore.org/documents/dces/. Title A name given to the resource Science, Engineering and Medicine Academic Paper An academic paper Abstract The abstract of the paper As the adoption of Large Language Models (LLMs) is becoming ubiquitous, understanding their pitfalls is becoming increasingly urgent. LLMs are known to hallucinate and may be subject to cascading hallucinatory generations due to their autoregressive structure and exposure bias. This paper first introduces the topic of hallucination cascade, covering relevant literature. Then an experiment is presented on the propagation of hallucination through long answer text generation, for all combinations of five LLMs - Qwen2 (0.5B), Qwen2 (7B), Llama3.1 (8B), Gemma2 (9B) and Mistral-NeMo (12B) - at five levels of temperature: 0, 0.75, 1, 1.25, 2. In a novel methodology, responses are generated by the five LLMs, then labelled by two more LLMs as hallucinatory or not on both a token and sentence level. The labelling is done with clear instructions and additional context to ensure its quality. Correlational evidence was found on both granularity levels that hallucinatory tokens are positively autocorrelated at lag 1, and that autocorrelation monotonically decreases as lag increases (token level only), supporting the cascading hallucination hypothesis. Finally, some hallucination mitigating techniques are presented, with both research and application in mind. All the code used for the empirical part of this paper (Section 3) can be found at https://github.com/LucMekouar/URSS_2024. Meta Tags Meta tags Large Language Model, LLM, Machine Learning, Natural Language Processing, Hallucination Dublin Core The Dublin Core metadata element set is common to all Omeka records, including items, files, and collections. For more information see, http://dublincore.org/documents/dces/. Subject The topic of the resource Computer Science Date A point or period of time associated with an event in the lifecycle of the resource 2024 Contributor An entity responsible for making contributions to the resource Prof Bo Chen Title A name given to the resource Investigating Hallucination Cascades in Autoregressive Large Language Models Creator An entity primarily responsible for making the resource Luc Mekouar Hallucination Large Language Model LLM Machine learning Natural Language Processing