Limitations of Using AI Systems in the Literature Research Process

However, these AI systems have some limitations and are not recommended as the sole tool for more robust and comprehensive scientific work. Let's take a closer look at some of their limitations.

Limitation I – Inaccuracy and Hallucinations

Another limitation of using these AI systems in the research context is the potential for inaccuracies in AI-generated content. Despite their advanced technologies, they are not infallible. For example, they sometimes generate content that does not fully align with the specific requirements of a given prompt. This discrepancy shows that while the tool can generate relevant academic content, its search and synthesis algorithms do not always capture the nuanced details or desired focus of the user query accurately.

In a research article published in 2024, six AI-powered research assistants were tested for hallucinations in references. Reference hallucination occurs when an AI system fabricates or incorrectly generates citation details such as article titles, journal names, or author names that do not match real sources. In the study, all AI systems were tested with the same 10 medical prompts, each requesting 10 references per query. Elicit and SciSpace showed minimal hallucinations, while ChatGPT 3.5 and Bing exhibited critical hallucination rates. *It must be noted that some of these systems have improved their mechanisms since the article was published.

Since the introduction of ChatGPT, researchers have repeatedly noted its tendency to hallucinate sources or cite non-existent references. These errors occur not only occasionally but systematically and compromise the reliability of the tool for scientific research, where accuracy and factual correctness are crucial. Although ChatGPT and comparable systems provide fast and comprehensive overviews, the results must always be critically evaluated and checked against primary sources.

To address these challenges, various measures are being implemented. Developers improve training processes using fact-checked datasets and implement algorithms that cross-check information with reliable sources such as peer-reviewed databases. For example, the latest versions of ChatGPT now include specialized agents that specifically search for scientific content and retrieve real publication data, significantly reducing the risk of inaccurate results.

Limitation II – Limited Access to Publications

Many relevant and influential articles are not open access but are behind paywalls of scientific publishers, and access is only possible by payment. Typically, universities must have subscriptions with publishers for students, researchers, and faculty to access the articles. The Freie Universität Berlin has subscriptions with some scientific publishers (e.g., Springer, Wiley, and Taylor & Francis). Therefore, researchers at the university can access not only open-access articles but also articles behind paywalls from publishers with whom the university holds subscriptions.

AI systems rely heavily on open-access publications, which, although increasing, do not cover all fields equally. Some disciplines are better represented in open-access publications or preprints (scientific manuscripts made publicly available before formal peer review) than others. For example, fields like physics, which rely heavily on preprints, are well represented, while areas like chemistry are less covered.

Thus, AI systems that rely on open-access publications may not provide a comprehensive overview of publications on a particular topic, and many highly relevant publications that are not open access could be missed if only these AI systems are used for literature research.

Moreover, these AI systems are opaque about how they retrieve and index scientific articles. Scite.ai states that it uses open-access repositories like PubMed Central and identifies TDM articles via Unpaywall and Crossref. They also report including directories from “over a dozen” publishers and processing updates “from daily to monthly.” Elicit limits results to publications indexed in Semantic Scholar.

Limitation III – Excessive User Dependence

Since these AI systems automate various aspects of literature research, students and researchers might become too dependent on these functions. Such over-reliance could lead to a superficial understanding of the topic and, in the long term, impair the development of critical thinking and writing skills.

 

 

Limitation IV – Query in Question Form

These AI systems allow users to enter a search query in the form of a question. However, this feature could be a limitation, as researchers must ensure their questions are well-formulated. Otherwise, the quality of the results could be affected.