“You can’t solve a problem on the same level that it was created.

You have to rise above it to the next level.” – Albert Einstein

I will never forget the words of a mentor who offered me a piece of sage advice that’s reflexively become my go-to when I face a challenge today. I ask myself this one basic question:

What are we trying to solve for?

I remind myself that this fundamental line of questioning must be my point of reference. My foundation in tackling research is defined by the question of what I’m trying to solve for and exactly how I should apply bullet-proof methodology, while advancing the population’s health.

The Mission and the Advent of AI

In my experience, you can never underestimate the drive towards human innovation. We are born innovators. Its pursuit is a great motivator with its own game-changing allure that drives success. Under the current landscape, many have acknowledged AI as an innovation that “will change the world” (Bill Gates; https://www.cnbc.com/2023/02/10/bill-gates-says-ai-like-chatgpt-is-the-most-important-innovation.html).

In an attempt to make sense of AI which is so often presented as a solution in the systematic review process, I recently read a compelling interview with a faculty expert on AI at the University of Virginia (https://news.darden.virginia.edu/2023/06/01/roshni-raveendhran-explores-the-psychology-of-technology/). Dr. Raveendhran considered what people can learn from their interactions with technologies and how people can leverage technologies to empower themselves “to provide their best work and bring their full self to work.” Her emphasis on the relationship we have with technology is a critical consideration to me as I begin to understand and optimize AI for systematic reviews.

My Framework: AI and Its Potential on My Beloved Methodology

As we consider the application of AI in systematic reviews, several key issues are important considerations for any systematic review team:

1. As evidence bases rapidly grow, the potential burden of screening grows.
2. With the vastness of so many evidence bases, review teams have to be sufficiently equipped to execute. Supporting these larger reviews demands more training and funding.
3. The ripple effect of such growth with corresponding review burden has associated implications with quality of the review process at the top.

With such challenges offered by many systematic review platforms, I firmly believe that anyone contemplating a systematic review using AI should be aware of several key, counterbalancing factors.

Balancing the Relationship between AI and the Systematic Review Methodology

Applying my understanding of a human-centric perspective, thanks to Dr. Raveendhran’s inspiration, I have identified the following guiding principles in my pursuit of technology-assisted systematic review methodology:

1. AI is a technological innovation that serves to inform me, but not to replace me or my work.
2. AI requires more complexity than I have knowledge of, but I know it requires training from humans. Therefore, like any human-to-human relationship, you ultimately get what you put into it.
3. The systematic review methodology was never defined to be easy or fast; therefore, AI should not be considered a complete solution to make things easy or fast.

Instead, AI is a solution that can inform a review team on the quality of their work and provide guidance to increase efficiency while sustaining the human-drive quality that is required by the systematic review methodology. But several methodological considerations should be considered when any systematic review team contemplates the compelling benefits of AI, including:

1. BIAS: If AI is trained on biased data, there is a risk of perpetuating inherent biases in the evidence base. This is exactly contradictory to what the methodology intended to do. The power is in the hands of the human reviewer to minimize this risk.
2. ALGORITHM TRANSPARENCY: The ‘black box’ of AI algorithms is not apparent to me, or many others. Therefore, the guiding principle of transparency cannot be ignored.
3. REPRODUCIBILITY: The methodology was defined to be reproducible and AI should not change that. Again, AI is an aid to reviewers in many ways but should not be identified as a replacement.
4. QUALITY CONTROL: AI is not an auto-pilot solution for systematic reviews. Even with the support of AI, the methodological process must be actively guided by a human to ensure the methodological quality expected with the gold standard of evidence synthesis.
5. HUMAN-AI COLLABORATION: Just like any relationship, the collaboration between AI and humans in the systematic review process can be valuable if both division of effort and interaction is consistent and thoughtful.
6. LACK OF EXPERTISE: A lack of expertise in AI coupled with a lack of expertise in the systematic review methodology has to be avoided to make this collaboration successful. I believe this to be most important consideration.

Boundaries must be respected and we have a responsibility to maintain awareness of the boundaries of the systematic review methodology, why it was so specifically prescribed, and what the methodology was intended to solve.

In the end, AI is a tool that may complement our work by allowing us to take advantage of massive new gains in computational powers and data analytics on a scale never before possible. However, AI is not a solution that should be expected to adhere to the strictest methodological standards that should be our constant benchmark of scientific reference. In my estimation, within the unbiased framework of systematic review methodology, the heaviest lifting of all must still rest on the shoulders of human intelligence and effort, but the potential for profound collaboration is undeniable.