Journalists Club – Nursing Education Network

This journal club reviews the newest evidence on AI-powered qualitative evaluation when it comes to what it enables, where it fails, and learn how to use it responsibly. Can artificial intelligence significantly support qualitative research without compromising interpretive depth?

: Cook, D. A., Ginsburg, S., Sawatsky, A. P., Kuper, A., & D’Angelo, J. D. (2025). Artificial intelligence supporting qualitative data evaluation: guarantees, approaches, pitfalls. Academic medicine10-1097.

: The authors critically assess how AI can support qualitative data evaluation through three activities:

1. exploratory case study using ChatGPT-4 on three narrative datasets
2. an outline of the scope of qualitative data evaluation supported by artificial intelligence
3. conceptual evaluation of guarantees, pitfalls and ethics.

: :

• Ready-to-use prompts provided accurate, short summaries but weren’t effective for higher-order tasks (e.g., thematic evaluation, cross-topic observations); after iterative prompting, a tool for counting and summarizing keywords emerged, while several tasks remained unsatisfactory.

• Of these, 104 were original research; acceleration of publication in 2023–2024 (n=64). Common approaches included inductive topic/theme discovery (n=70), keyword discovery (n=39), rubric-based coding (n=30), sentiment evaluation (n=28), and discourse evaluation (n=13). Many studies used unsupervised learning (n=75), with frequent use of natural language processing, pre-trained transformers, and other neural methods.

• : Computer-aided qualitative data evaluation predates current large language models by many years (e.g., NVivo/ATLAS.ti/MAXQDA features equivalent to search, word frequency, encoder compatibility). The latest wave of AI is increasing accessibility and scope relatively than inventing qualitative data evaluation.

: AI can speed up transcription and translation at human-competitive quality; support purposeful sampling and large-scale corpus evaluation; data cleansing assistance; and facilitate developer training and collaborative workflows between people.

: The article highlights the necessity for a “man in the loop”. AI tools, especially multilingual models, match probabilistic patterns, not create meaning; they risk missing rare/nuanced phenomena, hallucinations, and repetitive bias. Plug-and-play analyzes by insufficiently trained users pose risks, potential erosion of collaborative conclusions, and privacy and security issues. Limited context windows and incomplete knowledge also limit performance.

: Researchers need to know what’s under the hood, be reflective about how AI shapes findings, and be transparent about how AI is utilized in data processing and evaluation.

: :

1. Apply AI where it’s clearly strong (transcription, translation, summarization, retrieval).
2. Reserve interpretive/thematic findings for trained qualitative researchers using audit trails and triangulation.
3. Plan for ethics, privacy and model bias; document AI roles and parameters and maintain rigorous coder calibration and reflective practice.

: Artificial intelligence has a protracted history of supporting qualitative data evaluation, and offers powerful but limited capabilities because of modern large-scale language models and natural language processing. Its value is maximized when embedded in a careful qualitative methodology under explicit human supervision.