Keywords: smoking cessation, patient education, primary care, digital health

Background:

Artificial intelligence (AI) has emerged as a promising tool to support smoking cessation in primary care, particularly for populations underserved by traditional interventions. However, the quality of AI-generated smoking cessation advice remains understudied, especially in low-resource settings and among vulnerable groups such as adolescents.

Research questions:

This study aims to evaluate AI-generated responses to smoking cessation questions for patient education in primary care, comparing different AI programs in terms of knowledge, readability, and quality.

Method:

Ten publicly accessible AI programs were prompted in Turkish with 24 standardized, open‐ended smoking cessation questions framed as a patient consultation. Two family medicine specialists independently assessed each response’s readability using Ateşman’s Readability Index, reliability using the DISCERN instrument, and accuracy and motivational interviewing quality using a bespoke rubric and OARS (Open questions, Affirmations, Reflections, Summaries) framework. Inter‐rater agreement was evaluated via intraclass correlation. Descriptive statistics were computed for readability scores, DISCERN ratings, and accuracy grades.

Results:

All AI programs provided at least partially correct answers to all questions. The average readability score was 54.90 (medium difficulty)according to Atesman’s Index. The mean DISCERN score was 66±5.2, indicating excellent quality. Three AI programs incorporated core motivational interviewing skills. The most accurately answered question concerned e-cigarettes' harm compared to traditional cigarettes, while medication advice was least evidence-based.

Conclusions:

Free AI chatbots deliver reliably accurate and moderately readable smoking cessation advice, supporting their potential role as patient education adjuncts in primary care—particularly for individuals with at least a high school education. Further research should compare AI‐assisted versus clinician‐led interventions on smoking cessation outcomes.

Points for discussion: