PERFORMANCE OF GENERATIVE ARTIFICIAL INTELLIGENCE MODELS IN PUBLIC DENTISTRY EXAM QUESTIONS: A COMPARATIVE STUDY OF THE ACCURACY RATE IN COLLECTIVE ORAL HEALTH

Abstract

The advancement of generative artificial intelligence has sparked interest in its application in the health field, including dentistry. However, studies evaluating the performance of these tools in specific contexts of public dentistry, such as solving public service examination questions, are still scarce. Therefore, this study aimed to evaluate and compare the accuracy rates of three generative artificial intelligence models: ChatGPT, Gemini, and DeepSeek, in their free versions, in solving 100 public service examination questions in the area of ​​public oral health. The questions were extracted from public exam banks for dentists conducted between 2016 and 2026, covering topics such as oral epidemiology, SUS (Brazilian Unified Health System) public policies, health surveillance, social determinants, and service management. Each question was applied individually to the three models using a standardized prompt, without a history of previous conversations, and the accuracy rate was calculated considering each correct answer as equivalent to 1 percentage point. The results demonstrated that ChatGPT achieved the best performance (75 correct answers), followed by Gemini (47 correct answers) and DeepSeek (23 correct answers). The differences between all pairs were statistically significant (p < 0.001), with ChatGPT outperforming Gemini by 28 percentage points and DeepSeek by 52 percentage points. It is concluded that, among the free models tested, only ChatGPT would achieve the minimum score for approval in most public examinations for dental surgeons in collective oral health, while Gemini and DeepSeek, in the versions evaluated, did not prove to be reliable tools for this purpose.