AI-Powered Medical Residency Application Evaluator

Project Aim

The project aimed to eliminate manual work for doctors in evaluating residency applications submitted through the ERAS® System in the US. By leveraging large language models (LLMs), the system aimed to increase the accuracy of the evaluation process, which was previously based on simple and often unjust criteria like word count and keyword presence.

My Role

As the solo developer, I was responsible for designing and implementing the entire backend system using Golang and PostgreSQL. Specifically, I:

• Gathered necessary information and algorithms from doctors to understand the evaluation process.

• Developed a rules-based algorithm to enhance the accuracy of the evaluations.

• Took measures to ensure the consistency and reliability of the evaluation results, given the high stakes involved.

• Integrated the system with both OpenAI and Ollama models to support online and offline evaluations.

Description & Technologies

The system was developed using the following technologies:

• Backend: Golang, for robust and efficient server-side performance.

• Database: PostgreSQL, for secure and reliable data management.

• Models: Compatibility with OpenAI and Ollama models, allowing flexible evaluation options.

Challenges

One of the main challenges was the responsibility of determining people's futures through automated evaluations. To address this, I implemented extensive measures to guarantee the accuracy and consistency of the results. This included testing the system across different models and ensuring each request was processed multiple times to validate results.

Outcome

The Residency Application Evaluation System successfully automated the evaluation of residency applications, eliminating subjective and manual errors. It significantly improved the accuracy and fairness of the evaluation process, making it a valuable tool for both doctors and applicants.

Key Aspects

• Automated Evaluation: Streamlined and more accurate evaluation process.

• Model Compatibility: Flexibility to use both OpenAI and Ollama models.

• Reliability: High assurance of consistent and accurate evaluation results.

• End-to-End Solution: Comprehensive backend and database integration for seamless functionality.

Final Thoughts

This project stands as a significant advancement in evaluating residency applications by combining advanced technology with a user-centric approach. It sets a new standard for accuracy and fairness in the medical residency application process.