Residency Intelligence: AI-Powered Medical Residency Application Evaluator

Project Aim

Residency Intelligence aimed to eliminate manual work for doctors in evaluating residency applications submitted through the ERAS® System in the US. By leveraging large language models (LLMs), the system transforms the evaluation process, which was previously based on simple and often unjust criteria like word count and keyword presence.

Visit Residency-Intelligence.com to explore our AI-powered evaluation platform.

My Role

As the solo developer of Residency Intelligence, I was responsible for designing and implementing the entire backend system using Golang and PostgreSQL. Specifically, I:

• Gathered necessary information and algorithms from doctors to understand the evaluation process
• Developed a rules-based algorithm to enhance the accuracy of the evaluations
• Took measures to ensure the consistency and reliability of the evaluation results, given the high stakes involved
• Integrated the system with both OpenAI and Ollama models to support online and offline evaluations

Description & Technologies

Residency Intelligence was developed using the following technologies:

• Backend: Golang, for robust and efficient server-side performance
• Database: PostgreSQL, for secure and reliable data management
• Models: Compatibility with OpenAI and Ollama models, allowing flexible evaluation options

Key features include:

• AI-powered analysis for unprecedented accuracy and consistency
• Sophisticated rules-based algorithm derived from medical professionals
• Dual model support for both online and offline evaluations
• Multiple validation processes to ensure reliable results
• High-performance backend that handles large volumes of applications

Challenges

One of the main challenges was the responsibility of determining people's futures through automated evaluations. To address this, I implemented extensive measures to guarantee the accuracy and consistency of the results. This included testing the system across different models and ensuring each request was processed multiple times to validate results.

Outcome

Residency Intelligence successfully automated the evaluation of residency applications, eliminating subjective and manual errors. It significantly improved the accuracy and fairness of the evaluation process, making it a valuable tool for both doctors and applicants. The platform reduces evaluation time by up to 90%, allowing medical professionals to focus on strategic decision-making.

Key Aspects

• Time Efficiency: Reduces evaluation time dramatically, freeing medical professionals for more valuable tasks
• Enhanced Accuracy: Sophisticated algorithms ensure comprehensive evaluations beyond simple metrics
• Consistent Results: Eliminates human biases and fatigue factors that can affect manual evaluations
• Seamless Integration: Works with existing electronic application systems with minimal workflow changes
• Dual Model Support: Flexibility to use both OpenAI and Ollama models for various needs

Technology Stack

• Golang: High-performance server-side infrastructure for efficient processing
• PostgreSQL: Enterprise-grade data management ensuring security and reliability
• OpenAI & Ollama: Integration with state-of-the-art language models for superior evaluation capabilities

Final Thoughts

Residency Intelligence represents a significant advancement in evaluating medical residency applications by combining advanced AI technology with a fairness-focused approach. It sets a new standard for accuracy, consistency, and equity in the medical residency application process, ultimately helping institutions select candidates based on true merit rather than superficial metrics.