In a blog post published last week, Meta introduced Meditron, a suite of open-source large multimodal foundation models tailored specifically to the medical domain. Built on Meta Llama 2 and trained on carefully curated, high-quality medical data sources, Meditron has been developed through collaboration between researchers at EPFL's School of Computer and Communication Sciences, Yale School of Medicine, and humanitarian organisations like the International Committee of the Red Cross (ICRC).
The primary goal of Meditron is to provide life-saving advice and guidance in low-resource medical settings, where access to the right information at the right time can make all the difference. By compressing complex medical information into an accessible conversational interface, Meditron empowers healthcare workers to diagnose and treat patients more effectively, even in underserved areas.
In emergency scenarios, where fast and accurate medical response is critical, Meditron can provide crucial support to healthcare workers. Additionally, in low-resource settings where access to medical expertise may be limited, Meditron can assist in diagnosing and treating patients, thereby improving healthcare outcomes and reducing disparities.
Professor Mary-Anne Hartley from Yale, who co-leads the project, emphasises the importance of open-access and open-source technology in ensuring equitable access to medical knowledge. By making Meditron fully open-access and open-source, from data to weights, with clear documentation, the team aims to empower innovation in resource-constrained settings and ensure that the populations and needs of these areas are adequately represented.
One major challenge is funding, as the team has chosen not to commercialise the project to maintain the neutrality required for impartial validation. To conserve costs, experiments started on the smaller Llama 2 7B before scaling up to 70B, and the team focused on carefully curating medically validated textual documents to ensure the quality of the training data.
Another crucial aspect of the Meditron project is the Massive Online Open Validation and Evaluation (MOOVE) initiative. Doctors from around the world, particularly those in low-resource settings, are actively participating in MOOVE by asking Meditron challenging questions and critically evaluating its answers. This feedback will be incorporated into future iterations of the model, ensuring that Meditron continues to improve and adapt to the real-world needs of healthcare professionals.