A new tool could transform how scientists assess predictive models. The InterModel Vigorish (IMV) quantifies how much better one model is than another when predicting binary outcomes. By integrating the mathematics of gambling, the model offers a simple and clear way to compare predictive performance across different fields.
From predicting election outcomes to diagnosing diseases, accurate models play a crucial role in decision-making. However, comparing different predictive models to determine which one performs best has long been a challenge, especially when results vary across different contexts.
The IMV overcomes this complexity and difficulty in comparing models by providing a standardised way to measure which models are better for different binary outcomes (such as a positive or negative result). In an article published in PLOS One, an international team of researchers outline how IMV estimates the extent to which how much more accurate one model is over another when dealing with binary outcomes.
Ben Domingue, Associate Professor at Stanford University’s Graduate School of Education, explains ‘Binary outcomes matter because they turn complex predictions into clear yes/no or success/failure decisions like determining whether a patient will develop a disease, or if someone will qualify for a mortgage. The IMV builds a measure that tells you not just which model is better, but by how much.’
The study demonstrates the effectiveness of using the IMV to quantify predictive improvements in health, social, and physical fields. Using data from the Health and Retirement Study (a survey of over 37,000 individuals over age 50 in the USA), the authors were able to quantify through the IMV how simple demographic factors like age and a person’s grip strength added a small amount of value when predicting mortality, while detailed medical information significantly improved breast cancer predictions.
Charlie Rahal, Associate Professor at the Leverhulme Centre for Demographic Science and Oxford Population Health’s Demographic Science Unit, said ‘The IMV offers a valuable addition to existing tools, providing a flexible and intuitive way to communicate predictive gains in a way that allows researchers and decision-makers to compare models effectively across problems.’
The team hopes the IMV will become a widely used benchmark for predictive modelling. With replication code available in R, Python, and MATLAB, researchers across different fields can apply it to problems ranging from predicting children’s life course outcomes to making election forecasts.
