Many monogenic disorders cause a characteristic facial morphology. Artificial intelligence can support physicians in recognizing these patterns by associating facial phenotypes with the underlying syndrome through training on thousands of patient photographs. However, this ‘supervised’ approach means that diagnoses are only possible if the disorder was part of the training set. To improve recognition of ultra-rare disorders, we developed GestaltMatcher, an encoder for portraits that is based on a deep convo
GestaltMatcher enables clinicians to match patients with facial similarity and thus, possibly diagnose patients with an ultra-rare disorder, or delineate a new syndrome in similar patients. This new algorithm is available as a web service and more details about the method can be found in our Nature Genetics paper . Furthermore, we provide code snippets and data for reproducing our results. The data portal GestaltMatcher DB can also be used to query for any medical images by gene or disorder.
See also Hellen Lesmann's talk about GestaltMatcher Database.