Study Questions Accuracy of Melanoma Phone Apps
Use of apps may delay diagnosis and harm users, authors warn (Jan. 16)
The performance of smartphone applications in assessing the risk of melanoma is highly variable, and three of four applications incorrectly classified 30% or more of melanomas as unconcerning, according to a report published online in JAMA Dermatology.
To measure the performance of smartphone applications that evaluate photographs of skin lesions and provide the user with feedback about the likelihood of malignancy, researchers at the University of Pittsburgh Medical Center tested the sensitivity, specificity, and positive and negative predictive values of four smartphone applications:
- Application 1 uses an automated algorithm to detect the border of the lesion, although it also allows manual input to confirm or to change the detected border. The application gives an assessment of “problematic,” “okay,” or “error.”
- Application 2 uses an automated algorithm to evaluate an image that has been uploaded by the user. The output given is “melanoma” or “looks good.”
- Application 3 asks the user to upload an image to the application and then to position it within a box to ensure that the correct lesion is analyzed. The output given by the application is “high risk,” “medium risk,” or “low risk.”
- Application 4 does not use an automated analysis algorithm to evaluate images; rather, each image is sent to a board-certified dermatologist for evaluation, and that assessment is returned to the user within 24 hours.
The authors included 188 images of lesions (128 benign and 60 melanoma) in the analysis, each of which was evaluated by the four smartphone applications. The test results were recorded as positive, negative, or unevaluable.
The sensitivity of the four applications ranged from 7% to 98%; specificity ranged from 30% to 94%; positive predictive value ranged from 33% to 42%; and negative predictive value ranged from 65% to 97%.
The highest sensitivity for a diagnosis of melanoma was observed for the application that sends the image directly to a board-certified dermatologist for analysis, while the lowest sensitivity was associated with applications that use automated algorithms to analyze images.
The authors suggest that reliance on these applications — which are not subject to regulatory oversight — and not seeking medical consultation can delay the diagnosis of melanoma and potentially harm users.
“Physicians must be aware of these applications, because the use of medical applications seems to be increasing over time… The dermatologist should be aware of those relevant to our field to aid us in protecting and educating our patients,” the authors concluded.
Source: JAMA Dermatology; January 16, 2013.