The AI-based application capable of recognising various medications, developed by the staff of the University of Pécs Faculty of Pharmacy, could make hospital medicine delivery and quality assurance a lot easier and efficient. The development was recognised with the UP Research Innovation Award, and it has been tested in a clinical environment. Right now, the hardware prototype development and training the algorithm to recognise more medication is in progress.
Written by Miklós Stemler
At the moment, the application developed by dr. Amir Reza Ashraf assistant professor and his team can recognise 30 types of orally administered medications. They hope that in a few years, the application could provide significant support in monitoring hospital medication distribution, making it more efficient and safe. The Pécs development is a good example of using artificial intelligence in a way to support human work instead of replacing it and also of developing an application useful in the day to day life, by determining achievable goals.
Although there have been many attempts of developing various AI medicine recognition applications since the spread of image recognition algorithms, these all fell into the same trap: it is simply impossible, or incredibly difficult to create a model that could identify tens of thousands of medications with complete security, considering that this number is constantly growing with the development of new pills. Dr. Amir Reza Ashraf has had to face this same problem when he started to deal with this topic a few years ago at the start of his PhD studies.
Eliminating mistakes
“One of the research topics of the Department of Pharmaceutics was illegal medicine distribution and online medicine distribution, and about six years ago we started to make advances in distinguishing real and fake medicine by object recognition. My colleagues who were dealing with this made hundreds of images, and the AI model trained on these showed adequate results, but it turned out that a huge amount of data would be needed, and the project was shelved. I have been interested in artificial intelligence for a long time and when my thesis consultant, dr. András Fittler mentioned the previous attempt, it roused my curiosity. I started to dig into the topic and started working on it” – remembers dr. Amir Reza Ashraf.
The researcher has decide at the start of the project that he will focus on an application usable in a clinical environment, which has to bug advantages: one, the number of medications used in hospitals is limited, therefore adding all of the data is not an unattainable goal, and second, there is a real demand for such an application.
“I was thinking of something to support the work of hospital workers even when first coming up with the idea. We all know that hospitals are hectic, there are new patients coming in constantly, and it is important that everyone gets the right dosage of the right medication. Medication dosage automats are a good solution, but these are mostly available in larger hospitals. Therefore it would be important for workers of smaller institutions or departments to have a device that could help eliminate those rare, but internationally documented cases where a patient does not get the medication prescribed to them”.
Truly in short supply
The first, rudimentary version of the application was able to recognise 10 types of medication, which proved that the idea was tenable. It also showed that there was a significant international interest in it as well.
“I sent an abstract to the 2022 Vienna conference of the European Association of Hospital Pharmacists (EAHP) just to test the waters, and to my surprised they informed me that I was nominated for an award and also asked me for a presentation. That is when I realised that while there are many similar attempts, there were none that uses the medication list of a specific clinic and supply system, like ours does. This was a huge boost.”
The next step was the further development of the algorithm, tripling the medications it recognised, and the enthusiastic researcher also developed a mobile app, tested in the UP Clinical Centre, the Kaposi Mór Teaching Hospital in Kaposvár and the Health Centre in Komló. This was followed by a publication in the renown journal Artificial Intelligence in Medicine, and the university innovation application, where its success brough the completion of the application within arm’s reach.
“After getting my publication accepted, dr. András Fittler encouraged me to join the innovation application, where as a happy coincidence, the topic this year was artificial intelligence. We handed in the Dr. Ferenc Jakab Proof of Concept (PoC) application that granted us 10 million Hungarian forints for the realisation of our project”.
An extra tool in the hands of healthcare workers
Dr. Amir Reza Ashraf hopes that the 10 million HUF funding will allow the application to get its own hardware with an integrated camera system to improve performance, and also help the further training of the model to take it closer to the point of practical usability.
“The current mobile application is user friendly, but the efficacy of image recognition is restricted by the camera of the phone. We are working on a 3D printed, integrated camera prototype where the pills would be put into the device. We are also developing the model, the goal is for it to recognise 80 medications instead of the current 30. This is a large enough number to cover the list of medications used in smaller hospitals and departments, therefore for tis point onwards, we could test and develop the application in a real life environment.”
According to the plans, they have 18 months for this stage, after which they can start looking for potential industry or professional partners. Dr. Amir Reza Ashraf is already very satisfied with their results.
“I was very sceptical at the beginning, and seeing for the first time how the application differentiates between two almost indistinguishable pills was a real revelation. It is also important for me that we are not taking the jobs professionals, but only assisting the work of department pharmacists, doctors and nurses; if all goes to plan, we are only giving an extra tool to healthcare professionals.”
Photo:
Szabolcs Csortos/Univ Pécs