At Epic Systems this summer, I worked with 5 other interns to build a Siri-like system in medical context.
The idea was that doctors need a more convenient way to access patient information when they’re not sitting at their desk. There was an existing iOS app called Haiku developed for the doctors, but it was relatively hard to navigate and required a lot of effort to get patient data. Our project was to reduce the effort of doctors by building a personal assistant in the Haiku app that is capable of handling voice commands such as”What does my schedule look like?” or “Prescribe 20mg of Aspirin for him (the patient)”.

Figure 1. The Haiku app in which our personal assistant was integrated

I worked with Nathan, another intern from Northwestern, on the iOS UI part for the first two months while the other team members worked on implementing machine learning algorithms and sentence parsing. We created a simple and intuitive UI that fits the design of the original app so that the doctors will know immediately how to use it. We also tested embedding Dragon Dictation in our personal assistant but it was not accurate enough to be used in our case.

After presenting personal assistant to the rest of the company and interns, many interns who arrived earlier than us left. So Nathan and I took over and moved on to improve algorithms for the last month. My other teammates have built a naïve bayes classifier and I focused on trying to find out why it failed on some cases. I improved it so that it can recognize multiple types of medications as a single entity using medication tagger.

At the end of our internship, Nathan and I also added another feature to predict the next word when the doctor is typing a note. I collected some sample notes and built a n-gram model for prediction. I also encorporated the medication tagger so that when the algorithm predicts the doctor is going to type a medication,it will provide customized choices of possible medication based on the patient’s medical history.

 

Some of the challenges we’ve faced are:

1. Not enough training data.
2. Learning Objective-c and Cache programming language in a short period of time and immediately start programming with them.
3. Think outside of the box and solve problems not mentioned in the original n-gram algorithm, such as an undefined (almost infinite) vocaburary size.

 

P.S. Due to a non-disclosure contract I signed, I cannot give out example pictures of what the app looked like.