Crowdtesting for Mobile Healthcare
March 24, 2014
For generations, innovations that created direct impact and value were personified by lone geniuses, the likes of Thomas Edison and Steve Jobs toiling away in their laboratories. Today’s reality, however, paints a different picture; the confluence of political, economic, social, and technological forces forge a diverse, and often complex ecosystem that shapes, fosters and demands innovation. The medical technology field, especially in the developed economies, long profited from the convergence of forces that spurred innovations, such as prosthetic limbs, hearing aids, imaging technologies, and less-invasive cardiovascular procedures, which have reduced recovery times and greatly improved healthcare outcomes.
However, according to PricewaterhouseCooper’s (PwC) report titled, “Operating performance in the Medtech industry: Trends and imperatives,” Research and Development (R&D) activities are not generating as much value and growth as they historically did. PwC’s study of 56 global medical technology companies revealed that the impact of R&D on revenue growth declined at an average annual rate of 10 percent between 2005 and 2011. These companies compensated for this decline through cost cutting measures and by increasing operational efficiencies.
The most significant challenge faced by the medical device industry is that the fundamental nature of innovation has changed dramatically. It was a world built on incremental innovations with hardware focus. The notion that the players could demand price premiums for these innovations is slowly disintegrating. This phenomenon is accelerated by the emergence of modular systems that complement hardware ubiquitously available among healthcare stakeholders and by shift to software-centricity.
The adoption of mobile technology at breakneck speed has also transformed the healthcare industry, creating an area of innovation — Mobile Healthcare (mHealth), fueled by numerous mobile applications developed and released by companies and developers alike for general use by lifestyle consumers, patients, and healthcare professionals. In 2012, the number of medical application users reached 247 million and the global revenue from mHealth apps grew to $1.3 billion, and continues to grow on an upward trending curve.
With mobile applications, regular smartphones could be easily converted into an effective healthcare platform that patients come to rely on.
Disease symptoms and medication side effects can be easily and progressively tracked, logged, and electronically shared with healthcare practitioners. Gathering diagnostic data, such as blood pressure, heartbeat rate, and even much more complex kinds of diagnostics like antibiotic resistance and eye diseases, can be automated. Appointments with healthcare specialists can be scheduled with a single click and these meetings improve in quality as patients could systematically record and share their concerns with these experts before a physical visit. In case of physical discomfort, sickness or diseases, patients could browse through a comprehensive FAQ database through a mobile device to see available medications and options before physically consulting a practitioner.
In extreme scenarios, as in cancer, patients could gain access to a community of patients, where they could discuss their problems, ask questions, get inspiration from success stories, participate in a information exchanges, gain moral support, and stay positive, knowing that they are not alone in their battle for cure. Healthcare organizations could curate knowledge from diverse information sources and provide structured access to healthcare consumers where and when they need it. Pharmaceutical companies could use mobile channels to disseminate information, organize webinars, work directly with patients, and so on. The possibilities are boundless.
Surveys reveal that mobile applications have become increasingly important to both, patients and doctors alike. Research conducted by Dutch physicians showed that over 60 percent of doctors use medical apps on their mobile devices. 83 percent of them use mobile apps to find information; 47 percent utilize them for reference purposes, and about 40 percent use them for support during consultations.
However, another study published by the Department of Neurology in the Academic Medical Center, University of Amsterdam also discusses the dangers, lack of regulation, and proposes quality assurance guidelines for mobile healthcare apps. The authors of the study agree that medical apps have tremendous potential, but also underline the alarming lack of knowledge about risks these apps pose. Regulation and guidance are urgently required.
Furthermore, medical apps should be peer-reviewed by healthcare experts and quality control measures should be streamlined to safeguard the quality of care. Because doctors and patients rely on the information and tools around cure, ensuring quality and safety are of paramount importance in gaining their trust in medical apps. Awareness among medical professionals is absolutely required, so that they can make informed choices about the apps they use in clinical care knowing that some apps may contain unreliable, non-peer-reviewed content.
Hence, it’s easy to recognize that mHealth is a very complex and diverse ecosystem; mHealth applications require the highest degree of accuracy from both device and software. The medical app frontier is an entirely new development for established regulatory bodies like the US Food and Drug Administration (FDA). In July 2011 in its “Draft Guidance for Industry and Food and Drug Administration Staff: Mobile Medical Applications,” the FDA proposed that certain types of mobile apps targeted for medical use be considered medical devices and placed under its scrutiny before they can be released for public use.
However, the FDA clearance processes for medical apps do not require comprehensive clinical testing to ensure safety and quality as the processes for drugs and medications. With the increasing complexity of software systems, the possibility of failure through software defects increases exponentially, reinforcing the need for stringent quality assurance measures. Even though FDA’s guidelines on medical apps has been finalized, it only addresses a subset of the entire medical app space.
According to the study “Medical apps for smartphones: lack of evidence undermines quality and safety,” although no harm caused by medical apps has been reported, without app quality assurance and safety standards, it’s only a matter of time before some medical errors happen and unintended harm to patient occurs. To ensure quality of medical apps, authors of the study suggest:
- Official certification marks that guarantee quality
- Peer review system implemented by physicians’ associations or patient organizations
- Making high quality apps more findable by adding them to hospital or library collections
Presently, the feedback and ranking mechanism in place at app stores, while good for games and less critical apps, is highly inadequate for medical apps. The medical community, app store operators, regulatory bodies, auditors, patients, and other stakeholders need to get more involved in evaluating medical apps, at least on the following aspects:
- Validation and certification of software quality that endorses apps’ fitness for purpose and use
- Peer reviews by diverse stakeholders — physicians, medical associations, patients, regulatory agencies, subject matter experts, etc.
- Continuous monitoring of user feedbacks, complaints through establishment of vigilance systems
The responsibility of addressing the aforementioned requirements falls on the medical app development companies; they are required to validate the apps they develop — a responsibility requiring a spectrum of expertise and experience. Many medical app development companies have neither the bandwidth nor the resources to handle comprehensive quality assurance and validation. Though several journals now include app reviews by doctors and healthcare practitioners, evaluating the apps from a 360-degree perspective requires cross-functional skills and knowledge.
For example, medical app design is an area of paramount importance when developing mobile medical apps. Medical apps are intricate tools placed in the hands of doctors and patients, which can significantly influence healthcare outcomes. Hence, these apps must be designed and developed with usability in mind and addresses human factors, including user errors that result from unintuitive design. As the article “Developing a Mobile Medical App? Don’t Forget Human Factors!” points out, medical app developers should consider at least the following human factors:
- Preliminary analyses (such as hazard analyses, contextual inquiries, task analyses, heuristic analyses, and human factors expert reviews)
- Exploratory human factors research
- Formative human factors usability testingHuman factors validation testing
The FDA guidance also recommends app manufacturers to follow the human factors guidelines, even when their apps are not subject to regulation. This strategy is perceived to be the best way to both, mitigate risks and encourage adoption, while simultaneously avoiding high costs associated with unnecessary redesigns and product recalls.
From a software engineering perspective, even small changes in software could dramatically change the way a mobile medical app works. These changes may stem from requirement changes, design modifications, or amendments required by regulatory agencies. Ensuring the continued quality, accuracy and safety of mobile medical apps requires that any software modifications undergo a life cycle of software quality assurance and validation process. Regression testing should be as comprehensive as possible to demonstrate that the changes were implemented correctly and, did not adversely impact other parts of the product.
FDA’s final guidance addresses these perspectives required of mHealth apps and states that regulating mobile medical apps will be a high priority from 2013. The ramifications of this decision implies that medical app development companies are forced to implement rigorous testing procedures to ensure compliance — a development that’s perceived to be associated with high costs, complexity, and scope. Raising the bars might improve the overall healthcare outcomes, but may simultaneously deter aspiring developers and entrepreneurs from developing healthcare IT solutions.
Complex challenges such as these often spur innovative, creative, and simple solutions, but in the case of mHealth apps, these challenges can be thoroughly addressed using the help of professional communities and users, in other words through crowdsourcing testing and evaluation.
Crowdtesting, a variant of outsourced software testing, undertaken with a community comprising of subject matter experts, professional testers, legal consultants, software engineers, healthcare professionals, patients, usability experts, and others easily scales the scope and support required by a medical app development company to ensure that their solutions have been evaluated at all angles and comprehensively tested. The concept of crowdtesting, traditionally used to ensure quality of general purpose web and mobile apps, subjects the medical app to an exhaustive battery of tests performed under real-world conditions on a variety of devices with a group of professionals plus an identified target users group, before the medical app is released to the marketplace. Crowdtesting encompasses the rigor of traditional, formal software testing methods and complements these formal methodologies by emulating real-world conditions — a development that many medical app companies haven’t had access to before.
The fundamental concept of crowdtesting is the idea of distributing a specific quality assurance problem among a community of experts, specifically curated to solve the task. These experts are inherently motivated to solve the challenges for, either monetary or non-monetary rewards. Medical app development companies could avail crowdtesting services ranging from fully managed testing services offering to a self-service concept, where the company only seeks access to the testing platform and the expert community when needed.
In addition, the company could save on associated overheads, such as hardware and software costs because the community already owns various combinations of different device types, operating system and language versions — a situation that’s almost impossible to simulate in an internal lab. Crowdtesting opens a large pool of software testers and domain experts to medical app companies, so selection of target groups could be achieved at different levels of granularity.
As the crowdtesting process is fast and flexible by design, it could easily be integrated into existing software release cycles and delivery models, so that the shortcomings could be identified and corrected immediately before the software is released. Crowdtesting also permits testing at different stages of software development — from the prototype evaluation to a final check at the end of the beta stage. The ability to leverage real medical app users and devices helps achieve a substantial cost reduction during development and maintenance phases of the medical app lifecycle. Furthermore, human factors enumerated by FDA could be progressively validated through the various stages of software development through post-deployment and maintenance. As a result, the software quality increases exponentially while simultaneously relieving the app development company of important domain considerations beyond their area of expertise and freeing up their internal resources to focus on core operations.
In closing, crowdtesting delivers tremendous benefits to medical app development companies. Navigating the murky waters of regulatory bodies, software engineering, healthcare ecosystem, consumer psychology, etc. is an exhaustive endeavor in itself. Medical app development companies and entrepreneurs do not have to steer the waters alone. A trusted crowdtesting partner like Passbrains helps cruise these muddy waters by complementing your core skills and providing valuable mission-critical feedback from a broad and diverse panel of medical app users, healthcare practitioners and stakeholders, who have much to gain from your innovation.