The Value of Testing

The crop damage caused by fall armyworm has put millions of livelihoods at stake across Sub-Saharan Africa. Its rapid spread has been projected to cause losses valuing $2.5 billion to $6.2 billion per year if left unabated.^[1] Fall armyworm has caused havoc for smallholder farmers across the region and become a serious threat to food security.

To help those affected, Feed the Future has been simultaneously trying to solicit new tools that communicate emerging research and information on how to battle the invasive pest to African farmers while also trying to ensure that those tools perform accurately and consistently. The Fall Armyworm Tech Prize is striking this balance–the competition is incentivizing innovators to submit solutions, but will only pay to reward those tools that are successfully tested in the field. One critical step in the prize process is testing to ensure that the 20 finalists selected actually work for the end-users’ contexts. Testing enables us to confidently say that the proposed solutions stand a good chance of mitigating the spread of fall armyworm.

Testing in the Field

While we encouraged prize finalists to hone their solutions by independently testing their functionality and user interface, especially in their respective countries of implementation, the prize itself features a neutral third-party test with the Centre for Agriculture and Bioscience International (CABI) in a single locale in Uganda. This offered finalists an opportunity to continue to have their solutions tested, validated, and improved while also informing the prize’s judging process.

CABI designed the field test protocol and conducted the tests this August with these considerations in mind:

• Whether the solution does what it says it will do to address fall armyworm (truth in advertising);
• Whether users feel that the solution works; and
• Whether the solution functions in the field.

In short, the prize’s overseers want to understand what types of solutions work so they can reward the best entries and encourage their scale up.

A tester captures an image of a FAW caterpillar using a mobile phone, Photo Credit: CABI

Testing Experiences

 CABI’s Plantwise program, which has plant clinics throughout Uganda, selected two test sites in the Wakiso and Mukono districts, home to many farmers battling fall armyworm.

Testers using a mobile phone to identify a pest using image recognition, Photo Credit: CABI

 Field testing reveals benefits and issues that come with introducing new tools and approaches. Often, users immediately recognize what they want and what they do not like about the tools. During its field testing, CABI observed that although testers were aware that some of the prototypes had multiple platforms (for example, Unstructured Supplementary Service Data or USSD, short message service or SMS, apps, and interactive voice response), they tended to interact more with a platform they preferred. If they did not like a platform—or experienced a bug or delay— they were more likely to stop using it altogether and move on. While this engagement and resulting feedback can be understandably frustrating for the finalists, it does offer valuable insights into user preferences, user-experience barriers that need to be addressed, and preconceptions that may need to be rethought.

Field testing also brings a number of logistical challenges. In addition to making sure that testers are available, fed, and compensated for their time during testing, CABI had to ensure the prototypes could be tested in a rural context. The testing areas had limited internet access and no electricity, and when the devices were able to get a data connection, it was weak and slow to respond—particularly when downloading information and processing images. CABI had to source Wi-Fi routers linked to a reliable internet service provider to ensure internet access, as well as a mobile generator to charge both devices and routers directly in the field. Access roads to the farms are also very poor, especially during rainy seasons. These logistics underscored that while digital tools and approaches have a growing market in rural Africa, there are real infrastructure challenges that can constrain their use and may need to be addressed alongside their introduction.

Testing Lesson

Despite careful preparations such as bringing in routers and a mobile generator, the testing process was far from seamless. With the routers and generator, most of the digital solutions relying on software or online applications were easily downloaded. The solutions using hardware were another story. Their shipping and customs process introduced expensive delays. The Fall Armyworm Tech Prize team had to hustle, use imagination, and find private couriers to ensure we were able to test the solutions that involve hardware without stalling the next phase of the competition. We’ve been reminded to keep an expansive view of the kinds of solutions being tested and ensure we’re prepared for them.

We also faced challenges in designing an equitable testing protocol that reflected the diversity of prototypes submitted and needs of the eventual users. The testing protocol needed to be flexible enough to fairly capture feedback from solutions that incorporate tactics ranging from drones to behavior change communications. Instead of rating the prototypes against a fixed set of criteria—such as a predetermined idea of what the ideal solution looks like—CABI only set out some basic parameters of functionality and then let the users express their preferences without implicitly guiding them.

A tester filling in a survey form, Photo Credit: CABI

 The Fall Armyworm Tech Prize is now entering its final judging phase. Our judges will be guided by CABI’s independent equivalence test report, which ensures clarity in the testing methodology so it can comparably, fairly, and transparently explain the results. In turn, the judges will be better able to understand and compare the usability, accessibility, and potential impact of the entries.

With those factors in mind, the judges will recommend the winners and Feed the Future will support and promote solutions urgently needed by smallholder farmers fighting to protect their crops from fall armyworm.

Aerial view of the farm where testing took place, Photo Credit: CABI

Written By Bhavik Doshi (Challenge Prize Centre, Nesta) and Marin Hirschfeld (Centre for Agriculture and Bioscience International)

^[1] https://www.cabi.org/news-and-media/2018/cabi-warns-of-rapid-spread-of-crop-devastating-fall-armyworm-across-asia/

Featured Image, A tester configuring one of the solutions, Photo credit: CABI