The International Wheat and Maize Improvement Center (CIMMYT) is the world’s primary source of breeding material for wheat and corn (maize). Founded in 1943 and vaulted to international recognition in the 1960’s and 70’s, partly through the work of U Minnesota alum and Nobel prize Laureate Norman Borlaug, CIMMYT was the original model for the centers that now comprise the CGIAR. Over the years, CIMMYT has accumulated a large database of pedigree, trait and passport information for millions of wheat genotypes that include finished varieties, germplasm accessions, and (advanced) breeding lines. This information is critical for breeding programs to track the inheritance of desired traits (e.g., yield, disease resistance, drought tolerance, baking quality) for selections that are made for subsequent generations. The coefficient of parentage (COP), also known as the inbreeding coefficient, is a particularly useful metric when considering any two genotypes as potential parents for breeding. Formally this quantity indicates the likelihood that for any gene, the copies that occur in both genotypes are descended from a common ancestor.

GEMS Informatics worked with CIMMYT to analyze a subset of their large database, the 2012-2024 spring and durum wheat international nursery dataset. This dataset included 8.5 Million genotypes, 11.8 Million genotype aliases (e.g., internal cross names, commercial release names, abbreviations), and 18 Million genotypic relationships. We set out to accomplish 3 tasks:

1. Build a python code repository with a scalable infrastructure. GEMS Informatics designed an approach using SQLAlchemy and SQLite that can accommodate 100’s of millions of genotypes and their relationships. This effort was successful and took 6 months. All CIMMYT’s data can be ingested in just 1 hour on a contemporary laptop.
2. Resolve naming discrepancies identified in the CIMMYT data. GEMS staff analyzed all common_name and cross_name designations among the 8.5 million genotypes and putatively identified 544 pairs of genotypes in CIMMYT’s genebank that may be duplicative, and hence require consolidation in their database. It is a testament to the care that CIMMYT staff have employed that there were only 544 “typos” among the names for these 8.5 million genotypes. Examples include common typographical errors (e.g., C0723595 and CO723595; II53.546 and 1153.546), punctuation variants (e.g., 4715D(5B) and 47-1-5D (5B); DARTS-IMPERIAL and DART´S IMPERIAL), compound word variants (e.g., PLAN ALTO and PLANALTO; YANG MAI 6 and YANGMAI 6), language variants (ALGERIAN and ALGERIEN; FEDERATION and FEDERACION) and misspellings (e.g., AEGILOP UMBELLULATA and ARGILOPS UMBELLULATA; ATALANTA and ATLANTA; AUBAKOMUGI and AOBAKOMUGHI).
3. Provide harmonized pedigree datasets and query capabilities via an API to CIMMYT. All of the following questions can now be answered via API queries to the database: What are the parents of any wheat genotype? What are the pedigree entries at any arbitrary level? (level 1 = parents; level 2 = grandparents; level 3 = great grandparents; …) What is the full recursive pedigree for any genotype? (ideally traced back recursively to landraces, if possible) What are the known aliases for any wheat genotype? What is the matrix of pairwise COP values for any pair or list of genotypes?

Future Work

Cleaning and harmonizing wheat pedigrees worldwide. Previously PedTools, developed by GEMS in 2017 could support modest pedigree sizes involving thousands of genotypes, initially targeted to wheat pedigrees for US and Canadian varieties. This work enabled  the GEMS team to further enhance their PedTools infrastructure so that it can scale to collections with genotype counts numbering 10 million - 100 million. This makes it suitable to expand to CIMMYT’s full wheat genebanks as well as publicly accessible repos like GrainGenes and GRIN. Further, PedTools has the ability to match genotypes across organizations so it may serve to unify the wheat pedigree collections across countries, CIMMYT and public repositories, mapping accessions at each center to each other. 

Additional crops: The original incarnation of PedTools was used to harmonize ~10,000 soybean pedigrees for a UMN soybean breeding project. Our soybean breeding collaborator is currently digitizing decades of old printed variety breeding information. So we plan to revisit that harmonizing effort with the new version of PedTools. Soybean breeders don’t use the Purdey notation (variety 1 / variety 2 // variety 3) for pedigrees, but instead utilize an arithmetic notation (e.g., ((variety 1 x variety 2) x variety 3)). With this in mind the underlying architecture of PedTools has been designed to accommodate a plugin of any custom set of rules to parse variety names and pedigrees for specific crop communities. In this manner, in the future we can write a new parser to ingest a new format of pedigrees and the rest of the PedTools machinery remains unchanged since the internal representation of varieties and their relationships is the same. 
With this potential for expandability, we plan to engage pedigree data curators for other crops at various CGIAR Centers and elsewhere to help standardize their pedigrees and thus streamline and accelerate trait discovery and varietal development efforts.

Photo credit: A. Morgounov/CIMMYT.

Funding. The activities described here were conducted with support from the Government of Mexico and Minnesota State Government MnDRIVE funding made available to GEMS Informatics.

Potatoes are a pervasive staple and specialty crop the world over, but so too are the pests and diseases that affect potato yields, tuber quality and farmer profitability. However, like the productive part of the crop itself, many potato diseases develop below ground, incurring costly damage well before the farmer becomes aware of the problem. Getting a better handle on the spatial extent, depth and temporal variation of soil temperature, moisture and other environmental variables that affect the development of potato diseases is key to modeling the field-scale risks posed by these threats. This is especially so if the aim is to model disease development in ways that provide farmers with actionable (real-time) information to mitigate or manage the crop production and profitability outcomes of these diseases.

Verticillium wilt is a long standing scourge of potato farmers. In related work, we estimate this particular soil borne fungi is a threat to almost 72% of the world’s potato growing area. V. wilt infections first become evident above ground when the plant’s lower leaves wither and die. Symptoms progress upwards until the entire plant yellows and wilts. The disease causes early senescence of the plant, which results in economically significant yield losses and tuber discoloration. In some instances, costly fumigation can be an effective mitigation strategy, while long rotations (3 years or more) with other crops can reduce the inoculum load of this long-lived disease at a particular site.

 
Creating fit-for-purpose biotic threat models that reveal the potential risks associated with V. wilt and other crop diseases at field scale and beyond is a core research focus of the GEMS Biotic Threat Analytics Lab. Pest risk prediction models and timely access to the targeted information products they enable helps farmers and others prioritize disease intervention on local (and neighboring) farms, informs a host of post-farm supply-chain decisions that rely on prospective crop production outcomes, feeds valuable information into early warning systems, and informs crop breeding strategies.

Digging Deeper into Above- and Below-Ground Environmental Data

Appropriately scaled environmental data both above and below ground data are required to informatively model the field-level risks posed by V. wilt (and other crop pests and diseases). While there are  several relevant gridded environmental datasets to hand, most are at coarser resolutions that extend well beyond the area extent of a typical potato field or farm. Moreover, these datasets often lack relevant below ground variables (e.g., soil moisture and temperature, at variable depths) that in combination with other variables are required to develop and deploy actionable pest prediction models of soil-born biotic threats. To rectify these two shortcomings, we turned to our GEMS Sensing team to provide real-time sensing of the needed environmental data. 
 

To best align our environmental sensing efforts with incidence and severity information on V. wilt, we also paired up with Dr. Ashish Ranjan’s Lab in the University of Minnesota’s (UMN) Department of Plant Pathology. Ashish conducts extensive V. wilt trials at UMN’s potato disease nursery located at the U’s Sand Plains Research Center in Becker, Minnesota. 

Siting Sensors to Reap the Biggest Predictive Bang for the Buck!

In 2023 we ran a test deployment of two GEMS sensing systems in the V. wilt resistance screening blocks at Becker, MN. Each system was configured with 3 above ground sensors (temperature, barometric pressure, and relative humidity) and 5 below ground sensors (soil moisture, temperature, permittivity, bulk soil electrical conductivity, and porosity). The above ground sensors were deployed in 3 replicates, and the below ground sensors at 3 depths. Our statistical assessment of these real-time data indicated that one set of above ground sensors coupled with below ground sensors at two depths yielded the optimal sensor configuration. 
 

For the 2024 growing season we scaled up our sensing efforts to 17 sensing stations, each with 3 above ground sensors and 5 below ground sensors. Fifteen sensor systems were deployed in the research plots where select potato varieties are screened for V. wilt by the Ashish Lab, plus 2 sensing systems for benchmarking in the (disease free) potato breeding plots at Becker managed by Dr. Laura Shannon in UMN’s Department of Horticultural Science. 

The precise placement of each sensing system was informed by an environmental profiling exercise prior to field deployment. First we digitized the boundaries of each of the 16 blocks used in the V. wilt screening nursery then overlaid that on gridded data we accessed from GEMS Exchange on 10 variables of potential relevance for disease risk modeling; including elevation, slope, available water storage (AWS) and soil organic carbon stock estimate (both at 3 depths throughout the rootzone). Our aim was to sense as much environmental variation from within the study area as possible in the process of generating our targeted below (and above) ground environmental variables.
 

The deployed location of each sensor is marked by the red dot in image #3, where in this instance each disease nursery block is overlaid on just one (i.e., elevation) of the 10 environmental variables we used to select a site for each sensor. 
 

The wealth of high-resolution, real-time (every 15 minutes) environmental data generated by this deployment is now being analyzed and integrated with correspondingly geo-tagged V wilt field data from the Ashish Lab. Field-scale predictive pest models are also being prototyped drawing directly on these novel, environment-linked-to-disease data sets to both develop and ground truth our modeling results. Working with our industry partners, PepsiCo, we look forward to further refining and then geographically scaling up the deployment of these predictive models to provide real-time, fit-for-purpose insights into dealing with this (and other) pesky potato diseases.     

Verticillium wilt Image credit: Utah State University

​This activity supported in part by MnDRIVE Global Food Ventures, University of Minnesota

Revolutionizing Agriculture

In recent years, the fusion of artificial intelligence (AI), machine learning (ML), and the Internet of Things (IoT) has opened up unprecedented opportunities for agricultural innovation. One groundbreaking development in this arena is the implementation of real-time geoinformation systems, which promise to revolutionize agri-environment research by enhancing data quality, scalability, and cost-efficiency.

The Rise of Spatial IoT in Agriculture

As the agricultural sector increasingly relies on AI and ML for knowledge discovery, the need for large, high-quality datasets has become paramount. Spatial IoT technologies, which involve deploying internet-connected sensors throughout agricultural environments, have emerged as a crucial tool in this data-driven landscape. These sensors collect real-time, high-resolution geospatial and temporal data, enabling researchers to monitor and analyze agricultural systems with unprecedented precision.

Challenges in IoT Implementation

Despite its potential, the implementation of IoT in agriculture presents significant challenges. Managing large fleets of devices while maintaining data quality is a complex task. Scientists often start with one-off prototypes, but scaling these to thousands of internet-connected devices requires overcoming numerous technical and logistical hurdles.

Case Studies in IoT System Development

The University of Minnesota’s Real-Time GeoInformation Systems Lab has been at the forefront of addressing these challenges. Since 2019, the lab has developed and deployed over 2,727 IoT devices across four continents. This extensive deployment has provided valuable insights into creating a generalizable, open-source spatial IoT system tailored for agricultural research. This work was summarized in a recent pre-print on Arxiv.com (Runck et al. 2024).
One key aspect of the lab's work has been the iterative development of the IoT system, progressing through three major and fourteen minor versions. Each iteration has refined the system's capabilities, from improving sensor accuracy to enhancing data transmission reliability. The current version of the system is designed to be scalable, ensuring that it can be deployed widely while maintaining high data quality.

Practical Applications

The applications of these IoT systems are diverse and impactful. For instance, in irrigation management, real-time data on soil moisture and temperature help optimize water usage, crucial in regions facing water scarcity. Similarly, in plant winterkill research, sensors monitor microclimates to understand the conditions leading to crop damage in cold environments. These insights enable farmers to adopt preventive measures, safeguarding crop yields.
Another notable application is in meteorological observations. Deploying IoT systems for weather monitoring provides granular data that enhance the accuracy of weather forecasts, which is vital for agricultural planning and risk management. For example, in Minnesota and Malawi, extensive networks of weather stations equipped with IoT sensors collect data that support both local farmers and broader agricultural research initiatives. However, paying attention to data quality, access and interoperability matters, often coupled with fit-for-purpose analytic pipelines, is key to ensuring real-time, geo-sensed data lead to actionable, data-driven informatics products.

The Role of Open Source in Scaling IoT

Open-source technology plays a crucial role in the scalability of IoT systems. By making design files and code publicly available, researchers can build on existing work, ensuring broader adoption and continuous improvement. This collaborative approach aligns with the scientific principles of transparency and reproducibility, fostering innovation across the agricultural research community.

Moving Forward: GEMS Sensing Service

To support the ongoing development and deployment of IoT systems, the University of Minnesota has established GEMS Sensing, a service organization within its GEMS Informatics Center. This initiative aims to provide turnkey IoT solutions for researchers, ensuring that the technology is accessible and sustainable. By offering both internal and external sales models, GEMS Sensing facilitates public-private partnerships, driving further advancements in digital agriculture.

Conclusion

The integration of real-time geoinformation systems into agricultural research marks a significant leap towards smarter, more sustainable farming practices. By harnessing the power of spatial IoT, researchers can collect and analyze data at an unprecedented scale and resolution, paving the way for innovative solutions to some of agriculture's most pressing challenges. As these technologies continue to evolve, the future of agriculture looks increasingly data-driven and resilient, promising enhanced productivity and sustainability for the global food system.

Image: Generated with Firefly. of A modern agricultural field with IoT sensors placed at various points.

​This activity supported in part by MnDRIVE Global Food Ventures, University of Minnesota

APIs: Now well-documented and much easier to use

One of the big frustrations in using computing to solve large, multidisciplinary challenges is managing data sets from different disciplines. Often, you have to go to each individual site and download the entire dataset. Then you have to parse out the subset of data fields you want within the geographies, spatial resolutions and time periods you care about. It is still the case that a few groups provide their data in the form of an Application Programmer Interface (API), where the data are served in a structured form with clear metadata documentation. Data can be sliced and diced how you like, selecting subsets of geography, time, and variables of interest. Once you sign up and obtain an API key, it just takes a few lines of code in Python or R to establish a connection and query at will!

GEMS has been building out a portfolio of APIs since 2021 across a range of useful datasets seeking to span the full Genetics x Environment x Management x Socioeconomic data landscape. Those who tried GEMS Exchange before will know that we used to have a middle layer managed by RapidAPI. Users found that cumbersome and confusing, so we are now using our own Apache APISIX server within our own web pages to serve you your key and monitor usage. We’re confident that your experience will be super easy this time around. Let’s get you started!

First, check out which APIs might interest you at our GEMS Exchange page. To obtain your API key simply click here for key. (Note you will need to have a Globus.org account, which is free – or you can connect via your academic institution, Google account, or ORCID). Once you know which APIs interest you, explore our collection of Jupyter notebooks in Github that give you practical guidance on how to use them. Many of the APIs we offer use the GEMS Grid which help ensure they are interoperable. And the GEMS Grid itself has recently been made open source, so you can place your own data sets on the Grid and interoperate with the community.

We are always happy to hear of useful datasets that could be added to the GEMS gridded collection in Exchange, so by all means reach out with suggestions or queries here.

​This activity supported in part by MnDRIVE Global Food Ventures, University of Minnesota

We are delighted to announce that we have just released the GEMS Grid code library, where the code is under the open source Apache 2.0 license, which allows anyone to use the code for commercial or non-commercial purposes – you simply need to provide attribution to GEMS Informatics when you use or modify it.

Just before we at GEMS Informatics started developing Application Programmer Interfaces (APIs) in agriculture for GEMS Exchange, GEMS geospatial expert Jeffery Thompson worked with others in the GEMS team and colleagues at NSIDC to develop the GEMS Grid, a hierarchical discrete global gridding system. This Grid has allowed us to provide data sets at different resolutions ranging from 36 km to 1 m, and still have them remain functionally interoperable. The interoperability is possible because we have written the code to allow users to project data onto the grid, aggregate data to coarser resolutions, and, notably, also disaggregate data to finer resolutions. The latter operation is ordinarily a difficult problem, but is made easier, as I discuss below, since we enable the users of our code to thoughtfully address it in a standardized, replicable way.

Many problems in agriculture (e.g., understanding the spatial location of crop production) require equal area parcels of land to do proper calculations. Working with strict lat-lon coordinates won’t suffice as areas near the equator are significantly different in size as areas near the poles. The GEMS Grid preserves equal-area assumptions as it divides land, so you can do these calculations with confidence, and preserve aggregation-disaggregation consistency in the data, even if you are not a GIS expert.

So let’s look at the 5 options for disaggregation that GEMS geospatial developer Olena Boiko included in the GEMS grid toolbox, schematically described in the figure she developed above.

Option 1. Value transference. 
In this case, if you were to subdivide a 3 km² resolution grid cell into 9 x 1 km² cells, this option would be appropriate for any value that is deemed roughly constant throughout the area applied. Examples would be rainfall in inches or grain yield in bushels / acre.

Option 2. Even value division. 
Sometimes the quantity measured in a cell represents a cumulative value for the area in which it is reported. In this case, if the parent cell is homogenous, then splitting it up into 9 equal-area pieces would require that you divide the value in each equivalent cell by a factor of 9. Examples where this selection makes sense include grain production in bushels, crop acreage, and population.

Option 3. Value transference with a mask. 
This one is similar to Option 1 except we are no longer making the assumption that the distribution of values in the parent cell is spatially homogeneous. For example suppose you were measuring grain yield, but you knew that 3 of your nine cells had buildings occupying them (see white areas in the Figure). In this case you only transfer your values to 6 remaining cells (colored peach) that have arable land. Cells are binary with this option (i.e., either allowed a value or not).

Option 4. Even value division with a mask. 
Analogously, you can mask out cells in the value division case when you know that your daughters cells are not all equal. This is just like the case in Option 3, except you divide your parent-cell value evenly by the number of viable daughter cells. In this pictorial example, there are 6 viable daughter cells, so each gets a value of 900/6 = 150. This would be appropriate if you were computing grain production in bushels and you had a total value that needed to be split up across the 6 arable daughter parcels.

Option 5. Flexible division with a mask. 
This scenario is the most flexible, and allows the user to create a master mask with arbitrary weights at each daughter cell. It allows you to block off daughter cells entirely, and prescribe the relative weights of all remaining daughter cells. This is ideal for situations where you are allocating crop distributions and you want to avoid certain land use features (e.g., lakes, forests, housing) and probabilistically distribute the remaining crop areas (e.g., with higher probability near soils with a high SSURGO National Commodity Crop Productivity Index).

I’m confident these flexible disaggregation tools will provide much easier, more accurate, and replicable solutions for your particular spatial analytic problem. So please give them a try!

​This activity supported in part by MnDRIVE Global Food Ventures, University of Minnesota

Assessing BMP Effectiveness for Water Quality Outcomes with Remote Sensing

Virtually everyone in Minnesota and bordering states whose profession has a direct stake in Water Conservation assembled in Downtown St. Paul’s RiverCentre October 17-18 for the annual Water Resources Conference. I had the honor of chairing a Special Session at the conference titled “Assessing BMP Effectiveness for Water Quality Outcomes with Remote Sensing.” The session featured two very foundational efforts that we are attempting to bridge together in partnered research at GEMS: (1) The Minnesota Department of Agriculture’s (MDA) Minnesota Agricultural Water Quality Certification Program (MAWQCP), directed by Brad Jordahl Redlin, and (2) the Department of Forest Resources’ remote-sensed Water Quality assessment for our 10,000+ lakes, led by Leif Olmanson.

The 90-minute session was organized so that each of the three speakers (one had dropped out last minute for logistical reasons) had 20 minutes to speak plus two minutes for burning questions. At the end the audience had 15 minutes for questions with the entire panel of speakers. There were about 30 very inquisitive people in the audience, who peppered the speakers with questions throughout.

Updates on the MDA’s certification program

Brad Jordahl Redlin kicked off the session with an outstanding talk that outlined the principles on which the certification program he founded and directs was created, and highlighted the progress that has been made. Literally over a million acres have now been certified, spread across over 1400 producers. In exchange for regulatory certainty for a period of time, these producers allow his government agency to (1) assess their farm practices, (2) suggest new management practices (sometimes with additional incentives such as subsidized equipment from a grant), and (3) carry out audits of their new practices. The program, which has been operational for nearly a decade, is well beyond the stage of corralling early-adopters. New participants span the spectrum of producer demographics in age, farm size, and more.
 

Scaling lake quality measurement

Leif Olmanson stepped up to review an activity that he first spearheaded twenty years ago – using remote sensing to measure the water quality of Minnesota lakes. In the interim, he has overseen steady progress using new generations of satellites (from Landsat to Sentinel), multiple metrics (clarity, chlorophyll, organic matter), and the frequency of data reporting (from once every 5 years to ~weekly lake snapshot and monthly pixel-level composites of all 10,000+ lakes). Data is made available to the casual user via a friendly Lake Browser interface, and in numeric form pixel-by-pixel for data wizards on GEMS Exchange.
 

GEMS bridging the two programs

David Porter rounded out the session discussing the improvements he has made in creating algorithms that properly remove clouds and aerosols from the satellite images. This is a prerequisite for feeding all of the regularly spaced (~5 day) images of a field through a growing season into a sophisticated Recurrent Neural Network (RNN, a type of Machine Learning algorithm) that his colleague Anubha Agrawal has been preparing. As described in this case study, David, Anubha, and I intend to use the RNN trained on known, certified management practices (e.g., strip till / no till, cover crops) from the MAWQCP to make a prediction of what practices growers are making on uncertified farmland. We then will trace these producer fields to the downstream lakes in each watershed, and see what covariates (e.g., slope, soil type, distance to waterway) affect the correlation of % farm practice adoption vs. downstream lake quality in a watershed.

​This activity supported in part by MnDRIVE Global Food Ventures, University of Minnesota