There has been a new release of the Spatial Production Allocation Model with updates in the countries listed below.

In the notes below, you will notice one of the main corrections that was applied to each country was “fixing of 0 and -999 inconsistencies”.  Wondering what that means?  When we process  subnational agricultural production statistics for each country we make a distinction between when we have data for that administrative district, and that value is equal to zero, and when we know that district exists, yet we have no production data for that district.

In other words, “no data” (-999) is not the same as “zero”.  The more precise we can be with this distinction in the input agricultural production statistics, the better out outputs.

So how do we fix these inconsistencies?  When we revisit a country for re-run we re-visit input statistics.  When we see that larger administrative units (Regions, or Level 1) report zero, the smaller administrative units with that unit (districts, or level2) within that unit should also be equal to zero, not -999 as in this example.  The user changes these -999 values to zero and re-runs the model for that country.  This would be the fix in “wheat” example below.  However, the -999 value under rice is fine since this indicates we don’t have statistics for that district, not that the value is zero.

Excerpt from ReadMe.txt associated with this release:

Bangladesh … complete new dataset from [1], calibrated to 2006-2007, and National data for totals

India … corrected groundnut data (use information from ICRISAT [2], but only at state level),
some corrections for wheat and rice using [3] and fixing of 0 and -999 inconsistencies in all crops
Burundi … fixing of 0 and -999 inconsistencies
Congo DR … fixing of 0 and -999 inconsistencies
Rwanda … fixing of 0 and -999 inconsistencies
Tanzania … fixing of 0 and -999 inconsistencies
Uganda … fixing of 0 and -999 inconsistencies
Zimbabe … fixing of 0 and -999 inconsistencies
Kenya … complete new dataset from Kenya [4] calibrated to avg 2004-2006,FAO calibration
South Africa … fixing of 0 and -999, use only level1 data in the runs, ALSO include WHEAT (which had previously been missed)
DUPLICATE pixels in South Africa and adjoining countries were fixed (eliminated)
Nigeria … new data set at national and level1 [5] for cassava, millet, groundnut, sorghum, other pulses (bean), maize, yam, rice and cotton.
Aggregate 37 states into 31 (to match previous Nigeria data)
rest of the crops from previous Nigeria data, with totals from FAO avg(2005-2007).
Assume sugar-cane area and production = 0 since data is not consistent with reports.

New SPAM runs since V3.0.5:
Centr.African Rep … fixing of 0 and -999 inconsistencies, run all crops
Gabon … fixing of 0 and -999 inconsistencies, run all crops
Liberia … fixing of 0 and -999 inconsistencies, run all crops
Mauritania … fixing of 0 and -999 inconsistencies, run all crops
Togo … fixing of 0 and -999 inconsistencies, run all crops

[1] Yearbook of Agricultural Statistics of Bangladesh,2008.
[2] ICRISAT, District-wise yearly area and production for 18 States from 1999 to 2006. Worksheet provided to project.
[3] IndiaStat, e-yearbook 2010 for 5 Indian regions.
[4] KIPPRA, 2007. Kenya Agricultural Sector Data Compendium. Volume Two: Crop production.
Kenya Institute for Public Policy Research and Analysis (KIPPRA) and Ministry of Agriculture, Nairobi, Kenya.
[5] 2007 Nigeria NBS, CBN, NCC Survey and FAO countrystat, both for year 2006.

Bangladesh – complete new dataset from [1], calibrated to 2006-2007, and National data for totals
India -  corrected groundnut data (use information from ICRISAT [2], but only at state level), some corrections for wheat and rice using [3] and fixing of 0 and -999 inconsistencies in all crops
Burundi … fixing of 0 and -999 (no data)  inconsistencies in input statistics

Congo DR … fixing of 0 and -999 (no data)  inconsistencies in input statistics
Rwanda … fixing of 0 and -999 (no data)  inconsistencies in input statistics
Tanzania … fixing of 0 and -999 (no data)  inconsistencies in input statistics
Uganda …fixing of 0 and -999 (no data)  inconsistencies in input statistics
Zimbabwe …fixing of 0 and -999 (no data)  inconsistencies in input statistics
Kenya … complete new dataset from Kenya [4] calibrated to avg 2004-2006,FAO calibration
South Africa … fixing of 0 and -999, use only level1 data in the runs, ALSO include WHEAT (which had previously been missed)
DUPLICATE pixels in South Africa and adjoining countries were fixed (eliminated)
Nigeria … new data set at national and level1 [5] for cassava, millet, groundnut, sorghum, other pulses (bean), maize, yam, rice and cotton.  Aggregate 37 states into 31 (to match previous Nigeria data) rest of the crops from previous Nigeria data, with totals from FAO avg(2005-2007). Assume sugar-cane area and production = 0 since data is not consistent with reports.

SPAM2000v3.r5 Draws from these sources:

[1] Yearbook of Agricultural Statistics of Bangladesh,2008.
[2] ICRISAT, District-wise yearly area and production for 18 States from 1999 to 2006. Worksheet provided to project.
[3] IndiaStat, e-yearbook 2010 for 5 Indian regions.
[4] KIPPRA, 2007. Kenya Agricultural Sector Data Compendium. Volume Two: Crop production.
Kenya Institute for Public Policy Research and Analysis (KIPPRA) and Ministry of Agriculture, Nairobi, Kenya.
[5] 2007 Nigeria NBS, CBN, NCC Survey and FAO countrystat, both for year 2006.

Having just returned from the ESRI’s User Conference I couldn’t wait to try their new *free* mapmaking service ArcGIS online. Below you will find my first attempt at making a map, aided by the free time afforded by a bout with early morning insomnia and by someone’s prior labor prepping and posting many Africa centric layers to a server.

I suspect that someone is Todd Slind at SpatialDev given the Map Service was posted by slindy23. Thank you Todd (I think) for posting such complete and relevant agriculture data Africa-wide. It was one of the only Africa layers listed, though I imagine with time there will be a much richer library available, especially as people tap in to the power and usability of this medium.

This map is a “map sandwich,” as Bernie Szukalski calls it, where I’ve selected the Terrain Basemap, overlayed my content, then overlayed a reference layer of World Boundaries and Places. I tried to make my content layers transparent enough for the relief to come though. I selected the landcover which I believe comes from GLOBCOVER and farming systems (Reference: Dixon, J., A. Gulliver, D. Gibbon, and M. Hall (ed.), 2001: Farming systems and poverty: Improving farmers’ livelihoods in a changing world. FAO and World Bank, Rome and Washington, D.C., 412 pp. [available online from http://www.fao.org/docrep/003/Y1860E/y1860e00.HTM] )

I configured the pop-ups to show the name of the farming system and made the layer opaque enough you could see the extent of its reach. I clicked Save, Share, then Embed. Nothing fancy, but nevertheless very satisfying.

One critique I have is that this workflow doesn’t bring along the legends with the embeddable map, whereas creating a web app or sharing by other means does bring along the legends. Bernie– something to work on.

Stay tuned– I think there will be more of these to come.

View Larger Map

How does agricultural production compare within as well as between countries?  This data vis ranks the top five crops within each country in West Africa by harvested area.

Within the total harvested area of those top five crops, what share of area is allocated to each crop?  In other words, of each crop’s importance, how important is it in relation to the other important crops?  See each square’s color to indicate rank and its size to indicate share of harvested area in relation to the other four.

A couple that stand out to me personally — Cashew nuts in Guinea-Bissau, and cocoa beans in Ivory Coast as number one, and by a lot! How does this look to you MapSpam readers? Please comment.  - Susana

Data Source: FAOSTAT (Averaged 2004 – 2008, retrieved May 9, 2011)

DataVis: HarvestChoice

Production is reported in tons.
(Note groundnut production is measured in the shell)

Yield units in SPAM are kilograms / hectare.
More specifically, yields are reported for wet crops, as in the amount harvested on the field per hectare.

To the tunes of Cumbia and Cassava chips crunching, the Harvest Choice team convened a cassava map viewing party in hopes of mining our collective knowledge for information to validate our SPAM outputs, specifically, “Area Harvested Cassava” in Sub Saharan Africa.

With the presence of our visitor, the encyclopedic Chris Legg, and his expertise in cassava cultivation in Africa, we explored a .kml file SPAM Area harvested of cassava on Google Earth country by country and evaluated the plausibility of reported SPAM results.  We recorded our collective findings on new polygons created in Google Earth and attached comments.

In the spirit of experimental innovation, you are invited to join in on the discussion. Where do you think cassava is grown?  In the links below we provide viewing access to SPAM results for cassava: area harvested.

The first map is a raster covering per-pixel harvested area cassava.  The second is an example of how to collect comments about the data by drawing overlaying polygons over problematic or exceptional areas of accuracy.  Feel free to browse both, open in Google Earth, then create a “Feedback Polygons” layer of your own.

View Larger Map

View Larger Map

Here’s a suggested framework to guide your polygon creation in Google Earth. Most important is to leave a detailed comment in the description box of every new feature explaining WHY you agree / disagree with the data.  Please be as specific as possible.  Lastly, save and export your .kml file from Google Earth, then email back a copy to Harvest Choice which will serve as the central repository.
Polygon codings:

• Red – I don’t think so: There is likely a problem here
• Yellow – Maybe: Data needs a second look
• Green – That looks about right

Any comments, questions, or feedback (thank you!) can be sent to: s.crespo@cgiar.org

In the near future we hope to formalize the implementation of these kinds of feedback systems, then immediately feed collective feedback back into the re-running of our SPAM model.

In the meantime, we invite you to hold your own mini-map viewing party, download and open the .kmz files in google earth, create “Commenting” polygons of your own, then send it back to us.

Round the table at the Cassava Map Party

Since launching the new MapSpam.info in April 2010 and tracking SPAM downloads, we have reached a milestone with an ever growing number of users exploring the global dataset for a surprising diversity of ways.

In the past six months there have been over fifty downloads (not including those by the HarvestChoice team itself!).

Thanks to our community for sharing the range of ways in which you implement SPAM data, illustrated in the word cloud below. We look forward to reaching the next milestone– 500? Any guesses on when that will take place? Feel free to comment below.

Browse maps, query by pixel, and filter results by admin unit in the new data portal

Thank you for visiting MapSpam’s new site, designed to serve data to our growing community with speed and efficiency.

The drive behind this new format is to provide a platform where users can contribute to SPAM’s development and dissemination through an increased ability to be in dialogue with us, at HarvestChoice, where SPAM is developed.

Please peruse our new data portal where you will be able to browse and download mapped results, query SPAM results per pixel, as well as visualize and filter results by administrative unit and technology.

Then, feel free to leave a comment anywhere on the site, or here for more general comments.  Finally, follow Map_Spam on twitter for periodic updates on our dataset releases and progress.

Inconsistent formatting, units and administrative names make flagging errors even more difficult

Many SPAM users have been wondering why SPAM updates take so long to release.   While the actual execution of the model can be done one country at a time in one day, data preparation for even one country can take months.

One of the biggest delays occurs around the acquisition and “cleaning” of subnational crop production statistics.  Before the data can even be verified, it needs to be formatted correctly and inputted into the Harvest Choice subnational database.  Then,  it goes through a series of analyses aggregating numbers up to the next administrative level, then checking reported yields against those from an outside source, usually FAOstat.

Crop names and administrative unit names also need to be harmonized and verified before the data can be coded to match a time-stamped shapefile representing the maximum number of administrative units for which the incoming crop statistics report data.

Note “The Pipeline“ posts will always indicate which country is being processed as well as which ones will come next.

Please browse the many ways of visualizing SPAM data and feel free to comment on its veracity.

You can leave comments throughout the site, or submit more general comments here.