Research · shadow mode · does not change any advice
Sakia Lab: can a small learned model add anything to the physics?
The irrigation advice comes from a transparent FAO-56 water balance. Here we asked a harder question, in public: does a 106-tree CatBoost model (43 KB), trained on free satellite evapotranspiration of Kairouan cropland, track the satellite better than a calendar, and does it carry over to other places in Africa? The protocol was written before the full run; every later change is logged. The answers below include the ones we did not hope for.
Run it yourself, in your browser
Loading the weather and running the model…
What the model estimates is the average evapotranspiration of cropland around this place as a satellite would see it, from the weather alone. It is not your crop, not your field, and it does not change any irrigation advice.
How close to the satellite? (Kairouan, held-out years 2024–2025)
Error against satellite evapotranspiration, in mm/day (shorter is better). 92 cropland pixels in 14 weather cells. Intervals resample weather cells, because pixels in one cell share the same weather.
- Monthly climatology (calendar only)0.220
- Water balance, FAO-56 style0.261(+0.041 vs calendar; 95 %: 0.027 to 0.062)
- CatBoost, weather + calendar (43 KB, runs offline)0.201(−0.019 vs calendar; 95 %: -0.039 to -0.010)
- M1 + month as categorical feature0.200(−0.020 vs calendar; 95 %: -0.040 to -0.011)
- Hybrid: water balance + CatBoost correction0.205(−0.015 vs calendar; 95 %: -0.038 to -0.003)
- M1 + NDVI (partly circular, not deployable)0.139(−0.080 vs calendar; 95 %: -0.115 to -0.048)
Reading: weather alone brings a modest gain over the calendar (about 9 %; R² 0.43 → 0.52), clearly better than the crude water balance, and the learned model is tiny. By season the gain is in autumn (0.114 vs 0.176) and winter, and zero in spring (0.338 vs 0.343), when cereals green up and weather alone does not know the crop calendar. Adding NDVI (M2) helps a lot, but it shares its sensor with the label, so we do not count it.
What did not work, on purpose published
- Month as a categorical feature adds nothing (0.200 against 0.201): a cyclical day-of-year is enough, and the numeric-only model is the one that fits in a browser.
- The hybrid (physics + learned correction) is no better than the pure model (0.205).
- The “not sure” safeguard rule was not met. The disagreement between the model and the water balance tracks the water balance's error (within-month rank correlation about 0.5), but only 0.063 better than a naive disagreement (95 %: 0.028 to 0.097), below the 0.10 we had fixed in advance. So it is information only, and it never triggers “ask a technician”.
- No irrigated pixel in the Kairouan sample (most cropland there is rainfed): this study says little about irrigated fields.
Does it travel? The pipeline does; the trained model only partly
Every ingredient (MODIS, Open-Meteo, ESA WorldCover) covers all of Africa, so another region means changing one box in the code. We applied the Kairouan model, unchanged, to three other regions. Error in mm/day; “local calendar” is each region's own monthly average computed on its own data (optimistic for the calendar).
| Region | Pixels / weather cells | Local calendar | Kairouan model | Verdict |
|---|---|---|---|---|
| Sidi Bouzid, Tunisia (near) | 21 / 4 | 0.160 | 0.123 | Carries over |
| Haouz (Marrakech), Morocco | 39 / 9 | 0.202 | 0.172 | Carries over |
| Gezira, Sudan (Sahel) | 40 / 14 | 0.337 | 0.585 | Fails: needs local training |
What we conclude: a model trained in one climate regime does not carry to another (the Sahelian rainy-season regime is not Kairouan's). The honest way to extend Sakia across Africa is to train one small model per regime, which this pipeline does with about 90 minutes of free public data per region. Sidi Bouzid has only 4 weather cells, so its apparent success is fragile.
How to check us
Protocol, amendment log, every number and the code are public: github.com/ag-algolab/sakia/tree/main/ml. The satellite label is itself a model product, so we only ever say “closer to the satellite”, never “more accurate”. What the AI does elsewhere in Sakia. Also measured: Sakia on a weak connection, against five other sites.