Accurate prediction is not profitable advice: profit-based evaluation of machine learning nitrogen recommendations in winter wheat
Organizations: University of Sheffield
Abstract
Nitrogen rates for winter wheat are set before the season, under unknown prices and weather. The standard UK advice does not respond to prices, yet recent price swings moved the most profitable rate by tens of kilograms per hectare. Machine learning is often proposed as the fix. However, it is usually judged on prediction accuracy, and accurate prediction does not by itself make the recommended rate more profitable. Our insight is to score nitrogen advice directly by the profit it forgoes on measured yield response curves. We build a test bench on 892 such curves from two long running UK experiments, and sweep the nitrogen to grain price ratio to cover all price scenarios. On this bench, machine learning fails as a predictor. No model recovers the best rate within farm tolerance, and the benchmark noise shows none can. At normal prices, every model also loses to the standard advice on profit. The gain sits elsewhere. A simple correction step applied after the model cuts profit losses by a quarter, while better models and extra features give no gain. The same frozen correction cuts losses by 43% at the second site without any retraining. A hybrid of standard advice plus a damped correction removes bias and trims rare large losses. The same price sweep also prices emission cuts, at a cost comparable to current carbon prices. Machine learning therefore pays as a profit scored correction to standard advice, not as its replacement.
Figures & tables
| Period | AN, £/t | Wheat, £/t | N price, £/kg | |
|---|---|---|---|---|
| 2020 (Jan) | 234 | 165 | 0.68 | 4.1 |
| 2021 (RB209 ref.) | 345 | 200 | 1.00 | 5.0 |
| 2021 (Sep) | 395 | 200 | 1.14 | 5.7 |
| 2022 (Jul peak) | 841 | 280 | 2.44 | 8.7 |
| 2023 (Jan) | 700 | 190 | 2.03 | 10.7 |
| 2023 (May) | 390 | 185 | 1.13 | 6.1 |
| Design | Median loss, t ha -1 | Worst 10% | Bias, kg N ha -1 | Large errors |
|---|---|---|---|---|
| RB209 alone | 0.035 | 0.562 | 42 | |
| ML alone | 0.070 | 0.872 | 62 | |
| Hybrid: RB209 damped ML | 0.033 | 0.518 | 37 |
| Change tested | What the operation does | Effect |
| Change the model or its features | ||
| Pick safer predictions | Choose the rate with the smallest predicted downside, not the highest mean profit. | % |
| Add protein features | Add grain protein measurements as extra model inputs. | % |
| Add weather features | Add season rainfall, temperature, and radiation as extra inputs. | % |
| Average several training runs | Train the model several times with different seeds; average the predictions. | % |
| Use the median, not the mean | Combine those runs by median instead of mean. | % |
| Quantity | t ha -1 grain | @160 | @200 | @240 | @280 |
|---|---|---|---|---|---|
| RB209 median loss, | 0.058 | 9 | 12 | 14 | 16 |
| RB209 median loss, | 0.246 | 39 | 49 | 59 | 69 |
| RB209 worst 10%, | 3.45 | 553 | 691 | 829 | 967 |
| Hybrid vs ML alone | 0.037 | 6 | 7 | 9 | 10 |
| Correction step gain | 0.019 | 3 | 4 | 5 | 5 |
| Woburn transfer gain | 0.032 | 5 | 6 | 8 | 9 |
Decision-Focused Learning
Decision-focused learning trains predictive models to improve the quality and cost of the decisions they inform.
See all 50 papersNon-Linear Model-Based Sequential Decision-Making in Agriculture
Technical Report on the Turba Fertilizer Machine Learning Stack in Morocco
turba-client provides programmatic access to publicly accessible site profiles, crop-specific target-yield spaces, and N, PO, and KO recommendation workflows; turba-data distributes analysis-ready snapshots; and turba-models packages crop-specific machine learning surrogates of recommendation outputs. The architecture links upstream retrieval, versioned analytical snapshots, reproducible cross-model benchmarking, and loadable offline surrogates while preserving the distinction between recommendation-system outputs, observed agricultural data, and model-generated predictions. The first dataset was constructed from 44,096 unique ESA WorldCereal locations. Scenario expansion across supported cereal workflows generated 132,017 crop-location recommendation requests under a medium target-yield setting. The resulting 22-variable dataset spans 10 regions, 66 provinces, and 1,149 communes. Nine regression families were evaluated under a fixed deterministic 80/20 protocol, and the current release packages five best-performing crop-specific models. The machine learning task is recommendation-function emulation rather than prediction of observed crop response. The stack provides a reproducible basis for spatial and temporal validation, uncertainty estimation, field-trial comparison, and future integration with additional data.