The situation

Nigeria's downstream fuel market is described by a regulator. The regulator publishes the numbers - refinery utilisation, national stock sufficiency, product supply, prices - and those numbers are in a PDF. Once a month.

Everything useful can be built on top of them: a market dashboard, a price tracker, a tool that helps a station manager decide when to restock, an exchange for distributors. All of it depends on one unglamorous piece of engineering - getting the figures out of the document and into a database, cleanly, repeatedly, without a human retyping them.

And then, once the data is in, there is a second problem that is harder than the first: what do you do with the days in between? The PDF is monthly. A product that is monthly is a report. A product people use is live.

The ingestion

The ingest script does something deliberately boring. It enumerates the regulator's public document container, finds the current period's publication, downloads it, extracts the text, and parses the figures out of it.

The part worth designing carefully is the parsing, because a PDF is a rendering, not a structure. What looks like a table on the page is a set of text runs at coordinates, and a table that is a clean grid in one month can be a slightly different grid in the next. So the parser:

• locates figures by their labels, not by their position - the label "Refinery utilisation" is more stable than "column 3";
• reads the period from the document itself, so a figure is never filed under the month the script happened to run;
• records the source: which document, which page, which label the number came from;
• refuses anything it cannot identify with confidence, and says so.

That last point is the one that makes the rest of the product possible.

The rule: a figure with no source is not published

Every number in the database carries a pointer back to the document it came from. The interface can show that pointer, so anyone can check a figure against the original. And a number that failed to parse never reaches the product at all - it is not shown as zero, and it is not guessed from a neighbouring cell.

This is unglamorous and it is the whole design. A market intelligence product that quietly contains one wrong number is worth nothing to the people who would use it, because they cannot tell which number is wrong. Keeping the source attached to the figure means the product can be trusted in the specific sense that every claim can be checked.

The monthly problem

With trustworthy data in place, the product still has to answer a question that arrives daily: what changed?

There are three honest answers, and the product needs all three.

1. The regulator published something new. The market moved for a stated reason, and the data supports the move. The dashboard can say so.
2. The market moved and nobody said why. The data shows it; the cause is not in the data. The interface can show the change without attaching a story to it.
3. Nothing has changed since the last publication. This is a real state, not a missing feature. If the last publication is three weeks old, the honest thing is to show that it is three weeks old.

The temptation in a product like this is to interpolate. Take the monthly figure, smooth it, project it to today, and show a live-looking line. It looks better. It is also a fabrication - the number on the screen is a guess wearing the same clothes as the real ones, and a user cannot tell them apart. The market intelligence layer in FuelLink does not interpolate. It shows the published figure, its date, and how old it is.

The other half of the product

A dashboard nobody has to open is a report. The part that makes the data useful is the part that lets somebody act on it.

That is the exchange and the companion app: stations and distributors posting what they need and what they have, with the market data visible beside the transaction. The intelligence is not the product; it is the context that makes a decision in the market less blind.

Which is also why the market data has to be right. A restocking decision made against a figure that was guessed is worse than no figure at all.

What this generalises to

If you are building a product on top of somebody else's published data - a regulator, a statistics office, an open data portal - three things decide whether it is worth using:

• Ingest on a schedule, with the source attached. The data is the product; you have to be as careful with it as with your own database.
• Never fill a gap with a guess that looks like data. Age the last real figure and show it as aged.
• Be explicit about what the data does not explain. The most common failure in this kind of product is the confident narrative attached to a number that only says what it says.

All three of those are cheap to do correctly at the start, and very expensive to retrofit.