Can I have a postcode to postcode drive time and distance lookup table?
With online mapping data increasingly accessible to the public, journey planning seems easier than ever. However, once you move beyond individual journeys to look at onward travel, planning for a fleet of vehicles, or a network of locations, things quickly get complicated. One question new clients often put to us is “Can we have a simple table of travel times between each postcode?” Unfortunately, this seemingly simple question comes with some significant challenges.
When do you need it?
There are around 1.77 million live postcodes in the UK (1,766,067 active postcodes as at June 2020 according to the National Statistics Postcode Directory/NSPD plus 874,449 terminated postcodes). However, the exact figure changes on a monthly basis as new postcodes are created and old ones terminated. This means to do a postcode to postcode lookup you would need that many results squared, that’s about 3.1 trillion variations! How long would it take to produce the drive time and distance lookup tables for this? Well, if we assume it averages just one second to calculate the best time and distance between each postcode we can calculate the compute time needed to generate our data. 3.1 trillion rows will take 3.1 trillion seconds at one second per postcode to postcode pair. A year is just under 32 million seconds (31,536,000).
This means that the pure compute time is almost 100,000 years (98,902 years to be exact)!
To allow for variations in travel time for peak, off-peak, HGV, and shortest distance driven (irrespective of road or traffic conditions), we’d need 4 calculations, so we are looking at a compute time of 400,000 years for a single end to end process. Using more computers can speed it up:
- If we have 4 computers, one for each task, we would have needed to start processing when the first buildings were constructed in Egypt.
- If we can speed the process up to 10 per second by using better code, we could have started processing 10,000 years ago during the last ice age as the final megafauna, like the woolly mammoth and sabre tooth tiger went extinct.
- 10 computers per task (not forgetting there are 4 results to calculate) and we could have started processing 1,000 years ago when King Canute was on the throne in England, Denmark and Norway.
- 100 computers per task; we could have started the processing whilst worrying about Spanish influenza, the Russian Revolution and World War 1.
- 1,000 computers per task and we could have been listening to Dizzie Rascal or Lady Gaga topping the charts.
- Even 20,000 computers would still take 6 months! To get a real time process we would need 200,000 computers and it would still take 18 days to process all 4 sets of data.
What's your budget?
Now we have an idea about how long it would take to produce postcode to postcode drive time and distance tables, how much storage space do you think we will need?
The sector to sector table is about 7.5Gb and has 65 million rows. Our postcode to postcode table is 27,000 times as big. This means our new data table in an unindexed form would be 387,297 terabytes or 378 petabytes (406,109,876,742 Mb).
Up to 2014 Google had indexed about 200 terabytes of data. Our one data table is 200 times as much data as Google has handled in 16 years.
There is a 60 terabyte disk available. We would need 6,500 of them, they are expected to retail between $30,000 and $40,000 each, so let us be optimistic and say $195 million in total. They would draw power at 1 watt per terabyte so we would need a power station producing 385 Mw to keep just the disks running (about an eighth of the output of a new nuclear power station, that retail at around £20 billion each).
A realistic yet accurate alternative
So it would seem that postcode to postcode data tables are a non-starter. Fortunately, we have an alternative solution that can return time and distance for any 2 postcodes in less than 2 seconds. Our TimeTravel product offers postcode sector to postcode sector drive times and distances for all of the UK without the massive amounts of infrastructure or forward planning we’ve previously described. We also update the system as new postcodes and road networks alter the calculations to keep it accurate.
The TimeTravel data table has distance and time values for peak, off-peak, HGV and shortest distance driven (irrespective of traffic conditions or time of day). It even accounts for Ferry crossing times, where travel to that part of the UK goes over water. To process all combinations takes days, rather than thousands of years, and we provide regular updates to our clients so that their customers and fleet drivers have the most accurate drive times available within seconds.