Tiny project: where to live in London

I had an idea this afternoon for a little app to do something that I have wanted done at various times.

Namely, something to help me find a good place to live in London based on commute times to a number of locations.

For example, given your workplace, a family member’s house, a friend’s house, your housemate’s workplace etc. what is the best place to live based on average commute time.

I’m trying to get better at doing tiny projects which I can build quickly, rather than bigger projects I never finish, so I set myself a challenge to deploy something to the web in an afternoon, which at least partially solved the problem I had posed.

I had a rough idea about how it might work, and so I scribbled some notes and got going trying to prove it. Here are the notes I came up with, just to give an idea of the process (I apologise for my writing…):

Ugly UI:

Use the TFL journey planner API, split London into a grid of latitude/longitude locations, and try testing out how long it takes to get from the middle of each grid tile to the locations the user has specified:

Hopefully this will be the data we end up with, a series of tile locations in a dictionary, along with the journey times to each of our user’s locations:

And lastly, to avoid going off on a tangent, these are the things I wanted to prove I could do in some form:

In case that one is too illegible, the things I wanted to prove were:

  • Split London into a grid of tiles, defined by the latitude and longitude of the centre of the tile.
  • Calculate travel times from the centre of each grid tile to the user specified locations.
  • Sort the results based on the best average commute time and fairest split of times, and report back with the best ones.

It ended up working quite well.

First of all I played about with the Transport For London (TFL) API, and figured out that if I registered and got an API key, I could make 500 requests a minute for free. More than enough for my purposes for now.

Then I used Postman to test the API, pass it some locations, and see what form the data came back in. This enabled me to get a simple JavaScript script written which I ran with Node.JS and which allowed me to find the time taken for the shortest journey from point A to point B.

I had to remind myself how to make http requests from inside Node.JS, and ended up using axios which appears to be quite a nice http client, based around promises.

At this point, I had proved that I could pass the API a postcode, and a latitude/longitude coordinate, and get the shortest journey time between the postcode and the lat/long coordinate.

Next, I had to figure out how to split London up into a grid of lat/long coordinates. This was fairly hacky. I ended up clicking on google maps roughly where I wanted to define the North, South, East and West limits of ‘London’, and copying the values it returned as hard coded values for my program, in order to structure my grid of coordinates.

This highly fuzzy methodology eventually gave me the following piece of code:

const generateGrid = () => {
  const bottom = 51.362;
  const top = 51.616;
  const left = -0.3687;
  const right = 0.1722;

  const gridHeight = 6;
  const gridWidth = 10;

  const heightIncrement = (top - bottom) / gridHeight;
  const widthIncrement = (right - left) / gridWidth;

  const grid = [];

  let centeredPoint = [bottom, left];

  for (let i = 0; i < gridWidth; i++) {
    for (let j = 0; j < gridHeight; j++) {
      grid.push([...centeredPoint]);
      centeredPoint[0] += heightIncrement;
    }
    centeredPoint[1] += widthIncrement;
    centeredPoint[0] = bottom;
  }

  return grid;
};

The grid is wider than it is tall, because London is wider than it is tall, which I think makes sense…

So now I had an array of coordinates, representing the centres of my grid tiles spanning London, and the ability to figure out how long it takes to get from one of these coordinates to a given postcode.

I wrote some quite ugly code to loop through all the tiles, and calculate the commute times to each of the user locations, and to calculate the average of these times, and the spread of commute times (the difference between the largest and smallest time):

const getGridJourneys = async (grid, locationsToCheck) => {
  const gridJourneys = {};

  console.log("requesting");

  let withAverageAndSpread = {};

  await axios
    .all(
      grid.map(([gridLat, gridLong]) => {
        return axios.all(
          locationsToCheck.map((location) => {
            return axios
              .get(
                `https://api.tfl.gov.uk/journey/journeyresults/${gridLat},${gridLong}/to/${location}?app_id=${appId}&app_key=${appKey}`
              )
              .then((res) => {
                const key = `${gridLat},${gridLong}`;
                if (gridJourneys[key]) {
                  gridJourneys[key].push(fastestJourneyTime(res.data));
                } else {
                  gridJourneys[key] = [fastestJourneyTime(res.data)];
                }
              })
              .catch((err) => {
                console.error(err.response.config.url);
                console.error(err.response.status);
                return err;
              });
          })
        );
      })
    )
    .then(() => {
      console.log("request done");
      withAverageAndSpread = Object.keys(gridJourneys).reduce(
        (results, gridSquare) => {
          return {
            ...results,
            [gridSquare]: {
              journeys: gridJourneys[gridSquare],
              spread: gridJourneys[gridSquare].reduce(
                (prev, curr) => {
                  const newHighest = Math.max(curr, prev.highest);
                  const newLowest = Math.min(curr, prev.lowest);

                  return {
                    highest: newHighest,
                    lowest: newLowest,
                    diff: newHighest - newLowest,
                  };
                },
                {
                  lowest: Number.MAX_SAFE_INTEGER,
                  highest: 0,
                  diff: 0,
                }
              ),
              average:
                gridJourneys[gridSquare].reduce((prev, curr) => {
                  return prev + curr;
                }, 0) / gridJourneys[gridSquare].length,
            },
          };
        },
        {}
      );
    });
  return {
    gridJourneys,
    withAverageAndSpread,
  };
};

And with a bit more fiddling (and flushing out a lot of bugs), I had my program deliver me results in the following format

[
  {
    location: '51.404,-0.044',
    averageJourneyTime: 60,
    spread: 4,
    journeys: [ 58, 62 ]
  },
  {
    location: '51.446,-0.206',
    averageJourneyTime: 61,
    spread: 5,
    journeys: [ 58, 63 ]
  }
]

So now, in theory I had proved that my program worked, and after putting in some values, the answers I got from it seemed to make sense.

I was pretty happy given this had only taken an hour or so, and considered leaving it there, but I decided to spend another few hours and deploy it to the web.

I started a new Express.JS project, using their scaffolding tool, and pasted all my semi-working code into it. This took a bit of wrangling to make everything work as it did before, but wasn’t too bad.

Then I spent half an hour or so reminding myself how to use Jade/Pug to put together HTML templates, and how routing works in Express.

Eventually I ended up with two views:

A simple form

extends layout

block content
  h1 Where should I live in London
  p Add postcodes of up to 5 locations you want to be able to travel to
  form(name="submit-locations" method="get" action="get-results") 
    div.input
      span.label Location 1
      input(type="text" name="location-1")
    div.input
      span.label Location 2
      input(type="text" name="location-2")
    div.input
      span.label Location 3
      input(type="text" name="location-3")
    div.input
      span.label Location 4
      input(type="text" name="location-4")
    div.input
      span.label Location 5
      input(type="text" name="location-5")
    div.actions
      input(type="submit" value="Where should I live?")

and a results page

extends layout

block content
  h1 Results
  ul
  each item in data[0]
    li 
    div Location: #{item.location}
    div Average journey time: #{item.averageJourneyTime}
    div Journey times: #{item.journeys}
    div Journey time spread: #{item.spread}
    a(href="https://duckduckgo.com/?q=#{item.location}&va=b&t=hc&ia=web&iaxm=maps" target="_blank") Find out more

along with the routing for the results page

router.get("/get-results", async function (req, res, next) {
  const locations = Object.values(req.query)
    .filter((v) => !!v)
    .map((l) => l.replace(" ", ""));

  const results = await getResults(locations);

  if (results[0]?.length === 0) {
    res.render("error", {
      message: "Sorry that did not work. Please try again in a minute!",
      error: {},
    });
  } else {
    res.render("results", { data: results });
  }
});

For context, here is the getResults method

const getResults = async (locationsToCheck) => {
  const { gridJourneys, withAverageAndSpread } = await getGridJourneys(
    generateGrid(),
    locationsToCheck
  );

  const sorted = [
    ...Object.keys(gridJourneys).sort((a, b) => {
      if (withAverageAndSpread[a].average < withAverageAndSpread[b].average) {
        return -1;
      }
      if (withAverageAndSpread[a].average === withAverageAndSpread[b].average) {
        return 0;
      }
      if (withAverageAndSpread[a].average > withAverageAndSpread[b].average) {
        return 1;
      }
    }),
  ];

  const sortedListWithDetails = sorted.map((key) => {
    return {
      location: key
        .split(",")
        .map((i) => i.slice(0, 6))
        .join(","),
      averageJourneyTime: Math.round(withAverageAndSpread[key].average),
      spread: Math.round(withAverageAndSpread[key].spread.diff),
      journeys: withAverageAndSpread[key].journeys,
    };
  });

  return [sortedListWithDetails.slice(0, 5)];
};

I added the generic error message, because the API is rate limited, and if the rate is exceeded, the app doesn’t handle it very well…

I then used Heroku to deploy the app for free, which was, as ever a dream.

Here is a video of the app in action.

And here is the deployed app (assuming it is still up when you are reading this!)

https://where-should-i-live-london.herokuapp.com/

And here is the code.

Overall, I really enjoyed this little exercise, and while there are obviously huge improvements that could be made, it is already a better option (for me at least), than trying individual areas of London one at a time and checking CityMapper to see how long it takes to get to the places I care about.

If I come back to it, I might look into displaying the results on a map, which would allow me to clarify better that the results represent tiles, rather than a specific granular location.

I love how easy it is to quickly prototype and build things using free tooling these days, and it was really refreshing to write a server side web app instead of another single page application. I really believe that for quick proof of concept work, and prototypes, Express.JS and Heroku is a powerful combination. The code is nothing special but it is enough to prove the idea, and to get something running which can be improved upon if I want to later.

Leave a Reply

Your email address will not be published. Required fields are marked *