Unlucky 13 - the M1 junction's weirdness explained

As a frequent traveller to Bedford via Milton Keynes and the A421, I have often wondered why this road suddenly and inconveniently veers south-east and travels alongside the M1 motorway for a couple of miles before meeting it at junction 13, and then goes north again on the other side, adding several unnecessary miles to the journey. Having discovered that junction 13 itself in its original form featured in road enthusiasts' website CBRD's list of bad junctions, piecing together the various reader comments solved the mystery of the bigger picture and why the entire road layout in that area is a bit strange, so here is the explanation.

Why does the A421 go down to the motorway junction and back up the other side? Read on to find out...