Shortly after, the duchess, anxious to retain their former chaplain in the neighbourhood, gave Crabbe a letter to Thurlow, asking him to exchange the two livings in Dorsetshire for two other, of more value, in the Vale of Belvoir.

Crabbe waited on the Chancellor with the letter, but Thurlow was, or affected to be, annoyed by the request.

It was a thing, he exclaimed with an oath, that he would not do "for any man in England." However, when the young and beautiful duchess later appealed to him in person, he relented, and presented Crabbe to the two livings of Muston in Leicestershire, and Allington in Lincolnshire, both, within sight of Belvoir Castle, and (as the crow flies) not much more than a mile apart.

To the rectory house of Muston, Crabbe brought his family in February 1789.

His connection with the two livings was to extend over five and twenty years, but during thirteen of those years, as will be seen, he was a non-resident.