Two mulattoes, Will and Frank, bought of Mary Lee in 1768, cost L61.15 and L50, and Will became famous as a body servant; Adam and Frank, bought of the same owner, cost L38.

He bought five more slaves in 1772.

Some writers say that this was his last purchase, but it is certain that thereafter he at least took a few in payment of debts.
In 1786 he took a census of his slaves on the Mount Vernon estate.

On the Mansion House Farm he had sixty-seven, including Will or Billy Lee, who was his "val de Chambre," two waiters, two cooks, three drivers and stablers, three seamstresses, two house maids, two washers, four spinners, besides smiths, a waggoner, carter, stock keeper, knitters and carpenters.

Two women were "almost past service," one of them being "old and almost blind." A man, Schomberg, was "past labour." Lame Peter had been taught to knit.