Charles Augustus Murray wrote of it in his quaint "Travels in North America," published in 1839) "situated on an eminence commanding a view of the town, and of the bold, sweeping course of the Rappahannoc." Murray also tells of the beautiful garden, with its great box trees and its huge slave-built terraces, stepping down to the water like a giant's stairway.
In this house my companion and I were guests, and as I won the toss for the choice of rooms, mine was the privilege of sleeping in the historic west bedchamber, the principal guest room, and of opening my eyes, in the morning, upon a lovely wall all paneled in white-painted wood.
I shall always remember the delightful experience of awakening in that room, so vast, dignified, and beautiful, and of lying there a little drowsy, and thinking of those who had been there before me.

This was the room occupied by George and Martha Washington when they stopped for a few days at Chatham on their wedding journey; this was the room occupied by Madison, by Monroe, by Washington Irving, and by Robert E.Lee when he visited Chatham and courted Mary Custis, who became his wife.

And, most wonderful of all to me, this was the room occupied by Lincoln when he came to Fredericksburg to review the army, while Chatham was Union headquarters, and the embattled Lee had headquarters in the old house known as Brompton, still standing on Marye's Heights back of the river and the town.

It is said that Lee during the siege of Fredericksburg never trained his guns on Chatham, because of his sentiment for the place.

As I lay there in the morning I wondered if Lee had been aware, at the time, that Lincoln was under the roof of Chatham, and whether Lincoln knew, when he slept in "my" room, that Washington and Lee had both been there before him.
War, I thought, not only makes strange bedfellows, but strange combinations in the histories of bedrooms.
Then the maid rapped for the second time upon my door, and though this time I got up at once, my ruminations made me scandalously late for breakfast.
After breakfast came the motor, which was to take us to the battlefields, its driver a thin dry-looking, dry-talking man, with the air of one a little tired of the story he told to tourists day in and day out, yet conscientiously resolved to go through with it.