A photograph shows it as it was in 1880, with the tree from which it took its name in front, and the Henry W.Tyson Fifth Avenue Market adjoining it.

"Fifth Avenue" quotes from Mr.John T.
Mills, Jr., whose father owned the cottage: "My mother planted the old willow tree," said Mr.Mills, "and I remember distinctly the Orphan Asylum fire.

The only reason our home was not destroyed was that father ran the Bull's Head stages which carried people downtown for three cents, and the ruffians did not care to destroy the means of transportation." There were many vacant lots in this section of Fifth Avenue at the time of the Civil War, and a small shanty below the Willow Cottage was the only building that stood between Madison Avenue and Fifth Avenue.

On the north-west corner of Fifth Avenue and Forty-fourth Street, then considered far north, stood a three-story brick building.
The stockyards were between Fifth Avenue and Fourth Avenue from Forty-fourth to Forty-sixth Street, and Madison Avenue was not then cut through.

The stockyards were divided into pens of fifty by one hundred feet, into which the cattle were driven from runs between the yards.