The first James had greatly enlarged and strengthened its works defensive.

He had added thirty feet to the height of David's Tower, which now served as a watch-station over all the rock, and in his last days he had begun to build the great hall which the Chancellor had but recently finished.
It was here that presently the feast was set.

The banquet-hall ran the width of the keep, and the raised dais in the centre was large enough to seat the whole higher baronage of Scotland, among whom (as the Earl of Douglas thought with some scorn) neither of his entertainers, Crichton and Livingston, had any right to place themselves.
But the question where the Lady Sybilla was bestowed soon occupied the Douglas more than any thought of his own safety or of the loyalty of his entertainers.

Sybilla, however, was neither in the courtly cavalcade which met them at the entrance of the park, nor yet among the more numerous ladies who stood at the castle yett to welcome to Edinburgh the noble and handsome young lords of the South.
Douglas therefore concluded that de Retz, discovering some part of the love that was between them, or mayhap hearing of it from some spy or other at Crichton Castle, had secluded his sweetheart.

He loosened his hand on the rein to lay it on the sword-hilt, as he thought of this cruelty to a maid so pure and fair.
Sholto kept his company very close behind him as they rode up the High-street, a gloomy defile of tall houses dotted from topmost window to pavement with the heads of chattering goodwives, and the flutter of household clothing hung out to dry.
At the first defences of the castle Douglas called Sholto and said: "Your fellows are to be lodged here on the Castle Hill.