Berkhamsted Castle was first built as a motte-and-bailey castle in the late 11th century. Kept in royal hands, it was occupied by key figures of the Middle Ages, such as Thomas Becket, Richard, Earl of Cornwall, and the Black Prince. Much of its stone was plundered during the 16th century.

The lodge situated at the site of the castle dates from 1865.