This prominent vantage point and strategic site was quickly recognised by the Normans, who built a huge timber fortress here by 1100 AD. In the years shortly after the Norman invasion of 1066, William the Conqueror is believed to have ordered defences to be built against a still hostile and rebellious native community. The fortunes of the castle and its owners, the Stafford family, fluctuated greatly. Earl Ralph, a founder member of the Order of the Garter, built a massive stone keep on top of the motte in 1347. In 1444, Humphrey Stafford was created Duke of Buckinghamshire and the castle reached its heyday.