ramlingham Castle's location was probably fortified in some way a long time before the Normans first starting building their castle on it. After the Conquest Roger Bigod, a Norman knight who may have assisted in the invasion, was granted lands in Essex and Sussex. In 1101 he was given the lands of Framlingham by Henry I. Roger died in 1107 and his first son died on the 'White Ship' disaster along with William Adelin in 1120.

Roger's second son, Hugh, inherited the lands and became the Earl of Norfolk. Hugh probably built a motte type castle at Framlingham around 1140. During the Civil War between Stephen and Matilda Hugh changed sides as it pleased him, but when Henry II came to the throne, Hugh was confirmed as owner of the lands at Framlingham. Hugh had assisted Henry with his claim to the throne and was awarded accordingly. Hugh soon fell out with Henry and the castle was confiscated by the king. In 1165, Hugh raised enough money to buy the castle back off the Crown, but Henry built Orford Castle nearby to keep Hugh under control. After Hugh's involvement with the Baron's revolt in 1173, the castle again was confiscated by the Crown and destroyed. When Richard became King, Hugh was dead, and his son Roger now in favour with the King was able to regain ownership of Framlingham. Roger rebuilt the castle consisting of the thirteen towers of the curtain wall and the defensive earthworks.

In 1476 Framlingham Castle passed to John Howard, the Duke of Norfolk when he married into the Mowbray family. Previously the castle had belonged to the Mowbrays but the last male descendant died without an heir.

The castle became Crown property when King Henry VIII was either granted it or confiscated it. The castle became the home of Mary Tudor during the period when Lady Jane Grey threatened to take the English Crown and it was at Framlingham that Mary's supporters gathered in 1553 to fight for her cause.