He fought on the Lancastrian side at the battle of Towton but, together with his father, later transferred his allegiance to the Yorks. He became well known and influential at court when his sister Elizabeth married Edward IV and the Woodvilles became a force to be reckoned with. After his sister's marriage to the King, Anthony became a Knight of the Garter and was given the honour of Lordship of the Isle of Wight. He was promoted to be Lieutenant of Calais and captain of the king's armada, among many other honorary positions.

It is said he fought a two day 'duel' with the Bastard of Burgundy, which is an indication of his fighting prowess. They started out on horseback and ended up fighting with daggers, until the King intervened to prevent death, which would not have been welcomed by anyone. Later Anthony was part of the group which went to Bruges, there to negotiate the marriage of Edward IV's sister Margaret to Charles the Bold.

Anthony's father and brother were both beheaded after the battle of Edgecote and he succeeded to the title of Earl Rivers. He accompanied Edward of March into exile in 1470 and fought with him at Barnet, where he was wounded. Edward thought highly of him and appointed him councillor to the young Prince of Wales, essentially putting his son into Anthony's care and keeping. This was pivotal in Anthony's future, although at the time it was a highly lucrative and important position to hold. Later he was actually made a guardian of the young prince.

It was probably during this time that he became a writer, and also began his pilgrimages to holy sites. His pilgrimage to Rome resulted in Pope Sextus IV investing him with the title of defender and director of papal causes in England.

• He was a serious patron of the arts, working with Caxton in translating and producing books. He was responsible for the first ever book printed in England, The Dictes and Sayings of the Philosophers. Overall Anthony Woodville was known as a courtier and writer, as well as a devout and devoted man.
• In 1483 Edward appointed his brother Richard of Gloucester as protector for his young son Edward. When the king died in April 1483, Richard intercepted the group heading for London with the boy, now officially Edward V, ordered the arrest of Earl Rivers and those with him and had them imprisoned at Sherriff Hutton Castle, near York. Later they were all executed at Pontefract Castle. He died on the 25th June 1483, a victim of his position, of the paranoia of the time and of the circumstances and problems left behind by Edward IV.