He became Earl of Desmond when his father died in 1536. His father was de facto earl. In fact, James' 1st cousin once removed, also named James Fitzgerald, was 13th earl from 1534-1541, but his cousin was never in de facto possession.

This James began by repudiating the daughter of Viscount Fermoy, by whom he had a son, Thomas Ruadh (red). The Earl then married the daughter of O'Carroll, by whom he had two sons, Gerald or Garrett, and John. His second wife having died, he married the daughter of M'Carty, who was mother to James. After the Earl's death a doubt existed regarding the legitimacy of the eldest son, Thomas Ruadh; and Garret, the son of Desmond by his second wife, the daughter of O'Carroll, was named Earl. The name of the Countess of Desmond was Mora ni O'Carroll, which appears from a Fiant of Henry VIII AD 1541 (202)--Grant of English liberty to Mora Ene O'Carroll, wife of the Earl of Desmond and her issue.32