Born on October 6, 1853, in Providence, Rhode Island, Frederick Burgess graduated from Brown University in 1873 before pursuing his theological studies at the General Theological Seminary in New York. He was ordained as a deacon in 1876 and was later ordained into the priesthood in 1878 by the Rt. Rev. Thomas M. Clark. Notably, ministry ran in his family; his uncle, Bishop George Burgess, had served as the first bishop of Maine. Frederick married Caroline G. Bartow, and the couple had four sons—Julian Howard, Edgar Bartow, George, and Frederick—prior to Caroline's passing in 1894.
Following his ordination, Burgess served a variety of parishes across Massachusetts, Connecticut, Pennsylvania, and Detroit. In 1898, the same year he received an honorary Doctor of Divinity from Brown University, he became the rector of Grace Church in Brooklyn.
In 1901, Burgess was elected the second Bishop of Long Island. He was officially consecrated on January 15, 1902, at Grace Church in Brooklyn Heights by the Rt. Rev. Henry C. Potter, Bishop of New York. He served faithfully in this episcopate until he died in office on October 15, 1925. Bishop Burgess was subsequently laid to rest in the crypt of the Cathedral of the Incarnation.