Born in Newton, Massachusetts, known today to be located in Canton, Massachusetts , Sherman established a legal career in Litchfield County, Connecticut despite a lack of formal education. After a period in the Connecticut House of Representatives, he served as a Justice of the Superior Court of Connecticut from 1766 to 1789. He represented Connecticut at the Continental Congress and signed the Continental Association, which provided for a boycott against Britain following the imposition of the Intolerable Acts. He was also a member of the Committee of Five that drafted the Declaration of Independence. He later signed both the Articles of Confederation and the United States Constitution. In 1784, he was elected as the first mayor of New Haven, Connecticut.