Conable was born in Warsaw, New York on November 2, 1922. Conable graduated from Cornell University in 1942. He then enlisted in the Marines and fought in the Battle of Iwo Jima. After the war, he received his law degree from Cornell University Law School in 1948. He later re-enlisted and fought in the Korean War.
In 1962, Conable was elected as a Republican to the New York State Senate. Two years later, Conable was elected to the U.S. House of Representatives in 1964. He was reelected nine more times. He was voted by his colleagues the "most respected" member of Congress. He refused to accept personal contributions larger than $50. For many years, Conable was the ranking minority member of the House Ways and Means Committee, where he authored a tax code provision that made so-called 401(k) and 403(b) defined-contribution retirement plans possible.
A long-time ally of Richard Nixon, Conable supported Nixon until the White House released a tape of Nixon instructing his Chief of Staff H. R. Haldeman to obstruct the FBI investigation. This caused Conable to join those asking for Nixon's resignation.
Conable did not seek reelection in 1984. In 1986, President Ronald Reagan appointed him president of the World Bank. Conable was the first career politician appointed to lead the bank. In May 1987, Conable reorganized the bank to promote efficiency and reduce waste. He retired in 1991.