In 1941, he started undergraduate coursework in engineering at Georgia Southwestern College in nearby Americus, Georgia. The next year, Carter transferred to the Georgia Institute of Technology in Atlanta, where civil rights icon Blake Van Leer was president. While at Georgia Tech, Carter took part in the Reserve Officers' Training Corps. In 1943, he received an appointment to the Naval Academy from U.S. Representative Stephen Pace, and Carter graduated with a Bachelor of Science in 1946. He was a good student, but was seen as reserved and quiet, in contrast to the academy's culture of aggressive hazing of freshmen

In 1952, Carter began an association with the Navy's fledgling nuclear submarine program, led by then-Captain Hyman G. Rickover. Rickover had high standards and demands for his men and machines, and Carter later said that, next to his parents, Rickover had the greatest influence on his life. Carter was sent to the Naval Reactors Branch of the Atomic Energy Commission in Washington, D.C., for three-month temporary duty.

On 12 December 1952, the NRX reactor suffered a partial meltdown due to operator error and mechanical problems in the shut-off systems. For test purposes, some fuel channels had been disconnected from high-pressure water cooling and were connected by hoses to a temporary cooling system; one low-power channel was cooled only by airflow.

Carter was ordered to Chalk River to lead a U.S. maintenance crew that joined other American and Canadian service personnel to assist in the shutdown of the reactor. The painstaking process required each team member to don protective gear and be lowered individually into the reactor for 90 seconds at a time, limiting their exposure to radioactivity while they disassembled the crippled reactor. When Carter was lowered in, his job was simply to turn a single screw. During and after his presidency, Carter said that his experience at Chalk River had shaped his views on atomic energy and led him to cease the development of a neutron bomb.

He served in the inactive Navy Reserve until 1961 and left the service with the rank of lieutenant. Carter's awards include the American Campaign Medal, World War II Victory Medal, China Service Medal, and National Defense Service Medal. As a submarine officer, he also earned the "dolphin" badge.

USS Jimmy Carter (SSN-23) is the third and final Seawolf-class nuclear-powered fast-attack submarine in the United States Navy. Commissioned in 2005, she is named for the 39th president of the United States, Jimmy Carter, the only president to have qualified on submarines. The only submarine to have been named for a living president, Jimmy Carter is also one of the few vessels, and only the third submarine of the US Navy, to have been named for a living person.

While at the Academy, Carter fell in love with Rosalynn Smith, a friend of his sister Ruth. The two wed shortly after his graduation in 1946, and were married until her death on November 19, 2023