A native of Fayetteville, Kelso had returned to live in the Tennessee town in 2003.
He was in Norfolk to attend the graduation of his grandson from high school, according to the Elk Valley Times in Fayetteville.
As CNO, Kelso oversaw revolutionary changes within the OPNAV staff and changed the means by which the Navy processed and made decisions, according to Naval News Service.
Kelso helped to transform the Naval organization and the processes by which information could be shared and considered, the NNS noted.
Kelso strongly advocated the integration of women into the Navy, particularly in the wake of the 1991 Tailhook Convention during which numerous incidents of sexual assault and harassment were found to have occurred.
Following his graduation from the U.S. Naval Academy in 1956, he served aboard the USS Oglethorpe (AKA 100). After attending submarine school, he was assigned to the submarine USS Sabalo (SS 302). He also served aboard the nuclear-powered attack submarine USS Pollack (SSN 603), USS Daniel Webster (SSBN 626) and USS Sculpin (SSN 590).
From January 1969 to August 1971, Kelso served as commanding officer, U.S. Naval Nuclear Power School in Bainbridge, Md. Following tours included the USS Finback (SSN 670); staff of commander, Submarine Force, U.S. Atlantic Fleet; and commanding officer, USS Bluefish (SSN 675).
From September 1975 to July 1977, Kelso was executive assistant to the Commander in Chief, U.S. Atlantic Command and U.S. Atlantic Fleet and Supreme Allied Commander Atlantic.
He served as commander, Submarine Squadron SEVEN until reporting as division director, Submarine Distribution Division in the Naval Military Personnel Command, and Section Head of the Submarine Programs Section in the Office of the Deputy Chief of Naval Operations (Manpower, Personnel and Training) in September 1978.
After his selection for flag rank, Rear Adm. Kelso served as director of Office of Program Appraisal, Office of the Secretary of the Navy.
In 1985, he became commander of 6th Fleet and NATO commander Naval Striking Force and Support Forces Southern Europe.
In 1986, he was promoted to admiral and became commander in chief, U.S. Atlantic Fleet. He became Supreme Allied Commander Atlantic and Commander in Chief, U.S. Atlantic Command in November 1988. He became chief of Naval Operations in 1990.
His awards included the Defense Distinguished Service Medal, the Navy Distinguished Service Medal (three awards), Legion of Merit (four awards), Meritorious Service, Navy Commendation and Navy Achievement Medals.
Burial for Kelso will be held Saturday in Fayetteville. He is survived by his wife, Georgeanna; four children; and several grandchildren.