The aircraft entered service in early 1937, soon seeing action in pitched aerial battles at the start of the Second Sino-Japanese War, including air-to-air battles with the Republic of China Air Force's Boeing P-26C Model 281 "Peashooters" in what was the world's first-ever aerial dogfighting and kills between monoplane fighters built of mostly metal.
Chinese Nationalist pilots, primarily flying the Curtiss Hawk III, put up a valiant fight against the Japanese, but the A5M was the better of almost every fighter aircraft it encountered. Though armed with only a pair of 7.7 mm machine-guns, the new fighter proved effective and damage-tolerant, with excellent maneuverability and robust construction. Later on A5M's also provided much-needed escorts for the then-modern but vulnerable Mitsubishi G3M bombers.
The Mitsubishi team continued to improve the A5M, working through versions until the final A5M4, which carried an external underside drop tank to provide fuel for extended range.
The A5M's most competitive adversary in the air was the Polikarpov I-16, a fast and heavily armed fighter flown by both Chinese Air Force regulars and Soviet volunteers. Fierce air battles in 1938, especially on 18 February and 29 April, ranked among the largest air battles ever fought at the time. The battle of 29 April saw 67 Polikarpov fighters (31 I-16s and 36 I-15 bis) against 18 G3Ms escorted by 27 A5Ms. Each side claimed victory: the Chinese/Soviet side claimed 21 Japanese aircraft (11 fighters and 10 bombers) shot down with 50 Japanese airmen killed and two captured having bailed out while losing 12 aircraft and 5 pilots killed; the Japanese claimed they lost only two G3Ms and two A5Ms shot down with no less than 40 Chinese planes shot down.
104 A5M aircraft were modified to accommodate a two-seater cockpit. This version, used for pilot training, was dubbed the A5M4-K. K version planes continued to be used for pilot training long after standard A5Ms left front-line service.
Almost all A5Ms had open cockpits. A closed cockpit was tried but found little favor among Navy aviators. All had fixed, non-retractable undercarriage. Wheel spats were a feature of standard fighters but not training planes.
The Flying Tigers encountered the Type 96, although not officially, and one was shot down at Mingaladon airfield, Burma on 29 January 1942.
Some A5Ms remained in service at the end of 1941 when the United States entered World War II in the Pacific. US intelligence sources believed the A5M still served as Japan's primary Navy fighter, when in fact the A6M 'Zero' had replaced it on first-line aircraft carriers and with the Tainan Kokutai in Taiwan. Other Japanese carriers and Kokutai (air groups) continued to use the A5M until production of the Zero caught up with demand. The last combat actions with the A5M as a fighter took place at the Battle of the Coral Sea on 7 May 1942, when two A5Ms and four A6Ms of the Japanese carrier Shoho fought against US planes that sank their carrier.
In the closing months of the war most remaining A5M airframes were used for kamikaze attacks.