"Shinzo Abe was transported to (the hospital) at 12:20 pm. He was in a state of cardiac arrest upon arrival. Resuscitation was administered. However, unfortunately he died at 5:03 pm," said Hidetada Fukushima, professor of emergency medicine at Nara Medical University Hospital.
The attack came before noon in the country's western region of Nara, where Abe, 67, had been delivering a stump speech with security present, but spectators able to approach him easily.
Footage broadcast by NHK showed him standing on a stage when a man dressed in a grey shirt and brown trousers begins approaching from behind, before drawing something from a bag and firing.
At least two shots appear to be fired, each producing a cloud of smoke.
As spectators and reporters ducked, a man was shown being tackled to the ground by security. He was later arrested on suspicion of attempted murder, reports said.
Local media identified the man as 41-year-old Tetsuya Yamagami, citing police sources, with several media outlets describing him as a former member of the Maritime Self-Defense Force, the country's navy.
He was wielding a weapon described by local media as a "handmade gun", and NHK said he told police after his arrest that he "targeted Abe with the intention of killing him".
The assassination of the country's best-known politician comes despite Japan's strict gun laws and with campaigning under way ahead of upper house elections on Sunday.
Shinzo Abe smashed records as Japan's longest-serving prime minister, championing ambitious economic reform and forging key diplomatic relationships while weathering scandals.
Abe was a sprightly 52 when he first became prime minister in 2006, the youngest person to occupy the job in the postwar era.
He was seen as a symbol of change and youth, but also brought the pedigree of a third-generation politician groomed from birth by an elite, conservative family.
Abe's first term was turbulent, plagued by scandals and discord, and capped by an abrupt resignation.
After initially suggesting he was stepping down for political reasons, he acknowledged he was suffering an ailment later diagnosed as ulcerative colitis.
He ran again, and Japan's revolving prime ministerial door brought him back to office in 2012.
On the international stage, Abe took a hard line on North Korea, but sought a peacemaker role between the United States and Iran.
He prioritized a close personal relationship with Donald Trump in a bid to protect Japan's key alliance from the then-US president's "America First" mantra, and tried to mend ties with Russia and China.
Abe had been due to stay on until late 2021, giving him an opportunity to see out one final event in his historic tenure -- the postponed Tokyo 2020 Olympics Games.
But in a shock announcement, he stepped down in August 2020, with a recurrence of ulcerative colitis ending his second term, too.
Numerous tributes to the slain former PM have been issued.
NATO chief Jens Stoltenberg said he was "deeply saddened by the heinous killing" of Abe.
British Prime Minister Boris Johnson Friday hailed the leadership role taken by Abe and said the UK stands with Japan "at this dark and sad time."