In a speech in London, Sunak said while AI could bring opportunities for economic growth and the chance to solve problems that were once thought to be "beyond us," it would also raise "new dangers and new fears."
"The responsible thing for me to do is to address those fears head on, giving you the peace of mind that we will keep you safe, while making sure you and your children have all the opportunities for a better future that AI can bring," he said.
"Doing the right thing, not the easy thing, means being honest with people about the risks from these technologies."
Sunak's speech comes ahead of a two-day international gathering starting next Wednesday at Bletchley Park, central England, where top British codebreakers cracked Nazi Germany's "Enigma" code.
Bringing together world leaders, experts and others, it aims to "build a shared global understanding of the risks" posed by AI, according to Sunak's office.
During his speech, he announced that Britain would create an AI Safety Institute to examine and test new types of AI and explore its risks, including social harms like bias and misinformation.
The only people currently testing the safety of AI are the organizations developing it, Sunak noted. They should not be relied on to "mark their own homework," he said.
'Risks'
A government study also published Thursday, written with the support of 50 experts, warns that AI has the potential to enhance terrorist capabilities in developing weapons, planning attacks and producing propaganda.
It says generative AI, which creates text and images from written prompts, significantly increases risks to safety and security.
"By 2025, generative AI is more likely to amplify existing risks than create wholly new ones, but it will increase sharply the speed and scale of some threats," the paper says.
Risks in the digital sphere, such as cyber-attacks, fraud, scams, impersonation and child sexual abuse images, are most likely to manifest and to have the highest impact, it adds.
Global regulation is incomplete and "highly likely" failing to anticipate future developments, the report notes.
It recommends industry, academia, civil society, governments and the public all collaborate to help regulate the sphere.
AI is tipped to increasingly perform tasks more efficiently than humans, but Sunak stressed the technology should be thought of as more of a "co-pilot," that can help people do their job.
He cited the example of a care worker who could use the technology to help with paperwork.
However, the British leader noted AI's future role in an evolving labour market was unpredictable.
The best way to prepare the country for those possible changes, he said, was to educate people in a way that will properly equip them for the future.
Forum