The proposals, outlined in a draft document seen by AFP, apply to migrants who have been denied asylum but cannot be deported and who have been living in Germany for at least five years.
Around 130,000 such migrants will be granted a "right of opportunity to stay" lasting for one year, according to the document.
After that, if they can prove they have a reliable source of income and a sufficient command of the German language, they will be granted official permanent residency.
The draft bill includes plans to make it easier for migrants to be joined by family members, especially if they are skilled workers, and to improve access to vocational and language courses.
It also proposes measures to make it easier to deport asylum seekers with criminal convictions.
"These people, who have made a life for themselves in Germany over a long period of residence, are to be offered a perspective under residence law and given a chance to obtain the necessary requirements for legal residence," according to the document.
The changes must still be voted through in the Bundestag and Bundesrat lower and upper houses of parliament before becoming law.
Critics said the plans do not go far enough and accused Germany's coalition government of watering down a promise to completely overhaul the country's migration system.
"The aim is good and fair: to give a chance to people who so far have no secure status," said Joshua Hofert, a board member at the Terre des Hommes NGO, but "the paradigm shift announced by the coalition is not yet in sight".