Rishi Sunak is facing a significant rebellion from Tory MPs over legislation to revive his plan to send some asylum seekers to Rwanda.
Over 30 backbenchers on the right of the party are backing plans to change the bill next week to make it harder for people to appeal deportation.
The amendments underline the scale of Tory division over the policy, which the PM has made a priority.
Ministers insist the bill allows only a "vanishingly small" number of appeals.
Those backing the amendments include former Tory leader Sir Iain Duncan Smith and a clutch of former cabinet ministers, including former home secretary Suella Braverman.
Ex-migration minister Robert Jenrick, who resigned over the draft law last month and is a leading rebel, said the current bill would fail to prevent a "merry-go-round" of individual appeals.
The Rwanda policy dominated Prime Minister's Questions, with Labour leader Sir Keir Starmer arguing the prime minister had been "taken hostage by his own party".
He added that reports over the weekend that he had doubts about the policy when he was chancellor showed Mr Sunak had been "caught red handed".
"He knows the Rwanda gimmick won't work - but he can't be honest about it because he's too scared of his own MPs," he added.
In reply, the prime minister said the government would deliver on its promise to halt migrants crossing the Channel in small boats.
Mr Sunak said Labour had not put forward a "single practical idea" to reduce illegal migration, and "can never be trusted to stop the boats".
The government introduced the bill last month, after its plan to send asylum seekers to the east African country was ruled unlawful by the Supreme Court.
The legislation seeks to declare in UK law that Rwanda is a safe country to send refugees to - thereby stopping flights being grounded on legal grounds.
Ministers could ignore emergency orders from the European Court of Human Rights (ECHR) to suspend a flight to Rwanda while an individual legal case was being heard.
But rebels insist it will still allow the policy to be derailed by a wave of individual appeals, and want to tighten the circumstances in which they would be allowed.
They also want to make it the default position that ministers would ignore injunctions from the ECHR blocking flights.
The amendments are destined to fail at the vote next Tuesday as they will struggle to attract the required support from Labour MPs to overturn the government's majority.
However, the rebels could put the government's bill in jeopardy if they vote it down at a later stage if it remains unamended.
A group of 29 MPs would be big enough to overturn Mr Sunak's 56-seat majority, if they were to vote with Labour, which opposes the Rwanda policy.
Speaking to BBC Radio 4's Today programme, Mr Jenrick did not rule out voting the whole bill down if their proposed changes were not successful.
However, he added he was "not looking ahead to that" and the rebels were hoping to "win the argument".
He said the current version of the bill was "guaranteed to fail" because it would not provide a "sustainable deterrent" to people crossing the Channel to claim asylum.
He added that appeals to deportations should only be allowed in a limited number of cases, such as for women who are pregnant and those unfit to fly.
Tory rebels had threatened to vote against the bill before Christmas - but in the end they only abstained, meaning passed its initial parliamentary stage fairly comfortably.
Ministers could make concessions to the rebels in order to win them round, but it is thought this is unlikely to happen until the bill reaches its final stages in Parliament.
And the government is also facing pressure from Tory MPs on the more liberal One Nation wing of the party, who insist the bill cannot be tightened further without breaching international law.
The Rwandan government has also threatened to pull out of the scheme if it does not comply with international obligations.
Downing Street said Mr Sunak would consider amendments put forward by Tory MPs, and "engagements" were taking place with backbenchers.
However, his spokesperson told reporters the government would not accept any changes that could "jeopardise" the UK's deal with Rwanda.
"We need a bill with a respectable legal argument and one which does not collapse the scheme," they added.