Minister for Housing Eoghan Murphy has called talk of a Sinn Féin motion of no confidence in him as "an unnecessary distraction and possibly reckless behaviour".
Speaking as he arrived at the AGM of the Irish Council for Social Housing he said the housing issue is never about him.
He said he has confidence in the Rebuilding Ireland project, his desire is to see it through and to have thousands of homes built and people living in them.
Fianna Fáil with the confidence and supply agreement is there to make sure Government can do that, he said.
It comes as Minister Murphy's party colleague Patrick O'Donovan said Fine Gael will not consider going into Government with Sinn Féin.
Mr O'Donovan claimed that such a move would involve risking the security of the State.
Sinn Féin leader Mary Lou McDonald told her party's Ard Fheis at the weekend that they would not be excluded from government.
She also said it will be the decision of the people, and not the leaders of Fianna Fáil or Fine Gael, to determine the makeup of the next Government.
However, speaking on RTÉ's Morning Ireland, Mr O'Donovan said that this is "one deal that will not be done".
The Limerick County TD said Sinn Féin was not a suitable party for government because of its opposition to the Special Criminal Court and the Offences Against the State Act.
The Minister of State for Public Procurement, Open Government and eGovernment said that in order to have any sort of arrangement with Fine Gael, there must be unequivocal support for the Special Criminal Court.
He said the SCC is "the last line of defence between the citizens of this country and gangland criminals".
In addition, he said, there were too many gaps between Sinn Féin's economic policies and that of Fine Gael.
Mr O'Donovan added that Fine Gael was not working towards an election and said the party did not want a snap election.
The confidence and supply agreement between Fine Gael and Fianna Fáil, which facilitates a minority Government, expires after the next budget.
Meanwhile, Tánaiste Simon Coveney has said the party's position on Sinn Féin has not changed.
"The speeches and grandstanding over the weekend has not changed anything," he said.
Mr Coveney said he works constructively with Sinn Féin on building confidence in divolved government in Northern Ireland, but Fine Gael has said many times that Sinn Féin's policies are not compatible with what they are trying to do for the country.
He said he did not see any possibility of a Sinn Féin/Fine Gael partnership or alliance in Government any time soon.
Asked about a possible snap election, he said the confidence and supply agreement with Fianna Fáil involved three budgets, and then a review.
It will be up to both parties if it is to progress beyond then but Fine Gael does not want an election.
When asked about the testing of political slogans, he said a minority Government has to be ready for an election should it happen, and Fine Gael will be.