Fine Gael TD John Deasy has accused his party headquarters of being "negligent" regarding problems within the party's Waterford constituency organisation.
He also said that he has been "giving everything" to his work as a special envoy to the United States, working for emigrants' rights, which has prevented him from "being on the ground" in the constituency in recent years.
The vote was proposed by a brother of Senator Paudie Coffey.
The meeting was attended by Fine Gael councillors, unsuccessful local election candidates and ordinary party members.
It heard "severe and sustained criticism" of the sitting TD, according to one long-standing member.
"Speaker after speaker came out strongly against John Deasy and his work rate as sitting Fine Gael TD," this member said.
However, in response, Mr Deasy has hit out at the local party organisation and also headquarters in Dublin for not doing anything, according to him, about internal divisions in recent years.
"If Fine Gael wants to be competitive in Waterford in the next general election, the entire organisation needs to be stood down immediately and reviewed," he told RTÉ News.
"For years, warnings have been given with regard to the behaviour of individuals within the organisation and they haven't been acted upon."
He said party headquarters has "ignored" documentary evidence regarding divisions within the local party.
"Everyone in the highest levels of Fine Gael has known there's been a problem in Waterford for years and they've ignored it. Fine Gael headquarters have been negligent with regard to dealing with this issue."
He said the situation has prevailed for "20 years" and goes back to when his late father and former minister for agriculture Austin Deasy was a TD.
Asked if has been neglecting his role as a constituency TD, he said: "I've basically given everything to my role in the US and, in my opinion, that has gone well. That has prevented me from being on the ground in the last few years and being active in the Dáil."
Asked if he still wants to stand at the next general election for Fine Gael, he said: "I'm not going there yet."
In a statement, a spokesman for Fine Gael said Mr Deasy had been doing "important work on a special visa scheme for Irish citizens" since his appointment as a special envoy to the US.
Party general-secretary Tom Curran will "soon be consulting with members and elected representatives in Waterford on finalising the party's general election ticket for the constituency," the spokesman said.
The Fine Gael organisation in Waterford has long been divided into two general camps: those supporting Mr Deasy, whose home base is Dungarvan, and those supporting Mr Coffey, who is based in Portlaw.
Mr Deasy has been a TD since 2002 and re-elected at every national poll since then, while Mr Coffey won a seat alongside him in 2011, but lost it at the general election of 2016, despite holding a minister of state portfolio for some of that Dáil term.
Mr Coffey was the sole candidate chosen by the local party at the selection convention held in November of 2017, when a general election seemed imminent.
Mr Deasy, who was recovering from hip surgery at the time and also spending much time in the US on envoy business, did not put his name forward.
There has been speculation since then about whether or not the party would add him to the Waterford ticket.
The "no confidence" motion passed unanimously by members last night was in his performance as a TD and also his prospects as a candidate at the next election.
Former senator questions 'engagement' by Deasy
Former Fine Gael councillor and senator Maurice Cummins questioned how much Mr Deasy knows about what's been going on in the local organisation in recent years.
"The question of him being in America is a red herring, as far as I'm concerned," he said, adding that Mr Deasy did not "engage" with party chairmen in the constituency even before his appointment as a special envoy.
Mr Cummins also disputed Mr Deasy's account of why supporters have stopped going to meetings. "I'd say the reason that they're not going to meetings is because they're embarrassed of his performance as a TD."
Mr Cummins, a former Seanad leader, also called on party chiefs to step in and take action in the Waterford party organisation and said the TD should look at his own future.
However, Deasy supporter and recent local election candidate Fiona Dowd said there had been problems within the party in Waterford for years.
"This has been going on a long time," she said, adding that the meeting on Monday night, which she attended but did not speak at, had made her rethink her future in Fine Gael.
"I for one will absolutely never go to another Waterford constituency meeting, with that mentality that's there," she said of the no confidence vote and some of the anti-Deasy sentiment expressed on Monday night.
She also said she will "absolutely not" be canvassing for Fine Gael at the next general election.