House Democratic Caucus chairman on Conyers: 'The right thing will be done'

"I believe at the end of the day, the right thing will be done," Crowley said, adding that the Michigan congressman has gone back to his home district and is taking counsel from family at the moment. "I think that accountability will be had."

Crowley and California Democratic Rep. Linda Sanchez both refused to call for his resignation though and continued to punt handling of the issue to the Ethics Committee, saying they have asked for the process to be expedited.

"Calling for the resignation of someone does not actually create the resignation," Crowley told CNN. "So, the reality is we have a process in place and we are calling for an expedited process with the Ethics committee."

"I don't know all the facts, I don't know the specific allegations," Rep. Linda Sanchez said. "It appears that there is more than one complainant which does heighten my sense of there may be something there. But again, I can't sit and judge a member and call for their resignation unless there has -- unless I have been party to hearing all the evidence and the defense of all the evidence."

The House Ethics Committee announced last week it had opened an investigation into allegations against Conyers after BuzzFeed reported that he settled a wrongful dismissal complaint in 2015 after allegedly sexually harassing a staffer. A separate woman told CNN Tuesday that Conyers made three sexual advances toward her when she worked for him in his district office in Detroit from 1997 to 2005.

Conyers has denied wrongdoing in both cases and has repeatedly said he will not resign from Congress. On Sunday, he stepped down as the top Democrat on the House Judiciary Committee.