International Trade Secretary Liam Fox said it would be “easier” for Boris Johnson to “just give an explanation” about the row recorded at his home.
The MP said reports of the row between Mr Johnson and his partner should not be a “distraction” from policy discussion in the leadership race.
But fellow leadership rival Jeremy Hunt said “what happens in people’s personal lives is really a matter for them”.
It comes after a neighbour called police and recorded a loud row at the home Mr Johnson shares with his partner Carrie Symonds in London.
Defending his actions, Tom Penn told the Guardian he was worried about his neighbours’ safety, adding: “I hope that anybody would have done the same thing.”
He said he began recording from inside his flat in Camberwell, south London, after he heard “slamming and banging” in the early hours on Friday.
In the recording – heard by the Guardian, but not by the BBC – Ms Symonds reportedly could be heard telling the Tory MP to “get off me” and “get out of my flat”.
Speaking on the BBC’s Andrew Marr Show about the row, Mr Fox – who backs Mr Hunt – said: “It’s always easier to just give an explanation.
“The key thing is then how you get on to the issues. What we can’t have is it being a distraction from explanations about wider policy.”
He said it was “fair” for candidates to be asked questions about their character, but added: “I’m not sure what we’ve seen over the last few days is a fair reflection of that.”
When asked about the incident, Mr Hunt steered clear of calling for Mr Johnson to explain what happened.
He told Sky News: “I think what happens in people’s personal lives is really a matter for them.
“What people care about is who is going to be the wise prime minister who navigates this country out of the biggest constitutional crisis in our lifetimes.”
Also in his interview on the Andrew Marr Show, Mr Fox dismissed suggestions that Mr Johnson was a potential security risk.
Recalling Mr Johnson’s previous role in government, he said: “Do you think Theresa May would make him foreign secretary if there were genuine worries about him being a security risk?”
“I think we have to get away from these distractions and talk about policy issues.”
Meanwhile, speaking to John Pienaar on BBC Radio 5 Live, Chief Secretary to the Treasury Liz Truss said Mr Johnson had a proven record, so “people know what he’s like in office”.
Asked about the row, she said: “There’s no point asking me. I believe it’s a private matter – I don’t think the public are concerned about that.
“Boris served for eight years as mayor of London, did a brilliant job; he’s served as foreign secretary – people know what he’s like in office, and that’s what’s important.”
But shadow communities secretary Andrew Gwynne said Mr Johnson was “completely unsuitable” to be prime minister.
Speaking on Sky News, he said: “In one sense, of course, it is a private matter, but when you’re running for public office, when you are wanting to be the prime minister of the UK, then these matters are in the public interest.
“I’ve long held the view that Boris Johnson is unsuitable to be prime minister of this country.”
On Saturday, Mr Johnson repeatedly avoided questions about the incident as he and Mr Hunt made their pitches to Tory party members on why they should succeed Theresa May as prime minister.
When the event moderator, Iain Dale, accused him of ducking the question, Mr Johnson did not respond directly, instead saying: “People are entitled to ask me what I want to do for the country.”
Mr Dale was heckled by some in the audience when he continued to press the MP, but later defended his persistence.
“There will have been lots of other people in the audience who didn’t boo, and who actually did want to hear the answer to that question,” he told the BBC.
It was the first of 16 events, or hustings, to choose the next Conservative party leader – and prime minister – following Theresa May’s resignation after she failed to get her Brexit deal through Parliament.
She remains in office until her successor is found.
Conservative Party members will vote for their next leader after an initial list of 10 candidates to replace Mrs May was whittled down to Mr Hunt and Mr Johnson following a series of votes by Tory MPs.
Members will receive their ballots between 6 and 8 July, with the new leader expected to be announced in the week beginning 22 July.