It comes amid growing political tension and just days after the country confirmed more than one million coronavirus infections.
Brazil recorded its highest number of daily deaths on 4 June; the seven-day average seems to have plateaued since.
President Jair Bolsonaro's decision to oppose lockdowns and focus on the economy has been hugely divisive.
Two health ministers - both doctors - have left their posts as deaths and infections have surged. The first was sacked by Mr Bolsonaro, the second resigned after disagreeing with the far-right president.
On Sunday, Brazil's health ministry announced that 641 more deaths had been registered in the past 24 hours, taking the total to 50,617. Over the same period it also registered more than 17,000 new infections.
Only the US has fared worse overall, with 2.2 million cases and nearly 120,000 deaths.
Brazil has recently been recording about 1,000 deaths a day, although figures at weekends tend to be lower. Many experts believe the lack of testing nationally - some of them say the level is 20 times less than needed - suggests the overall figures could be considerably higher.
The northern states of Amazonas, Pará and Ceará have seen more than 12,000 deaths in total, but it is São Paulo and Rio de Janeiro that have seen the biggest spikes, now standing at 12,500 and 8,800 fatalities respectively.
The World Health Organization (WHO) has also recorded the biggest one-day increase in global cases, with most new infections in the Americas.
The WHO says that, of the 183,000 new cases reported globally in 24 hours, more than 60% were from North and South America.
Mexico, Peru and Chile have been particularly badly hit, aside from the US, and on Sunday Argentina passed 1,000 deaths.
Latin America and the Caribbean now have more than two million infections.
Police in the capital, Brasilia, kept rival groups apart as they marched outside Congress and the Supreme Court buildings. One anti-Bolsonaro protester there, Nilva Aparecida, told AFP: "[We are here] in defence of democracy and asking for Bolsonaro to leave. This president who usurped the people's power and today is committing atrocities. He does not deserve to govern our country."
Other large gatherings took place in São Paulo and Rio de Janeiro.
Opponents of Mr Bolsonaro are also calling for him to be impeached amid a growing political crisis. He is under investigation for allegedly trying to interfere with the police for political motives, which he denies, while the Supreme Court is carrying out two separate inquiries into his allies.
On Thursday, a former aide and family friend was arrested after being accused of corruption.
Mr Bolsonaro's supporters say Congress and the Supreme Court are trying to curb his powers.