Myanmar’s leader Aung San Suu Kyi called off a trip to Indonesia on Monday after protests there over her country’s renewed crackdown on its Rohingya minority, a flare-up of violence that has killed dozens of people and exposed the limits of her authority over the military.

Foreign affairs spokesman Kyaw Zaya said the protests, including another set for Friday in Jakarta that is also expected to touch on the Rohingya crisis, contributed to Ms. Suu Kyi’s decision to postpone her visit, which was planned for that same day.

Adding to the tensions, Indonesian police in recent days arrested several alleged Islamic State sympathizers they say were suspected of planning bomb attacks on several buildings in Jakarta, including the Myanmar Embassy.

The Rohingya, like the vast majority of Indonesians, are Muslim.

Hundreds have fled the northern reaches of the state of Rakhine since the purge began in early October, after a group of militant Rohingyas attacked security outposts in the area, killing nine police and seizing firearms.

Satellite images released this month by U.S.-based Human Rights Watch, and video footage smuggled out by Rohingya activists, show villages in the area around Maungdaw being torched as the stream of refugees grows.

Aid agencies and journalists have been barred from the area, where more than 150,000 people have now been left without food or medical care as the Myanmar armed forces conduct what they describe as “clearance operations.” At least 69 insurgents have been killed, authorities say.

The crisis is the worst since 2012, when Buddhist mobs attacked some of the 1.1 million Rohingya living in Rakhine, who are subject to travel and work restrictions in addition to being denied citizenship in Myanmar, also known as Burma.

The U.S., which recently dropped its last remaining sanctions against Myanmar, says it has raised concerns with the government. The State Department called last week for an independent investigation and “open media access.”

ENLARGE

Rohingya Muslims from Myanmar in a refugee camp in Teknaf, in Bangladesh’s Cox’s Bazar, on Saturday. PHOTO: AGENCE FRANCE-PRESSE/GETTY IMAGES

Presidential spokesman Zaw Htay this month accused Rohingya militants of burning their own villages to darken Myanmar’s international reputation, and denied allegations from Rohingya activists that soldiers have raped Rohingya women. He has said any other country would take action if insurgents had raided outposts and stolen weapons.

Ms. Suu Kyi, who formally serves as state counselor rather than president, has taken a cautious approach. Speaking in Japan this month, she said that blame shouldn’t be cast until all the facts are known.

Some international researchers say Ms. Suu Kyi’s silence is providing a free hand for the Myanmar army to clear and burn villages, with the goal of forcing entire communities over the border to refugee camps in Bangladesh.

A regional official with the U.N.’s refugee agency, John McKissick, called it a campaign of ethnic cleansing while experts at the International State Crime Initiative, a research center based at Queen Mary University of London, accuse Ms. Suu Kyi’s government of mimicking the previous military regime by turning a deaf ear to allegations of human-rights abuses.

The reality, though, is that the former Nobel Peace Prize winner, who came to power last year after a landmark election, still faces constraints.

Under Myanmar’s new constitution, the army retains control of the Defense and Interior ministries and border control, meaning her government must share power with the armed forces.

That leaves her with a dilemma: Whether to hold her tongue about her former captors’ treatment of the Rohingya and face a worsening humanitarian crisis, or speak out and risk damaging the working relationship that she has developed with Tatmadaw, as the armed forces are known.

ENLARGE

An effigy of Nobel Peace Prize laureate Aung San Suu Kyi was later burned during a rally by several Islamic groups against the persecution of Rohingya Muslims in Myanmar after Friday prayers in Dhaka, Bangladesh. PHOTO: A.M. AHAD/ASSOCIATED PRESS

Some analysts point out that she isn’t powerless. Ms. Suu Kyi has a strong international support, having spent years under house arrest, and could likely influence the military if she chooses to do so.

“There is a symbiotic relationship between Ms. Suu Kyi and the military. She needs them to govern effectively, and the Tatmadaw needs her to help legitimize itself to the rest of the world,” said Richard Horsey, a Myanmar analyst and consultant based in Yangon.

Going against the army could prove politically unpopular, too. There is a strong anti-Muslim sentiment in Myanmar, fanned by Buddhist nationalists who argue that Islam is gaining too much ground too quickly. They have become more vocal since the Oct. 9 attack on the police outposts.

Meanwhile, researchers say the situation in Rakhine is worsening as Bangladesh turns back people attempting to cross the border.

Penny Green, director of the International State Crime Initiative, says the current round of repression echoes previous crackdowns in 1977-78 and 1991-92, when hundreds of thousands of Rohingya fled Myanmar.

—Myo Myo in Yangon contributed to this article.

Corrections & Amplifications:
Penny Green is director of the International State Crime Initiative. An earlier version of this article incorrectly stated the name of the Initiative.

Write to James Hookway at james.hookway@wsj.com