He said audits being conducted of several foreign firms operating in the country have found possible criminal activity. He declined to name specific companies or details, but said he would reveal more in March when the audits would be complete.

Major Western companies operating in Congo include Glencore PLC, Randgold Resourcesand Ivanhoe Mines Ltd. Executives from these companies either declined to comment or weren’t immediately available to do so.

Mr. Yuma’s comments—made Tuesday at a mining conference in Cape Town, South Africa—ratchet up pressure on the miners that have bet big on Congo’s mineral wealth. They have recently been on the defensive amid a run-up in prices for some of the country’s most valuable commodities, which include copper, cobalt, diamonds and gold.

“They’ve been stealing money from Gécamines for many years,” Mr. Yuma said.

He was flanked as he spoke by members of both houses of Congo’s parliament, some of whom wrote the country’s controversial new mining code. That new legislation—yet to be signed by President Joseph Kabila —will classify metals such as cobalt and tantalum, a rare blue-gray metal used in smartphones, as “strategic substances,” allowing government to tax them at higher rates.

The proposed changes and the comments by Mr. Yuma come amid a boom in the price of cobalt, a metal used in cellphones and a key material for making lithium ion batteries for electric vehicles.

Cobalt prices have doubled in the last 12 months, as technology and automobile companies rush to secure supplies. Congo holds some of the world’s richest cobalt reserves.

Higher cobalt prices “give us full reason” to renegotiate joint ventures, Mr. Yuma said.

“A bigger piece of the cake, that’s the ultimate goal,” he said, adding that eventually, he would like to revert to 100% ownership of all mines in Congo.

It wasn’t immediately clear whether Mr. Yuma’s comments signaled a formal shift in government policy or a gambit to renegotiate contracts on more preferential terms. However, people familiar with the matter say miners who operate in the country are rattled.

Congo’s Minister of Mines Martin Kabwelulu Labilo didn’t return requests for comment.

Mr. Yuma said recent audits of mining companies operating in Congo—undertaken by global accounting firm Mazars on behalf of Gécamines—had revealed corrupt and potentially criminal behavior and warned that companies could face prosecution by Congolese courts.

“As long as African countries offered mining codes and fiscal regimes that were reasonably investor-friendly, companies were prepared to take the risks of limited or nonexistent infrastructures and skills bases, as well as political volatility,” Mark Bristow, chief executive of Randgold, which owns gold mines in Congo, said in a speech in Cape Town on Tuesday. “Start making unreasonable demands, however, and investors will vote with their feet.”

The audits are due to conclude in March, at which point Gécamines will invite the miners to sit down and renegotiate joint ventures, based on individual findings, Mr. Yuma said.