A number of people were also feared trapped in and around the unit, he said.
Some videos of the incident surfaced on social media showing the fire with intermittent explosions taking place at the site and people running to save themselves.
Three persons were killed and 40 others injured in the blaze at the crackers unit, Harda Collector Rishi Garg told PTI.
A number of people were feared trapped in and around the unit, he said.
Madhya Pradesh Chief Minister Mohan Yadav spoke to the officials concerned and sought details of the incident, an official said.
The CM also directed minister Uday Pratap Singh, Additional Chief Secretary Ajit Kesari and Director General Home Guard Arvind Kumar to rush to Harda by a helicopter.
Burn units at hospitals in Indore, Bhopal and the AIIMS in the state capital were directed to make necessary arrangements for any emergency, the official said.
Fire brigades were also rushed from Indore and Bhopal to tackle the blaze.
The chief minister has also called a meeting in connection with the incident, the official added.
