The move comes amid heightened tensions across the US over allegations of police brutality, and particularly after the recent deaths of black men at the hands of officers.
The President's executive order will ban police use of equipment such as explosive-resistant vehicles with tracked wheels like those seen on military tanks, a fact sheet released by the White House said.
Police departments' requests for other military-style gear, such as as MRAPs (mine-resistant ambush protected) vehicles and riot shields, will require added justification prior to approval under the President's order.
The federal government is also exploring ways to recall prohibited equipment already distributed to departments across the country.
Mr Obama unveiled the new restrictions in a speech in Camden, New Jersey, on Monday.
"We've seen how militarised gear can sometimes give people the feeling like there's an occupying force as opposed to a force that's a part of the community that's protecting them and serving them," he said.
"It can alienate and intimidate local residents and send the wrong message.
"So we're going to prohibit some equipment made for the battlefield that is not appropriate for local police departments."
Clashes between police and protesters in the wake of the fatal shooting of Michael Brown Jr in Ferguson, Missouri, and the death of Freddie Gray in Baltimore have fuelled calls for changes to police tactics.
Similar incidents in New York, Cleveland and South Carolina have also highlighted divisions between some communities and police.
The issue of militarised police took centre stage last year during the riots in Ferguson. Critics questioned police use of full body armour and armoured trucks to dispel demonstrators.
At the time, Mr Obama said: "There is a big difference between our military and our local law enforcement and we don't want those lines blurred."
The President ordered a review of the programmes that provide the equipment.
