Xavier Bettel married Gauthier Destenay on Friday in a private ceremony at the capital's town hall.
The couple are among the first to benefit after the country changed its laws to allow gay marriages last year.
The union comes after Iceland's then-PM Johanna Sigurdardottir married her same-sex partner in 2010.
Mr Bettel formed a government in December 2013 to become Luxembourg's first openly gay prime minister.
He announced that he and his civil partner Mr Destenay, an architect, would be getting married just months after lawmakers approved the law change in June 2014.
'Honest in politics'
"I could have hidden it or repressed it and been unhappy my whole life," Mr Bettel reportedly told a Belgian broadcaster the night before the nuptials.
"But I told myself that if you want to be a politician and be honest in politics, you have to be honest with yourself and accept that you are who you are."
Former Belgian Prime Minister Elio di Rupo was the EU's first openly gay national leader.
The Luxembourg newly-weds are not expected to go away on a honeymoon, as the PM has to attend an economic forum in Kazakhstan, according to AFP news agency.
A friend of the couple told the French news agency Mr Bettel was keen to keep his private life out of the public spotlight.
Pierre Leyers, a journalist in Luxembourg, told the BBC that the country's media and public were respecting the PM's wishes.
"It is something quite ingrained in society - we are quite reserved and do not like things to be too public," he said.
Among the guests spotted in the crowd was Belgian Prime Minister Charles Michel,according to Luxemburger Wort newspaper.
Correspondents say Luxembourg was previously a conservative Catholic country but attitudes have changed in recent years, in part due to the high number of immigrants and foreign workers.
Mr Bettel defeated conservative PM Jean-Claude Juncker in 2013 after Mr Juncker's almost two decades in power.
