The negotiations were launched at a ceremony in the capital of Ecuador, Quito, where the talks will be held.
Delegates stood side by side as the Colombian national anthem was played.
The talks come just months after the government signed a peace agreement with Colombia's largest rebel group, the Farc.
The chief ELN negotiator, Pablo Beltran, urged both sides to rally around the points that united them and leave aside their differences.
The top government representative, Juan Camilo Restrepo, said he expected to draw from the lessons of the negotiations with the Farc to reach a peace accord with the ELN.
They both agreed that this was a once-in-a-generation opportunity for Colombia to achieve peace.
'No more kidnappings'
"Colombia and the world realise that it's very unlikely that this opportunity for peace will come before us again," said Mr Restrepo.
He called on the rebels to officially suspend its kidnapping policy during the negotiations.
The ELN relies on the ransom obtained from kidnappings or "retentions" to finance their activities.
Mr Beltran said peace would not be achieved through more repression.
"We need a political solution. We are willing to take responsibility for the mistakes we have made, but we expect the other side to do the same," he said.
"This generation may not have another opportunity to achieve peace," said Ecuadorean Foreign Minister Guillaume Long.
Ecuador is hosting the first round of negotiations. Brazil, Chile, Cuba, Norway and Venezuela are acting as guarantors.
The ELN, or National Liberation Army, is Colombia's second largest rebel group.
On Monday, it released a soldier it had been holding hostage for two weeks.
The soldier, Freddy Moreno, was handed over to delegates from the International Committee of the Red Cross in Arauca province.
The talks were due to begin at the end of October. But they were delayed as the government refused to sit down for formal negotiations while the rebels still held Odin Sanchez, a former congressman.
Mr Sanchez was released last Thursday, after 10 months in captivity.
The ELN was founded in 1964 with the stated aim of fighting Colombia's unequal distribution of land and riches, inspired by the Cuban revolution of 1959.
The government signed a revised peace agreement with Colombia's largest rebel group, the Farc, in November, after four years of negotiations in the Cuban capital, Havana.
Members of the Farc have been gathering in "transition zones", where they are to demobilise and lay down their weapons under the supervision of United Nations monitors.
The last of the Farc rebels are expected to reach the designated demobilisation areas by Wednesday, government officials said.