Twins for Costello and Krall

Los Angeles (E! Online) - Hopefully, between the two of them,

Elvis Costello and

Diana Krall know plenty of lullabies.

The musically inclined couple welcomed their first children together Wednesday in New York, twin sons Dexter Henry Lorcan and Frank Harlan James.

"We are ecstatic," Costello, 52, and Krall, 42, said in a statement, adding that "mother and sons are doing splendidly."

The happy occasion also happened to fall on their third wedding anniversary. The Grammy-nominated artists tied the knot at

Elton John's castle in London, eight months after meeting each other at the 2003

Grammy Awards. It was the first trip down the aisle for Krall and the third for her bespectacled mate.

Costello also has a son, Matthew, with his first wife, Mary Burgoyne.

Krall apparently wasn't surprised to learn that she was carrying two babies at once. She told People in September that she was genetically predisposed to doubling up.

"I have twins on my mother's side, as well," the jazz songstress said. Her latest album, From This Moment On, dropped in September and scored a Grammy nod Thursday for Best Jazz Vocal Album.

Costello seemingly hasn't had a moment's rest this year, either. The Rock and Roll Hall of Famer released My Flame Burns Blue, a compilation of recordings from the 2004 North Sea Jazz Festival, in February, followed by The River in Reverse, a collaboration with New Orleans R&B artist

Allen Toussaint, in June.

Both offerings are up for 

Grammys this year—The River was nominated for Best Pop Vocal Album and Costello got a nod for Best Instrumental Arrangement Accompanying Vocalist for My Flame.

The British singer-songwriter was also commissioned last year to pen a chamber opera for the Danish Royal Opera, but now some of the new songs will instead be combined with tunes from 1993's The Juliet Letters to be performed in the opera company's studio theater.