Comedians have always wanted to be rock stars. Now there’s a new crop of albums from entertainers across the comic spectrum.

Elton John and Brandi Carlile’s new album Who Believes in Angels? debuts at No. 1 on two Billboard charts, becoming his first ...