Song writer Jimmy Van Heusen was at Crosby’s house one evening for dinner, and to discuss a song for the movie Going My Way. During the meal one of the children began complaining about how he didn’t want to go to school the next day. The singer turned to his son and said to him, “If you don’t go to school, you might grow up to be a mule. Do you wanna do that?”
Van Heusen thought this clever rebuke would make a good song for the movie. He pictured Crosby, who played a priest, talking to a group of children acting much the same way as his own child had acted that night. Van Heusen took the idea to his partner lyricist Johnny Burke, who approved. They wrote the song.
The first recording of "Swinging on a Star" took place in Los Angeles on February 7, 1944, released as Decca Records no. 18597. The Williams Brothers quartet, including Andy Williams, sang backup vocals to Crosby. It was featured in the Little Lulu cartoon, Bout with a Trout.