Music therapy has been shown in recent years to provide multiple health benefits related to emotional wellness. In turn, maintaining a healthy emotional state has proven to be effective for patients undergoing treatment, such as Parkinson's patients or patients suffering from stress and anxiety. We propose fine-tuning MusicGen, a music-generating transformer model, to create short musical clips that assist patients in transitioning from negative to desired emotional states. Using low-rank decomposition fine-tuning on the MTG-Jamendo Dataset with emotion tags, we generate 30-second clips that adhere to the iso principle, guiding patients through intermediate states in the valence-arousal circumplex. The generated music is evaluated using a music emotion recognition model to ensure alignment with intended emotions. By concatenating these clips, we produce a 15-minute "music medicine" resembling a music therapy session. Our approach is the first model to leverage Language Models to generate music medicine. Ultimately, the output is intended to be used as a temporary relief between music therapy sessions with a licensed therapist.
The following samples are generated with our model, following the iso principle the generated audio traverses different emotions starting from the initial state and ending to the desired state.
Initial Prompt: Sad Ambient Piano Initial State: Sad Desired State: Happy
Initial Prompt: Stressed pizzicato strings and high-pitch clarinet sounds Initial State: Stressed Desired State: Relaxed
Initial Prompt: Upset dramatic rock guitars, drums and bass Initial State: Upset Desired State: Calm
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-sad-to-1?si=4b80b034343142df80f67bf4e33ff00b&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-stressed-1?si=b14f85c9ec15479fbcb2284b69f348d1&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-upset-to-1?si=a74ea1ba8bee4d0db4d943e54e658146&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing
Initial Prompt: Sad Ambient Piano Initial State: Sad Desired State: Happy
Initial Prompt: Stressed pizzicato strings and high-pitch clarinet sounds Initial State: Stressed Desired State: Relaxed
Initial Prompt: Upset dramatic rock guitars, drums and bass Initial State: Upset Desired State: Calm
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-sad-to?in=emmanouil-karystinaios/sets/generative-music-medicine-musicgen-large&si=d6b007f7944549168715dbd333460f04&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-stressed?in=emmanouil-karystinaios/sets/generative-music-medicine-musicgen-large&si=fbf13e0a8cc349ffaef37f094f2b3c77&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing
https://soundcloud.com/emmanouil-karystinaios/music-medicine-from-upset-to?in=emmanouil-karystinaios/sets/generative-music-medicine-musicgen-large&si=d674f47b81184c21b98cbfaf3ba389a2&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing