Music 14/102 - Music, Information, Neuroscience (Winter 2017)

Professor Michael Casey

Department of Music, Department of Computer Science


Hallgarten Hall Seminar Room

Course Description

This course explores how the brain represents, learns, and reacts to music, covering musical ability, preference, reward, emotion, and creativity. Drawing from new results in neuroscience, music cognition, and music informatics, topics include: neural codes for pitch, rhythm, timbre, structure, and style; measuring musical performance, listening, and imagination with EEG and fMRI; brain-computer interfaces for music; and music composition and performance using biofeedback.

Recommended background

You should have taken at least two of the following courses, or their equivalents: a) MUS1 (knowledge of basic music theory); b) COSC1 (Python) or COSC10 (Java) or equivalent; and c) PSYC 1 (Introduction to Psychology), PSYCH6 (principles of cognition and neuroscience), or PSYCH10 (Experimental Design, Methodology, and Data Analysis Procedures). E.g. any two of MUS, COSC, and PSYC/NEURO at the intro level.

Some familiarity with music theory and programming will be necessary to comprehend and engage with materials in this course. This should include knowledge of basic Western tonal music theory―such as notation of scales, rhythm, cadences, chords, keys―as well as experience with script programming in a modern language such as Python or Java. Music graduate students will need to be familiar with a computer music programming environment such as Python, Java/Processing, Max/MSP, Supecollider, or PureData.

Expected Workload

3 hours in class, occassional individual meetings, and 6-10 hours per week reading and homework. There are two parts to the homework, a lab portion (involving some computer scripting) and an analysis component (requiring use of computational tools to apply music cognition and neuroscience concepts). Graduate students must complete an additional assignment each week tailored to individual field of study such as: music composition, empirical research in cognition / neuroscience, or a detailed written research survey.

Class Timetable / Room Allocation


All sessions will take place in Hallgarten Hall Room 101 (Main Seminar Room).

Reading

Two required readings will be assigned each week, seminar discussion leaders to be allocated each week. You will be assessed partly on your ability to understand and respond to the concepts in the readings. See the course bibliography.

Assessment

CriteriaAllocation
Weekly Assignments (Labs, Writing, Responses)15%
Participation/Preparedness, In-Class Contributions15%
Seminar Presentation20%
Final Project Presentations, Writeup, and Implementation50%
Graduate Students: Additional Weekly Assignment Plus Publishable-Quality Final ProjectDetermines: LP, P, HP
(Extra Credit for UG)

Office Hours

Individual meetings by arrangement, Thursdays 3:00-5:00, Hallgarten Hall room 103. Note: to supplement your understanding of the course materials and to develop a successful project individual meetings with the instructor are highly recommended.

Weekly Schedule

SYLLABUS MUS14/102 Winter 2015, 3A
Prof. Michael Casey
Dept. Music and Dept. Computer Science, Dartmouth College
   
MODULE 1. Music Cognition and Neuroscience MUSIC (Listening and Analysis)
INFORMATION (Computation)
NEUROSCIENCE
Required Reading / Listening Supplemental Reading / Listening

Week 1
January 5th

Fundamentals
[MUS] Introduction: Time-scales of music and memory, how well do we hear and listen ?
[INFO] The Ear as Time-Frequency analyzer, Ohm's acoustical law
[NEURO] Music Neuroanatomy I: The ear, cochlea, cochlear nucleus/nerve, frequency, loudness,auditory pathway, tonotopy, superior olive, thalamus
(Byrne, 1997-A) (Byrne, 1997-B)
(Rasch, 1984)



Consonance
[MUS] (Dis)pleasing concurrence: taming the consonance hydra. Where does the circle of fifths come from?
[INFO] Consonance/dissonance, tuning systems, Helmholtz, sensory and musical dissonance
[NEURO] Sensory dissonance, critical bands, masking, spatialization, binaural beats
(Van De Geer, Levelt & Plomp, 1962) (Plomp & Levelt, 1965)
(McKinney, Tramo & Delgutte, 2001)

Week 2
January 12th

Pitch
[MUS] Pitch space: pitch-class and pitch-interval transformations, geometries of tonality
[INFO] Absolute / relative pitch, chroma, height, scale, contour, sequence
[NEURO] Music Neuroanatomy II: primary auditory cortex (core): temporal gyri/sulci, planum temporale (PT), hechel's gyrus (HG), Brodmann areas 41 & 42, 22
(Rasch and Plomp, 1999) (Janata et al., 2002)
(Lee, Janata, Front, Hanke, & Granger, 2011)



Rhythm
[MUS] Grouping, hierarchy, rhythm, groove, pitch and timing deviations (expression)
[INFO] Tonal hierarchy: chords, key, melody; rhythmic hierarchy: tactus, beat, meter, rhythm, groove
[NEURO] Music Neuroanatomy III: secondary auditory cortex: belt and parabelt, pre-frontal cortex
(Krumhansl, 2000) (Tomic & Janata, 2008)
(Janata & Grafton)

Week 3
January 18th

Timbre
[MUS] The Rest is (Timbre and) Noise: timbre space, timbre and pitch, timbre and genre
[INFO] Formants; dynamics; spectral measures: flux, centroid, deviation; genre and timbre
[NEURO] Bilateral auditory cortex asymmetry, neural correlates of timbre and environmental sound
(Grey, 1977) (Halpern, Zatorre, Bouffard & Johnson, 2004)
(Krumhansl, 1989)



Auditory Scene Analysis
[MUS] Spatial hearing, polyphony, auditory scenes
[INFO] Auditory scenes, mixtures, latent components, time-frequency atoms
[NEURO] Auditory object formation, auditory streams, spectro-temporal, receptive fields
(Bregman and Ahad, 1990a) (Bregman and Ahad, 1990b)
(Pressnitzer,Suied, and Shamma, 2011)
MODULE 2. Musical Affect, Expectation, and Reward
(MID TERM PRESENTATIONS)

   

Week 4
January 26th

Emotion
[MUS] Emotion and meaning in music: goosebumps, musical preference, long term music affect, social factors

[NEURO] Dopamine, neural correlates of affect Reward systems: Frontal lobe, Limbic system, Amygdila,
   



Expectation
[MUS] Sweet Anticipation: implication-realization, music and affect
[INFO] Information theory, music, probability, and expectation. Information dynamics of a performance.

   

Week 5
February 2nd

Neurofeedback
[MUS] Reflexive Composition: bio-sensing and neuro-feedback in music, multi-modality and synaesthesia
[INFO] Analysis of fMRI, EEG, neuro-feedback, regression and classification of biological signals, synaesthesia
[NEURO] Biological signals, EEG, heart rate, feedback, galvanometric skin response, facial / corregator musclar response, other affect sensors, cross-modal affect
   



Neuromusic
[MUS] Music: Lucier “Music for Solo Performer”, Rosenbloom “On Being Invisible”, Miranda


   
MODULE 3. Music,Brain, and Health
(PRELIMINARY PROJECT PRESENTATIONS)

   

Week 6
February 9th

Music and Health 1: Memory
[MUS] GUEST SPEAKER 1:(TBA)

[NEURO] Alzheimer's Disease, Music: Non-Expert Higher-Order Composition as Clinical Intervention
(Barrett, Grimm, Robins, Wildschut, Sedikides, & Janata, 2010) (Janata, 2009)
(Janata, 2012)
 


   

Week 7
February 16th

Music and Health 2
[MUS] GUEST SPEAKER 2: Yune Lee (U. Penn.):

[NEURO] Dyslexia, Autism, Speech, and Music. fMRI studies in the Neuroscience of Sound, Speech, Music, Memory, and Affect
   
 


   

Week 8
February 23rd

Music and Health 3
[MUS] GUEST SPEAKER 3: (TBA)

[NEURO] Music, Health, and Medicine: cortical stroke, dementia, major depression, coma, parkinson's Disease, Williams' Syndrome
   
 


   

Week 9
March 2nd

Music and Health 4
[MUS] GUEST SPEAKER 4: (TBA)

[NEURO] Musical Expertise: Musical Performance and the Brain
   
 


   
MODULE 4. Final Projects
FINAL PROJECT PRESENTATIONS

   

Week 10
March 9th

Final Project
[MUS] FINAL PROJECT PRESENTATIONS
[INFO] FINAL PROJECT PRESENTATIONS
[NEURO] FINAL PROJECT PRESENTATIONS
   

Bibliography

Fundamentals

(Byrne, 1997-A) Byrne, J. (Ed.), Neuroscience Online, An Electronic Textbook for the Neurosciences. Chapter 12 (Auditory System I)
(Byrne, 1997-B) Byrne, J. (Ed.), Neuroscience Online, An Electronic Textbook for the Neurosciences. Chapters 13 (Auditory System II)
(Rasch, 1984) Rasch, R. (1984). Theory of Helmholtz-beat frequencies. Music Perception, Vol. 1, pp. 308-322.

Consonance

(Van De Geer, Levelt & Plomp, 1962) Van de Geer, J.P., Levelt, W.J.M. & Plomp, R. (1962). The connotation of musical consonance. Acta Psychologica, Vol. 20, pp. 308-319.
(Plomp & Levelt, 1965) Plomp, R. & Levelt, W.J.M. (1965). Tonal consonance and critical bandwidth. Journal of the Acoustical Society of America, Vol. 38, pp. 548-560.
(McKinney, Tramo & Delgutte, 2001) McKinney MF, Tramo MJ, Delgutte B. (2001) Neural correlates of the dissonance of musical intervals in the inferior colliculus. In Physiological and Psychophysical Bases of Auditory Function, DJ Breebaart, AJM Houtsma, A Kohlrausch, VF Prijs, and R Schoonhoven (eds). Maastricht: Shaker: pp. 83-89.
(Huron, 2009) Huron, D. (2009). Notes on Theories of Consonance and Dissonance, Music 829B, Ohio State University.
(Tenney, 1988) Tenney, J. (1988). A History of "Consonance" and "Dissonance." White Plains, NY: Excelsior, 1988; New York: Gordon and Breach.

Pitch

(Rasch and Plomp, 1999) Rasch, R., and R. Plomp. The Perception of Musical Tones. in Deutsch, D. (Ed.) (1998) The Psychology of Music. 2nd ed. San Diego, CA: Academic Press.
(Janata et al., 2002) Janata, P. Birk, J. L., Van Horn, J. D., Leman, M., Tillmann, B., Bharucha, J. J. (2002) The Cortical Topography of Tonal Structures Underlying Western Music, Science, No. 13, Vol. 298 no. 5601 pp. 2167-2170
(Lee, Janata, Front, Hanke, & Granger, 2011) Lee, Y. S., Janata, P., Frost, C., Hanke, M., & Granger, R. (2011). Investigation of melodic contour processing in the brain using multivariate pattern-based fMRI. NeuroImage, 57(1)293-300.
(De Cheveigne, 2005) De Cheveigne, A., Pitch perception models, in Plack, C. and Oxenham. A. (eds) (2004)Pitch. New York: Springer Verlag.
(Cedolin & Delgutte, 2005) Cedolin L. and Delgutte B. (2005) Representations of the pitch of complex tones in the auditory nerve. In: Auditory signal processing: Physiology, Psychoacoustics, and Models, Pressnitzer D, de Cheveigne A, McAdams S, Collet L (eds). Springer: pp. 107-116.

Rhythm

(Krumhansl, 2000) Krumhansl, C. (2000) Rhythm and Pitch in Music, Psychological Bulletin, Vol. 126, No. 1, pp. 159-179
(Tomic & Janata, 2008) Tomic, S. T., & Janata, P. (2008). Beyond the beat: modeling metric structure in music and performance. Journal of the Acoustical Society of America. 124(6): 4024–4041.
(Janata & Grafton) Janata, P., & Grafton, S. T. (2003). Swinging in the brain: shared neural substrates for behaviors related to sequencing and music. Nature Neuroscience, 6(7), 682-687.
(Janata & Tomic) Janata, P., Tomic, S. T., & Haberman, J. (2012). Sensorimotor coupling in music and the psychology of the groove. Journal of Experimental Psychology: General, 141(1): 54–75.

Timbre

(Grey, 1977) Multidimensional Perceptual Scaling of Musical Timbres, Journal of the Acoustical Society of America, Vol. 61.
(Halpern, Zatorre, Bouffard & Johnson, 2004) Halpern, A. R., Zatorre, R. J., Bouffard, M., Johnson, J. A. (2004) Behavioral and Neural Correlates of Perceived and Imagined Musical Timbre, Neuropsychologia, Vol. 42, pp. 1281-129
(Krumhansl, 1989) Krumhansl, C., Why is Musical Timbre so Hard to Understand? In S. Nielsen and O. Olsson (Eds.) (1989) Structure and Perception of Electroacoustic Sound and Music Amsterdam, Elsevier: Etcetera Media 8H6.
(Samson, Zatorre, &Ramson, 1997) Samson, S., Zatorre R. J., and Ramsay J. O. (1997) Multidimensional Scaling of Synthetic Musical Timbre: Perception of Spectral and Temporal Characteristics, Canadian Journal of Experimental Psychology, Vol. 51(4), pp. 307-15.
(Casey, Thompson, Kang, Raizada, Wheatley, 2012) Casey, M., Thompson, J., Kang, O., Wheatley, T. (2012) Population codes representing musical timbre for high-level fMRI categorization of music genres. In Advances in Machine Learning and the Interpretation of Neuroimaging, Springer: Lecture Notes in Computer Science, Vol. 7263, pp. 34–41

Auditory Scene Analysis

(Bregman and Ahad, 1990a) Bregman, A. and Ahad, P. (1990) Auditory Scene Analysis: The Perceptual Organization of Sound, (Demonstrations Booklet), MIT Press.
(Bregman and Ahad, 1990b) Bregman, A. and Ahad, P. (1990) Auditory Scene Analysis, (Demonstrations: AUDIO EXAMPLES), MIT Press.
(Pressnitzer,Suied, and Shamma, 2011) Pressnitzer, D., Suied, C., and Shamma, S. Auditory Scene Analysis: the Sweet Music of Ambiguity, Frontiers in Human Neuroscience, Vol. 5, 158, pp. 1-11.
(Klapuri and Virtanen, 2008) Klapuri, A. and Virtanen, T. (2008) Progress Towards Automatic Music Transcription, in D. Havelock, S. Kuwano, and M. Vorländer (Eds) Handbook of Signal Processing in Acoustics, Springer, pp. 277-303
(Klapuri, 2008) Klapuri, A. (2008) Multipitch Analysis of Polyphonic Music and Speech Signals Using an Auditory Model, IEEE Transactions on Audio, Speech, and Language Processing Vol. 16 , No. 2, pp. 255-266.
(Smaragdis, Raj, and Shashanka, 2006) Smaragdis, P, Raj, B., and Shashanka, M. (2006) A Probabilistic Latent Variable Model for Acoustic Modeling, Proceedings of Neural Information Processing Systems
(Abdallah and Plumbley, 2006) Abdallah, S. and Plumbley, M. (2006) Unsupervised Analysis of Polyphonic Music by Sparse Coding, IEEE Transactions on Neural Networks Vol. 17, No. 1, pp. 179-96.
(Casey and Westner, 2000) Casey, M. and Westner, A. (2000) Separation of Mixed Audio Sources by Independent Subspace Analysis, Proc. Int. Comput. Music Conf. (ICMC) pp. 154-161

Music and Health 1: Memory

(Barrett, Grimm, Robins, Wildschut, Sedikides, & Janata, 2010) Barrett, F. S., Grimm, K. J., Robins, R. W., Wildschut, T., Sedikides, C., & Janata, P. (2010). Music-evoked nostalgia: Affect, memory, and personality. Emotion. 10(3): 390–403.
(Janata, 2009) Janata, P. (2009). The neural architecture of music-evoked autobiographical memories. Cerebral Cortex, 19, 2579-2594.
(Janata, 2012) Janata, P. (2012). Effects of Widespread and Frequent Personalized Music Programming on Agitation and Depression in Assisted Living Facility Residents With Alzheimer- Type Dementia. Music and Medicine, 4(1), 8–15.

Other On-line Reading Resources (Some On-Campus or via Dartmouth VPN Only)

Huron, D. (1999), Music 829B: Consonance and Dissonance, Course Notes from Ohio Sate University.
Haines, Duane E. (2013) Fundamental Neuroscience for Basic and Clinical Applications, Fourth Edition
Schnupp, J., Nelken, I., and King, A. (Eds.) (2011) Auditory neuroscience [electronic resource] : making sense of sound. MIT Press.
Altenmüller, E., Schmidt, S., and Zimmermann, E. (Eds.) (2013) Evolution of Emotional Communication: From Sounds in Nonhuman Mammals to Speech and Music in Man, Oxford Scholarship Online.
Nalbantian, S., Matthews, P. M., and McClelland, J. L. (Eds.) (2011) The Memory Process, MIT Press.
Siu-Lan Tan, Annabel J. Cohen, Scott D. Lipscomb, and Roger A. Kendall (Eds.) (2013) The Psychology of Music in Multimedia, Oxford Scholarship Online
Timour Klouche and Thomas Noll (Eds.) (2007) Mathematics and Computation in Music
David Aldridge and Jörg Fachner (Eds.) (2006) Music and altered states [electronic resource] : consciousness, transcendence, therapy and addiction. J. Kingsley PUblishers.

Software and Data Resources

Required (We will learn how to install and use these software tools in class)

Homework will involve modeling aspects of musical listening, so there will be some guided script-level programming. We'll be using the free anaconda framework (Python 2.7 version): Python 2.7 - Anaconda Scientific Distribution We'll use the open source Bregman Music and Audio Analysis Toolkit to learn about and use musical models: Bregman Music and Audio Analysis Toolkit You may also want to use the following excellent resources:

Optional