Speech Recognition and Understanding (11-751/11-551)

Course title: Speech Recognition and Understanding
Course number: 11-751/11-551
Department: Language Technologies Institute (LTI)
Units: 12
Semester: Fall 2000

Time: Monday and Wednesday from 11:30 am to 12:50 pm
First day of class: August 28
Place: Old Student Center 206
Final exam: Fri, December 15, 8:30am - 11:30am DH1217(new!!!)
Grading: Will be based on homeworks (30%), project (30%), and final(40%).

Instructors: Alex Waibel
E-mail: ahw@cs.cmu.edu
Office Hours: By Appointment
Teaching Assistant : Yue Pan
Office: NSH 2602j
Phone: 8-5477
E-mail: ypan@cs.cmu.edu
TA office hours: Friday 3-4pm

Course home page: http://www.is.cs.cmu.edu/11-751

Course Description

Prerequisites: Permission From Instructor (Undergraduates)

sound mathematical background
knowledge of basic statistics
good computing skills

No prior experience with speech recognition is necessary. This course is primarily for graduate students in LTI, CS, Robotics, ECE, HCI, Psychology, or Computational Linguistics. Others by prior permission of instructor.

Note that 11-751 has been approved for 1 CS coreunit for CSD students.

Course description:
The technology to allow humans to communicate by speech with machines or by which machines can understand when humans communicate with each other is rapidly maturing. This course provides an introduction to the theoretical tools as well as the experimental practice that has made the field what it is today. We will cover theoretical foundations, essential algorithms, major approaches, experimental strategies and current state-of-the-art systems and will introduce the participants to ongoing work in representation, algorithms and interface design. This course is suitable for graduate students with some background in computer science and electrical engineering, as well as for advanced undergraduates.

Text:

Waibel and Lee, Readings in Speech Recognition, Morgan Kaufman Publishers, San Mateo, CA, 1990.
Rabiner and Juang, Fundamentals of Speech Recognition, Prentice Hall Signal Processing Series, Englewood Cliffs, NJ, 1993.
Jelinek, Statistical Methods for Speech Recognition, MIT Press, Cambridge, MA, 1997.

Method of evaluation:
Grading will be based on regular homework/lab assignments, a course project and a final exam.

Topics to be covered:

Speech

Speech production and perception
Digital signal processing for speech signals

Recognition Approaches

Template-based recognition
Hidden Markov modeling
Artificial neural networks

Search Techniques
Language Processing

Language modeling
Spoken language understanding

Speech system design

Robust recognition
Speech interface design
Multimodal interfaces
Applications, Systems

Lecture Schedule / Syllabus (tentative)

Date	Lecturer	Topic
Mon Aug 28	Waibel	Course Overview
Wed Aug 30	Waibel	Introduction to the Speech Recognition Problem
Mon Sep 4	No Class	Labor Day
Wed Sep 6	No Class	ISL Speech Workshop
Mon Sep 11	Waibel	Speech production, Signal Processing
Wed Sep 13	Rogina	Digital Signal Processing of Speech - Intro
Mon Sep 18	Pan	Signals Lab - NSH 2602
Wed Sep 20	Waibel	Vector Quantization, Template Matching I
Mon Sep 25	Waibel	Template-based recognition II
Wed Sep 27	Stern	Digital Signal Processing - Advanced Concepts
Mon Oct 2	Waibel	Hidden Markov Models I
Wed Oct 4	Waibel	Hidden Markov Models II
Mon Oct 9	Pan	Homework Review
Wed Oct 11	Woszczyna	Hidden Markov Models III
Mon Oct 16	Rosenfeld	Language Modeling I, (week of ICSLP)
Wed Oct 18	Rosenfeld	Language Modeling II, (Week of ICSLP)
Mon Oct 23	No Class	Mid-Semester Break
Wed Oct 25	Pan	Review, Project Definitions Due
Mon Oct 30	Schultz	Acoustic Modeling I
Wed Nov 1	Eskenazi	Phonetics and prosody
Mon Nov 6	Schultz	Acoustic Modeling II
Wed Nov 8	Schultz	Speaker Adaptation
Mon Nov 13	Waibel	Neural Networks I
Wed Nov 15	Waibel	Neural Networks II
Mon Nov 20	Gavalda	NLP
Wed Nov 22	No Class	Thanksgiving Holiday
Mon Nov 27	Waibel	Speech Translation
Wed Nov 29	Denecke	Dialog-based interfaces
Mon Dec 6	Waibel	Multimodal Interfaces
Sat Dec 16	Project presentations	ISL Lab, NSH2602

Homeworks and Tests

(Available only from cmu.edu and uka.de domains. )

Homework 1a. Out: Sep 18; Due: Sept 27 (link to Matlab help, CMU Matlab page)
Homework 1b. Out: Sep 27; Due: Oct 11
Homework 2. Out: Oct 30; Due Nov 8 (link to Statistical Language Modeling Toolkit)
Homework 3. Out: Nov 21; Due: Nov 29
Solution 1-2.

Term Project

(Available only from cmu.edu and uka.de domains. ) Project Ideas. (Drafts)

Slides

(Available only from cmu.edu and uka.de domains.)

Ivica Rogina's Automatic Speech Recognition on-line course from the Interactive Systems Labs at the Universitaet Karlsruhe (work in progress).
first set of slides (covering lectures 1 and 2)(1998).
Richard Stern's lectures on Signal Processing for Robust Speech Recognition(2000), Signal Processing for Speech Applications(1998) and Signal Processing Based on Auditory Physiology and Perception(1998).
Monika Woszczyna's lectures on Search(2000), Acoustic Modeling(1999) and Speech Translation(1999).
Roni Ronsenfeld's review on Statistical Language Modeling
Marsal Gavalda's lecture on Natural Language Understanding and The Soup Parser.
Michale Finke's lectures on Speaker Adaptation and Pronunciation Modeling.
Review and overview slides from partial review lecture.
Alon Lavie's lecture on Spoken NLP.
Alex Rudnicky's slides on Dialog Systems(1999).
Matthias Denecke's slides on Diaglog System(2000).

Mantained by ypan@cs.cmu.edu. Last modified: Dec 4, 2000