Advanced Topics in Natural Language Processing

• Time: Wednesday and Friday 2pm-3:15pm
• Place: zoom link sent to class mailing list
• Instructor: Prof. Heng Ji (Email: hengji@illinois.edu);
• TA: Vicki Qi Zeng (Email: qizeng2@illinois.edu) and Pengfei Yu (Email: pengfei4@illinois.edu)
• Instructor and TA Office Hours: Wednesday and Friday 3:15pm-3:45pm
• Course Folder: Google Drive (Please log in with your illinois email account).

Lectures and Schedules

Date	Lecture	Reading	Assignment
08/25	Overview	Reading on Embeddings
08/27	Language Encoding and Decoding	Reading on DNN for NLP
09/01	Multilingual Multimedia Encoding
09/03	Deep Neural Networks for NLP	Reading on Joint IE
09/08	Meta-learning for NLP
09/10	Joint Neural Models for Information Extraction	Reading on Multilingual Multimedia IE	Assignment 1 (due on 10/13)
09/15	Multilingual Information Extraction
09/17	Multimedia Information Extraction	Reading on Open IE and Schema Induction
09/22	Open-domain Information Extraction
09/24	Schema Induction and Knowledge Acquisition	Reading on Misinformation and QA
09/29	Misinformation Detection
10/01	Question Answering	Reading on Knowledge Conditioned NLG
10/06	Natural Language Generation
10/08	Natural Language Generation	Reading on Prompt Learning	Project Proposal Due 10/20, see Project Proposal Requirement
10/13	Knowledge Controlled Natural Language Generation
10/15	Prompt Learning (guest lecture by Pengfei Liu (CMU))	Reading on Reading Comprehension	Assignment 2 (due on 10/29)
10/20	Project proposal presentation
10/22	Project proposal presentation (Cont')	Reading on Summarization
10/27	Project proposal presentation (Cont')
10/29	Reading Comprehension	Reading on Dialogue Systems
11/03	Summarization		Assignment 3 (due on 11/17)
11/05	Dialogue Systems	Reading on Deep learning from Graphs for NLP
11/10	Guest lecture on Deep Learning from Graphs for NLP
11/12	Guest lecture on Deep Learning from Graphs for NLP	NLP and Computational Social Science	Assignment 4 (due on 12/03)
11/17	NLP and Computational Social Science
11/19	NLP and Computational Social Science (Cont')
11/24	Fall Break (No Class)
11/26	Fall Break (No Class)
12/01	Final Project Presentations		Final project presentation 12/01 1pm CT
12/03	Final Project Presentations
12/08	Final Project Presentations
12/10	Final Project Presentations		Final Paper Due

Course Description

In this course we will teach advanced topics in natural language processing, ranging from general techniques such as deep learning for NLP to specific topics such as information extraction, question answering, reading comprehension, summarization, dialogue systems, and natural language generation. Review of classic as well as state-of-the-art techniques and remaining challenges, and exploration of recent proposals for meeting these challenges. Intended for graduate students doing research in natural language processing.

 

Grading

• The instructor gives an overview tutorial (50-60mins), followed by Q/A and paper discussions (15-25mins).
• 4 Assignments (40 pts in total): participate in shared tasks and submit through Codalab, the grades will be based on your system’s rank in the class. If a submission is successfully complete then it will get 5 basic points, and then the additional points are based on the system's relative rank in the class between [0, 2], and 0-3 points will be assigned to analysis and report writing. Well-written error analysis and discussions will receive extra points.
• Term Project (55 pts in total, 5pts proposal, 25pts project implementation, 25pts final report writing/presentation) Team work is encouraged, but a team cannot be more than 4 students. Everyone is encouraged to submit papers to NAACL/ACL based on the term projects.
• Reading assignment (5 points each session): Each student is required to sign up one paper critique session, 3mins with or without slides.

Course Materials

The instructor will write most of the slides and hand-outs, give survey about the best papers from

• ACL Papers: ACL Anthology.

The following books may provide some useful background:

• Daniel Jurafsky and James Martin. Speech and Language Processing, Prentice-Hall (second edition).
• Christopher D. Manning and Hinrich Schutze. Foundations of Statistical Natural Language Processing. MIT Press.
• Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning. MIT Press.
• Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning.

Prerequisites

• Programming in Python
• Already took NLP and machine learning courses
• Familiar with Probability and Statistics
• Research background in NLP or related fields; it's targeted for students who work on related areas.
• Interests in languages and lingusitics
• Solid background in algorithms
• Good programming skills
• Sufficient mathematical background

Class Policy

• Restricted Academic Integrity
• NO Incompleteness are accepted
• NO Late Assignments are accepted
• NO cell phones and internet surfing are allowed in the lecture room
• Don't come late to the class, if you are late more than 10 minutes, simply skip it in order not to disturb the lecture

Mailing List

• https://groups.google.com/g/cs546

Office Hours and Paper Critique

• Please sign up for office hours and find room information here.
• Please sign up for paper critiques here.