ML for NLP / CS475 / Spring 2021 KAIST
All contents in this document are tentative.
Teaching Staff
Alice Oh (Professor), Jiseon Kim (TA), Dongkwan Kim (TA)
When you send emails, please email to all TAs and prof. Oh. [Click me to see our emails.]
alice.oh@kaist.edu, jiseon_kim@kaist.ac.kr, dongkwan.kim@kaist.ac.kr
And put "CS475" to the title. (e.g., [CS475] Do we have a class on thanksgiving day?)
Course Description
This course will cover advanced and state-of-the-art machine learning for text data. ML methods covered will include graphical models, Bayesian inference, nonparametric models, and deep learning. By the end of the course, students will be able to
- Understand important concepts in NLP
- Read current research papers in NLP
- Implement some of the basic ML models for NLP
- Conduct replication studies based on a recent NLP+ML paper
- Communicate in written and spoken English about NLP+ML research
Course Addition
Due to resource limitation, we decided not to approve requests for course addition. We ask for your kind understanding regarding this matter.
Prerequisites
- You need to have good programming skills in Python.
- You need to have a basic understanding of ML concepts. You do not need to have taken CS376 or any other undergraduate ML course, but you need to know concepts such as train vs test data, clustering vs classification, accuracy/precision/recall, overfitting, and basic classification models such as SVM, random forest, etc. You can learn these concepts as we go along, but you may find some lectures and papers difficult to understand if you do not put in extra time to learn these concepts.
- We will use well-known frameworks for machine learning. You may start with little prior experience and learn these libraries during this semester, but that will require extra time and effort. Note that we do not provide any lectures about learning them.
- The topic of the course includes Korean NLP. You do not need to be fluent in Korean, but you need to know what the Korean alphabet (Hangeul) is and how they combine to form syllables and words.
Materials
- Papers from JMLR, ICML, NeurIPS, IJCAI, AAAI, ICLR, ACL, EMNLP, ArXiv, etc.
- Jacob Eisenstein, Natural Language Processing
Schedule (Subject to Change)
All the deadlines are 23:59 unless specified.
| Date |
Contents |
Important Deadlines |
Homework |
| 2021.03.03 |
Introduction |
|
|
| 2021.03.08, 2021.03.10 |
N-gram Language Models |
Team Building (2021.03.12) |
HW1 out (2021.03.08) |
| 2021.03.15, 2021.03.17 |
Word Vectors & Distributed Semantic |
|
HW1 end (2021.03.21) |
| 2021.03.22, 2021.03.24 |
Neural Network Basics |
|
|
| 2021.03.29, 2021.03.31 |
Recurrent Neural Networks |
|
HW2 out (2021.03.29) |
| 2021.04.05, 2021.04.07 |
No Class / Transformers |
|
HW2 end (2021.04.11) |
| 2021.04.12, 2021.04.14 |
Project Proposal |
Slide Submission (the day before your presentation) |
|
| 2021.04.19, 2021.04.21 |
No Class (Midterm) |
|
|
| 2021.04.26, 2021.04.28 |
Neural Languague Models (ELMo, BERT, etc.) |
|
HW3 out (2021.04.26) |
| 2021.05.03 |
No Class |
|
HW3 end (2021.05.09) |
| 2021.05.10, 2021.05.12 |
AI for Social Good / Ethics in NLP |
|
|
| 2021.05.17 |
Paper Presentation |
Slide Submission (the day before your presentation) |
|
| 2021.05.24, 2021.05.26 |
Paper Presentation |
Slide Submission (the day before your presentation) |
|
| 2021.05.31, 2021.06.02 |
QA & MT / Korean NLP and Multilinguality |
|
|
| 2021.06.07, 2021.06.09 |
Final Project Presentation (2h/day) |
Slide Submission (the day before your presentation) |
|
| 2021.06.14, 2021.06.16 |
No Class (Finalterm) |
Final Report (2021.06.14) |
|
Homeworks (Subject to Change)
- BOW
- RNN Family
- BERT
Team Projects
- You will form teams of three or four, and as a team, pick one NLP paper from ACL, EMNLP, NAACL, TACL, NeurIPS, ICML, and ICLR, published in 2018 to 2021, and replicate it. You will be required to change at least one thing – dataset, model, or research question. More details will be given out during the first week of class.
- https://uilab-kaist.github.io/cs475-mlnlp-spring-2021/project
Evaluation
Your grade will be a combination of the following:
- Homework 40%
- Team Project 50%
- Proposal 5%
- Paper presentation 10%
- Final presentation 20%
- Written report 10%
- Teamwork 5% (Note that any team may get up to -25% if there is a serious problem with teamwork)
- Peer Review Participation 10%