cs475-mlnlp-spring-2021

CS475 team project

Schedule

• slide submission & Peer grading:
  • slide submission deadline: 23:59, the day before your presentation
  • Peer grading: Real-time
• Week 7 (4/12, 4/14): Project proposal, Peer grading
• Week 12, 13 (5/17, 5/24, 5/26): Replication presentation, Peer grading
• Week 15 (6/7, 6/9): Final presentation, Peer grading
• Week 16 (6/14): Final report,Teamwork report

Goal

This project aims to replicate an existing paper on NLP fields in machine learning, and then make meaningful improvements/extensions in terms of research question, machine learning model, features, datasets, or evaluation metrics.

Teams

This is a team-based project. Please form teams of 4 people per group, 3 allowed for two or three groups.

Project Grading (50% of Course Grade)

• Team Project (50%)
  • Proposal presentation: 5%
  • Paper presentation: 10%
  • Final presentation: 20%
  • Final report: 10%
  • Teamwork report: 5%
• Peer review participation: 10%

Note that any team may get up to -25%p for project scores if there is a serious problem with teamwork. That means, the project grade could be 25/50, even if all presentation and report scores are perfect. This can apply to an individual student, or to the entire team.

Deliverables

All project team members should present an important part in the presentation (speak for at least one minute each). You should present your work in presentation session.

• Proposal presentation: 5-8 minute presentation
• Replication presentation: 7-10 minute presentation
• Final presentation: 10-15 minute presentation
• Final report: 4-6 pages report (+ unlimited references) and slides used for final presentation
• Peer grading: peer gradings for proposal, replication presentation, final presentation, and final report
• Teamwork report: (individually written/submitted) description of what each team member did

Instruction

Choose a paper with the following criteria.

• Recent paper published in 2018 to 2021
• Technical paper that includes a machine learning model and a medium- to large-sized dataset
• NLP paper from ACL, EMNLP, NAACL, TACL, NeurIPS, ICML, and ICLR
• If your paper deviates from the above criteria, or if your project is not based on replication of a previous paper, please email the teaching staff

Based on the paper, there are two options you can choose:

1. Replicate their method and apply it to your own research problem.
2. Replicate their method and improve it. You should find a problematic issue of the existing work and try to resolve/alleviate it.

Project Proposal

Give a 5-8 minute proposal presentation that includes the following:

• Problem: What is the topic of your paper? What is the specific task being tackled? Why is it important to solve this problem?
• Related work: What existing papers have dealt with this problem? What were their approaches?
• Replication approach: Which paper will you use for replication and why?
• Improvement approach: How will you improve upon the replication paper?
• Data: Describe the dataset you will use, even if it is the same one as in the replication paper. If the replication paper does not work with openly available data, explain how you will gather your own data. If using openly available data but you will work with a subset of the data, explain why and how you will select the subset.
• Plan: Explain a week-by-week plan of completing this project, clearly indicating which tasks will be done by which team member, and which tasks will be done together.

Replication Presentation

Give a 7-10 minute replication presentation that includes the following:

• Introduction: Please briefly explain your problem, approach, and dataset.
• Replication results:
  • How did you replicate the original paper? What problems did you face during the replication, and how did you solve them?
  • Describe the experiment setting, results of baseline implementation, replication of the chosen models and experiments. You may not have complete replication results, in which case just explain the results you have so far and a concrete plan of completing the rest.
• Plan:
  • Again, how will you improve or modify the original paper? Please have a more concrete plan for this based on the replication results.

Final Presentation & Final Report

Give a 10-15 minute final presentation and submit a 4-6 page (+ unlimited references) report in ACL format. Below is the structure of the final report. The final presentation should contain the same content but in a presentation style.

• Abstract: Summary of your work
• Introduction: Background and your motivation for the problem
• Approach: Your approach to solving the problem
• Data & Experiments: Data, model, algorithm, comparisons to baseline models, etc
• Results:
  • Replication results
  • Results of your improved approach: use graphs, tables, whatever is most appropriate to present your results
  • Interpretation of results: how do the results answer your research problem?
• Discussions: Your improvements and the limitations
• References (unlimited pages)

Peer Grading

• We will probably use Google form (TBD) and it will be done in the lecture.
• Everyone (individually) should grade and comment on
  • Proposal presentations
  • Replication presentations
  • Final presentations & reports

Teamwork Report

• We will use Google form
• Everyone should contribute, and everyone should learn something new
• Everyone (individually) should write
  • Who did what for each task (proposal presentation, replication presentation, project presentation, final report) in detail
  • Any grievances
  • Any praises and special recognitions
  • Due date: same as the final report evaluation