Natural Language Processing

Instructor: Dr. Chandresh Kumar Maurya

Lecture : Online (over Google meet, see calendar invite)

Time : Class on Sat from 3-5 pm, Tutorial on Thu 3-4pm. Thu 9am will be reserved

Office : 415 in POD A building

Meeting hours: 16:00-17:00 PM

**TA : M Ratna **

Text Books:

1. Daniel Jurafsky & James H. Martin : *Speech and Language Processing* : Pearson Education India : India : 2013 : 9789332518414  (2nd and 3rd ed.)
2. Manning and Schutze : *"Statistical Natural Language Processing* : MIT Press : Cambridge, MA : 1999 : 0262133601

What is this course about?

Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, etc. In the last decade, deep learning (or neural network) approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering. In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning/machine learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand basics of NLP.

Prerequisites:

  1. Proficiency in Python
  2. College Calculus, Linear Algebra
  3. Basic Probability and Statistics
  4. Foundations of Machine Learning

Grading policy:

  1. Assignments– There will be weekly assignments.
  2. Projects– One Major that will lead to a short paper
  3. MSE
  4. ESE

Projects:

  • Pick a paper from ACL 2021 proceedings link.
  • Submit a proposal in Stanford overleaf template ( link ) following the guidelines given in the template by Sat, Jan 15. You can form a team of at most 3 people (5 marks).
  • At the end of the course, you will submit your paper in the ACL format link.

Assignments

  1. See LMS for assignments

Lecture Slides (Videos are only accessible inside IIT Indore)

Date Topics Reading/References Slides/Papers/demo
Dec 30 Introduction to NLP Read chapter 1 and 2 in Jurafsky's book PDF,Video ,Video
Jan 8 Words, Morpholoy, stemming, lemmatization, edit distance Read chapter 2 in Jurafsky's book PDF,Video1,Video2 Video3
Jan 15 Language Model Read chapter 3 in Jurafsky's book 3re ed. PDF, Video1,Video2
Jan 22 Language Models Read chapter 3 in Jurafsky's book 3re ed. PDF, Video1,Video2
Jan 22 Neural Language Models See the slides and videos PDF, Video1
Jan 29 Text Classification, Naive Bayes and sentiment analysis See the slides and videos, NB notes PDF, Video1, Video2, Video3
Feb 10 POS Tagging (HMM) See the slides and videos PDF, Video1, Video2, Video3
Feb 12 Named Entity Recoginition (NER) See the slides and videos PDF, Video
Feb 17 Constituenty parsing (CFG, CKY) See the slides and videos PDF1, PDF2 Video1, Video2
March 10 Dependency Parsing See the slides and videos PDF, Video1,Video2
March 31 Topics Models See the slides and videos (see also wiki page for LDA) PDF, Video1,Video2
April 4 Machine Translation (including NMT) See the slides and videos PDF1, PDF2 Video1,Video2
April 7 Information Extraction See the slides and videos PDF1, PDF2, Video1,Video2
April 18 Question Answering See the slides and videos PDF, Video

Tutorials

Date Topics Reading/References Slides/Papers/demo
Jan 1 Introduction to regex, tokenization, etc. Read chapter 1 and 2 in Jurafsky's book PDF,Video
Jan 6 tokenization issues, normalization, etc. Read chapter 1 and 2 in Jurafsky's book PDF,Video1, Video2, Notebook
Jan 13 Tensorflow introduction See the notebook and tensorflow doc Video, Notebook
Jan 20 Langauge Modeling Demo using Hugging face library See the notebook and video Notebook, Video
Jan 27 Neural LM See the notebook and video Notebook, Video
March 6 CKY Algorithm See the notebook and video Notebook, Video
March 29 Dependency Parsing See the notebook and video Notebook, Video
April 9 Machine Translation Demo See the notebook and video Notebook, Video
April 16 QA Demo See the notebook and video Notebook, Video