Study Material / Introduction to Large Language Models (LLMs) (November 2025)
Introduction to Large Language Models (LLMs)
November 2025
3 credits
12 weeks
Prof. Tanmoy Chakraborty, Prof. Soumen Chakraborti
IIT Delhi, IIT Bombay
Practice
Solutions
This course introduces the fundamental concepts underlying Large Language Models (LLMs). It starts with an introduction to the various problems in NLP, and discusses how to approach the problem of language modeling using deep learning. It describes the architectural intricacies of Transformers and the pre-training objectives of the different Transformer-based models. It also discusses the recent advances in LLM research, including LLM alignment, prompting, parameter-efficient adaptation, hallucination, bias and ethical considerations. This course prepares a student to comprehend, critique and approach various research problems on LLMs.
Last updated in October 2025.
Week 1
Course Introduction
- Course Introduction
- Introduction to NLP (NLP Pipeline, Applications of NLP)
Week 2
Introduction to Statistical Language Models
- Statistical Language Models: Advanced Smoothing and Evaluation
Week 3
Introduction to Deep Learning
- Perceptron, ANN, Backpropagation, CNN
- Introduction to PyTorch
Week 4
Word Representation and Tokenization
- Word2Vec, fastText
- GloVe
- Tokenization Strategies
Week 5
Neural Language Models and Attention
- CNN, RNN
- LSTM, GRU
- Sequence-to-Sequence Models, Greedy Decoding, Beam search
- Other Decoding Strategies: Nucleus Sampling, Temperature Sampling, Top-k Sampling
- Attention in Sequence-to-Sequence Models
Week 6
Introduction to Transformers
- Self and Multi-Head Attention
- Positional Encoding and Layer Normalization
- Implementation of Transformers using PyTorch
Week 7
Pre-Training Strategies and HuggingFace
- Pre-Training Strategies: ELMo, BERT (Encoder-only Model)
- Pre-Training Strategies: Encoder-decoder and Decoder-only Models
- Introduction to HuggingFace
Week 8
Instruction Tuning and Alignment
- Instruction Tuning
- Prompt-based Learning
- Advanced Prompting Techniques and Prompt Sensitivity
- Alignment of Language Models with Human Feedback (RLHF)
Week 9
Retrieval-Augmented Systems
- Open-book question answering: The case for retrieving from structured and unstructured sources
- Retrieval-augmented inference and generation
- Key-value memory networks in QA for simple paths in KGs
- Early HotPotQA solvers, pointer networks, reading comprehension
- REALM, RAG, FiD, Unlimiformer
- KGQA (e.g., EmbedKGQA, GrailQA)
Week 10
Knowledge Graphs
- Representation, completion
- Tasks: Alignment and isomorphism
- Distinction between graph neural networks and neural KG inference
Week 11
Parameter-efficient Adaptation and Interpretability
- Parameter-efficient Adaptation (Prompt Tuning, Prefix Tuning, LoRA)
- An Alternate Formulation of Transformers: Residual Stream Perspective
- Interpretability Techniques
Week 12
Modern Models and Ethical Considerations
- Overview of recently popular models such as GPT-4, Llama-3, Claude-3, Mistral, and Gemini
- Ethical NLP – Bias and Toxicity
- Conclusion
Courses
Wildlife Ecology (2025)Conservation Geography (2025)Psychology of Learning (2025)Introduction to Large Language Models (LLMs) (2025)Educational Leadership (2025)Forests and their Management (2025)Conservation Economics (2025)Psychology of Learning (2024)Wildlife Ecology (2024)Google Cloud Computing Foundations (2024)
Made by Arsh
Copyright © 2025 Arsh Saxena.
Made with 
in India.
in India.