|
Lecturer(s)
|
-
Kouřil Emil, Ing. Ph.D.
-
Nemo 19 Martin, Ing. Ph.D.
-
Průša Jaroslav, Ing.
|
|
Course content
|
1. Introduction, N-gram language models. 2. Neural networks. Backpropagation algorithm. Partial derivatives. 3. Word vectors. 4. Convolutional neural networks in NLP. 5. Recurrent neural networks. 6. Vanishing gradient. 7. LSTM, tagging. 8. Encoder-decoder architecture, attention principle, machine translation. 9. Transformer architecture. 10. BERT and other pretrained models. 11. Text generation and generative models. 12. Prompting models.
|
|
Learning activities and teaching methods
|
Lecture supplemented with a discussion, Lecture with practical applications, E-learning, Discussion, Multimedia supported teaching, Students' portfolio, One-to-One tutorial, Individual study, Students' self-study, Lecture, Practicum
- Practical training (number of hours)
- 26 hours per semester
- Contact hours
- 26 hours per semester
- Individual project (40)
- 60 hours per semester
- Preparation for formative assessments (2-20)
- 10 hours per semester
- Preparation for an examination (30-60)
- 40 hours per semester
|
| prerequisite |
|---|
| Knowledge |
|---|
| having an overview of basic methods of probability and statistics |
| solving computer tasks at the level of Bachelor degree in Computer Science or a similar field |
| having an overview of basic methodshaving an overview of basic methods of probability and statistics |
| Skills |
|---|
| decompose tasks into simpler units |
| solve linear algebra problems |
| implement more advanced programs in an imperative programming language |
| Competences |
|---|
| N/A |
| learning outcomes |
|---|
| Knowledge |
|---|
| be familiar with multilingual text processing |
| be familiar with basic text summarization methods |
| be familiar with evaluating the success of natural language processing methods |
| describe the principles of natural language processing and text data retrieval |
| Skills |
|---|
| train language models |
| create algorithms for sentence parsing |
| create algorithms for automatic evaluation of semantic similarity of words sentences and documents |
| create named entity recognition algorithms |
| create machine learning algorithms |
| apply machine learning to natural language processing |
| Competences |
|---|
| N/A |
| N/A |
| teaching methods |
|---|
| Knowledge |
|---|
| Self-study of literature |
| Practicum |
| Lecture supplemented with a discussion |
| Interactive lecture |
| Discussion |
| One-to-One tutorial |
| E-learning |
| Multimedia supported teaching |
| Skills |
|---|
| Individual study |
| Competences |
|---|
| Interactive lecture |
| assessment methods |
|---|
| Knowledge |
|---|
| Test |
| Oral exam |
| Skills |
|---|
| Seminar work |
| Competences |
|---|
| Oral exam |
|
Recommended literature
|
-
Aurélien Géron. Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O'Reilly Media, 2017. ISBN 1491962291.
-
Delip Rao, Brian McMahan. Natural Language Processing with PyTorch: Build Intelligent Language Applications Using Deep Learning. ISBN 1491978236.
-
François Chollet. Deep Learning with Python. 2017. ISBN 9781617294433.
-
Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA, USA, 1999.
-
Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning (Adaptive Computation and Machine Learning series). MIT press, 2016. ISBN 9780262035613.
-
Jacob Eisenstein. Introduction to Natural Language Processing (Adaptive Computation and Machine Learning series). MIT Press, 2019. ISBN 0262042843.
-
Jurafsky, Daniel; Martin, James H. Speech and language processing. 2nd ed. Harlow: Pearson, 2014. ISBN 978-1-292-02543-8.
|