All of the [ PyTorch | Transformers ]
Recurring Blog | Last Updated: June 09, 2023
1 min readJun 2, 2022
I am a Ph.D. Candidate at LCS2 Lab. I work primarily in the area of conversational systems for mental health. My day-to-day tasks involve complex PyTorch coding and making sophisticated use of deep learning models for the NLP use cases.
There are plenty of resources on the internet to learn applied deep learning for NLP use cases using PyTorch. Here, I am just collating all relevant resources which I have found helpful during my learning course.
Initially, this blog may look scattered, but I will keep organizing it. This will be a continuously updating blog for many years. All your suggestions really matter. Do comment!!!
Pytorch:
- This one single video is enough to cover all of the basic PyTorch
https://www.youtube.com/watch?v=c36lUUr864M
Credit: Patrick Lober
Pytorch x Transformer:
- The best of the best https://nlp.seas.harvard.edu/2018/04/03/attention.html
Transformer Theory:
- Attention:
(1) [Undoubtedly the best] Transformer blog by Jay Alammar
https://jalammar.github.io/illustrated-transformer/
(2) (a better source) https://sebastianraschka.com/blog/2023/self-attention-from-scratch.html
(3) (second best maybe) https://theaisummer.com/self-attention/ - BERT Inner Working:
(1) https://www.topbots.com/bert-inner-workings/
(2) https://gmihaila.medium.com/%EF%B8%8F-bert-inner-workings-1c3054cd1591
Transformer Pretraining & Finetuning:
- Don't miss out on the Hugging Face course. It is reallllly good.
- Pretrain: https://www.topbots.com/pretrain-transformers-models-in-pytorch/
- BERT Finetune: https://www.topbots.com/fine-tune-transformers-in-pytorch/
- BERT Finetune: https://medium.com/grabngoinfo/transfer-learning-for-text-classification-using-pytorch-5e52045d55f8
- GPT2 Classification: https://www.topbots.com/gpt2-text-classification-using-hugging-face-transformers/
Some Extra Tips and Tricks:
