Create a Large Language Model from Scratch with Python

Mark as Favorite Share
image

What Will You Learn?

In this course, you'll learn to construct your own large language model from the ground up, delving into the intricacies of data handling, mathematics, and transformer architectures. Practical implementation will be carried out using Python, providing you with hands-on experience in building and understanding the mechanics of large language models.

About This Course

Provider: Youtube
Format: Online
Duration: 6 hours to complete [Approx]
Target Audience: Beginners
Learning Objectives: Upon completion this free course, you will be able to construct a large language model from scratch, gaining expertise in data handling, mathematical concepts, and transformer architectures, all implemented using Python.
Course Prerequisites: NA
Assessment and Certification: NA
Instructor: Freecodecamp
Key Topics: Large Language Model, Python Programming, Artificial Intelligence
Topic Covered: 
  2. - Introduction
  3. - Install Libraries
  4. - Pylzma build tools
  5. - Jupyter Notebook
  6. - Download wizard of oz
  7. - Experimenting with text file
  8. - Character-level tokenizer
  9. - Types of tokenizers
  10. - Tensors instead of Arrays
  11. - Linear Algebra heads up
  12. - Train and validation splits
  13. - Premise of Bigram Model
  14. - Inputs and Targets
  15. - Inputs and Targets Implementation
  16. - Batch size hyperparameter
  17. - Switching from CPU to CUDA
  18. - PyTorch Overview
  19. - CPU vs GPU performance in PyTorch
  20. - More PyTorch Functions
  21. - Embedding Vectors
  22. - Embedding Implementation
  23. - Logits and Reshaping
  24. - Generate function and giving the model some context
  25. - Logits Dimensionality
  26. - Training loop + Optimizer + Zerograd explanation
  27. - Optimizers Overview
  28. - Applications of Optimizers
  29. - Loss reporting + Train VS Eval mode
  30. - Normalization Overview
  31. - ReLU, Sigmoid, Tanh Activations
  32. - Transformer and Self-Attention
  33. - Transformer Architecture
  34. - Building a GPT, not Transformer model
  35. - Self-Attention Deep Dive
  36. - GPT architecture
  37. - Switching to Macbook
  38. - Implementing Positional Encoding
  39. - GPTLanguageModel initalization
  40. - GPTLanguageModel forward pass
  41. - OpenWebText download and Survey of LLMs paper
  42. - How the dataloader/batch getter will have to change
  43. - Extract corpus with winrar
  44. - Python data extractor
  45. - Adjusting for train and val splits
  46. - Adding dataloader
  47. - Training on OpenWebText
  48. - Training works well, model loading/saving
  49. - Pickling
  50. - Fixing errors + GPU Memory in task manager
  51. - Command line argument parsing
  52. - Porting code to script
  53. - Prompt: Completion feature + more errors
  54. - nnModule inheritance + generation cropping

0 Comments

No reviews yet !!

Please login first

Course Tags
AI Course AI Free Course Large Language Model Course Top Free Course Free AI Course Large Language Model Free Course