Part 10: PyTorch FSDP, End to End Walkthrough

FSDP provides a comprehensive framework for large model training in PyTorch. In this video, we walk through a working code base for a T5 grammar checker that is fully integrated with FSDP to quickly give you a feel for FSDP in action. Start getting comfortable with FSDP in under 14 minutes! Source of this PyTorch […]

Saving and Loading a PyTorch Neural Network (3.3)

Learn how to load and save PyTorch neural network models. It can take a long time to train a model. Saving the model allows you to keep a copy of the trained model. PyTorch stores neural networks in the Pickle file format. Code for This Video: https://github.com/jeffheaton/t81_558_deep_learning/blob/pytorch/t81_558_class_03_3_save_load.ipynb ~~~~~~~~~~~~~~~ COURSE MATERIAL ~~~~~~~~~~~~~~~ 📖 Textbook – Coming […]

Part 3: FSDP Mixed Precision training

Modern AI training rarely requires the full precision included with FP32. FSDP allows you to switch to fine grained, mixed precision training and potentially improve training performance over 70%. Learn how in under 7 minutes! Source of this PyTorch AI Video

How to Calculate a Mandelbrot Set

In this video short I show how to perform the calculations needed for a mandlebrot set. How to do this in Python: https://www.youtube.com/watch?v=WCsWY3p8Jms #shorts #mandelbrot #fractal Source of this machine learning/AI Video

SingularityNET: Deep Funding | Governance Voting Experiment

Deep Funding has always been about the community. In the current way of running this program, the community decides which projects are awarded a grant by casting a vote. However, this decision process can be extended to a lot of other topics concerning the governance of Deep Funding itself. In this video, Jan Horlings, VP […]

Assessing Data Quality at Shopify with Wendy Foster – #592

Today we’re back with another installment of our Data-Centric AI series, joined by Wendy Foster, a director of engineering & data science at Shopify. In our conversation with Wendy, we explore the differences between data-centric and model-centric approaches and how they manifest at Shopify, including on her team, which is responsible for utilizing merchant and […]

Part 1: Accelerate your training speed with the FSDP Transformer wrapper

Want to learn how to accelerate your transformer model training speed by up to 2x+? The transformer auto-wrapper helps FSDP better understand your model’s optimal wrapping points. This can accelerate your training throughput by 2x+ vs the default wrapper. Learn how to implement it in under 5 minutes! Source of this PyTorch AI Video

Early Stopping in PyTorch to Prevent Overfitting (3.4)

It can be difficult to know how many epochs to train a neural network for. Early stopping stops the neural network from training before it begins to seriously overfitting. Generally too many epochs will result in an overfit neural network and too few will be underfit. Code for This Video: https://github.com/jeffheaton/t81_558_deep_learning/blob/pytorch/t81_558_class_03_4_early_stop.ipynb ~~~~~~~~~~~~~~~ COURSE MATERIAL ~~~~~~~~~~~~~~~ […]

Two Minute Papers: NVIDIA’s New AI: Beautiful Simulations, Cheaper! 💨

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper “NeuralVDB: High-resolution Sparse Volume Representation using Hierarchical Neural Networks” is available here: https://developer.nvidia.com/rendering-technologies/neuralvdb https://blogs.nvidia.com/blog/2022/08/09/neuralvdb-ai/ https://arxiv.org/abs/2208.04448 📝 The paper with the water simulation is available here: https://users.cg.tuwien.ac.at/zsolnai/gfx/photorealistic-material-editing/ ❤️ Watch these videos in early access on our Patreon page or join […]

Evolution of Jobs and Skills in Tech | CogX 2019

Join the CogX Global Leadership Summit and Festival of AI and Breakthroughs Technology – June 8th to 10th 2020 – https://cogx.co/ Subscribe to our epic newsletters for free https://cognitionx.com/newsletter-subscribe/ Evolution of Jobs and Skills in Tech. Presentation given at CogX 2019, on the Future of Work and Education Dirk Pilat; Deputy Director, Directorate for Science, […]

Transformers for Tabular Data at Capital One with Bayan Bruss – #591

Today we’re joined by Bayan Bruss, a Sr. director of applied ML research at Capital One. In our conversation with Bayan, we dig into his work in applying various deep learning techniques to tabular data, including taking advancements made in other areas like graph CNNs and other traditional graph mining algorithms and applying them to […]

SingularityNET: Ambassador Townhall Meeting #18 | September 13th, 2022

Recording of our last week’s Community Ambassador Townhall Meeting. Notes: https://miro.com/app/board/uXjVO0WVUBA=/ Dework: https://app.dework.xyz/singularitynet-ambas/ GitBook: https://snet-ambassadors.gitbook.io/home/ The next Town Hall meeting will be going live and recorded on September 20th, at 18:00 UTC, on our Discord server. To join upcoming ambassador meetings, please make sure that you’re part of our Discord community! Discord: https://discord.gg/snet Special thanks […]

NVIDIA GTC 2022 – My Session Picks

Registration Link: https://nvda.ws/3vNRiLb (please use this one) NVIDIA Deep Learning Institute: https://www.nvidia.com/en-us/training/ My picks: Keynote https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1657326563846001zRzJ Roundtable https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1656098868289001rHuQ CUDA https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1655232120108001VJ29 https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/165471356386700183uX Career https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1656524764448001i2bG AWS GPU https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1659634536518001hKdn PyTorch Production https://www.nvidia.com/gtc/session-catalog/?tab.catalogallsessionstab=16566177511100015Kus#/session/1655499808507001ohEi Source of this machine learning/AI Video

Introduction to PyTorch

PyTorch is an open-source framework for prototyping, building, and deploying machine learning systems. Whether you’re a researcher in academia or a hobbyist dabbling in machine learning, there’s something for you in the PyTorch ecosystem! Suraj Subramanian and Justin Jeffress, Developer Advocates for PyTorch share a quick and easy-to-follow overview to get you started using PyTorch. […]

Introduction to PyTorch for Deep Learning with Python (3.2)

An introduction to PyTorch in Python. This video sets the foundation for the rest of the course by introducing TensorFlow and how it is accessed with Python. PyTorch neural networks are demonstrated as a higher level abstraction for deep neural networks. Code for This Video: https://github.com/jeffheaton/t81_558_deep_learning/blob/pytorch/t81_558_class_03_2_pytorch.ipynb Quick overview of Mandelbrot: https://www.youtube.com/shorts/mKbm0Qj80NI ~~~~~~~~~~~~~~~ COURSE MATERIAL ~~~~~~~~~~~~~~~ […]

Two Minute Papers: NVIDIA’s AI: Amazing DeepFakes And Virtual Avatars!

❤️ Check out Weights & Biases and say hi in their community forum here: https://wandb.me/paperforum 📝 The paper “One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing” is available here: https://nvlabs.github.io/face-vid2vid/ Try it out: http://imaginaire.cc/vid2vid-cameo/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: – https://www.patreon.com/TwoMinutePapers – https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join […]