PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke et al.: https://arxiv.org/abs/1912.01703
#ArtificialIntelligence #deepLearning #PyTorch
🔗 PyTorch: An Imperative Style, High-Performance Deep Learning Library
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.
Paszke et al.: https://arxiv.org/abs/1912.01703
#ArtificialIntelligence #deepLearning #PyTorch
🔗 PyTorch: An Imperative Style, High-Performance Deep Learning Library
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.
Машинное обучение, AI, нейронные сети, Big Data (VK)
Tensors | Deep Learning with PyTorch
https://www.youtube.com/watch?v=hXMoTDoehFY
Tensors | Deep Learning with PyTorch
https://www.youtube.com/watch?v=hXMoTDoehFY
YouTube
Tensors | Deep Learning with PyTorch
Tensors | Deep Learning with PyTorchComplete playlist - Deep Learning with PyTorch: https://www.youtube.com/playlist?list=PL1w8k37X_6L8oJGLWdzeOSRVTI6mL8vw7#...
Data Science / Machine Learning / AI / Big Data (VK)
PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models
Rozemberczki et al.: https://arxiv.org/abs/2104.07788
#MachineLearning #ArtificialIntelligence #PyTorch
PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models
Rozemberczki et al.: https://arxiv.org/abs/2104.07788
#MachineLearning #ArtificialIntelligence #PyTorch
Forwarded from Machinelearning
PyTorch представил усовершенствованные методы Activation Checkpointing (AC), цель которых - снижение потребления памяти при обучении.
Традиционный подход в
eager mode сохраняет промежуточные активации для обратного прохода, что зачастую приводит к значительному расходу ресурсов. AC позволяет не сохранять эти тензоры, а вычислять их заново при необходимости, тем самым жертвуя вычислительным временем ради экономии памяти.Новая техника – Selective Activation Checkpoint (SAC). В отличие от обычного AC, который затрагивает всю выбранную область, SAC дает гранулярный контроль над тем, какие операции следует пересчитывать, а какие – сохранять. Это достигается за счет использования
policy_fn, определяющей, нужно ли сохранять результаты конкретной операции. SAC будет полезен для избегания перевычисления ресурсоемких операций, например, матричных умножений.Для
torch.compile стала доступна Memory Budget API. Эта функция автоматически применяет SAC с оптимальной политикой, исходя из заданного пользователем бюджета памяти (от 0 до 1). Бюджет 0 соответствует обычному AC, а 1 – поведению torch.compile по умолчанию. @ai_machinelearning_big_data
#AI #ML #Pytorch
Please open Telegram to view this post
VIEW IN TELEGRAM
Please open Telegram to view this post
VIEW IN TELEGRAM