We demonstrate the possibility of what we call sparse learning: accelerated training of deep neural networks that maintain sparse weights throughout training while achieving dense performance...

An overview of the evolution of NLP for writing text

Misspelling Oblivious Embeddings (MOE) is a new model for word embeddings that are resilient to misspellings, improving the ability to apply word embeddings to real-world situations, where misspellings are common.

Cartoons receive increasingly attention and have a huge global market.
Cartoon person recognition has a wealth of application scenarios. However,
there is no large and high quality dataset for...

NVIDIA DGX SuperPOD trains BERT-Large in just 47 minutes, and trains GPT-2 8B, the largest Transformer Network Ever with 8.3Bn parameters Conversational AI is an essential building block of human…

Discussions:
Hacker News (64 points, 3 comments), Reddit r/MachineLearning (219 points, 18 comments)


Translations: Simplified Chinese, French, Korean, Russian, Turkish






This year, we saw a dazzling application of machine learning. The OpenAI GPT…

Deep Reinforcement Learning (DRL) has become a powerful methodology to solve
complex decision-making problems. However, DRL has several limitations when
used in real-world problems (e.g., robotics...

Many automated processes such as auto-piloting rely on a good semantic segmentation as a critical component. To speed up performance, it is common to downsample the input frame. However, this comes at the cost of missed small objects and reduced accuracy…

A new paper by Liu, Jian, He et al introduces RAdam, or “Rectified Adam”. It’s a new variation of the classic Adam optimizer that provides…

The BAIR Blog

Your daily dose of AI research from AK

Recent explainability related studies have shown that state-of-the-art DNNs
do not always adopt correct evidences to make decisions. It not only hampers
their generalization but also makes them...

Python API for LVIS Dataset. Contribute to lvis-dataset/lvis-api development by creating an account on GitHub.

Autonomous cars need to learn how to drive like a local

We’re releasing the 774 million parameter GPT-2 language model after the release of our small 124M model in February, staged release of our medium 355M model in May, and subsequent research with partners and the AI community into the model’s potential for…

Simple transformer implementation from scratch in pytorch. - pbloem/former