Applying deep learning to Airbnb search

Story of how #Airbnb research team moved from using #GBDT (gradient boosting) to #NN (neural networks) for search, with all the metrics and hypothesises.

Link: https://blog.acolyer.org/2019/10/09/applying-deep-learning-to-airbnb-search/

Implementing Transfer Learning in PyTorch

Fine-tuning and feature extraction with PyTorch

Link: https://medium.com/analytics-vidhya/transfer-learning-in-pytorch-f7736598b1ed

#PyTorch #novice #entrylevel #beginner

ODS breakfast in Paris! See you this Saturday (26th) at 10:30 at Malongo Café, 50 Rue Saint-André des Arts.

Efficient multi-lingual language model fine-tuning

Most of the world’s text is not in English. To enable researchers and practitioners to build impactful solutions in their domains, understanding how our NLP architectures fare in many languages needs to be more than an afterthought.
In this post, we introduce our latest paper that studies multilingual text classification and introduces #MultiFiT, a novel method based on #ULMFiT.

MultiFiT, trained on 100 labeled documents in the target language, outperforms multi-lingual BERT. It also outperforms the cutting-edge LASER algorithm-even though LASER requires a corpus of parallel texts, and MultiFiT does not.

Post: http://nlp.fast.ai/classification/2019/09/10/multifit.html…
Paper: https://arxiv.org/abs/1909.04761
Tweet: https://twitter.com/seb_ruder/status/1186744388908654597?s=20

#NLP #DL #FineTuning

Learning a unified embeding for visual search at #Pinterest

How Pinterest created unified embeddings for images from different fields searching instead three different by use #multitask approach.

Link: https://blog.acolyer.org/2019/10/11/learning-a-unified-embedding-for-visual-search-at-pinterest/

#Search #CV #embeddings

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)

The approach casts every language problem as a text-to-text task. For example, English-to-German translation – input: "translate English to German: That is good." target: "Das ist gut." or sentiment ID – input: "sentiment: This movie is terrible!", target: "negative"

Transfer learning for NLP usually uses unlabeled data for pre-training, so they assembled the "Colossal Clean Crawled Corpus" (C4), ~750GB of cleaned text from Common Crawl.

Compared to different architectural variants including encoder-decoder models and language models in various configurations and with various objectives. The encoder-decoder architecture performed best in our text-to-text setting.

More at the thread by the tweet: https://twitter.com/colinraffel/status/1187161460033458177?s=20

Paper: https://arxiv.org/abs/1910.10683
Code/models/data/etc: https://github.com/google-research/text-to-text-transfer-transformer

#NLP #DL #transformer

How Trip Inferences and Machine Learning Optimize Delivery Times on Uber Eats

Article on how business task can be decomposed to ML problem

Link: https://eng.uber.com/uber-eats-trip-optimization/

#Uber #ml #taskdesign #analytics

Two papers stating random architecture search is a competitive (in some cases superior) baseline for NAS methods.

These are papers demonstrating that Neural Architecture Search can be stohastic.

Paper 1: https://arxiv.org/abs/1902.08142
Paper 2: https://arxiv.org/abs/1902.07638

#NAS #nn #DL

​​ICCV 2019 papers

ICCV 2019 – one of the major tier A conferences on Computer Vision. These are papers presented at the conference. We are definitely going to post short descriptions of the most influential ones, but if you don't want to wait, here is the link:

Link: http://openaccess.thecvf.com/ICCV2019.py

#CV #Papers

FUNIT: Few-Shot Unsupervised Image-to-Image Translation

A team of NVIDIA researchers has defined new AI techniques that give computers enough smarts to see a picture of one animal and recreate its expression and pose on the face of any other creature. The work is powered in part by generative adversarial networks (GANs), an emerging AI technique that pits one neural network against another.

Blog: https://blogs.nvidia.com/blog/2019/10/27/ai-gans-pets-ganimals/
Paper: https://arxiv.org/abs/1905.01723
Сode: https://github.com/NVlabs/FUNIT
GANimal app: http://nvidia-research-mingyuliu.com/ganimal/

#CV #GAN #ICCV

YOLACT: Real-time Instance Segmentation

Fully-convolutional model for real-time instance segmentation that achieves 29.8 mAP on MS COCO at 33.5 fps evaluated on a single Titan Xp, which is significantly faster than any previous competitive approach. They obtain this result after training on only one GPU.


video: https://www.youtube.com/watch?v=0pMfmo8qfpQ
paper: https://arxiv.org/abs/1904.02689
code: https://github.com/dbolya/yolact

#yolo #instance_segmentation #segmentation #real_time

🎃Moscow Data Halloween on the 31st of October

It’s gonna be one of the most unusual data science meetups!

We will have several Black ML talks, Data Science PPT Karaoke from Hell, costume contest with prizes, lots of fun and afterparty.

Registration link: https://corp.mail.ru/ru/press/events/678/

NLP News: Deep Learning Indaba, EurNLP, ML echo chamber, Pretrained LMs, Reproducibility papers

The famous Sebastion Ruder (Research scientist @ DeepMindAI) wrote an interesting article about the last NLP news

article: http://newsletter.ruder.io/issues/deep-learning-indaba-eurnlp-ml-echo-chamber-pretrained-lms-reproducibility-papers-199557
tweet: https://twitter.com/seb_ruder/status/1186567939232817153?s=20

#NLP #News #Conference

🏆 Moscow ML Trainings meetup on the 2nd of November

ML Trainings are based on Kaggle and other platform competitions and are held regularly with free attendance and a live stream. Winners and top-performing participants discuss competition tasks, share their solutions, and results.

Program and the registration link - https://corp.mail.ru/ru/press/events/682/
Live stream link - https://youtu.be/VNsXzK4C7gg
* Note: this time all the talks will be in Russian. Usually, we have one talk in English. @mltrainings

ODS breakfast in Paris! See you this Saturday (2nd of November) at 10:30 at Malongo Café, 50 Rue Saint-André des Arts.

6-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypoints

It's deep learning approach to category-level 6D object pose tracking on RGB-D data. this method tracks in real-time novel object instances of known object categories such as bowls, laptops, and mugs. 6-PACK learns to compactly represent an object by a handful of 3D keypoints, based on which the interframe motion of an object instance can be estimated through keypoint matching.
These keypoints are learned end-to-end without manual supervision to be most effective for tracking. Their experiments show that the method substantially outperforms existing methods on the NOCS category-level 6D pose estimation benchmark and supports a physical robot to perform simple vision-based closed-loop manipulation tasks.

preprint: https://arxiv.org/abs/1910.10750
code: https://github.com/j96w/6-PACK
tweet: https://twitter.com/RobobertoMM/status/1187617487837257733?s=20
video: https://www.youtube.com/watch?v=INBjNZsnfy4


#CV #DL #PatternRecognition

Keras Tuner

Fully-featured, scalable, easy-to-use hyperparameter tuning for Keras & beyond.

It supports RandomSearch, BayesianOptimization, and Hyperband. It can run locally or in a distributed setting. It's possible to have both multi-device single-model training (one machine training one model over 8 GPUs) and distributed search (many models in parallel) at the same time


documentation: https://keras-team.github.io/keras-tuner/
tweet: https://twitter.com/fchollet/status/1189992078991708160?s=21

#DL #keras #Tuning #BayesianOptimization

​​🔥DeepMind’s AlphaStar beats top human players at strategy game StarCraft II

AlphaStar by Google’s DeepMind can now play StarCraft 2 so well that it places in the 99.8 percentile on the European server. In other words, way better than even great human players, achieving performance similar to gods of StarCraft.

Solution basically combines reinforcement learning with a quality-diversity algorithm, which is similar to an evolutionary algorithm.

What’s difficult about StarCraft and how is it different to recent #Go and #Chess AI solutions: even finding winning strategy (StarCraft is famouse to closeness to rock-scissors-paper, not-so-transitive game design, as chess and go), is not enough to win, since the result depends on execution on different macro and micro levels at different timescales.

How that is applicable in real world: basically, it is running logistics, manufacture, research with complex operations and different units.

Why this matters: it brings AI one step closer to running real business.

Blog post: https://deepmind.com/blog/article/AlphaStar-Grandmaster-level-in-StarCraft-II-using-multi-agent-reinforcement-learning
Nature: https://www.nature.com/articles/d41586-019-03298-6
ArXiV: https://arxiv.org/abs/1902.01724
Nontechnical video: https://www.youtube.com/watch?v=6eiErYh_FeY

#Google #GoogleAI #AlphaStar #Starcraft #Deepmind #nature #AlphaZero

​​SinGan: Learning a Generative Model from a Single Natural Image

Best Paper Award at #ICCV2019. A generative model, which learns from a single natural image, and then generates random samples.

ArXiV: https://arxiv.org/pdf/1905.01164v2.pdf
Github: https://github.com/tamarott/SinGAN

#GAN #ICCV #BestPaperAward

Matus Telgarsky’s Deep Learning Theory course

Course syllabus, lecture handout materials from Illinois university.

Link: http://mjt.cs.illinois.edu/courses/dlt-f19/

#MOOC #DL #Theory #Course