A set of tutorials and courses on Geometric Deep Learning:
https://people.lu.usi.ch/bronstem/teaching_tutorial.html
#deep_learning #geometric_deep_learning #machine_learning

Lectures Slides of Signal Processing for Machine Learning Course by Stanfrod University
http://web.stanford.edu/class/ee269/slides.html
#mathematics #machine_learning

Solving Rubik’s Cube with a Robot Hand

This is fascinating, make sure you read it.

Summary: OpenAI team trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The neural networks are trained entirely in simulation, using the same reinforcement learning code as OpenAI Five paired with a new technique called Automatic Domain Randomization (ADR). The system can handle situations it never saw during training, such as being prodded by a stuffed giraffe. This shows that reinforcement learning isn’t just a tool for virtual tasks, but can solve physical-world problems requiring unprecedented dexterity.

https://openai.com/blog/solving-rubiks-cube/
#reinforcement_learning #machine_learning #robotics

A must read document for deep learning & machine learning practitioners

https://www.deeplearningbook.org/contents/guidelines.html
#deep_learning #machine_learning

Self-training with Noisy Student improves ImageNet classification

New state-of-the-art supervised+unsupervised algorithm on ImageNet

https://arxiv.org/abs/1911.04252
#machine_learning #neural_networks #meta_learning

How Relevant is the Turing Test in the Age of Sophisbots?

Popular culture has contemplated societies of thinking machines for generations, envisioning futures from utopian to dystopian. These futures are, arguably, here now-we find ourselves at the doorstep of technology that can at least simulate the appearance of thinking, acting, and feeling. The real question is: now what?

https://arxiv.org/pdf/1909.00056.pdf
#machine_learning #technology #ethics

A great discussion with Sebastian Thrun about various topics such as: Flying Cars, Autonomous Vehicles, and Education

https://www.youtube.com/watch?v=ZPPAOakITeQ
#self_driving_cars #education #artificial_intelligence #machine_learning

A fruitful relationship between neuroscience and AI

https://deepmind.com/blog/article/Dopamine-and-temporal-difference-learning-A-fruitful-relationship-between-neuroscience-and-AI
#reinforcement_learning #machine_learning #neuroscience #artificial_intelligence

Complete Statistical Theory of Learning

https://www.youtube.com/watch?v=Ow25mjFjSmg
#statistics #machine_learning
#theory

https://t.co/fHiVaFDere?amp=1
#AI #machine_learning #education

TensorFlow Quantum: An Open Source Library for Quantum Machine Learning

https://ai.googleblog.com/2020/03/announcing-tensorflow-quantum-open.html
#quantum_computing #machine_learning #quantum_machine_learning

Crafting Papers on Machine Learning

This paper provides some useful hints and advice for preparing machine learning papers. Besides, consider that it is not meant to cover all types of papers.

https://icml.cc/Conferences/2002/craft.html
#machine_learning #writing

Model-based evolutionary algorithms: a short survey

Abstract: The evolutionary algorithms (EAs) are a family of nature-inspired algorithms widely used for solving complex optimization problems. Since the operators (e.g. crossover, mutation, selection) in most traditional EAs are developed on the basis of fixed heuristic rules or strategies, they are unable to learn the structures or properties of the problems to be optimized. To equip the EAs with learning abilities, recently, various model-based evolutionary algorithms (MBEAs) have been proposed. This survey briefly reviews some representative MBEAs by considering three different motivations of using models. First, the most commonly seen motivation of using models is to estimate the distribution of the candidate solutions. Second, in evolutionary multi-objective optimization, one motivation of using models is to build the inverse models from the objective space to the decision space. Third, when solving computationally expensive problems, models can be used as surrogates of the fitness functions. Based on the review, some further discussions are also given.

https://link.springer.com/article/10.1007/s40747-018-0080-1
#evolutionary_algorithm #machine_learning

At the Interface of Algebra and Statistics

Abstract: This thesis takes inspiration from quantum physics to investigate mathematical structure that lies at the interface of algebra and statistics. The starting point is a passage from classical probability theory to quantum probability theory. The quantum version of a probability distribution is a density operator, the quantum version of marginalizing is an operation called the partial trace, and the quantum version of a marginal probability distribution is a reduced density operator. Every joint probability distribution on a finite set can be modeled as a rank one density operator. By applying the partial trace, we obtain reduced density operators whose diagonals recover classical marginal probabilities. In general, these reduced densities will have rank higher than one, and their eigenvalues and eigenvectors will contain extra information that encodes subsystem interactions governed by statistics. We decode this information, and show it is akin to conditional probability, and then investigate the extent to which the eigenvectors capture "concepts" inherent in the original joint distribution. The theory is then illustrated with an experiment that exploits these ideas. Turning to a more theoretical application, we also discuss a preliminary framework for modeling entailment and concept hierarchy in natural language, namely, by representing expressions in the language as densities. Finally, initial inspiration for this thesis comes from formal concept analysis, which finds many striking parallels with the linear algebra. The parallels are not coincidental, and a common blueprint is found in category theory. We close with an exposition on free (co)completions and how the free-forgetful adjunctions in which they arise strongly suggest that in certain categorical contexts, the "fixed points" of a morphism with its adjoint encode interesting information.

Introductory Video: https://youtu.be/wiadG3ywJIs

Thesis: https://arxiv.org/abs/2004.05631

#statistics #machine_learning #algebra #quantum_physics

Tips for Publishing Research Code

This repository represents several important tips for releasing research code in Machine Learning (with official NeurIPS 2020 recommendations). These recommendations have been gathered based on analysis of more than 200 Machine Learning repositories, these recommendations facilitate reproducibility and correlate with GitHub stars - for more details, see our our blog post.

https://github.com/paperswithcode/releasing-research-code
#research_paper #machine_learning #neurIPS

The Notorious Difficulty of Comparing Human and Machine Perception

Abstract: With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed towards comparing information processing in humans and machines. These works have the potential to deepen our understanding of the inner mechanisms of human perception and to improve machine learning. Drawing robust conclusions from comparison studies, however, turns out to be difficult. Here, we highlight common shortcomings that can easily lead to fragile conclusions. First, if a model does achieve high performance on a task similar to humans, its decision-making process is not necessarily human-like. Moreover, further analyses can reveal differences. Second, the performance of neural networks is sensitive to training procedures and architectural details. Thus, generalizing conclusions from specific architectures is difficult. Finally, when comparing humans and machines, equivalent experimental settings are crucial in order to identify innate differences. Addressing these shortcomings alters or refines the conclusions of studies. We show that, despite their ability to solve closed-contour tasks, our neural networks use different decision-making strategies than humans. We further show that there is no fundamental difference between same-different and spatial tasks for common feed-forward neural networks and finally, that neural networks do experience a "recognition gap" on minimal recognizable images. All in all, care has to be taken to not impose our human systematic bias when comparing human and machine perception.

https://arxiv.org/abs/2004.09406
#machine_learning

TensorFlow Probability: Learning with confidence

TensorFlow Probability (TFP) is a Python library built on TensorFlow that makes it easy to combine probabilistic models and deep learning on modern hardware (TPU, GPU). It's for data scientists, statisticians, and ML researchers/practitioners who want to encode domain knowledge to understand data and make predictions with uncertainty estimates. In this talk we focus on the "layers" module and demonstrate how TFP "distributions" fit naturally with Keras to enable estimating aleatoric and/or epistemic uncertainty.

Website: https://www.tensorflow.org/probability

Introduction Video: https://www.youtube.com/watch?v=BrwKURU-wpk
#tensorflow #machine_learning

Mathematics for Machine Learning

Summary: The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.

https://mml-book.github.io/book/mml-book.pdf
#machine_learning #mathematics

Yann LeCuN advice for an undergraduate student who aspires to become a Machine Learning Scientist in the field of Deep Learning

(0) take all the continuous math and physics class you can possibly take. If you have the choice between “iOS programming” and “quantum mechanics”, take “quantum mechanics”. In any case, take Calc I, Calc II, Calc III, Linear Algebra, Probability and Statistics, and as many physics courses as you can. But make sure you learn to program.
(1) Take an AI-related problem you are passionate about.
(2) think about it on your own
(3) once you have formed your own idea of it, start reading the literature on the problem
(4) you will find that (a) your ideas were probably a bit naive but (b) your view of the problem is slightly different from what was done before.
(5) Find a professor in your school that can help you make your ideas concrete. It might be difficult. Professors are busy and don’t have much time for undergrads. The ones with the most free time are the very junior, the very senior, and the ones who are not very active in research.
(6) If you don’ find a professor with spare time, hook up with a postdoc or PhD student in his/her lab.
(7) ask the professor if you can attend his/her lab meetings and seminars or sit in his/her class.
(8) Before you graduate, try to write a paper about your research or release a piece of open source code.
(9) Now apply to PhD programs. Forget about the “ranking” of the school for now. Find a reputable professor who works on topics that you are interested in. Pick a person whose papers you like or admire.
(10) Apply to several PhD programs in the schools of the above-mentioned professors and mention in your letter that you’d like to work with that professor but would be open to work with others.
(11) ask your undergrad professor to write a recommendation letter for you. It’s maximally efficient if your undergrad professor is known by your favorite PhD advisor.
(12) if you don’t get accepted in one of your favorite PhD programs, get a job at Facebook or Google and try to get a gig as an engineer assisting research scientists at FAIR or Google Brain.
(13) publish a papers with the research scientists in question. Then re-apply to PhD programs and ask the FAIR or Google scientists you work with to write a recommendation letter for you.

https://www.quora.com/What%E2%80%99s-your-advice-for-undergraduate-student-who-aspires-to-be-a-research-scientist-in-deep-learning-or-related-field-one-day
#machine_learning

Machine Learning & Computational Statistics Course

Course Intro: This course covers a wide variety of topics in machine learning and statistical modeling. While mathematical methods and theoretical aspects will be covered, the primary goal is to provide students with the tools and principles needed to solve the data science problems found in practice.

https://davidrosenberg.github.io/ml2016/#home
#machine_learning #statistics #course