Marvins' Memos
A curated collection of essential AI papers for audio learners. Listen on Apple Podcasts and Spotify.
Listen on Apple Podcasts:
Listen on Spotify:
Season 1 : 30 Essential AI Papers
“If you really learn all of these, you’ll know 90% of what matters today” - Ilya Sutskever.
The First Law of Complexodynamics
Audio : (Spotify)
Paper: https://scottaaronson.blog/?p=762
The Unreasonable Effectiveness of Recurrent Neural Networks
Audio : (Spotify)
Paper: https://karpathy.github.io/2015/05/21/rnn-effectiveness/
Understanding LSTM Networks
Audio : (Spotify)
Paper: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
Recurrent Neural Network Regularization
Audio : (Spotify)
Paper: https://arxiv.org/abs/1409.2329v5
Keeping Neural Networks Simple by Minimizing the Description Length of the Weights
Audio : (Spotify)
Paper: https://www.cs.toronto.edu/~hinton/absps/colt93.pdf
Pointer Networks
Audio : (Spotify)
Paper: https://arxiv.org/abs/1506.03134v2
ImageNet Classification with Deep Convolutional Neural Networks
Audio : (Spotify)
Paper: https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
Order Matters: Sequence to Sequence for Sets
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1511.06391
GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism
Audio : (Spotify)
Paper: https://arxiv.org/abs/1811.06965
Deep Residual Learning for Image Recognition
Audio : (Spotify)
Paper: https://arxiv.org/abs/1512.03385
Multi-Scale Context Aggregation by Dilated Convolutions
Audio : (Spotify)
Paper: https://arxiv.org/abs/1511.07122
Neural Message Passing for Quantum Chemistry
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1704.01212
Attention is All You Need
Audio : (Spotify)
Paper: https://arxiv.org/abs/1706.03762
Neural Machine Translation by Jointly Learning to Align and Translate
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1409.0473
Identity Mappings in Deep Residual Networks
Audio : (Spotify)
Paper: https://arxiv.org/abs/1603.05027
A Simple Neural Network Module for Relational Reasoning
Audio : (Spotify)
Paper: https://arxiv.org/abs/1706.01427
Variational Lossy Autoencoder
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1611.02731
Relational Recurrent Neural Networks
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1806.01822
Quantifying the Rise and Fall of Complexity in Closed Systems: the Coffee Automaton
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1405.6903
Neural Turing Machines
Audio : (Spotify)
Paper: https://arxiv.org/abs/1410.5401
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Audio : (Spotify)
Paper: https://arxiv.org/pdf/1512.02595
Scaling Laws for Neural Language Models
Audio : (Spotify)
Paper: https://arxiv.org/abs/2001.08361
A Tutorial Introduction to the Minimum Description Length Principle
Audio : (Spotify)
Paper: https://arxiv.org/pdf/math/0406077
Machine Super Intelligence
Audio : (Spotify)
Paper: https://www.vetta.org/documents/Machine_Super_Intelligence.pdf
Kolmogorov Complexity and Algorithmic Randomness
Audio : (Spotify)
Paper: https://www.lirmm.fr/~ashen/kolmbook-eng-scan.pdf
Better & Faster Large Language Models Via Multi-token Prediction
Audio : (Spotify)
Paper: https://arxiv.org/pdf/2404.19737
Dense Passage Retrieval for Open-Domain Question Answering
Audio : (Spotify)
Paper: https://arxiv.org/pdf/2004.04906.pdf
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Audio : (Spotify)
Paper: https://arxiv.org/abs/2005.11401v4
Zephyr: Direct Distillation of LM Alignment
Audio : (Spotify)
Paper: https://arxiv.org/pdf/2310.16944.pdf
Lost in the Middle: How Language Models Use Long Contexts
Audio : (Spotify)
Paper: https://arxiv.org/abs/2307.03172
Top 30 Papers Round Up
Audio : (Spotify)
Paper: https://aman.ai/primers/ai/top-30-papers/#ilya-sutskevers-top-30-reading-list