research reading list

• Generative Spoken Language Modeling from Raw Audio – see 2021-09-10

• Ruder 2019 – The 4 Biggest Open Problems in NLP

• How can we improve peer review in NLP – 2020
• An Adversarial Review of “Adversarial Generation of Natural Language” – Yoav Goldberg blog post from 2017

• Marasović 2018 – NLP’s generalization problem, and how researchers are tackling it

• Optimal Neural Program Synthesis from Multimodal Specifications – 2020 paper from TAUR lab
• [course] Structured Prediction – Yoav Artzi
• Learning Differentiable Programs with Admissible Neural Heuristics – 2020 shah20_learn_differ_progr_with_admis_neural_heuris

• Formal Language Theory Meets Modern NLP – 2021

• Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer – 2021
• VideoBERT: A Joint Model for Video and Language Representation Learning – 2019

• [course] Language Grounding to Vision and Control – Katerina Fragkiadaki

• Conneau and Lample 2018 – Word Translation without Parallel Data

• LambdaNetworks: Modeling Long-Range Interactions Without Attention
• Attention is Not Explanation – 2019 jain19_atten_is_not_explan

• Recent Advances in Language Model Fine-tuning – 2021

-Leveraging Pre-trained Checkpoints for Sequence Generation Tasks

• Topological Planning with Transformers for Vision-and-Language Navigation – 2020
• Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training – 2021

• Compositional planning in Markov decision processes: Temporal abstraction meets generalized logic composition

• machine learning will not solve NLU – 2021
  • hn thread

• Sequence-to-Sequence Model for Trajectory Planning of Nonprehensile Manipulation Including Contact Model
• Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments – 2018

• Parr and Howard 2018 – The Matrix Calculus You Need for Deep Learning parr18_matrix_calcul_you_need_deep_learn
• The Modern Mathematics of Deep Learning
• Deep Learning Goodfellow and Bengio
• Math for Deep Learning – Faisal et al

• Backprop as Functor:A compositional perspective on supervised learning – 2021
• HaskTorch – Justin Le

• Learning Differentiable Programs with Admissible Neural Heuristics – 2020

• Machine Learning: The Great Stagnation – Mark Saroufim

• How to avoid machine learning pitfalls: a guide for academic researchers

• Generative Adversarial Transformers

• Formal reasoning about programs – Chlipala
• CS 4110 Cornell
• harvard programming languages
• Hoare Logic
• NEU compilers
• UPenn PL
• 6.820 MIT
• UPenn Haskell