Ruder 2019 – The 4 Biggest Open Problems in NLP
1. Natural language understanding
1.1. program synthesis
- underlying NLU are the logical form representations of language
- let's try predicting them directly
1.2. embodied intelligence
- with massive compute, could place agent in the real world and try to get it to learn language from the ground up
- in the meantime, can ground in simulated environment
1.3. inductive bias
- related to the above, what sort of priors should we build into the model
1.5. cog sci
- build approaches inspired by brain science
2. NLP for low resource languages
2.1. universal language model
2.2. cross lingual representations
- aligning word embeddings for different languages
3. reasoning about large/many documents
- what does unsupervised language modelling look like for very long documents?
- how do life long learning and memory work for language models
4. datasets and evaluation
Created: 2024-07-15 Mon 01:26