Overview
In just the past couple of years, we have seen significant advances in the capabilities of (Large) Language Models. One of the most striking capabilities of these systems is knowledge retrieval — Language Models can answer a diverse set of questions, which differ substantially in the domain knowledge needed for their responses, and their input structure. The precise methods for knowledge retrieval vary from the language model directly generating a response (parametric approaches) to a combination of generation and referencing an external knowledge corpus, e.g. retrieval augmented generation, to primarily using an external knowledge corpus with language model embeddings (semi-parametric approaches.)
Despite the rapid advances, there remain many pressing open questions on the limits of knowledge retrieval with language models, and connections between these different approaches. How factual are generated responses, and how does this vary with question complexity, model scale, and importantly, different methods of knowledge retrieval? How important is the role of (self-supervised/supervised) pretraining? What are the tradeoffs between few-shot (prompt based) approaches and finetuning when adapting to novel domains? And relatedly, to what extent do different knowledge retrieval approaches generalize to unseen settings?
This workshop seeks to bring together a diverse set of researchers across NLP, Machine Learning and Theory to discuss these questions. We hope to share current findings and challenges, identify promising directions for future study, and most importantly, build a community around this topic at this pivotal time.
Call for Papers
We welcome all submissions related to knowledge retrieval and language models, including:
- Methods for grounding language model responses.
- Open-domain retrieval and question answering.
- Improving and evaluating factuality.
- Bias and fairness in knowledge retrieval.
- Empirical studies of scale, transfer and generalization.
- Analysis of knowledge retrieval and language model phenomena.
Submission URL: openreview.net/group?id=ICML.cc/2022/Workshop/KRLM
Format: All submissions must be in PDF format. Submissions are limited to four content pages, including all figures and tables; unlimited additional pages containing references and supplementary materials are allowed. Reviewers may choose to read the supplementary materials but will not be required to. Camera-ready versions may go up to five content pages.
Style file: You must format your submission using the provided LaTeX style file, which is adapted from the official ICML 2022 LaTeX style file. Please include the references and supplementary materials in the same PDF as the main paper. The maximum file size for submissions is 50MB. Submissions that violate the style (e.g., by decreasing margins or font sizes) or page limits may be rejected without further review.
Dual-submission policy: We welcome ongoing and unpublished work. We also welcome papers that are under review at the time of submission, or that have been recently accepted.
Submissions and accepted papers: Workshop submissions and reviews will be private. The camera-ready version of accepted papers will be shared on the workshop webpage. However, the hosting is not an official proceeding, so the papers can be subsequently / concurrently submitted to other venues.
In person presentation: Accepted papers are expected to be presented in person.
Important Dates
- Submission deadline: May 27, 2022 AOE
- Notification to authors: June 13, 2022
- Camera-ready deadline: July 8th, 2022 AOE
Schedule
| Time | Event | Speaker | |
|---|---|---|---|
| 8:45 am - 9:00 am | Opening Remarks | Workshop Organizers | |
| 9:00 am - 9:30 am | Invited Talk 1 | Nils Reimers | |
| 9:30 am - 10:00 am | Invited Talk 2 | Sebastian Riedel | |
| 10:00 am - 10:30 am | Break | ||
| 10:30 am - 11:00 am | Invited Talk 3 | John Schulman | |
| 11:00 am - 12:00 pm | Poster Session | ||
| 12:00 pm - 1:30 pm | Lunch Break | ||
| 1:30 pm - 2:00 pm | Invited Talk 4 | Jimmy Lin | |
| 2:00 pm - 2:30 pm | Invited Talk 5 | Ellie Pavlick | |
| 2:30 pm - 3:00 pm | Contributed Talks |
1. Dialog Inpainting: Turning Documents into Dialogs 2. Huge Frozen Language Models as Readers for Open-Domain Question Answering 3. LinkBERT: Pretraining Language Models with Document Links |
|
| 3:00 pm - 3:30 pm | Break | ||
| 3:30 pm - 4:00 pm | Invited Talk 6 | Danqi Chen | |
| 4:00 pm - 4:30 pm | Invited Talk 7 | Quoc Le | |
| 4:30 pm - 5:30 pm | Panel Discussion | Panelists: Kenton Lee, Danqi Chen, Fernando Diaz, Stella Biderman Moderator: Sasha Rush |
|
| 5:30 pm - 5:40 pm | Closing Remarks | Workshop Organizers |
Accepted Papers
- Michihiro Yasunaga, Jure Leskovec, Percy Liang. LinkBERT: Pretraining Language Models with Document Links. Contributed Talk.
- Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, Yusuke Iwasawa. Large Language Models are Zero-Shot Reasoners.
- Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Mike Green, Qazi Mamunur Rashid, Kelvin Guu. Dialog Inpainting: Turning Documents into Dialogs. Contributed Talk.
- Yoav Levine, Ori Ram, Daniel Jannai, Barak Lenz, Shai Shalev-Shwartz, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham. Huge Frozen Language Models as Readers for Open-Domain Question Answering. Contributed Talk.
- Jie Huang, Hanyin Shao, Kevin Chang. Are Large Pre-Trained Language Models Leaking Your Personal Information?
- Disha Shrivastava, Hugo Larochelle, Daniel Tarlow. Repository-Level Prompt Generation for Large Language Models of Code. Poster
- Man Luo, Mihir Parmar, Jayasurya Sevalur Mahendran, Sahit Jain, Samarth Rawal, Chitta Baral. SCONER: Scoring Negative Candidates Before Training Neural Re-Ranker For Question Answering. Poster Video
- Sebastian Hofstätter, Jiecao Chen, Karthik Raman, Hamed Zamani. Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling.
- Gyuwan Kim, Jinhyuk Lee, Barlas Oguz, Wenhan Xiong, Yizhe Zhang, Yashar Mehdad, William Yang Wang. Bridging the Training-Inference Gap for Dense Phrase Retrieval. Poster
- Minki Kang, Jin Myung Kwak, Jinheon Baek, Sung Ju Hwang. Knowledge-Consistent Dialogue Generation with Knowledge Graphs.
- Ahmad Pouramini. Matching pre-training and Fine-tuning Methods for Knowledge Retrieval from pretrained Language Models.
- Uri Alon, Frank F. Xu, Junxian He, Sudipta Sengupta, Dan Roth, Graham Neubig. Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval. Poster Video
- Weijia Shi, Julian Michael, Suchin Gururangan, Luke Zettlemoyer. Nearest Neighbor Zero-Shot Inference.
