Zhuyun Dai | Towards Deeper Language Understanding in Large-Scale Information Retrieval

Information retrieval (IR) systems face large document collections and diverse language patterns. It brings challenges for using advanced NLP techniques in retrieval.  In this talk, I will share our efforts in using deep neural networks to improve language understanding in IR.  First, I will share our work in applying Transformers to generate context-aware bag-of-words document representations. These context-aware representations significantly boost retrieval accuracy while the simple bag-of-words ensures efficiency. Next, I will share our recent explorations in complementing bag-of-words with machine-learned embeddings, and show how our method is used in a zero-shot setting to help people access COVID-19 articles.

Zhuyun Dai is a sixth-year Ph.D. student at LTI, advised by Jamie Callan. Her research is at the intersection of information retrieval, deep learning, and natural language processing.  She received a BS degree in computer science from Peking University, and an MS degree from LTI, CMU.

Qizhe Xie | Self-training with Noisy Student improves ImageNet classification

We present Noisy Student Training, a semi-supervised learning approach that works well even when labeled data is abundant. Noisy Student Training achieves 88.4% top-1 accuracy on ImageNet, which is 2.0% better than the state-of-the-art model that requires 3.5B weakly labeled Instagram images. On robustness test sets, it improves ImageNet-A top-1 accuracy from 61.0% to 83.7%, reduces ImageNet-C mean corruption error from 45.7 to 28.3, and reduces ImageNet-P mean flip rate from 27.8 to 12.2. Noisy Student Training extends the idea of self-training and distillation with the use of equal-or-larger student models and noise added to the student during learning. On ImageNet, we first train an EfficientNet model on labeled images and use it as a teacher to generate pseudo labels for 300M unlabeled images. We then train a larger EfficientNet as a student model on the combination of labeled and pseudo labeled images. We iterate this process by putting back the student as the teacher. During the learning of the student, we inject noise such as dropout, stochastic depth, and data augmentation via RandAugment to the student so that the student generalizes better than the teacher. 

Qizhe Xie is a PhD student at Carnegie Mellon University, advised by Eduard Hovy. His research interests include Deep Learning, Natural Language Processing and Computer Vision. He worked at Google Brain as a student researcher for two years under the guidance of Quoc Le (The inventor of Seq2Seq and AutoML).

Junjie Hu | XTREME: A Massively Multilingual Multi-task Benchmarkfor Evaluating Cross-lingual Generalization

Much recent progress in applications of machine learning models to NLP has been driven by benchmarks that evaluate models across a wide variety of tasks. However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing. To this end, we introduce the Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME) benchmark, a multi-task benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that while models tested on English reach human performance on many tasks, there is still a sizable gap in the performance of cross-lingually transferred models, particularly on syntactic and sentence retrieval tasks. There is also a wide spread of results across languages. We release the benchmark at https://sites.research.google/xtreme. The codes used for downloading data and training baseline models are available at https://github.com/google-research/xtreme to encourage research on cross-lingual learning methods that transfer linguistic knowledge across a diverse and representative set of languages and tasks.

Junjie Hu is a PhD student in Language Technologies Institute, at Carnegie Mellon University (CMU), working with the late Jaime Carbonell and Graham Neubig. His research focuses on indirect supervision for natural language generation and understanding over diverse domains, languages, and modalities. In particular, he works on developing learning algorithms for machine translation, cross-lingual adaptation, and multi-modal language generation.

Xuezhe Ma | FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flows

Most sequence-to-sequence (seq2seq) models are autoregressive; they generate each token by conditioning on previously generated tokens. In contrast, non-autoregressive seq2seq models generate all tokens in one pass, which leads to increased efficiency through parallel processing on hardware such as GPUs. However, directly modeling the joint distribution of all tokens simultaneously is challenging, and even with increasingly complex model structures accuracy lags significantly behind autoregressive models. In this paper, we propose a simple, efficient, and effective model for non-autoregressive sequence generation using latent variable models. Specifically, we turn to generative flow, an elegant technique to model complex distributions using neural networks, and design several layers of flow tailored for modeling the conditional density of sequential latent variables. We evaluate this model on three neural machine translation (NMT) benchmark datasets, achieving comparable performance with state-of-the-art non-autoregressive NMT models and almost constant decoding time w.r.t the sequence length.

Xuezhe Ma recently received his PhD degree from Language Technologies Institute at Carnegie Mellon University, where he worked with Eduard Hovy. Before that, he received his B.E and M.S from Shanghai Jiao Tong University. His research interests fall in areas of natural language processing and machine learning, particularly in deep learning and representation learning with applications to linguistic structured prediction and deep generative models. Xuezhe has interned at Allen Institute for Artificial Intelligence (AI2) and earned the AI2 Outstanding Intern award. His research has been recognized with outstanding paper award at ACL 2016 and best demo paper nomination at ACL 2019.

Haohan Wang | High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks

We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN). We first notice CNN's ability in capturing the high-frequency components of images. These high-frequency components are almost imperceptible to a human. Thus the observation leads to multiple hypotheses that are related to the generalization behaviors of CNN, including a potential explanation for adversarial examples, a discussion of CNN's trade-off between robustness and accuracy, and some evidence in understanding training heuristics.

Haohan Wang is a PhD student at LTI, CMU, working with Professor Eric P. Xing; his research interest centers around the robustness issues of machine learning, covering a spectrum of applications including computer vision, NLP, and computational biology.

Shrimai Prabhumoye | Controllable Text Generation: Should machines reflect the way humans interact in society?

The 21st century is witnessing a major shift in the way people interact with technology and Natural Language Generation (NLG) is playing a central role. Users of smartphones and smart home devices now expect their gadgets to be aware of their situation, and to produce natural language outputs in interactions. ‘We identify three aspects of human communication to make machines sound human-like - style, content and structure. This talk provides deep learning solutions to controlling these variables in neural text generation. This talk first outlines the various modules which could be manipulated to perform effective controllable text generation. It also provides a novel solution for style transfer using back-translation. Furthermore, it introduces two new tasks to leverage information from unstructured documents into the dialogue response generation process and Wikipedia edit generation process. At the end, it provides a discussion on the ethical considerations of the applications of controllable text generation.

Shrimai Prabhumoye is a PhD student at Carnegie Mellon University working with Prof. Alan W Black and Prof. Ruslan Salakhutdinov. Her thesis focuses on controlling style, content and structure in text generation and the ethical considerations of this technology. She was also the co-designer of the Computational Ethics for NLP course offered at CMU. During her masters she was leading the CMU Magnus team in the Alexaprize competition in 2017.

Paul Michel | Weight Poisoning Attacks on Pre-trained Models

Recently, NLP has seen a surge in the usage of large pre-trained models. Users download weights of models pre-trained on large datasets, then fine-tune the weights on a task of their choice. This raises the question of whether downloading untrusted pre-trained weights can pose a security threat. In this paper, we show that it is possible to construct ``weight poisoning'' attacks where pre-trained weights are injected with vulnerabilities that expose ``backdoors'' after fine-tuning, enabling the attacker to manipulate the model prediction simply by injecting an arbitrary keyword. We show that by applying a regularization method, which we call RIPPLe, and an initialization procedure, which we call Embedding Surgery, such attacks are possible even with limited knowledge of the dataset and fine-tuning procedure. Our experiments on sentiment classification, toxicity detection, and spam detection show that this attack is widely applicable and poses a serious threat. Finally, we outline practical defenses against such attacks.

Paul Michel is a 4th year PhD student in the Language Technologies Institute at Carnegie Mellon University, advised by Graham Neubig. His research interests are in NLP and machine learning, with a focus on learning in the face of distributional shift.

Junwei Liang | The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction

In this talk, I will present our latest work at CVPR 2020, multi-future trajectory prediction. In this work, we study the problem of predicting the distribution over multiple possible future paths of people as they move through various visual scenes. We make two main contributions. The first contribution is a new dataset, created in a realistic 3D simulator, which is based on real world trajectory data, and then extrapolated by human annotators to achieve different latent goals. This provides the first benchmark for quantitative evaluation of the models to predict multi-future trajectories. Pedestrian trajectory prediction has been an important research direction for traffic safety and self-driving applications. Project page: https://next.cs.cmu.edu/multiverse/index.html

Junwei Liang is a third-year Ph.D. student at LTI, advised by Prof. Alexander Hauptmann. He received the master of language technologies degree from LTI in 2017. His research interests include large-scale computer vision, natural language processing and machine learning. His recent work on person future prediction attracts wide attention on Github and from Chinese media. His work on video event reconstruction and shooter localization have received much media coverage including from CBS.

Hira Dhamyal | Detecting Gender Differences in Perception of Emotion in Crowdsourced Data

Do men and women perceive emotions differently? Popular convictions place women as more emotionally perceptive than men. Empirical findings, however, remain inconclusive. Most prior studies focus on visual modalities. In addition, almost all of the studies are limited to experiments within controlled environments. Generalizability and scalability of these studies has not been sufficiently established. We study the differences in perception of emotion between genders from speech data in the wild, annotated through crowdsourcing. While we limit ourselves to a single modality (i.e. speech),our framework is applicable to studies of emotion perception from all such loosely annotated data in general. We addresses multiple serious challenges related to making statistically viable conclusions from crowdsourced data. Overall we aim to establish a reliable novel framework for perceptual studies from crowdsourced data and secondly we aim to study statistically the differences in speech-based emotion perception between genders.

Hira Dhamyal is 2nd year Master student at the Language Technologies Institute, Carnegie Mellon University. Her research interests includes multiple topics in speech and has worked on topics including speaker verification, emotion identification and emotion expression and perception.