Publications
2023
Multitrack Music Transformer: Learning Long-Term Dependencies in Music with Diverse Instruments
Hao-Wen Dong, Ke Chen, Shlomo Dubnov, Julian McAuley, and Taylor Berg-Kirkpatrick
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper
code
demo
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
Yusong Wu*, Ke Chen*, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper
code
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji, Julian McAuley, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper
code
demo
ClimaBench: A Benchmark Dataset For Climate Change Text Understanding in English
Tanmay Laud*, Daniel Spokoyny*, Tom Corringham, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2301.04253
paper
2022
An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David K Evans, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
HeLo: Learning-Free Lookahead Decoding for Conversation Infilling
Ivan Lee and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: EMNLP
paper
Prompt Consistency for Zero-Shot Task Generalization
Chunting Zhou*, Junxian He*, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig
Findings of the Association for Computational Linguistics: EMNLP
paper
Checklist Models for Improved Output Fluency in Piano Fingering Prediction
Nikita Srivatsan and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper
Improving Choral Music Separation through Expressive Synthesized Data from Sampled Instruments
Ke Chen, Hao-Wen Dong, Yi Luo, Julian McAuley, Taylor Berg-Kirkpatrick, Miller Puckette, and Shlomo Dubnov
International Society for Music Information Retrieval Conference (ISMIR)
paper
code
demo
Non-Parametric Temporal Adaptation for Social Media Topic Classification
Fatemehsadat Mireshghallah, Nikolai Vogler, Junxian He, Omar Florez, Ahmed El-Kishky, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2209.05706
paper
UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis
Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, and Dimitrios Dimitriadis
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper
Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription
Nikolai Vogler, Jonathan Parkes Allen, Matthew Thomas Miller, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: NAACL-HLT
paper
Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context
Daniel Spokoyny, Ivan Lee, Zhao Jin, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: NAACL-HLT
paper
HOLM: Hallucinating Objects with Language Models for Referring Expression Recognition in Partially-Observed Scenes
Volkan Cirik, Louis-Philippe Morency, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Deep Performer: Score-to-Audio Music Performance Synthesis
Hao-Wen Dong, Cong Zhou, Taylor Berg-Kirkpatrick, and Julian McAuley
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper
code
TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music
Ke Chen, Shuai Yu, Cheng-i Wang, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper
code
Towards a Unified View of Parameter-Efficient Transfer Learning
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig
International Conference on Learning Representations (ICLR)
paper
Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis–Hastings
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper
Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data
Ke Chen*, Xingjian Du*, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
AAAI Conference on Artificial Intelligence (AAAI)
paper
code
An Unsupervised Masking Objective for Abstractive Multi-Document News Summarization
Nikolai Vogler, Songlin Li, Yujie Xu, Yujian Mi, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2201.02321
paper
2021
Investigating Robustness of Dialog Models to Popular Figurative Language Constructs
Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Truth-Conditional Captions for Time Series Data
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Scalable Font Reconstruction with Dual Latent Manifolds
Nikita Srivatsan, Si Wu, Jonathan T. Barron, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
Fatemehsadat Mireshghallah and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Efficient Nearest Neighbor Language Models
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
An Empirical Evaluation of End-to-End Polyphonic Optical Music Recognition
Sachinda Edirisooriya, Hao-Wen Dong, Julian McAuley, and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper
code
Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music
Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick, and Julian McAuley
International Society for Music Information Retrieval Conference (ISMIR)
paper
code
demo
Neural Representation Learning for Scribal Hands of Linear B
Nikita Srivatsan, Jason Vega, Christina Skelton, and Taylor Berg-Kirkpatrick
ICDAR Workshop on Computational Paleography
paper
Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction
Maria Ryskina, Eduard Hovy, Taylor Berg-Kirkpatrick, and Matthew R. Gormley
SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
paper
Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation
Varun Gangal, Harsh Jhamtani, Eduard Hovy, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: ACL-IJCNLP
paper
Unsupervised Enrichment of Persona-grounded Dialog with Background Stories
Bodhisattwa Prasad Majumder, Taylor Berg-Kirkpatrick, Julian McAuley, and Harsh Jhamtani
Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP)
paper
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, and Robert Sim
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper
Paraphrastic Representations at Scale
John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2104.15114
paper
An Empirical Study of Extrapolation in Text Generation with Scalar Control
Aashi Jain and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2104.07910
paper
2020
Learning Sparse Prototypes for Text Generation
Junxian He, Taylor Berg-Kirkpatrick, and Graham Neubig
International Conference on Neural Information Processing Systems (NeurIPS)
paper
Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
An Empirical Investigation of Contextualized Number Prediction
Daniel Spokoyny and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
A Bilingual Generative Transformer for Semantic Sentence Embedding
John Wieting, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Narrative Text Generation with a Latent Discrete Plan
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: EMNLP
paper
Discovering Music Relations with Sequential Attention
Junyan Jiang, Gus Xia, and Taylor Berg-Kirkpatrick
Workshop on NLP for Music and Audio (NLP4MusA)
paper
MusPy: A Toolkit for Symbolic Music Generation
Hao-Wen Dong, Ke Chen, Julian McAuley, and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper
code
demo
Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm
Ke Chen, Cheng-i Wang, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
International Society for Music Information Retrieval Conference (ISMIR)
paper
Refer360°: A Referring Expression Recognition Dataset in 360° Images
Volkan Cirik, Taylor Berg-Kirkpatrick, and Louis-Philippe Morency
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
A Probabilistic Generative Model for Typographical Analysis of Early Modern Printing
Kartik Goyal, Chris Dyer, Christopher Warren, Max G’Sell, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Phonetic and Visual Priors for Decipherment of Informal Romanization
Maria Ryskina, Matthew R. Gormley, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
A Probabilistic Formulation of Unsupervised Text Style Transfer
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper
Domain Adaptation via Context Prediction for Engineering Diagram Search
Harsh Jhamtani and Taylor Berg-Kirkpatrick
European Conference on Information Retrieval (ECIR)
paper
Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods
Maria Ryskina, Ella Rabinovich, Taylor Berg-Kirkpatrick, David R. Mortensen, and Yulia Tsvetkov
Proceedings of the Society for Computation in Linguistics (SCiL)
paper
2019
A Deep Factorization of Style and Structure in Fonts
Nikita Srivatsan, Jonathan T. Barron, Dan Klein, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper
Learning Rhyming Constraints using Structured Adversaries
Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
Bohan Li, Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick, and Yiming Yang
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections
Junxian He, Zhisong Zhang, Taylor Berg-Kirkpatrick, and Graham Neubig
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Simple and Effective Paraphrastic Similarity from Parallel Translations
John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Beyond BLEU: Training Neural Machine Translation with Semantic Similarity
John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, and Graham Neubig
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Modeling Self-Repetition in Music Generation using Generative Adversarial Networks
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Machine Learning for Music Discovery Workshop (ML4MD)
paper
An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper
2018
Unsupervised Text Style Transfer using Language Models as Discriminators
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick
International Conference on Neural Information Processing Systems (NeurIPS)
paper
Speaker-Follower Models for Vision-and-Language Navigation
Daniel Fried*, Ronghang Hu*, Volkan Cirik*, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein**, and Trevor Darrell**
International Conference on Neural Information Processing Systems (NeurIPS)
paper
Modeling Online Discourse with Coupled Distributed Topics
Nikita Srivatsan, Zachary Wojtowicz, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Learning to Describe Differences Between Pairs of Similar Images
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Unsupervised Learning of Syntactic Structure with Invertible Neural Projections
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper
Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data
Harsh Jhamtani*, Varun Gangal*, Eduard Hovy, Graham Neubig, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Visual Referring Expression Recognition: What Do Systems Actually Learn?
Volkan Cirik, Louis-Philippe Morency, and Taylor Berg-Kirkpatrick
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper
A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
Kartik Goyal, Graham Neubig, Chris Dyer, and Taylor Berg-Kirkpatrick
AAAI Conference on Artificial Intelligence (AAAI)
paper
Using Syntax to Ground Referring Expressions in Natural Images
Volkan Cirik, Taylor Berg-Kirkpatrick, and Louis-Philippe Morency
AAAI Conference on Artificial Intelligence (AAAI)
paper
SPINE: SParse Interpretable Neural Embeddings
Anant Subramanian*, Danish Pruthi*, Harsh Jhamtani*, Taylor Berg-Kirkpatrick, and Eduard Hovy
AAAI Conference on Artificial Intelligence (AAAI)
paper
2017
Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick
International Conference on Machine Learning (ICML)
paper
Efficient Correlated Topic Modeling with Topic Embedding
Junxian He*, Zhiting Hu*, Taylor Berg-Kirkpatrick, Ying Huang, and Eric P. Xing.
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)
paper
Automatic Compositor Attribution in the First Folio of Shakespeare
Maria Ryskina, Hannah Alpert-Abrams, Dan Garrette, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Differentiable Scheduled Sampling for Credit Assignment
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
2016
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints
Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein
Annual Meeting of the Association for Computational Linguistics (ACL)
paper
Using accelerometers to remotely and automatically characterize behavior in small animals
Talisin T. Hammond, Dwight Springthorpe, Rachel E. Walsh, and Taylor Berg-Kirkpatrick.
Journal of Experimental Biology, 219(11)
paper