Publications

Please see Google Scholar for the most updated publication list.

2023

EEBO-Verse: Sifting for Poetry in Large Early Modern Corpora Using Visual Features
Danlu Chen, Nan Jiang, and Taylor Berg-Kirkpatrick
International Conference on Document Analysis and Recognition (ICDAR)
paper

Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction
Keren Shao, Ke Chen, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
arXiv preprint arXiv:2308.02723
paper code

MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
Ke Chen, Yusong Wu, Haohe Liu, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
arXiv preprint arXiv:2308.01546
paper demo code

CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models
Hao-Wen Dong, Xiaoyu Liu, Jordi Pons, Gautam Bhattacharya, Santiago Pascual, Joan Serrà, Taylor Berg-Kirkpatrick, and Julian McAuley
IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)
paper demo

Who Rpinted Shakespeare’s Fourth Folio?
Christopher N Warren, Samuel V Lemley, DJ Schuldt, Elizabeth Dieterich, Laura S DeLuca, Max G’Sell, Taylor Berg-Kirkpatrick, Kari Thomas, Kartik Goyal, and Nikolai Vogler
Shakespeare Quarterly, quad021
paper

Multitrack Music Transformer: Learning Long-Term Dependencies in Music with Diverse Instruments
Hao-Wen Dong, Ke Chen, Shlomo Dubnov, Julian McAuley, and Taylor Berg-Kirkpatrick
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper demo code

Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation
Yusong Wu*, Ke Chen*, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper code

Membership Inference Attacks against Language Models via Neighbourhood Comparison
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: ACL
paper

Text Conditional Alt-Text Generation for Twitter Images
Nikita Srivatsan, Sofia Samaniego, Omar Florez, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2305.14779
paper

Smaller Language Models are Better Black-box Machine-Generated Text Detectors
Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2305.09859
paper

Universal Source Separation with Weakly Labelled Data
Qiuqiang Kong, Ke Chen, Haohe Liu, Xingjian Du, Taylor Berg-Kirkpatrick, Shlomo Dubnov, and Mark D. Plumbley
arXiv preprint arXiv:2305.07447
paper code

CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji, Julian McAuley, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper demo code

Jointly modeling products and resource pages for task-oriented recommendation
Brendan Duncan, Surya Kallumadi, Taylor Berg-Kirkpatrick, and Julian McAuley
World Wide Web (WWW), Industry Track
paper

Contrastive Attention Networks for Attribution of Early Modern Print
Nikolai Vogler, Kartik Goyal, Kishore PV Reddy, Elizaveta Pertseva, Samuel V. Lemley, Christopher N. Warren, Max G’Sell, and Taylor Berg-Kirkpatrick
AAAI Conference on Artificial Intelligence (AAAI)
paper code

ClimaBench: A Benchmark Dataset For Climate Change Text Understanding in English
Tanmay Laud*, Daniel Spokoyny*, Tom Corringham, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2301.04253
paper code

2022

An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David K Evans, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

An Empirical Analysis of Memorization in Fine-tuned Autoregressive Language Models
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

HeLo: Learning-Free Lookahead Decoding for Conversation Infilling
Ivan Lee and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: EMNLP
paper

Prompt Consistency for Zero-Shot Task Generalization
Chunting Zhou*, Junxian He*, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig
Findings of the Association for Computational Linguistics: EMNLP
paper

Checklist Models for Improved Output Fluency in Piano Fingering Prediction
Nikita Srivatsan and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper

Improving Choral Music Separation through Expressive Synthesized Data from Sampled Instruments
Ke Chen, Hao-Wen Dong, Yi Luo, Julian McAuley, Taylor Berg-Kirkpatrick, Miller Puckette, and Shlomo Dubnov
International Society for Music Information Retrieval Conference (ISMIR)
paper demo code

Non-Parametric Temporal Adaptation for Social Media Topic Classification
Fatemehsadat Mireshghallah, Nikolai Vogler, Junxian He, Omar Florez, Ahmed El-Kishky, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2209.05706
paper

UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis
Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, and Dimitrios Dimitriadis
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper

Lacuna Reconstruction: Self-supervised Pre-training for Low-Resource Historical Document Transcription
Nikolai Vogler, Jonathan Parkes Allen, Matthew Thomas Miller, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: NAACL-HLT
paper

Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Context
Daniel Spokoyny, Ivan Lee, Zhao Jin, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: NAACL-HLT
paper

HOLM: Hallucinating Objects with Language Models for Referring Expression Recognition in Partially-Observed Scenes
Volkan Cirik, Louis-Philippe Morency, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
Fatemehsadat Mireshghallah, Kartik Goyal, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Deep Performer: Score-to-Audio Music Performance Synthesis
Hao-Wen Dong, Cong Zhou, Taylor Berg-Kirkpatrick, and Julian McAuley
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper

HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper code

TONet: Tone-Octave Network for Singing Melody Extraction from Polyphonic Music
Ke Chen, Shuai Yu, Cheng-i Wang, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
paper code

Towards a Unified View of Parameter-Efficient Transfer Learning
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig
International Conference on Learning Representations (ICLR)
paper

Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis–Hastings
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper

Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data
Ke Chen*, Xingjian Du*, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
AAAI Conference on Artificial Intelligence (AAAI)
paper code

An Unsupervised Masking Objective for Abstractive Multi-Document News Summarization
Nikolai Vogler, Songlin Li, Yujie Xu, Yujian Mi, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2201.02321
paper

2021

Investigating Robustness of Dialog Models to Popular Figurative Language Constructs
Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Truth-Conditional Captions for Time Series Data
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Scalable Font Reconstruction with Dual Latent Manifolds
Nikita Srivatsan, Si Wu, Jonathan T. Barron, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Style Pooling: Automatic Text Style Obfuscation for Improved Classification Fairness
Fatemehsadat Mireshghallah and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Efficient Nearest Neighbor Language Models
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

An Empirical Evaluation of End-to-End Polyphonic Optical Music Recognition
Sachinda Edirisooriya, Hao-Wen Dong, Julian McAuley, and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper code

Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music
Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick, and Julian McAuley
International Society for Music Information Retrieval Conference (ISMIR)
paper demo code

Neural Representation Learning for Scribal Hands of Linear B
Nikita Srivatsan, Jason Vega, Christina Skelton, and Taylor Berg-Kirkpatrick
ICDAR Workshop on Computational Paleography
paper

Comparative Error Analysis in Neural and Finite-state Models for Unsupervised Character-level Transduction
Maria Ryskina, Eduard Hovy, Taylor Berg-Kirkpatrick, and Matthew R. Gormley
SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology
paper

Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation
Varun Gangal, Harsh Jhamtani, Eduard Hovy, and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: ACL-IJCNLP
paper

Unsupervised Enrichment of Persona-grounded Dialog with Background Stories
Bodhisattwa Prasad Majumder, Taylor Berg-Kirkpatrick, Julian McAuley, and Harsh Jhamtani
Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing (ACL-IJCNLP)
paper

Privacy Regularization: Joint Privacy-Utility Optimization in Language Models
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, and Robert Sim
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper

Paraphrastic Representations at Scale
John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2104.15114
paper

An Empirical Study of Extrapolation in Text Generation with Scalar Control
Aashi Jain and Taylor Berg-Kirkpatrick
arXiv preprint arXiv:2104.07910
paper

2020

Learning Sparse Prototypes for Text Generation
Junxian He, Taylor Berg-Kirkpatrick, and Graham Neubig
International Conference on Neural Information Processing Systems (NeurIPS)
paper

Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

An Empirical Investigation of Contextualized Number Prediction
Daniel Spokoyny and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

A Bilingual Generative Transformer for Semantic Sentence Embedding
John Wieting, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Narrative Text Generation with a Latent Discrete Plan
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Findings of the Association for Computational Linguistics: EMNLP
paper

Discovering Music Relations with Sequential Attention
Junyan Jiang, Gus Xia, and Taylor Berg-Kirkpatrick
Workshop on NLP for Music and Audio (NLP4MusA)
paper

MusPy: A Toolkit for Symbolic Music Generation
Hao-Wen Dong, Ke Chen, Julian McAuley, and Taylor Berg-Kirkpatrick
International Society for Music Information Retrieval Conference (ISMIR)
paper demo code

Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm
Ke Chen, Cheng-i Wang, Taylor Berg-Kirkpatrick, and Shlomo Dubnov
International Society for Music Information Retrieval Conference (ISMIR)
paper

Refer360°: A Referring Expression Recognition Dataset in 360° Images
Volkan Cirik, Taylor Berg-Kirkpatrick, and Louis-Philippe Morency
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

A Probabilistic Generative Model for Typographical Analysis of Early Modern Printing
Kartik Goyal, Chris Dyer, Christopher Warren, Max G’Sell, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Phonetic and Visual Priors for Decipherment of Informal Romanization
Maria Ryskina, Matthew R. Gormley, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

A Probabilistic Formulation of Unsupervised Text Style Transfer
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper

Domain Adaptation via Context Prediction for Engineering Diagram Search
Harsh Jhamtani and Taylor Berg-Kirkpatrick
European Conference on Information Retrieval (ECIR)
paper

Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods
Maria Ryskina, Ella Rabinovich, Taylor Berg-Kirkpatrick, David R. Mortensen, and Yulia Tsvetkov
Proceedings of the Society for Computation in Linguistics (SCiL)
paper

2019

A Deep Factorization of Style and Structure in Fonts
Nikita Srivatsan, Jonathan T. Barron, Dan Klein, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper

Learning Rhyming Constraints using Structured Adversaries
Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper

A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
Bohan Li, Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick, and Yiming Yang
Conference on Empirical Methods in Natural Language Processing and International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
paper

Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections
Junxian He, Zhisong Zhang, Taylor Berg-Kirkpatrick, and Graham Neubig
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Simple and Effective Paraphrastic Similarity from Parallel Translations
John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Beyond BLEU: Training Neural Machine Translation with Semantic Similarity
John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, and Graham Neubig
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Modeling Self-Repetition in Music Generation using Generative Adversarial Networks
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Machine Learning for Music Discovery Workshop (ML4MD)
paper

An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Berg-Kirkpatrick
International Conference on Learning Representations (ICLR)
paper

2018

Unsupervised Text Style Transfer using Language Models as Discriminators
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, and Taylor Berg-Kirkpatrick
International Conference on Neural Information Processing Systems (NeurIPS)
paper

Speaker-Follower Models for Vision-and-Language Navigation
Daniel Fried*, Ronghang Hu*, Volkan Cirik*, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein**, and Trevor Darrell**
International Conference on Neural Information Processing Systems (NeurIPS)
paper

Modeling Online Discourse with Coupled Distributed Topics
Nikita Srivatsan, Zachary Wojtowicz, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Learning to Describe Differences Between Pairs of Similar Images
Harsh Jhamtani and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Unsupervised Learning of Syntactic Structure with Invertible Neural Projections
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick
Conference on Empirical Methods in Natural Language Processing (EMNLP)
paper

Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data
Harsh Jhamtani*, Varun Gangal*, Eduard Hovy, Graham Neubig, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Visual Referring Expression Recognition: What Do Systems Actually Learn?
Volkan Cirik, Louis-Philippe Morency, and Taylor Berg-Kirkpatrick
Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)
paper

A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
Kartik Goyal, Graham Neubig, Chris Dyer, and Taylor Berg-Kirkpatrick
AAAI Conference on Artificial Intelligence (AAAI)
paper

Using Syntax to Ground Referring Expressions in Natural Images
Volkan Cirik, Taylor Berg-Kirkpatrick, and Louis-Philippe Morency
AAAI Conference on Artificial Intelligence (AAAI)
paper

SPINE: SParse Interpretable Neural Embeddings
Anant Subramanian*, Danish Pruthi*, Harsh Jhamtani*, Taylor Berg-Kirkpatrick, and Eduard Hovy
AAAI Conference on Artificial Intelligence (AAAI)
paper

2017

Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick
International Conference on Machine Learning (ICML)
paper

Efficient Correlated Topic Modeling with Topic Embedding
Junxian He*, Zhiting Hu*, Taylor Berg-Kirkpatrick, Ying Huang, and Eric P. Xing.
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)
paper

Automatic Compositor Attribution in the First Folio of Shakespeare
Maria Ryskina, Hannah Alpert-Abrams, Dan Garrette, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Differentiable Scheduled Sampling for Credit Assignment
Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

2016

Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints
Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein
Annual Meeting of the Association for Computational Linguistics (ACL)
paper

Using accelerometers to remotely and automatically characterize behavior in small animals
Talisin T. Hammond, Dwight Springthorpe, Rachel E. Walsh, and Taylor Berg-Kirkpatrick.
Journal of Experimental Biology, 219(11)
paper