• Home
• Papers
• Tutorials
• Teaching
• Team
• CV

Trevor Cohn

Associate Professor
ARC Future Fellow

Computing and Information Systems
The University of Melbourne
Language Technology and Knowledge Discovery research groups

Google Scholar

DBLP

Understanding of human language by computers has been a central goal of Artificial Intelligence since its beginnings, with massive potential to improve communication, provide better information access and automate basic human tasks. My research focuses on technologies for automatic processing of human language, with several applications including automatic translation (akin to Google and Bing's translation tools). My core focus is on probabilistic machine learning modelling of language applications, particularly handling uncertain or partly observed data and structured prediction problems.

News

• Melbourne will host ACL in July 2018, at the Melbourne Convention Centre, with Tim Baldwin, Karin Verspoor and myself serving as the local chairs.
• Co-organising the ALTA 2016 workshop to be held in Monash Caulfield campus in Melbourne, on the 5th - 7th of December 2016.
• I'll be giving a tutorial on succinct data structures for NLP with Matthias Petri at COLING 2016 in Osaka, Japan on 12th December, 2016.
• My student Oliver Adams won the Best Short Paper Award at EMNLP 2016! My group also has several other papers appearing at the conference and a TACL presentation (listed below).

Current Projects

• Efficient storage and access to text count data: An application to unlimited order language modelling. 2016 – 2017. Google Research Award, $US 85k.
• Learning Deep Semantics for Automatic Translation between Human Languages. 2016 – 2019. ARC Discovery with Reza Haffari, $450k.
• Ariel: Analysis of Rare Incident-Event Languages. 2015 – 2018. DARPA LORELEI (sub-contract), $300k.
• Adaptive Context-Dependent Machine Translation for Heterogeneous Text. 2014 – 2018. ARC Future Fellowship, $730k.
• Pheme: Computing Veracity Across Media, Languages, and Social Networks. 2014 – 2017. EU FP7 with Kalina Bontcheva and others, £494k.

Selected Papers

Continuous Representation of Location for Geolocation and Lexical Dialectology using Mixture Density Networks
Afshin Rahimi, Trevor Cohn and Timothy Baldwin. In Proceedings of EMNLP, 2017.

Learning how to Active Learn: A Deep Reinforcement Learning Approach
Meng Fang, Yuan Li and Trevor Cohn. In Proceedings of EMNLP, 2017.

Towards Decoding as Continuous Optimisation in Neural Machine Translation
Cong Duy Vu Hoang, Gholamreza Haffari and Trevor Cohn. In Proceedings of EMNLP, 2017.
Abstract

×

Towards Decoding as Continuous Optimisation in Neural Machine Translation

We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. The resulting optimisation problem is then tackled using constrained gradient optimisation. Our powerful decoding framework, enables decoding intractable models such as the intersection of left-to-right and right-to-left (bidirectional) as well as source-to-target and target-to-source (bilingual) NMT models. Our empirical results show that our decoding framework is effective, and leads to substantial improvements in translations generated from the intersected models where the typical greedy or beam search is infeasible.

PDF

Sequencing bias in Crowd-sourced annotation for NLP
Nitika Mathur, Timothy Baldwin and Trevor Cohn. In Proceedings of EMNLP, 2017.

Modelling the Working Week for Multi-Step Forecasting using Gaussian Process Regression
Pasan Karunaratne, Shanika Karunasekera, Masud Moshtaghi, Aaron Harwood and Trevor Cohn. In Proceedings of IJCAI, 2017.

Compressed Nonparametric Language Modelling
Ehsan Shareghi, Trevor Cohn and Gholamreza Haffari. In Proceedings of IJCAI, 2017.

Topically Driven Neural Language Model
Jey Han Lau, Timothy Baldwin and Trevor Cohn. In Proceedings of ACL, 2017.

Model Transfer for Tagging Low-resource Languages without Bilingual Corpora
Meng Fang and Trevor Cohn. In Proceedings of ACL, 2017.

A Neural Model for User Geolocation and Lexical Dialectology
Afshin Rahimi, Trevor Cohn and Timothy Baldwin. In Proceedings of ACL, 2017.

DyNet: The Dynamic Neural Network Toolkit
Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, Kevin Duh, Manaal Faruqui, Cynthia Gan, Dan Garrette, Yangfeng Ji, Lingpeng Kong, Adhiguna Kuncoro, Gaurav Kumar, Chaitanya Malaviya, Paul Michel, Yusuke Oda, Matthew Richardson, Naomi Saphra, Swabha Swayamdipta, Pengcheng Yin. In arXiv preprint, 2017.
Abstract

×

DyNet: The Dynamic Neural Network Toolkit

We describe DyNet, a toolkit for implementing neural network models based on dynamic declaration of network structure. In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives. In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input. Dynamic declaration thus facilitates the implementation of more complicated network architectures, and DyNet is specifically designed to allow users to implement their models in a way that is idiomatic in their preferred programming language (C++ or Python). One challenge with dynamic declaration is that because the symbolic computation graph is defined anew for every training example, its construction must have low overhead. To achieve this, DyNet has an optimized C++ backend and lightweight graph representation. Experiments show that DyNet's speeds are faster than or comparable with static declaration toolkits, and significantly faster than Chainer, another dynamic declaration toolkit.

PDF Code

Context-Aware Prediction of Derivational Word-forms
Ekaterina Vylomova, Ryan Cotterell, Trevor Cohn and Timothy Baldwin. In Proceedings of EACL (short), 2017.

Robust Training under Linguistic Adversity
Yitong Li, Trevor Cohn and Timothy Baldwin. In Proceedings of EACL (short), 2017.

Pairwise Webpage Coreference Classification using Distant Supervision
Shivashankar Subramanian, Timothy Baldwin, Julian Brooke and Trevor Cohn. In Proceedings of WWW (posters), 2017.

Cross-Lingual Word Embeddings for Low-Resource Language Modeling
Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird and Trevor Cohn. In Proceedings of EACL, 2017.
Abstract

×

Cross-Lingual Word Embeddings for Low-Resource Language Modeling

Most languages have no formal writing system and at best a limited written record. However, textual data is critical to natural language processing and particularly important for the training of language models that would facilitate speech recognition of such languages. Bilingual phonetic dictionaries are often available in some form, since lexicon creation is a fundamental task of documentary linguistics. We investigate the use of such dictionaries to improve language models when textual training data is limited to as few as 1k sentences. The method involves learning cross-lingual word embeddings as a pre-training step in the training of monolingual language models. Results across a number of languages show that language models are improved by such pre-training.

PDF

Multilingual Training of Crosslingual Word Embeddings
Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird and Trevor Cohn. In Proceedings of EACL, 2017.
Abstract

×

Multilingual Training of Crosslingual Word Embeddings

Crosslingual word embeddings represent lexical items from different languages using the same vector space, enabling crosslingual transfer. Most prior work constructs embeddings for a pair of languages, with English on one side. We investigate methods for building high quality crosslingual word embeddings for many languages in a unified vector space. In this way, we can exploit and combine strength of many languages. We obtained high performance on bilingual lexicon induction, monolingual similarity and crosslingual document classification tasks.

PDF

Learning a Lexicon and Translation Model from Phoneme Lattices
Oliver Adams, Graham Neubig, Trevor Cohn, Steven Bird, Quoc Truong Do and Satoshi Nakamura. In Proceedings of EMNLP (short papers), 2016.
Winner of Best Short Paper Award
Abstract

×

Learning a Lexicon and Translation Model from Phoneme Lattices

Language documentation begins by gathering speech. Manual or automatic transcription at the word level is typically not possible because of the absence of an orthography or prior lexicon, and though manual phonemic transcription is possible, it is very slow. On the other hand, translations of the minority language into a major language are more easily acquired. We propose a method to harness such translations to improve automatic phoneme recognition. The method assumes no prior lexicon or translation model, instead learning them from phoneme lattices and translations of the speech being transcribed. Experiments demonstrate phoneme error rate improvements against two baselines and the model's ability to learn useful bilingual lexical entries.

PDF