Keynotes

abstract Anomaly mining finds high-stakes applications in various real-world domains such as cybersecurity, finance, environmental monitoring, to name a few. Therefore, it has been studied widely and a large body of detection techniques exists. Today, many real-world settings necessitate detection at speed for streaming/evolving data, and/or detection at scale for massive datasets stored in a distributed environment.

Despite the plethora of detection algorithms, selecting an algorithm to use on a new task as well as setting the values for its hyperparameter(s), known as the model selection problem, is an open challenge for unsupervised anomaly detection. This issue is only to be exacerbated with the recent advent of detectors based on deep neural networks that exhibit a long list of hyperparameters. The challenge stems from two main factors: the lack of labeled data and the lack of a widely accepted anomaly loss function. Toward automation, one can explore internal evaluation strategies, or capitalize on the experience from historical detection tasks through meta-learning. However, the problem remains far from solved.

In deployment, many real-world use cases of anomaly detection require the flagged anomalies from a detector to be screened or audited by a human expert, typically for vetting purposes, where taking automatic actions can be costly (e.g. directly charging a flagged medical provider with fraud). While a vast majority of the literature focuses on novel detection algorithms, as humans are often involved with(in) the process, anomaly mining also concerns various human-centric problems that are beyond mere detection, namely explanation, human interaction, and fairness. These aspects of the field are under-studied and pose many open challenges.

bio Leman Akoglu is the Heinz College Dean's Associate Professor. Her research broadly spans machine learning and data mining, and focuses on graph mining, pattern discovery and anomaly detection, with applications to fraud and event detection in many real-world domains. Dr. Akoglu is a recipient of the SDM/IBM Early Career Data Mining Research Award (2020), National Science Foundation CAREER Award (2015) and the US Army Research Office Young Investigator Award (2013). Her research has won several publication awards, most recently The Most Influential Paper (PAKDD 2020), Best Research Paper (SIAM SDM 2019), and Best Student Machine Learning Paper Runner-up (ECML PKDD 2018). Her research has been supported by the NSF, US ARO, DARPA, Adobe, Facebook, Northrop Grumman, PNC Bank, and PwC.

Leman Akoglu

Carnegie Mellon University

links

• Download abstract
• Personal website

abstract Computational Linguistics and Natural Language Processing have changed considerably in the past few decades. Early research focused on representing and using linguistic knowledge in computational processes such as parsers, while these days the field focuses on practically-useful tasks such as information retrieval and chatbots. Currently our Deep Learning models have little to do with linguistic theory.

For example, the Oracle Digital Assistant is built on top of generic 'Foundation' Deep Learning models. An intermediate Focusing step adapts these models to specific enterprise domains. Transfer Learning is used to refocus these models onto specific customer-oriented tasks such as Intent Classification, Named Entity Recognition, as well as more advanced models such as text-to-SQL sequence-to-sequence models. These technologies have revolutionised the application of NLP to practical problems with commercial relevance, enabling us to build better systems faster and cheaper than ever before.

Linguistic insights aren't gone from the field, however; they play a critical role in data manufacturing and evaluation. This talk explain how we use hundreds of different evaluations to understand the strengths and weaknesses of our models in the Oracle Digital Assistant, and how we automatically use this in hyper-parameter tuning. It also describes areas where additional research is still required before we can claim that NLP has become an engineering field.

bio Mark Johnson is Chief AI Scientist, Oracle Digital Assistant and a Professor of Language Sciences, Dept of Computing at Macquarie University in Sydney, Australia.

He has been active in Computational Linguistics and Natural Language Processing for decades. He was President of the Association for Computational Linguistics (ACL) and ACL’s SIGDAT, a Founding Fellow of the ACL, and an Editor-in-Chief for the Transactions of the ACL. His research spans topics such as syntactic parsing, semantic analysis and knowledge representation. He has over 200 publications across a wide range of NLP and AI topics. Most recently his research focuses on practical applications of Deep Learning to Conversational AI.

Mark Johnson

Oracle Corporation

links

• Download abstract
• Personal website

abstract Even though the term “knowledge graph” has become commonplace, there is still a fair bit of hype surrounding this technology. The potential usefulness of applications built on top of knowledge graphs is clear and they are, in fact, already commonly used to connect and link data silos in meaningful ways, as well as to represent lexical, structural, and symbolic information in machine-readable form. Using this representation, one can perform tasks such as inferencing, data validation, and link prediction in a wide range of domains, including the web, autonomous systems, finance, supply chain management, and biomedicine. Other well-known tasks include entity-based search and recommender systems, question answering, chatbots, and attribute/type ranking, all of which typically require data in a knowledge graph to be combined with some form of behavioral information, such as clicks, mentions, or purchases. Thus, it is clear that knowledge graphs are able to provide a plethora of possible functionalities. In this talk I will reflect on my hands-on experience using knowledge graphs in both industry and academia, explain what their strengths and weaknesses are, provide examples of how they create value, and dive into specific challenges and open issues.

bio Edgar Meij is the head of the Artificial Intelligence (AI) Discovery group in Bloomberg’s Engineering department. He leads several teams of researchers and engineers who develop systems that provide question answering capabilities, smart contextual suggestions with severe latency constraints. They also build and maintain the Bloomberg Knowledge Graph, together with its advanced machine learning-based analytics that is used to generate accurate, timely, and contextual financial insights. Edgar holds a PhD in computer science from the University of Amsterdam and has an extensive track record in information retrieval, natural language processing, and machine learning. Before joining Bloomberg, Edgar worked at Yahoo! Labs on all aspects related to entities in the context of web search.

Edgar Meij

Bloomberg

links

• Personal website

abstract As predictive models are increasingly being deployed in high-stakes decision making (e.g., loan approvals), there has been growing interest in developing post hoc techniques which provide recourse to individuals who have been adversely impacted by predicted outcomes. For example, when an individual is denied loan by a predictive model deployed by a bank, they should be informed about reasons for this decision and what can be done to reverse it. While several approaches have been proposed to tackle the problem of generating recourses, these techniques rely heavily on various restrictive assumptions. For instance, these techniques generate recourses under the assumption that the underlying predictive models do not change. In practice, however, models are often updated for a variety of reasons including data distribution shifts. There is little to no research that systematically investigates and addresses these limitations.

In this talk, I will discuss some of our recent work that sheds light on and addresses the aforementioned challenges, thereby paving the way for making algorithmic recourse practicable and reliable. First, I will present theoretical and empirical results which demonstrate that the recourses generated by state-of-the-art approaches are often invalidated due to model updates. Next, I will introduce a novel algorithmic framework based on adversarial training to generate recourses that remain valid even if the underlying models are updated. I will conclude the talk by presenting theoretical and empirical evidence for the efficacy of our solutions, and also discussing other open problems in the burgeoning field of algorithmic recourse.

bio Himabindu (Hima) Lakkaraju is an assistant professor at Harvard University focusing on explainability, fairness, and robustness of machine learning models. She has also been working with various domain experts in policy and healthcare to understand the real-world implications of explainable and fair ML. Hima has been named as one of the world’s top innovators under 35 by both MIT Tech Review and Vanity Fair. Her research has also received best paper awards at SIAM International Conference on Data Mining (SDM) and INFORMS, and grants from NSF, Google, Amazon, and Bayer. Hima has given keynote talks at various workshops at ICML, NeurIPS, AAAI, and CVPR, and her research has also been showcased by popular media outlets including the New York Times, MIT Tech Review, TIME magazine, and Forbes. More recently, she co-founded the Trustworthy ML Initiative to enable easy access to resources on trustworthy ML and to build a community of researchers/practitioners working on the topic.

Himabindu Lakkaraju

Harvard University

links

• Download abstract
• Personal website