Prof. Alistair Knott - School of Engineering and Computer Science, Victoria University of Wellington

How can computational linguists contribute to the social good?

Like most AI language researchers these days, I divide my time between being amazed by the quality of our new large language models, and worrying about how these models will impact society. In an attempt to reconcile these two instincts, my work for the last few years has focussed on contributing to conversations about LLM regulation, where people who understand the technology at issue have many important roles to play.

In this talk I’ll discuss a few ongoing conversations I’m involved in - and hopefully connect with people at ALTA who are having similar conversations. One discussion is about transparency for AI-generated content. My group at the Global Partnership on AI helped to define a clause in the EU’s AI Act, requiring providers of AI content generators to ensure their content is ‘detectable’. A key debate at present is about enforcement: what can we require of providers, given the state-of-the-art in AI content detection schemes? Another discussion is about transparency of social media platforms. The EU’s Digital Services Act contains provisions allowing vetted external researchers to access the largest platforms, to study ‘societal risks’ (and their mitigation). Now these provisions are in place, what are the most important questions that should be asked, and what methods are most suitable? (I co-founded a grouping of social data science researchers, to help answer that question.) A final discussion is about accountability for AI training sets. I am particularly interested in the labelled datasets created by Silicon Valley companies, to align their LLMs, and to train their harmful content classifiers. A great deal of power resides with the annotators who provide labels for these datasets. But we often know little about how annotators are chosen, and about the guidelines they follow. My group at GPAI has proposed that labels could be provided ‘democratically’, by consulting a representative sample of citizens or users, through a process akin to opinion polling. I’ll finish by discussing how this idea could feature in the governance of ’sovereign LLMs’ developed in Australia and New Zealand.

Bio

Ali Knott is Professor of AI at Victoria University of Wellington. He studied Philosophy and Psychology at Oxford, and did his postgrad and postdoc work at in Edinburgh, in computational linguistics. He moved to New Zealand in 1998, and participated in ALTA during its heady early years.

Ali spent many years researching how language is implemented in the human brain. His main interest was in how language interfaces to the sensory and motor systems, to enable us to talk about what we see and do. He presented his theory about this interface in a book published by MIT Press. He extended the theory in the New Zealand AI company Soul Machines. One application was an embodied model of an 18-month-old toddler, ‘BabyX’.

For the last 10 years, Ali’s main focus has been on the social impacts and governance of AI. He is involved in many international AI policy discussions, mainly through his work as co-lead of a project on Social Media Governance, coordinated by the Global Partnership on AI (now part of the OECD). This work has had several impacts on EU tech legislation, including on provisions about AI content detection (in the AI Act), and provisions to allow vetted researchers access to the largest online platforms (in the Digital Services Act). Ali also participated in the Christchurch Call to eliminate Terrorist and Violent Extremist Content Online, and contributes to the Forum for Information and Democracy. He has advised the New Zealand government on many questions of AI policy, and often looks across the ditch to see how policy is progressing in Australia.

Prof. Mark Johnson - School of Computing, Macquarie University and Chief AI Scientist, Oracle

Mark Johnson keynote

How LLMs Change NLP

LLMs have made large-scale industrial application of NLP easier than ever before. This talk surveys the strengths and weaknesses of LLMs as qualitative rather than quantitative models, and focuses on the critical role of training data and post-training. Rather than focusing on superficial linguistic form, NLP development now involves understanding information sources and crafting the information flow between agents. Evaluation and testing are emerging as key challenges for LLM-powered applications.

Bio

Mark Johnson is a Professor of Language Science (CORE) in the School of Computing at Macquarie University. He is also the Chief AI Scientist, Oracle Digital Assistant at Oracle Corporation, where he develops chatbots and digital assistants. The Oracle Digital Assistant division develops novel deep learning models to power the next generation of Conversational AI using semantic parsing.

Mark Johnson has worked on a wide range of topics in computational linguistics, but his main area of research is natural language understanding, especially syntactic parsing and semantic analysis, and their applications to text and speech processing.

Panel Discussion: “Teaching NLP & Using NLP for teaching”

Moderator: Aditya Joshi

Panel Members:

  • Massimo Piccardi, Professor, University of Technology Sydney
  • Jey Han Lau, Senior Lecturer, University of Melbourne
  • Thomas Elton, PhD Student, The University of Sydney