ECS 289G — Human-AI Teaming

Human-AI Teaming (HAT) describes integrated teams consisting of humans and AI agents working together as a unit. These teams harness the combined strengths of humans and AI to enhance performance in complex tasks. Three core topics this course will cover include: (1) Developing AI technologies that are reliable, understandable, and ethically responsible, in order to more effectively work with human users. (2) Creating interfaces and interaction protocols to facilitate effective communication and interaction between humans and AI systems. (3) Designing human-AI teaming systems for data-driven decision-making in real-world applications by combining humans’ creativity, emotional intelligence, and strategic thinking with AI’s computational power, pattern recognition, and analytics capabilities.

Course Learning Objectives

• Describe the core concepts and methods in HAT research
• Explain VIS roles and know how to use it can enhance HAT
• Identify the foundations and trends in HAT research & applications
• Develop creative and critical thinking through reading and discussing SOTA research papers
• Improve presentation skills for target users and different stakeholders in academia and industry
• Acquire rapid prototyping abilities for research demonstration

Lecture Information

Lecture Time: 1:10 - 2:00 PM Monday, Wednesday and Friday

Location: Olson Hall 147

Instructor: Dongyu Liu, 2123 Kemper

Instructor Office Hours: by appointment (location: Kemper 2123) [Zoom Link]

Teaching Assistant: Alex Dewey

Discussion Channels:

• Piazza (Canvas Integrated) We use Piazza for Q&A as it is more formal and well-organized. Our course staff will constantly check it. We encourage active participation by posting and answering questions on Piazza. If you're exceptionally helpful to other students, we'll consider you for extra credit.
• Campuswire (code: HIDE) We have created this platform for free discussion among students. Please follow our course logistics and make sure to respect with each other.

Textbook: n/a

Additional Tutorials: Check the Recourses Page

Grading Breakdown:

    Class attendance                                       10%

    Weekly reading notes                               24% (4% * 6)

    In-class presentation                                 34% (9% conf + 9% conf + 9% ted + 7% q&a)

    Final project                                               32% 

 

SCHEDULE

Welcome to our brand-new course! We're excited to embark on this learning journey with you. As we navigate through this course together, we ask for your patience and understanding. The instructor is dedicated to continually refining and tailoring the course structure to best meet your learning outcomes. 

Week 1: April 1 — Introduction I

Mon 4/1	Lecture: Introduction to Human-AI Teaming + Course Overview
Wed 4/3	Lecture: The Value of Visualization
Fri 4/5	Lecture: Visual Analytics for Decision-Making + Presentation Styles

Week 2: April 8 — Introduction II 

Mon 4/8	Lecture: Understanding Humans & Human-Centered Design TODO Sign up for your presentations in Week 3&4 Reading  Pick one paper from the Week 3 Reading List and write a summary (<= 300 words).
Wed 4/10	Lecture: Human-AI Interaction
Fri 4/12	Lecture: Trust, Transparency, and Explainability

Week 3: April 15 — Primer on HAI, HCI, and VIS [Reading list W3]

Mon 4/15	Conf Talk by [HIDE]: The design of everyday things (Chapter 1). Conf Talk by [HIDE]: The design of everyday things (Chapter 2). DUE  Reading W3 (please submit before 11:55am) Reading  Pick one paper from the Week 4 Reading List and write a summary (<= 300 words).
Wed 4/17	Conf Talk by [HIDE]: The science of visual data communication: What works (pages 1-15).  Conf Talk by [HIDE]. Views on visualization.
Fri 4/19	Conf Talk by [HIDE]: Interaction Design: beyond human-computer interaction (Chapter 1).  Conf Talk by [HIDE]. Human-centered AI: The role of Human-centered Design Research in the development of AI.

Week 4: April 22 — eXplainable AI [Reading list W4]

Mon 4/22	Conf Talk by [HIDE]: Sibyl: Understanding and addressing the usability challenges of machine learning in high-stakes decision making Ted Talk by [HIDE]: How AI-Powered Generative Images Are Transforming the Role of Graphic Designers DUE  Reading W4 (please submit before 11:55am) Reading  Pick one paper from the Week 5 Reading List and write a summary (<= 300 words). TODO Sign up for your presentations in Week 5&6
Wed 4/24	Conf Talk by [HIDE]: Feature visualization Lecture: Presentation Reflections
Fri 4/26	Conf Talk by [HIDE]: Rulematrix: Visualizing and understanding classifiers with rules Ted Talk by [HIDE]: AI and Agency for All: A Call for an Equitable Decision-Making System

Week 5: April 29 — Humans Using, Perceiving, and Interacting with AI  (Reading list W5)

Mon 4/29	Project Lecture: Planning, Tools, and Final Projects   DUE  Reading W5 (please submit before 11:55am) Reading  Pick one paper from the Week 6 Reading List and write a summary (<= 300 words).
Wed 5/1	Conf Talk by [HIDE]: Beyond accuracy: The role of mental models in human-AI team performance Ted Talk by [HIDE]: Ideas to Code
Fri 5/3	Conf Talk by [HIDE]: Will you accept an imperfect AI? exploring designs for adjusting end-user expectations of AI systems Ted Talk by [HIDE]: The evolution of creating 3D world from human and AI

Week 6: May 6    — Ethics and Fairness (Reading list W6)

Mon 5/6	Conf Talk by [HIDE]: My model is unfair, do people even care? visual design affects trust and perceived bias in machine learning Ted Talk by [HIDE]: Designing with Data AI-Driven Sustainable Design DUE  Reading W6 (please submit before 11:55am)
Wed 5/8	Ted Talk by [HIDE]: When morality meets autonomous vehicles Project Lecture: Planning, Tools, and Final Projects II
Fri 5/10	Ted Talk by [HIDE]: Fireflies.ai: Illuminating Collaboration in the Digital Age Ted Talk by [HIDE]: Collaborative Care: How Human-AI partnerships are transforming Healthcare

Week 7: May 13  — Advanced Research Topics 

Mon 5/13	Lecture: Human-Centered NLP Reading  Pick one paper from the Week 8 Reading List and write a summary (<= 300 words).
Wed 5/15	Lecture: Collaborating with LLMs
Fri 5/17	Lecture: GenAI for UI and VIS design

Week 8: May 20  — Evaluation (Reading list W8)

Mon 5/20	Lecture: Evaluating Interactive Systems I DUE  Reading W8 (please submit before 11:55am) Reading  Pick one paper from the Week 9 Reading List and write a summary (<= 300 words).
Wed 5/22	Lecture: Evaluating Interactive Systems II Ted Talk by [HIDE]: Self-Driving Cooperation
Fri 5/24	Conf Talk by [HIDE]: Vistext: A benchmark for semantically rich chart captioning Conf Talk by [HIDE]: What can AI do for me? evaluating machine learning interpretations in cooperative play

Week 9: May 27  — Generative AI (Reading list W9)

Mon 5/27	No Lectures & Presentations due to Memorial Day DUE  Reading W9 (please submit before 11:55am)
Wed 5/29	Conf Talk by [HIDE]: AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts Conf Talk by [HIDE]: LEVA: Using large language models to enhance visual analytics
Thu 5/31	Conf Talk by [HIDE]: Let the chart spark: Embedding semantic context into chart with text-to-image generative model Conf Talk by [HIDE]: Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

Week 10: June 3  — Final Week

Mon 6/3	Project  Video + Pitch Talk (5 minutes each student)
Wed 6/5	Project  Poster display, discussion, and brainstorming
Thu 6/6	Instruction Ends

Annotations to be used:

DUE  For marking any deadlines

Reading  For marking reading tasks

Project  For marking project-relevant tasks

TODO For marking any other todo tasks