Schedule

Weeks 1

Lecture: Foundations and Introduction to CUI

This foundational lecture traces the evolution of conversational interfaces from early experiments to contemporary AI systems. Students will explore the historical journey from ELIZA (1966) through ALICE, to modern assistants like Siri, Alexa, and ChatGPT. The session distinguishes between task-oriented systems (goal-driven, domain-specific) and open-domain systems (general conversation), examining their architectural differences and appropriate use cases.

The lecture examines CUI applications across multiple industries, with particular focus on Conversational Recommender Systems that understand user preferences through dialogue and provide explainable recommendations, Conversational Search featuring mixed-initiative interactions and contextual query refinement, and CUI for Healthcare including patient intake systems, medication reminders, and mental health support with critical privacy considerations. Students will also explore Educational CUIs such as intelligent tutoring systems and adaptive language learning companions, as well as Enterprise CUIs for customer service automation and internal knowledge management. Throughout the course, positions CUI within the broader HCI landscape, examining relationships with GUI, VUI, and emerging multimodal interfaces.

Week 2

Lecture: Design Principles, Communication Theory, and Information Architecture for CUI

This lecture establishes theoretical and practical foundations for effective conversation design by integrating communication theory with actionable design principles. Central to this foundation is Grice's Maxims of Conversation, which provide essential guidelines for cooperative communication. Students learn to apply the Maxim of Quantity (providing appropriate amounts of information), the Maxim of Quality (ensuring truthfulness and evidence-based responses), the Maxim of Relation (maintaining relevance to user goals), and the Maxim of Manner (being clear, brief, and orderly) to conversational interface design.

The lecture explores turn-taking mechanisms, including adjacency pairs, transition relevance places, and overlap management, essential for creating natural conversational flows. Students examine repair strategies for conversational breakdowns, learning to distinguish between self-initiated self-repair and other-initiated repair, implement progressive clarification techniques, and design graceful degradation strategies when conversations go off track. These communication principles directly inform the information architecture for conversation, including conversation state management (local versus global context), intent hierarchies and taxonomies, entity recognition and slot-filling strategies, and methods for preserving context across multiple conversational turns.

Design principles for effective CUI build upon this theoretical foundation, addressing personality and tone consistency, progressive disclosure in conversation, the balance between error prevention and error recovery, and cognitive load management throughout dialogue interactions. Students learn how these principles manifest differently in text versus voice interfaces and how to adapt their designs accordingly.

Critique 1: Good and bad of CUI design

Week 3

Lecture: Large Language Models in CUI

This lecture explores the transformative impact of Large Language Models on conversational interface design and implementation. The session is structured in three parts, each addressing crucial aspects of LLM integration in CUI development.

Part I: Building LLM-based CUI examines how LLMs fundamentally reshape conversational interface capabilities. Students explore LLM-powered voice assistants and the unique interaction patterns that emerge when combining speech interfaces with language models, including opportunities for natural conversation, deep context understanding, and dynamic personalization. The lecture addresses critical challenges including latency management, response consistency, and hallucination prevention. 

Part II: Building CUI with LLM Support shifts focus to leveraging LLMs as design tools. Students learn the ProtoChat methodology, which combines crowd feedback with LLM generation for rapid prototype development, enabling quick iteration based on user testing while maintaining conversation consistency. The lecture covers expert-sourced knowledge integration through Retrieval-Augmented Generation (RAG), teaching students to combine LLM flexibility with verified expert knowledge, implement proper citation mechanisms, and balance generative capabilities with factual accuracy requirements.

Part III: Responsible Use of LLMs in CUI addresses ethical and practical considerations essential for deployment. Students examine public health and safety implications in sensitive domains, learning to identify benefits and risks, implement bias detection and mitigation strategies, and build user trust through transparency. The lecture emphasizes verifiability and factual accuracy, teaching students to evaluate generative responses, implement fact-checking mechanisms, balance creativity with accuracy, and follow ethical guidelines for responsible deployment.

Tutorial: CUI Foundations with LLM Integration

Students apply communication theory to real-world conversational design challenges through hands-on exercises. The module begins with the analysis of provided chat transcripts, where students identify violations of Grice’s Maxims. Based on these insights, they then redesign the dialogues to better align with the Cooperative Principle, aiming to balance the four maxims in a contextually appropriate way.

Following this redesign, students implement LLM-driven conversational interfaces to realize their improved dialogues. Starting with the basic integration of the OpenAI API, students build a functioning dialogue loop that reflects their revised conversation flows. They also experiment with system parameters such as temperature and prompt engineering to explore how different settings impact conversational quality, coherence, and user experience.

Assignment 1: Released CUI Design & Prototype

Week 4

Lecture: Multimodal and Multi-agent CUI

This lecture explores advanced CUI architectures incorporating multiple interaction modalities and agent collaboration.

The multimodal integration section establishes principles for combining interaction channels effectively. Students learn to distinguish modal equivalence from modal synergy, understand modality switching triggers, and apply accessibility-first design. Technical patterns include voice-visual combinations (Alexa Show, Google Nest Hub), text-gesture interactions in AR, and haptic feedback integration. Adaptive modality selection covers context-aware switching, user preference learning, and environmental adaptation for factors like noise and privacy.

The multi-agent systems portion introduces coordination architectures for conversational agents. Students examine centralized versus decentralized approaches, role-based agent design, and inter-agent communication protocols. Collaborative problem-solving covers task decomposition, agent specialization, and conflict resolution. The lecture addresses presenting multiple agents coherently, managing user attention across agents, and establishing trust in multi-agent responses.

Tutorial: Exploring Multimodal Design Patterns

Students complete a simplified design exercise using the provided design templates (available on course website). They sketch three screens for a campus coffee shop assistant: an opening screen with both voice prompt and menu buttons, a response screen showing drink options after voice command "Show me cold drinks", and an error screen for unrecognized voice input. Students use Google's Material Design voice interaction guidelines (material.io/design/sound/about-sound.html - open access) and Microsoft's Fluent Design System documentation (microsoft.github.io/fluent-ui-web-components - open source) as reference materials. This structured exercise introduces multimodal concepts through observation and basic sketching.

Week 5

Lecture: User Research and Prototyping for CUI

This combined lecture addresses the complete lifecycle from research through prototype development, emphasizing rapid iteration for conversational interfaces.

User research methods adapt traditional HCI approaches for conversation's temporal nature. Wizard of Oz methodology teaches protocol design balancing realism with research goals and meaningful insight extraction without full implementation. Longitudinal studies include diary study adaptations, conversation log analysis, and temporal pattern identification. Contextual inquiry covers in-situ observation, environmental factor documentation, and multi-stakeholder perspective gathering.

Prototyping strategies progress from low to high fidelity. Students learn paper prototyping for conversation flows, storyboarding for interaction narratives, and script-based testing. Digital tools include Voiceflow for voice/chat design, Botmock for multi-platform flows, and Figma for high-fidelity interfaces. Progressive enhancement guides movement from scripts to rules to AI, with iterative complexity addition and validation at each level. The lecture emphasizes the research-to-prototype pipeline: translating findings to design decisions, rapid iteration cycles, and stage-appropriate validation strategies.

Tutorial: Figma Prototyping Workshop

This hands-on workshop guides students through creating high-fidelity conversational prototypes in Figma. Students will design chat interface components (message bubbles, input fields, system indicators), create interactive conversation flows with multiple branches, incorporate multimodal elements identified in Week 3, and demonstrate error handling and help flows.

Week 6

Lecture: CUI Implementation I – Structure, Logic, and Frameworks

This technical lecture introduces the architectural components and frameworks for building conversational interfaces. Students learn about core CUI components (NLU pipeline, dialogue manager, response generator, context store), comparison of rule-based vs. statistical vs. hybrid approaches, popular frameworks (Dialogflow, Microsoft Bot Framework, BotPress, Voiceflow), and web technologies for chat interfaces (WebSocket, REST APIs, event-driven architecture). The session provides criteria for selecting appropriate tools based on project requirements and introduces basic implementation patterns that will be expanded in Week 7.

Tutorial: Implementation Workshop – Setting Up Your CUI

This practical workshop helps students translate their prototypes into working code. Students will set up a basic web-based chat interface (HTML/CSS/JavaScript), implement simple pattern matching for intent recognition, create a dialogue management structure using state machines or decision trees, and integrate mock responses and basic API calls. The session emphasizes code organization, debugging strategies, and preparing a foundation for advanced features to be added in Week 7.

Week 7

Lecture: CUI Implementation II – Conversational Agents

This lecture deepens the implementation of intelligent conversational agents by examining how generative models, autonomous agents, and memory architectures can be combined to create more lifelike and adaptive CUIs. Building on earlier discussions of task-oriented systems, this session explores how recent advancements (e.g., Generative Agents, AutoGPT, and memory-augmented LLM frameworks) enable conversational interfaces to exhibit long-term planning, reflective behaviors, and context persistence.

The lecture begins with an introduction to Generative Agents, drawing on the influential 2023 simulation of AI personas in a sandbox town environment. Students will explore how these agents integrate symbolic memory, reactive dialogue, and narrative coherence to simulate believable social behavior. We will discuss the architectural components—planning modules, memory retrieval, and LLM-driven behavior generation—and reflect on what this means for the boundaries between conversation, cognition, and character.

Next, the session introduces the concept of autonomous conversational agents through systems like AutoGPT, and LangChain-based agents. These tools chain together LLM capabilities with external tools, iterative goal decomposition, and memory buffers to create systems that can pursue open-ended tasks across multiple dialogue turns or user sessions. Students will analyze how such agents blend instruction following, tool use, and reflection, and what new possibilities they enable for domains such as customer service, tutoring, and companionship.

Tutorial: Iterative Development and Mid-Project Demo

In the tutorial session, students will extend their chatbot projects by incorporating advanced agent features such as dynamic memory, self-reflection, or tool use. In preparation for the mid-project demo, students will define a minimal working scenario—such as a multi-step goal or simulated social interaction—and test how their system handles information recall, user adaptation, or evolving intent. Peer feedback will focus on the coherence of agent behavior, the design of memory flows, and the expressive flexibility of the conversation. Instructors will guide students in scoping their ambitions realistically while experimenting with cutting-edge agent capabilities.

Week 8

Lecture: System-Level Testing of CUIs

This lecture focuses on how to test, debug, and evaluate conversational systems from a technical standpoint, emphasizing both the predictability and resilience of CUIs in real-world contexts. The session begins with a conceptual overview of system-level robustness, introducing techniques for fallback design based on confidence thresholds, ambiguity detection, and intent disambiguation. Students will learn how conversational agents can recover from errors using clarification loops or repair sub-dialogues, and how these mechanisms contribute to perceived system intelligence.

The lecture introduces a range of testing frameworks and tools, including Rasa's story-driven test suite, Dialogflow's session debugger, and Botium’s cross-platform bot testing environment. These tools enable automated testing of dialogue paths, intent accuracy, and edge-case coverage. For generative systems, the session explores LangChain’s agent tracing tools, OpenAI function call debugging, and methods for inspecting memory flows across dialogue turns. Students are encouraged to think about testability as a design principle, considering how predictable architecture supports iterative development. Throughout, testing is framed not only as a technical process but as a foundation for building user trust and deployment readiness.

Tutorial: Debugging and Test Log Review

In the tutorial, students will apply these testing strategies to their own chatbot prototypes. Working with real or simulated input, they will identify failure cases in intent detection, dialogue transitions, and memory recall. Students will use logs and test suites to inspect how their systems handle ambiguity, user interruptions, and off-script queries. Tutorials will introduce structured error taxonomies to help categorize failure types and inform design revisions. By the end of the session, teams will generate a test report summarizing key weaknesses, implemented fixes, and next steps for improving robustness.

Assignment 2: Released CUI Implementation, Evaluation & Ethical Reflection

Week 9

Lecture: Evaluating CUIs through User Studies

This lecture explores how to evaluate conversational user interfaces through empirical user studies, emphasizing a mixed-methods approach that blends qualitative insight with quantitative rigor. The session introduces usability testing methodologies such as think-aloud protocols, cognitive walkthroughs, and A/B comparisons, guiding students to select appropriate techniques based on the maturity and purpose of their system. Drawing from HCI evaluation literature and UX research, students will learn how to design test tasks, recruit participants, and define meaningful success criteria.

Students will explore performance metrics relevant to dialogue systems, including task success rate, average number of turns, help requests, user satisfaction, and perceived naturalness. The session also addresses the challenges of evaluating generative and multi-agent CUIs, where responses are often non-deterministic and interaction patterns vary across sessions. In this context, frameworks for analyzing conversation quality—such as coherence, engagement, and appropriateness—are introduced, along with discussion of how to code qualitative transcripts using thematic analysis.

Tutorial: Conduct and Analyze a Usability Test

Students will conduct user testing of their chatbot systems, either remotely or in class, using structured test plans. They will collect both observational and interactional data, annotate conversation logs, and analyze findings using a combination of metrics and coding schemas. Emphasis will be placed on interpreting user reactions and identifying usability breakdowns. Based on their evaluation, students will develop a design brief for system improvement, supported by annotated evidence from their tests. The tutorial positions usability evaluation not as a validation step, but as a generative moment in the design process.

Week 10

Lecture: Responsible AI in CUIs I – Ethics, Bias, and Surveillance

This session marks the first of two lectures on Responsible AI in conversational systems, focusing on the ethical risks and social implications of CUIs. Students will examine how voice interfaces and chatbots encode normative assumptions through choices in gender presentation, accent, tone, and personality. The lecture introduces case studies where commercial agents (e.g., Siri, Alexa, and customer-facing bots) have perpetuated stereotypes or introduced surveillance concerns, particularly through passive data collection and emotional manipulation.

Theoretical frameworks such as Value-Sensitive Design (Friedman et al.), Fairness, Accountability, and Transparency (FAT*), and Contextual Integrity (Nissenbaum) will provide analytical lenses for interpreting these issues. Through these perspectives, students will explore how system design intersects with power, visibility, and consent, especially for marginalized or vulnerable user populations. Discussion extends to training data bias, reinforcement of hegemonic norms, and the ethical tensions of designing for “likability” over honesty.

Tutorial: Critical Review of CUI Case Studies

In this session, students will conduct a structured ethical audit of their team projects using a checklist developed from the frameworks introduced in class. The audit includes analysis of data handling practices, representational fairness, accessibility, and transparency. Students will reflect on the implicit values embedded in their system, and develop a short ethical considerations statement to accompany their final submission. The tutorial supports students in moving from reactive to proactive approaches to AI ethics, equipping them with practical tools for real-world deployment contexts.

Week 11

Lecture: Responsible AI in CUIs II – Anthropomorphism, Human Agency

This lecture continues the exploration of how human-like qualities and social roles are embedded in conversational user interfaces, turning attention to the psychological, social, and design implications of anthropomorphism and perceived agency. Building on prior discussions of trust and representation, this session investigates how users attribute mind, emotion, or intentionality to CUIs—often regardless of their technical capabilities.

Students will examine theories of anthropomorphism, including Epley’s Three-Factor Theory (elicited agent knowledge, effectance motivation, and sociality motivation), and how these tendencies shape user expectations. Through examples from Replika, AI companions, and multimodal assistants like XiaoIce and Pi.ai, the lecture discusses how visual, linguistic, and behavioral cues can lead users to interpret chatbots as socially or emotionally aware—sometimes leading to overtrust, emotional bonding, or role confusion.

The session also explores the design of agentic behavior in CUIs, questioning what it means for a system to “have agency” in different interaction contexts. Concepts from actor-network theory, HCI agency frameworks, and human-agent teaming will help students consider how agency can be distributed, designed, or constrained. Special attention is given to multi-agent CUIs and systems where conversational entities simulate internal deliberation, disagreement, or cooperation, inviting users into more complex social dynamics.

Tutorial: Ethical Audit of Final Project

In this session, students will analyze their own projects through the lens of anthropomorphism and agency. Each team will reflect on how their system presents itself—whether as a tool, a partner, a persona, or something in-between—and how these design choices shape user expectations and interaction patterns. They will assess the linguistic style, visual representation (if applicable), and behavioral framing of their agent, drawing on theories discussed in the lecture.

Critique 2: Trust, Inclusion and Ethical Design of CUI