Keynotes
Mixed Reality and Generative AI in Healthcare: From Rehearsal to Responsible Ca
Professor Pan Hui is Chair Professor of Computational Media and Arts and Director of the Center for Metaverse and Computational Creativity at the Hong Kong University of Science and Technology (Guangzhou). He also holds appointments as Chair Professor of Emerging Interdisciplinary Areas at the Hong Kong University of Science and Technology and Nokia Chair in Data Science at the University of Helsinki. Previously, Professor Hui served as Senior Research Scientist and subsequently Distinguished Scientist at Telekom Innovation Laboratories (T-Labs) in Germany. His industry research experience also includes positions at Intel Research Cambridge and Thomson Research Paris. His research has received support from major industry partners, including Nokia, Deutsche Telekom, Microsoft Research, and China Mobile. He has published more than 600 research papers, received over 40,000 citations, and holds 32 European and US patents. Professor Hui is an International Fellow of the Royal Academy of Engineering, a Member of Academia Europaea, an IEEE Fellow, and an ACM Distinguished Scientist. He is also a founding member of the INTERPOL Expert Group on Metaverse and was a member of the World Economic Forum’s Global Future Council on the Future of Metaverse. He earned his PhD in Computer Science from the University of Cambridge.
Abstract
Mixed reality and generative AI are opening new possibilities for clinical training and patient support. But expanding their role requires answering a harder question: What should we entrust to these systems, and what evidence should justify that trust? Drawing on our research, this keynote examines that question across three settings: clinical training, support beyond the clinic, and AI interactions without clinician oversight. In training, mixed-reality simulations move beyond fixed scenarios, allowing simulated patients’ conditions to evolve as trainees assess and act. AI-powered standardized patients extend rehearsal to clinical communication. For patient support, we explore wearables developed with rehabilitation clinicians and tools that help patients understand medical procedures. We also examine failures that matter when people turn to AI without a clinician present. In a workshop involving seventy clinicians, educators, and researchers, subtle changes in wording shifted models’ sensitivity to risk, while models sometimes agreed when they should have challenged a premise. Together, these projects highlight both opportunities to extend human expertise and the need to test the limits of that extension. The keynote considers what evidence is needed to move from promising demonstrations to justified roles in care. The goal is healthcare in which rehearsal comes before the stakes are real, patients understand what is about to happen to them, and systems are entrusted only with responsibilities they have demonstrated they can fulfil.
Pan Hui
Professor, Hong Kong University of Science and Technology
Nicos Maglaveras
Professor of Medical Informatics Aristotle University of Thessaloniki Greece
Personalised health driven by digital health systems and multi-source health/environmental data, ML/AI/DL analytics and predictive models
Nicos Maglaveras received the diploma in electrical engineering from the Aristotle University of Thessaloniki (A.U.Th.), Greece, in 1982, and the M.Sc. and Ph.D. degrees in electrical engineering with an emphasis in biomedical engineering from Northwestern University, Evanston, IL, in 1985 and 1988, respectively. He is currently a Professor of Medical Informatics, A.U.Th. He served as head of the graduate program in medical informatics at A.U.Th, as Visiting Professor at Northwestern University Dept of EECS (2016-2019), and is a collaborating researcher with the Center of Research and Technology Hellas, and the National Hellenic Research Foundation.
His current research interests include biomedical engineering, biomedical informatics, ehealth, AAL, personalised health, biosignal analysis, medical imaging, and neurosciences. He has published more than 500 papers in peer-reviewed international journals, books and conference proceedings out of which over 160 as full peer review papers in indexed international journals. He has developed graduate and undergraduate courses in the areas of (bio)medical informatics, biomedical signal processing, personal health systems, physiology and biological systems simulation.
He has served as a Reviewer in CEC AIM, ICT and DGRT D-HEALTH technical reviews and as reviewer, associate editor and editorial board member in more than 20 international journals, and participated as Coordinator or Core Partner in over 45 national and EU and US funded competitive research projects attracting more than 16 MEUROs in funding. He has served as president of the EAMBES in 2008-2010. Dr. Maglaveras has been a member of the IEEE, AMIA, the Greek Technical Chamber, the New York Academy of Sciences, the CEN/TC251, Eta Kappa Nu and an EAMBES Fellow.
Abstract
The last years saw a steep increase in the number of wearable sensors and systems, mhealth and uhealth apps both in the clinical settings and in everyday life. Further large amounts of data both in the clinical settings (imaging, biochemical, medication, electronic health records, -omics), in the community (behavioral, social media, mental state, genetic tests, wearable driven bio-parameters and biosignals) as well as environmental stressors and data (air quality, water pollution etc.) have been produced, and made available to the scientific and medical community, powering the new AI/DL/ML based analytics for the identification of new digital biomarkers leading to new diagnostic pathways, updated clinical and treatment guidelines, and a better and more intuitive interaction medium between the citizen and the health care system.
Thus, the concept of connected and translational health has started evolving steadily, connecting pervasive health systems, using new predictive models, new approaches in biological systems modeling and simulation, as well as fusing data and information from different pipelines for more efficient diagnosis and disease management.
In this talk, we will present the current state-of-the-art in personalized health care by presenting cases from COVID-19 and COPD patients using advanced wearable vests and new technology sensors including lung sound and EIT, new outcome prediction models in COVID-19 ICU patients fusing X-Rays, lung sounds, and ICU parameters transformed via AI/ML/DL pipelines, new approaches fusing environmental stressors with -omics analytics for chronic disease management, and finally new ML/AI-driven methodologies for predicting mental health diseases including suicidality, anxiety, and depression.
