Understanding the human in the design of cyber-human discovery systems for data-driven astronomy

High-quality, usable, and effective software is essential for supporting astronomers in the discovery-focused tasks of data analysis and visualisation. As the volume, and perhaps more crucially, the velocity of astronomical data grows, the role of the astronomer is changing. There is now an increased reliance on automated and autonomous discovery and decision-making workflows rather than visual inspection. We assert the need for an improved understanding of how astronomers (humans) currently make visual discoveries from data. This insight is a critical element for the future design, development and effective use of cyber-human discovery systems, where astronomers work in close collaboration with automated systems to gain understanding from continuous, real-time data streams. We discuss how relevant human performance data could be gathered, specifically targeting the domains of expertise and skill at visual discovery, and the identification and management of cognitive factors. By looking to other disciplines where human performance is assessed and measured, we propose four early-stage applications that would: (1) allow astronomers to evaluate, and potentially improve, their own visual discovery skills; (2) support just-in-time coaching; (3) enable talent identification; and (4) result in user interfaces that automatically respond to skill level and cognitive state. Throughout, we advocate for the importance of user studies and the incorporation of participatory design and co-design practices into the planning, implementation and evaluation of alternative user interfaces and visual discovery environments.