ASAQ - Artificial Social Agent Questionnaire
ASAQ - Artificial Social Agent Questionnaire
The Artificial Social Agent Questionnaire (ASAQ)[1][2] is a scientific instrument designed to systematically measure human experiences when interacting with artificial social agents (ASAs). ASAs are, for instance, chatbots, virtual agents, conversational agents, and social robots. The ASAQ provides standardised measurements for assessing key constructs like believability, sociability, usability, and trust. The instrument is available in two versions: a 90-item long version for detailed evaluation[1], and a 24-item short version for rapid assessment[1].
The ASAQ addresses the need for a standardised evaluation tool in ASA research, enabling cross-study comparisons and replication of findings[3]. It's development followed a community-driven approach. This means that the community's interest determined what the questionnaire aimed to measure, rather than a specific theory. Development of the ASAQ began in 2018 at the Intelligent Virtual Agent conference in Sydney, Australia[4], and culminated in the publication of the validated instrument in 2025[1]. The ASAQ was developed by an international working group of over 120 researchers[5], coordinated through the Open Science Foundation platform
The ASAQ has been studied, examining reliability, content validity, construct validity, and cross-validity[1]. The questionnaire has been translated into English[1], Chinese Mandarin[6], Dutch[7], and German[7], with additional translations in development[5]. The questionnaire is available in the 4TU repository.
Questionnaire Constructs
The ASAQ is structured around 19 core constructs, each capturing a particular aspect of the human-agent interaction experience. Three of these constructs are further divided into a combined total of eleven dimensions.
| No. | ID | Construct Name | Construct Definition |
|---|---|---|---|
| 1 | Agent Believability | The extent to which a user believes that the artefact is a social agent | |
| 1.1 | HLA | Human-Like Appearance | The extent to which a user believes that the social agent appears like a human |
| 1.2 | HLB | Human-Like Behavior | The extent to which a user believes that the social agent behaves like a human |
| 1.3 | NA | Natural Appearance | The extent to which a user believes that the social agent's appearance could exist in or be derived from nature |
| 1.4 | NB | Natural Behavior | The extent to which a user believes that the social agent's behaviour could exist in or be derived from nature |
| 1.5 | AAS | Agent's Appearance Suitability | The extent to which the agent's appearance is suitable for its role |
| 2 | AU | Agent's Usability | The extent to which a user believes that using an agent will be free from effort (future process) |
| 3 | PF | Performance | The extent to which a task was well performed (past performance) |
| 4 | AL | Agent's Likeability | The agent's qualities that bring about a favourable regard |
| 5 | AS | Agent's Sociability | The agent's quality or state of being sociable |
| 6 | Agent's Personality | The combination of characteristics or qualities that form an individual's distinctive character | |
| 6.1 | APP | Agent's Personality Presence | To what extent the user believes that the agent has a personality |
| 6.2 | Agent's Personality Type* | The particular personality of the agent | |
| 7 | UAA | User Acceptance of the Agent | The willingness of the user to interact with the agent |
| 8 | AE | Agent's Enjoyability | The extent to which a user finds interacting with the agent enjoyable |
| 9 | UE | User's Engagement | The extent to which the user feels involved in the interaction with the agent |
| 10 | UT | User's Trust | The extent to which a user believes in the reliability, truthfulness, and ability of the agent (for future interactions) |
| 11 | UAL | User Agent Alliance | The extent to which a beneficial association is formed |
| 12 | AA | Agent's Attentiveness | The extent to which the user believes that the agent is aware of and has attention for the user |
| 13 | AC | Agent's Coherence | The extent to which the agent is perceived as being logical and consistent |
| 14 | AI | Agent's Intentionality | The extent to which the agent is perceived as being deliberate and has deliberations |
| 15 | AT | Attitude | A favourable or unfavourable evaluation toward the interaction with the agent |
| 16 | SP | Social Presence | The degree to which the user perceives the presence of a social entity in the interaction |
| 17 | IIS | Interaction Impact on Self-Image | How the user believes others perceive the user because of the interaction with the agent |
| 18 | Emotional Experience | A self-contained phenomenal experience. They are subjective, evaluative, and independent of the sensations, thoughts, or images evoking them | |
| 18.1 | AEI | Agent's Emotional Intelligence Presence | To what extent the user believes that the agent has an emotional experience and can convey its emotions |
| 18.2 | Agent's Emotional Intelligence Type* | The particular emotional state of the agent | |
| 18.3 | UEP | User's Emotion Presence | To what extent the user believes that his/her emotional state is caused by the interaction or the agent |
| 18.4 | User's Emotion Type* | The particular emotional state of the user during or after the interaction with the agent | |
| 19 | UAI | User Agent Interplay | The extent to which the user and the agent have an effect on each other |
Notes: the numbering following <construct number>.<dimension number>. In italics are the constructs that are measured indirectly through dimensions. * Dimension not measured in the ASAQ.
Scale and perspective
The constructs and dimensions are assessed through a series of statements (i.e., questionnaire items), where participants indicate their level of agreement on a seven-point scale. Responses range from -3 ("strongly disagree") to +3 ("strongly agree"), with 0 representing a neutral stance ("neither agree nor disagree"). Using first and third-person perspective, the ASAQ can be used to assess a user's own experience with an ASA or to evaluate someone else's interaction with an agent.
The ASAQ instrument
ASAQ Representative Sets
The ASAQ Representative Sets serve as the normative datasets for interpreting ASAQ scores, providing essential context for understanding how an ASA scores across constructs and dimensions. The ASAQ Representative Set 2024[1] was developed during ASAQ validation and includes 1,066 participant ratings of 29 agents using a third-person perspective. A second dataset, the ASAQ Representative Set 2025, is pending publication and is based on first-person reports from 666 participants interacting with 10 commonly used agents (e.g., ChatGPT, Siri, Roomba). These representative sets offer researchers benchmarks for comparing their ASA's scores against familiar agents, enabling interpretation through percentile ranks or relative positioning. They also support study planning by providing effect size estimates and guidance for sample size decisions[1]. Additional ASAQ representative sets are expected to be developed over time, and all will be made publicly available online.
ASAQ Translations
Currently, translations of the ASAQ to several languages are available. Notably, the validated Dutch[7], German[7] and Chinese[6] versions of the ASAQ have been developed.
ASAQ Charts
Two types of ASAQ charts have been developed to visualise an ASA's interaction profile, each serving a distinct purpose[1]. The ASAQ Chart displays scores on the original scale, ranging from -3 to +3, reflecting the raw mean responses for each of the 24 constructs and dimensions. The Percentile ASAQ Chart presents the same constructs using percentile ranks, allowing researchers to compare their ASA's performance against an ASAQ representative set. In both charts, the centre shows the overall ASAQ score or its corresponding percentile score. Scripts for producing ASAQ charts are provided in the 4TU data repository.
Using ASAQ
A YouTube tutorial is available that introduces the ASAQ and explains how researchers can use it. The tutorial covers how to apply the questionnaire, present the results, calculate appropriate sample sizes, and understand existing evidence on the ASAQ's reliability and validity.
This article "ASAQ - Artificial Social Agent Questionnaire" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:ASAQ - Artificial Social Agent Questionnaire. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
- ↑ 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 Fitrianie, Siska; Bruijnes, Merijn; Abdulrahman, Amal; Brinkman, Willem-Paul (2025-05-01). "The Artificial Social Agent Questionnaire (ASAQ) — Development and evaluation of a validated instrument for capturing human interaction experiences with artificial social agents". International Journal of Human-Computer Studies. 199. doi:10.1016/j.ijhcs.2025.103482. ISSN 1071-5819. Unknown parameter
|article-number=ignored (help) - ↑ Fitrianie, Siska; Bruijnes, Merijn; Abdulrahman, Amal; Brinkman, Willem-Paul (2025-08-11). Artificial Social Agent Questionnaire (Report). American Psychological Association. doi:10.1037/t95286-000.
- ↑ Ioannidis, John P. A. (2022-08-25). "Correction: Why Most Published Research Findings Are False". PLOS Medicine. 19 (8). doi:10.1371/journal.pmed.1004085. ISSN 1549-1676. PMC 9410711 Check
|pmc=value (help). PMID 36007233 Check|pmid=value (help). Unknown parameter|article-number=ignored (help) - ↑ "Program". The workshop on Methodology and/of Evaluation of IVAs. 2018-11-04. Retrieved 2025-07-11.
- ↑ 5.0 5.1 "ASAQ: Artificial Social Agent Questionnaire". asaq.ewi.tudelft.nl. Retrieved 2025-07-04.
- ↑ 6.0 6.1 Li, Fengxiang; Fitrianie, Siska; Bruijnes, Merijn; Abdulrahman, Amal; Guo, Fu; Brinkman, Willem-Paul (2023-10-30). "Mandarin Chinese translation of the Artificial-Social-Agent questionnaire instrument for evaluating human-agent interaction". Frontiers in Computer Science. 5. doi:10.3389/fcomp.2023.1149305. ISSN 2624-9898. Unknown parameter
|article-number=ignored (help) - ↑ 7.0 7.1 7.2 7.3 Albers, Nele; Bönsch, Andrea; Ehret, Jonathan; Khodakov, Boleslav A.; Brinkman, Willem-Paul (2024-12-26). "German and Dutch Translations of the Artificial-Social-Agent Questionnaire Instrument for Evaluating Human-Agent Interactions". Proceedings of the ACM International Conference on Intelligent Virtual Agents. IVA '24. New York, NY, USA: Association for Computing Machinery. pp. 1–4. doi:10.1145/3652988.3673928. ISBN 979-8-4007-0625-7. Search this book on
