Key Information for the Task
What Am I Being Asked To Do?
Read mental health related post and their chatbot responses and tell us whether the responses are inappropriate or not (this should take approx. 5 minutes).
What Is This Study About and What Procedures Will You be Asked to Follow?
This study is about analyzing the behaviors of chatbots in mental health related contexts. You will be asked to read and analyze posts and their response generated from different chatbots.
Are There Any Risks or Discomforts you Might Experience by Being in this Study?
The mental health related posts and their responses from chatbots are uncensored and may contain self-harm and suicide ideation content. We understand that such sensitive content can be disturbing or upsetting. If you have queries, concerns, or negative reactions to any of the content, please either email us (Ashutosh Baheti at firstname.lastname@example.org, or Professor Alan Ritter at email@example.com) or reach out for support online if in crisis.
What Are the Reasons You Might Want to Volunteer For This Study?
The data annotated as a part of this research study will help researchers better and safer chatbots that can engage with users in a positive way. Furthermore, for every successfully annotated form you will receive $0.80 (amount set while considering federal minimum wage).
Do You Have to Take Part in the This Study?
The participation in this study is completely voluntary. Any annotator can stop and leave at any point of time without losing the rewards earned on previous completed tasks.
We want to analyze the possibly inappropriate language generated by a chatbot to different kinds of inputs related to mental health situations. You will be asked to evaluate 5 posts and their chatbot responses per task and categorize them in predefined categories.
The participation in this study is voluntary. Stopping participation at any point won't affect the rewards you have earned on completed tasks. We want the answers to the questions from an American cultural perspective. Subjects located in the EU are not allowed to join this study.
We will show you posts with their responses generated by one of our chatbots. For each post, indicate if it is talking about a potential mental health related situation and identify whether the responses generated by chatbots are plausible and appropriate. Participation in this research study is voluntary. You may skip or stop in the middle of any task.
Risks and Discomforts:
The posts were downloaded from the (uncensored) internet. The content of the posts or the chatbot response may be related to self-harm or suicide. We do not agree with any such harmful/malicious content and will assume that all workers are annotating in good faith. If you have queries, concerns, or negative reactions to any of the content, feel free to email us.
Although the information collected in this study may not directly benefit to you, we will very much appreciate your help in identifying inappropriate chatbot behavior, since this is something chatbots have no clue about. This carefully annotated data will help researchers develop better models that will be able to identify and avoid/correct generating inappropriate language when discussing mental health related problems.
Compensation to You:
For every successfully annotated task, you will be rewarded with $0.80 (set in consideration of the Federal minimum wage). However, if we feel that you are not properly following our annotation guidelines or your responses are widely different from expected then we might put a quota on the number of tasks you can do in the future.
Storing and Sharing your Information:
Your participation in this study is gratefully acknowledged. By agreeing to participate in this research study, you consent for your de-identified information/data to be securely stored in cloud servers and shared with qualified researchers/experts who want to further the research on mental health related safety in chatbots. Future researchers will not have a way to identify you.
We do not ask for your name or other traceable information and will anonymize the data to the best of our extent. Only the researchers involved in this study will be allowed to analyze the anonymized data. Participants can choose to excluded from this study and ask to remove their annotated data from the study at any point. Kindly email us if you want us to exclude your data.
Costs to You:
There will be no costs (other than your time) to you if you decide to participate in this research study.
Questions about the Study:
If you have queries, concerns, or feedback regarding the format of the study, kindly email us (Ashutosh Baheti at firstname.lastname@example.org, or Professor Alan Ritter at email@example.com). Note: Feedback for improving the research study are always welcome!
Questions about Your Rights as a Research Participant:
- Your participation in this study is voluntary. You do not have to be in this study if you don't want to be.
- You have the right to change your mind and leave the study at any time without giving any reason and without penalty.
- Any new information that may make you change your mind about being in this study will be given to you.
- To save a copy of the consent form and instructions, you can save/print this webpage.
- You do not waive any of your legal rights by signing this consent form.
If you have any questions about your rights as a research participant, you may contact Ms. Kelly Winn, Georgia Institute of Technology Office of Research Integrity Assurance, at (404) 385-2175