نماذج الإجابة على الأسئلة (QA) تستخدم أنظمة المسترد والقارئ للإجابة على الأسئلة.يمكن الاعتماد على البيانات التدريبية من قبل أنظمة ضمان الجودة أو تعكس عدم المساواة من خلال ردودهم.يتم تدريب العديد من نماذج QA، مثل تلك الخاصة ب DataSet Squad، على مجموعة فرعية من مقالات ويكيبيديا التي تشفص لتحيزاتها الخاصة وإعادة إنتاج عدم المساواة في العالم الحقيقي.فهم كيفية تأثير البيانات التدريبية على التحيز في أنظمة ضمان الجودة يمكن أن تبلغ الأساليب لتخفيف عدم المساواة.نقوم بتطوير مجموعتين من الأسئلة لأسئلة النطاق المغلقة والفتوة على التوالي، والتي تستخدم أسئلة غامضة لتحقيق نماذج QA للتحيز.نطعم ثلاثة أنظمة ضمان الجودة في التعلم العميق مع مجموعات الأسئلة الخاصة بنا وتقييم الردود على التحيز عبر المقاييس.باستخدام مقاييسنا، نجد أن نماذج QA المجال المفتوحة تضخيم التحيزات أكثر من نظيرهم المغلقة من النطاق واقتراح أن يتحيزات في سطح المسترد بسهولة أكبر بسبب حرية الاختيار أكبر.
Question answering (QA) models use retriever and reader systems to answer questions. Reliance on training data by QA systems can amplify or reflect inequity through their responses. Many QA models, such as those for the SQuAD dataset, are trained and tested on a subset of Wikipedia articles which encode their own biases and also reproduce real-world inequality. Understanding how training data affects bias in QA systems can inform methods to mitigate inequity. We develop two sets of questions for closed and open domain questions respectively, which use ambiguous questions to probe QA models for bias. We feed three deep-learning-based QA systems with our question sets and evaluate responses for bias via the metrics. Using our metrics, we find that open-domain QA models amplify biases more than their closed-domain counterparts and propose that biases in the retriever surface more readily due to greater freedom of choice.
References used
https://aclanthology.org/
Current textual question answering (QA) models achieve strong performance on in-domain test sets, but often do so by fitting surface-level patterns, so they fail to generalize to out-of-distribution settings. To make a more robust and understandable
We tackle multi-choice question answering. Acquiring related commonsense knowledge to the question and options facilitates the recognition of the correct answer. However, the current reasoning models suffer from the noises in the retrieved knowledge.
Large pre-trained language models (PLMs) have led to great success on various commonsense question answering (QA) tasks in an end-to-end fashion. However, little attention has been paid to what commonsense knowledge is needed to deeply characterize t
Question answering (QA) models for reading comprehension have been demonstrated to exploit unintended dataset biases such as question--context lexical overlap. This hinders QA models from generalizing to under-represented samples such as questions wi
Pre-trained language models learn socially harmful biases from their training corpora, and may repeat these biases when used for generation. We study gender biases associated with the protagonist in model-generated stories. Such biases may be express