Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userSoloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching
المنفرد المنفرد: بناء روبوتات المهام على نطاق واسع مع التعلم النقل والتدريس الجهاز
42
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
مجردة نحن نقدم طريقة جديدة، منفردا، 1 التي تستخدم التعلم نقل وتدريس الجهاز لبناء روبوتات المهام على نطاق واسع. نقوم بمعلمات أنظمة الحوار الموجهة نحو المهام المعيارية الكلاسيكية باستخدام نموذج لغة تراجع تلقائي يستند إلى المحولات، والتي تضم وحدات حوار مختلفة في نموذج عصبي واحد. نحن قبل التدريب مسبقا، على برج الحوار غير المتجانس، نموذج توليد الاستجابة المتطلم، الذي يمكن أن يؤدي إلى إنشاء ردود حوار حول أهداف المستخدم ومعرفة العالم الحقيقي لإنجاز المهمة. يمكن تكييف النموذج المدرب مسبقا بكفاءة لإنجاز مهام جديدة مع حفنة من مربعات الحوار الخاصة بمهام المهام عبر تدريس الجهاز، حيث يتم إنشاء عينات التدريب من قبل معلمي البشر يتفاعلون مع النظام. تبين التجارب أن (ط) ينشئ المنفرد من المنازل من من من بين الفن في معايير الحوار الموجهة نحو المهام المدروسة جيدا، بما في ذلك Camrest676 و Mulotwoz؛ (2) في إعدادات الطاقة الدقيقة القليلة، يتفوق المنفر المنفرد بشكل كبير على الطرق الحالية؛ (3) استخدام التدريس الآلي يقلل بشكل كبير من تكلفة وضع العلامات للضبط الجميل. تتوفر النماذج والأدوات المدربة مسبقا في HTTPS://aka.ms/Soloist.
Abstract We present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish new tasks with a handful of task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the system. Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly outperforms existing methods; and (iii) the use of machine teaching substantially reduces the labeling cost of fine-tuning. The pre-trained models and codes are available at https://aka.ms/soloist.
References used
https://aclanthology.org/
rate research
Read More
This paper illustrates our approach to the shared task on large-scale multilingual machine translation in the sixth conference on machine translation (WMT-21). In this work, we aim to build a single multilingual translation system with a hypothesis t
hat a universal cross-language representation leads to better multilingual translation performance. We extend the exploration of different back-translation methods from bilingual translation to multilingual translation. Better performance is obtained by the constrained sampling method, which is different from the finding of the bilingual translation. Besides, we also explore the effect of vocabularies and the amount of synthetic data. Surprisingly, the smaller size of vocabularies perform better, and the extensive monolingual English data offers a modest improvement. We submitted to both the small tasks and achieve the second place.
We present the results of the first task on Large-Scale Multilingual Machine Translation. The task consists on the many-to-many evaluation of a single model across a variety of source and target languages. This year, the task consisted on three diffe
rent settings: (i) SMALL-TASK1 (Central/South-Eastern European Languages), (ii) the SMALL-TASK2 (South-East Asian Languages), and (iii) FULL-TASK (all 101 x 100 language pairs). All the tasks used the FLORES-101 dataset as the evaluation benchmark. To ensure the longevity of the dataset, the test sets were not publicly released and the models were evaluated in a controlled environment on Dynabench. There were a total of 10 participating teams for the tasks, with a total of 151 intermediate model submissions and 13 final models. This year's result show a significant improvement over the known base-lines with +17.8 BLEU for SMALL-TASK2, +10.6 for FULL-TASK and +3.6 for SMALL-TASK1.
Non-Autoregressive machine Translation (NAT) models have demonstrated significant inference speedup but suffer from inferior translation accuracy. The common practice to tackle the problem is transferring the Autoregressive machine Translation (AT) k
nowledge to NAT models, e.g., with knowledge distillation. In this work, we hypothesize and empirically verify that AT and NAT encoders capture different linguistic properties of source sentences. Therefore, we propose to adopt multi-task learning to transfer the AT knowledge to NAT models through encoder sharing. Specifically, we take the AT model as an auxiliary task to enhance NAT model performance. Experimental results on WMT14 En-De and WMT16 En-Ro datasets show that the proposed Multi-Task NAT achieves significant improvements over the baseline NAT models. Furthermore, the performance on large-scale WMT19 and WMT20 En-De datasets confirm the consistency of our proposed method. In addition, experimental results demonstrate that our Multi-Task NAT is complementary to knowledge distillation, the standard knowledge transfer method for NAT.
This paper describes TenTrans large-scale multilingual machine translation system for WMT 2021. We participate in the Small Track 2 in five South East Asian languages, thirty directions: Javanese, Indonesian, Malay, Tagalog, Tamil, English. We mainly
utilized forward/back-translation, in-domain data selection, knowledge distillation, and gradual fine-tuning from the pre-trained model FLORES-101. We find that forward/back-translation significantly improves the translation results, data selection and gradual fine-tuning are particularly effective during adapting domain, while knowledge distillation brings slight performance improvement. Also, model averaging is used to further improve the translation performance based on these systems. Our final system achieves an average BLEU score of 28.89 across thirty directions on the test set.
Domain adaption in syntactic parsing is still a significant challenge. We address the issue of data imbalance between the in-domain and out-of-domain treebank typically used for the problem. We define domain adaptation as a Multi-task learning (MTL)
problem, which allows us to train two parsers, one for each do-main. Our results show that the MTL approach is beneficial for the smaller treebank. For the larger treebank, we need to use loss weighting in order to avoid a decrease in performance be-low the single task. In order to determine towhat degree the data imbalance between two domains and the domain differences affect results, we also carry out an experiment with two imbalanced in-domain treebanks and show that loss weighting also improves performance in an in-domain setting. Given loss weighting in MTL, we can improve results for both parsers.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
