تركز معالجة اللغة الطبيعية الحالية بقوة على زيادة الدقة.يأتي التقدم بتكلفة نماذج فائقة الثقيلة مع مئات الملايين أو حتى مليارات المعلمات.ومع ذلك، فإن المهام النحوية البسيطة مثل وضع العلامات على جزء من الكلام (POS) أو تحليل التبعية أو التعرف على الكيان المسمى (NER) لا تحتاج إلى أكبر النماذج لتحقيق نتائج مقبولة.تمشيا مع هذا الافتراض، نحاول تقليل حجم النموذج الذي ينفذ بشكل مشترك جميع المهام الثلاثة.نقدم Comboner: أداة خفيفة الوزن، أوامر ذات حجم أصغر من المحولات الحديثة.يعتمد على مدمج الكلمات الفرعية المدربة مسبقا بنية الشبكة العصبية المتكررة.يعمل COMBONER على بيانات اللغة البولندية.يحتوي النموذج على مخرجات لوضع العلامات على نقاط البيع والتحليل التبعية و NER.تحتوي ورقةنا على بعض الأفكار من ضبط النموذج الدقيق والتقارير عن نتائجها الإجمالية.
The current natural language processing is strongly focused on raising accuracy. The progress comes at a cost of super-heavy models with hundreds of millions or even billions of parameters. However, simple syntactic tasks such as part-of-speech (POS) tagging, dependency parsing or named entity recognition (NER) do not require the largest models to achieve acceptable results. In line with this assumption we try to minimize the size of the model that jointly performs all three tasks. We introduce ComboNER: a lightweight tool, orders of magnitude smaller than state-of-the-art transformers. It is based on pre-trained subword embeddings and recurrent neural network architecture. ComboNER operates on Polish language data. The model has outputs for POS tagging, dependency parsing and NER. Our paper contains some insights from fine-tuning of the model and reports its overall results.
References used
https://aclanthology.org/
State-of-the-art approaches to spelling error correction problem include Transformer-based Seq2Seq models, which require large training sets and suffer from slow inference time; and sequence labeling models based on Transformer encoders like BERT, wh
We describe the DCU-EPFL submission to the IWPT 2021 Parsing Shared Task: From Raw Text to Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilit
Accurate recovery of predicate-argument structure from a Universal Dependency (UD) parse is central to downstream tasks such as extraction of semantic roles or event representations. This study introduces compchains, a categorization of the hierarchy
We describe the NUIG solution for IWPT 2021 Shared Task of Enhanced Dependency (ED) parsing in multiple languages. For this shared task, we propose and evaluate an End-to-end Seq2seq mBERT-based ED parser which predicts the ED-parse tree of a given i
Human language encompasses more than just text; it also conveys emotions through tone and gestures. We present a case study of three simple and efficient Transformer-based architectures for predicting sentiment and emotion in multimodal data. The Lat