Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userNovel Natural Language Summarization of Program Code via Leveraging Multiple Input Representations
تلخيص اللغة الطبيعية الجديدة من رمز البرنامج عبر الاستفادة من العديد من تمثيلات الإدخال
168
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
إن الافتقار إلى وصف رمز برنامج معين بمثابة عقبة كبيرة لتلك المطورين الجديدة في قاعدة التعليمات البرمجية لفهمها. أبلغت معالجة هذه المشكلة، العمل السابق في تلخيص التعليمات البرمجية، مهمة إنشاء وصف التعليمات البرمجية تلقائيا نظرا لأن نموذج التعلم الإضافي الذي تم تدريبه على إنتاج واجهة برمجة API (واجهة برمجة التطبيقات) أظهرت نتائج واعدة عند تطبيقها على ملخص كود نموذج. ومع ذلك، فإن رموز مختلفة لها ملخصات مختلفة يمكن أن يكون لها نفس مجموعة تسلسل API. إذا قمنا بتدريب نموذج لتوليد ملخصات بالنظر إلى تسلسل API، فلن يكون النموذج قادرا على التعلم بفعالية. ومع ذلك، نلاحظ أن تسلسل API لا يزال من الممكن أن يكون مفيدا ولم يتم استخدامه بنشاط. يقترح هذا العمل نهجا جديدا للمهاميات الجديدة التي تدرب في وقت واحد اثنين من المهام المماثلة: 1) تلخيص شفرة معينة (رمز إلى ملخص)، و 2) تلخيص تسلسل API معين (تسلسل API إلى ملخص). نقترح تشفير رواية على مستوى التعليمات البرمجية بناء على بيرت قادرة على التعبير عن دلالات التعليمات البرمجية، والحصول على تمثيلات لكل سطر التعليمات البرمجية. عملنا هو أول عمل تلخيص التعليمات البرمجية يستخدم نموذج اللغة المدربين تدريبا مسبقا باللغة والسياق الطبيعي في تشفيره. نقيم نهجنا باستخدام مجموعات بيانات شائعة (Java and Python) التي تم استخدامها على نطاق واسع في الدراسات السابقة. تظهر نتائجنا التجريبية أن نهجنا متعدد المهام لدينا يحسن على خطوط الأساس ويحقق الحالة الجديدة من بين الفن.
The lack of description of a given program code acts as a big hurdle to those developers new to the code base for its understanding. To tackle this problem, previous work on code summarization, the task of automatically generating code description given a piece of code reported that an auxiliary learning model trained to produce API (Application Programming Interface) embeddings showed promising results when applied to a downstream, code summarization model. However, different codes having different summaries can have the same set of API sequences. If we train a model to generate summaries given an API sequence, the model will not be able to learn effectively. Nevertheless, we note that the API sequence can still be useful and has not been actively utilized. This work proposes a novel multi-task approach that simultaneously trains two similar tasks: 1) summarizing a given code (code to summary), and 2) summarizing a given API sequence (API sequence to summary). We propose a novel code-level encoder based on BERT capable of expressing the semantics of code, and obtain representations for every line of code. Our work is the first code summarization work that utilizes a natural language-based contextual pre-trained language model in its encoder. We evaluate our approach using two common datasets (Java and Python) that have been widely used in previous studies. Our experimental results show that our multi-task approach improves over the baselines and achieves the new state-of-the-art.
References used
https://aclanthology.org/
rate research
Read More
Learning a good latent representation is essential for text style transfer, which generates a new sentence by changing the attributes of a given sentence while preserving its content. Most previous works adopt disentangled latent representation learn
ing to realize style transfer. We propose a novel text style transfer algorithm with entangled latent representation, and introduce a style classifier that can regulate the latent structure and transfer style. Moreover, our algorithm for style transfer applies to both single-attribute and multi-attribute transfer. Extensive experimental results show that our method generally outperforms state-of-the-art approaches.
The span-based model enjoys great popularity in recent works of sequence segmentation. However, each of these methods suffers from its own defects, such as invalid predictions. In this work, we introduce a unified span-based model, lexical unit analy
sis (LUA), that addresses all these matters. Segmenting a lexical unit sequence involves two steps. Firstly, we embed every span by using the representations from a pretraining language model. Secondly, we define a score for every segmentation candidate and apply dynamic programming (DP) to extract the candidate with the maximum score. We have conducted extensive experiments on 3 tasks, (e.g., syntactic chunking), across 7 datasets. LUA has established new state-of-the-art performances on 6 of them. We have achieved even better results through incorporating label correlations.
Automatically extracting keyphrases from scholarly documents leads to a valuable concise representation that humans can understand and machines can process for tasks, such as information retrieval, article clustering and article classification. This
paper is concerned with the parts of a scientific article that should be given as input to keyphrase extraction methods. Recent deep learning methods take titles and abstracts as input due to the increased computational complexity in processing long sequences, whereas traditional approaches can also work with full-texts. Titles and abstracts are dense in keyphrases, but often miss important aspects of the articles, while full-texts on the other hand are richer in keyphrases but much noisier. To address this trade-off, we propose the use of extractive summarization models on the full-texts of scholarly documents. Our empirical study on 3 article collections using 3 keyphrase extraction methods shows promising results.
Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and i
nference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. We also conduct experiments on our newly proposed benchmark to show that the augmentation effect is not only attributed to memorization. Further ablation studies and a qualitative analysis provide more insights into our approach.
Sarcasm detection is one of the top challenging tasks in text classification, particularly for informal Arabic with high syntactic and semantic ambiguity. We propose two systems that harness knowledge from multiple tasks to improve the performance of
the classifier. This paper presents the systems used in our participation to the two sub-tasks of the Sixth Arabic Natural Language Processing Workshop (WANLP); Sarcasm Detection and Sentiment Analysis. Our methodology is driven by the hypothesis that tweets with negative sentiment and tweets with sarcasm content are more likely to have offensive content, thus, fine-tuning the classification model using large corpus of offensive language, supports the learning process of the model to effectively detect sentiment and sarcasm contents. Results demonstrate the effectiveness of our approach for sarcasm detection task over sentiment analysis task.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
