Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userExtending Neural Keyword Extraction with TF-IDF tagset matching
تمديد استخراج الكلمات الرئيسية العصبية مع TF-IDF Tagset مطابقة
298
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
استخراج الكلمات الرئيسية هي مهمة تحديد الكلمات (أو تعبيرات متعددة الكلمة) التي تصف أفضل وثيقة معينة وخدمة في بوابات الأخبار لربط مقالات من مواضيع مماثلة. في هذا العمل، نطور وتقييم أساليبنا على أربع مجموعات بيانات جديدة تغطي لغات أقل ممثلة تمثيلا، لغات غنية بالمظورة في صناعة وسائل الإعلام الإخبارية الأوروبية (الكرواتية، الإستونية، اللاتفية والروسية). أولا، نؤدي تقييم اثنين من أساليب المحولات العصبية الخاضعة للإشراف، والتكلمة العصبية القائمة على المحولات لتحديد الكلمات الرئيسية (TNT-KID) وتمثيل التشفير ثنائي الاتجاه من المحولات (بيرت) مع وجود ذاكرة عشوائية قصيرة الأجل طويلة الأجل الطويلة الأجل (BILSTM) رأس التصنيف CRF)، ومقارنتها بموجب تردد خط الأساس - تعتمد على تردد المستندات (TF-IDF) مقرها. بعد ذلك، نظهر أنه من خلال الجمع بين الكلمات الرئيسية التي تم استرجها من قبل كل من الأساليب القائم على المحولات العصبية وتوسيع المجموعة النهائية من الكلمات الرئيسية ذات التقنية القائمة على TF-IDF غير المدعومة، يمكننا تحسين استدعاء النظام بشكل كبير، مما يجعلها مناسبة للاستخدام ك نظام التوصية في بيئة مجلس الإعلام.
Keyword extraction is the task of identifying words (or multi-word expressions) that best describe a given document and serve in news portals to link articles of similar topics. In this work, we develop and evaluate our methods on four novel data sets covering less-represented, morphologically-rich languages in European news media industry (Croatian, Estonian, Latvian, and Russian). First, we perform evaluation of two supervised neural transformer-based methods, Transformer-based Neural Tagger for Keyword Identification (TNT-KID) and Bidirectional Encoder Representations from Transformers (BERT) with an additional Bidirectional Long Short-Term Memory Conditional Random Fields (BiLSTM CRF) classification head, and compare them to a baseline Term Frequency - Inverse Document Frequency (TF-IDF) based unsupervised approach. Next, we show that by combining the keywords retrieved by both neural transformer-based methods and extending the final set of keywords with an unsupervised TF-IDF based technique, we can drastically improve the recall of the system, making it appropriate for usage as a recommendation system in the media house environment.
References used
https://aclanthology.org/
rate research
Read More
In the paper, we deal with the problem of unsupervised text document clustering for the Polish language. Our goal is to compare the modern approaches based on language modeling (doc2vec and BERT) with the classical ones, i.e., TF-IDF and wordnet-base
d. The experiments are conducted on three datasets containing qualification descriptions. The experiments' results showed that wordnet-based similarity measures could compete and even outperform modern embedding-based approaches.
This work revisits the information given by the graph-of-words and its typical utilization through graph-based ranking approaches in the context of keyword extraction. Recent, well-known graph-based approaches typically employ the knowledge from word
vector representations during the ranking process via popular centrality measures (e.g., PageRank) without giving the primary role to vectors' distribution. We consider the adjacency matrix that corresponds to the graph-of-words of a target text document as the vector representation of its vocabulary. We propose the distribution-based modeling of this adjacency matrix using unsupervised (learning) algorithms. The efficacy of the distribution-based modeling approaches compared to state-of-the-art graph-based methods is confirmed by an extensive experimental study according to the F1 score. Our code is available on GitHub.
The rapid flow of information and the abundance of text data on the Internet have brought about the urgent demand for the construction of monitoring resources and techniques used for various purposes. To extract facets of information useful for parti
cular domains from such large and dynamically growing corpora requires an unsupervised yet transparent ways of analyzing the textual data. This paper proposed a hybrid collocation analysis as a potential method to retrieve and summarize Taiwan-related topics posted on Weibo and PTT. By grouping collocates of 臺灣 Taiwan' into clusters of topics via either word embeddings clustering or Latent Dirichlet allocation, lists of collocates can be converted to probability distributions such that distances and similarities can be defined and computed. With this method, we conduct a diachronic analysis of the similarity between Weibo and PTT, providing a way to pinpoint when and how the topic similarity between the two rises or falls. A fine-grained view on the grammatical behavior and political implications is attempted, too. This study thus sheds light on alternative explainable routes for future social media listening method on the understanding of cross-strait relationship.
Term weighting schemes are widely used in Natural Language Processing and Information Retrieval. In particular, term weighting is the basis for keyword extraction. However, there are relatively few evaluation studies that shed light about the strengt
hs and shortcomings of each weighting scheme. In fact, in most cases researchers and practitioners resort to the well-known tf-idf as default, despite the existence of other suitable alternatives, including graph-based models. In this paper, we perform an exhaustive and large-scale empirical comparison of both statistical and graph-based term weighting methods in the context of keyword extraction. Our analysis reveals some interesting findings such as the advantages of the less-known lexical specificity with respect to tf-idf, or the qualitative differences between statistical and graph-based methods. Finally, based on our findings we discuss and devise some suggestions for practitioners. Source code to reproduce our experimental results, including a keyword extraction library, are available in the following repository: https://github.com/asahi417/kex
Byte-pair encoding (BPE) is a ubiquitous algorithm in the subword tokenization process of language models as it provides multiple benefits. However, this process is solely based on pre-training data statistics, making it hard for the tokenizer to han
dle infrequent spellings. On the other hand, though robust to misspellings, pure character-level models often lead to unreasonably long sequences and make it harder for the model to learn meaningful words. To alleviate these challenges, we propose a character-based subword module (char2subword) that learns the subword embedding table in pre-trained models like BERT. Our char2subword module builds representations from characters out of the subword vocabulary, and it can be used as a drop-in replacement of the subword embedding table. The module is robust to character-level alterations such as misspellings, word inflection, casing, and punctuation. We integrate it further with BERT through pre-training while keeping BERT transformer parameters fixed--and thus, providing a practical method. Finally, we show that incorporating our module to mBERT significantly improves the performance on the social media linguistic code-switching evaluation (LinCE) benchmark.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
