With the prosperity of smart contracts and the blockchain technology, various security analyzers have been proposed from both the academia and industry to address the associated risks. Yet, there does not exist a high-quality benchmark of smart contract vulnerability for security research. In this study, we propose an approach towards building a high-quality vulnerability benchmark. Our approach consists of two parts. First, to improve recall, we propose to search for similar vulnerabilities in an automated way by leveraging the abstract vulnerability signature (AVS). Second, to remove the false positives (FPs) due to AVS-based matching, we summarize the detection rules of existing tools and apply the refined rules by considering various defense mechanisms (DMs). By integrating AVS-based code matching and the refined detection rules (RDR), our approach achieves higher precision and recall. On the collected 76,354 contracts, we build a benchmark consisting of 1,219 vulnerabilities covering five different vulnerability types identified together by our tool (DOUBLADE) and other three scanners. Additionally, we conduct a comparison between DOUBLADE and the others, on an additional 17,770 contracts. Results show that DOUBLADE can yield a better detection accuracy with similar execution time.
Download