Treffer: Cheap, Rigorous, and Transparent: How Web-scraping with Python can Improve Collecting Grey Literature for Systematic Literature Reviews.

Gathering non-conventional literature, such as grey literature, from web-based sources for use in a systematic literature review is at times an arduous task. Often, the processes used to do so are difficult for other researchers to repeat. Compounding this issue is the cost that researchers bear, in either paying for desktop-based applications, or paying external researchers who have programming experience to design applications and tools for them. To address these issues, this article presents a methodology for researchers to systematically gather grey literature from online repositories using the computer programming language Python. Utilising a well-known data extraction technique (Web-Scraping), this article exhibits the code used to scrape policy documents from the International Energy Agency's online policy database. A flowchart for the different stages of this process is also introduced to aid in addressing the technical, legal, and ethical elements of web-scraping that researchers must also be aware of when undertaking this approach. Finally, as a proof of concept, the results from the method are also presented. [ABSTRACT FROM AUTHOR]

Copyright of Grey Journal (TGJ) is the property of TextRelease and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)