Open Research Tools and Technologies
Gazouilloire: a command line tool for long-term tweets collection
Many open-source libraries provide an interface for the Twitter API. However, most people use these tools in temporary scripts for a one-time tweets collection. Moving to a robust application for collecting and indexing tweets over long periods of time requires some programming knowledge that most social science researchers do not master. In order to meet this need, the medialab has developed gazouilloire, a tool that makes it possible to easily configure the collection parameters (keywords searched, language of tweets, location of tweets, etc.) and can then be launched from the command line.
Gazouilloire combines two methods to collect tweets from the Twitter API ("search" and "filter") in order to maximize the number of collected tweets, and automatically fills the gaps in the collection in case of connexion errors or reboots. It also provides a large range of features that are not directly available from the free Twitter API: collecting during specific periods of time, resolving redirected urls, downloading only certain types of media contents (only photos and no videos, for example) or unfolding Twitter conversations. The user can then choose to export the tweets in csv format, and select the fields that will form the columns of the table.
Social science researchers have already used gazouilloire for a wide variety of studies: measuring online activity during COVID-19 lockdown, studying the public discourse of anti-vaxxers, or monitoring urban nature policies, among many others...
Additional information
| Type | devroom |
|---|
More sessions
| 2/6/21 |
As technology advances, so as our maps. In this talk, we will explore the ever growing open map data that can help us understand, validate, and explore socio-economic indicators with the aid of network theory and machine learning techniques.
|
| 2/6/21 |
We will describe in this talk how to combine crowdsourcing approaches with scientific expertise in Digital Humanities projects, and some of the issues that are at stake. The talk will focus on Recital, a Digital Humanities project aiming at gaining insights on 18th-century theater through the analysis of its accounting books. It combines crowdsourcing, using the ScribeAPI free software, producing results that need to be evaluated and validated by scientific expertise, which requires appropriate ...
|
| 2/6/21 |
This talk will focus on our experiences with making open source tools for the study of social media platforms (amongst others, DMI-TCAT for Twitter, the YouTube Data Tools, and 4CAT for forum-like platforms such as Reddit and 4chan) in the context of social science and humanities research. We will discuss questions of reliability and reproducibility, but also how tools are taking part in shaping which questions are being asked and how research is done in practice - making open source ...
|
| 2/6/21 |
The World Wide Web’s original design as a vast open documentary space built around the concept of hypertext made it a fantastic research field to study networks of actors of a specific field or controversy and analyse their connectivity. Navicrawler, IssueCrawler, Hyphe... Over the past 15 years, a variety of web crawling tools, most often free and open source, have been developped by or for social sciences research labs across the world. They provide means to engage with the web as a research ...
|
| 2/6/21 |
PANDORÆ : Retrieving, curating and exploring enhanced corpi through time and space Mapping the state of research in a particular field has been made easier through commercial services providing API-based bibliometric-enhanced corpuses retrieval. Common assertions such as “the use of CRISPR technologies has skyrocketed in laboratories all around the world since 2012” can now be easily verified in both quantitative and qualitative perspectives using those platforms. Such services as ...
|
| 2/6/21 |
This is a live panel session which gathers speakers from three lightning talks about web mining tools and technologies.
|
| 2/6/21 |
This talk aims to give a user’s perspective on FLOSS tools for open research in social science. It will be based on personal experience with a team project that aimed to analyze the Twitter follow graph of last year’s FOSDEM and CHAOSScon participants. The project used open source tools and agile management: data was collected with a command line tool (Twarc), network visualization was done with Gephi, and Framagit provided a collaborative framework for managing code, data, visualization and ...
|
