Data Scientist / Programmer
The 2018 World Cup is upon us! If you’re tempted to do a little betting, or you’re taking part in a friendly forecast competition with friends or colleagues, read on. In this tutorial, we’ll learn how to use R and the ‘elo’ package to create Elo rankings for the 32 teams in the tournament, and how to use those rankings to predict the result of football matches.
I recently had to import a lot of CSV files into a MySQL database. Given that I didn’t know the data and the format of the files very well, I wrote this short R script. It prints a data.frame that indicates, for each variable in the file:
Anybody wanting to learn R from scratch in 2018 will find an incredible wealth of tutorials, interactive learning websites, and high-quality videos at their disposal—almost to a point where it’s difficult to know where to start! This is of course a good thing, and is mainly due to R’s quickly growing popularity, with a constant stream of new users from both industry and academia wanting to learn the fundamentals.
But I’ve found that once you reach a certain level of confidence with the language, it becomes more difficult to find material for intermediate/advanced users who wish to become really good at R programming. But these materials do exist—they just tend to be mentioned and highlighted less frequently by the community.
Hence this post, where I’ve tried to gather a variety of books, courses and resources that should be beneficial to you, if you’re at that level where you don’t need another tidyverse tutorial, but wish you could get advanced insights from seasoned R programmers.
The purpose of this tutorial is to show a concrete example of how web scraping can be used to build a dataset purely from an external, non-preformatted source of data.
Our example will be the website Fivebooks.com, which I’ve been using for many years to find book recommendations. As explained on the website itself, Fivebooks asks experts to recommend the five best books in their subject and explain their selection in an interview. Their archive consists of more than one thousand interviews (i.e. five thousand book recommendations), and they add two new interviews every week.
Our objective will be to use R, and in particular the rvest package, to
gather the entire list of books recommended on Fivebooks, and see which
ones are the most popular.
If your main data is stored in an SQL database, creating a connection to query this database directly from R can save you hours of tedious data exports. The process is usually straightforward, but I recently had to set up a connection to Ingres. Unfortunately, a simple Google query wasn’t quite enough to find good documentation, since Ingres isn’t as common as other relational database management systems these days.
Whether used in academia, industry or journalism, working with R involves importing and exporting a lot of data. While the basic functions to read and write files are known to all users, different methods have been developed over the years to optimise this process.
In this article, we’ll have a look at the most efficient ways to read and write permanent files (i.e. in plain-text formats such as CSV), and to save and load binary files, a solution often overlooked by R users but much better suited to regular analysis of a given dataset.
WHIP est un observatoire des dissensions internes aux groupes parlementaires de l’Assemblée nationale, sous la forme d’une application qui collecte et analyse tous les votes contraires au mot d’ordre de chaque groupe lors des scrutins publics.
Les données agrégées permettent de surveiller la tendance générale dans le temps, ainsi que l’apparition de “frondeurs” au sein de chaque groupe parlementaire. WHIP permet également de consulter l’historique des votes de chaque députée, et leur conformité avec le reste du groupe.
WHIP est accessible ici.