Around 2015, I discovered the very inspiring ideas promoted by the effective altruism community. As summarised here:

“Effective altruism is about answering one simple question: how can we use our resources to help others the most? Rather than just doing what feels right, we use evidence and careful analysis to find the very best causes to work on. But it’s no use answering the question unless you act on it. Effective altruism is about following through. It’s about being generous with your time and your money to do the most good you can.”

Effective altruists focus their efforts, for example, on optimal ways to fight extreme poverty, improve the long-term future, and reduce animal suffering. If you’re interested in learning more, I recommend this great introduction. I also published an interview with Will MacAskill, one of effective altruism’s key figures, on Five Books.

Since 2015, I have donated regularly to effective causes recommended by GiveWell and the Effective Altruism Funds. In 2018, I took the Giving What We Can Pledge, making a commitment to giving at least 10% of my income to the organisations that can do the most good with it.

“I recognise that I can use part of my income to do a significant amount of good. Since I can live well enough on a smaller income, I pledge that for the rest of my life or until the day I retire, I shall give at least ten percent of what I earn to whichever organisations can most effectively use it to improve the lives of others, now and in the years to come. I make this pledge freely, openly, and sincerely.” —The Giving What We Can Pledge

Between 2018 and 2021, I had the pleasure of being part of the editorial team at Five Books. I published several interviews on artificial intelligence, data, and programming, as well as a variety of other subjects:

• Historical Change and Economic Ideology, with Thomas Piketty
• Computer Science for Data Scientists, with Hadley Wickham
• Effective Altruism, with Will MacAskill
• Data Science, with Roger D. Peng
• Existential Risks, with The Centre for the Study of Existential Risk
• Learning Python and Data Science, with Vicki Boykis
• Computer Science and Programming, with Ana Bell
• Modern Japanese Literature, with Linda Flores

Five Books is a comprehensive library of knowledge, curating book recommendations on any topic you might want to read about. We ask leaders in a field to make book recommendations in their area of expertise and explain their choices in an interview. Our interviews range from engaging introductions to specialist subjects—such as the Dutch Masters, Globalization, “Anti-memoirs”, Existentialism, Violence (or its decline)—to in-depth explorations of more personal themes and inspirations, from the likes of John Kerry, Ian McEwan, Margaret Drabble and Elif Shafak.

The 2018 World Cup is upon us! If you’re tempted to do a little betting, or you’re taking part in a friendly forecast competition with friends or colleagues, read on. In this tutorial, we’ll learn how to use R and the ‘elo’ package to create Elo rankings for the 32 teams in the tournament, and how to use those rankings to predict the result of football matches.

I recently had to import a lot of CSV files into a MySQL database. Given that I didn’t know the data and the format of the files very well, I wrote this short R script. It prints a data.frame that indicates, for each variable in the file:

• Its data type in your R data.frame;
• Some more information about the data (range for integers and dates, maximum decimal places for floats, maximum length for strings);
• The corresponding data type in MySQL 8.0;
• Whether the column includes missing values.

Anybody wanting to learn R from scratch will find an incredible wealth of tutorials, interactive learning websites, and high-quality videos at their disposal—almost to a point where it’s difficult to know where to start! This is of course a good thing, and is mainly due to R’s quickly growing popularity, with a constant stream of new users from both industry and academia wanting to learn the fundamentals.

But I’ve found that once you reach a certain level of confidence with the language, it becomes more difficult to find material for intermediate/advanced users who wish to become really good at R programming. But these materials do exist—they just tend to be mentioned and highlighted less frequently by the community.

Hence this post, where I’ve tried to gather a variety of books, courses and resources that should be beneficial to you, if you’re at that level where you don’t need another tidyverse tutorial, but wish you could get advanced insights from seasoned R programmers.

The purpose of this tutorial is to show a concrete example of how web scraping can be used to build a dataset purely from an external, non-preformatted source of data.

Our example will be the website Fivebooks.com, which I’ve been using for many years to find book recommendations. As explained on the website itself, Fivebooks asks experts to recommend the five best books in their subject and explain their selection in an interview. Their archive consists of more than one thousand interviews (i.e. five thousand book recommendations), and they add two new interviews every week.

Our objective will be to use R, and in particular the rvest package, to gather the entire list of books recommended on Fivebooks, and see which ones are the most popular.

If your main data is stored in an SQL database, creating a connection to query this database directly from R can save you hours of tedious data exports. The process is usually straightforward, but I recently had to set up a connection to Ingres. Unfortunately, a simple Google query wasn’t quite enough to find good documentation, since Ingres isn’t as common as other relational database management systems these days.

Whether used in academia, industry or journalism, working with R involves importing and exporting a lot of data. While the basic functions to read and write files are known to all users, different methods have been developed over the years to optimise this process.

In this article, we’ll have a look at the most efficient ways to read and write permanent files (i.e. in plain-text formats such as CSV), and to save and load binary files, a solution often overlooked by R users but much better suited to regular analysis of a given dataset.