A data science blog
When I started collecting and analysing Australian Prime Minister Scott Morrison’s speeches and interview transcripts over a year ago for a blog post and Shiny app, I hardly believed he would still be Prime Minister come the next election. With the election today, this is very possibly Scott Morrison’s last day as PM. In honour of the occasion, this is a special edition post about ScoMo’s language use during the election campaign. Read More…
In my previous post I created a Shiny app for exploring the text of Australian Prime Minister Scott Morrison’s speeches and interviews. While I was pleased with the app, one thing I didn’t like was having to run scraping and processing scripts manually to keep the data feeding the app up to date. This post describes how I used GitLab CI/CD and and AWS S3 to automate my scripts and save their outputs so that I could use them in the ScoMoSearch app. Read More…
Since becoming Prime Minister in 2018, Scott Morrison has spoken publicly almost every day. Transcripts of all speeches, interviews, and press conferences are available on the Prime Minister’s website, which means we can analyse ScoMo’s language over this whole period (August 2018 - March 2021). This dataset includes 988 transcripts. In this post I will: Compare frequently used words and phrases before and during the pandemic. Read More…
To follow up my two previous posts where I analysed ingredient lists and ratings for dessert recipes, I decided to make a Shiny app to explore the data interactively. You can find the app here. (As a quick aside, to deploy my Shiny app on AWS I relied heavily on Charles Bordet’s excellent and comprehensive guide.) You can select an ingredient either by clicking a point on the plot or by selecting it from the dropdown. Read More…
In a previous post, I explored ingredient lists from dessert recipes from the website taste.com.au. In this post, I’ll be using that dataset to identify ingredients that influence a recipe’s rating (whether negatively or positively). As a reminder, my main questions are: How well does ingredient composition predict the rating of a recipe? Which individual ingredients contribute to high and low scores? Which combinations of ingredients contribute to high and low scores? Read More…
If you enjoy cooking, you might be familiar with a book called the Flavour Bible. You can look up an ingredient and find lists of other ingredients that go well with it, with an emphasis on interesting or unusual combinations. It’s an amazing resource for discovering creative combinations of ingredients. In this series of posts, I’ll try to create something similar using a dataset of ingredient lists and their ratings. Read More…
One consequence of not having ever learned programming in any systematic way is that sometimes I come across a very unexpected behaviour that I don’t really have the language to describe (or more problematically, to Google). Recently when this happened I was led down a very interesting rabbit hole to the idea of short circuit evaluation. Importantly, I learned that my mental model for how logical operators work was not quite right. Read More…
I’m a big fan of Doctor Who, so when the new series came out earlier this year I was inspired to do some exploratory analysis of scripts from the show. Doctor Who is about a time-travelling alien (the Doctor) who explores the universe with a series of companions, generally getting himself into lots of trouble and saving the world a few times along the way. The Doctor has the ability to regenerate instead of dying, which means that the show can fairly gracefully replace the actor playing him. Read More…
When I first started learning R, I hated it. My first programming language was Python, so R’s lack of list comprehensions and f-strings seemed like major shortcomings. Having enjoyed learning Python, I had assumed learning my second language would be similar. Instead, I had three frustrating failed attempts to get started with R over several years before I finally got it. I now enjoy programming in R, but it’s been a bumpy road. Read More…