I’ve created a deeply middle-class Git scraper project which tracks the prices of a basket of goods sold by the British online supermarket, Ocado.
For example, Lurpak butter:
I’ve been looked for an excuse to use Git scraping for ages, and this idea came up as my wife and I were commiserating over how much food prices are increasing at the moment.
The project is the codeinthehole/food-scraper
repo — see,
in particular, the product overview file.
It uses Github actions to call a Python CLI application once a day that:
- Fetches prices for a list of products.
- Updates a price archive file if any prices have changed.
- Builds a price chart image file for each product (using
matplotlib
). - Updates an
overview.md
file that collates all the chart images.
Any changes are committed back to the repo.
Ultimately, it’s an act of contortion, but it’s neat to be able to build a simple scheduler project that doesn’t require a database and runs for free on Github.