Donagh Horgan

Building Docker images from private GitHub repos

Donagh Horgan on docker | 25 Oct 2019

Docker has a nice build context feature that lets you specify a git repository URL as the location to build an image from. So, for instance, you can roll your own Redis image like this: docker build -t myredis:5 https://github.com/docker-library/redis.git#master:5.0 But…

Slides and code from my RebelCon talk

Donagh Horgan | 20 Jun 2019

For anyone who is interested, I've just posted slides for my talk here. You can also check out the source code for the pipeline I demonstrated on GitHub.…

Exporting environment variables from dotenv files

Donagh Horgan on bash | 07 Jun 2019

Here's a nice shortcut for bash that exports all environment variables from a dotenv (.env) file into the current terminal session.…

Configuring Airflow DAGs with YAML

Donagh Horgan on airflow | 17 Apr 2019

The Apache Airflow UI is nice to look at, but it's a pretty clunky way to manage your pipeline configuration. One alternative is to store your DAG configuration in YAML and use it to set the default configuration in the Airflow database when the DAG is first run.…

Cork Open Data Dashboard

Donagh Horgan on open data, influxdb, grafana, docker, projects, cork, analytics, parking, holt-winters | 31 Aug 2017

The Cork Open Data Dashboard is a visualisation and analysis tool for several of the datasets on Cork City's open data portal, built with open source tools including Docker, InfluxDB and Grafana. The dashboard is essentially an expanded and refined version of the parking dashboard I developed last year, and…

Rolling your own analytics dashboard with InfluxDB, Grafana and Docker

Donagh Horgan on docker, influxdb, grafana, analytics, cork, parking, open data | 30 Nov 2016

For a pet project, I decided to build my own dashboard for charting and analysing time series metrics. There are lots of options out there for doing this. For data aggregation and querying, you can use Graphite, Prometheus, InfluxDB, OpenTSDB, Riak, DalmatinerDB and more. And for visualisation, there's Graphite (again)…

Called Kian? It's probably because of this guy.

Donagh Horgan on kian egan, westlife, granger causality, linear regression | 19 Sep 2016

The number of babies called Kian in Ireland increased sharply after the band Westlife became popular. Is this relationship coincidental? Or is there some deeper meaning? The Central Statistics Office in Ireland provide various data sets on birth rates and names. Pairing this data with Westlife's single release history leads…

Strategies for controlling service startup order with Docker Compose

Donagh Horgan on docker | 06 Sep 2016

As of version 1.6.0, Docker Compose supports a depends_on option that allows you to specify dependencies between the various services that you have defined. For instance, if we specify that service B depends on service A, then Compose will start service A first, and then service B.…