Nick Hibberd

Share

Data Lake fisherman, Engineer at Ambiata
github.com/nhibberd
@nbhibberd

Drinking tea and writing Haskell at Ambiata.

YOW! Lambda Jam 2016 Brisbane

Immutable Infrastructure Deployment with Functional Design Patterns

TALK –  VIEW SLIDES WATCH VIDEO

Transforming a large stream of data from one form into another. This seems like the ideal task for functional programming, a clear input, transform, output pipeline. However, things get very complex, very quickly. Once that stream of data is hundreds of terabytes a day, requiring co-ordination of hundreds of servers, with hundreds of software configurations, the functions transforming the data are the least of our problems. How can we use functional programming and the tools we are familiar with to get the same properties we value from our data transformation functions when writing code to manage our infrastructure and deployment?

This talk will step through the complexity of building and operating a large scale continuous deployment and co-ordination system; from server administration and change management through to wrangling Amazon Web Services. We will delve into the implementation of our production deployment system (in Haskell), looking specifically at its functional design: including a simple but effective purely functional DSL for describing deployment interactions; the techniques we utilize to ensure auditability and correctness; as well as the overall patterns we use in the end-to-end design of our daemon in order to keep this an easy to understand and maintain piece of software.

From this talk, the audience will take away a knowledge of how to apply many functional programming techniques together in an effective way to build robust commercial software. On their own these techniques appear simple, but the demonstration of them in a real production system will hopefully provide the context required for the audience to actually apply these techniques in their own code.