Aws

For several years now, I’ve been running all my AWS instances with a recent long-term support edition of Ubuntu server, ZFS as the root file system. Ubuntu 20.04 LTS was released today, so I’ve updated my Packer templates - the original inspiration for the amazon-ebssurrogate builder - to support it! Ubuntu 20.04 has some nice new features, including: ZFS on Linux version 0.8.3, Linux Kernel version 5.4, including io_uring and built-in support for WireGuard, Up-to-date compiler and runtime packages for a variety of platforms. [Read More]

Last year, I posted about how to build Ubuntu 16.04 AMIs using HashiCorp Packer’s ebs-surrogate builder. Since then Ubuntu 18.04 has been released, and the process has changed a little.

[Read More]

Some recent image building work I was doing required images based on Windows Server. One of my more popular posts, Windows AMIs Without the Tears, detailed the fraught endeavor of making the WinRM management system work for this purpose, but since then Microsoft have substantially improved the situation by committing engineering effort to porting OpenSSH to Windows.

[Read More]

For all applications of importance or significance, we recommend using ZFS. On bare metal servers, ZFS is king of the hill, but on AWS and Linux it is still gaining traction. Data integrity guarantees as well as features such as “instantaneous” snapshots, compression, quotas, and the ability to send/receive datasets make ZFS very compelling. In this post, we’re going to demonstrate how to build from-scratch AMIs booting Ubuntu Linux with a ZFS root file system.

[Read More]

IAM Policy Documents are ubiquitous in AWS - they are used not only for standalone policies you might attach to users or roles, but also for S3 bucket policies, SNS topic policies and more. Unfortunately, the JSON syntax can be error prone to hand write, and the default mechanism for creating policies in many configuration management tools is template rendering.

[Read More]

One of the easiest ways of building resilience into a system running in AWS is to use an autoscaling group. Generally speaking, I use one for any service which is required to self-heal - even when aiming to maintain a steady number of instances, as is desirable when running servers for Consul and Nomad, as well as a whole host of other clustered systems. Unhealthy instances can simply be replaced, usually without operator intervention, and launch configurations can be used to simplify upgrading clustered software one instance at a time.

[Read More]

Twice in the last couple of weeks I’ve helped automate build infrastructure in AWS, first for Event Store and then secondly for another company. Both times we got 90% of the way there using great tools like Terraform and Packer, and fell at the last hurdle: how do you build Windows images in an automated fashion (i.e. no point-and-click)?

[Read More]