Matt Healy - Web Developer
Perth, Western Australia

A neat feature of Amazon Web Service's API Gateway service is that it can integrate directly with other AWS services. The most common use of API Gateway is to integrate directly with a Lambda function, typically to perform an action like update a DynamoDB table or send a message to an SQS queue. But, sometimes it is not actually necessary to use a Lambda function at all - by taking advantage of API Gateway's AWS service integration, we can avoid this intermediary step altogether and build a much more efficient and resilient architecture.

I recently had a need to provision a simple API endpoint that would accept a JSON payload and store the data to be processed later. Typically I would look to use API Gateway backed by a basic Lambda function to accept the JSON data and store it in an SQS queue. Instead, for this project I opted to use the API Gateway AWS service integration with the SQS

Read More

I recently wrote an API endpoint for a project at VaultRealEstate which required me to generate a month-by-month commission summary breakdown for a sales agent. The API endpoint accepts an arbitrary start and end month and should return a JSON object showing all the distinct months in that date range and the commission performance for those months (for example, for displaying a bar chart).

I found that this was actually not a trivial problem, and built-in Python libraries like datetime can't really solve this problem nicely. There are third party libraries like dateutil which can solve the problem, and normally I would immediately go for a library like this. However, this project is hosted on AWS Lambda and I'm conscious about the deployment size of the project getting bigger with each dependency introduced, so I really like to only introduce a dependency when it's necessary.

I found

Read More

Something awesome about using AWS is the ability to auto-scale your workloads. At the risk of over-simplifying it, you set up a Launch Configuration and an Autoscaling Group to which the Launch Configuration should apply. The Launch Configuration defines the EC2 instance type, security groups, base AMI image, and other attributes for your Autoscaling Group. Once this is all set up, you can have new machines spun up automatically based on the load experienced by your application.

Something not so awesome about using AWS is when you need to update the base AMI image for your launch configuration. For example, you may have installed a new required system package or code library to your application, and you need to ensure that all your instances now and in the future have this package installed. You create a new AMI image, but now what?

It turns out that for some reason AWS does not allow

Read More

S3 Sync - Danger!

I recently had to revisit the backup strategy for a web design business run by a friend of mine. The business only ran a single server, containing a MySQL database and Apache web server, serving about ten Wordpress websites for clients. We previously had a very simple backup script which ran every night. The script would create a tar archive of the entire htdocs area, and a MySQLdump of the database, and FTP it all across to an FTP account on a different server. This approach has a couple of drawbacks:

  1. It's insecure. The data is sent across the Internet via an insecure protocol, FTP. This could have been remedied by using a secure file transfer protocol such as SFTP.

  2. It's inefficient. Every night, we're sending an entire snapshot to the backup server. Obviously, there's a whole lot of data on the server that never changes, so there is really no need to back it up every night. To

Read More

foodloggr

It's been a while since I've written a post here, so I thought January 1st would be as good a time as any to write one.

I've been busy working on a little side project for the last few months. foodloggr is a basic web application which allows you to simply and easily track your food consumption.

The inspiration for creating foodloggr came in June 2015. I had decided to go the whole month of June without eating any junk food at all - no fast food, no soft drink, no alcohol, no sugary crap. To track my progress, I thought I would use an app. I'd used various calorie counting apps in the past, but I always gave up using them for various reasons:

  • too US centric
  • polluted with poor user-submitted content
  • takes too long to find a close enough match

"It shouldn't be this hard", I thought! That's when I decided to build it myself. A few months of evening and weekend work later,

Read More