Developing AWS Serverless Applications - Offline vs Deployed?
When you're developing a service project there are multiple different ways to check that what you’re building is working. You can use test driven design, manual testing with postman, or hook your app directly to the APIs that you're creating.
With all three of these processes you need to decide one thing: is your service going to be running locally or in AWS? We're going to be looking at the key advantages and disadvantages for both ways of working. Make sure you read to the end as I'm going to be giving my opinion on which of these is the best way of working.
If you prefer to learn by watching videos then check out the video version of this article here.
Offline VS Deployed in AWS
In this article we’re comparing developing and testing your serverless projects offline vs testing them directly in the cloud. When I’m talking about ‘testing’ I’m talking about when you’re writing a Lambda function or vtl query and you want to see if it will work. You might be tweaking a single line between each test and you want quick feedback on whether your last line of code does what you expect.
This is not talking about testing your main branch code. This should always be done in AWS. This is just about testing your changes as you write your code.
The two options are
- Using tools like Serverless Offline with Dynamo Local to run the whole system offline.
- Deploying every change to AWS to test it
Serverless Offline
Serverless offline is a plugin for the serverless framework that allows you to run your lambda functions and API Gateway locally. It uses your serverless config file and creates an API that is almost identical to what you’ll get once it’s deployed.
You can also use other plugins to extend this functionality. Plugins like DynamoDB Local allow you to run local versions of different AWS services.
If you want to learn to set up serverless offline you can follow this video tutorial.
Advantages
It's super fast - with webpack hot-reload it can be 0.2 seconds between saving a change and being able to test it and see the results. If you’re using Postman, then that’s just clicking one button to re-send a request.
All of the logs are shown in your terminal - super quick and easy.
You can seed the DB each time you start running offline so you know you always start with a consistent DB state. This can also making testing specific scenarios much easier.
Full Isolation - If one developer makes changes or calls their APIs, it won’t change the behaviour of anyone else on your team.
Runs offline - If you’re without good networking then this works just the same. This probably not much of a benefit nowardays but there have been times when travelling by plane or train where I’ve been able to develop and test applications with no internet connection.
Disadvantages
The biggest one is definitely that not all services have great mocks. Whilst Dynamo and API gateway have awesome mocks, if you want to test Cognito, Athena or Lex then you just can’t. Services like step functions or Eventbridge have mocks but require adding extra config or endpoints to your service. There are a lot of services mocked but you can’t guarantee that they’ll function identically to the real service.
If you work with multiple microservices then testing them working together can be complex. How does a deployed service call your service that is running locally?
Your local machine can affect how your service runs. You might have globally installed packages. Is your NodeJS or Python the exact same version as you’re going to run in AWS?
Deployed to AWS
Explaining this is pretty simple. You deploy every change you make to AWS and test real APIs that trigger all of the real AWS Services.
Advantages
Your testing against the real services so it is almost guarenteed to function identically in production.
No more “it works on my machine” comments or inconsistencies between environments.
You can test any service that AWS offers. This is a massive advantage if you're using services beyond the really common services.
If you’re just creating a function and testing as you go then you can re-deploy just the code for a single Lambda which takes a few seconds.
You can stream logs straight from cloudwatch to your terminal.
Disadvantages
If you’re working as part of a team conflicts can arise. If all development is done against one stage, you could update records in the DB that someone else is testing against. If two people are editing the same Lambda then you’re just over writing each other. This is also why testing this way is not 100% perfect. The reason that your change works may be that someone else has also made their changes to the DB or another lambda which make yours work.
When you make any changes to the serverless.yml/ts file, you have to do a full serverless deploy. This can take up to a minute if you’re working on a large repo with compile and webpack adding to the time. If you’re creating a new API then this should only happen once a feature, but if you’re doing something with step functions or other services then it could be a real frustration.
Lambda logs can be SLOW - whilst they do work, I tested and found that it took between 8-14s to see the log changes in my terminal. That’s WAY slower than running locally.
So what do I do?
So we’ve talked about the advantages and disadvantages of both local and deployed development processes. So which do I do?
The answer isn’t quite as black and white as always do X. It depends on the project and the type of work I’m doing in there.
If the work is based on creating some APIs that might interact with Dynamo and implement some business logic then I’m going with offline. The time from making a change to seeing the result (response and logs) is fractions of a second.
If I’m working with multiple micro-services or using aws services that don’t have as good mocks then I go with deploying. You just can’t reliably mock some of the AWS services. Things may go slightly slower but with this kind of work, you’re often doing more solution design than just development work so the extra time between testing can be good for thinking over what you’ve done or planning the next bit of the solution.
From start to finish on a project I’ll almost certainly be doing bits of both.