Production-Ready Python Serverless Web APIs

After weeks of intensive research and a dozen small projects, you have now decided to use Serverless Framework to develop a huge production Python web API and deploy it to AWS Lambda. But how exactly do you transition from the simple app structure you saw in tutorials to the mature production-ready form you need? That's exactly the same problem we faced over the past 24 months. In this talk, I will share my team's experience on how we solved this problem.

Directory Structure - how to arrange your Python code

  • Controllers, Handlers, Helpers, Models, Gateways: Building your app's directories for single responsibility between components
  • Multi-repository vs. Single-repository approach: The pros and cons
  • Lambdalith (monolith Lambda) vs. Lambda Functions

Python development in the context of AWS Lambda

  • Don't be afraid to use Exceptions - monitoring your app's performance with the error count in Lambda
  • Use print statements deliberately - using CloudWatch to create searchable logs and metric filters to create alarms out of your logs
  • Use AWS Lambda Layers to package Python packages with OS Native dependencies (i.e psycopg2, cryptography)

Tools of the trade: Leveraging AWS Services to reduce the development you have to do in-house and focus on the features that make your client's business stand out

  • Lambda - as your compute engine.
  • API Gateway - for your routes and data validation
  • DynamoDB - as your NoSQL database
  • SQS - for your queue service
  • Parameter Store - for storing configuration
  • Step Functions - orchestrating workflows in AWS
  • Cognito - using AWS' ready-made authentication service
  • ECS - when your workloads reach past 15mins, turn to ECS containers to finish the job

Black Belt Lambda: using Serverless Framework features to get ahead of Lambda's limitations and ensure best practices

  • Using AWS Layers
  • Prune your Lambda Applications regularly
  • Managing environment variables
  • Have a local environment