Machine Learning Inference on AWS Lambda Functions powered by AWS Graviton2 Processors

  • use cases of using serverless for inference
  • how, by using Graviton2, it becomes both faster and cheaper to use it for making predictions
  • benchmarks and the link to the repository with code and libraries where you can try them with your models

Serverless for machine learning

  • You really need to optimize speed for the response time
  • You have a model which requires GPU or a lot of RAM

Graviton2 AWS Lambda update

Benchmarks and links to the repo

  • Framework: Scikit-Learn
  • Data: Binary classification dataset generated synthetically
  • Model: SVM classifier
  • Train: 512 samples with different number of features
  • Test: 1024 samples with different number of features
  • Number of cycles: 40
  • Lambda Configuration: 10GB RAM, 5-minute limit
  • Training on average takes the same or less time on Graviton2 compared to x86-based Lambda
  • Inference takes more time on Graviton2 Lambda when the number of features is small and becomes faster than x86-based Lambda when the number of features is large
  • Graviton2 is definitely better at complex operations than x86-based Lambda by being both faster and cheaper.
  • Depending on your case you may need to use one or another so the best option is to run your workflow on both and check which one performs better. To check Lambda performance on your case feel free to use libraries from the repo https://github.com/ryfeus/lambda-packs

Conclusion

--

--

--

I'm a senior machine learning engineer at Instrumental, where I work on analytical models for the manufacturing industry, and AWS Machine Learning Hero.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rustem Feyzkhanov

Rustem Feyzkhanov

I'm a senior machine learning engineer at Instrumental, where I work on analytical models for the manufacturing industry, and AWS Machine Learning Hero.

More from Medium

Deploy Simple Business Rules in AWS Using SageMaker Hosting

Writing Custom Operators and Hooks for MWAA

Timeseries databases demystified

How to build a data lake from scratch — Part 2: Connecting the components