![]() The easiest way to do this is to use the AWS CLI tool, so go over there and grab your version. The next step would be to create or publish a new AWS Lambda layerīased on our zipped dependencies. Windows users should have a command for zipping available in their context menu. Next, we zip this folder: zip -r scikit-learn.zip python. When the docker process has finished, you should have the scikit-learn package along with all dependencies within the newly created python folder. The reason for choosing python as output directory is due to the requirements by the AWS platform to put Python dependencies into either a python directory or into python/lib/python/site-packages. ![]() While this is happening inside the container, because the container works in a directory that is mounted, the output will also be available in the host system. All this happens inside a directory called foo, which is just a working directory we use inside the container so that we do not mess with other things. For our platform, we use Python 3.8, but there is support for all major Python versions, so you can easily change the version to your needs.Īfter docker has pulled the image, it will run the command pip install -r requirements.txt -t python, which will install all dependencies listed in the file requirements.txt to the target directory specified by the -t option. The lambci/lambda image offers various Python builds. ![]() As always with docker, the first time you run this command, the image has first to be downloaded from DockerHub. And of course, the most simple solution does also work: Just use an absolute path like C:/Users/user/scikit-learn-layer.ĭocker run will execute the given command pip install in a container that is based on the given image lambci/lambda. For the windows command line or the Anaconda Prompt you can use %CD%, for the Windows PowerShell it is $. This command is not available for Windows (but of course, again, in WSL). $(pwd) is a bash expression that allows to use the current working directory (that is what pwd returns) as a variable for another command (here the -v option of docker). Pip install -r requirements.txt -t python $ docker run -rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.8 \ Next, we run docker and mount our current working directory, assuming, you already have a working docker installation: Of course, feel free to add any specific version or constraint that suits your case. In our case, this is scikit-learn, so this is the only line in our requirements.txt file. Mirroring the AWS Lambda function base systemĪs mentioned in the first article, for mirroring the Lambda environment, the best available option is docker-lambda, which replicates the AWS Lambda environment almost identically.įirst, we create a new folder, for example scikit-learn-layer, and create a requirements.txt in which we put all the dependencies that we want later to be part of the Lambda layer. Easy, isn’t it? In the following, I am going to explain each of these steps. In general, the idea is to setup a system that is identical or close to the system AWS Lambda Layers are based on, then install the dependencies with pip like in any other Python project, and finally ship these dependencies as a Lambda layer. How we managed to setup a scikit-learn AWS Lambda Layer Create a new AWS Lambda function from the AWS Management Console.Connecting the layer to an AWS Lambda function.Mirroring the AWS Lambda function base system. ![]() How we managed to setup a scikit-learn AWS Lambda Layer.Once you understood the main steps, you are able to setup your own layers in no time. So I started a journey into the depth of AWS Lambda Layers. In particular, we wanted to support the well-known machine learning library scikit-learn. In this article you will learn about creating your own AWS Lambda Layer to support any Python package you may need. This article continues where How to Run External Python Libraries in AWS Cloud ended.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |