Trying to run a jupyter notebook from AWS Lambda - getting: "errorMessage": "No such kernel named python3"

Question

I am trying to run a Jupyter notebook from Lambda using the following code which uses papermill:

import os
import boto3
import subprocess

# to add paths
import sys
# pip install custom package to /tmp/ and add to path
subprocess.call('pip install papermill -t /tmp/ --no-cache-dir'.split(), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
sys.path.insert(1, '/tmp/')

# papermill to execute notebook
import papermill as pm
def lambda_handler(event, context):
    s3 = boto3.resource('s3')
    print('Here')
    s3.meta.client.download_file("testappend","ForTrigger.ipynb", "/tmp/juptest.ipynb")
    print('Here')
    pm.execute_notebook('/tmp/juptest.ipynb', '/tmp/juptest_output.ipynb', kernel_name='python3')
    print('Here')
    s3_client.upload_file("/tmp/juptest_output.ipynb", "testappend","temp/ForTriggerOutput.ipynb") 

The program throws this error:

"errorMessage": "No such kernel named python3",
"errorType": "NoSuchKernel"

I am not sure how to go about finding a list of available kernels. Please help.

Thanks in advance.

Answer 1

I see you have code in your function to install just papermill. By default papermill will just install jupyter and ipykernel essential as part of its dependencies and that does not include python3 kernel. Add an explicit ipykernel install in your init code as well.

pip3 install ipykernel

Here is another way of running Jupyter Notebooks using Papermill as serverless function. You might find this useful.