Kedro simple project run error: ModuleNotFoundError followed by ValueError

Question

Full code available on: https://github.com/braKhAbid/kedro_mnist

I am trying to setup a simple python ML project in Kedro. It is very simple, one pipeline of three nodes, a data loading node, a model computing node and a model evaluating node that just prints the accuracy. This is just to learn to use Kedro for an upcoming project.

Despite the simplicity I am struggling to make this work. Basically my kedro run outputs

ModuleNotFoundError: No module named 'kedro_mnist.pipelines.'

followed by

KeyError: 'pipeline'

The above exception was the direct cause of the following exception:

and finally

ValueError: Failed to find the pipeline named 'pipeline'. It needs to be generated and returned by the
'register_pipelines' function.

I tried debugging using Google and ChatGPT. ChatGPT made me create an additional hooks.py file where I register my pipeline, but it didn't solve the problem.

Any help would be greatly appreciated. I am using Kedro 0.18.13.

I expect normally a float value to be printed in the console.

The full error traceback is:

PS D:\Files\IA divers\kedro_mnist\kedro-mnist> kedro run --pipeline pipeline
[10/18/23 11:49:05] INFO     Kedro project kedro-mnist                                                   session.py:364
                    WARNING  c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\project\__init warnings.py:109
                             __.py:359: UserWarning: An error occurred while importing the
                             'kedro_mnist.pipelines..ipynb_checkpoints' module. Nothing defined therein
                             will be returned by 'find_pipelines'.

                             Traceback (most recent call last):
                               File
                             "c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\project\__ini
                             t__.py", line 357, in find_pipelines
                                 pipeline_module = importlib.import_module(pipeline_module_name)
                               File "c:\users\rashid\anaconda3\lib\importlib\__init__.py", line 127, in
                             import_module
                                 return _bootstrap._gcd_import(name[level:], package, level)
                               File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
                               File "<frozen importlib._bootstrap>", line 991, in _find_and_load
                               File "<frozen importlib._bootstrap>", line 961, in
                             _find_and_load_unlocked
                               File "<frozen importlib._bootstrap>", line 219, in
                             _call_with_frames_removed
                               File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
                               File "<frozen importlib._bootstrap>", line 991, in _find_and_load
                               File "<frozen importlib._bootstrap>", line 973, in
                             _find_and_load_unlocked
                             ModuleNotFoundError: No module named 'kedro_mnist.pipelines.'

                               warnings.warn(

┌─────────────────────────────── Traceback (most recent call last) ────────────────────────────────┐
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\session\session.py:381 in run        │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\project\__init__.py:137 in inner     │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
KeyError: 'pipeline'

The above exception was the direct cause of the following exception:

┌─────────────────────────────── Traceback (most recent call last) ────────────────────────────────┐
│ c:\users\rashid\anaconda3\lib\runpy.py:194 in _run_module_as_main                                │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\runpy.py:87 in _run_code                                           │
│                                                                                                  │
│ in <module>:7                                                                                    │
│                                                                                                  │
│   4 from kedro.framework.cli import main                                                         │
│   5 if __name__ == '__main__':                                                                   │
│   6 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ > 7 │   sys.exit(main())                                                                         │
│   8                                                                                              │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\cli\cli.py:211 in main               │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\click\core.py:1157 in __call__                       │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\cli\cli.py:139 in main               │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\click\core.py:1078 in main                           │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\click\core.py:1688 in invoke                         │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\click\core.py:1434 in invoke                         │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\click\core.py:783 in invoke                          │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\cli\project.py:453 in run            │
│                                                                                                  │
│ c:\users\rashid\anaconda3\lib\site-packages\kedro\framework\session\session.py:383 in run        │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
ValueError: Failed to find the pipeline named 'pipeline'. It needs to be generated and returned by the
'register_pipelines' function.

Answer 1

Can you share the repo it's quite hard to debug from this? I would also recommend (if you haven't already) to run through the Spaceflights tutorial.

Answer 2

Looks like the error comes from here.

And the problem seems to come from a .ipynb_checkpoints directory generated by Jupyter that is inside kedro_mnist/pipelines. In fact, I can reproduce locally.

This is a bug in Kedro: https://github.com/kedro-org/kedro/issues/3194