Snakemake workflows work independently but fail when imported as a module

Question

I have two working pipelines. The fist pipeline takes input from a config file and then creates a .csv output file. The second pipeline takes in the .csv output of the previous one. When I run these pipelines independently they work as expected. I am trying to combine these pipelines by importing them as modules in a third Snakefile. However the snakefile throws an error and I am unable to figure out what I am doing wrong.

My snakemake version is 8.6

The directory structure looks like this:

• combined_pipelines
  • pipeline 1
  • pipeline 2
  • combined-pipeline

Here is the content of the Snakefile in which I am importing the working pipelines.

from snakemake.utils import min_version
min_version("6.0")


module metadata:
    snakefile:
        "../../metadata/workflow/metadata.smk"
    replace_prefix: '../../metadata'
    config:
        'config/metadata.yaml'

use rule * from metadata as metadata_*

module download:
    snakefile:
        "../../download/workflow/downloads.smk"
    replace_prefix: "../../download"
    config:
        'config/download.yaml'

use rule * from download as download_*

rule all:
    input:
        rules.metadata_all.input,
        rules.download_all.input
    default_target: True

Here is the error I receive when running snakemake --cores 1 -n:

Traceback (most recent call last):
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/cli.py", line 1961, in args_to_api
    dag_api = workflow_api.dag(
              ^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 328, in dag
    return DAGApi(
           ^^^^^^^
  File "<string>", line 6, in __init__
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 438, in __post_init__
    self.workflow_api._workflow.dag_settings = self.dag_settings
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/api.py", line 385, in _workflow
    workflow.include(
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/workflow.py", line 1391, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/home/bayraktar/PycharmProjects/combined_pipelines/run_pipe/workflow/Snakefile", line 16, in <module>
    "../../download/workflow/downloads.smk"
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/workflow.py", line 2130, in decorate
    module.use_rules(
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/modules.py", line 82, in use_rules
    modifier = WorkflowModifier(
               ^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/modules.py", line 207, in __init__
    self.path_modifier = PathModifier(replace_prefix, prefix, workflow)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bayraktar/miniforge3/envs/snakemakefull/lib/python3.12/site-packages/snakemake/path_modifier.py", line 32, in __init__
    for prefix, replacement in replace_prefix.items():
                               ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'items'

Answer 1

The error message clearly explains the problem: the replace_prefix is expected to be a dictionary, not a string. And logically to replace something you need to specify what to replace and what to replace with. So it should look like that:

replace_prefix: {'../../metadata' : 'absolute_path_to_metadata'}