Could not find module ‘Criterion.Main’ when trying to benchmark haskell

Question

I cannot get Criterion to work. I followed the tutorial here, installing Criterion by doing the following.

cabal update
cabal install -j --disable-tests criterion

when I try ghc -O Fibber.hs or ghc -O --make Fibber I receive the error

[1 of 1] Compiling Main             ( Fibber.hs, Fibber.o )

Fibber.hs:1:1: error:
    Could not find module ‘Criterion.Main’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
  |
1 | import Criterion.Main
  |

I expect tests and diagnostics to be run and outputted to the console and an HTML file. trying with stack results in the same error. I have solved this problem before with my own modules by including them in my .cabal file like so

library
  Exposed-Modules:
    Geometry.Sphere
    Geometry.Cuboid
    Geometry.Cube

But doing this same in this case does not resolve the issue. I have also tried this SO solution, but to no avail. I receive basically the same error:

/Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/Fibber.hs:3:1: error:
    Could not find module ‘CriterionMain’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
  |
3 | import CriterionMain
  | ^^^^^^^^^^^^^^^^^^^^
Received ExitFailure 1 when running
Raw command: /Users/me/.stack/programs/aarch64-osx/ghc-9.2.2/bin/ghc-9.2.2 -i -i/Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/ -hide-all-packages -fdiagnostics-color=always -packagebase -O2 /Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/Fibber.hs
Run from: /Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/
Standard output:

[1 of 1] Compiling Main             ( /Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/Fibber.hs, /Users/me/Documents/projects/haskell/performance/criterion-1.5.13.0/Criterion/Fibber.o )

this is my .cabal file.

name:           criterion
version:        1.5.13.0
synopsis:       Robust, reliable performance measurement and analysis
license:        BSD3
license-file:   LICENSE
author:         Bryan O'Sullivan <[email protected]>
maintainer:     Ryan Scott <[email protected]>
copyright:      2009-2016 Bryan O'Sullivan and others
category:       Development, Performance, Testing, Benchmarking
homepage:       http://www.serpentine.com/criterion
bug-reports:    https://github.com/haskell/criterion/issues
build-type:     Simple
cabal-version:  >= 1.10
extra-source-files:
  README.markdown
  changelog.md
  examples/LICENSE
  examples/*.cabal
  examples/*.hs
tested-with:
  GHC==7.4.2,
  GHC==7.6.3,
  GHC==7.8.4,
  GHC==7.10.3,
  GHC==8.0.2,
  GHC==8.2.2,
  GHC==8.4.4,
  GHC==8.6.5,
  GHC==8.8.4,
  GHC==8.10.7,
  GHC==9.0.2,
  GHC==9.2.1

data-files:
  templates/*.css
  templates/*.tpl
  templates/*.js

description:
  This library provides a powerful but simple way to measure software
  performance.  It provides both a framework for executing and
  analysing benchmarks and a set of driver functions that makes it
  easy to build and run benchmarks, and to analyse their results.
  .
  The fastest way to get started is to read the
  <http://www.serpentine.com/criterion/tutorial.html online tutorial>,
  followed by the documentation and examples in the "Criterion.Main"
  module.
  .
  For examples of the kinds of reports that criterion generates, see
  <http://www.serpentine.com/criterion the home page>.

flag fast
  description: compile without optimizations
  default: False
  manual: True

flag embed-data-files
  description: Embed the data files in the binary for a relocatable executable.
               (Warning: This will increase the executable size significantly.)
  default: False
  manual: True

library
  exposed-modules:
    Criterion
    Criterion.Analysis
    Criterion.IO
    Criterion.IO.Printf
    Criterion.Internal
    Criterion.Main
    Criterion.Main.Options
    Criterion.Monad
    Criterion.Report
    Criterion.Types

  other-modules:
    Criterion.Main.Options.Internal
    Criterion.Monad.Internal

  other-modules:
    Paths_criterion

  build-depends:
    -- TODO: Eventually, we should bump the lower version bounds to >=2 so that
    -- we can remove some CPP in Criterion.Report. See #247.
    aeson >= 1 && < 2.1,
    ansi-wl-pprint >= 0.6.7.2,
    base >= 4.5 && < 5,
    base-compat-batteries >= 0.10 && < 0.13,
    binary >= 0.5.1.0,
    binary-orphans >= 1.0.1 && < 1.1,
    bytestring >= 0.9 && < 1.0,
    cassava >= 0.3.0.0,
    code-page,
    containers,
    criterion-measurement >= 0.1.1.0 && < 0.2,
    deepseq >= 1.1.0.0,
    directory,
    exceptions >= 0.8.2 && < 0.11,
    filepath,
    Glob >= 0.7.2,
    microstache >= 1.0.1 && < 1.1,
    js-chart >= 2.9.4 && < 3,
    mtl >= 2,
    mwc-random >= 0.8.0.3,
    -- TODO: Depend on optparse-applicative-0.17 as the minimum (see #258)
    optparse-applicative >= 0.13 && < 0.18,
    parsec >= 3.1.0,
    statistics >= 0.14 && < 0.16,
    text >= 0.11,
    time,
    transformers,
    transformers-compat >= 0.6.4,
    vector >= 0.7.1,
    vector-algorithms >= 0.4
  if impl(ghc < 7.6)
    build-depends:
      ghc-prim
  if !impl(ghc >= 8.0)
    build-depends:
      fail == 4.9.*,
      semigroups

  default-language: Haskell2010
  ghc-options: -Wall -funbox-strict-fields
  if impl(ghc >= 6.8)
    ghc-options: -fwarn-tabs
  if flag(fast)
    ghc-options: -O0
  else
    ghc-options: -O2

  if flag(embed-data-files)
    other-modules: Criterion.EmbeddedData
    build-depends: file-embed < 0.1,
                   template-haskell
    cpp-options: "-DEMBED"

Executable criterion-report
  Default-Language:     Haskell2010
  GHC-Options:          -Wall -rtsopts
  Main-Is:              Report.hs
  Other-Modules:        Options
                        Paths_criterion
  Hs-Source-Dirs:       app

  Build-Depends:
    base,
    base-compat-batteries,
    criterion,
    optparse-applicative >= 0.13

  if impl(ghc < 7.6)
    build-depends:
      ghc-prim

  if !impl(ghc >= 8.0)
    build-depends:
      semigroups

test-suite sanity
  type:                 exitcode-stdio-1.0
  hs-source-dirs:       tests
  main-is:              Sanity.hs
  default-language:     Haskell2010
  ghc-options:          -Wall -rtsopts
  if flag(fast)
    ghc-options:        -O0
  else
    ghc-options:        -O2

  build-depends:
    HUnit,
    base,
    bytestring,
    criterion,
    deepseq,
    tasty,
    tasty-hunit

test-suite tests
  type:                 exitcode-stdio-1.0
  hs-source-dirs:       tests
  main-is:              Tests.hs
  default-language:     Haskell2010
  other-modules:        Properties

  ghc-options:
    -Wall -threaded     -O0 -rtsopts

  build-depends:
    QuickCheck >= 2.4,
    base,
    base-compat-batteries,
    criterion,
    statistics,
    HUnit,
    tasty,
    tasty-hunit,
    tasty-quickcheck,
    vector,
    aeson

test-suite cleanup
  type:                 exitcode-stdio-1.0
  hs-source-dirs:       tests
  default-language:     Haskell2010
  main-is:              Cleanup.hs

  ghc-options:
    -Wall -threaded     -O0 -rtsopts

  build-depends:
    HUnit,
    base,
    base-compat,
    bytestring,
    criterion,
    deepseq,
    directory,
    tasty,
    tasty-hunit

source-repository head
  type:     git
  location: https://github.com/haskell/criterion.git

Answer 1

One option is to use a dependency-solved REPL.

% cabal repl --build-depends criterion
> :l Fibber.hs

Another is to create a cabal package.

% mkdir Fibber
% mv Fibber.hs Fibber
% cd Fibber
% cabal init
<follow prompts, and say you're building an executable with filename Fibber.hs>
% vi Fibber.cabal
<add criterion to the build-depends: stanza>
% cabal run Fibber # or whatever executable name you gave it in the cabal init step

One advantage of the latter is that as you discover additional dependencies, you can make a record of all of them in a file, instead of having to remember to pass all of them every time when you start cabal repl. Another is that your code will be compiled, rather than interpreted; rather important for performance testing, which is presumably what you're doing if you need criterion!

Answer 2

5 minutes Cabal explanation

Just a brief and not complete explanation about how cabal works so you can understand the error.

A project is made up of components: libraries and "runnables" (made up name for the sake of explanation). Each component has its own set of dependencies under build-depends tag.

Libraries do not have an entry point, so they don't need a main function and do not need to specify main-is. A cabal file has only one public library which is the name of the package, at the top of the file tagged as name. You can have internal libraries, but thats another topic. In the criterion's cabal file you see:

• The name is criterion (first line)
• The library build configuration under library tag. (dependencies, exposed modules, etc...)

Runnables do need a main function and a module marked as the main-is. Runnables can be a executables, test suites or benchmarks. AFAIK, labeling a runnable as one of them, only affects the way cabal compiles/install them via commands: cabal build, cabal run, cabal test, cabal bench, etc... For example cabal build will compile (by default) the library component and the executable components generating binary files one per executable, but cabal test will compile the library component and the test components, and then run the tests. In theory you can use Criterion.Main.defaultMain as the main function for a normal executable (define as en executable component in cabal file), in this case, cabal will generate an standalone exe which will run the benchmark on execution.

In the criterion's cabal file you see on exectuable and three test suites:

• Executable named criterion-report under app folder
• Test suites sanity, test and cleanup under test folder with different main files each

Each of this components has different dependencies sets, and in particular, all of them depend on the criterion library which is the component defined in the same cabal file under section library, as explained above.

So what's goint on with my error?

So essentialy, you want the Fibber.hs file to be a "runnable" file, either as a standalone binary or as a benchmark suite in a project. Therefore, you need to add it to a cabal file as you prefer. As a suggestion, avoid adding it to criterion's cabal file otherwise you'll be compiling criterion from source. Follow @DanielWagner answer and create a new project in which Fibber.hs is a main file and add criterion to its dependencies, or use the dependency-repl alternative.