Shake - Best way to delete intermediate files?

12 views Asked by At

I would like to have a Shake rule which deletes its own dependency — and passes lint mode.

There are multiple ways of doing this, but I am wondering whether there is a way to work with the build system, rather than hiding information from it.

For example, I have a directory of PNG files which I would like to batch-convert to JPEG XL. After successful conversion of an image, I would like to throw away the original PNG (accepting a small loss of quality in return for a 90% smaller file).

import Development.Shake
import Development.Shake.FilePath

main :: IO ()
main = shakeArgs shakeOptions $ do
  phony "convert" $ do
    pngs <- getDirectoryFiles "" ["*.png"]
    need [png -<.> "jxl" | png <- pngs]

  "*.jxl" %> \out -> do
    let src = out -<.> "png"
    need [src]
    putInfo $ "Converting " ++ src ++ " -> " ++ out
    cmd_ "cjxl --quiet" [src, out] -- This command is part of libjxl
    putInfo $ "Removing " ++ src
    liftIO $ removeFiles "." [src]

Running shake convert --lint produces the following messages:

Converting scan_00028.png -> scan_00028.jxl
# cjxl (for scan_00028.jxl)
Removing scan_00028.png
Converting scan_00027.png -> scan_00027.jxl
# cjxl (for scan_00027.jxl)
Removing scan_00027.png
Lint checking error - 3 values have changed since being depended upon:
  Key:  getDirectoryFiles  [scan_*.png]
  Old:  (scan_00027.png scan_00028.png,"")
  New:
       
  Key:  scan_00027.png
  Old:  (Just File {mod=0xCB980C50,size=0x3286CCB,digest=NEQ},"")
  New:  <missing>
       
  Key:  scan_00028.png
  Old:  (Just File {mod=0xCB980C50,size=0x2EEF34A,digest=NEQ},"")
  New:  <missing>

Is there a way I can declare to Shake that those dependencies have been removed intentionally?

I am aware of multiple ways to work around the lint errors, such as:

  • removeFilesAfter - this works, but is liable to fail when free disk space is low. I would prefer the PNG file to be removed promptly after conversion, rather than batched at the end of the build.
  • getDirectoryFilesIO "" ["*.png"] - just disabling dependency tracking is not what I want, I don't think.
  • shakeArgsPrune - effectively the same as removeFilesAfter.
  • withTempDir - the Action which produces the intermediate PNG file could also do conversion to JXL. But it seems messier to combine different things into one step. Image acquisition and conversion can't run in parallel. Also I wouldn't want to throw away the source image if format conversion fails.
  • Development.Shake.Forward - This would probably work well if the only thing I wanted was conversion. But I have other build rules, for which dependency information would be helpful.

Thanks

0

There are 0 answers