TechQA.

Convert(ImageMagick) killed when batch processing jpgs

32 views Asked by At

Linux CentOS7 Version: ImageMagick 6.9.11-22 Q16 x86_64 2020-06-29 https://imagemagick.org

I have a script that converts a set of jpgs into another set of jpgs.

convert -strip -interlace Line -resize 1024x1280\> -quality 100 temp/highres-*.jpg $Accession/${Accession,,}-%d.jpg

This has worked fine until I tried to convert a set of 223 files. After running for a couple minutes it returns "Killed"

running convert -list resource
Resource limits:
  Width: 214.7MP
  Height: 214.7MP
  List length: unlimited
  Area: 12.173GP
  Memory: 5.66844GiB
  Map: 11.3369GiB
  Disk: unlimited
  File: 768
  Thread: 5
  Throttle: 0
  Time: unlimited

passing -debug All, output of the last image

  open temp/highres-171.jpg[0] (Heap Memory, 1279x2105 20.5406MiB)
2024-01-16T01:25:31+00:00 0:09.886 8.520u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-171.jpg] JPEG Trace: "End Of Image"
2024-01-16T01:25:31+00:00 0:10.118 8.560u 6.9.11 Resource convert[62520]: resource.c/RelinquishMagickResource/1259/Resource
  Map: 263875B/0B/11.3369GiB
2024-01-16T01:25:33+00:00 0:11.353 8.720u 6.9.11 Policy convert[62520]: policy.c/IsRightsAuthorized/605/Policy
  Domain: Path; rights=Read; pattern="temp/highres-172.jpg" ...
2024-01-16T01:25:33+00:00 0:11.442 8.730u 6.9.11 Blob convert[62520]: blob.c/OpenBlob/2817/Blob
    read 3 magic header bytes
2024-01-16T01:25:33+00:00 0:11.621 8.740u 6.9.11 Resource convert[62520]: resource.c/AcquireMagickResource/439/Resource
  Map: 252111B/252111B/11.3369GiB
2024-01-16T01:25:33+00:00 0:11.695 8.750u 6.9.11 Resource convert[62520]: resource.c/RelinquishMagickResource/1259/Resource
  Map: 252111B/0B/11.3369GiB
2024-01-16T01:25:33+00:00 0:11.705 8.750u 6.9.11 Cache convert[62520]: cache.c/DestroyPixelCache/1175/Cache
  destroy 
2024-01-16T01:25:33+00:00 0:12.219 8.830u 6.9.11 Policy convert[62520]: policy.c/IsRightsAuthorized/605/Policy
  Domain: Path; rights=Read; pattern="temp/highres-172.jpg" ...
2024-01-16T01:25:33+00:00 0:12.248 8.830u 6.9.11 Blob convert[62520]: blob.c/OpenBlob/2817/Blob
    read 3 magic header bytes
2024-01-16T01:25:34+00:00 0:12.268 8.830u 6.9.11 Cache convert[62520]: cache.c/DestroyPixelCache/1175/Cache
  destroy 
2024-01-16T01:25:34+00:00 0:12.278 8.830u 6.9.11 Policy convert[62520]: policy.c/IsRightsAuthorized/605/Policy
  Domain: Coder; rights=Read; pattern="JPEG" ...
2024-01-16T01:25:34+00:00 0:12.445 8.850u 6.9.11 Policy convert[62520]: policy.c/IsRightsAuthorized/605/Policy
  Domain: Path; rights=Read; pattern="temp/highres-172.jpg" ...
2024-01-16T01:25:34+00:00 0:12.503 8.870u 6.9.11 Blob convert[62520]: blob.c/OpenBlob/2817/Blob
    read 3 magic header bytes
2024-01-16T01:25:34+00:00 0:12.711 8.890u 6.9.11 Resource convert[62520]: resource.c/AcquireMagickResource/439/Resource
  Map: 252111B/252111B/11.3369GiB
2024-01-16T01:25:34+00:00 0:12.847 8.900u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Start of Image"
2024-01-16T01:25:34+00:00 0:12.885 8.910u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "JFIF APP0 marker: version 1.01, density 150x150  1"
2024-01-16T01:25:34+00:00 0:12.891 8.910u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Quantization Table 0  precision 0"
2024-01-16T01:25:34+00:00 0:12.900 8.910u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Quantization Table 1  precision 0"
2024-01-16T01:25:34+00:00 0:12.909 8.910u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Start Of Frame 0xc0: width=1268, height=2097, components=3"
2024-01-16T01:25:34+00:00 0:12.927 8.910u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 1: 2hx2v q=0"
2024-01-16T01:25:34+00:00 0:12.988 8.930u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 2: 1hx1v q=1"
2024-01-16T01:25:34+00:00 0:13.047 8.940u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 3: 1hx1v q=1"
2024-01-16T01:25:34+00:00 0:13.144 8.950u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Huffman Table 0x00"
2024-01-16T01:25:34+00:00 0:13.207 8.960u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          0   1   5   1   1   1   1   1"
2024-01-16T01:25:35+00:00 0:13.266 8.970u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          1   0   0   0   0   0   0   0"
2024-01-16T01:25:35+00:00 0:13.327 8.980u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Huffman Table 0x10"
2024-01-16T01:25:35+00:00 0:13.398 8.990u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          0   2   1   3   3   2   4   3"
2024-01-16T01:25:35+00:00 0:13.465 9.000u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          5   5   4   4   0   0   1 125"
2024-01-16T01:25:35+00:00 0:13.510 9.010u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Huffman Table 0x01"
2024-01-16T01:25:35+00:00 0:13.628 9.020u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          0   3   1   1   1   1   1   1"
2024-01-16T01:25:35+00:00 0:13.659 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          1   1   1   0   0   0   0   0"
2024-01-16T01:25:35+00:00 0:13.667 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Define Huffman Table 0x11"
2024-01-16T01:25:35+00:00 0:13.675 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          0   2   1   2   4   4   3   4"
2024-01-16T01:25:35+00:00 0:13.684 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "          7   5   4   4   0   1   2 119"
2024-01-16T01:25:35+00:00 0:13.694 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "Start Of Scan: 3 components"
2024-01-16T01:25:35+00:00 0:13.717 9.030u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 1: dc=0 ac=0"
2024-01-16T01:25:35+00:00 0:13.809 9.050u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 2: dc=1 ac=1"
2024-01-16T01:25:35+00:00 0:13.872 9.050u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "    Component 3: dc=1 ac=1"
2024-01-16T01:25:35+00:00 0:13.933 9.060u 6.9.11 Coder convert[62520]: jpeg.c/JPEGWarningHandler/399/Coder
  [temp/highres-172.jpg] JPEG Trace: "  Ss=0, Se=63, Ah=0, Al=0"
2024-01-16T01:25:35+00:00 0:14.145 9.080u 6.9.11 Coder convert[62520]: jpeg.c/ReadJPEGImage_/1392/Coder
  Interlace: nonprogressive
2024-01-16T01:25:35+00:00 0:14.155 9.080u 6.9.11 Coder convert[62520]: jpeg.c/ReadJPEGImage_/1394/Coder
  Data precision: 8
2024-01-16T01:25:35+00:00 0:14.159 9.090u 6.9.11 Coder convert[62520]: jpeg.c/ReadJPEGImage_/1396/Coder
  Geometry: 1268x2097
2024-01-16T01:25:35+00:00 0:14.159 9.090u 6.9.11 Coder convert[62520]: jpeg.c/JPEGSetImageQuality/969/Coder
  Quality: 75 (exact)
2024-01-16T01:25:35+00:00 0:14.159 9.090u 6.9.11 Coder convert[62520]: jpeg.c/JPEGSetImageSamplingFactor/1059/Coder
  Colorspace: RGB
2024-01-16T01:25:35+00:00 0:14.160 9.090u 6.9.11 Coder convert[62520]: jpeg.c/JPEGSetImageSamplingFactor/1086/Coder
  Sampling Factors: 2x2,1x1,1x1
2024-01-16T01:25:35+00:00 0:14.162 9.090u 6.9.11 Resource convert[62520]: resource.c/AcquireMagickResource/439/Resource
  ListLength: 1B/1B/9.22337EB
2024-01-16T01:25:35+00:00 0:14.162 9.090u 6.9.11 Resource convert[62520]: resource.c/AcquireMagickResource/439/Resource
  Area: 2.659MB/2.659MB/12.1729GB
2024-01-16T01:25:35+00:00 0:14.163 9.090u 6.9.11 Resource convert[62520]: resource.c/AcquireMagickResource/439/Resource
  Memory: 20.2865MiB/3.37146GiB/5.66844GiB
2024-01-16T01:25:35+00:00 0:14.163 9.090u 6.9.11 Cache convert[62520]: cache.c/OpenPixelCache/3990/Cache
  open temp/highres-172.jpg[0] (Heap Memory, 1268x2097 20.2865MiB)
  Killed

It appears to die on highres-172.jpg, but no reason given?

I tried using a for loop

for file in temp/*.jpg; do
    f=${file%.*}
    convert -strip -interlace Line -resize 1024x1280\> -quality 100 temp/highres-*.jpg T11000/T11000-${f##*-}.jpg
done

passing one file at a time to convert and it still is killed, once it hits the first kill it is killed on every subsequent call to convert until I stop the process.

I'm mystified!

Scott

bash imagemagick jpeg imagemagick-convert
Original Q&A
1

There are 1 answers

0
ScottD On

I have solved the issue.

1.) a bone head mistake, not realizing the policy file entries were all by default commented out!

2.) Had to reboot server after making initial changes.

working like a champ now...

Scott

Related Questions in BASH

Related Questions in IMAGEMAGICK

Related Questions in JPEG

Related Questions in IMAGEMAGICK-CONVERT

Popular Questions

Popular Tags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json

Trending Questions