chef-clients are unable to run docker cookbook api commands like docker_registry, docker_image and docker_container. The docker cli commands in the recipe are working fine and are able to connect and pull from dockerhub and private docker registry hosted on Nexus. chef-client error states “malformed header”, no new changes were made to chef-server, chef-client, cookbooks or any of the versions. Tried upgrading the chef-client and docker cookbook to a newer version but still same error. This happened to both dev and prod (separate chef-servers for both)nodes at almost the same time and clients started failing. About 40 nodes failing at sync with the same error “malformed header”.
chef-client = v15.1.36
docker cookbook = v8.0.1
Here is the error,
"status": "failure", "run_list": "["recipe[testinator::ufw]","recipe[testinator::smb]"]", "total_res_count": "33", "data": { "exception": { "class": "#<Excon::Error::Socket: docker_image[dperson/samba] (testinator::smb line 360) had an error: Excon::Error::Socket: malformed header (Excon::Error::ResponseParse)>", "message": "docker_image[dperson/samba] (testinator::smb line 360) had an error: Excon::Error::Socket: malformed header (Excon::Error::ResponseParse)", "backtrace":
This is a bug in Excon 0.108.0, which is used by Chef. Ref: malformed header (Excon::Error::ResponseParse) in version 0.108.0 · Issue #845 · excon/excon