Git LFS doesn't work anymore with Yocto: git doesn't fetch "large files" since we upgraded our tools to these versions:
Yocto kirkstone
gitlab 16.7.3 and gitlab 16.8
git version 2.34.1
git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
Kubuntu 22.04: Linux linux-dev 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
When I execute the Yocto command "bitbake tep2020ihm", I get multiple "Smudge" errors, like that one:
ERROR: tep2020ihm-1.22-r0 do_unpack: Bitbake Fetcher Error: FetchError( <...>) failed with exit code 128, output:\nDownloading 3rd-party/aghaf/lib/aghaf.dll (6.6 MB)\nError downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll
When I'm cloning the project manually (to another directory on the dev server, by typing manually the command "git clone https://...", it works, including the large files. When the Yocto fetcher does it, it does not work (smudge errors for large files).
If I put the mention "lfs=0" in the SRC_URI variable of the .bb file (the Yocto recipe for my project), the fetcher works, but instead of fetching the file's content, it fetches the "LFS blob" instead:
version https://git-lfs.github.com/spec/v1
oid sha256:dba4dab327a3b420509ac9f466acde6c657698ccb1854522d20d3cfb35a493c1
size 173735936
I'm producing below the result of the command "git lfs logs last" in case it may clarify a point that I forgot to mention:
git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1)
git version 2.34.1
$ git-lfs filter-process
Error downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd): batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution
Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd)
github.com/git-lfs/git-lfs/errors.newWrappedError
github.com/git-lfs/git-lfs/errors/types.go:225
github.com/git-lfs/git-lfs/errors.Wrapf
github.com/git-lfs/git-lfs/errors/errors.go:85
github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile
github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:119
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:78
github.com/git-lfs/git-lfs/commands.smudge
github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
github.com/git-lfs/git-lfs/commands/command_filter_process.go:122
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra/command.go:902
github.com/git-lfs/git-lfs/commands.Run
github.com/git-lfs/git-lfs/commands/run.go:105
main.main
github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
runtime/proc.go:250
runtime.goexit
runtime/asm_amd64.s:1571
Smudge error
github.com/git-lfs/git-lfs/errors.newWrappedError
github.com/git-lfs/git-lfs/errors/types.go:225
github.com/git-lfs/git-lfs/errors.NewSmudgeError
github.com/git-lfs/git-lfs/errors/types.go:311
github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge
github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:87
github.com/git-lfs/git-lfs/commands.smudge
github.com/git-lfs/git-lfs/commands/command_smudge.go:127
github.com/git-lfs/git-lfs/commands.filterCommand
github.com/git-lfs/git-lfs/commands/command_filter_process.go:122
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra/command.go:902
github.com/git-lfs/git-lfs/commands.Run
github.com/git-lfs/git-lfs/commands/run.go:105
main.main
github.com/git-lfs/git-lfs/git-lfs.go:33
runtime.main
runtime/proc.go:250
runtime.goexit
runtime/asm_amd64.s:1571
Current time in UTC:
2024-01-18 17:17:38
ENV:
LocalWorkingDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git
LocalGitDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git
LocalGitStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git
LocalMediaDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/objects
LocalReferenceDirs=
TempDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/tmp
ConcurrentTransfers=8
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic,lfs-standalone-file,ssh
UploadTransfers=basic,lfs-standalone-file,ssh
GIT_EXEC_PATH=/usr/lib/git-core
GIT_CEILING_DIRECTORIES=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0
GIT_PREFIX=
GIT_CONFIG_PARAMETERS='core.fsyncobjectfiles'='0' 'gc.autoDetach'='false' 'core.pager'='cat'
Client IP addresses:
There is a bug in yocto git fetcher which make that nobranch is incompatible with lfs option. you have to use the branch=master and lfs=1 to make it work.