So I created a private podfile repository where I store my podfile with source linked to private pods (code below), I succsefully added source and pod lines into my Podfile and I was able to use this private SDK.
My problem is when I try to upload app to testflight I got an error saying that binary files are not permitted and indeed my archive contains .bundle file which I think shoudn't be there
Here is my podfile
Pod::Spec.new do |s|
s.name = "HigoSDK.precompiled"
s.version = "0.4.0"
s.summary = "HigoSDK for iOS"
s.description = <<-DESC
HigoSDK framework
DESC
s.homepage = "homepage_url"
s.license = "Summer Agency License"
s.author = "Summer Agency"
s.platform = :ios, "13.0"
s.swift_version = '5.0'
s.source = { :http => "url/higo-sdk-0.4.0.zip" }
s.vendored_frameworks = 'Precompiled/HigoSDK.xcframework'
s.resource = 'Precompiled/HigoSDK.bundle'
s.static_framework = true
s.dependency 'RxSwift'
s.dependency 'RxCocoa'
s.dependency 'DifferenceKit'
s.dependency 'Kingfisher'
s.dependency 'Swinject'
s.dependency 'KeychainAccess'
s.dependency 'SwiftJWT'
s.dependency 'lottie-ios'
And here is the screen from archive where u can see HigoSDK.bundle in wrong file format 
I'm new to swift development so I don't really know if I added this library correctly (propably not)