Read .xz file from S3 in C#

Question

I am trying to read file from S3 using below code:

 var dir = new S3DirectoryInfo(_amazonS3Client, bucketName, folderName);
                IS3FileSystemInfo[] files = dir.GetFileSystemInfos();
                if(files.Length > 0)
                {
                    _bucketKey = files[0].Name;
                    var request = new GetObjectRequest
                    {
                        BucketName = bucketName,
                        Key = $"{folderName}/{_bucketKey}"
                    };
                    using (GetObjectResponse response = await _amazonS3Client.GetObjectAsync(request))
                    using (Stream responseStream = response.ResponseStream)
                    using (var reader = new StreamReader(responseStream))
                    {
                        var responseBody = reader.ReadToEnd();
                        File.WriteAllText(newFilePath, responseBody);
                    }
                }

It is working fine for a non-compressed file, however, I need some suggestions on how could I read the .xz file? My file is like DummyData_2020-07-21.csv.xz

Answer 1

I found a solution using SharpCompress and below code worked for me

                    var request = new GetObjectRequest
                    {
                        BucketName = bucketName,
                        Key = $"{folderName}/{bucketKey}"
                    };

                    using (var obj = _amazonS3Client.GetObject(request))
                    {
                        obj.WriteResponseStreamToFile($"{fileLocation}{bucketKey}");
                    }

                    using (Stream xz = new XZStream(File.OpenRead($"{fileLocation}{bucketKey}")))
                    using (TextReader reader = new StreamReader(xz))
                    {
                        var responseBody = reader.ReadToEnd();
                        File.WriteAllText(newFilePath, responseBody);
                    }