Why has the textfile's encoding changed from UTF-8 to ANSI (and how to solve this)?

Question

I'm sending a .txt file as an attachment from an Android device using Indy.

When I download this, the .txt file's encoding has changed from UTF-8 to ANSI and is displaying it's content in lines next to eachother instead of underneath eachother.

So what have I been doing wrong, and how to solve this?

Functional code used to send the mail:

Body := TStringList.Create;
IdSMTP := TIdSMTP.Create(nil);
SSLHandler:= TIdSSLIOHandlerSocketOpenSSL.Create(Form1);
IdSMTP.IOHandler:= SSLHandler;
IdSMTP.UseTLS:= utUseExplicitTLS;

IdSMTP.Host := 'smtp.gmail.com';
IdSMTP.Port := 587;
IdSMTP.AuthType := satDefault;
IdSMTP.Username := AppData.MailAddrSender;
IdSMTP.Password := Appdata.MailPassword;

IdMessage := TIdMessage.Create(nil);
IdMessage.From.Name := aName;
IdMessage.From.Address := AppData.MailAddrSender;
IdMessage.Subject := 'E-mail subject';

PathString := System.IOUtils.TPath.Combine(System.IOUtils.TPath.GetDocumentsPath, (PathString + 'Log.txt'));
Body.Add('Mail todays log,');
if FileExists(PathString) then
  TIdAttachmentFile.Create(IdMessage.MessageParts,PathString);
try
  for I := 0 to Body.Count -1 do
    IdMessage.Body.Add(Body.Strings[I]);
  IdEmailAddressItem := IdMessage.Recipients.Add;
  IdEmailAddressItem.Address := AppData.MailAddrReceiver;

  try
    IdSMTP.Connect;
    try
      if IdSMTP.Authenticate then
      IdSMTP.Send(IdMessage);

    finally
      IdMessage.Free;
      IdSMTP.Disconnect;
    end;

  except
    on E: Exception do
    //exception handling here
  end;
finally
  IdSMTP.Free;
  SSLHandler.DisposeOf;
  Body.DisposeOf;
end;

Thank you for your time.

Answer 1

When I download this, the .txt file's encoding has changed from UTF-8 to ANSI

When using TIdAttachmentFile, the raw bytes of the file are sent as-is. Indy does not change an attached file's encoding.

TIdAttachmentFile defaults its ContentType property value based on the extension of the filename that you pass to the constructor. In this case, the extension is .txt so the ContentType should be defaulting to text/plain. The text/... media type has a charset attribute associated with it, but you are not setting the attachment's Charset property in your code. Without that, the recipient will interpret the data using a default charset instead (typically us-ascii per RFC 822). So it is likely that the recipient is detecting the text/plain media type with a missing charset and is displaying the file data in ASCII/ANSI because it does not know the data is actually UTF-8 encoded instead (since the file is an attachment, the recipient should be saving the raw bytes as-is if saving the attachment to a new file).

If you know for a fact that the file you are attaching is encoded in UTF-8, you should be explicit about it:

if FileExists(PathString) then
begin
  Attachment := TIdAttachmentFile.Create(IdMessage.MessageParts, PathString);
  Attachment.ContentType := 'text/plain';
  Attachment.Charset := 'utf-8';
end;

and is displaying it's content in lines next to eachother instead of underneath eachother.

That sounds more like a linebreak issue than an encoding issue. For instance, if the original file on Android is using LF linebreaks, but the recipient only supports CRLF linebreaks instead.

If you want, you can have Indy normalize the linebreaks to CRLF. You would simply load the file data into a TIdText instead of a TIdAttachmentFile, and then be sure to set the TIdText.ContentDisposition property to attachment (otherwise it will default to inline), and set the TIdText.FileName, so the data is still treated as an attachment by the recipient:

if FileExists(PathString) then
begin
  Attachment := TIdText.Create(IdMessage.MessageParts, nil);
  Attachment.Body.LoadFromFile(PathString, TEncoding.UTF8);
  Attachment.ContentType := 'text/plain';
  Attachment.Charset := 'utf-8';
  Attachment.ContentDisposition := 'attachment';
  Attachment.FileName := ExtractFileName(PathString);
end;

Now, with all of that said, there are some other minor coding issues with your use of Indy's components in general. I would suggest something more like this instead:

try
  IdSMTP := TIdSMTP.Create(nil);
  try
    SSLHandler := TIdSSLIOHandlerSocketOpenSSL.Create(IdSMTP);
    IdSMTP.IOHandler := SSLHandler;
    IdSMTP.UseTLS := utUseExplicitTLS;

    IdSMTP.Host := 'smtp.gmail.com';
    IdSMTP.Port := 587;
    IdSMTP.AuthType := satDefault;
    IdSMTP.Username := AppData.MailAddrSender;
    IdSMTP.Password := Appdata.MailPassword;

    IdMessage := TIdMessage.Create(nil);
    try
      IdMessage.From.Name := aName;
      IdMessage.From.Address := AppData.MailAddrSender;
      IdMessage.Subject := 'E-mail subject';

      IdEmailAddressItem := IdMessage.Recipients.Add;
      IdEmailAddressItem.Address := AppData.MailAddrReceiver;

      IdMessage.Body.Add('Mail todays log,');

      PathString := System.IOUtils.TPath.Combine(System.IOUtils.TPath.GetDocumentsPath, PathString + 'Log.txt');
      if FileExists(PathString) then
      begin
        // or TIdText...
        Attachment := TIdAttachmentFile.Create(IdMessage.MessageParts, PathString);
        Attachment.ContentType := 'text/plain';
        Attachment.Charset := 'utf-8';
      end;

      IdSMTP.Connect;
      try
        IdSMTP.Send(IdMessage);
      finally
        IdSMTP.Disconnect;
      end;
    finally
      IdMessage.Free;
    end;
  finally
    IdSMTP.Free;
  end;
except
  on E: Exception do
    //exception handling here
end;