Need to read a file, of any extension one byte at a time XE5 into dynamic array

Question

I already tried to read the file into a TFileStream but that is where I got stuck the file is inserted into the TFileStream but I am unable to read the bytes of the file, I haven't programmed in a while, please help me.

I also tried to read it into a normal File

 var
   myFile    : File;
   byteArray : array of byte;
   oneByte   : byte;
   i, count  : Integer;

 begin
   // Try to open the Test.byt file for writing to
   AssignFile(myFile, 'C:\Users\theunie\Desktop\Toets\Test2.txt');

   // Reopen the file for reading only
   FileMode := fmOpenRead;
   Reset(myFile, 1);   // Now we define one record as 1 byte

   // Display the file contents
   // Start with a read of the first 6 bytes. 'count' is set to the
   // actual number read
   ShowMessage('Reading first set of bytes :');
   setlength(ByteArray,sizeof(myfile));
   BlockRead(myFile, byteArray, sizeof(myFile), count);

   // Display the byte values read
   for i := 0 to count do
     ShowMessage(IntToStr(byteArray[i]));

   // Now read one byte at a time to the end of the file
   ShowMessage('Reading remaining bytes :');
   while not Eof(myFile) do
   begin
     BlockRead(myFile, oneByte, 1);   // Read and display one byte at a time
     ShowMessage(IntToStr(oneByte));
   end;
   Freeandnil(byteArray);
   // Close the file for the last time
   CloseFile(myFile);
 end;

as well as this

  procedure TForm1.Button1Click(Sender: TObject);
var
tf  : TFileStream;  //My Filestream
ar  : array of byte;//The dynamic array I want to read it into
k   : integer;//count
s   : string;//I want to display this at the end
begin
k   := 0;
tf  := TFileStream.Create('C:\Users\Theunie\Desktop\Test2.txt',fmOpenReadwrite);
try
  inc(k);
  SetLength(ar,k);
  ar[k-1] := tf.Read(ar[k-1],tf.size);
finally
s   := inttostr(ar[0]) +';';
for k := 1 to length(ar) do
begin
  s := s + ';' + IntToStr(ar[k]);
end;
FreeAndNil(ar);
end;
RichEdit1.Lines.Add(s);
end;

Answer 1

Is the file large ? Can it fit into RAM all at once ?

You basically have two simple options to create a DynArray out of the file, but they are only recommended for small to middle files.

1: http://www.freepascal.org/docs-html/rtl/classes/tbytesstream.html

 var BS: TBytesStream; b: byte; L, i: integer;
 begin
   BS := TBytesStream.Create;
   try
     BS.LoadFromFile('c:\boot.ini');
     L := High(BS.Bytes);
     for i := 0 to L do begin
       b := BS.Bytes[i];
       ShowMessage( IntToStr( b ) );
     end;
   finally
     BS.Destroy;
   end;
 end;

2: use IOUtils classes

• http://docwiki.embarcadero.com/Libraries/XE8/en/System.IOUtils.TFile.ReadAllBytes
• http://www.malcolmgroves.com/blog/?p=447
• http://www.programdevelop.com/4641326/
• Why do the System.IOUtils functions and TStreamReader use fmShareCompat?

et cetera

 var BS: TBytes; b: byte; L, i: integer;
 begin
   BS := TFile.ReadAllBytes('c:\boot.ini');
   L := High(BS);
   for i := 0 to L do begin
     b := BS[i];
     ShowMessage( IntToStr( b ) );
   end;
 end;

---

Conversely, to save the array's content into the file you would use something like How to convert TBytes to Binary File? (using MemoryStream)

---

Regarding your attempts.

http://wiki.freepascal.org/File_Handling_In_Pascal

1. as it already was noted above, SizeOf has no relation to the files, that is the memory size of a File variable type. If you want to stick with old TurboPascal API, then you have to use FileSize function to set the size at once. For small files it would work okay, for large files the very approach "read it all to memory at once, then process" is wrong.

2. inc(k); SetLength(ar,k); - in a +1 loop - that is a very bad idea, it means a heap fragmentation and copying and re-copying and re-re-copying gorwing data buffer time and again. That is Length*Length/2 scaling, and also might be badly damaging heap memory structure (google about heap fragmentation).
   When you can - you need to check the FileSize in before and set the array to

3. FreeAndNil(byteArray); - totally wrong. Arrays are not objects. You can not use Destroy/Free/FreeAndNil over them. How to clean a dynarray then ? Well, you may just do nothing, as dynarrays are one of auto-ref-counted types, like strings and interfaces etc. As long as your procedure exits, Delphi would automatically free the memory from the local variables. However if you want to clean a dynarray in the middle of the procedure you can do it via SetLength( MyDynArray, 0 ) or a shortcut MyDynArray := nil

4. BlockRead(myFile, byteArray, sizeof(myFile), count) is wrong on misuse of SizeOf. But it has another fault also: ByteArray variable is basically a pointer, so it is just 4 (four!) bytes ( 8 bytes in Win64 code), so you just overwrite all the call-stack. You really should better use modern type-safe API instead. But if you want to stick with old unsafe low-level API then you have to be very clear upon the low-level implementation of the variables of different types. Basically you want to read the file content not into the pointer to buffer, but into the buffer being pointed at, so it should be like BlockRead(myFile, byteArray[0], MyFileAndArraySize, count). Then if only a part of your file was read - count < MyFileAndArraySize - you would BlockRead(myFile, byteArray[count], MyFileAndArraySize - count, count1), then BlockRead(myFile, byteArray[count+count1], MyFileAndArraySize - count - count1, count2) and so on. Tedious even when you would understand how bytes are running around in low-level types...

5. ar[k-1] := tf.Read(ar[k-1],tf.size); - that is just absolutely wretched. Check http://www.freepascal.org/docs-html/rtl/classes/tstream.read.html - the result is how many bytes were actually read. So instead of filling your array with the file content, you feel it with "how many bytes were read in one attempt?" instead. You better utilize tf.ReadBuffer procedure instead then.

Yet if you wanted to go via portions tf.Read it should be something like

k := 0;
SetLength(ar, tf.Size);
while k < tf.Size do begin
  k := k + tf.Read( ar[k], tfSize - k);
end;

But again, you have much easier tools to work with small files in modern Delphi

---

One more problem in your code is in

s   := inttostr(ar[0]) +';';
for k := 1 to length(ar) do
begin
  s := s + ';' + IntToStr(ar[k]);
end;

It is a so-called "one-off error".

While strings for hystorical reasons are indexed from 1 to Length(s) and usually do not have 0th element, dynarrays are not.

Dynamic arrays are indexed from 0 = Low(ArrayVarName) to High(ArrayVarName) = Length(ArrayVarName) - 1. So your loop tries to read the memory past the end of array, outside of array itself.

Another error is that you start it with TWO semicolons, like "10;;20;30;40....."

It is typical when you got tired or is not very attentive. So you'd better avoid indexing arrays at all. Below is the working code for turning dynamic array into string from Delphi XE2

procedure TForm1.Button1Click(Sender: TObject);
var DynamicArray: TBytes;
    SB: TStringBuilder; iSL: IJclStringList;
    s1,s2: string; b: byte;
begin
   DynamicArray := TBytes.Create( 10, 20, 30, 40, 50 );

   SB := TStringBuilder.Create;
   try
     for b in DynamicArray do begin
       if SB.Length > 0 then
          SB.Append( ';' );
       SB.Append( b );
     end;

     s1 := SB.ToString;
   finally
     SB.Destroy;  // you must do it in Delphi for Windows
   end;

   iSL := JclStringList();
   for b in DynamicArray do
     iSL.Add( IntToStr( b ) );
   s2 := iSL.Join( ';' );
   iSL := nil;  // you may skip it, Delphi would do on exit from procedure

   ShowMessage( 'Dynamic array of bytes to string'
                + ^M^J'   with Delphi RTL: ' + s1
                + ^M^J'   with J.E.D.I. Code Library: ' + s2);

end;

A bit more about dynamic arrays:

• https://delphihaven.wordpress.com/2009/12/09/dynamic-arrays-as-reference-types/
• http://blog.marcocantu.com/blog/2014_september_dynamic_arrays_delphixe7.html