C++ How to create byte[] array from file (I don't mean reading file byte by byte)?

Question

I have a problem I neither can solve on my own nor find answer anywhere. I have a file contains such a string:

01000000d08c9ddf0115d1118c7a00c04

I would like to read the file in the way, that I would do manually like that:

char fromFile[] =
"\x01\x00\x00\x00\xd0\x8c\x9d\xdf\x011\x5d\x11\x18\xc7\xa0\x0c\x04";

I would really appreciate any help.

I want to do it in C++ (the best would be vc++).

Thank You!

Answer 1

int t194(void)
{
   // imagine you have n pair of char, for simplicity, 
   // here n is 3 (you should recognize them)
   char pair1[] = "01"; // note:
   char pair2[] = "8c"; //   initialize with 3 char c-style strings
   char pair3[] = "c7"; //

   {
      // let us put these into a ram based stream, with spaces
      std::stringstream ss;
      ss << pair1 << " " << pair2 << " " << pair3;

      // each pair can now be extracted into
      // pre-declared int vars
      int i1 = 0;
      int i2 = 0;
      int i3 = 0;

      // use formatted extractor to convert
      ss >> i1 >> i2 >> i3;

      // show what happened (for debug only)
      std::cout << "Confirm1:"  << std::endl;
      std::cout << "i1: " << i1 << std::endl;
      std::cout << "i2: " << i2 << std::endl;
      std::cout << "i3: " << i3 << std::endl << std::endl;

      // output is:
      // Confirm1:
      // i1: 1
      // i2: 8
      // i3: 0
      // Shucks, not correct.
      // We know the default radix is base 10
      // I hope you can see that the input radix is wrong,
      // because c is not a decimal digit, 
      // the i2 and i3 conversions stops before the 'c'
   }

   // pre-delcare
   int i1 = 0;
   int i2 = 0;
   int i3 = 0;
   {
      // so we try again, with radix info added
      std::stringstream ss;
      ss << pair1 << " " << pair2 << " " << pair3; 
      // strings are already in hex, so we use them as is

      ss >> std::hex  // change radix to 16
         >> i1 >> i2 >> i3;

      // now show what happened
      std::cout << "Confirm2:"  << std::endl;
      std::cout << "i1: " << i1 << std::endl;
      std::cout << "i2: " << i2 << std::endl;
      std::cout << "i3: " << i3 << std::endl << std::endl;

      // output now:
      // i1: 1
      // i2: 140
      // i3: 199
      // not what you expected?  Though correct, 
      // now we can see we have the wrong radix for output

      // add output radix to cout stream
      std::cout << std::hex  // add radix info here!
                << "i1: " << i1 << std::endl
                // Note: only need to do once for std::cout
                << "i2: " << i2 << std::endl
                << "i3: " << i3 << std::endl << std::endl
                << std::dec;

      // output now looks correct, and easily comparable to input
      // i1: 1
      // i2: 8c
      // i3: c7

      // So: What next?
      //     read the entire string of hex input into a single string
      //     separate this into pairs of chars (perhaps using    
      //       string::substr())
      //     put space separated pairs into stringstream ss
      //     extract hex values until ss.eof()
      //     probably should add error checks
      // and, of course, figure out how to use a loop for these steps
      //
      // alternative to consider:
      //   read 1 char at a time, build a pairing, convert, repeat 
   }



   //
   // Eventually, you should get far enough to discover that the
   // extracts I have done are integers, but you want to pack them
   // into an array of binary bytes.
   //
   // You can go back, and recode to extract bytes (either 
   // unsigned char or uint8_t), which you might find interesting.
   //
   // Or ... because your input is hex, and the largest 2 char
   // value will be 0xff, and this fits into a single byte, you
   // can simply static_cast them (I use unsigned char)

   unsigned char bin[] = {static_cast<unsigned char>(i1),
                          static_cast<unsigned char>(i2),
                          static_cast<unsigned char>(i3) };

   // Now confirm by casting these back to ints to cout
   std::cout << "Confirm4:  "
             << std::hex << std::setw(2) << std::setfill('0')
             << static_cast<int>(bin[0]) << " "
             << static_cast<int>(bin[1]) << " "
             << static_cast<int>(bin[2]) << std::endl;

   // you also might consider a vector (and i prefer uint8_t)
   // because push_back operations does a lot of hidden work for you
   std::vector<uint8_t>  bytes;
   bytes.push_back(static_cast<uint8_t>(i1));
   bytes.push_back(static_cast<uint8_t>(i2));
   bytes.push_back(static_cast<uint8_t>(i3));

   // confirm
   std::cout << "\nConfirm5:  ";
   for (size_t i=0; i<bytes.size(); ++i)
      std::cout << std::hex << std::setw(2) << std::setfill(' ')
                << static_cast<int>(bytes[i]) << " ";
   std::cout << std::endl;

Note: The cout (or ss) of bytes or char can be confusing, not always giving the result you might expect. My background is embedded software, and I have surprisingly small experience making stream i/o of bytes work. Just saying this tends to bias my work when dealing with stream i/o.

   // other considerations:
   //
   // you might read 1 char at a time.  this can simplify
   //    your loop, possibly easier to debug
   //    ... would you have to detect and remove eoln?  i.e.  '\n'
   //    ... how would you handle a bad input 
   //        such as not hex char, odd char count in a line
   //
   // I would probably prefer to use getline(),
   //    it will read until eoln(), and discard the '\n'
   //    then in each string, loop char by char, creating char pairs, etc.
   //
   // Converting a vector<uint8_t> to char bytes[] can be an easier
   //    effort in some ways.  A vector<> guarantees that all the values
   //    contained are 'packed' back-to-back, and contiguous in
   //    memory, just right for binary stream output
   //
   //    vector.size() tells how many chars have been pushed
   //
   // NOTE: the formatted 'insert' operator ('<<') can not
   //       transfer binary data to a stream.  You must use
   //       stream::write() for binary output.
   //
   std::stringstream ssOut;

   // possible approach:
   // 1 step reinterpret_cast 
   // - a binary block output requires "const char*"

   const char* myBuff = reinterpret_cast<const char*>(&myBytes.front());

   ssOut.write(myBuff, myBytes.size()); 
   // block write puts binary info into stream

   // confirm
   std::cout << "\nConfirm6:  ";

   std::string s = ssOut.str();  // string with binary data

   for (size_t i=0; i<s.size(); ++i)
   {
      // because binary data is _not_ signed data, 
      // we need to 'cancel' the sign bit
      unsigned char ukar = static_cast<unsigned char>(s[i]); 

      // because formatted output would interpret some chars 
      //   (like null, or \n), we cast to int
      int  intVal = static_cast<int>(ukar);  

      // cast does not generate code

      // now the formatted 'insert' operator 
      // converts and displays what we want
      std::cout << std::hex << std::setw(2) << std::setfill('0') 
                << intVal << " ";
   }
   std::cout << std::endl;
   //
   //
   return (0);
} // int t194(void)

Answer 2

I think I'd use a proxy class for reading and writing the data. Unfortunately, the code for the manipulators involved is just a little on the verbose side (to put it mildly).

#include <vector>
#include <algorithm>
#include <iterator>
#include <iostream>
#include <iomanip>
#include <string>
#include <sstream>

struct byte {
    unsigned char ch;

    friend std::istream &operator>>(std::istream &is, byte &b) {
        std::string temp;
        if (is >> std::setw(2) >> std::setprecision(2) >> temp) 
            b.ch = std::stoi(temp, 0, 16);       
        return is;
    }

    friend std::ostream &operator<<(std::ostream &os, byte const &b) {
        return os << "\\x" << std::setw(2) << std::setfill('0') << std::setprecision(2) << std::hex << (int)b.ch;
    }
};

int main() {
    std::istringstream input("01000000d08c9ddf115d1118c7a00c04");
    std::ostringstream result;

    std::istream_iterator<byte> in(input), end;
    std::ostream_iterator<byte> out(result);

    std::copy(in, end, out);

    std::cout << result.str();
}

I do really dislike how verbose iomanipulators are, but other than that it seems pretty clean.

Answer 3

You can try a loop with fscanf

unsigned char b;    
fscanf(pFile, "%2x", &b);

---

Edit:

#define MAX_LINE_SIZE 128
FILE* pFile = fopen(...);
char fromFile[MAX_LINE_SIZE] = {0};
char b = 0;
int currentIndex = 0;
while (fscanf (pFile, "%2x", &b) > 0 && i < MAX_LINE_SIZE)
    fromFile[currentIndex++] = b;

Answer 4

The below snippet should be helpful!

std::ifstream input( "filePath", std::ios::binary );
std::vector<char> hex((
        std::istreambuf_iterator<char>(input)), 
        (std::istreambuf_iterator<char>()));
std::vector<char> bytes;
for (unsigned int i = 0; i < hex.size(); i += 2) {
    std::string byteString = hex.substr(i, 2);
    char byte = (char) strtol(byteString.c_str(), NULL, 16);
    bytes.push_back(byte);
}
char* byteArr = bytes.data()

Answer 5

Here is a function that takes a string as in your description, and outputs a string that has \x in front of each digit.

#include <iostream>
#include <algorithm>
#include <string>

std::string convertHex(const std::string& str)
{
    std::string retVal;
    std::string hexPrefix = "\\x";
    if (!str.empty())
    {
        std::string::const_iterator it = str.begin();
        do
        {
            if (std::distance(it, str.end()) == 1)
            {
                 retVal += hexPrefix + "0";
                 retVal += *(it);
                 ++it;
            }
            else
            {
                retVal += hexPrefix + std::string(it, it+2);
                it += 2;
            }
        } while (it != str.end());
    }
    return retVal;
}

using namespace std;

int main()
{
     cout << convertHex("01000000d08c9ddf0115d1118c7a00c04") << endl;
     cout << convertHex("015d");
}

Output:

\x01\x00\x00\x00\xd0\x8c\x9d\xdf\x01\x15\xd1\x11\x8c\x7a\x00\xc0\x04
\x01\x5d

Basically it is nothing more than a do-while loop. A string is built from each pair of characters encountered. If the number of characters left is 1 (meaning that there is only one digit), a "0" is added to the front of the digit.

Answer 6

The way I understand your question is that you want just the binary representation of the numbers, i.e. remove the ascii (or ebcdic) part. Your output array will be half the length of the input array.

Here is some crude pseudo code.

For each input char c:

if (isdigit(c)) c -= '0';
else if (isxdigit(c) c -= 'a' + 0xa;  //Need to check for isupper or islower)

Then, depending on the index of c in your input array:

if (! index % 2) output[outputindex] = (c << 4) & 0xf0;
else output[outputindex++] = c & 0x0f;