Initializing, constructing and converting struct to byte array causes misalignment

Question

I am trying to design a data structure (I have made it much shorter to save space here but I think you get the idea) to be used for byte level communication:

/* PACKET.H */    
#define CM_HEADER_SIZE          3
#define CM_DATA_SIZE            16
#define CM_FOOTER_SIZE          3
#define CM_PACKET_SIZE       (CM_HEADER_SIZE + CM_DATA_SIZE + CM_FOOTER_SIZE)
// + some other definitions

typedef struct cm_header{  
  uint8_t       PacketStart;   //Start Indicator 0x5B [
  uint8_t       DeviceId;       //ID Of the device which is sending
  uint8_t       PacketType;
} CM_Header;

typedef struct cm_footer {
  uint16_t      DataCrc;        //CRC of the 'Data' part of CM_Packet
  uint8_t       PacketEnd;      //should be 0X5D or ]
} CM_Footer;

//Here I am trying to conver a few u8[4] tp u32 (4*u32 = 16 byte, hence data size)
typedef struct cm_data {
  union {
      struct{
        uint8_t           Value_0_0:2;
        uint8_t           Value_0_1:2;
        uint8_t           Value_0_2:2;
        uint8_t           Value_0_3:2;          
      };
    uint32_t              Value_0;
  };
  //same thing for Value_1, 2 and 3
} CM_Data;

typedef struct cm_packet {
  CM_Header Header;
  CM_Data Data;
  CM_Footer Footer;
} CM_Packet;

typedef struct cm_inittypedef{
  uint8_t               DeviceId;
  CM_Packet             Packet;        
} CM_InitTypeDef;

typedef struct cm_appendresult{
  uint8_t Result;
  uint8_t Reason;
} CM_AppendResult;

extern CM_InitTypeDef cmHandler;

The goal here is to make reliable structure for transmitting data over USB interface. At the end the CM_Packet should be converted to an uint8_t array and be given to data transmit register of an mcu (stm32).

In the main.c file I try to init the structure as well as some other stuff related to this packet:

/* MAIN.C */
uint8_t packet[CM_PACKET_SIZE];
int main(void) {
    //use the extern defined in packet.h to init the struct
    cmHandler.DeviceId = 0x01; //assign device id
    CM_Init(&cmHandler); //construct the handler
    //rest of stuff   

    while(1) {
        CM_GetPacket(&cmHandler, (uint8_t*)packet);
        CDC_Transmit_FS(&packet, CM_PACKET_SIZE);
    }
}

And here is the implementation of packet.h which screws up everything so bad. I added the packet[CM_PACKET_SIZE] to watch but it is like it is just being generated randomly. Sometimes by pure luck I can see in this array some of the values that I am interested in! but it is like 1% of the time!

/* PACKET.C */
CM_InitTypeDef cmHandler;

void CM_Init(CM_InitTypeDef *cm_initer) {
  cmHandler.DeviceId = cm_initer->DeviceId;

  static CM_Packet cmPacket;
  cmPacket.Header.DeviceId = cm_initer->DeviceId;
  cmPacket.Header.PacketStart = CM_START;
  cmPacket.Footer.PacketEnd = CM_END;
  cm_initer->Packet = cmPacket;
}

CM_AppendResult CM_AppendData(CM_InitTypeDef *handler, uint8_t identifier, 
                              uint8_t *data){    

    CM_AppendResult result;

    switch(identifier){
      case CM_VALUE_0:
        handler->Packet.Data.Value_0_0 = data[0];
        handler->Packet.Data.Value_0_1 = data[1];
        handler->Packet.Data.Value_0_2 = data[2];
        handler->Packet.Data.Value_0_3 = data[3];
        break;

       //Also cases for CM_VALUE_0, 1 , 2
       //to build up the CM_Data sturct of CM_Packet

    default:
      result.Result = CM_APPEND_FAILURE;
      result.Reason = CM_APPEND_CASE_ERROR;
      return result;
      break;

    }    

    result.Result = CM_APPEND_SUCCESS;
    result.Reason = 0x00;
    return result;
}

void CM_GetPacket(CM_InitTypeDef *handler, uint8_t *packet){
    //copy the whole struct in the given buffer and later send it to USB host
    memcpy(packet, &handler->Packet, sizeof(CM_PACKET_SIZE));
}

So, the problem is this code gives me 99% of the time random stuff. It never has the CM_START which is the start indicator of packet to the value I want to. But most of the time it has the CM_END byte correctly! I got really confused and cant find out the reason. Being working on an embedded platform which is hard to debugg I am kind of lost here...

Answer 1

If you transfer data to another (different) architecture, do not just pass a structure as a blob. That is the way to hell: endianess, alignment, padding bytes, etc. all can (and likely will) cause trouble.

Better serialize the struct in a conforming way, possily using some interpreted control stream so you do not have to write every field out manually. (But still use standard functions to generate that stream).

Some areas of potential or likely trouble:

• CM_Footer: The second field might very well start at a 32 or 64 bit boundary, so the preceeding field will be followed by padding. Also, the end of that struct is very likely to be padded by at least 1 bytes on a 32 bit architecture to allow for proper alignment if used in an array (the compiler does not care you if you actually need this). It might even be 8 byte aligned.
• CM_Header: Here you likely (not guaranteed) get one uint8_t with 4*2 bits with the ordering not standardized. The field my be followed by 3 unused bytes which are required for the uint32_t interprettion of the union.
• How do you guarantee the same endianess (for >uint8_t: high byte first or low byte first?) for host and target?
• In general, the structs/unions need not have the same layout for host and target. Even if the same compiler is used, their ABIs may differ, etc. Even if it is the same CPU, there might be other system constraints. Also, for some CPUs, different ABIs (application binary interface) exist.