How can I configure Ceedling for an IAR Embedded Workbench project?

Question

I'm trying to develop a new feature for an embedded application and I'd like to do so using a test-driven approach.

The project is written in pure C and is being developed using IAR Embedded Workbench 6.60.1.5104. I'm targeting an LPC1788, which is a Cortex-M3 device, and all development is being done on a 64-bit Windows 7 machine. Right now I'm more in favour of getting unit tests to run on the PC rather than on the target hardware (RAM is quite limited).

I came across a useful book on the subject called Test Driven Development for Embedded C and that pointed me towards tools like Unity, CppUTest, Ceedling, etc. After looking into this stuff, I think my best choice is to configure Ceedling (which uses Unity) for my project. However, I'm not sure exactly what steps I need to take to configure Ceedling to work with my current IAR toolchain.

I've installed Ceedling and created the "blinky" example project and I'm trying to build and test it using the IAR toolchain. I've added iccarm.exe to my path and edited blinky/project.yml as given below:

---

# Notes:
# This is a fully tested project that demonstrates the use
# of a timer ISR to blink the on board LED of an Arduino UNO
:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: TRUE
  :use_auxiliary_dependencies: TRUE
  :build_root: build
  :release_build: TRUE
  :test_file_prefix: test_

#You'll have to specify these
:environment:
  - :mcu: atmega328p
  - :f_cpu: 16000000UL 
  - :serial_port: COM8  #change this to the serial port you are using!!!
  - :objcopy: avr-objcopy
  # Uncomment these lines if you are using windows and don't have these tools in your path
  # - :path:
    # - C:\mingw\bin
    # - C:\WinAVR-20100110\bin
    # - C:\WinAVR-20100110\utils\bin
    # - #{ENV['PATH']}

:extension:
  :executable: .bin

:release_build:
  :output: blinky

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support

:defines:
  # in order to add common defines:
  #  1) remove the trailing [] from the :common: section
  #  2) add entries to the :common: section (e.g. :test: has TEST defined)
  :commmon: &common_defines []
  :test:
    - *common_defines
    - TEST
  :test_preprocess:
    - *common_defines
    - TEST

:tools:
  :release_compiler:
    :executable: avr-gcc
    :arguments:
      - ${1}
      - -DTARGET
      - -DF_CPU=#{ENV['F_CPU']}
      - -mmcu=#{ENV['MCU']}
      - -Iinclude/
      - -Wall
      - -Os
      - -c
      - -o ${2}
  :release_linker:
    :executable: avr-gcc
    :arguments:
      - -mmcu=#{ENV['MCU']}
      - ${1}
      - -o ${2}.bin

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

:tools:
  :test_file_preprocessor:
    :executable: iccarm
    :name: 'IAR test file preprocessor'

  :test_includes_preprocessor:
    :executable: iccarm
    :name: 'IAR test includes preprocessor'

  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'

  :test_linker:
    :executable: iccarm
    :name: 'IAR test linker'

  :release_compiler:
    :executable: iccarm
    :name: 'IAR release compiler'

  :release_linker:
    :executable: iccarm
    :name: 'IAR release linker'

:plugins:
  :load_paths:
    - vendor/ceedling/plugins
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
...

The only difference between this and the default project.yml is the content under the second :tools section.

My guess is that I'm heading in the right direction, but I'm not sure whether iccarm.exe is the correct executable to use for all these parts of the toolchain and what arguments I need to pass.

If I can configure Ceedling to build and test the blinky project using an IAR toolchain, I'm hoping I should be able to apply the same configuration for my actual project. If I try running rake now, I get the following output:

$ rake


Test 'test_BlinkTask.c'
-----------------------
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - build/test/preprocess/files/test_BlinkTask.c
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_extractor.rb:18:in `readlines'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_extractor.rb:18:in `extract_base_file_from_preprocessed_expansion'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_file_handler.rb:14:in `preprocess_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:40:in `preprocess_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:12:in `block in setup'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_helper.rb:33:in `preprocess_test_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:25:in `preprocess_test_and_invoke_test_mocks'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/test_invoker.rb:42:in `block in setup_and_invoke'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/test_invoker.rb:32:in `setup_and_invoke'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/tasks_tests.rake:11:in `block (2 levels) in <top (required)>'
Tasks: TOP => default => test:all
(See full trace by running task with --trace)

--------------------
OVERALL TEST SUMMARY
--------------------

No tests executed.

I assume this is because the test file preprocessor should be copying test files under the build/test/preprocess/files directory, which currently doesn't happen.

After a bit of digging around I found this example configuration file for Unity that looks like it may be helpful. It's geared towards an IAR EW/Cortex M3 environment like the one I'm using. This may give some indication of what configuration options I need to specify in my Ceedling project.yml:

If I can get Ceedling to build and test the blinky project using an IAR toolchain, I'm hoping I can adapt it to work with my actual project. Any help would be appreciated.

Answer 1

It was a struggle but I believe I've managed to configure Ceedling to help test my project. Hopefully this will be useful to anyone else looking to use Ceedling within IAR projects.

The Ceedling CLI has a command (ceedling new <proj_name>) that allows you to create new projects with the structure Ceedling expects. You can also specify the name of an existing project in which case it only adds the necessary files to make it Ceedling-compatible, which is what I did with my project.

For reference, my project structure looked something like this after performing this step:

.
├── build
│   ├── artifacts
│   │   └── test
│   ├── docs
│   ├── exe
│   ├── list
│   ├── logs
│   ├── obj
│   ├── temp
│   └── test
│       ├── cache
│       ├── dependencies
│       ├── list.i
│       ├── mocks
│       ├── out
│       ├── results
│       ├── runners
│       └── tests.map
├── project.yml
├── rakefile.rb
├── src
│   └── main
│       ├── c
│       │   ├── canDatabase.c
│       ├── include
│       │   ├── canDatabase.h
│       ├── python
│       └── resources
├── test
│   ├── support
│   └── test_canDatabase.c
├── <my_project>.dep
├── <my_project>.ewd
├── <my_project>.ewp
├── <my_project>.eww
├── vendor
│   └── ceedling
│       ├── docs
│       ├── lib
│       ├── plugins
│       └── vendor
└── version.properties

After that, I looked over the reference manuals for the IAR tools and studied the output from IAR Embedded Workbench when building sample projects, as @user694733 suggested. I used this information to edit my project.yml as given below:

:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: FALSE
  :use_auxiliary_dependencies: TRUE
  :build_root: build
  :release_build: FALSE
  :test_file_prefix: test_

:environment:
  - :path:
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin'
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\common\bin'
    - #{ENV['PATH']}

:extension:
  :executable: .out

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/main/c/**
    - src/main/include/**
    - src/main/resources/**
  :support:
    - test/support

:defines:
  :commmon: &common_defines []
  :test:
    - *common_defines
    - TEST
  :test_preprocess:
    - *common_defines
    - TEST

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
    - :callback
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

:tools:
  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'
    :arguments:
      - -D _DLIB_FILE_DESCRIPTOR=1
      - --debug
      - --endian=little
      - --cpu=Cortex-M3
      - -e
      - --fpu=None
      - -Ol
      - --preprocess "build/test/list"
      - --dlib_config "C:/Program Files (x86)/IAR Systems/Embedded Workbench 6.5/arm/INC/c/DLib_Config_Normal.h"
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -o "${2}"
      - --diag_suppress=Pa050
      - '"${1}"'

  :test_linker:
    :executable: ilinkarm
    :name: 'IAR test linker'
    :arguments:
      - --vfe
      - --redirect _Printf=_PrintfFull
      - --redirect _Scanf=_ScanfFull
      - --semihosting
      - --config "C:/Program Files (x86)/IAR Systems/Embedded Workbench 6.5/arm/config/generic_cortex.icf"
      - --map "build/test/tests.map"
      - -o "${2}"
      - '"${1}"'

  :test_fixture:
    :executable: cspybat
    :name: 'CSpyBat test runner'
    :arguments:
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armproc.dll"'
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armsim2.dll"'
      - '"${1}"'
      - --plugin "C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armbat.dll"
      - --backend -B
      - --endian=little
      - --cpu=Cortex-M3
      - --fpu=None
      - --semihosting

:plugins:
  :load_paths:
    - vendor/ceedling/plugins
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
...

This seems to be a suitable configuration for testing code designed to work on a Cortex-M3 device.

I also edited rakefile.rb to ensure that the generated test files are cleaned before each test run, as this was necessary to have the test results get printed consistently.

PROJECT_CEEDLING_ROOT = "vendor/ceedling"
load "#{PROJECT_CEEDLING_ROOT}/lib/ceedling.rb"

Ceedling.load_project

task :default => %w[ clean test:all ]

I was then able to define and run unit tests. Below is an excerpt from test_canDatabase.c:

#include "unity.h"
#include "canDatabase.h"

uint32_t actualId;
uint8_t actualPayload[8];
uint8_t actualPayloadLen;
uint8_t actualCanPort;

void mockHandler(uint32_t id, uint8_t payload[8], uint8_t payloadLen, uint8_t canPort)
{
  actualId = id;
  actualPayloadLen = payloadLen;
  actualCanPort = canPort;

  for (int i=0; i < payloadLen; i++)
  {
    actualPayload[i] = payload[i];
  }
}

void setUp(void)
{
  actualId = 0;
  actualPayloadLen = 0;
  actualCanPort = 0;

  for (int i=0; i < 8; i++)
  {
    actualPayload[i] = 0;
  }

  CANDB_Init(mockHandler);
}

void tearDown(void) {}

void test_Register_Tx_Definition()
{
  // GIVEN a CAN Tx message definition.
  CAN_TX_MESSAGE_DEFINITION_T definition;
  definition.id = 0;

  // WHEN we register the definition in the CAN database.
  int err = CANDB_RegisterTxDefinition(definition);

  // THEN the database should return SUCCESS (0x0).
  TEST_ASSERT_EQUAL_MESSAGE(0x0, err, "Registration should succeed");
}

void test_Register_Tx_Definition_Twice()
{
  // GIVEN a CAN Tx message definition.
  CAN_TX_MESSAGE_DEFINITION_T definition;
  definition.id = 0;

  // WHEN we register the definition once.
  CANDB_RegisterTxDefinition(definition);

  // AND we register the definition again.
  int err = CANDB_RegisterTxDefinition(definition);

  // THEN the database should return SUCCESS (0x0).
  TEST_ASSERT_EQUAL_MESSAGE(0x0, err, "Re-registration should succeed");
}

I'm now able to run automated tests by invoking "ceedling" from a terminal (project root is the current working directory):

$ ceedling
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.


Cleaning build artifacts...
(For large projects, this task may take a long time to complete)



Test 'test_canDatabase.c'
-------------------------
Generating runner for test_canDatabase.c...
Compiling test_canDatabase_runner.c...
Compiling test_canDatabase.c...
Compiling unity.c...
Compiling canDatabase.c...
Compiling cmock.c...
Linking test_canDatabase.out...
Running test_canDatabase.out...

-----------
TEST OUTPUT
-----------
[test_canDatabase.c]
  - ""
  - "     IAR C-SPY Command Line Utility V6.6.0.2752"
  - "     Copyright 2000-2013 IAR Systems AB."
  - ""
  - ""

-------------------
FAILED TEST SUMMARY
-------------------
[test_canDatabase.c]
  Test: test_Register_More_Than_Max_Allowed_Definitions
  At line (84): "Expected 1 Was 0. Registration > CANDB_MAX_TX_DEFINITIONS should fail"

  Test: test_Activate_Tx_Definition_With_Hardcoded_Payload
  At line (124): "Expected 0x00000001 Was 0x00000000. Incorrect ID"

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  4
PASSED:  2
FAILED:  2
IGNORED: 0

Answer 2

For anyone configuring Ceedling for an IAR Embedded Workbench for Renesas RX project:

:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: FALSE
  :use_auxiliary_dependencies: TRUE
  :build_root: build
  #:release_build: FALSE
  :test_file_prefix: test_
  :which_ceedling: gem
  :ceedling_version: 0.31.1
  :default_tasks:
    - test:all

:environment:
- :path:
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 8.3_2\rx\bin'
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 8.3_2\common\bin'
    - #{ENV['PATH']}

:extension:
  :executable: .out

:paths:
  :test:
    - +:test/**
    - -:test/support
  :support:
    -  test/support
  :source:
    - src/**
:defines:
  :common: &common_defines []
  :test:
    - *common_defines
    - TEST
  :test_preprocess:
    - *common_defines
    - TEST

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: FALSE
  :plugins:
    - :ignore
    - :callback
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8
    
:tools:
  :test_compiler:
    :executable: iccrx
    :name: 'IAR C/C++ Compiler for RX'
    :arguments:
      - '"${1}"'
      - --dlib_config "C:\Program Files (x86)\IAR Systems\Embedded Workbench 8.3_2\rx\lib\dlrxfllf.h"
      - -D  MCU1_COMPILATION_FLAG 
      - -D _DLIB_FILE_DESCRIPTOR=1
      - -D _DLIB_FULL_LOCALE_SUPPORT=1
      - -D _DLIB_FORMATTED_MULTIBYTE=1
      - -D _DLIB_STRTOD_HEX_FLOAT=1
      - --debug # or - -r 
      - --endian=little
      - --core=RXv2
      - -e
      - -Ol
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -D$: COLLECTION_DEFINES_TEST_AND_VENDOR
      - -o "${2}"
      - --diag_suppress=Pa050
      - --diag_suppress=Pe826 
      - --diag_suppress=Pm011 
      - --diag_suppress=Pe161
      - --diag_suppress=Pe223
      - '"${1}"'

  :test_linker:
    :executable: ilinkrx #IAR ILINK Linker
    :name: 'IAR Linker for RX'
    :arguments:
      - --vfe  #Very Fast Executable mode.
      - --redirect _Printf=_PrintfFull
      - --redirect _Scanf=_ScanfFull
      - --debug_lib
      - --config "C:\Program Files (x86)\IAR Systems\Embedded Workbench 8.3_2\rx\config\lnkr5f565ne.icf" 
      - --map "build\test\tests.map"
      - -o "${2}"
      - '"${1}"'

  :test_fixture:
    :executable: cspybat
    :name: 'CSpyBat test runner'
    :arguments:
      - '"C:/Program Files (x86)/IAR Systems/Embedded Workbench 8.3_2/rx/bin/rxproc.dll"'
      - '"C:/Program Files (x86)/IAR Systems/Embedded Workbench 8.3_2/rx/bin/rxsim.dll"'
      - '"${1}"'
      - --plugin "C:/Program Files (x86)/IAR Systems/Embedded Workbench 8.3_2/rx/bin/rxbat.dll"
      - --backend -B
      - --double 32

  :release_compiler:
    :executable: iccrx
    :name: 'IAR release compiler'

  :release_linker:
    :executable: ilinkrx
    :name: 'IAR release linker'

  :test_file_preprocessor:
    :executable: iccrx
    :name: 'IAR RX Test File Preprocessor'
    :arguments: [] 

  :test_file_preprocessor_directives:
    :executable: iccrx 
    :name: 'IAR RX Test File Preprocessor Directives'
    :arguments: [] 

  :test_includes_preprocessor:
    :executable: iccrx 
    :name: 'IAR RX Test Includes Preprocessor'
    :arguments: [] 

:libraries:
  :placement: :end
  :flag: "-l${1}"
  :path_flag: "-L ${1}"
  :system: []    
  :test: []
  :release: []

:plugins:
  :load_paths:
    - "#{Ceedling.load_path}"
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
    - xml_tests_report

#defaults
:module_generator:
  :project_root: ./
  :source_root: src/ 
  :test_root: test/
...

Happy Testing !

Answer 3

Your answer was super helpful and I'm working on similar approach for Cortex-M4 with IAR 7.5 and thought of sharing my portions of project.yml additions.

:tools:
  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'
    :arguments:
      - -D _DLIB_FILE_DESCRIPTOR=1 
      - --diag_suppress Pa050,Pa082,Pa039,Pe186
      - --no_cse
      - --no_unroll
      - --no_inline
      - --no_code_motion
      - --no_tbaa
      - --no_clustering
      - --no_scheduling
      - --debug
      - --endian=little
      - --cpu=Cortex-M4
      - -e
      - --fpu=VFPv4_sp
      - --dlib_config "C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\INC\c\DLib_Config_Normal.h"
      - --preprocess "build/test/list"
      - -On
      - --c++
      - --no_exceptions
      - --no_rtti
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -o "${2}"
      - '"${1}"'

  :test_linker:
    :executable: ilinkarm
    :name: 'IAR test linker'
    :arguments:
      - --vfe
      - --redirect _Printf=_PrintfFull
      - --redirect _Scanf=_ScanfFull
      - --semihosting
      - --keep __checksum
      - --entry __iar_program_start
      - --place_holder __checksum,4,.checksum,4
      - --define_symbol __checksum_begin=0x8020000
      - --define_symbol __checksum_end=0x80dfffb
      - --no_exceptions 
      - --config "C:\Users\jseinfeld\RubymineProjects\m4\common\cb\proj\flash.icf"
      - --map "build/test/m4.map"
      - -o "${2}"
      - '"${1}"'

  :test_fixture:
    :executable: cspybat
    :name: 'CSpyBat test runner'
    :arguments:
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armproc.dll"'
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armsim2.dll"'
      - '"${1}"'
      - --plugin "C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armbat.dll"
      - --backend -B
      - --endian=little
      - --cpu=Cortex-M4
      - --fpu=VFPv4_sp
      - --semihosting