Why doesn't Ceedling compile the files included in the production code?

[Parminder Singh]

This is my situation:

src/tracelog_manager/tracelog_manager.h:

#ifndef TRACELOG_MANAGER_H
#define TRACELOG_MANAGER_H

typedef enum {
    TRL_UNINITIALIZED,
    TRL_INITIALIZED,
    TRL_INIT_FAILED
} trl_status_t;

trl_status_t tracelog_manager_init();

#endif //TRACELOG_MANAGER_H

src/tracelog_manager/tracelog_manager.c:

#include "tracelog_manager.h"
#include "tracelog_interface.h"

trl_status_t tracelog_manager_init()
{
    return tracelog_interface_init();
}

src/port/tracelog_interface.h

#ifndef TRACELOG_INTERFACE_H
#define TRACELOG_INTERFACE_H
#include "tracelog_manager.h"

trl_status_t tracelog_interface_init();

#endif //TRACELOG_INTERFACE_H

src/port/tracelog_interface.c

#include "tracelog_interface.h"

trl_status_t tracelog_interface_init()
{
    return TRL_INITIALIZED;
}

test/tracelog_manager/test_tracelog_manager.c

#include "unity.h"
#include "tracelog_manager.h"

//#include "mock_tracelog_interface.h"
//#include "tracelog_interface.h"
//TEST_FILE("tracelog_interface.c")

void setUp(void)
{
}

void tearDown(void)
{
}

void test_just_init()
{
    trl_status_t status = tracelog_manager_init();
    TEST_ASSERT_EQUAL_INT(TRL_INITIALIZED, status);
}

What I'm experiencing is that tracelog_interface.c is not compiled, so during the linking phase I get the error undefined reference to 'tracelog_interface_init'.
To solve the problem I have to uncomment one of the 3 commented lines in test_tracelog_manager.c, but I don't like any of these 3 solutions.
Why do I need to explicitly tell Ceedling to compile the tracelog_intarface.c file?
Ceedling/gcc should already know from the list of files included by tracelog_manager.c that tracelog_interface.c is needed for compiling/linking.

[Mark VanderVoord]

Hi.

This is a good question, and one which speaks to what Ceedling can/n't guess. Ceedling is very good add making educated guesses about things, which sometimes makes people assume it can read their minds. But it can't... and in this case, for good reason.

Ceedling manages unit tests, interaction tests, and integration tests. We, as the test writers, often treat them interchangeably because it's convenient to use the same tools and the same frameworks for both. Why learn different tools for all of these things when there is so much overlap? But they are different things and therefore have different needs. A unit test wants to focus on a single unit (C file in this case). An interaction test wants to focus on how a single unit interacts with other units, but couldn't care less about how those other units behave internally. An integration test cares about how a collection (a pair or other small number of units) work together in detail.

Ceedling doesn't know which of these you're building until you tell it. You do this by specifying if you want the real version of a file (by either #includeing or using TEST_SOURCE_FILE), or a mock version of that file. It then understands what needs you have and can build the appropriate things in the background.

If you've worked with large C projects, you've probably run into the same issue with your release builds. The makefile (or whatever build system your using) needs to explicitly be set up to include the files required for the port / configuration that you're using. The difference is that these release builds usually require you to learn another tool (autoconf, or whatnot) to set these up, whereas here, Ceedling allows you to specify it through simple hints in your files.

I hope that makes sense?

[Parminder Singh]

It makes some sense, but not completely (I'm a noob with unit test frameworks).

If I #include mock_tracelog_interface.h Ceedling knows that I want to use the mock version of tracelog_interface.h (I want to do "unit test"?)

If I don't tell Ceedling to compile tracelog_interface.c but this is used by tracelog_manager.c, why can't Ceedling assume that I want to do an interaction test?

As for the integration tests, I still don't understand what they are.

[Mark VanderVoord]

Sorry about piling terminology onto the conversation. I don't mean to distract us too much. Hopefully I can clear this up a little instead of making it more confusing:

• unit test -- include one header or TEST_SOURCE_FILE which is getting tested... other headers can be included, but they don't have related source files or we're ignoring them. We're testing just one source file at a time.
• interaction test -- include one header or TEST_SOURCE_FILE which is getting tested, and one or (often) more mock_ headers. We're now testing how this unit interacts with others, but fully controlling how the other units act through mocks
• integration test -- include more than one header or TEST_SOURCE_FILE which are getting tested together. This is usually done if a couple units are made to work VERY closely together and are almost useless without eachother. Occasionally, these are very valuable tests to run. Often, it's a sign that you should redesign things to be more independent.

The more things you can push into "unit tests" the easier your code is going to be to maintain and reuse. They're fairly independent from one another, and the testing doesn't depend on other units.

This is a gray area for linking, though, which might be where your situation is: A true unit test doesn't really need the other source or mock modules... it just needed the header so it can understand types, defines, etc... In this case, you can often get away without including the header at all in the test... but sometimes the linker isn't happy with that (because it's looking for a missing function or whatnot). Then you're still stuck #including the header (or the mock header), because it's needed to compile. At that point, Ceedling builds the related source or mock because it's needed to link, even if it's not called from the test. It's just a bit of overhead added to keep the linker happy. In these situations, it's often better to #include the mock version of the header, just so that it doesn't include all the source (which might include another source file, which might include another, etc etc etc).

[Michael Karlesky]

@parmi93 To add a little more explanation towards your core question… The reason Ceedling doesn't do what you've asked about is because while it's theoretically possible in some cases, it's super hard to do practically speaking and all but impossible in many cases.

In order for Ceedling to understand what set of files should be built together with a test file requires a deep knowledge of the code that isn't readily available. Files include other files. Macros expand in complicated ways (that may impact #include statements themselves). Symbols can be externd in source code with an expectation that the symbol is defined elsewhere; in a release build this is usually no problem but becomes quite complex in a build of a subset of release code under test. Making sense of what should be compiled and linked gets all but impossible if your project includes parallel / overlapped trees of source code for different builds of the same project codebase.

Digging all this out requires spidering through all the code, parsing it, processing all macros and preprocessor statements, and building a knowledge graph to pull out the needed symbols, locations, and file paths. Even with that, because of how unit tests carve out areas of release code there could still be gaps. Next you need to worry about all the different C language standards and the various language extensions that proprietary compilers can include. There is no cross-platform tool able to do all that and pull all that knowledge out of a test and the files that its test executable build would require. The gcc tooling is the closest thing. But, even if we ignore proprietary extensions, the GNU preprocessor and compiler do not expose quite enough of the needed details to adequately do everything I've just mentioned. Even in the upcoming 0.32 release where we completely rebuilt Ceedling's concept of preprocessing — that heavily depends on the GNU cpp tool — we couldn't get to perfect, just mostly there.

The only way we could automatically cause Ceedling to have the required knowledge would be to mandate certain strict project layouts, naming conventions, etc. Only a small fraction of projects would meet this standard, and only a small number of developers would be willing to write new code with such constraints.

So, in the end, a test author must tell Ceedling which files should be a part of a test executable's build. We opted to do this by convention wherever possible and by code when the convention breaks rather than introduce another build file format that complements test code. You've already grasped the two conventions and one code option. On the upside, this acts as a kind of executable documentation and often tends to squeeze out ambiguity and bad practices in source code.

I am the original author of Ceedling. I can tell you that I very much wanted to create what you are asking about when I wrote the first version almost twenty years ago now. It's nearly impossible to do in a practical way.

Does this help answer your question?

[Parminder Singh]

Thank you both so much for your time and these detailed answers.
I just started using unit test frameworks, my knowledge is based only on the book "Test-Driven Development for Embedded C" by James W. Grenning, and I'm trying to apply all this to a new project.

[Michael Karlesky]

@parmi93 You're very welcome.

If you wouldn't mind sharing your experience on learning with Grenning's book, we'd love to hear it. We have our own educational resources on testing and using these tools, but they are packaged differently than a book. We need to put in some effort there.

We'd also love to hear anything you're willing to speak to if you're interested in commercial products and services built around the open source tools. We're moving in that direction because we've heard the market ask for it and because a revenue stream will greatly help develop and maintain the freely available tools better than they have been.

[e-mail] or [e-mail]