Debugging AIStatefulTask

Stil writing a testsuite for statefultask. This is extremely hard, since statefultask can do so much and so complex that it is unthinkable what it all can do and then test that. So instead, I have to do a "Monte Carlo" test: I just do (pseudo) random calls to its interface and that way hope to reach every possible internal state of the library (aka, reproduce every possible bug). This leads to a lot of assertions firing and so far they fired for a good reason: I'm not allowed to call everything at random; the "random" testsuite has to be made smarter so it avoids things that aren't allowed. I managed to get it working that a single threaded, single task runs indefinitely, switching randomly between two states. The next step is to 'insert' random calls simulated to be from other threads (at random points in the code, by adding a lot of macros at many points in the library that are hooks to do this simulation). The following is the debug output of the first assert that I get after inserting my first random cont() (from a call to 'probe'): // Only ever insert control function calls when we're not inside a critical area of mSubState. if (!m_sub_state_locked) { // It should only ever happen (by design of the task) that cont() is called when the task is idle. if (it->first.task_state.idle || !executing()) { // Insert a cont() once every 50 times. int randomnumber = std::uniform_int_distribution<>{0, 50}(m_rand); if (randomnumber == 0) cont(); } } The assert that fires is this one: // If this fails then the run state didn't change and neither idle() nor yield() was called (sub_state_r->idle is false). // Or, another thread called cont() or signalled() immediately after we called idle(). That is a race condition and that // other thread should have called advance_state() instead. ASSERT(!(need_new_run && !mYieldEngine && sub_state_r->run_state == run_state && !(sub_state_r->skip_idle || // advance_state was called. sub_state_r->aborted || // abort was called. sub_state_r->finished))); // finish was called. The debug output is: STATEFULTASK: Calling montecarlo->run() WARNING : Object file /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0 does not have debug info. Address lookups inside this object file will result in a function name only, not a source file location. 7f901d737700 STATEFULTASK: Entering AIAuxiliaryThread::mainloop() STATEFULTASK: Entering AIStatefulTask::run(0, NA, abort_parent = false, on_abort_signal_parent = true, default_engine = nullptr) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::reset() [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::multiplex(initial_run) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_reset) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_reset) [0x2a7f3d0] STATEFULTASK: Running state bs_reset [0x2a7f3d0] STATEFULTASK: Base state changed from bs_reset to bs_initialize; need_new_run = true [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_initialize) [0x2a7f3d0] STATEFULTASK: Running state bs_initialize [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::set_state(MonteCarlo_alpha) [0x2a7f3d0] STATEFULTASK: Base state changed from bs_initialize to bs_multiplex; need_new_run = true [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Running state bs_multiplex / MonteCarlo_alpha [0x2a7f3d0] NOTICE : randomnumber = 1 STATEFULTASK: Entering AIStatefulTask::idle() [0x2a7f3d0] NOTICE : Looped 63 times, calling cont(). STATEFULTASK: Entering AIStatefulTask::cont() [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::multiplex(schedule_run) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Running state bs_multiplex / MonteCarlo_alpha [0x2a7f3d0] NOTICE : randomnumber = 1 STATEFULTASK: Entering AIStatefulTask::idle() [0x2a7f3d0] NOTICE : Looped 16 times, calling cont(). STATEFULTASK: Entering AIStatefulTask::cont() [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::multiplex(schedule_run) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Running state bs_multiplex / MonteCarlo_alpha [0x2a7f3d0] NOTICE : randomnumber = 6 STATEFULTASK: Entering AIStatefulTask::advance_state(MonteCarlo_beta) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Copying advance_state to run_state, because it is larger [MonteCarlo_beta > MonteCarlo_alpha] STATEFULTASK: Running state bs_multiplex / MonteCarlo_beta [0x2a7f3d0] NOTICE : randomnumber = 2 STATEFULTASK: Entering AIStatefulTask::yield(gMainThreadEngine) [0x2a7f3d0] STATEFULTASK: Adding stateful task [0x2a7f3d0] to gMainThreadEngine STATEFULTASK: Entering AIStatefulTask::multiplex(normal_run) [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::begin_loop(bs_multiplex) [0x2a7f3d0] STATEFULTASK: Running state bs_multiplex / MonteCarlo_beta [0x2a7f3d0] NOTICE : randomnumber = 1 STATEFULTASK: Entering AIStatefulTask::idle() [0x2a7f3d0] STATEFULTASK: Entering AIStatefulTask::cont() [0x2a7f3d0] COREDUMP : /home/carlo/projects/feniks/montecarlo/montecarlo/statefultask/AIStatefulTask.cxx:552: void AIStatefulTask::multiplex(AIStatefulTask::event_type, AIEngine*): Assertion `!(need_new_run && !mYieldEngine && sub_state_r->run_state == run_state && !(sub_state_r->skip_idle || sub_state_r->aborted || sub_state_r->finished))' failed. The question is: is this assert correct? Did it fire because we're violating a rule on calling cont()? I guess it is... a call to cont() doesn't void the next call to idle(), as advance_state() does.