Preempting Flaky Tests via Non-Idempotent-Outcome Tests @ICSE '22

文章目录[隐藏]

Nonidiomatic test are one of Flaky tests category. In the previous TACAS work, the relation between polluters(modify the shared state) and victims(read from the shared state) are found but there exists a lot false positive for it to have Order dependent variables inside, from our previous tests on these testcases. Also for the latent polluters/victims that potentially read/write the variables. e.g.

//shared variables x,y,z are initialized into 0
void t1(){ assert x==0; } // victim
void t2(){ x = 0; } // polluter
void t3(){ assert y==0; } // latent-victim
void t4(){ z=1; } // latent-polluter
void t5(){ assert w=0;w=1;}

The NIO tests means passes in the first run but fails in the second when run twice consecutively.

The NIO will self pollutes the state that its own assertions depend on

  • NIO $\in$ latent-polluter ^ latent-victim $\in$ latent-writer

Examples


Real Examples