in the logfile i also see some suspicious message ‘syntax error on token ‘package’, import expected…’
looks like an internal problem, as the source compiles perfectly.
i find this message in all logs - independend of my sources,
Hi @dvholten
This seems to occur when writing tests, and you mention in the comment this is in all logs. I have a couple of follow up questions to help determine what’s triggering this:
- Does this also occur for every test?
- Are you able to get any tests from the plugin?
- For the method here, you are seeing an R013 output code - do you see other output codes for other methods?
Kind regards,
Jane @ Diffblue
well, things are tricky. it seems, i messed up my project’s libraries with some failed update and rollback.
i made an independent, fresh project and can produce Diffblue-tests in that like before.
i carefully checked and copied the relevant jar-files from the small, working project into ‘the real one’ - no success.
the initial environment check is green - the Diffblue app seems busy producing tests (and counts these in license-view), but only one test is actually written into the testfile. The logfile indicates, that it actually does some work (it starts many threads), but writing the results fails.
watching the many trials (&errors) i have the impression, that Diffblue works like this:
at first, a project-built is tried. after that, the environment is checked. after that, the jar-file corresponding to the requested source-file is used for some fuzzing. one or more tests per source-method are produced and all are cleverly merged into the output file.
what exactly is used during the test-generation phase? what is the role of Junit and Mockito? It’s clear, that these must be known to produce the right tests - but are these actually run to generate the tests?
i assume the problem to be in some remote indirect-indirect dependent library jar, not in the main level jars. Any idea how to check the dependents during test-generation?
greetings from sunny Hamburg
some more data…
i can produce 1 (one) test per class. usually, it is the first method, when i request tests for all methods in the class. when i request a test just for method 5, i get one test for method 5. further single requests are ignored (with that error-msg). when i delete the generated testfile, i can get another test for another method.
when everything worked, diffblue often generated more than one junit test per method, here i get just one.
it seems, the problem is writing the generated tests into a non-empty file.
Thank you for this in-depth analysis @dvholten
Following your comment we have investigated internally, and found it seems the reason you see this output is that there is a change which makes the java syntactically invalid in between Cover reading and trying to write the files.
Are you or intellij changing any test files while Cover is writing them?
To answer the other questions in your comments;
what exactly is used during the test-generation phase?
- Cover reads the Byte code
what is the role of Junit and Mockito? It’s clear, that these must be known to produce the right tests - but are these actually run to generate the tests?
- The tests Cover writes uses the Junit framework, and any mocking will use Mockito. Therefore, Cover will make sure those dependencies are present to ensure tests will compile and run later.
I hope that answers your questions, please let me know if you have any more.
Kind regards,
Jane @ Diffblue (from slightly rainy Oxford 
)
thank for resonse - on a friday afternoon…
no - i patiently wait for the diffblue to finish.
while investigating this thing, i encountered an intersting effect (which is gone in the meantime):
for some reason, i had two folders in intelliJ marked as test-root-source (tests and test).
generating tests for a class resultet in a ‘duplicate class’ warning from intelliJ, which turned out to be true: the short testClass (only the constructor) was written in /tests, while another class with same name and all the other tests was written into the /test - folder tree. It seems, the main thread, started from IntelliJ, uses one view of the world, while the worker-threads, which analyze all the other methods, have a different opinion. Of course, the environment should be tidy, but i am sure, there is something which could be enhanced in your code. (that was before the august update)
another question which came up: as Diffblue is an IntelliJ-plugin, i assume, it uses all the infrastructure of IntelliJ-project settings. What about Maven (or Ant). Is it really independent of external build-tools? What is going on in the initial build phase (before the environment check) - is it just a compilation to make sure that everything is valid? what is the target?
as i assumed some incompatible libraries in the project, i setup a new branch, checked out all sources, removed all library references and pulled them in one-by-one as maven dependencies. During that process, i made the error with the two test-trees. for a few hours i could produce some shiny new tests. somehow that broke again.
slowly, i am running out of ideas…
anyway - have a nice weekend and best greetings to the collegues
dvh
btw - who is fumbling with my sandbox? i turn it off in the plugin configuration, the environment-checker shows a warning, the generated tests change their messages, so fine, so good - but after a while, the sandbox is set enabled again.
not sure, if that has to do with IntelliJ-restart or something else.
bug or feature?
Thanks for bringing this to our attention - this is a bug, we found after each restart of IntelliJ the sandboxing option was returning to the default (on). Our engineers have located and resolved the issue - the fix will be available in the next version (2023.08.02) released next week. In the meantime; each time you restart IntelliJ, the sandboxing option will need to be manually switched off.
Jane @ Diffblue
hi jane
hope the weekend wasn’t too bad… silver medal is better that 4th place
thanks for info about the sandbox thing.
i the meantime, i am back to normal diffblue operation. i set up a fresh project, based on maven, battled with the dependencies and the ‘interesting’ interactions between IntelliJ and the maven pom-settings. little by little, i regained the testgeneration.
there is something wrong with IntelliJ project settings, the diffblue-logfile says, tests are generated in …/src/test/java - but they (sometimes) end up in …/target/generated-test-sources, which seemed to come from somewhere between IDE and maven-plugins - and which is a very volatile location.
Your boffins definetly should check the parameters, which determine the folder where the tests are written to. I am sure, there is more than one source, and the various sources are not always congruent. As the plugin depends on IntelliJ, the IDE and maven must be in sync (or, no maven at all - live was much simpler before that crept in)
IIRC, in my first mail i proposed to do something with @NotNull annotations to method parameters. I understand there are several of these annotations, perhaps with slightly different meaning, but IMO, diffblue should not create tests which stuff a null into an annotated method parameter.
i am happy that the test generation is productive again.
keep up the good work
dvh
1 Like