i would love to used to use it change code in ways that compiles and see if test fails. Coverage metric sometimes doesn't really tell you if some piece of code is covered or not.
sesm 17 hours ago||
Coverage metric can tell if lines of code were executed, but they can't tell if execution result was checked.
taberiand 16 hours ago||
I believe that's called mutation testing. Using an LLM to perform the mutation sounds like a great idea
rgmerk 5 hours ago||
LLMs are not suitable for mutation testing. Mutation testing needs to be fast to be useful (because you need to generate and test a lot of mutated versions); an LLM-based mutator would be extremely slow as well as error-prone.