Improve detection of paths in GO_COV reports - #265
akash-manna-sky wants to merge 30 commits into
Conversation
… improve asInt method
| private String buildPackagePath(final List<String> parts, final int startIndex) { | ||
| return parts.subList(startIndex, parts.size() - 1).stream() | ||
| .collect(Collectors.joining(".")); |
There was a problem hiding this comment.
Replace with
String.join(".", parts.subList(startIndex, parts.size() - 1));
And extract "." as parameter so you don't need to duplicate with "/"
| if (parts.size() == 1) { | ||
| return new PathInfo(StringUtils.EMPTY, StringUtils.EMPTY, 0); | ||
| } | ||
| if (parts.size() == PATH_LENGTH_TWO) { |
There was a problem hiding this comment.
| if (parts.size() == PATH_LENGTH_TWO) { | |
| if (parts.size() == 2) { |
| if (parts.size() == PATH_LENGTH_TWO) { | ||
| return new PathInfo(parts.get(0), parts.get(0), 1); | ||
| } | ||
| if (parts.size() == PATH_LENGTH_THREE) { |
There was a problem hiding this comment.
PMD is sometimes too silly: add a suppression for this method.
| if (parts.size() == PATH_LENGTH_THREE) { | |
| if (parts.size() == 3) { |
| var module = new ModuleNode(moduleName); | ||
| modules.add(module); | ||
| return module; |
There was a problem hiding this comment.
While that seems to work technically, it is a bad practice to modify the elements of the parameter.
Can you add the result in the caller?
There was a problem hiding this comment.
This one still is open? Never ever change a parameter.
|
Can you check, if the new example in #263 works? |
…overage profile test
…er handling of path parts
| } | ||
|
|
||
| @SuppressWarnings("PMD.AvoidLiteralsInIfCondition") | ||
| private PathInfo determinePathStructure(final List<String> parts) { |
There was a problem hiding this comment.
This logic does not look correct, because the module name can be from "a" to "a/b/c/d/e/f/g" etc. The only way to determine what it actually is, is based on the information parsed from go.mod.
There was a problem hiding this comment.
PS: this approach as a fallback when go.mod is not available would be of course fine.
| ext/stat/minmax.go:9.27,11.3 1 1 | ||
| ext/stat/minmax.go:12.2,12.24 1 1 | ||
| hello.test/main.go:3.13,5.2 1 1 | ||
| hello.test/cpu/main.go:3.13,5.2 1 1 |
There was a problem hiding this comment.
Note, I updated the example to have an additional corner case for determining, which module to use.
… and update demo coverage profile
|
Please have a look. @uhafner |
Did you have a chance to look at the comments from @egonelbre? |
Yes, I looked his concern. My approach is basically heuristic-based approach. I was waiting for your suggestion. Anyway, I have a plan, will push the changes shortly. |
|
@akash-manna-sky https://github.com/egonelbre/exp/blob/main/coverage_example/parse/main.go contains Go code on how to handle module files and use them to parse coverage. Extracting the module name can be done via regex easily as well, because the go.mod doesn't have block comments. |
…path resolution and enhance path parsing logic
…plitting and improve go.mod parsing logic
…module registry initialization
|
Please review the changes @uhafner |
|
Is there anything need to update? Please review. @uhafner |
uhafner
left a comment
There was a problem hiding this comment.
Looks almost good now. I still find a hard to see what is working now differently. Can you please add more text to the description in the PR as the current text does not say anything useful for users.
| private static final long serialVersionUID = -4511292826873362408L; | ||
|
|
||
| private static final PathUtil PATH_UTIL = new PathUtil(); | ||
| private static final Pattern PATH_SEPARATOR = Pattern.compile("/"); |
| if (org == null) { | ||
| private String determineContainerName(final String fullPath) { | ||
| var normalizedPath = fullPath.replace('\\', '/'); | ||
| var parts = Arrays.stream(PATH_SEPARATOR.split(normalizedPath)).toList(); |
There was a problem hiding this comment.
| var parts = Arrays.stream(PATH_SEPARATOR.split(normalizedPath)).toList(); | |
| var parts = StringUtils.split(normalizedPath, PATH_SEPARATOR); |
| if (parts.isEmpty()) { | ||
| return StringUtils.EMPTY; | ||
| } | ||
| return org + "/"; | ||
|
|
||
| if (parts.size() >= 3 && parts.get(0).contains(".")) { | ||
| return parts.get(0) + "/" + parts.get(1); | ||
| } | ||
|
|
||
| return parts.get(0); |
There was a problem hiding this comment.
It makes no sense to convert to a list, just use the array
| var module = new ModuleNode(moduleName); | ||
| modules.add(module); | ||
| return module; |
There was a problem hiding this comment.
This one still is open? Never ever change a parameter.
| return createPathPartsFromModule(normalizedPath, moduleMatch); | ||
| } | ||
|
|
||
| var parts = Arrays.stream(PATH_SEPARATOR.split(normalizedPath)).toList(); |
…ling and simplify module retrieval logic
|
I added a short PR description regarding this PR. I also implemented your requested changes. Please review the changes. @uhafner |
| * Creates an empty module registry. | ||
| */ | ||
| ModuleRegistry() { | ||
| this.modules = new ArrayList<>(); |
There was a problem hiding this comment.
The module registry does not make sense. It has a list of modules that is always empty. Can you check what the registry is good for?
egonelbre
left a comment
There was a problem hiding this comment.
It looks like the latest change completely removes the module handling code, without module handling an accurate path cannot be resolved.
I should mention there does not seem to be any tests for having multiple module files and a corresponding coverprofile. This should make it easier to see when essential pieces are being removed.
https://github.com/egonelbre/exp/tree/main/coverage_example/demo contains a decent example that should work with the expected output shown in #263 (comment)
|
What should I do now? @uhafner |
Did you already have a look at the example? Does your code correctly resolve the files? I did not find time yet to make me familiar with the reports. Maybe it makes sense to go a step back and re-evaluate the problem? I try to find some time to look into the problem a little be deeper. |
|
Another option would be to ask in the corresponding channels of the GoCov project, how the mapping should be and why they do not include the corresponding information in the report format. All other coverage tools in the world report their files correctly with absolute or relative paths, I see no reason why GoCov should not follow the same approach. |
|
The relevant change was done in https://go-review.googlesource.com/c/go/+/122478. The main reasoning for using package path instead of filenames is that a lot of go tooling use package path as input. Also, the package path allows to transfer the coverprofile to other computers and have it still correctly resolve to correct absolute path (assuming you are on the exact same commit). As for finding the correct mapping -- one approach I didn't think of earlier is to let people build the "module name" -> "directory" mapping with Of course that would be a little bit more inconvenient to setup than passing in the relevant go.mod files. The conversion logic from package path to absolute path would still stay the same though as I previously suggested. |
Yes, it seems my code resolve the files. And please take your time and look into the problem. I will commit the changes after your suggestions. I am trying to figure out the solutions and looking into the |
| assertThat(report.getAllFileNodes()).map(FileNode::getFileName).containsExactlyInAnyOrder( | ||
| "sum.go", "minmax.go", "main.go", "main.go"); | ||
| } | ||
| } |
There was a problem hiding this comment.
It looks like the test case I outlined in #263 (comment) is missing. Similarly I don't see any go.mod in the testsuite or module parsing, which are necessary to correctly resolve paths.
☀️ Quality MonitorTests Coverage for New Code 〰️ Line Coverage: 97.35% — 4 missed lines Coverage for Whole Project 〰️ Line Coverage: 95.96% Style Bugs API Problems 🚫 Revapi: No warnings Vulnerabilities 🛡️ OWASP Dependency Check: No vulnerabilities Software Metrics 🌀 Cyclomatic Complexity: 1019 (total) 📌 Reference ResultsDelta reports computed against the reference results of 20d104b in workflow run 31372098981. 🚦 Quality GatesOverall Status: ✅ SUCCESS✅ Passed Gates
Created by Quality Monitor v4.15.0 (#82d77af). More details are shown in the GitHub Checks Result. |
GO_COV issues with resolving paths
Issue #263 reported that Go coverage reports were losing their directory structure and failing to locate source files. For example,
example.com/project/internal/alpha.gowas incorrectly parsed as justalpha.go. To resolve this, I implemented aModuleRegistry()solution that parsesgo.modfiles to extractmodule-to-directorymappings. It then uses a longest-prefix matching approach to accurately translate coverage paths to file system paths (e.g.,ext/sum.go→./hack/ext/sum.go). Whengo.modfiles are not available, the implementation falls back to intelligent heuristics to infer the correct path structure.From the demo coverage profile:
Fixes #263
Testing done
Submitter checklist