[BUG]: Certain test targets fail to execute bazel coverage commands

[Rd4dev]

Bazel Coverage

Bazel provides a way to analyze the coverage of test targets using command

bazel coverage //:test_target

or

bazel test --collect_code_coverage //:test_target

But many targets in the oppia-android codebase seem to fail to analyze the coverage data.

Few examples of failing test targets

//app:src/sharedTest/java/org/oppia/android/app/help/HelpActivityTest
//app:src/sharedTest/java/org/oppia/android/app/home/HomeActivityTest
//app:src/test/java/org/oppia/android/app/home/HomeActivityLocalTest

Few examples of passing test targets

//utility/src/test/java/org/oppia/android/util/math:MathTokenizerTest
//domain/src/test/java/org/oppia/android/domain/auth:FirebaseAuthWrapperImplTest

Suspects

Few suspected reasons/findings why this occurs are:

Please look into the following issues for more history

• Fix part of #5343: Exempt targets incompatible with code coverage #5480
• [Feature Request]: Introduce support for code coverage & enforcement (GSoC'24 4.1) #5343

As mentioned in the above issues

• Many tests especially within the app module that are not modularized (tests that don't have their own explicit bazel target specified) seem to fail running coverages (though running bazel test succeed on the same).

  • Also, some of the unmodularized tests in the app module did pass eg: SpotlightFragmentTest.

  • Though the SpotlightFragmentTest passes, the produced coverage.dat file does not include the SourceFile (SF:) to SpotlightFragment.kt (specified under gen_rule) while it includes SourceFile to SpotlightManager.kt, SpotlightShape.kt (other files in the same package specified under kt_android_library), etc, which raises a doubt if the files that are specified under gen_rule causes issues with collecting the coverages data.

• Few non modularized tests when moved to its own target (modularized) do execute the coverage command with necessary SourceFile being included in the produced coverage report [comment addressing this].

• Other suspects: Incorrect instrumentation filter being set, improper naming and positoning of the test files relative to their corresponding source files.

The reasons are not yet clearly identified.

The PR #5480 lists all the source files that exhibit the behaviour of failing coverage runs on its test targets and identifies the following:

115/230 failures occurred from coverage.dat missing after running bazel coverage.
32/230 failures occurred from the test file not actually existing where it was expected to be.
81/230 failures occurred from a failure to instrument one or more of the classes needed for runtime.
2/230 failures occurred from a JaCoCo access issue (likely indicating some significant build configuration issue somewhere in the toolchain).

Steps To Reproduce

bazel coverage //app:src/test/java/org/oppia/android/app/home/HomeActivityLocalTest

Expected Behavior

Passing coverage analysis with log

INFO: Elapsed time: 292.080s, Critical Path: 129.12s
INFO: xx processes: xx internal, xx linux-sandbox, xx worker.
INFO: Build completed successfully, xx total actions
 //app:src/test/java/org/oppia/android/app/home/HomeActivityLocalTest PASSED in 8.1s
/home/.../__main__/bazel-out/k8-fastbuild/testlogs/
app/src/test/java/org/oppia/android/app/home/HomeActivityLocalTest/coverage.dat

Executed 1 out of 1 test: 1 test passes.

Screenshots/Videos

No response

What device/emulator are you using?

No response

Which Android version is your device/emulator running?

No response

Which version of the Oppia Android app are you using?

No response

Additional Context

No response

[Rd4dev]

The following incompatible files are exempted as they lack test files in the correct location.

No. of files with misplaced test files: 32

• app/src/main/java/org/oppia/android/app/administratorcontrols/appversion/AppVersionActivity.kt
• app/src/main/java/org/oppia/android/app/devoptions/markchapterscompleted/MarkChaptersCompletedActivity.kt
• app/src/main/java/org/oppia/android/app/devoptions/markchapterscompleted/MarkChaptersCompletedFragment.kt
• app/src/main/java/org/oppia/android/app/devoptions/markstoriescompleted/MarkStoriesCompletedActivity.kt
• app/src/main/java/org/oppia/android/app/devoptions/markstoriescompleted/MarkStoriesCompletedFragment.kt
• app/src/main/java/org/oppia/android/app/devoptions/marktopicscompleted/MarkTopicsCompletedActivity.kt
• app/src/main/java/org/oppia/android/app/devoptions/marktopicscompleted/MarkTopicsCompletedFragment.kt
• app/src/main/java/org/oppia/android/app/devoptions/vieweventlogs/ViewEventLogsActivity.kt
• app/src/main/java/org/oppia/android/app/devoptions/vieweventlogs/ViewEventLogsFragment.kt
• app/src/main/java/org/oppia/android/app/help/faq/FAQListFragment.kt
• app/src/main/java/org/oppia/android/app/help/faq/faqsingle/FAQSingleActivity.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/LicenseListActivity.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/LicenseListFragment.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/LicenseTextViewerActivity.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/LicenseTextViewerFragment.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/ThirdPartyDependencyListActivity.kt
• app/src/main/java/org/oppia/android/app/help/thirdparty/ThirdPartyDependencyListFragment.kt
• app/src/main/java/org/oppia/android/app/home/recentlyplayed/RecentlyPlayedFragment.kt
• app/src/main/java/org/oppia/android/app/home/topiclist/TopicSummaryViewModel.kt
• app/src/main/java/org/oppia/android/app/player/state/ImageRegionSelectionInteractionView.kt
• app/src/main/java/org/oppia/android/app/walkthrough/end/WalkthroughFinalFragment.kt
• app/src/main/java/org/oppia/android/app/walkthrough/topiclist/WalkthroughTopicListFragment.kt
• app/src/main/java/org/oppia/android/app/walkthrough/welcome/WalkthroughWelcomeFragment.kt
• domain/src/main/java/org/oppia/android/domain/oppialogger/survey/SurveyEventsLogger.kt
• instrumentation/src/java/org/oppia/android/instrumentation/application/EndToEndTestGcsResourceModule.kt
• instrumentation/src/java/org/oppia/android/instrumentation/application/EndToEndTestImageParsingModule.kt
• instrumentation/src/java/org/oppia/android/instrumentation/application/EndToEndTestNetworkConfigModule.kt
• utility/src/main/java/org/oppia/android/util/accessibility/FakeAccessibilityService.kt
• utility/src/main/java/org/oppia/android/util/extensions/StringExtensions.kt
• utility/src/main/java/org/oppia/android/util/parser/html/HtmlParser.kt
• utility/src/main/java/org/oppia/android/util/parser/html/ListItemLeadingMarginSpan.kt
• utility/src/main/java/org/oppia/android/util/platformparameter/PlatformParameterSingleton.kt

(generated via the testfilecheck script)

While files in the app module may still exhibit the same behavior, moving test files in other modules to their correct locations should generate appropriate coverage data.

Not sure if this is the right issue to reference, but for now adding it here.

[Rd4dev]

@BenHenning, should the test files be moved to their correct locations (adding builds as needed) to enable coverage for these files - #5481 (comment)? Tried this with the test for StringExtensions.kt, and it works, or any blocker keeping them as they are?

StringExtension.kt coverage report

Source: utility/src/main/java/org/oppia/android/util/extensions/StringExtensions.kt

Test: utility/src/test/java/org/oppia/android/util/extensions/StringExtensionsTest.kt