Android Malware
Detection Test Plan

Introduction

This document describes the Test Plan for the Testing Ground Labs (TGL) Android Malware Detection Test 2026 August C (Consumer Products). The Test is designed to independently assess the efficiency of consumer security solutions for Android OS in detecting currently widespread malicious mobile applications.

AMTSO Standard Compliance Statement
This Test has been designed to comply with the Anti-Malware Testing Standards Organization, Inc. (“AMTSO”) Testing Protocol Standard for the Testing of Anti-Malware Solutions, Version 1.3 (the “Standard”). This Test Plan has been prepared using the AMTSO Test Plan Template and Usage Directions, Version 2.4. Testing Ground Labs is solely responsible for the content of this Test Plan.

Android is a mobile operating system developed by Google. It is based on a modified version of the Linux kernel and other open-source software and is designed primarily for touchscreen mobile devices such as smartphones and tablets. In addition, Google has developed Android TV for televisions, Android Auto for cars, and Wear OS for wrist watches, each with a specialized user interface. Variants of Android are also used on game consoles, digital cameras, PCs, and other electronics.

The popularity and openness of the Android ecosystem introduce numerous security risks for end users and organizations. Malicious activities on Android devices may include:

• Sending messages to premium-rate SMS numbers that incur additional charges.
• Exfiltrating sensitive or personal information to unauthorized third parties.
• Enrolling the device into botnets to execute remote commands for spam, DDoS attacks, and other abuse.
• Allowing unauthorized parties to monitor phone calls, SMS, and app communications.
• Enabling blackmail or extortion by leaking or threatening to leak sensitive data.
• Tricking users into disclosing financial information such as account numbers and dates of birth.
• Serving as a pivot to compromise other systems, including PCs connected to the smartphone.

To protect systems and data from these and other threats, dedicated Android-based security applications exist and must be evaluated regularly against evolving malware. This Test is intended to provide an independent and repeatable assessment of the malware detection capabilities of consumer Android security solutions under realistic conditions.

Methodology

The Test methodology is designed to be transparent, repeatable, and aligned with AMTSO best practices. The detailed process is as follows.

1. Test Device Preparation
   Several Android-based mobile devices are prepared and clean backup images are created. The primary platform for this Test is:
   • Android 14 on Samsung Galaxy S22 (256 GB internal storage).
2. Sample Collection
   Approximately 1,000–2,000 Android malware samples and 300–500 different clean Android application installers are collected and delivered in a mixed set to the internal storage of the mobile devices. Malware samples are collected from multiple sources (including China region–based feeds and other international channels). Clean applications are obtained from Google Play and other legitimate app stores.
3. Product Installation
   Selected consumer security applications are installed on the physical mobile devices using their default configuration.
4. Product Updates
   Each security application and its antivirus bases (signatures and related components) are updated to the latest available versions before testing begins and as needed during the Test Period.
5. On-Demand Scan (Static Detection)
   A full scan of the mixed collection is run by each security application. Malware detection rates and false positive detections are recorded.
6. On-Execution Detection (Dynamic Behavior)
   Each malicious sample that was not detected during the on-demand scan is subsequently installed and executed. Any detection that occurs during or after execution is recorded.
7. Benign Application Handling
   Installation of clean applications is not performed. They are scanned only, to measure the false positive detection rate without unnecessary risk to device stability.

Environment

The primary test environment for this Test is:

• Device Model: Samsung Galaxy S22
• Operating System: Android 14
• Internal Storage: 256 GB
• Network: Internet-connected, with access to product cloud services where applicable

Malicious and non-malicious Android installers are collected by the Testing Ground Labs threat collection system during the two months preceding the Test commencement date. Malware is sourced from multiple feeds and regions (including China region–based sources). Non-malicious (clean) applications are used to measure false positive detection rates. Candidates for legitimate sample testing include newly released apps collected from multiple public app stores (including, but not limited to, Google Play).

Malicious and legitimate applications are independently verified by Testing Ground Labs to the extent reasonably practical, to ensure that each sample is correctly categorized as malicious or non-malicious at the time of testing.

Upon completion of the full Test, Testing Ground Labs will provide each participating vendor with data for their own non-optimal results (missed malware samples and false positives). TGL does not share a vendor’s detailed results or data with other vendors. Any tested security vendor may request the hashes of their missed samples and false positives.

Control Procedures

All security solutions in the Test will be granted access to their cloud reputation, update, and other backend services, to match typical real-world deployment conditions for end users.

Instructions for enabling and exporting logging within each product must be provided by the Participant to Testing Ground Labs upon request. Logs may be used to investigate unexpected behaviors, disputed results, or technical issues during the Test.

Any configuration information needed for automatic or manual product updates during the Testing Period must be disclosed by the Participant. Testing Ground Labs will endeavor to ensure that all tested products are up to date prior to and during testing, in line with normal end-user expectations.

Dispute Process

The dispute process runs for eight business days commencing from the end of the Test (i.e., following Milestone 2: Test Report First Edition). Please refer to Section 6 (Schedule) for additional timing details.

The general Dispute Process operates as follows:

1. Testing Ground Labs provides each vendor with detailed results only for their own security solution, including hash values associated with any missed malware samples and false positive detections.
2. The vendor responds within eight business days to Testing Ground Labs, providing fact-based disagreements or evidence regarding any sub-optimal results, if applicable.
3. Testing Ground Labs reviews each submission and responds with a decision indicating whether the dispute is accepted or denied. If accepted, appropriate corrections (including re-tests, where feasible) may be reflected in the final Test results and report.