How to Test Third-Party SDKs for Performance & Security Risks

In this article, we’ve covered the techniques and practical steps for testing third-party SDKs for performance and security risks. It also describes the tools for the trade and what to look for when running such tests for SDKs.

Key reasons why thorough vetting of SDKs for performance and security is crucial:

Performance Risks

• Resource Consumption — Poorly optimized SDKs can gobble up too much CPU, system memory, and battery resources, leading to slow app performance, crashes, and rapid battery drain. 
• Network Overheads — Inefficient or excessive data transactions by SDKs can result in excessive network overheads (such as frequent connections and unencrypted communication).
• Compatibility Issues — Some SDKs may not function or perform the same across different devices, operating system versions, or hardware configurations.
• Main Thread Slowing Up or Getting Blocked — SDKs that often execute long-running operations on the main thread can cause sluggish performance or even block it on some occasions, freezing your app’s UI.
• Increased App Size — Some third-party SDKs can bloat your app with too much unnecessary code, thus, increasing your app package size.

Security Risks

• Data Breaches and Privacy Violations — Third-party SDKs, especially from less reputable sources, can collect and transmit data without proper encryption, user consent, or transparency. This can lead to privacy violations and data breaches, exposing sensitive user information. Note that the risk isn't always caused intentionally; poor coding practices by the SDK developers can also lead to such problems.
• Malware and Backdoors — Malicious SDKs can tinker with your software or your system and introduce malware or backdoors into your app, allowing attackers to compromise devices, steal data, or perform other unauthorized actions. While intentional malice is possible, most vulnerabilities are unintentional coding errors that can be exploited.
• Undetected Bugs or Security Flaws — SDKs can have bugs or security flaws that could have gone undetected or missed while QA and QC checks. Bad actors can exploit these weak parts of the SDK to compromise your app, gain unauthorized access, or execute arbitrary code.
• Compliance Issues — SDKs must comply with relevant regulations like GDPR, CCPA, HIPAA, etc.  This is especially critical for apps related to healthcare, security, and other sensitive sectors. Using non-compliant SDKs can lead to legal troubles and hefty fines.
• Dependency Risks — Many SDKs often rely on other libraries and modules to implement functionalities within them. Vulnerabilities in even one of these dependencies channeled through the SDKs can also put your apps at risk.

Steps to test third-party SDKs for performance:

Prerequisites 

Let’s consider testing a mobile SDK for performance. Make sure you have the following supplies:

1. Download the SDK — Obtain the SDK binary code file to be tested (e.g., .jar, .aar, .dll).
2. SDK Documentation — Make sure you have the official SDK docs with instructions to integrate with your app, its API specs and keys, and security guidelines.
3. Demo Application — SDK publishers typically include a demo of how it's integrated with another app. Make sure to keep a demo app at hand that already using the SDK that you’re gonna test. 
4. Sandbox Environment — Setup a testing environment (preferably, a virtual sandbox to keep things isolated and contained in case of errors) configured to match the real-world conditions. This can include using similar hardware, software, network, or database setups.
5. Test Data — Prepare a set of realistic but anonymized test data to avoid divulging sensitive information while running tests.

Also Read: SDKs vs. API: What’s the Difference and When to Use Each?

SDK Performace Testing Techniques w/ Steps to Run Each

Key techniques for running performance checks for third-party SDK:

• Performance Profiling  
• Load and Stress Testing
• Memory Usage Analysis 
• Network Performance Testing

Let’s quickly expand on each of the techniques mentioned with:

• Steps to Run 
• What to Look for When Using Each

Performance Profiling 

Performance profiling involves analyzing the SDK's resource usage to identify performance bottlenecks.

Steps for Profiling SDK’s Performance

1. Integrate the SDK with the test application within the sandbox environment.
2. Use profiling tools (e.g., Android Profiler, Xcode Instruments) to monitor CPU usage, memory allocation, and network activity.
3. Run the app with the SDK integrated and simulate user interactions.
4. Analyze the profiling data to identify performance issues, such as long-running tasks or excessive memory allocations and usage.
5. Document and report the findings

Also Read: How to Fix ANR (Application Not Responding) Errors in Android?

What to Look For

• High CPU usage during specific SDK operations
• Excessive memory allocation or memory leaks if any
• Slow response times or delays caused by the SDK

Load and Stress Testing

Load and stress testing allows you to check the SDK's performance under variable loads and network conditions.

Steps for Running Load and Stress Tests for SDK

1. Start things off by simulating expected user loads to test SDK performance under normal conditions using tools like Apache JMeter and LoadView.
2. Monitor the SDK's response time, throughput, and resource usage.
3. Increase the load beyond normal operational capacity to test how the SDK handles stress.
4. Confirm that the SDK does not crash or cause significant issues when overloaded.
5. Document and report the results.

What to Look For

• Increased response times or delays under heavy load times
• Error rates or crashes due to resource exhaustion
• Inability to handle too many concurrent users or requests 
• Performance hits and bottlenecks under extreme loads

Memory Usage Analysis

Memory usage analysis focuses on identifying memory leaks and excessive memory consumption by the SDK.

Steps for Memory Usage Analysis 

1. Start by using Android Profiler or Xcode Instruments (Allocations and Leaks) to monitor the SDK's memory usage over time.
2. Look for memory that isn't released properly, indicating potential memory leaks. Use tools like LeakCanary (Android) and Instruments Leaks Template (iOS) to identify memory leaks.
3. Analyze the SDK's memory allocation and deallocation patterns. 
4. Document and report the findings.

What to Look For

• Memory leaks that cause the app's memory usage to grow over time
• Identify areas of excessive memory consumption
• Unnecessarily stored data in memory when the SDK is running

Also Read: How to Fix Error Code Out of Memory in Chrome

Network Performance Testing

Network performance testing helps evaluate the SDK's network usage and its impact on network performance. 

Steps for Network Performance Testing 

1. Use network monitoring tools (e.g., Wireshark, Charles Proxy) to capture and analyze the network traffic generated by the SDK.
2. Measure the SDK's data usage, connection frequency, and latency.
3. Simulate different network conditions, such as low bandwidth or high latency.
4. Document and report the results.

What to Look For

• Data throughput rates
• Large or unnecessary network requests
• Latency and packet losses
• SDK performance under different network conditions

Steps to test third-party SDKs for security risks:

Prerequisites 

The prerequisites for running security tests for SDKs are very similar to those for the performance tests. Here too, we’ve considered security testing for Mobile SDKs. 

Here’s a list of prerequisites you can borrow for security testing of SDKs:

• SDK binary code file 
• Official SDK documentation
• Demo app with the integrated SDK
• A sandbox environment 
• Security testing tools (included with the techniques described below)
• Real-world setup and conditions (hardware, software, network, etc)
• SDK Source Code (Having access to the SDK’s source code can be invaluable for its in-depth analysis. While not always provided by the original publishers, it’s good to have if you can access it. But, it's NOT a hard rule)

SDK Security Testing Techniques w/ Steps to Run Each

Key techniques for running security checks for third-party SDKs:

• Static Analysis 
• Dynamic Analysis
• Integration Testing
• Penetration Testing 

Static Analysis 

Static Analysis is the process of examining the SDK's code without executing it. This technique helps identify vulnerabilities such as insecure coding practices, hardcoded credentials, and potential backdoors.

Steps for Static Analysis

1. After downloading the SDK file, decompile/unpack the downloaded SDK file with tools like MobSF, APKTool, and JadX. 
2. Now, reverse engineer the decompiled file to access the source code and other available resources. Review the SDK's code manually by looking for potential vulnerabilities, insecure code logic, and suspicious code patterns.
3. Next, use static analysis tools, such as SonarQube or Checkmarx to scan the code for common vulnerabilities and code smells.
4. Document and report the findings.

What to Look For

• Sensitive data like API keys, login information, database, or user credentials stored as plain text within the app's preferences or local storage.
• Insecure deeplinks and webviews
• Use of weak or outdated cryptographic algorithms
• Improper input validation methods
• Client-side SQL injections
• Unnecessary permissions or access to sensitive resources
• Potential backdoors or hidden logic
• Data leaks (e.g., transmitting sensitive data without encryption)

Also Read: API Testing 101

Dynamic Analysis 

Dynamic analysis involves testing the SDK in a sandbox environment to observe its behavior, API, and network traffic interactions, and potential vulnerabilities during execution.

Steps for Dynamic Analysis

1. Check the official SDK docs for integration steps and connect with the test application.
2. Use network monitoring tools (e.g., Wireshark, tcpdump) to capture and analyze the network traffic generated by the SDK.
3. Intercept function calls within the SDK to monitor inputs, outputs, and side effects.
4. Try understanding the logic and flow of the SDK to detect potential security flaws.
5. Log, document, and report any inconsistencies.

What to Look For

• Data being sent to servers in cleartext (HTTP) instead of HTTPS 
• Unexpected network connections or data transactions to unknown or suspicious servers
• Excessive data usage or frequent communication with remote servers
• Unauthorized access to system resources
• Unwanted cryptographic functions running through networks
• Runtime activities
• Weak authentication mechanisms for data transfers

Integration Testing

Integration testing for an SDK evaluates how well it integrates with your app, focusing on security aspects, such as input validation, error and exception handling, secure data exchange between app components, compatibility, and others.

Steps for SDK Integration Testing 

1. Integrate the SDK with the test app as per the docs.
2. Identify any issues that might occur with different configurations or misconfigurations.
3. Look for any security or error warnings that may have been included in the SDK documentation while integrating it.
4. Check for any hardcoded API or encryption keys that may have been exposed while the SDK integrates with the target components in your app.
5. Document and report the findings.

What to Look For

• Data validation between the app and the SDK.
• Unclear errors and exception handling mechanisms.
• Unexpected interactions with other components of your app.
• Data corruption, data losses, or app crashes during integration procedures.
• Compatibility issues with devices or operating system versions.

Penetration Testing 

Penetration testing involves simulating real-world attacks to evaluate the SDK's security defenses against potential exploits.

Steps for Penetration Testing 

1. Define the scope to Identify aspects of the SDK and app to be tested.
2. Do a threat modeling for flagging potential threats and attack vectors specific to the SDK.
3. Using the SDK docs, gather information about the SDK's functionalities, endpoints, and data flows.
4. Next, use manual expertise or pen-testing tools (e.g., OWASP ZAP, Metasploit framework) to simulate various attacks, such as Brute Force, SQL injections, XSS, and DDoS. 
5. Assess the impact of successful exploits, such as data access or code execution.
6. Record all findings, steps taken, and vulnerabilities discovered. 
7. Finally, create detailed reports with evidence and logs.

What to Look For

• Hidden vulnerabilities within the code.
• Other weaknesses in the SDK's security mechanisms.