Before you find a video game tester job, you need to know roles of a game development team, their main tasks and responsibilities, and the workflow.

Game Development Team

Generally, a game development team is composed of producer, planner, programmer, art, and game tester.

A producer is the manager of the project who is in charge of game market research and development, game operation, team management, and all affairs of the project.

The responsibility of a planner includes plot, system rules, numerical design, and game level planning.

While for the game artists, scenario, UI, original painting, and animation are their main responsibilities.

Usually, programmers are divided into front-end engineers and back-end engineers. The front-end engineers take charge of the presentation and logic of the client-side, while the back-end engineer is responsible for the verification of logic and data of the server-side.

Game development process

● Producer: Set the goal of the project.

● Planner: Split the project goal into detailed requirements.

● Programmer: Present the requirements with code and fine art while the tester writes test cases.

● Tester: Quality assurance of the project, and give feedback of the defects found to the team.

After learning the roles and their tasks in the game development team, we are going to figure out the main task of a game tester and the basic workflow of game testing.

What Do Game Testers Do?

◮ Functional testing

Functional testing is the most common type of game testing, the main testing method is black box testing. Functional testing is mainly used to verify whether the function meets the requirements.

Functional testing mainly focus on the correctness of the function instead of considering the underlying code structure and code errors of the game.

Functional testing usually starts with the interface and tries to simulate the possible operations of the user.

◮ Performance testing of client-side

Testers need to monitor the following data:

CPU utilization of client-side

Memory utilization of client-side

Network traffic of client-side

Power consumption of client-side

Frame per second of client-side

Monitoring tools:

IOS: The instrument that comes with Xcode

Android: Emmage and GT (both open-source tools, you may need to root your phone)

◮ Stress test of server-side

CPU utilization of server-side

Memory utilization of server-side

System Throughput (TPS)

Transaction response time

Transaction success rate

◮ Compatibility test (mobile game)

Model adaptation test

Operating system compatibility test

Screen resolution compatibility test

Game version compatibility test

◮ Security test

Memory modification test

Encryption test of client-side

Decompilation test of client-side

Cyber Security Testing

◮ Interface test

Data test of each interface of the server, mainly through tools

Interface security test, repeat the request, check the interface processing

◮ Log test

Client log

Server log

◮ Weak network test

Different network conditions, game operation conditions, such as edge, 2g, 3g, 4g conditions

The operation of the game under different packet loss

Realize by setting network proxy through tools, commonly used fiddler, network link conditioner

◮ GM tools test

To test the function of the GM tools, you need to pay attention to the setting of the tool in the game

Data reading and storage of GM tools test

◮ SDK test

User data testing

Recharge, consumption test

Docking test with various channels

How to do Game Testing?

1). Have a game functions meeting with other teammates.

Understand the functional requirements

Put out possible risk points

Think about the key points and difficulties of functional testing. If you need tool support, put forward development requirements.

Think about the stuffs that can be optimized and propose discussions

2). Write test cases

Write test cases based on requirements

Focus on functional logic implementation

Pay attention to various special scenarios, such as boundary values, network interruptions, process interruptions, etc.

Pay attention to changes in requirements and adjust test cases in time.

3). Perform smoke testing

Quickly find out obvious bugs

Quickly ensure the fluent operation of main logic flow

Quickly clarify the function status

4). Perform detailed test

Carefully test each logical branch, resource, and configuration

Try to simulate every possible operation of the player

Test abnormal conditions, such as breakpoints, network disconnection, event interruption, process interruption, etc.

Test data reading, storage, network

5). Perform regression testing

6). CheckList

Let’s get started.

1. Testing Scenarios

The front-end performance test focuses on the user’s experience during gameplay, including whether it’s stuck, whether it’s crashing, whether the phone is hot, whether it consumes too much power, etc., regardless of whether the user’s phone is a high-end phone, a medium-end phone, or very low-configuration all need to pay attention to the user performance experience.

● Lag

The fluency of the game is a relatively basic performance indicator, especially in the core gameplay. From a testing perspective, pay more attention to the game’s fps data, frame lag rate, BigJank value, etc.

● Crash

Game players don’t want to have crashes when playing games, so pay attention to the crash rate when testing.

● Overheating

After the mobile phone heats up, in addition to the bad body feeling, some mobile phone manufacturers will choose to reduce the frequency or lock the core, which will cause the game lag. There are many reasons for mobile phone heating (such as running high-power apps, charging, bad batteries, the protective case can not dissipate heat, high ambient temperature, etc.). in this case, what testers will focus on is how to reduce the power consumption of the game, especially GPU and CPU usage.

● Too fast power consumption

Mobile phone battery milliampere is itself an important indicator of mobile phone, and fast power consumption is undoubtedly a flaw.

2. Test Tools

● perfdog

This performance tool is provided by Tencent. PerfDog supports all applications on the mobile platform (games, applications, browsers, mini-programs, mini-games, H5, back-end system processes, etc.)

● UWA

It provides services for game or VR products of unity and unreal engines, including resource monitoring (dynamic and static), performance diagnosis and optimization, Lua performance analysis, automated testing, community Q&A, etc.

UWA GOT is a local performance evaluation tool launched by UWA. It needs to install the corresponding SDK, without jailbreak or root.

● Test Writer

It is a codeless automated testing tool developed by 51Testing. Test Writer has 9 modules which include project management, test objects, test steps, test cases, test plans, test missions, automated testing, and test reports.

● Integration Test Platform (ITP)

Integration Test Platform (ITP) is an interface-oriented integration test automation tool independently developed by 51Testing, covering interface use case design, automated test script execution, automated test execution planning and task allocation, interface test management, and automated testers management and other functions. This tool is suitable for systems or products with high software reliability, high security, and high robustness, as well as for agile development processes.

● Test Platform

Test Platform is a software testing platform developed by 51Testing covering the whole process of software quality assurance such as test plan management, project management, demand management, test demand analysis, test case design, test execution management, defect tracking management, measurement statistics, defect analysis, etc. The platform helps customers fully manage the entire cycle of software testing.

3. Test Data

● Memory

Generally, we pay attention to:

① The size of pss memory (physical memory actually used)

iOS and Andriod devices are generally divided into three levels, and the pss memory standard of each level is different, and the standard value is gradually getting higher with the improvement of hardware equipment.

② Memory leak

The memory used becomes larger and larger after memory leak, it may have lag or crash due to slower and slower memory allocation.

③ Memory resource recurrence rate

Resource recurrence refers to the existence of two or more identical textures, grids, animations, audio, and other resources at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

● BigJank

PerfDog calculation method: If two conditions are met at the same time, it is considered to be a stuck Jank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2

Ø The current frame time-consuming> two movie frame time-consuming (1000ms/24*2=84ms).

If both conditions are met at the same time, it is considered to be a serious lag BigJank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2.

Ø The current frame time-consuming> three movie frame time-consuming (1000ms/24*3=125ms).

● FPS

Frame rate, standard requirements such as: the core game scene requires 90% of the value not lower than 25FPS by default;

● Fluency

For example, the default requirement of lag rate in the core game scene is not higher than 2%.

● CPU

For example, the average overall CPU usage (90%) is less than 60%, and the peak single-core CPU usage (90%) is less than 90%.

Also, pay attention to the single-core CPU usage, and whether all cores are in use, and whether the use is even, etc.

● Commonly Used Data in Tools

Ø CPU

① Frame Time: the time consumption of the same frame

② Rendering Time: The rendering time of the current frame; the higher the draw call, the greater the cost of this part.

③ Script Time: the current frame function time-consuming

④ Physics Time: the physics time of the current frame

Ø Memory

Memory in the code — Mono memory (mono memory, memory pool, only increase but not decrease);

Memory usage in resources — Native memory.

① Reserved Mono: generally the memory allocated by the script

② pss memory: generally used to locate multi-game battles, scene jumps, open and close to check whether there is a memory leak in the UI.

③ Total_reserved memory: unity_reserved memory + GFX memory + FMOD memory + Mono memory + Profiler memory

④ Mono reserved: The allocated mono memory, which only rises but does not fall, needs to be strictly controlled. Mono memory represents the memory allocated by the script in the game.

⑤ gfxdriver_reserved: it indicates the memory of the rendering module. If it is relatively high, the texture and Shader need to be optimized.

⑥ fmod_reserved: it indicates the memory of the audio module, if it is relatively high, the audio resources need to be optimized.

⑦ Recurrence rate of memory resources: Resource recurrence refers to the existence of two or more same resources such as textures, grids, animations, and audios at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

4. Commonly Used Optimization Methods and Strategies

Before optimizing the games, you should know:

① Performance optimization is a long-term process.

② Performance optimization is not only a matter of the program department but the result of the joint cooperation and efforts of the entire project team. Generally, each department of the project team will have corresponding full-time personnel.

③ Good optimization tools are essential.

④ Different devices and equipment show different art quality, so there are different optimization strategies for different models.

⑤ In different scenarios, the choice of time and space (time for space or space for time, CPU and GPU can be exchanged, CPU and memory can be exchanged, memory and disk can be exchanged)

⑥ Don't render unnecessary rendering things; don’t load unnecessary things. Compress what can be compressed.

⑦ Performance optimization is not a once-and-for-all thing, it is a protracted battle; even for long-term products that have been on the line for many years, the performance testers are always on their way.

● Scene optimization

① Pay attention to the hot data of the scene (for example, according to a calculation range unit of 2 meters), optimize the hot issues by adjusting the position of the object, optimizing the resource of the object, etc.

② Add key occluders to reduce the amount of rendering in the camera.

③ The use of LOD technology, or the use of imposter technology, etc.

④ Smooth rendering of distant objects including character monsters.

● Optimization of the number of people on the same screen

① Pay attention to the scene separation, such as the auction house and the transaction place, one indoor and one outdoor; what’s more, reduce the consumption of objects in the corresponding scene.

② Optimize the calculation strategy, give priority to rendering characters and scene objects close to you.

② Effect optimization, for example, the same type of effects will not be rendered after reaching a certain amount.

④ The user role is dynamic at the same screen, loading and releasing continuously, and smoothing (controlling the number of rendering methods by frame, etc.). Avoid stuttering caused by instant loading.

⑤ When fighting on the same screen, especially national warfare, it is the same as the role avatar.

⑥ Imposter (pretender, or substitute) technology: the basic principle is to use the camera to take pictures of all angles of the model, and then select different pictures to display according to the angle of the player’s camera.

● UI optimization

① Format

② Separate dynamic UI elements and static UI elements into different UIPanels as much as possible (UI rebuilding takes UIPanel as the unit), so as to control the refactoring caused by changed UI elements to a smaller scope as much as possible.

③ Atlas with and without channels, separate as far as possible.

④ Putting the UI resources of the same UI interface or the same function point in an atlas can reduce draw calls.

● GC optimization

GC will take up a lot of computing resources. If the memory is still not enough after GC and needs to be allocated again (unofficial data of unity is about 6MB at a time), then it will consume more resources. GC optimization generally includes:

① Object pool technology — — Reuse objects (such as bullets which require frequent generation and destruction)

② Reduce the amount of memory garbage (for example: do not repeatedly allocate heap memory in frequently called functions, which use caching technology)

③ Slow down the time of GC, do not operate GC at critical times; take the initiative to GC during non-sensitive periods.

④ Other technical methods, such as boxing, list, string, etc. lead to GC solutions.

● Memory optimization

The main reasons for memory performance are the following: excessive memory fragmentation (resolved by the memory pool), frequent memory creation and destruction (resolved by the object pool), slow memory loading, and excessive memory usage.

① Resources (textures, models, animations, sounds, etc.) are lossy compressed, and each resource has an optimal compression method and format.

② Script and configuration, timely uninstall, and split, etc.

③ Third-party shader library, redundant pre-compiled macro occupancy

④ AB package packaging and loading strategy will affect memory usage. AB automatic packaging tool (combination of public packages, the AB package should be within 10MB), the game type logic is strongly associated with the packaging mode, such as moba, which packs all the resources of a hero into one package.

● CPU optimization

Here are methods for CPU optimization:

Micro method: find problems in every frame (main method)

Macro method: Find problems from statistics. For example, we can find the problem for a battle from statistics on the time-consuming and call times of each module and function, and it is also convenient for comparison of different versions.

Classification:

① GC, GCAlloc time-consuming

② Rendering time-consuming (combine batch processing, reduce draw call: such as using dynamic and static batch processing, GPU Instance technology)

③ Logical time-consuming (such as circular search, centralized processing of a large number of server messages, etc.)

④ Other engine consumption (animation, physics)

● GPU optimization

GPU optimization ideas:

① Reduce rendering batches

② In the same draw call, reduce the rendering type.

③ Smooth rendering.

● Performance monitoring

① Internal monitoring

Have performance monitoring when uploading resources and performing version testing.

② External network performance monitoring.

● Power consumption

Optimization ideas:

① Calculation optimization: Algorithm, for loop optimization, Switch..case instead of if..else, avoid floating-point operations.

② Avoid improper use of Wake Lock

③ Use Job Scheduler to manage background tasks

④ Reduce brightness, etc.

5. Summary

The core idea of mobile game performance testing is more about designing reasonable test cases, analyzing the data after the test is completed, and providing reasonable optimization suggestions based on the problem data so as to provide efficient services to the R&D team and reflect value.

Let’s get started.

1. Testing Scenarios

The front-end performance test focuses on the user’s experience during gameplay, including whether it’s stuck, whether it’s crashing, whether the phone is hot, whether it consumes too much power, etc., regardless of whether the user’s phone is a high-end phone, a medium-end phone, or very low-configuration all need to pay attention to the user performance experience.

● Lag

The fluency of the game is a relatively basic performance indicator, especially in the core gameplay. From a testing perspective, pay more attention to the game’s fps data, frame lag rate, BigJank value, etc.

● Crash

Game players don’t want to have crashes when playing games, so pay attention to the crash rate when testing.

● Overheating

After the mobile phone heats up, in addition to the bad body feeling, some mobile phone manufacturers will choose to reduce the frequency or lock the core, which will cause the game lag. There are many reasons for mobile phone heating (such as running high-power apps, charging, bad batteries, the protective case can not dissipate heat, high ambient temperature, etc.). in this case, what testers will focus on is how to reduce the power consumption of the game, especially GPU and CPU usage.

● Too fast power consumption

Mobile phone battery milliampere is itself an important indicator of mobile phone, and fast power consumption is undoubtedly a flaw.

2. Test Tools

● perfdog

This performance tool is provided by Tencent. PerfDog supports all applications on the mobile platform (games, applications, browsers, mini-programs, mini-games, H5, back-end system processes, etc.)

● UWA

It provides services for game or VR products of unity and unreal engines, including resource monitoring (dynamic and static), performance diagnosis and optimization, Lua performance analysis, automated testing, community Q&A, etc.

UWA GOT is a local performance evaluation tool launched by UWA. It needs to install the corresponding SDK, without jailbreak or root.

● Test Writer

It is a codeless automated testing tool developed by 51Testing. Test Writer has 9 modules which include project management, test objects, test steps, test cases, test plans, test missions, automated testing, and test reports.

● Integration Test Platform (ITP)

Integration Test Platform (ITP) is an interface-oriented integration test automation tool independently developed by 51Testing, covering interface use case design, automated test script execution, automated test execution planning and task allocation, interface test management, and automated testers management and other functions. This tool is suitable for systems or products with high software reliability, high security, and high robustness, as well as for agile development processes.

● Test Platform

Test Platform is a software testing platform developed by 51Testing covering the whole process of software quality assurance such as test plan management, project management, demand management, test demand analysis, test case design, test execution management, defect tracking management, measurement statistics, defect analysis, etc. The platform helps customers fully manage the entire cycle of software testing.

3. Test Data

● Memory

Generally, we pay attention to:

① The size of pss memory (physical memory actually used)

iOS and Andriod devices are generally divided into three levels, and the pss memory standard of each level is different, and the standard value is gradually getting higher with the improvement of hardware equipment.

② Memory leak

The memory used becomes larger and larger after memory leak, it may have lag or crash due to slower and slower memory allocation.

③ Memory resource recurrence rate

Resource recurrence refers to the existence of two or more identical textures, grids, animations, audio, and other resources at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

● BigJank

PerfDog calculation method: If two conditions are met at the same time, it is considered to be a stuck Jank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2

Ø The current frame time-consuming> two movie frame time-consuming (1000ms/24*2=84ms).

If both conditions are met at the same time, it is considered to be a serious lag BigJank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2.

Ø The current frame time-consuming> three movie frame time-consuming (1000ms/24*3=125ms).

● FPS

Frame rate, standard requirements such as: the core game scene requires 90% of the value not lower than 25FPS by default;

● Fluency

For example, the default requirement of lag rate in the core game scene is not higher than 2%.

● CPU

For example, the average overall CPU usage (90%) is less than 60%, and the peak single-core CPU usage (90%) is less than 90%.

Also, pay attention to the single-core CPU usage, and whether all cores are in use, and whether the use is even, etc.

● Commonly Used Data in Tools

Ø CPU

① Frame Time: the time consumption of the same frame

② Rendering Time: The rendering time of the current frame; the higher the draw call, the greater the cost of this part.

③ Script Time: the current frame function time-consuming

④ Physics Time: the physics time of the current frame

Ø Memory

Memory in the code — Mono memory (mono memory, memory pool, only increase but not decrease);

Memory usage in resources — Native memory.

① Reserved Mono: generally the memory allocated by the script

② pss memory: generally used to locate multi-game battles, scene jumps, open and close to check whether there is a memory leak in the UI.

③ Total_reserved memory: unity_reserved memory + GFX memory + FMOD memory + Mono memory + Profiler memory

④ Mono reserved: The allocated mono memory, which only rises but does not fall, needs to be strictly controlled. Mono memory represents the memory allocated by the script in the game.

⑤ gfxdriver_reserved: it indicates the memory of the rendering module. If it is relatively high, the texture and Shader need to be optimized.

⑥ fmod_reserved: it indicates the memory of the audio module, if it is relatively high, the audio resources need to be optimized.

⑦ Recurrence rate of memory resources: Resource recurrence refers to the existence of two or more same resources such as textures, grids, animations, and audios at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

4. Commonly Used Optimization Methods and Strategies

Before optimizing the games, you should know:

① Performance optimization is a long-term process.

② Performance optimization is not only a matter of the program department but the result of the joint cooperation and efforts of the entire project team. Generally, each department of the project team will have corresponding full-time personnel.

③ Good optimization tools are essential.

④ Different devices and equipment show different art quality, so there are different optimization strategies for different models.

⑤ In different scenarios, the choice of time and space (time for space or space for time, CPU and GPU can be exchanged, CPU and memory can be exchanged, memory and disk can be exchanged)

⑥ Don't render unnecessary rendering things; don’t load unnecessary things. Compress what can be compressed.

⑦ Performance optimization is not a once-and-for-all thing, it is a protracted battle; even for long-term products that have been on the line for many years, the performance testers are always on their way.

● Scene optimization

① Pay attention to the hot data of the scene (for example, according to a calculation range unit of 2 meters), optimize the hot issues by adjusting the position of the object, optimizing the resource of the object, etc.

② Add key occluders to reduce the amount of rendering in the camera.

③ The use of LOD technology, or the use of imposter technology, etc.

④ Smooth rendering of distant objects including character monsters.

● Optimization of the number of people on the same screen

① Pay attention to the scene separation, such as the auction house and the transaction place, one indoor and one outdoor; what’s more, reduce the consumption of objects in the corresponding scene.

② Optimize the calculation strategy, give priority to rendering characters and scene objects close to you.

② Effect optimization, for example, the same type of effects will not be rendered after reaching a certain amount.

④ The user role is dynamic at the same screen, loading and releasing continuously, and smoothing (controlling the number of rendering methods by frame, etc.). Avoid stuttering caused by instant loading.

⑤ When fighting on the same screen, especially national warfare, it is the same as the role avatar.

⑥ Imposter (pretender, or substitute) technology: the basic principle is to use the camera to take pictures of all angles of the model, and then select different pictures to display according to the angle of the player’s camera.

● UI optimization

① Format

② Separate dynamic UI elements and static UI elements into different UIPanels as much as possible (UI rebuilding takes UIPanel as the unit), so as to control the refactoring caused by changed UI elements to a smaller scope as much as possible.

③ Atlas with and without channels, separate as far as possible.

④ Putting the UI resources of the same UI interface or the same function point in an atlas can reduce draw calls.

● GC optimization

GC will take up a lot of computing resources. If the memory is still not enough after GC and needs to be allocated again (unofficial data of unity is about 6MB at a time), then it will consume more resources. GC optimization generally includes:

① Object pool technology — — Reuse objects (such as bullets which require frequent generation and destruction)

② Reduce the amount of memory garbage (for example: do not repeatedly allocate heap memory in frequently called functions, which use caching technology)

③ Slow down the time of GC, do not operate GC at critical times; take the initiative to GC during non-sensitive periods.

④ Other technical methods, such as boxing, list, string, etc. lead to GC solutions.

● Memory optimization

The main reasons for memory performance are the following: excessive memory fragmentation (resolved by the memory pool), frequent memory creation and destruction (resolved by the object pool), slow memory loading, and excessive memory usage.

① Resources (textures, models, animations, sounds, etc.) are lossy compressed, and each resource has an optimal compression method and format.

② Script and configuration, timely uninstall, and split, etc.

③ Third-party shader library, redundant pre-compiled macro occupancy

④ AB package packaging and loading strategy will affect memory usage. AB automatic packaging tool (combination of public packages, the AB package should be within 10MB), the game type logic is strongly associated with the packaging mode, such as moba, which packs all the resources of a hero into one package.

● CPU optimization

Here are methods for CPU optimization:

Micro method: find problems in every frame (main method)

Macro method: Find problems from statistics. For example, we can find the problem for a battle from statistics on the time-consuming and call times of each module and function, and it is also convenient for comparison of different versions.

Classification:

① GC, GCAlloc time-consuming

② Rendering time-consuming (combine batch processing, reduce draw call: such as using dynamic and static batch processing, GPU Instance technology)

③ Logical time-consuming (such as circular search, centralized processing of a large number of server messages, etc.)

④ Other engine consumption (animation, physics)

● GPU optimization

GPU optimization ideas:

① Reduce rendering batches

② In the same draw call, reduce the rendering type.

③ Smooth rendering.

● Performance monitoring

① Internal monitoring

Have performance monitoring when uploading resources and performing version testing.

② External network performance monitoring.

● Power consumption

Optimization ideas:

① Calculation optimization: Algorithm, for loop optimization, Switch..case instead of if..else, avoid floating-point operations.

② Avoid improper use of Wake Lock

③ Use Job Scheduler to manage background tasks

④ Reduce brightness, etc.

5. Summary

The core idea of mobile game performance testing is more about designing reasonable test cases, analyzing the data after the test is completed, and providing reasonable optimization suggestions based on the problem data so as to provide efficient services to the R&D team and reflect value.

Let’s get started.

1. Testing Scenarios

The front-end performance test focuses on the user’s experience during gameplay, including whether it’s stuck, whether it’s crashing, whether the phone is hot, whether it consumes too much power, etc., regardless of whether the user’s phone is a high-end phone, a medium-end phone, or very low-configuration all need to pay attention to the user performance experience.

● Lag

The fluency of the game is a relatively basic performance indicator, especially in the core gameplay. From a testing perspective, pay more attention to the game’s fps data, frame lag rate, BigJank value, etc.

● Crash

Game players don’t want to have crashes when playing games, so pay attention to the crash rate when testing.

● Overheating

After the mobile phone heats up, in addition to the bad body feeling, some mobile phone manufacturers will choose to reduce the frequency or lock the core, which will cause the game lag. There are many reasons for mobile phone heating (such as running high-power apps, charging, bad batteries, the protective case can not dissipate heat, high ambient temperature, etc.). in this case, what testers will focus on is how to reduce the power consumption of the game, especially GPU and CPU usage.

● Too fast power consumption

Mobile phone battery milliampere is itself an important indicator of mobile phone, and fast power consumption is undoubtedly a flaw.

2. Test Tools

● perfdog

This performance tool is provided by Tencent. PerfDog supports all applications on the mobile platform (games, applications, browsers, mini-programs, mini-games, H5, back-end system processes, etc.)

● UWA

It provides services for game or VR products of unity and unreal engines, including resource monitoring (dynamic and static), performance diagnosis and optimization, Lua performance analysis, automated testing, community Q&A, etc.

UWA GOT is a local performance evaluation tool launched by UWA. It needs to install the corresponding SDK, without jailbreak or root.

● Test Writer

It is a codeless automated testing tool developed by 51Testing. Test Writer has 9 modules which include project management, test objects, test steps, test cases, test plans, test missions, automated testing, and test reports.

● Integration Test Platform (ITP)

Integration Test Platform (ITP) is an interface-oriented integration test automation tool independently developed by 51Testing, covering interface use case design, automated test script execution, automated test execution planning and task allocation, interface test management, and automated testers management and other functions. This tool is suitable for systems or products with high software reliability, high security, and high robustness, as well as for agile development processes.

● Test Platform

Test Platform is a software testing platform developed by 51Testing covering the whole process of software quality assurance such as test plan management, project management, demand management, test demand analysis, test case design, test execution management, defect tracking management, measurement statistics, defect analysis, etc. The platform helps customers fully manage the entire cycle of software testing.

3. Test Data

● Memory

Generally, we pay attention to:

① The size of pss memory (physical memory actually used)

iOS and Andriod devices are generally divided into three levels, and the pss memory standard of each level is different, and the standard value is gradually getting higher with the improvement of hardware equipment.

② Memory leak

The memory used becomes larger and larger after memory leak, it may have lag or crash due to slower and slower memory allocation.

③ Memory resource recurrence rate

Resource recurrence refers to the existence of two or more identical textures, grids, animations, audio, and other resources at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

● BigJank

PerfDog calculation method: If two conditions are met at the same time, it is considered to be a stuck Jank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2

Ø The current frame time-consuming> two movie frame time-consuming (1000ms/24*2=84ms).

If both conditions are met at the same time, it is considered to be a serious lag BigJank.

Ø The current frame time-consuming> the average consuming time of the previous three frames x 2.

Ø The current frame time-consuming> three movie frame time-consuming (1000ms/24*3=125ms).

● FPS

Frame rate, standard requirements such as: the core game scene requires 90% of the value not lower than 25FPS by default;

● Fluency

For example, the default requirement of lag rate in the core game scene is not higher than 2%.

● CPU

For example, the average overall CPU usage (90%) is less than 60%, and the peak single-core CPU usage (90%) is less than 90%.

Also, pay attention to the single-core CPU usage, and whether all cores are in use, and whether the use is even, etc.

● Commonly Used Data in Tools

Ø CPU

① Frame Time: the time consumption of the same frame

② Rendering Time: The rendering time of the current frame; the higher the draw call, the greater the cost of this part.

③ Script Time: the current frame function time-consuming

④ Physics Time: the physics time of the current frame

Ø Memory

Memory in the code — Mono memory (mono memory, memory pool, only increase but not decrease);

Memory usage in resources — Native memory.

① Reserved Mono: generally the memory allocated by the script

② pss memory: generally used to locate multi-game battles, scene jumps, open and close to check whether there is a memory leak in the UI.

③ Total_reserved memory: unity_reserved memory + GFX memory + FMOD memory + Mono memory + Profiler memory

④ Mono reserved: The allocated mono memory, which only rises but does not fall, needs to be strictly controlled. Mono memory represents the memory allocated by the script in the game.

⑤ gfxdriver_reserved: it indicates the memory of the rendering module. If it is relatively high, the texture and Shader need to be optimized.

⑥ fmod_reserved: it indicates the memory of the audio module, if it is relatively high, the audio resources need to be optimized.

⑦ Recurrence rate of memory resources: Resource recurrence refers to the existence of two or more same resources such as textures, grids, animations, and audios at the same time in the memory. Generally, the same resource is packaged into multiple AB packages. If these AB packages are loaded into the memory, there will be multiple copies of the same resource in the memory.

4. Commonly Used Optimization Methods and Strategies

Before optimizing the games, you should know:

① Performance optimization is a long-term process.

② Performance optimization is not only a matter of the program department but the result of the joint cooperation and efforts of the entire project team. Generally, each department of the project team will have corresponding full-time personnel.

③ Good optimization tools are essential.

④ Different devices and equipment show different art quality, so there are different optimization strategies for different models.

⑤ In different scenarios, the choice of time and space (time for space or space for time, CPU and GPU can be exchanged, CPU and memory can be exchanged, memory and disk can be exchanged)

⑥ Don't render unnecessary rendering things; don’t load unnecessary things. Compress what can be compressed.

⑦ Performance optimization is not a once-and-for-all thing, it is a protracted battle; even for long-term products that have been on the line for many years, the performance testers are always on their way.

● Scene optimization

① Pay attention to the hot data of the scene (for example, according to a calculation range unit of 2 meters), optimize the hot issues by adjusting the position of the object, optimizing the resource of the object, etc.

② Add key occluders to reduce the amount of rendering in the camera.

③ The use of LOD technology, or the use of imposter technology, etc.

④ Smooth rendering of distant objects including character monsters.

● Optimization of the number of people on the same screen

① Pay attention to the scene separation, such as the auction house and the transaction place, one indoor and one outdoor; what’s more, reduce the consumption of objects in the corresponding scene.

② Optimize the calculation strategy, give priority to rendering characters and scene objects close to you.

② Effect optimization, for example, the same type of effects will not be rendered after reaching a certain amount.

④ The user role is dynamic at the same screen, loading and releasing continuously, and smoothing (controlling the number of rendering methods by frame, etc.). Avoid stuttering caused by instant loading.

⑤ When fighting on the same screen, especially national warfare, it is the same as the role avatar.

⑥ Imposter (pretender, or substitute) technology: the basic principle is to use the camera to take pictures of all angles of the model, and then select different pictures to display according to the angle of the player’s camera.

● UI optimization

① Format

② Separate dynamic UI elements and static UI elements into different UIPanels as much as possible (UI rebuilding takes UIPanel as the unit), so as to control the refactoring caused by changed UI elements to a smaller scope as much as possible.

③ Atlas with and without channels, separate as far as possible.

④ Putting the UI resources of the same UI interface or the same function point in an atlas can reduce draw calls.

● GC optimization

GC will take up a lot of computing resources. If the memory is still not enough after GC and needs to be allocated again (unofficial data of unity is about 6MB at a time), then it will consume more resources. GC optimization generally includes:

① Object pool technology — — Reuse objects (such as bullets which require frequent generation and destruction)

② Reduce the amount of memory garbage (for example: do not repeatedly allocate heap memory in frequently called functions, which use caching technology)

③ Slow down the time of GC, do not operate GC at critical times; take the initiative to GC during non-sensitive periods.

④ Other technical methods, such as boxing, list, string, etc. lead to GC solutions.

● Memory optimization

The main reasons for memory performance are the following: excessive memory fragmentation (resolved by the memory pool), frequent memory creation and destruction (resolved by the object pool), slow memory loading, and excessive memory usage.

① Resources (textures, models, animations, sounds, etc.) are lossy compressed, and each resource has an optimal compression method and format.

② Script and configuration, timely uninstall, and split, etc.

③ Third-party shader library, redundant pre-compiled macro occupancy

④ AB package packaging and loading strategy will affect memory usage. AB automatic packaging tool (combination of public packages, the AB package should be within 10MB), the game type logic is strongly associated with the packaging mode, such as moba, which packs all the resources of a hero into one package.

● CPU optimization

Here are methods for CPU optimization:

Micro method: find problems in every frame (main method)

Macro method: Find problems from statistics. For example, we can find the problem for a battle from statistics on the time-consuming and call times of each module and function, and it is also convenient for comparison of different versions.

Classification:

① GC, GCAlloc time-consuming

② Rendering time-consuming (combine batch processing, reduce draw call: such as using dynamic and static batch processing, GPU Instance technology)

③ Logical time-consuming (such as circular search, centralized processing of a large number of server messages, etc.)

④ Other engine consumption (animation, physics)

● GPU optimization

GPU optimization ideas:

① Reduce rendering batches

② In the same draw call, reduce the rendering type.

③ Smooth rendering.

● Performance monitoring

① Internal monitoring

Have performance monitoring when uploading resources and performing version testing.

② External network performance monitoring.

● Power consumption

Optimization ideas:

① Calculation optimization: Algorithm, for loop optimization, Switch..case instead of if..else, avoid floating-point operations.

② Avoid improper use of Wake Lock

③ Use Job Scheduler to manage background tasks

④ Reduce brightness, etc.

5. Summary

The core idea of mobile game performance testing is more about designing reasonable test cases, analyzing the data after the test is completed, and providing reasonable optimization suggestions based on the problem data so as to provide efficient services to the R&D team and reflect value.

advance but testers check the system when processing. The focus of exploratory testing is more on testing as a “thinking” activity. ”

The test methods mentioned in the book Exploratory Testing are very useful for game testing. And this is what we will share today.

Guide Test Method

It is to review the functions strictly in accordance with the plan. This is the most basic and most important method.

Selling Point Test Method

This method is to encourage testers to watch the demo from the point of view of sales and understand the biggest selling point of the game. The selling point is not necessarily the core function of the game, but it must be given the same attention as the core function. For example, the biggest selling point of the mobile game “League of Goddess” is naturally the goddess, but the goddess is not the core function of the game, but auxiliary gameplay, but players are keen on this, it is very necessary to test more.

Landmark Test Method

The method is that if we want to go to a place, we usually set the flags representing the steps on the map, but no one stipulates where we should follow the steps. Since there are quite a lot of related functions in the game, the landmark test method can be said to be a very commonly used method when we test. For example, if the player kills a monster, the player will get +1 experience point(EXP). Will the player gain EXP by just hitting the last monster? Will the player gain experience point when the monster is killed while also dying?

Limit Test Method

Test the limit of the functional program. For example, one hundred players attacked a monster at the same time and check the result.

Late Night Test Method

This method is mainly used for some functions such as data backup and has little connection with game testing.

Museum Test Method

This method is used for long-term projects. Some old functions have been running normally. After new functions are released, whether the old functions are still normal.

Deep Alley Test Method

This method is mainly used for observing some points where no one pays attention, such as the function of HELP.

Overnight Test Method

This method is used to checking whether the function is normal when it runs continuously for a long time.

Long Path Test Method

Check the result that requires the most operation steps, and every step to reach this result.

Supermodel Test Method

You may think this method not very useful at first glance, isn’t it just looking at the interface? I watch it every day. But in fact, it is different to look at the interface with the mentality of finding bugs. For example, I look at an enhanced interface every day, but until I plan to find bugs, I really discovered that the tabbed text on this interface is a few pixels crooked compared to the text on other tabs. This is what I haven’t noticed before. Interface in the game is a very important thing, so the use of supermodel testing is of great help to us.

Cancel Test Method

Close the operation interface, close the application, switch the line when the program is processing to check whether there is bugs.

Sluggard Test Method

Choose the most trouble-free step to achieve the result, and check whether the result is correct.

Obsessive-compulsive Disorder Test Method

Keep repeating the operation to check whether the function responds normally. I think the most likely thing in the game is to use a mouse simulator to continuously click.

We use different methods to test different games and it is necessary to keep summarizing which methods are often used in the games.

For test engineers looking for game testing jobs, it is necessary to accumulate different game testing methods. For exploratory testing in game testing, I think that the methods mentioned above are very helpful. In software testing, you can use brain maps to write test methods and use cases. The more you write, the more proficient you will be.