Candidate: Mustafa Mohammed Al-tekreeti
Title: Methodologies for Evaluating User Centric Performance of Mobile Networked Applications.
Date: September 10, 2018
Time: 1:00 PM
Place: DC 1304
Supervisor(s): Naik, Kshirasagar
Abstract:
Performance is an important attribute of mobile software applications, having a direct impact on end-user's experience. One of the obstacles that make software performance testing difficult to pursue is the lack of performance requirements that complicates the process of verifying the correctness of the test case output. This problem is well known in software testing as the lack of required test oracles. Moreover, compared to other platforms, mobile apps' quality assurance is more challenging, since their functionality is affected by the surrounding environment. To address these challenges, we adopt model-based approach to evaluate how wireless network conditions impact the performance behaviour and end user experience of real-time mobile networked applications.
The thesis starts by defining the system model of software applications that we target, the network stack that the application is assumed to use to provide the service to the end user, and the metric used to capture the quality of the provided network service. Then, an analytical performance model that captures the application-network interactions is developed using the Markovian framework. To model realistic interactions with the network, the performance model is developed and solved using supplementary variable technique (SVT). The model is intensively verified with simulation.
Furthermore, two input network models are analytically developed. In both models, the mobile application is assumed to have a wireless network access through a Wi-Fi access point that implements IEEE 802.11 protocol. In the first model, data transfer is achieved using user datagram protocol (UDP), while in the second, data transfer is accomplished using transmission control protocol (TCP). Both models are verified using the well-known NS2 network simulator.
Third, we propose a model based test generation methodology to evaluate the impact of the interaction of the environment, the wireless network, and the app configurations on the performance of a real-time mobile networked app. The methodology requires four artefacts as inputs, namely, a behaviour model of the software under test, a network model, test coverage criteria, and a set of desired performance levels. The methodology consists of three steps: performance model development, test generation, and estimation of test execution parameters. To evaluate the end-user quality of experience, test generation is formulated as an inversion problem and solved as an optimization problem. To generate an efficient set of test cases, two test coverage criteria are proposed: user experience (UE) and user experience and input interaction (UEII). Test execution optimizations are inferred using a performance simulation model. To show the applicability of the methodology, two mobile networked app examples are used: multimedia streaming and web browsing. The effectiveness of the methodology is evaluated by comparing the time cost to design a test suite with random testing. The obtained results are very promising.