Development of a valid and reliable software customization model for SaaS quality through iterative method: perspectives from academia

Despite the benefits of standardization, the customization of Software as a Service (SaaS) application is also essential because of the many unique requirements of customers. This study, therefore, focuses on the development of a valid and reliable software customization model for SaaS quality that consists of (1) generic software customization types and a list of common practices for each customization type in the SaaS multi-tenant context, and (2) key quality attributes of SaaS applications associated with customization. The study was divided into three phases: the conceptualization of the model, analysis of its validity using SaaS academic-derived expertise, and evaluation of its reliability by submitting it to an internal consistency reliability test conducted by software-engineer researchers. The model was initially devised based on six customization approaches, 46 customization practices, and 13 quality attributes in the SaaS multi-tenant context. Subsequently, its content was validated over two rounds of testing after which one approach and 14 practices were removed and 20 practices were reformulated. The internal consistency reliability study was thereafter conducted by 34 software engineer researchers. All constructs of the content-validated model were found to be reliable in this study. The final version of the model consists of 6 constructs and 44 items. These six constructs and their associated items are as follows: (1) Configuration (eight items), (2) Composition (four items), (3) Extension (six items), 4) Integration (eight items), (5) Modification (five items), and (6) SaaS quality (13 items). The results of the study may contribute to enhancing the capability of empirically analyzing the impact of software customization on SaaS quality by benefiting from all resultant constructs and items.


4.
Tenants can create customization based on templates.

5.
Tenants can select their own workflow templates and items relating to SaaS application templates from the template repository.

6.
A set of components are provided in the application template which facilitates a variety of tenant needs. By making a choice from the relevant component set, tenants can personalize each customization point.

7.
When a tenant wishes to subscribe to the SaaS application, the capabilities of each feature within the system are analyzed to determine whether they ought to be assimilated within the application.
8. Configuration can manage incongruities by permitting the client to establish set pre-defined parameters and options within the context of the runtime.

9.
The configuration of the SaaS application involves disabling or excluding some features of the application.
Composition refers to techniques and solutions that bring together a distinct collection of pre-defined application components that jointly amount to a custom solution.
Please indicate to what relevancy you feel these statements represent the composition approach in the SaaS Multi-Tenant context.

2.
Composing different collaboration components is done according to the runtime of the SaaS application.
3. The composition of SaaS components takes into account the subset of components.

4.
The composition approach supports the decomposition of SaaS components.

5.
Performing the composition of SaaS application components considers the relationships and dependencies between these components.

SaaS Extension
Extension refers to techniques and solutions that that stretch the functionality of the application by implanting the custom code in pre-defined places of application's code.
Please indicate to what relevancy you feel these statements represent the extension approach in the SaaS Multi-Tenant context.

2.
The SaaS application provides a set of extension points which permit a customized service to be plugged in at virtually points in the application.

3.
Extending an existing object can happen at SaaS application runtime.

The SaaS service provider supplies an open platform
and an API, which allows developers to inject custom codes into business object layers.

5.
These extension points can either be replacements for existing objects or extensions to them.
6. An extension may be private to an individual tenant or shared by multiple tenants.

SaaS Integration
Integration refers to techniques and solutions that Implement third-party components designed to work with the application.
Please indicate to what relevancy you feel these statements represent the integration approach in the SaaS Multi-Tenant context.

SaaS Modification
Modification refers to techniques and solutions that alter the application design and other functional requirements of the application by way of alterations implemented on the source code.
Please indicate to what relevancy you feel these statements represent the modification approach in the SaaS Multi-Tenant context.

Part 3: SaaS Quality
Based on the definition provided for each quality attribute, please indicate to what relevancy you feel these statements represent the quality attributes of SaaS application that play an important role in customization. 13. Response time: There is defined time limit which is adhered to between a service request and a service response.

Comments and Suggestions
If there are any other statements, or further comments regarding the customization approaches or the quality attributes of SaaS applications that you think is needed and have not reflected in this survey, please add your remarks in the space provided below.

SaaS Configuration
Configuration refers to techniques and solutions that offer a pre-defined setting for the alteration of application functions within the pre-defined scope.
Please indicate to what relevancy you feel these statements represent the configuration approach in the SaaS Multi-Tenant context.

4.
Tenants can create customization based on templates.

5.
Tenants can select their desired workflow templates and items relating to SaaS application templates from the template repository.

6.
When a tenant wishes to subscribe to the SaaS application, the capabilities of each feature within the system are analyzed to determine whether they ought to be assimilated within the application.

7.
All Configurations established by the tenants have to be within the context of the runtime of the application.
8. An option of disabling or excluding some features of the SaaS application should be provided with the isolation effect across the tenants.

SaaS Composition
Composition refers to techniques and solutions that bring together a distinct collection of pre-defined application components that jointly amount to a custom solution.
Please indicate to what relevancy you feel these statements represent the composition approach in the SaaS Multi-Tenant context.

SaaS Extension
Extension refers to techniques and solutions that expand the functionality of the application by inserting the custom code in pre-defined places of application's code.
Please indicate to what relevancy you feel these statements represent the extension approach in the SaaS Multi-Tenant context. and an API, which allows developers to inject custom codes into business object layers.

5.
These injected codes can either be replacements for existing objects or extensions to them.
6. An extension may be private to an individual tenant or shared by multiple tenants.

SaaS Integration
Integration refers to techniques and solutions that implement third-party components designed to work with the application.
Please indicate to what relevancy you feel these statements represent the integration approach in the SaaS Multi-Tenant context.

SaaS Modification
Modification refers to techniques and solutions that alter the application design and other functional requirements of the application by means of alterations implemented to the source code.
Please indicate to what relevancy you feel these statements represent the modification approach in the SaaS Multi-Tenant context. 13. Response time: SaaS application adheres to a defined time limit between service request and service response.

Comments and Suggestions
If there are any other statements, or further comments regarding the customization approaches or the quality attributes of SaaS applications that you think is needed and have not reflected in this survey, please add your remarks in the space provided below.

Part 1: Demographics
Please mark your response for each of the following questions:

Part 2: SaaS Customization Approaches
The following statements describe the different customization approaches that may impact the quality of SaaS applications. The scales below represent opinions of equivalent weight and strength. You should select the responses which most closely correspond to your views.
For each question, you must choose one scale ONLY.

SaaS Configuration
Configuration refers to techniques and solutions that offer a pre-defined setting for the alteration of application functions within the pre-defined scope.
Please indicate the extent to which, in your view, the following statements represent the configuration approach in the SaaS Multi-Tenant context.

Questions Strongly Disagree
Disagree Neither Nor Agree Strongly agree

1.
Configuration typically maintains diversity by establishing pre-defined parameters, options, and components, and treats each tenant individually.

2.
Each tenant can configure the application in a standalone way by employing techniques to modify the functions of applications within established limits.

3.
SaaS providers have to develop and capture sets of services and plugins, from which tenants can make selections and perform configurations.

4.
Tenants can create customization based on templates.

5.
Tenants can select their desired workflow templates and items relating to SaaS application templates from the template repository.

6.
When a tenant wishes to subscribe to the SaaS application, the capabilities of each feature within the system are analyzed to determine whether they ought to be assimilated within the application.

7.
All Configurations established by the tenants have to be within the context of the runtime of the application.
8. An option of disabling or excluding some features of the SaaS application should be provided with the isolation effect across the tenants.

SaaS Composition
Composition refers to techniques and solutions that bring together a distinct collection of pre-defined application components that jointly amount to a custom solution.
Please indicate the extent to which you feel the following statements represent the composition approach in the SaaS Multi-Tenant context.

SaaS Extension
Extension refers to techniques and solutions that expand the functionality of the application by inserting the custom code in pre-defined places of application's code.
Please indicate the extent to which, in your view, you feel the following statements represent the extension approach in the SaaS Multi-Tenant context.

Questions Strongly Disagree
Disagree Neither Nor Agree Strongly agree

1.
The SaaS application is extended by adding custom code to extend the application through custom functionality.

2.
The SaaS application provides a set of extension points which permit a customized service to be plugged in at virtual points in the application.

3.
Injecting custom code into SaaS application has to be supported at the run time of the application.

Strongly Disagree
Disagree Neither Nor Agree Strongly agree

4.
The SaaS service provider supplies an open platform and an API, which allows developers to inject custom codes into business object layers.

5.
These injected codes can either be replacements for existing objects or extensions to them.
6. An extension may be private to an individual tenant or shared by multiple tenants.

SaaS Integration
Integration refers to techniques and solutions that implement third-party components designed to work with the application.
Please indicate the extent to which you feel the following statements represent the integration approach in SaaS Multi-Tenant context.

SaaS Modification
Modification refers to techniques and solutions that alter the application design and other functional requirements of the application by means of alterations implemented to the source code.
Please indicate the extent to which you feel the following statements represent the modification approach in the SaaS Multi-Tenant context.

Part 3: SaaS Quality
Based on the definition provided for each quality attribute, please indicate the extent to which you feel each quality attribute plays an important role in customization.

Questions Strongly Disagree
Disagree Neither Nor Agree Strongly agree

1.
Multi-tenancy: SaaS services can support instances of simultaneous access by multiple users for multiple tenants.

2.
Scalability: SaaS providers can manage growth or decline in the level of services.

3.
Availability: SaaS services can function within a specific time to satisfy users' needs.

4.
Reliability: SaaS application maintains operating and functioning under given conditions without failure within a given time period.

5.
Maintainability: Modifications to the application are made by SaaS provider to retain it in the condition of good repair.

6.
Security: the effectiveness of SaaS provider's controls on service data, access to the services, and the physical facilities from which service are provided.

7.
Usability: the ease with which SaaS application can be used to achieve tenant-specific-goal.

8.
Interoperability: SaaS service can easily interact with other services from the same SaaS provider or other providers.

9.
Efficiency: SaaS services effectively utilize resources to perform their functions. 13. Response time: SaaS application adheres to a defined time limit between service request and service response.