Testing and Validating of An Integrated Operational Excellence and Performance Improvement Framework for Organizations in the Sudanese Aviation Industry

The purpose of this study is to test and validate the conceptual operational excellence framework that will be adopted and implemented by organizations in the Sudanese aviation industry and to determine if it will result in improved organizational performance and competitive advantage, quantify the benefits therein, and in so doing, cater to as many concerns as possible as raised by business owners and managers. Many industries, especially the aviation industry, have experienced numerous booms and bust cycles, rarely reacting in a responsible way to either highs or lows. These conditions forced organizations to respond reactively to survive prevailing challenges and seek long-term success by achieving excellence in their operations. Assessing the reliability of any framework requires researchers to judge the soundness of the proposed framework about the appropriate assessment criteria. This assessment criteria should be able to evaluate the effectiveness and success of the use of the methodology. In this study, the proposed operational excellence framework will be judged based on three criteria: 1) Feasibility and Suitability, 2) Usability, 3) Utility. Three case companies were selected to assess the validity of the proposed operational excellence framework. The results and feedback provided by the case participants are analyzed to assess and evaluate the proposed operational excellence framework.


Development of the Assessment Criteria
Evaluating the quality of research outcome is essential if findings are to be utilized in practice and incorporated into operational excellence. Assessing the reliability of any framework requires researchers to judge the 'soundness of the proposed framework about the appropriate assessment criteria. This assessment criteria should evaluate the effectiveness and success of the use of the methodology (Tan, Platts, and Noble, 2004). (Platts, 1993) Furthermore, (Tan et al., 2004) suggest that successful tests of any practical methodology should constitute the following criteria: · Feasibility and Suitability Assessment -Does the framework is suitable for the organization and could be implemented? · Usability -How easily could the methodology be followed? · Utility (Usefulness) -Did the methodology provide beneficial results that met expectations? Hence this research adopted the feasibility, usability, and utility assessment criteria suggested by (Platts 1993) and (Tan et al., 2004) as described in Fig 1: which presents the Summary of the assessment criteria to evaluate the proposed framework.

Formulation of the Data Collection Action Plan
The data collection action plan based on the different assessment criteria to evaluate the proposed framework was designed to address the activities (steps to be taken), responsible persons, Data collection technique, Date collection time frame, monitoring, and evaluation indicators. Table 1 presents the data collection action plan.

Data collection instruments
After the data collection action plan is formulated, the next activity is to determine the research instruments used to gather the data. In this stage, the researcher acts as the facilitator. According to (Yin, 2003a) there are six possible sources of evidence for case studies: documents, archival records, interviews, direct observation, participant observation, and physical artifacts. Indeed, the case study's unique strength is "its ability to deal with a full variety of evidence-documents, artifacts, interviews, and observations" (Yin, 2003a). Case studies do not imply using a particular type of evidence and can be done using either qualitative or quantitative evidence.  Vol.13, No.15, 2021 Additionally, interviewing is one of the most important sources of case study information. Hence, in this research stage, the researcher adopted a combination of interviewing techniques and questionnaires as the primary sources of data. The questions during the interview were focused on what the participants feel about feasibility and suitability, usability and utility of the proposed operational excellence, as illustrated in table 2 below. The survey results were from the basis to validate of Operational Excellence framework for the aviation industry in Sudan. The participants were asked to answer the questions in this questionnaire. Table 2: Questions used to evaluate the feasibility and suitability, usability, and utility of the proposed operational excellence Assessment Criteria

Feasibility and Suitability Assessment
The framework is suitable to the organization and could be implemented

Applicability
The operational Excellence framework can be applied satisfactorily in the organization

Appropriateness
The framework approach/format for improvement is appropriate to the organizational context and internal processes

Implementation Intervals / Timing
The timing for the implementation of the framework is suitable and aligned with other organizational processes

Participation
It is easy to get people involved to join the implementation of the framework

Coherence
The framework is compatible with other performance improvement initiatives implemented by the organization

Usability Assessment
The framework could be easily implemented

Clarity
Operational Excellence framework purpose, concepts, description, and explanation are clear

Availability of information
It is easy to obtain the information required for the framework implementation.

Ease of use
The framework elements /processes /steps are easy to follow and implement

Flexibility
The framework and its implementation stages enable the organization to adapt quickly to new or changing environments

Integration
The framework can be integrated with other organizational processes and activities

Utility Assessment
The framework provides valuable results that can meet the expectation

Relevance
The operational Excellence framework is relevant to the organization and its context, e.g., Strategic fit, organization fit

Efficiency
The framework consumes suitable resources of time and people

Benefits/Effectiveness
There are many financial and other benefits and lessons learned associated with the framework application

Practicality
The framework methodology adapted or designed for actual use, i.e., it can be put into practice

Satisfaction
The expected results and outcomes of the Framework application meet the predefined expectations

Consistency
The elements of the framework are consistent, which enables conformity in the framework application

Selection of the Case Companies
Characteristics for selecting case study research versus other approaches focus on how or why kinds of research questions are directed at exploring and understanding some phenomenon in-depth (Yin, 2018). Once the decision is made to use a case study for the validation purpose, a critical decision involves selecting the case itself. (Yin, 2018) noted the two required elements as "defining the case and bounding the case." The former relates to clearly and concretely indicating the case, which can be a person, place, thing, organization, or phenomenon. The latter relates to scope-what is, and is not, included in the case, whether from time, structure, or other perspectives. Once the case has been identified, determine the data needed and how those data will be collected. The selection of suitable case companies with their willingness to collaborate with research and accessibility is essential for the methodology studied (Stuart, McCutcheon, Handfield, McLachlin, and Samson, 2002). This section provides the justification and explanation in selecting these case companies to assess and evaluate the proposed operational excellence framework. As per (Rowley, 2002), the case selection is determined by the research purpose, questions, propositions, theoretical context, and other constraints such as accessibility, resources, and time available.
The number of cases to study is determined by replication logic (Hersen and Barlow, 1976). According to (Yin, 2003b) two or three cases would be literal replications, while four or more might be conducted to pursue two different patterns of theoretical replications. (Stuart et al., 2002) stated that using well-known firms with good performance records will provide representative information, and hence it is worthwhile for an investigation. Therefore, for proving the validity of the methodology, three cases are selected for the assessment and validation of the proposed operational excellence Framework. Descriptions of the profile and background of all three cases are summarized and analyzed in the following section.

Profile and background of the Case Companies
Three case companies were selected to assess the validity of the proposed operational excellence framework. The case studies were named cases A, B, and C. The companies' profile includes sector and scope of work, type of ownership, years of operation, location and number of employees, and other relevant information.

Case A: Safat Aviation Group (SAG)
SAFAT Aviation Group has built its infrastructure to serve the aviation industry in Sudan and neighboring countries since 2007 and is now considered the pioneer and leading aviation industry in Sudan.
SAFAT Aviation Group is owned and operated by the SAFAT Aviation Group and currently represents a strategic role in Defense Industries System and includes homogenous companies that work together to develop the aviation industry in Sudan.
SAFAT Aviation Group has a distinguished relationship with pioneers in the aviation industry, where this relationship is based on the mutual benefits for SAFAT and its partners. These partnerships support the Group in terms of training, development, capacity building, and logistics support.
The Group's activities include Aircraft Maintenance and overhaul Services (MRO); Air Operation Service and Commercial Air Transport; Aviation Research And Development; Aircraft Manufacturing and Aviation Training Services.
SAFAT MRO COMPLEX is considered the primary subsidiary of SAFAT AVIATION GROUP and considered Approved Maintenance Organization (AMO), which undertakes aircraft maintenance, component, and accessories.
The scope of work performed under the approval "CLASS" (aircraft Base maintenance) is limited to Periodical Maintenance and Approved Major Repair.
The scope of work under the approval "CLASS" (Engines, Transmissions, Rotors, and Components) in Tests, Repairs, Parts replacement, Modifications, Calibration, and control, Within the limits of the approved capability lists.
The scope of work under the approval "CLASS" (Specialized Services) is radiographic, Ultra-Sonic, Eddy Current, Liquid Penetrant, Magnetic Particle, Welding, Painting, and Calibration.
In addition to aircraft maintenance and overhaul services, the SAFAT MRO complex provides logistic support services that are industry-focused, customized, and integrated suite of Aviation technical support services and spares services. Moreover, by adopting a holistic approach, the Complex creates optimized solutions that integrate project clearing, forwarding, transportation, warehousing systems, and processes.
SAFAT Air Operation Service and Commercial Air Transport consist of Air Transport Company, which establishes local, regional and international Air transportation services. In addition to Crop Protection Services CO, aerial spraying services national agricultural projects use a fleet of aerial spraying aircraft.
SAFAT Manufacturing Complex consists of Aeronautical Research Center which established to provide creative solutions (economical and high quality) to support the aviation industry in Sudan and Aircraft Manufacturing Center which manufacture and assemble lightweight airplanes for trainer aircraft in Sudan.

Figure 3: Result of Feasibility and Suitability Analysis
All evaluators from the three case studies agreed that the framework is compatible with other performance improvement initiatives implemented by their organizations and would support the aviation industry to achieve the required outcomes. Participants from SAFAT Aviation Group agreed that the framework approach/format for improvement is appropriate to the organizational context and internal processes. Hence it can lead to the achievement of the optimum results and outcomes. It was observed that the 'Participation' sub-criterion achieved the lowest score in all cases, and this means the proposed operational excellence framework needs to be improved, and this element is considered as one of the main areas for improvement in the future. In terms of the Implementation Intervals / Timing, the rating remained between 72 % and 81 % in Tarco aviation company and Green Flag Aviation company. In the case of SAFAT Aviation Group, the score is higher as the organization is more familiar with performance improvement techniques and different excellence frameworks. Overall, the 'Applicability' and 'Appropriateness' sub-criteria are high in all case companies, which validates the framework.

Usability Assessment
The analysis of the results and feedback from the three case studies showed the evaluator's opinions about the usability of the proposed operational excellence framework. Figure 4 shows the results of usability analysis.

Figure 4:
The Results of Usability Analysis All participants agreed that the Operational Excellence framework's purpose, concepts, description, and explanation were clear. It was observed that usability criteria including Clarity, Availability of information, Ease of use, Flexibility, and Integration, obtained an average score of 79.4% in all the case companies. This indicated that most participants had similar positive feedback regarding the framework implementation. The framework and its elements were found to be easy to follow by the participants. Additionally, all evaluators from the three case studies agreed that the available information gave them a clear understanding of the operational excellence framework and how to use it to achieve the required outcomes. The participants from SAFAT Aviation Group agreed that the framework could be integrated with other organizational processes and activities. Moreover, they suggested that providing a complete manual and examples as a guideline for implementation would better understand the practitioners. European Journal of Business and Management www.iiste.org ISSN 2222-1905(Paper) ISSN 2222-2839(Online) Vol.13, No.15, 2021

Utility Assessment
The utility assessment criteria obtained 80.4%, which is considered as the highest score amongst another criterion. This indicates that most participants had similar positive feedback on the usefulness of the proposed operational excellence framework. Figure 5 shows that the utility assessment obtained the highest score amongst the three main criteria of feasibility and suitability, usability, and utility. This validates the framework regarding Relevance, Efficiency, Benefits/Effectiveness, Practicality, Satisfaction, and Consistency. All participants from the case studies companies agreed that the framework for achieving operational excellence in the Sudanese aviation industry is a beneficial approach to determine issues, diagnose problems and opportunities for improvement systematically and rationally with minimum use of resources in terms of time, people and cost.

Figure 5:
The Results of Utility Analysis Participants from TARCO Aviation agreed that the elements in the framework were consistent, which enables conformity in the framework application and easy to get people to be involved in the improvement processes and projects. Green Flag Aviation rated the utility of framework at 79.5 %, SAFAT Aviation Group and TARCO Aviation rated the utility at 79.7 % and 82 %, respectively. The satisfaction of the framework was rated at 79 % based on the average scores of the three companies. This reflects the high confidence in the framework and its role in achieving operational excellence in the Sudanese aviation industry. Additionally, the framework implementation can also assist in achieving the targeted results and output with regards to organization strategy, environment, competitive advantages, and other business aspects. SAFAT Aviation Group, TARCO Aviation, and Green Flag Aviation confirmed that the Operational Excellence framework is relevant to the organization and its context, e.g., Strategic fit, organization fit. Additionally, applying such a comprehensive approach for continuous improvement is considered a simple way to determine the improvement projects, diagnose the root cause, develop and deploy robust and integrated solutions, and clear a road map to achieve their operational excellence journey.

Overall Assessment
The overall average result of the assessment was rated against the three main criteria of feasibility and suitability, usability, and utility. The assessors had a pretty high level of satisfaction with the feasibility and suitability, usability, and utility criteria, with a total average of 80.1%. The overall impression gained from the assessors was that the framework was advantageous in providing a systematic approach to achieving operational excellence in the Sudanese aviation Industry. Figure 6 shows the overall assessment result of the three criteria.

Conclusion:
A high level of acceptance was observed on the proposed operational excellence framework. All participants remarked that the proposed framework is very suitable for assessing and evaluating the current practice of performance improvement. Evaluators agreed that the developed framework is robust, rational, systematic, and practical. They considered it a new way in performance improvement approaches and appreciated that the new process improvement methodology could be used for all improvement projects. Moreover, they agreed on some strengths, including the framework giving a complete overview of the organization and addressing the overall management system. It is also a powerful tool to improve process performance and other management system elements, with clear relevance to operational processes and business aspects. It can be helpful in all types of improvement from small events to breakthrough improvement and transformation program, consequential in diagnosing of problems (root causes analysis) and developing of robust and innovative solutions before implementation, consider as decision support and decision-making tool, inspire intensive engagement of people and innovative ideas, and simple for training and implementation with low investments in training and piloting. Based on the above discussion, it is evident that implementing the proposed framework is needed to achieve Operational Excellence by the organizations in the Sudanese aviation industry.