Enterprise risk evaluation and continuous mitigation using the fuzzy-multiattribute decision making'A conceptual

Software development processes generally follow easily identifiable stages and become increasingly more challenging.

They differ from traditional manufacturing project stages in various ways, which make them very risky

and uncertain. Traditional software risk management practices are more focused on qualitative judgment and experiences;

however, with exponential growth in number and size, software companies require effective scientific

methodology for risk management. The main objective of this study is to increase the effectiveness of risk detection

and mitigation practices in the software development process. Another objective is to analyze these practices,

identify the areas for improvement, and develop a mechanism for their quantification. We also aim to identify the

processes, which cause problems and suggest strategy to eliminate or reduce the harmful effect of these processes

in minimum possible cost and time. Analytical hierarchy process and fuzzy set theory are suggested as effective

tools to achieve this objective.

Software development processes generally follow easily identifiable stages like project

planning, requirement definition, design, development, testing, integration, installation, acceptance

and support. However, they differ from traditional manufacturing project stages in

various ways. Firstly, we need to give time and cost estimates to the customers in advance

without actually knowing its exact nature, which make software projects very risky and uncertain.

Secondly, to judge the requirements in diverse domain areas with almost the same

set of manpower requires a lot of flexibility on the part of employees and management.

Then fast rate of changes in technologies, customer requirements, possibility of unexpected

number of employees leaving the organization make the task even more complicated. Integrating

the complete solution, implementing it in successful ways, and making the end-user

understand the software and changes in the traditional processes are all part of the software

development life cycle. More and more global companies are outsourcing their software

development projects to offshore software development destinations like India which are

626 NIRAJ KUMAR AND K. R. SRIVATHSAN

throwing new challenges in terms of requirements specifications, cultural differences, effective

communication, security of valuable information, more rigorous legal, governance and

compliance standards.

Because of challenges posed by software development risks many approaches have been

proposed to improve the development process. These approaches include various computeraided

software engineering (CASE) tools, enterprise project management tools, conceptual

tools like various quality, coding, process standards, models, and frameworks, and various

software engineering models like traditional waterfall and spiral model.

In some cases, modeling and design tools are widely used and include Rational Rose [1,

2], versioning control tools like CVS, various testing and bug-tracking tools like Bugzilla,

integrated development environment and automatic code generation tools like Visual Studio,

Jcreator, Dreamweaver and others. Requirements development and management have

always been critical in the implementation of software systems. Some automated tools are

also available to support requirements management. The use of these tools not only provides

support in the definition and tracing of requirements, but also opens the door to effective

use of metrics in characterizing and assessing testing. Metrics are important because of

the benefits associated with early detection and correction of problems with requirements

[3].

Enterprise project management tools are found to provide a wide range of functions.

Among these are scheduling, resource allocation, and cost estimating, budgeting, and collaborating.

It is important to note that these tools emphasize project performance relative to

resource consumption within a given set of time constraints (i.e. progress and dates). Some

of the popular enterprise project management tools include Welcome product suite, Microsoft

project 2002, Primavera P3e suite. While interest and investment in CASE and project

management tools are rising steadily, actual experiences with tools have exhibited more

ambiguity. There has been no systematic examination or formulation of the organizational

changes surrounding CASE tools [4]. The major challenge is to develop quality software in

a reliable and repeatable manner while improving productivity [5].

On the conceptual front all these challenges gradually led to development of capability

maturity model (CMM) and concept of 'Balanced Scorecard'. The concept of 'Balanced

Scorecard' [6] developed in the early 1990s by Kaplan and Norton represented an advance

in the field of measuring enterprise performance, providing a framework for companies to

evaluate both financial and 'non-financial', or 'extra-financial' measures such as quality,

customer and employee satisfaction. This framework was widely used for enterprise-level

planning, control and monitoring by software companies. However, it suffers from limitations

of over reliance on expert judgments for decision making.

CMM [7] was developed in the early 1990s by Carnegie Mellon's Software Engineering

Institute (SEI). Since then many versions of this model appeared and are widely accepted as

most rigorous and best standard by software industry throughout the world. What led to instant

success of this model is its ability to focus not only on external processes but also

provide a framework for continuous improvements of internal processes in the company.

ENTERPRISE RISK EVALUATION AND CONTINUOUS MITIGATION 627

CMM Level 2 KPAs CMM Level 3 KPAs CMM Level 4 KPAs CMM Level 5 KPAs

Requirements management Requirement development,

Technical solution,

product integration

Quantitative process

management

Defect prevention

Software project planning Verification, validation Software quality

management

Technology change

management

Software project monitoring

and control

Organization process

definition and focus

X Process change

management

Supplier agreement

management

Integrated software

management

X Continous improvement

and optimization

Measurement and analysis Organizational training,

integrated project

management

X X

Software configuration

management

Risk management,

integrated teaming

X X

Process and product quality

assurance

Integrated supplier

management, decision

analysis and resolution,

integrated supplier management,

integrated teaming

X X

The CMM establishes an yardstick against which it is possible to judge, in a repeatable

way, the maturity of an organization’s software process and compare it to the state of the

practice of the industry. The CMM can also be used by an organization to plan improvements

to its software development processes.

However, implementing the CMM framework is a very challenging problem for three

fundamental reasons. First is the issue of recognizing all input/output parameters in real

time, which influence various key process areas. Second, methodologies for quantification,

optimization, and continuous improvement of these processes are vague. Third is the issue

of synchronization of these processes with overall organizational objectives, profitability,

efficiency, and growth.

Our primary focus in this study is to develop a model to analyze risk management (CMM

level 3), quality management (CMM level 4), quantification of processes (CMM level 4),

and defect prevention, continuous improvement and optimization (CMM level 5). Then

based on analysis we also suggest improvement into various processes, which should be

cost effective and suited for particular organization culture.

Software development process analysis is an important first step towards making the

process more efficient and profitable. However, analysis is difficult not only due to its

complex nature because of large number of subsystems involved and dynamic interaction

and influence of one over another, but also because it is difficult to quantify their contribution

and influence on the software development process as a whole. For example, technical

ing any software company. However, technical expertise in itself can play an important role

in the quality of software delivered, at the same time it may also lead to cost and time overrun,

which may have negative consequences for the project. Similarly, the quality of software

also depends upon the processes adopted for quality control and on the amount of time

and funds available for the project. Also, the resources of an enterprise are limited so not all

sources of risk can be immediately eliminated and priorities need to be established. Studying

the various processes of a software development life cycle and measuring their impact

in quantitative terms is one of the important objectives of this study. Again identification of

key sources of risk and the ability to measure the level of its harmfulness on the system as a

whole is another important objective of this study. In this paper, various risk sources for a

software enterprise have been identified and a new approach based on analytical hierarchy

process [8] and fuzzy set theory [9'11] has been suggested as effective tool for enterprise

risk evaluation and continuous mitigation.

Many researchers have tackled problems related to software development processes, practices

and tools, multicriteria and multiattribute decision-making, risk management and

fuzzy set theory.

Wiegers [12] points out unstated expectations as major source of software project failure

which can lead to erroneous assumptions, unfulfilled dependencies, unexpected risks and

disappointed customers. He emphasizes the importance of documenting the requirements

thoroughly, precisely and without ambiguity. Mall [13] gives an overview of software engineering

practices and covers almost every aspect of software engineering in brief.

Ming and Smidts [14] deal with ranking of software engineering measures based on expert

opinion. Over reliance on expert opinion is a major limitation of this study. These theories,

methodologies and tools need to be subjected to rigorous test of practices to live up to

their perceived expectations and promises.

Many methodologies are available [9, 10] in the literature for multicriteria and multiattribute

decision-making including data envelopment analysis [15], analytical hierarchy

process [16], multiattribute utility theory [17], and Bayesian analysis and outranking methods

[19]. The AHP, designed to solve complex problems of multiple criteria involving both

qualitative and quantitative parameters, was proposed by Saaty in early 1970s. The application

of AHP is based on four basic principles, namely, decomposition, prioritization, synthesis

[22] have demonstrated the effectiveness of AHP to solve real-life industrial problems.

Kumar and colleagues [23'25] have tackled the problem of enterprise-level planning and

control with multiple criteria which include parameters like capacity, productivity, profitability,

environment and safety involving a number of decision-making units (DMUs).

The acceptance of term risk is universal and it penetrates every discipline of the society.

Each discipline visualizes risk in its own understanding. The concept of risk management

varies from macro to micro level. Several authors have highlighted the objectives of establishing

risk assessment and mitigation process [26, 27]. However, due to the dynamic nature

of interaction between various risk sources understanding of their relationship is

ENTERPRISE RISK EVALUATION AND CONTINUOUS MITIGATION 629

imprecise which can be equated with fuzziness. Further, they require far more detailed description

than is usually available for analysis. Application of the fuzzy set theory is an effective

tool to handle these kinds of real-life situations.

The theory of fuzzy set proposed by Zadeh [11] provides a strict mathematical framework

in which vague conceptual phenomena can be precisely and rigorously studied. It has

led to a paradigm shift in the way problems are solved in various disciplines. It can also be

considered as a modeling language well suited for situations in which fuzzy relations, criteria

and phenomena exist.

Most software development projects fail to deliver acceptable systems on time and within

budget. Many studies were conducted to study software project failure rate at global level

including the Conference Board Survey 2001. Their key findings suggest that 40% of software

projects failed to achieve their business case within one year of going live and project

costs were found to be on average 25% over budget from the original estimates. Traditionally,

user requirements, inadequate user documentation, excessive schedule pressure,

low quality, low user satisfaction, cost overruns were considered some of the major risk

sources for software projects. However, increasing concern over security, legal problems,

rising cost of software development, and HR-related problems necessitates that these factors

should also be taken into account in risk management [4, 7, 18].

Much of the failure could be avoided by managers proactively planning for dealing with

risk factors rather than waiting for problems to occur and then trying to react. Usually, this

reaction is too little and too late, because by the time the problem is fully recognized, the

schedule has already slipped, a considerable investment has been made, and the product

quality has suffered due to introduction of errors or workaround. Risk management is an

important tool to provide insight into potential problem areas and to identify, address, and

eliminate them before they derail the project. Software risk management is important because

it helps avoid disasters, rework, and overkill, but more importantly because it stimulates

win'win situations [3]. A typical risk assessment and mitigation system may follow

the following cycle (Fig. 1):

Typical internal and external factors causing risk in the software industry include the following:

630 NIRAJ KUMAR AND K. R. SRIVATHSAN

Sources of external and

internal risk in an enterprise

Risk identification Risk analysis and

prioritization Risk mitigation

New risk sources

Mitigated and

redundant risk

sources

ENTERPRISE RISK EVALUATION AND CONTINUOUS MITIGATION 631

exchange fluctuations, risk from increasing software development cost, risk from litigation

expense.

All the factors listed above are important sources of risk in the software industry. It is

also important to understand that some of these sources are critical and their impact is profound

across various processes of software development. Proper evaluation of all these factors

and identification of key risk sources and subsequently their reduction or elimination

with minimum cost and resources is necessary to make system more efficient and profitable

and to gain competitive advantage in the market.

Methodology to be adopted for this study is four fold?first to identify the major risk factors,

then able to quantify most of them, prioritize the factors based on their potential for

causing risk to the organization and finally developing a general-purpose risk management

software. These four stages can be summarized as

The constraints of time and cost make it impossible to eliminate all forms of risk in any enterprise,

so an effective way to eliminate key sources of risk and continuous improvement

of risk control process is required. Identifying and quantifying the key sources of risk that

should be eliminated is an important first step towards achieving this. Figure 2 displays

various potential risk sources identified and classified for a typical software enterprise.

Then harmfulness of each type of risk on the system as a whole is proposed to be determined.

Some factors like risk from project rejection by customers, risk due to theft or loss of

valuable information, risk from high cost of security management, risk from excessive inventory

level, financial risk, risk from currency fluctuations, risk from increasing software

development cost are relatively easy to quantify and estimate in monetary terms and their

harmfulness on the system as a whole can be determined. For example, consider risk due to

security-related problems. The cost of security-related problems on a software enterprise

ent of the software developed like cost of maintaining firewalls, gateways, etc. These costs

can result in increased capital and operating costs for software enterprise. Variable costs are

those that vary directly with the software developed. Methodology that examines the cost

impact of security-related problems can be based on crude estimates of the order of magnitude

of fixed and variable costs. For this, reliability analysis of failure due to securityrelated

problems is done and hazard function is determined.

Then cost model of security-related problems on software system can be given as:

= × + ×

ENTERPRISE RISK EVALUATION AND CONTINUOUS MITIGATION 633

where,

After evaluation of software development process and identification of the various forms of

risk and its degree of harmfulness, fuzzy clustering is used to cluster more harmful forms of

risk on the basis of their fuzzy correlation and categorize the various risk sources into key

634 NIRAJ KUMAR AND K. R. SRIVATHSAN

risk sources (Fig. 3). Expert grading method is used to estimate the correlation between objects.

The correlation between two risk sources is (0, 1). The bigger the number, the

stronger the correlation is. After generating tolerance matrix and transforming it into fuzzy

equivalence matrix, more harmful sources of risk that are strongly correlated can be clustered

into a single risk source. This stage is likely to bring down the number of unmanageable

risk sources into manageable few.

The priorities by which key forms of risk should be eliminated are ranked based on the following

important principles:

User-friendly computer software can be developed, which can quantify, cluster and rank the

various sources of risk by giving suitable input. Accordingly, management will be able to

generate useful information for elimination of risk sources and their effect on the software

development process.

ENTERPRISE RISK EVALUATION AND CONTINUOUS MITIGATION 635

Choosing the

most harmful forms

of risk sources

Identifying and classifying all forms of risk in software process

Evaluating the harmfulness of each type of risk

Clustering to identify the key sources of risk (Fuzzy clustering)

Dynamic

recycling for

total risk

Determining the primary elimination measure for each key risk source elimination

Ranking the key risk sources (Fuzzy comprehensive evaluation)

Identifying the eliminated object'the most harmful key risk source

Making systematic plan to eliminate this key risk source

Yes

No

Validation and implementation of the model to the scale, which is required for this kind of

research, is yet to be fully realized. However, preliminary attempt of its validation based on

information collected from a middle-level consultant of a fast-growing CMM-level 5 company

has been done. This company is situated at Technopark, Thiruvananthapuram, and it

relies on audits, reviews and testing for risk and quality management. However, no specific

information regarding quantification of quality and customer satisfaction level has been

provided.

Based on information, which is related with breakdown structure for software enterprise

in Fig. 2, data was collected in April 2005, which identifies two most important risk sources

636 NIRAJ KUMAR AND K. R. SRIVATHSAN

Category Two most important risk sources

in each category (Table II). However, these results are not conclusive, as the authors did not

verify practices and processes of the company.

A web-based system for risk evaluation, quantification and multicriteria decision-making

using Java-based technologies (JSP/Servlets), Mysql database and Apache tomcat application

server were already started (see Fig. 4 for screenshot of one such interface) and plan to

make it available in public domain after its completion.

Enterprise risk management is one of the most important strategic business tools to manage

effectively a variety of risks to gain competitive advantage and add value to the firm. This

study has identified the various external and internal risk sources in a software enterprise.

Authors have proposed a scientific model based on AHP and fuzzy set theory to prioritize

various risk sources and developed a framework for their continuous elimination by adopting

enterprise-specific strategy. The paper also emphasizes the dynamic interactions between

these factors and their suggested importance, quantification to judge the impact on

software enterprise as a whole. This model is highly flexible and customizable according to

specific enterprise need. Fuzzy clustering is proposed to cluster risk factors, which are

closely correlated to each other and are likely to have common source of problems to enable

their effective mitigation. However, quantifying (preferably in monetary terms) various risk

sources and improvements required for their effective mitigation can be some of the potential

directions in which this work can be extended.

Based on preliminary attempt of model validation with a CMM level-5 company it can be

said that uncertain requirements, change in requirements, reactive approach of security,

nonconformance with specifications and ineffective utilization of available resources are

some of the major risk sources for this enterprise. Software enterprises can apply this new

approach in their project and enterprise risk management to improve efficiency, performance,

profitability and to meet rising enterprise management challenges.

The authors would like to acknowledge Prof. Ashis Bhattacherjee, Department of Mining

Engineering, Indian Institute of Technology, Kharagpur, and anonymous reviewers for their

invaluable suggestions and encouragement. Thanks are also due to Mr K. Sreenivasa Rao,

1. Object Management Group, http://www.omg.org/gettingstarted/what is uml.htm

2. Rational Software Corporation, http://www.ibm.com/developerworks/rational/library/253.html

(J. L. Cochrane and M. Eeny, eds), University of South Carolina Press, Columbia, SC, USA, pp. 179'

201 (1973).

20. C. Rong, K. Takahashi, and J. Wang, Enterprise waste evaluation using the analytic hierarchy process and