Sign In

Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.

Forgot Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

You must login to ask question.

Please briefly explain why you feel this question should be reported.

Please briefly explain why you feel this answer should be reported.

Please briefly explain why you feel this user should be reported.

COM 560 UTD Computer Science Paper

Evolutionary Improvements of Quality Attributes: Performance
in Practice
Neil Ernst
Software Architecture Agile Quality Attributes
Continuous delivery practices, popularized in Jez Humble’s 2010
book Continuous Delivery, enable rapid and reliable software system
deployment by emphasizing the need for automated testing and building,
as well as closer cooperation between developers and delivery teams. As
part of the Carnegie Mellon University Software Engineering Institute’s (SEI) focus
on Agile software development, we have been researching ways to incorporate quality
attributes into the short iterations common to Agile development.
We know from existing SEI work on Attribute-Driven Design, Quality Attribute Workshops, and
the Architecture Tradeoff Analysis Method that a focus on quality attributes prevents
costly rework. Such a long-term perspective, however, can be hard to maintain in a high-tempo,
Agile delivery model, which is why the SEI continues to recommend an architecture-centric
engineering approach, regardless of the software methodology chosen. As part of our work in
value-driven incremental delivery, we conducted exploratory interviews with teams in these
high-tempo environments to characterize how they managed architectural quality attribute
requirements (QARs). These requirements–such as performance, security, and availability–have
a profound impact on system architecture and design, yet are often hard to divide, or slice, into
the iteration-sized user stories common to iterative and incremental development. This difficulty
typically exists because some attributes, such as performance, touch multiple parts of the system.
This blog post summarizes the results of our research on slicing (refining) performance in two
production software systems. We also examined the ratcheting(periodic increase of a specific
response measure) of scenario components to allocate QAR work.
Slicing and Allocation in Agile Projects
QARs are often provided in unstructured and unclear ways. QAR refinement is the process of
elaborating and decomposing a QAR into a specifiable, unambiguous form achievable in one
release and finding suitable design fragments. This process, also known as slicing or sizing, is
typically applied to user stories, short, textual requirements popularized by agile methodologies.
The requirements community calls such iterative refinement the Twin Peaks model because you
iteratively cross between the ‘peaks’ of requirements (the problem domain) and architecture (the
solution domain).
QAR refinement should produce a unit of work small enough to test, small enough to fit in an
iteration, and useful enough to produce value. It should separate abstract requirements into
constituent parts so they and their interrelationship can be studied. “[QAR refinement] is a design
activity,” a software developer recently posted on Twitter. “It surfaces our ignorance of the
problem domain.”
There are a number of ways to size requirements. Some methods are based purely in the problem
space, some are based on the work involved in satisfying the requirement, and some are a
mixture of analyzing the problem and possible solutions. These approaches include

And/Or decomposition: Split on conjunctions and and or in the requirement. For
example, for the requirement “display user name and account balance,” one piece would
deliver the user name, the other piece the account balance.
Acceptance or test criteria: Satisfy one criterion per slice. User stories (an Agile form of
requirement with user-focused textual descriptions) often have a list of reasons for
accepting the requirement (story) as done. For instance, “login function works, password
validated.” Each criterion is a slice.
Spike: Add exploratory spikes to backlog where steps are unknown–investigate versus
implement. Spikes are specific work dedicated to learning, such as the impact of a new
version of a key framework.
Horizontal slices: Slice according to architectural layers (such as database, UI, business
logic). Commonly seen as an anti-pattern since it creates false divisions of the work.
Operation type: Slice according to database operations. One story for create operations,
one for read, etc.
Hamburger slicing: Create horizontal slices that map steps in a use case, then extend
vertically according to improved quality criteria. For example, a user login use case has
steps for validating passwords, displaying user history, etc. These are then expanded
‘vertically’ for different options, such as login options OAuth, social media account, or
simple user:password.
In the two examples described below, we found the most common slicing approach was
ratcheting, popularized by Martin Fowler in his blog post “Threshold Testing.”
Our Approach
Our research question was
How do projects slice quality attribute requirements into iteration-sized pieces, and how are
these pieces allocated to a project plan or sprint backlog?
QARs, including performance, security, and modifiability, significantly influence architecture.
We conducted case studies of production systems (we omit identifying details to protect
confidentiality). For each case, we conducted interviews with team leads and architects to
understand the teams’ deployability goals and architectural decisions they made to enable
deployability. We then performed semi-structured coding to elicit common aspects across the
Based on our analysis, we described the quality attribute work as a series of increments,
representing the current state of the QAR, as the requirement is progressively refined and
enhanced. Table 1 below shows our conceptual framework. We used a scenario to define each
state, with each scenario consisting of a stimulus, some context or background, and a response to
that stimulus. Our approach is analogous to an Agile, “Behavior-Driven Development” (BDD)-
style”Given , When , Then ” model. The value column reports the outcome of that
increment. Based on our interviews, refinement happens not only in the stimulus response, but
also the context and the stimulus itself. In other words, one can tighten the response (from 10 to
1 ms, for example); one can loosen the context (for all customers instead of just certain
customers); or one can expand the range of stimuli (any order versus special order types). We
believe refinement of QARs using scenarios creates a reasonably sized chunk of work for
analysis, implementation, and possibly even testing during an iteration.
Project A
Project A developed financial support software for a mid-size firm. The software supported the
buying and selling of financial securities. Performance was a key concern, particularly at the
close of the financial day. Customers may have to pay interest if they have to borrow large sums
of money to hold sell orders overnight if they were not processed by the close of trading.
Development cycles were deliver-on-demand, rather than fixed iterations, while customers
provided feedback to the developers that informed the work in the subsequent state. Table 1
shows a summary of the state transitions. Below, we describe states that we identified in the first
These states were refined in three different dimensions (stimulus, context, response).

Stimulus: A-S4 to A-S5 illustrates refinement in the stimulus dimension, moving from a
single- to multi-user perspective. A second example is A-S2 to A-S3, which illustrates
moving from batch processing to individual transaction processing.
Context: The initial requirement focused development on the system behavior (simple
baseline case) to test ideas before dealing with complexities and uncertainties of the
environment. A-S1 to A-S2 illustrates refinement in the context dimension, moving from
manual processing to the autopilot solution, by adjusting the boundary of the system with
respect to its environment, including the user’s role. Moving from A-S4 to A-S5 accounts
for the complexities of the rotary algorithm in the environment.
Response: A-S3 to A-S4 illustrates refinement in the response dimension by ratcheting
the response measure. The response measure value was refined to less than 1 second for
processing individual transactions.
This analysis demonstrates that each refinement we captured reflects a change in the evolving
requirements context, such as increasing number of orders to be processed or reducing
performance threshold for a single order. The team also must make tradeoffs in the measurement
criteria, for example, first optimizing for throughput (A-S2) before dealing with latency (A-S3,
A-S5). With three dimensions to adjust (stimulus, context, response), ratcheting in one
dimension may require easing up on another to make progress.
Project A explained that separating performance-related from feature-related requirements in the
backlog or when planning and allocating work to iterations was not useful. Project A followed
evolutionary incremental development by doing performance-related analysis work concurrently
and loosely coupled from implementation sprint work. Separating feature work from
performance work is a common pattern that we observe in industry. As analysis work was
completed, the work was allocated to sprints. Well-understood changes refining existing features,
such as A-S1 through A-S3, were allocated to implementation sprints with minimal analysis.
However, in cases where significant analysis was needed (e.g., A-S4), the team created an
exploratory prototype to learn more about the problem and investigate alternative solutions while
continuing to mature the system and implement ongoing requirements. For A-S4 the changes
were more substantial, so the work was allocated to multiple sprints.
Project B
Project B, described in Table II, was also a financial system with stringent performance
requirements. The system functionality included high-speed, stock order processing. Like Project
A, the performance iterations are described as state transitions. This project was in the prerelease phase, focusing on evaluating the design to ensure it could meet requirements. The team
used a scenario-driven approach to investigate performance risks. Output of scenario analysis is
input to the evolving system design and architecture of the next state, as described below.
The analysis of this example also shows changes to the stimulus, context, and response, as
summarized below.

Stimulus: BS-3 shows variations in the stimulus with the artifact changing to focus on a
specific part of the system, the queue.
Context: B-S2 shows variations in the context by increasing the concurrently processed
orders to 1,000 orders while maintaining response time at 0.1 ms.
Response: The response goal remained consistent at 0.1 ms throughout all the state
transitions. In interviews, however, we discovered that this target was reached over
several rounds of tweaking the design, analyzing the response under increasingly
stringent conditions, such as described in B-S2, and making incremental improvements.
Project B did not separate feature-related and performance-related requirements in their software
development lifecycle either. Like Project A, the team conducted exploratory performance
analysis and design work concurrently with maturing other features and continuing
implementation. Work was integrated into implementation sprints as it became better defined
through analysis of design artifacts, prototyping, or both.
Ultimately, the purpose of refining QARs is to decompose a stakeholder need or business goal
into iteration-sized pieces. The process of allocation then takes those pieces and determines
when to work on them. This activity involves both analysis and design: refinement and allocation
are explorations of the problem and solution spaces, and evolutionary, iterative development
allows for course changes when new information is acquired. Developers work toward satisfying
cross-cutting concerns in the context of the effort and ultimate value.
We observed that developers refined performance requirements using a feedback-driven
approach, which allowed them to parse the evolving performance requirement to meet increasing
user expectations over time (expressed as state transitions). Within each state transition,
developers refined cross-cutting concerns into requirements by breaking them into their
constituent parts in terms of the scope of the system and response to stimuli in a given context.
The system and cross-cutting performance requirements evolve as stimuli, context, and response
are ratcheted.
We see evidence of projects that are better able to sustain their development cadence with a
combination of refinement and allocation techniques guided by measures for requirement
satisfaction, value, and development effort. As we retrospectively analyzed these examples, we
found that these teams did not follow a formal technique; however, they did have common
elements in how they refined the work into smaller chunks, enabling incremental requirements
analysis and allocation of work into implementation increments.
Martin Fowler describes ratcheting as a performance requirement in terms of increasing a
specific response measure. Based on what we learned from our interviews, we suggest that this
ratcheting concept can be broadened. For example, changes in the evolving context, such as
increasing the number of orders to be processed or reducing the performance threshold for a
single order, allow for breaking a cross-cutting concern to a reasonably sized chunk of work for
analysis, allocation, or testing. We suggest that these examples, which demonstrate ratcheting in
multiple dimensions, could be useful for teams struggling with how to break up and evolve
cross-cutting concerns during iterative and incremental development.
Additional Resources
This blog post is a summarization of a paper that was presented at the International Conference
on Software Maintenance and Evolution.
User Features
Submitted to
Instructor Name
Summer, 2018
Group Members
August, 2nd. 2010
1 of 10
User features for home EMC system
6.1 Potential user features
F1: The home EMC system might include lighting control.
F2: The home EMC might include an alarm system.
F3: The home EMC might include IP surveillance camera. .
F4: The home EMC might include an interface to link and control all systems and
electronic devices and lighting together.
F5: The home EMC might include a feature to turn music on or off anytime and
anywhere within the house.
F6: The home EMC might include a climate control system. .
F7: The home EMC might include a feature to turn the home theater (including TV,
satellite, projector, DVD changer, etc…)On with one-touch remote controller.
F8: The home EMC might include a feature to alert home owner when garage door is
left open.
F9: The home EMC system might include a feature to allow home owner to turn off TVs
in all rooms when it’s time for kids to do their home work.
F10: The home EMC system might include motion sensors.
F11: The home EMC system might include an interface to allow home owners to control
their homes from their Smart Phone or I- phone.
F12: The home EMC system might include a feature to control home easily and
F13: The home EMC system might include a flood sensor.
F14: The home EMC system might include automatic alarms.
F15: The home EMC system might include a feature to monitor home appliances
remotely from virtually anywhere.
F16: The home EMC system might include a feature to automate window shades to be
opened or closed based to sunlight.
F17: The home EMC system might include a feature to allow home owners to adjust
lights in the kitchen with a one-touch button.
F18: The home EMC system might include a feature to allow motion sensors to turn
lights off in unoccupied rooms.
F19: The home EMC system might include a water control system.
F20: The home EMC system might include a feature to raise bedroom temperature right
before wakeup time.
2 of 10
6.2 Prioritized user features for home EMC system
6.2.1 Essential User Features*
EF1: The home EMC system shall provide surveillance cameras to alert and notify home
owner when someone breaks into the house while at work or on vacation.
User classes that would use this feature: home owners
EF2: The home EMC system shall include an alarm system to notify home owner when
doors are left open.
User classes that would use this feature: home owners.
EF3: The home EMC system shall allow users to manage the use of energy and keep
house temperature to the preferred level.
User classes that would use this feature: home owners and guests.
EF4: The home EMC system shall include automatic timers to notify home owners when
locks are accessed.
User classes that would use this feature: home owners, guests, and systems technicians.
EF5: The home EMC system shall permit motion sensors to turn lights off in unoccupied
rooms to save energy.
User classes that would use this feature: home owners & thieves.
EF6: The home EMC system shall provide a glass break detector to notify home owners
when the there is a theft attempt.
User classes that would use this feature: home owners, system technicians, and thieves.
3 of 10
6.2.2 Desirable User Features*
DF1: The home EMC system should permit home owners to use outdoor security lights to
provide exterior home security.
User classes that would use this feature: home owners.
DF2: The home EMC system should enable home owners to turn living room into theatre
by providing a touch screen TV that combines audio and video entertainment
User classes that would use this feature: home owners.
DF3: The home EMC system should permit system technicians to make the lights turn on
when the security alarm goes off.
User classes that would use this feature: system technicians.
DF4: The home EMC system should allow home owners to control their home from onScreen via TV.
User classes that would use this feature: kids under18 years.
DF5: The home EMC system should provide a fire alarm system to go off when there is
fire at home.
User classes that would use this feature: system technicians, guests, and home owners
DF6: The home EMC system should allow the detection of water or flood inside the
User classes that would use this feature: guests.
4 of 10
6.2.3 Optional User Features*
OF1: The home EMC system could provide a garage door control to monitor and control
garage remotely.
User classes that would use this feature: home owners.
OF2: The home EMC system could allow home owners to use freeze alarms to prevent
potential damage to water pipelines.
User classes that would use this feature: home owners.
OF3: The home EMC system could permit home owners to keep lights on in a timed
User classes that would use this feature: kids under 18 years.
OF4: The home EMC system could provide a temperature sensor to notify home owners
when temperature reaches an extreme threatening level.
OF5: The home EMC system could provide driveway sensor alarm to protect driveway.
OF6: The home EMC system could include video capability to allow home owners to see
who is at the door without opening the door.
5 of 10
6.3 Prioritization of Desirable Features
Prioritization Matrix
home EMC system
prepared by:
Marwan Omar
July, 31.2010
DF1- The home
EMC system
should permit
technicians to
make the lights
turn on when
the security
alarm goes off.
DF2: The home
EMC system
should include
timers to
DF3-The home
EMC system
should provide
a fire alarm
system to go
off when there
is fire at home
DF4-The home
EMC system
should allow
home owners
to control their
home from onScreen via TV
6 of 10
DF5: The home
EMC system
should allow
the detection
of water or
flood inside the
DF6: The home
EMC system
should permit
home owners
to use outdoor
security lights
to provide
exterior home
7 of 10
6.4 Traceability of User Features to User Classes
8 of 10
6.5 Quality Attributes for home EMC system
Attribute Brief Description
The home EMC system shall provide timely response- services within 2-5
minutes from any incident’s occurrence to home owners to make using the
system convenient, easy, and enjoyable.
The home EMC shall provide safety tools and technologies (e.g. home
invasion detector and burglary alarm) to make home safe from intruders and
The home EMC system shall prevent theft and protect house and family by
offering security cameras, sound detectors, and motion detectors.
The home EMC system shall only have about 5-10 of scheduled down time
on a weekly basis. Any technical issues with the system or unexpected
maintenance requirements will be shared with stake holders such as home
owners and security service.
The home EMC system shall be accessible remotely via cell phone or laptop,
or I-phone.
RE: response time
SA: system safety
SE: system security
AV: system availability
AC: system accessibility
9 of 10
10 of 10
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Software Requirements1
Requirements are descriptions of the services that a software system must provide and the constraints under which it must operate
Requirements can range from high-level abstract statements of services or system constraints to detailed mathematical functional specifications
Requirements Engineering is the process of establishing the services that the
customer requires from the system and the constraints under which it is to be
developed and operated
Requirements may serve a dual function:
As the basis of a bid for a contract
As the basis for the contract itself
Requirements Documents
“If a company wishes to let a contract for a large software development project
it must define its needs in a sufficiently abstract way that a solution is not predefined. The requirements must be written so that several contractors can bid
for the contract, offering, perhaps, different ways of meeting the client organisation’s needs. Once a contract has been awarded, the contractor must write a
system definition for the client in more detail so that the client understands and
can validate what the software will do. Both of these documents may be called
the requirements document for the system.”
[A.M. Davis “Software Requirements: Objects, Functions and States”, Englewood
Cliffs, 1993]
2.1 Types of Requirement
User requirements
Statements in natural language plus diagrams of the services that the systems provides and its operational contraints.
Written for customers
System requirements
A structured document setting out detailed descriptions of the system services.
These notes are based on those provided by Ian Sommerville on his “Software Engineering”
text website
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Written as a contract between client and contractor
Software specification
A detailed software description which can serve as a basis for a design or
Written for developers
2.1.1 Example Definition and specifications
User Requirements definition:
The software must provide a means of representing and accessing external files
created by other tools
System Requirements specification:
1. The user should be provided with facilities to define the type of external files
2. Each external file type may have an associated tool which may be applied
to the file
3. Each external file type may be represented as a specific icon on the user’s
4. Facilities should be provided for the icon representing an external file to be
defined by the user
5. When a user selects an icon representing an external file, the effect of that
selection is to apply the tool associated with the type of external file to the
file represented by the selected icon
2.2 Functional, non-functional and domain requirements
Functional requirements
Statements of services that the system should provide, how the system
should react to particular inputs and how the system should behave in
particular situations
Non-functional requirements
Constraints on the services or functions offered by the system such as timing constraints, constraints on the development process, standards, etc.
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Domain requirements
Requirements that come from the application domain of the system that
reflect the characteristics of that domain
May be functional or non-functional
2.2.1 Functional requirements
Describe functionality or system services
Depend on the type of software, expected users and the type of system
where the software is used
Functional user requirements may be high-level statements of what the system should do; functional system requirements should describe the system
services in detail
The user shall be able to search either all of the initial set of databases or
select a subset from it
The system shall provide appropriate viewers for the user to read documents
in the document store
Every order shall be allocated a unique identifier (ORDER ID) which the
user shall be able to copy to the account’s permanent storage area
2.2.2 Non-functional requirements
Product requirements
– Requirements which specify that the delivered product must behave in
a particular way, e.g. execution speed, reliability etc.
Organisational requirements
– Requirements which are a consequence of organisational policies and
procedures, e.g. process standards used, implementation requirements
External requirements
– Requirements which arise from factors which are external to the system and its development process, e.g. interoperability requirements,
legislative requirements etc.
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Figure 2.1: Non-functional Requirements
Metrics for non-functional requirements
See figure ??.
2.2.3 Domain requirements
Describe system characteristics and features that reflect the domain
May be new functional requirements, constraints on existing requirements
or may define specific computations
If domain requirements are not satisfied, the system may be unworkble
Example: Library system
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Processed transactions/s
User/Event response time
Screen refresh time
Number of RAM chips
Ease of Use Training time
Number of help frames
Mean time to failure
Probability of unavailability
Rate of failure occurrence
Robustness % events causing failure
Time to restart after failure
Probability of data corruption
on failure
% target system dependent
Number of target systems
Figure 2.2: Metrics for non-functional requirements
Because of copyright restrictions, some received on-line documents must
be deleted immediately after printing
Example: Train protection system
Train deceleration shall be computed as

is 9.81 ms! * compensated gradient/alpha and where the
values of 9.81 ms ! /alpha are known for different types of train
Domain requirement issues
Requirements are expresed in the language of the application domain
– this is often not understood by software engineers developing the system
CS2 Software Engineering note 2
CS2Ah Autumn 2004
Domain specialists understand the area so well that they do not think of
making the domain requirements explicit
2.3 User requirements
Should describe functional and non-functional requirements so that they
are understandable by system users who don’t have detailed technical knowledge
User requirements are defined using natural language, tables and diagrams
2.3.1 Problems with natural language
– Readers and writers may not interpret words in the same way
– The same thing may be expressed in a number of different ways
Requirements amalgamation & confusion
– Several different requirements may be expressed together; functional
and non-functional requirements may be mixed together
Lack of modularisation
– NL structures are inadequate to structure system requirements
The requirements for a CASE tool for editing software design models include the
requirement for a grid to be displayed in the design window
“To assist in the positioning of entities on a diagram, the user may turn on a grid
in either centimetres or inches, via an option on the control panel”
This statement mixes up three different kinds of requirement
A conceptual functional requirement stating that the editing system should
provide a grid; it presents a rationale for this
A non-functional requirement giving information about the grid units
A non-functional user interface requirement defining how the grid is switched
on and off by the user
CS2 Software Engineering note 2
CS2Ah Autumn 2004
2.3.2 Revised requirement
“The editor shall provide a grid facility where a matrix of horizontal and
vertical lines provides a background to the editor window. This grid shall
be a passive grid where the alignment of entities is the user’s responsibility.
Rationale: A grid helps the user to create a tidy diagram with well-spaced entities.
Although an active grid, where entities ‘snap’ to grid lines can be useful, the
positioning is imprecise; the user is the best person to decide where entities
should be positioned.”
Concentrates on the essential system feature
Presents a rationale both its inclusion and for rejection of an automatic
alignment feature
Separate statements are needed to cover the other parts of the original requirement, e.g.
The grid can be turned on or off via an option in the control panel
The grid can be in centimetres or inches
The grid shall initially be in inches
2.3.3 Alternatives to NL specification
Structured natural language
– depends on defining standard forms or templates to express requirements specification
Design description languages
– similar to programming languages but with additional, more abstract
Graphical notations
– a graphical language, supplemented by text annotations, is used to
define functional requirements (e.g. use-case diagrams)
Mathematical/formal specifications
– based on mathematical concepts such as finite-state machines or sets;
unambiguous specifications reduce arguments between customers and
contractors but most customers don’t understand formal specifications
CS2 Software Engineering note 2
CS2Ah Autumn 2004
2.4 The Requirements Document
Official statement of what is required of the system developers
Should include both a definition and a specification of requirements
– specify external system behaviour
– specify implementation constraints
– be easy to change
(but changes must be managed)
– serve as a reference tool for maintenance
– record forethought about the life cycle of the system (i.e. predict changes)
– characterise responses to unexpected events
It is not a design document
– it should state what the system should do rather than how it should
do it
2.4.1 Requirements document structure
User requirements definition
System architecture
System requirements specification
System models
System evolution
CS2 Software Engineering note 2
CS2Ah Autumn 2004
2.4.2 Requirements document users
System customers
Specify the requirements and read them back to check that they meet their
needs; specify changes to the requirements
Development Managers
Use the requirements document to plan a bid for the system and to plan the
system development process
Implementation Programmers
Use the requirements to understand what system is to be developed
Test programmers
Use the requirements to develop validation tests for the system
Maintenance programmers
Use the requirements to help understand the system and the relationships between its parts
2.5 Summary
Requirements set out what the system should do and define constraints on
its operation and implementaion
Functional requirements set out services that the system should provide
Non-functional requirements constrain the system being developed or the
development process
User requirements are high-level statements of what the system should do
User requirements should be written using natural language, tables and
System requirements are intended to communicate the functions that the
system should provide
System requirements may be written in structured natural language, a PDL
or in a formal language
A software requirements document is an agreed statement of the system

Purchase answer to see full


Leave a comment