CIT420(1)1111.pptx

System procurement  Acquiring a system (or systems) to meet some identified organizational need.  Before procurement, decisions are made on:  Scope of the system  System budgets and timescales  High-level system requirements  Based on this information, decisions are made on wheth...

System procurement  Acquiring a system (or systems) to meet some identified organizational need.  Before procurement, decisions are made on:  Scope of the system  System budgets and timescales  High-level system requirements  Based on this information, decisions are made on whether to procure a system, the type of system and the potential system suppliers. Systems Engineering Overview 1 Decision drivers  The state of other organizational systems  The need to comply with external regulations  External competition  Business re-organization  Available budget Systems Engineering Overview 2 S o m Procurement and development e s y s t e m s p e c i f i c a System procurement processes Systems Engineering Overview 4 R e q Procurement issues u i r e m e n t s m a y h a T h e Contractors and sub-contractors p r o c u r e m e n t o f U s u System development a l l y f o l l o w s a p Systems development Systems Engineering Overview 8 T h r System requirements definition e e t y p e s o f r e q u P a r The system design process t i t i o n r e q u i r e m e R e q Requirements and design u i r e m e n t s e n g i n e e r Requirements and design spiral Systems Engineering Overview 12 T y p Sub-system development i c a l l y p a r a l l e l A f t System delivery and deployment e r c o m p l e t i o n , t h e System operation  Operational processes are the processes involved in using the system for its defined purpose.  For new systems, these processes may have to be designed and tested and operators trained in the use of the system.  Operational processes should be flexible to allow operators to cope with problems Systems Engineering Overview 15 L a r System evolution g e s y s t e m s h a v e Key points  System procurement covers all of the activities involved in deciding what system to buy and who should supply that system.  System development includes requirements specification, design, construction, integration and testing.  When a system is put into use, the operational processes and the system itself have to change to reflect changing business requirements.  Human errors are inevitable and systems should include barriers to detect these errors before they lead to system failure. Systems Engineering Overview 17  Differences, advantages and disadvantages between in- house development IT systems and industry standard IT system (Commercial Software)  In-House Developed Software Pros: The high level of customization; User interface will be more familiar and easy to use; Having access to knowledgeable support (Rather than dealing with technicians who may not understand your unique situation, you can get support from the individuals who have developed your software firsthand. They will understand any subtle nuances and minimize downtime from technical errors.)  In-House Developed Software Cons: May lack the knowledge and expertise to create sophisticated software capable of handling all the tasks you require; May lack scalability; May have difficulty in upgrading: May have difficulty in adapting to new platforms in the future;  Commercial Software Pros : Has the benefit of being extensively tested and used by other businesses; Quicker and smoother integration with other commercial software; Developed with team of highly skilled developers (almost guarantees that the software is created correctly) ; Fast deployment (In a fast-paced business world, this can save valuable time so you can get back to what’s important).  Commercial Software Cons: Can accrue high support and maintenance costs (In order to address any technical issues, you will be at the mercy of a vendor who sets and can change the price of service. There may also be a waiting period for support, and you may have to jump through hoops to reach a technician. This can end up costing time and slow operations);  Week 2 Systems Engineering Overview 23 Verification and Validation  Requirements engineering The process of establishing the services that the customer requires from a system and the constraints under which it operates and is developed. The requirements themselves are the descriptions of the system services and constraints that are generated during the requirements engineering process. 25  What is a requirement? It may range from a high-level abstract statement of a service or of a system constraint to a detailed mathematical functional specification. This is inevitable as requirements may serve a dual function – May be the basis for a bid for a contract - therefore must be open to interpretation; – May be the basis for the contract itself - therefore must be defined in detail; – Both these statements may be called requirements. 26  Types of requirement User requirements – Statements in natural language plus diagrams of the services the system provides and its operational constraints. Written for customers. System requirements – A structured document setting out detailed descriptions of the system’s functions, services and operational constraints. Defines what should be implemented so may be part of a contract between client and contractor. 27 User and system requirements 28  Functional and non-functional requirements Functional requirements – Statements of services the system should provide, how the system should react to particular inputs and how the system should behave in particular situations. – May state what the system should not do. Non-functional requirements – Constraints on the services or functions offered by the system such as timing constraints, constraints on the development process, standards, etc. – Often apply to the system as a whole rather than individual features or services. Domain requirements – Constraints on the system from the domain of operation 29  Usability requirements The system should be easy to use by medical staff and should be organized in such a way that user errors are minimized. (Goal) Medical staff shall be able to use all the system functions after four hours of training. After this training, the average number of errors made by experienced users shall not exceed two per hour of system use. (Testable non-functional requirement) 30 Metrics for specifying nonfunctional requirements Property Measure Speed Processed transactions/second User/event response time Screen refresh time Size Mbytes Number of ROM chips Ease of use Training time Number of help frames Reliability Mean time to failure Probability of unavailability Rate of failure occurrence Availability Robustness Time to restart after failure Percentage of events causing failure Probability of data corruption on failure Portability Percentage of target dependent statements Number of target systems 31  Domain requirements The system’s operational domain imposes requirements on the system. – For example, a train control system has to take into account the braking characteristics in different weather conditions. Domain requirements be new functional requirements, constraints on existing requirements or define specific computations. If domain requirements are not satisfied, the system may be unworkable. 32  Domain requirements problems Understandability – Requirements are expressed in the language of the application domain; – This is often not understood by software engineers developing the system. Implicitness – Domain specialists understand the area so well that they do not think of making the domain requirements explicit. 33 Example requirements for the insulin pump software system 3.2 The system shall measure the blood sugar and deliver insulin, if required, every 10 minutes. (Changes in blood sugar are relatively slow so more frequent measurement is unnecessary; less frequent measurement could lead to unnecessarily high sugar levels.) 3.6 The system shall run a self-test routine every minute with the conditions to be tested and the associated actions defined in Table 1. (A self-test routine can discover hardware and software problems and alert the user to the fact the normal operation may be impossible.) 34  Requirements engineering processes The processes used for RE vary widely depending on the application domain, the people involved and the organisation developing the requirements. However, there are a number of generic activities common to all processes – Requirements elicitation; – Requirements analysis; – Requirements validation; – Requirements management. In practice, RE is an iterative activity in which these processes are interleaved. 35 A spiral view of the requirements engineering process 36  Requirements elicitation and analysis Sometimes called requirements elicitation or requirements discovery. Involves technical staff working with customers to find out about the application domain, the services that the system should provide and the system’s operational constraints. May involve end-users, managers, engineers involved in maintenance, domain experts, trade unions, etc. These are called stakeholders. 37  Problems of requirements analysis Stakeholders don’t know what they really want. Stakeholders express requirements in their own terms. Different stakeholders may have conflicting requirements. Organisational and political factors may influence the system requirements. The requirements change during the analysis process. New stakeholders may emerge and the business environment may change. 38  Requirements elicitation and analysis Software engineers work with a range of system stakeholders to find out about the application domain, the services that the system should provide, the required system performance, hardware constraints, other systems, etc. Stages include: – Requirements discovery, – Requirements classification and organization, – Requirements prioritization and negotiation, – Requirements specification. 39 The requirements elicitation and analysis process 40  Process activities Requirements discovery – Interacting with stakeholders to discover their requirements. Domain requirements are also discovered at this stage. Requirements classification and organisation – Groups related requirements and organises them into coherent clusters. Prioritisation and negotiation – Prioritising requirements and resolving requirements conflicts. Requirements specification – Requirements are documented and input into the next round of the spiral. Problems of requirements elicitation Stakeholders don’t know what they really want. Stakeholders express requirements in their own terms. Different stakeholders may have conflicting requirements. Organisational and political factors may influence the system requirements. The requirements change during the analysis process. New stakeholders may emerge and the business environment change.  Requirements validation Concerned with demonstrating that the requirements define the system that the customer really wants. Requirements error costs are high so validation is very important – Fixing a requirements error after delivery may cost up to 100 times the cost of fixing an implementation error. 43  Requirements checking Validity. Does the system provide the functions which best support the customer’s needs? Consistency. Are there any requirements conflicts? Completeness. Are all functions required by the customer included? Realism. Can the requirements be implemented given available budget and technology Verifiability. Can the requirements be checked? 44 Requirements validation techniques Requirements reviews – Systematic manual analysis of the requirements. Prototyping – Using an executable model of the system to check requirements. Test-case generation – Developing tests for requirements to check testability. Requirements 45 Validation  Requirements reviews Regular reviews should be held while the requirements definition is being formulated. Both client and contractor staff should be involved in reviews. Reviews may be formal (with completed documents) or informal. Good communications between developers, customers and users can resolve problems at an early stage. Requirements 46 Validation  Review checks Verifiability – Is the requirement realistically testable? Comprehensibility – Is the requirement properly understood? Traceability – Is the origin of the requirement clearly stated? Adaptability – Can the requirement be changed without a large impact on other requirements?Requirements 47 Validation Requirements management Requirements management is the process of managing changing requirements during the requirements engineering process and system development. New requirements emerge as a system is being developed and after it has gone into use. You need to keep track of individual requirements and maintain links between dependent requirements so that you can assess the impact of requirements changes. You need to establish a formal process for making change proposals and linking these to system requirements. 48  Changing requirements The business and technical environment of the system always changes after installation. – New hardware may be introduced, it may be necessary to interface the system with other systems, business priorities may change (with consequent changes in the system support required), and new legislation and regulations may be introduced that the system must necessarily abide by. The people who pay for a system and the users of that system are rarely the same people. – System customers impose requirements because of organizational and budgetary constraints. These may conflict with end-user requirements and, after delivery, new features may have to be added for user support if the system is to meet its goals. 49  Changing requirements Large systems usually have a diverse user community, with many users having different requirements and priorities that may be conflicting or contradictory. – The final system requirements are inevitably a compromise between them and, with experience, it is often discovered that the balance of support given to different users has to be changed. 50 Requirements evolution 51 Verification and Validation Assuring that a software system meets a user's needs Verification vs. validation Verification: "Are we building the product right" – The software should conform to its specification Validation: "Are we building the right product" – The software should do what the user really requires V & V must be applied at each stage in the software process Two principal objectives – Discovery of defects in a system – Assessment of whether the system is usable in an operational situation  Static and dynamic verification STATIC – Software inspections – Concerned with analysis of the static system representation to discover problems – May be supplement by tool-based document and code analysis DYNAMIC – Software testing – Concerned with exercising and observing product behaviour – The system is executed with test data and its operational behaviour is observed  Static and dynamic V&V Static verification Requirements High-level Formal Detailed specification Program specification design design Dynamic Prototype validation  Program testing Can reveal the presence of errors, not their absence A successful test is a test which discovers one or more errors The only validation technique for non- functional requirements Should be used in conjunction with static verification to provide full V&V coverage  Types of testing Defect testing – Tests designed to discover system defects. – A successful defect test is one which reveals the presence of defects in a system. Statistical testing – Tests designed to reflect the frequency of user inputs – Used for reliability estimation  V & V goals Verification and validation should establish confidence that the software is fit for purpose This does not mean completely free of defects Rather, it must be good enough for its intended use – The type of use will determine the degree of confidence that is needed  V & V confidence Depends on system’s purpose, user expectations and marketing environment – Software function The level of confidence depends on how critical the software is to an organisation – User expectations Users may have low expectations of certain kinds of software – Marketing environment Getting a product to market early may be more important than finding defects in the program  Testing and debugging Defect testing and debugging are distinct processes Verification and validation is concerned with establishing the existence of defects in a program Debugging is concerned with locating and repairing these errors – Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the system error  Week 3 Systems Engineering Overview 62  V & V planning Careful planning is required to get the most out of testing and inspection processes Planning should start early in the development process The plan should identify the balance between static verification and testing Test planning is about defining standards for the testing process rather than describing product tests  The V-model of development Requir ements System System Detailed specification specification design design System Sub-system Module and Acceptance integration integration unit code test plan test plan test plan and tess Acceptance System Sub-system Service test integration test integration test The structure of a software test plan The testing process Requirements traceability Tested items Testing schedule Test recording procedures Hardware and software requirements Constraints  Software inspections Involve people examining the source representation with the aim of discovering anomalies and defects Do not require execution of a system – May be used before implementation May be applied to any representation of the system – Requirements, design, test data, etc. Very effective technique for discovering errors Many different defects may be discovered in a single inspection – In testing, one defect may mask another so several executions are required Reuse of domain and programming knowledge – Reviewers are likely to have seen the types of error that commonly arise  Inspections and testing Inspections and testing are complementary and not opposing verification techniques Both should be used during the V & V process Inspections can check conformance with a specification but not conformance with the customer’s real requirements Inspections cannot check non-functional characteristics such as performance, usability, etc.  Program inspections Formalised approach to document reviews Intended explicitly for defect DETECTION (not correction) Defects may be – logical errors – anomalies in the code that might indicate an erroneous condition (e.g. an uninitialized variable)  Inspection procedure System overview presented to inspection team Code and associated documents are distributed to inspection team in advance Inspection takes place and discovered errors are noted Modifications are made to repair discovered errors Re-inspection may or may not be required  Inspection teams Made up of at least 4 members Author of the code being inspected Inspector who finds errors, omissions and inconsistencies Reader who reads the code to the team Moderator who chairs the meeting and notes discovered errors Other roles are Scribe and Chief  Inspection checklists Checklist of common errors should be used to drive the inspection Error checklist is programming language dependent The 'weaker' the type checking, the larger the checklist Examples – Initialisation – Constant naming – Loop termination – Array bounds  Inspection rate 500 statements/hour during overview 125 source statement/hour during individual preparation 90-125 statements/hour can be inspected Inspection is therefore an expensive process Inspecting 500 lines costs about 40 person/hours  Automated static analysis Static analysers are software tools for source text processing They parse the program text and try to discover potentially erroneous conditions and bring these to the attention of the V & V team Very effective as an aid to inspections. A supplement to but not a replacement for Static analysis checks  Stages of static analysis Control flow analysis – Checks for loops with multiple exit or entry points, finds unreachable code, etc. Data use analysis – Detects uninitialized variables, variables written twice without an intervening assignment, variables which are declared but never used, etc. Interface analysis – Checks the consistency of routine and procedure declarations and their use Stages of static analysis (2) Information flow analysis – Identifies the dependencies of output variables – Does not detect anomalies, but highlights information for code inspection or review Path analysis – Identifies paths through the program and sets out the statements executed in that path – Also potentially useful in the review process Both these stages generate vast amounts of information – Handle with caution!  Cleanroom software development The name is derived from the 'Cleanroom' process in semiconductor fabrication. The philosophy is defect avoidance rather than defect removal Software development process based on: – Incremental development – Formal specification – Static verification using correctness arguments – Statistical testing to determine program reliability  The Cleanroom process Formally Error rework specify system Define Construct Formally Integrate software structured verify increment increments program code Develop operational Design Test profile statistical integrated tests system  Cleanroom process teams Specification team – Responsible for developing and maintaining the system specification Development team – Responsible for developing and verifying the software – The software is not executed or even compiled during this process Certification team – Responsible for developing a set of statistical tests to exercise the software after development – Reliability models are used to determine when reliability is acceptable  Verification and testing tools 1. Verification – For C programs, an old tool called “lint” – For Java programs, some nice tools integrate with Eclipse: CheckStyle and PMD – For distributed systems verification a Spin tool was designed  Developed in Bell Labs, starting in 1980. 2. Automated Testing – TestComplete is an automated testing environment for Win32,.NET and Windows Presentation Foundation (WPF) applications. – TestComplete provides extended support for testing Web Pages, Web Servers, Web Services and Projects created in the following development tools: Microsoft Visual C++/Borland C++ Builder, VB, Delphi, Java,.NET, and WPF.  Automated Testing Cont’… Siebel: Bug tracking tool from Oracle. Rational Robot: Load testing tool from IBM. Mercury winrunner: Performance testing tool from Hp SIPP: For Sip protocol testing. Ethereal: For sniffing packets. Test complete: For GUI automation  Key points Verification and validation are not the same thing. – Verification shows conformance with specification – Validation shows that the program meets the customer’s needs Test plans should be drawn up to guide the testing process Static verification techniques involve examination and analysis of the program for error detection Program inspections are very effective in discovering errors Program code in inspections is checked by a small team to locate software faults Static analysis tools can discover program anomalies which may be an indication of faults in the code The Cleanroom development process depends on Systems interface  Definition An interface is a shared boundary across which two or more separate components of a computer system exchange information. The exchange can be between software, computer hardware, peripheral devices, humans and combinations of these. A software interface may refer to a wide range of different types of interface at different "levels": an operating system may interface with pieces of hardware. Applications or programs running on the operating system may need to interact via streams, and in object oriented programs, objects within an application may need to interact via methods. A key principle of design is to prohibit access to all resources by default, allowing access only through well-defined entry points Interfaces between software components can provide: constants, data types, types of procedures, exception specifications and method signatures. Sometimes, public variables are also defined as part of an interface.  Definition Cont…  Systems interface are broadly defined as inputs or outputs with minimal or no human intervention ◦ Inputs from other systems (messages, EDI) ◦ Highly automated input devices such as scanners and bar code reader ◦ Inputs that are from data in external databases ◦ Outputs to external databases ◦ Outputs with minimal HCI ◦ Outputs to other systems ◦ Real-time connections (both input and output). Electronic Data Interchange (EDI) is the computer-to-computer exchange of business documents in a standard electronic format between business partners.  Definition Cont…. The engineering of interfaces is a critical function of the discipline of Systems Engineering. Many system problems result from inadequately defined interfaces Problems often surface during integration and test Interfaces provide the specifications of the relevant properties of a system or component that can be connected to other systems or components while instances of interaction are identified in order to specify the actual integration to other systems or components  Terms Definition Term Definition The system boundary that is presented by a Interface system for interaction with other systems. Describes the nature of the boundary presented Interface by a system or component in terms of properties Specification and functionality. An instance of an operational entity (system, Interaction organization, or services) interface. The interaction can connect to other operational entities according to its Interaction Specification. Describes how an operational entity (system, Interaction organization, or service) can effect another Specification operational entity when a connection exists.  API Application Programming Interface (API)is a set of clearly defined methods of communication between various software components. An API may include specifications for routines, data structures, object classes, and variables. An API can be: language-dependent, meaning it is only available by using the syntax and elements of a particular language. language-independent, written so that it can be called from several programming languages. This is a desirable feature for a service-oriented API that is not bound to a specific process or system and may be provided as remote procedure calls or web services. For example, a website that allows users to review local restaurants is able to layer their reviews over maps taken from Google Maps. The POSIX standard defines an API that allows a wide range of common computing functions to be written in a way such that they may operate on many different systems  API There are many benefits of API. General benefits of API are given below: 1. Efficiency: Data are created at once and made available to different systems. So, it helps to ease the distribution of data. API hides the complexity of program. 2. Wider Reach: Anyone can use API. Suppose, 'A' creates a website www.example.com. 'A' uses B's API. So, subscriber of A can get the knowledge and services of B. 3. Automation: Since API is a machine to machine interaction, people don't need to work every time when data are changed. 4. Partnership: API elaborates the partnership among the companies. As the company grows, API Provider Company needs to collaborate to each and every company that uses their API.