Systems Engineering Management Planning PDF

Systems Engineering Management Activities Planning and directing - Running the project according a plan Monitoring and reporting - Reporting to inside, outside Reviewing and auditing - At critical stages / moments in the process Topic...

Systems Engineering Management Activities Planning and directing - Running the project according a plan Monitoring and reporting - Reporting to inside, outside Reviewing and auditing - At critical stages / moments in the process Topics Technical review and auditing Testing and evaluating €€ Technical Risks management Con guration management Planning and directing / SEMP Jan Broenink Systems Engineering, 202200100 1 16-Managment-Risks.key - 25 October 2024  fi  Systems Engineering Management Planning Plan of management of the design and development process Initial Planning - timeline of what must be done Follow-on control - Keep on track - If deviation / delay is expected: change plans - Be ahead of problems Objective: right item at right location, at right time, at minimal costs Documented in SEMP — Systems Engineering Management Plan - Approved during a design review - Changes to be approved during a design review Project Planning / Risks - Updated during the project Jan Broenink Systems Engineering, 202200100 2 16-Managment-Risks.key - 25 October 2024  SEMP — Systems Engineering Management Plan Topics The work to be done / managed - Summary of the work in the project SE task description: Work Breakdown Structure - Parts of work: Work Packages - To design CIs and realise EPs - So all activities to be managed Schedule of tasks - Time line, Gantt chart Personnel - Project structure and responsibilities Document structure Cost projection According to a standard - Financial budget - Quite some mentioned in ASE - Materials For the case - WBS, Gantt, Milestones, - Integration & Veri cation plan Jan Broenink Systems Engineering, 202200100 3 16-Managment-Risks.key - 25 October 2024 fi  Example WBS 2B1 - 3B1… detailed out Levels 1, 2, 3 rather complete Tasks of Sec. 18.2.2 4 Jan Broenink Systems Engineering, 202200100 Figs 18.6, 18.7, SEaA 16-Managment-Risks.key - 25 October 2024  Schedule, Gantt chart Schedule Henry Gantt, 1861 - 1919 Hoover dam, Interstate Highway Bar chart: Gantt; Network: PERT - Tasks and dependencies - Estimate duration of tasks ‣ Most likely, pessimistic, optimistic ‣ Add slack time for robustness - Milestones - Events: review, deliver Cost projection - Personnel, materials Tools - Gantt Project, Excel templates, etc Update schedule Track progress - Task status (%% complete) - Effect of Risk mitigation - Update schedule 5 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Example: EU project DESTECS FP7-ICT-2009-4 STREP proposal 01/04/09 v9m DESTECS 5 Work Packages 1: Management 2: Methods 3: Tool development 4: Industrial Test cases 5: Dissemination 3 years, yearly iteration 8 partners ( = 7 locations) 4 countries 331 PM = 27 PY) 20 persons ( incl 7 full time) budget 3.62 M€ EU funded 2.74 M€ Figure 3: Project interdependencies. Task Dependencies - PERT chart (free format) PM: person month The chart shows a loop that starts and ends on task T3.3 that is resolved by proper timing: PY: person year Task 3.4 starts 202200100 to feed the case studies (T4.1 to T4.3) with state-of-the-art tools. The results 6 Jan Broenink Systems Engineering, are used in the methodology work (T2.1, T2.2 and T2.3) which will feed the work on new tools (T3.1 to T3.3). The resulting tools are then fed back to the case studies through T3.4. This closes the ‘loop’. For the second run through this loop (iteration two in year two), the case studies (T4.1 to T4.3) will provide requirements for improvements to the WP2 and WP3 teams. In the third year, this iteration takes place again. The deliverables after each year of WP2, WP3 and WP4 reflect this iteration structure. 16-Managment-Risks.key - 25 October 2024  01/04/09 v9m FP7-ICT-2009-4 DESTECS Gantt Chart (condensed) Project Month 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 Wp WP Lead PM start end 1 2 3 4 5 6 7 8 9 1 Project Management UT 23 1.1 project Management UT 23 1 36 D1.1 D1.2 D1.3 D1.2 D1.3 D1.2 D1.3 2 Design Space Exploration UNEW 87 D2.1 D2.2 D2.3 2.1 Support for Collaborative DSE UNEW 28 1 36 2.2 Support for Fault Tolerant Design UNEW 39 1 36 2.3 Model Structuring for Co-Sim UT 20 2 36 3 Tool Prototypes and Services IHA 109 D3.2 D3.3 D3.4 Proposal Part B: page 26 of 85 3.1 Open Platform IHA 22 1 27 Figure 2: Project timing. 3.2 Co-Simulation IHA 29 3 32 3.3 Dependability Exploration Tool UT 29 6 36 3.4 Tool Support for Ind Case studies IHA 16 1 36 D3.1 3.5 Extension to Matlab / Simulink CLP 13 19 36 D3.5 4 Industrial Case Studies CHESS 62 D4.1 D4.2 D4.3 4.1 Case Paper handling NEOP 21 1 36 4.2 Case Intelligent M&C VERH 21 1 36 4.3 Case People Transporter CHESS 20 1 36 5 Dissemination / Exploitation DTI 50 5.1 Website & on-line community UT 15 1 36 D5.1 D5.2 D5.2 D5.2 5.2 Industrial Follow Group DTI 17 7 30 D5.3 D5.4 D5.6 D5.5 / D5.8 5.3 Exploitation Report DTI 13 30 36 D5.6 / D5.7 5.4 Summerschool IHA 5 24 36 D5.9 M1 M2 M3 Changes: No dependencies indicated DTI stepped out, and IHA took over (Y1) Consortium Meeting Milestones (document deadlines) CHESS split up into 2 companies (Y3) STREP proposal Milestone: Annual review at Consortium Meeting are implicit task dependencies DESTECS 7 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Planning of a student research project MTP 4 10 21 28 32 33 Planning: 3 Phases INT 1 5 11 15 16 25 BSc 1 3 7 9 10 11 Main compartments of work Exploration Production Finalising Feasibility Study Production Fine Tuning Rolling forecast Arrange keeping-on-track Desk Research Documentation Reporting Progress meetings - On content P PP S D GL R P+A PIP (weekly) progress meeting P Proposal S Status update Progress meetings GL Green Llight PIPPF (monthly) progress meeting PP Project Plan D Demo R Submit Report P+A Presentation & Assessment - On overview, on planning - Risk analysis to tune / update planning 8 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Directing: run Project according to Plan Essential activity Implement / run SEMP Day-to-day direction - Where are we today in the project? Evaluation of planning: Risks Outcome Corrective action on planning - If needed!! Update / enhance plan - Change the planning ? ! From: http://en.wikipedia.org/wiki/Project_management 9 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Program Review - Are We on Track? Objective Assess degree of progress / schedule / costs Determine status of tasks Dealing with issues — Risk management Identify problems, rank in order of importance - w.r.t. end result… From: http://en.wikipedia.org/wiki/Project_management Handle each problem, most important rst - seek alternatives; check effects on - schedule / planning, also priority of tasks - costs of change, cost when nothing changes… - system performance / effectiveness - how risky this change is => risks associated to corrective actions - Plan resolving the problem External - management, ECP, CCP, etc - Aftermath - did the change really solve the problem? - no new problems induced? 10 Jan Broenink Systems Engineering, 202200100 Fig 19.12, SDaE 16-Managment-Risks.key - 25 October 2024 fi  Keep on Track — Risk Analysis When upcoming deadlines might be compromised i.e. some delay, some problematic tasks, etc Risk analysis of near-future / current tasks Does task contribute to results? Does omitting task affect results at upcoming mile stone negatively? Are alternatives possible? or alternative way of doing this task? - can the task be done in less time (less detailed, less precise???) Different order of tasks more feasible? - for instance, when delivery of parts takes longer than expected - so, changing priority of tasks Can somebody else do this task / give support? Tools / techniques risk management system budgets, margins, see SDaE Sec §4.8, ASE step C 3.4 - power, weight, space, accuracy, cycle time, service time, and also money = costs (in €) 11 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Program Review - Are We on Track? Time History: Project Flow and Cost TPM vs Requirement SEaA, Fig 19.5, 19.7 12 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Student project: On track? Updated planning 13 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024  Example: EU project DESTECS — I Proposal -> Description Of Work, including Tasks (= WBS, Tasks) Risk Management Plan: possible risks, plan to mitigate Gantt and PERT (rather basic); 3 cycles of 12 Month each 01/04/09 v9m FP7-ICT-2009-4 Project Month Risk Management Plan in proposal 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 Wp WP Lead PM start end 1 2 3 4 5 6 7 8 9 1 Project Management UT 23 1.1 project Management UT 23 1 36 D1.1 D1.2 D1.3 D1.2 D1.3 D1.2 D1.3 2 Design Space Exploration UNEW 87 D2.1 D2.2 D2.3 partner withdraws / key personnel leaves / unwilling to share info 2.1 Support for Collaborative DSE UNEW 28 1 36 2.2 Support for Fault Tolerant Design UNEW 39 1 36 2.3 Model Structuring for Co-Sim UT 20 2 36 3 Tool Prototypes and Services IHA 109 D3.2 D3.3 D3.4 Proposal Part B: page 26 of 85 3.1 Open Platform IHA 22 1 27 tasks run out of time; task results poor / completely different Figure 2: Project timing. 3.2 Co-Simulation IHA 29 3 32 3.3 Dependability Exploration Tool UT 29 6 36 3.4 Tool Support for Ind Case studies IHA 16 1 36 D3.1 3.5 Extension to Matlab / Simulink CLP 13 19 36 D3.5 Meeting structure - in the plan! 4 Industrial Case Studies CHESS 62 D4.1 D4.2 D4.3 4.1 Case Paper handling NEOP 21 1 36 4.2 Case Intelligent M&C VERH 21 1 36 4.3 Case People Transporter CHESS 20 1 36 5 Dissemination / Exploitation DTI 50 5.1 Website & on-line community UT 15 1 36 D5.1 D5.2 D5.2 D5.2 Strategy, Risk analysis: Executive Board, 2x year 5.2 Industrial Follow Group DTI 17 7 30 D5.3 D5.4 D5.6 D5.5 / D5.8 5.3 Exploitation Report DTI 13 30 36 D5.6 / D5.7 5.4 Summerschool IHA 5 24 36 D5.9 M1 M2 M3 FP7-ICT-2009-4 STREP proposal 01/04/09 v9m DESTECS - To discuss and mitigate risks on project level Consortium Meeting STREP proposal Milestone: Annual review at Consortium Meeting Operational, Project wide DESTECS - WP Leaders: monthly online meetings - Within Work Packages: biweekly or monthly online meetings Figure 3: Project interdependencies. The chart shows a loop that starts and ends on task T3.3 that is resolved by proper timing: Task 3.4 starts to feed the case studies (T4.1 to T4.3) with state-of-the-art tools. The results are used in the methodology work (T2.1, T2.2 and T2.3) which will feed the work on new 14 Jan Broenink Systems Engineering, 202200100 tools (T3.1 to T3.3). The resulting tools are then fed back to the case studies through T3.4. This closes the ‘loop’. For the second run through this loop (iteration two in year two), the case studies (T4.1 to T4.3) will provide requirements for improvements to the WP2 and WP3 teams. In the third year, this iteration takes place again. The deliverables after each year of WP2, WP3 and WP4 reflect this iteration structure. 1.3.8 Risk Management Every project, even if carefully planned, will face risks. Some risks can be foreseen and dealt with by contingency plans. In the DESTECS project we will carefully monitor these risks and ensure that the contingency plans stay up to date. Other risks cannot be foreseen and may lead to unexpected incidents. These incidents will be dealt with on an ad-hoc basis as described in Section 2.1. Risk assessment 16-Managment-Risks.key - 25 October 2024 During each EB meeting, the technical and project management decisions will be reviewed with respect to identification of potential areas of risk, both management risks and technical risks. When potential risks were found, they will be analysed to identify the cause(s), effects  Actual Risk Management EU project DESTECS Risk management to keep on track - Late staf ng Y1 - -> extra training courses, update Y1 plan - Case not suitable, design ows too diverse - -> other case; requirements formulation - Tool development too much spread over partners - concentrated by moving PM (thus €) - According to RMP: keep results and planning on level - Test results too slow to Methods Work Package - own pilot study (the line-following robot) Updates of the Formal Plan? - Split of one company at start Y3 - Technical and planning: no updates - Keep their key personnel, according to RMP - Organisational: partners changed legally Budget checks - -> update contracts incl €€ budgets - Person Months (PMs) and costs, in €€€ and % of total - -> update WBS Comments by reviewers at end of project: Very well managed project 15 Jan Broenink Systems Engineering, 202200100 16-Managment-Risks.key - 25 October 2024 fi fl  Production, use & retirement dr. ir. G.A. Folkertsma, CEng Autumn 2024 dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 1 / 23 Context Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Production process design 3. Conceptual design & stakeholder analysis I FAT and SAT / TRR and FQR 4. Scenarios & Use cases I Change management 5. Context & external interfaces I “Retirement” in a circular world 6. Requirements 7. Preliminary design & Architecting 8. Internal interfaces 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 2 / 23 Connecting Revisit: Verification Question (2 min) What is the difference between Verification and Validation? Question (2 min) What is the impact of a design change during the production phase? dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 3 / 23 Connecting Prior knowledge I What do you already know about Production, use & retirement? I Where are we in the design process? Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production I Where are we in the V-model? problem spec/design real/verif solution dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 4 / 23 Manufacturing Who and how many? dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 5 / 23 Manufacturing Scope Many / series production Few / “one-off” I Supply chain I Subsystem verification I Incoming goods inspection I FCA / PCA I Work instructions I “As-built TPD” I QA inspections I Outgoing logistics dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 6 / 23 Manufacturing Integration checks: FCA and PCA 1. The (sub)system does not build/cannot be made/ does not fit. I Executed on CIs 2. The (sub)system does not function. 3. Interface errors. I FCA: does it work? 4. The (sub)system is too slow. I PCA: was it built according to design? 5. The (sub)system does not meet main performance parameters. 6. Reliability is not met. dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 7 / 23 Manufacturing Integration checks: TRR I Internal I TEMP (MTP) check I Test plans (TAS) I (Intermediate) Test results (TAR) I Supporting documentation I Support, test equipment, and facilities dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 8 / 23 Manufacturing Integration checks: FQR I Formal I FAT / SAT I SVR: System Verification Record dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 9 / 23 Manufacturing Exercise: FAT and SAT for robot digger For your robot digger: a) Describe what the Factory Acceptance Test (FAT) looks like. b) Describe what the Site Acceptance Test (SAT) looks like. Remember your key customer needs, key drivers, scenarios and mission, and relate the FAT+SAT to those. After 15 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 10 / 23 break Break 15 minutes’ break dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 11 / 23 Operation OT&E I Proof of the pudding? I User or manufacturer? I Support systems I Monitoring I SLA dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 12 / 23 Operation FRACAS Failure Reporting, Analysis and Corrective Action System 1. Failure reporting (OT&E) 2. Failure analysis 3. Failure verification 4. Corrective action 5. Failure report and close-out 6. ID and control of failed items dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 13 / 23 Operation Left-shift Question (2 min) What is meant by left-shift? des ign eas cho yo ice fc integration issues han s changes or issues or… ges ge l ed es n ow hang k fc s to co Systems Engineering design timeline dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 14 / 23 Operation Change management I Impact: cost of changes I Compatibility I TPD I MTP ok, but TAS? I FCA ok, but PCA? I Supply chain I SVR I Work instructions I Careful change control dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 15 / 23 Operation Configuration Management (ASE § 8.5) I Establishing the Baselines I FBL I ABL I PBL Configuration Change Management I Configuration (or Change) Control Board I Class I vs. Class II I Request for Variance I Engineering Change Proposal I CCB dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 16 / 23 Operation Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 17 / 23 Circularity “ If it can’t be reduced, reused, repaired, rebuilt, refurbished, refinished, resold, recycled or composted, then it should be restricted, redesigned or removed from production. ” – Pete Seeger, AT 89 tr. 15 dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 18 / 23 Circularity out of our wasted products. (Urban Mining) Where the old economy is characterized by take make use and dispose, the upcoming circular economy aims to use and Closing the loop keep resources at the highest possible purpose. use of finite Damage to resources NOW the (future) from the past planet Take Make Use Dispose Share Unrecoverable Repair Use Losses Re-Use Refurbish Re-Use on part/module level Re-Use on material / chemical level / Re -cycling Downcycle loss Regenerate Lifecycle Losses life cycle consumption (e.g. energy, water) dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 19 / 23  the same design. Every viewport has its own focus points and has its own frame and terminology. With viewport, it is Circularity latterly meant to view with another focus and perspective and the viewport goals will help to do so. How? 0. BASELINE Agreed design goals - translation of customer needs in requirements 9. PHASE REVIEW EVALUATION 1. DESIGN SOLUTION Outbalancing (and iteration) of stakeholder Technical & functional solution for the user design baseline needs and set new baseline problem & needs in requirements fulfillment of customer voice (baseline) & stakeholders requirement balance customer technical 2. USABILIT BRANDING. SUSTAINABILIT Needs limits Optimize product user/stakeholder Balance product user experience experience and balance product environmental/ social footprint - footprint use use scenario s & value, cost price, quality & impact with its benefits environmental & benefit value proposition performance social impact 3rd Flower Model use scope error s footprint For Sustainable 7. SUPPL CHAIN Product Design Economically and reliable supply of product purchase make safety product safety components or critical & 3. SAFET & Secure operational safety of all logistics buy functions certification stakeholders in all product phases in design critical all (mis)use scenario s for (spare) design for service parts reliability (re) assembly & (re) manufacturing availability & (re)use maintenance & repair of functionality.RELIABILIT replace & refurbish Support Of Customer / Elongate. PRODUCT DESIGN upgrades Operation & Prevent / Shorten Translation of the (technical) design Interruption Of Operation solution into an affordable reliable producible product.. SERVICE Support Of Customer / Elongate Operation & Prevent / Shorten Interruption Of Operation dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 20 / 23 Circularity Exercise: circularity for robot digger 1. Come up with two or three footprint reduction ideas. Think about material, operation, energy, business model, retirement, … 2. Determine how to quantify† the impact of each. 3. Try to quantify the financial aspect of each. Consider the cost of development, goods, production, use. † Define a metric, even if you cannot give a number right now. After 10 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 21 / 23 Conclusion Questions Question (2 min) What is the role of the CCB? Question (2 min) What is FRACAS? Question (2 min) Name three things that should be considered for series production that are irrelevant* for a “one-off.” dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 22 / 23 Conclusion Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Production process design 3. Conceptual design & stakeholder analysis I FAT and SAT / TRR and FQR 4. Scenarios & Use cases I Change management 5. Context & external interfaces I “Retirement” in a circular world 6. Requirements 7. Preliminary design & Architecting Reading for next time: 8. Internal interfaces I ASE Ch 8 9. Budgets & money I SDE Ch 5 10. Decision-making 11. Risk management Case assigment: 12. FMEA, RMF SEMP: integration, verification, Gantt 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Production, use & retirement Autumn 2024 23 / 23  Verification dr. ir. G.A. Folkertsma, CEng Autumn 2024 dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 1 / 22 Context Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Verification methods 3. Conceptual design & stakeholder analysis I Design qualification vs. Quality 4. Scenarios & Use cases conformance 5. Context & external interfaces I Master test plan 6. Requirements I Test specifications & reports 7. Preliminary design & Architecting 8. Internal interfaces 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 2 / 22 Context Note: many new things; not in the books. More text on slides, but also take notes! dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 3 / 22 Connecting Revisit: Detailed design & integration Question (2 min) What is the goal of the Detailed Design stage? Question (2 min) Name two common integration issues. How will we know our design is good? → we will do verification! dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 4 / 22 Connecting Prior knowledge I What do you already know about Verification? I Where are we in the design process? Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production I Where are we in the V-model? problem spec/design real/verif solution dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 5 / 22 Verification activities What do we test when checking our requirements? Verification Validation I “veritas,” truth I “validus,” strong I “The evaluation of whether a product, I “The assurance that a product, service, or service, or system complies with a system meets the needs of the customer regulation, requirement, specification, or and other identified stakeholders. imposed condition. It often involves acceptance and suitability It is often an internal process.” with external customers.” I Did we build the product right? I Did we build the right product? dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 6 / 22 Verification activities Remember Bill… dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 7 / 22 Verification activities Verification methods Inspection visible provision implemented, e.g. transport handle Demonstration qualitative dynamic behaviour, e.g. hatch opens Test quantitative parameter proven in test procedure, e.g. speed Analysis quantitative parameter that will not be tested, e.g. maximum shock level Question (2 min) What do you think is the best verification method? dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 8 / 22 Verification activities Cost of verification … but don’t over-think it and be the too-analytical-academic(!) dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 9 / 22 Verification activities Examples ~ slide intentionally empty ~ dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 10 / 22 Verification activities Requirements for verification I Measurable? I Internal measuring sensors? I Measuring equipment + calibration I Enabling systems I Mock-ups I Material I Patients Testing strategy after the break… dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 11 / 22 Verification activities Exercise: verification of robot digger requirements For (some of) the requirements you wrote in lecture 6: a) What verification method seems best? b) Which extra measures, items, equipment do you need for verification? I Inspection D Demonstration T Test A Analysis After 15 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 12 / 22 break Break 15 minutes’ break dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 13 / 22 Verification process Testing through the lifecycle Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 14 / 22 Verification process ASE: DT&E I Functional testing I Interface testing I Environmental testing I Physical and configurational testing I Quality factor testing dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 15 / 22 Verification process Master Test Plan (TEMP, V&V plan) I Test strategy I Written early 1 7 I ASE: “TEMP” finalised* and approved (SDR) during Conceptual Design I Demcon: “MTP” delivered 2 6 after Product Definition I T&E outline (DT&E, AT&E, OT&E) I T&E resource summary 3 5 4 dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 16 / 22 Verification process Qualification DQ/QC Design Qualification Checked once to verify the design. Quality Conformance verified for every product (or spot check). Acceptance FAT Factory Acceptance Test → when leaving the (production) factory SAT Site Acceptance Test → when installed at the customer and incoming goods inspection, and medical testing, and… dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 17 / 22 Verification process TAS/TAR TAS Test & Analysis Specification TAR Test & Analysis Report medical vs. industrial dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 18 / 22 Verification process MTP vs. TAS Master Test Plan Test & Analysis Specification I Defines overall test approach I Specific test plan I Covers all development stages I Covers “logical set of requirements” I Describes right leg of V; approach of: I Describes: I component tests I related requirements (traceability) I integration tests I test method I verification tests I deliverables (TAR) I validation tests I conditions, setup, equipment I High-level description of test process I safety precautions I Detailed description of test(s) dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 19 / 22 Verification process Exercise: TAS for robot digger Write a test plan (TAS) for one of the requirements of the robot digger. After 10 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 20 / 22 Conclusion Questions Question (2 min) What are the four verification methods? Question (2 min) What is the difference between a TEMP and a TAS? Question (2 min) When is verification complete? dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 21 / 22 Conclusion Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Verification methods 3. Conceptual design & stakeholder analysis I Design qualification vs. Quality 4. Scenarios & Use cases conformance 5. Context & external interfaces I Master test plan 6. Requirements I Test specifications & reports 7. Preliminary design & Architecting 8. Internal interfaces Reading for next time: 9. Budgets & money I ASE Ch 6 + 7 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Verification Autumn 2024 22 / 22  Detailed design & integration dr. ir. G.A. Folkertsma, CEng Autumn 2024 dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 1 / 18 Context Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Detailed design phase 3. Conceptual design & stakeholder analysis I CDR 4. Scenarios & Use cases I Documentation of detailed design 5. Context & external interfaces I Integration plans 6. Requirements 7. Preliminary design & Architecting 8. Internal interfaces 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 2 / 18 Connecting Revisit: FMEA, RMF Question (1 min) What is the link between FMEA and safety? Question (1 min) Which factors contribute to the risk index, risk, or RPN? Question (2 min) What is the result of risk management for the product design? dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 3 / 18 Connecting In previous projects, what did you spend most time on? (How did that fit in the past 12 lectures?) dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 4 / 18 Connecting Prior knowledge I What do you already know about Detailed design & integration? I Where are we in the design process? Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production I Where are we in the V-model? problem spec/design real/verif solution dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 5 / 18 Detailed Design phase Goal What is the goal of the Detailed Design phase? “ Establish the Product BaseLine and pass the Critical Design Review. ” – ASE “ To define an SoI that meets its agreed-to stakeholder needs and requirements and can be produced, utilised, supported, and retired. ” – INCOSE “ Have a design that is complete: ready to start the build, integration and verification of the system. ” – Demcon QMS dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 6 / 18 Detailed Design phase Process dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 7 / 18 Detailed Design phase Critical Design Review I Evaluate detailed design I Determine readiness for production I Determine maturity of software I Determine design compatibility I Establish the Product Baseline (PBL) dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 8 / 18 Detailed Design phase Design documentation at Demcon PTS Product Test Specification DFMEA Design-FMEA CCL Critical Components List TAS Test & Analysis Specification HDD Hardware Design Description (mechanical, electrical, etc.) SDD Software Design Description IDD Interface Design Description PSD Part Specification Documents TPD Technical Product Documentation (drawings) (S)RMF (Security) Risk Management Files ) CIP/CEP Clinical Investigation/Evaluation Plan e xam n the CPL Critical Parts List i ot BoM Bill of Materials m sn y a c ron ( dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 9 / 18 Detailed Design phase Exercise: software development for robot digger 1. Make a plan for the software development. 2. ??? 3. profit! After 10 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 10 / 18 break Break 15 minutes’ break dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 11 / 18 break Aside: development approaches I Waterfall approach I Incremental development → overlapping waterfall I Evolutionary development I Spiral I Multiple V dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 12 / 18 Moving towards realisation Z integration dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 13 / 18 Moving towards realisation V-model ASE dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 14 / 18 Moving towards realisation Integration issues “ Many issues occur during the verification and integration process. ” – Gerrit Muller / SDE 1. The (sub)system does not build/cannot be made/does not fit. 2. The (sub)system does not function. 3. Interface errors. 4. The (sub)system is too slow. 5. The (sub)system does not meet main performance parameters. 6. Reliability is not met. dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 15 / 18 Moving towards realisation Exercise: integration contingencies For each of the six typical integration issues, describe: a) Which measures or activities in the SysEng approach until now helps prevent them; b) How you can catch these issues as soon as possible; c) What you can do once you encounter these issues. The issues were: 1. The (sub)system does not build/cannot be made/does not fit. 2. The (sub)system does not function. 3. Interface errors. 4. The (sub)system is too slow. 5. The (sub)system does not meet main performance parameters. 6. Reliability is not met. After 15 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 16 / 18 Conclusion Questions Question (2 min) What is the goal of the Detialed Design phase? Question (2 min) What are the five activities in the CDR? Question (2 min) What is documented during detailed design in a Hardware Design Description? dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 17 / 18 Conclusion Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Detailed design phase 3. Conceptual design & stakeholder analysis I CDR 4. Scenarios & Use cases I Documentation of detailed design 5. Context & external interfaces I Integration plans 6. Requirements 7. Preliminary design & Architecting Case assigment: 8. Internal interfaces Risk management 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Detailed design & integration Autumn 2024 18 / 18  FMEA, RMF dr. ir. G.A. Folkertsma, CEng Autumn 2024 dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 1 / 19 Context Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I RPN 3. Conceptual design & stakeholder analysis I Failure Mode and Effects Analysis (FMEA) 4. Scenarios & Use cases I Safety: Risk Management File 5. Context & external interfaces I Standards 6. Requirements 7. Preliminary design & Architecting 8. Internal interfaces 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 2 / 19 Connecting Revisit: Risk management Question (1 min) How do you measure risk? Question (2 min) How can you mitigate risk? dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 3 / 19 Connecting Prior knowledge I What do you already know about FMEA, RMF? I Where are we in the design process? Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production I Where are we in the V-model? problem spec/design real/verif solution dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 4 / 19 Robustness What-if: functionality is degraded Failure Modes & Effects Analysis I something breaks I device completely malfunctions I human error I performance specs not achieved I part under-performs I … I adverse environmental conditions I requirements fail I wear & tear DFMEA vs. PFMEA (and SFMEA and FMECA) dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 5 / 19 Robustness FMEA process 1. identify failure modes 2. predict / estimate effects 3. determine remedy dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 6 / 19 Robustness Risk Priority Number RPN = S × O × D Question (2 min) Why do you think detection was added to the equation? (recall R = P · C) dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 7 / 19 Robustness FMEA: RPN Unlikely-but-severe (black swan) Often-but-mild (gremlin) Run-of-the-mill I S high I S low I S medium I O low I O high I O medium I D easy I D medium I D medium Question (3 min) Which should be tackled first? RPN = 10 · 1 · 1 = 10 RPN = 1 · 10 · 5 = 50 RPN = 5 · 5 · 5 = 125 dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 8 / 19 Robustness Risk mitigation Severity Occurrence Detection I break the causal chain I remove root cause I inspection I add redundancy I over-dimension critical I predictive maintenance I shielding/armouring components I sensors I … I preventive maintenance I status I … leds/lamps/checks/logging I degradation/wear indicators I … I Have risk-related discussions with the customer / stakeholders. I Collect “lessons learned” for future improvements. dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 9 / 19 Robustness Exercise: FMEA for robot digger 1. Find a failure mode caused by each of these three: I) human error II) environmental conditions III) part breakdown (e.g. wear & tear) 2. Describe the effect in terms of system function, TPM or requirements. 3. Determine the RPN for each. 4. For the highest-priority item, design two mitigations, acting on different aspects. 5. Choose the best mitigation and calculate the new RPN. After 10 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 10 / 19 break Break 15 minutes’ break dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 11 / 19 Safety Standards ISO 9001 Quality Management System ISO 13485 Medical devices QMS IEC 60601 Technical standards for medical electrical equipment 2023/1230/EU Directive on machinery 2017-745/EU Regulation on […] medical devices (MDR) QA / RA dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 12 / 19 Safety Risk Management Analysis dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 13 / 19 Safety Risk index aspects I Severity: how bad S1 Slight injury, e.g. scratches, laceration, bruising, light wound S2 Serious injury (including fatality), e.g. broken or torn-out or crushed limbs, fractures, flesh P1 P2 P3 wounds requiring stitches. A1 A2 A1 A2 A1 A2 I Exposure: how often E1 Seldom E1 1 1 1 1 2 2 S1 E2 Often (default) E2 1 1 1 1 2 2 I Probability: how likely is the SOE* E1 2 2 2 3 3 4 S2 E2 3 4 4 5 5 6 P1 Almost impossible P2 Occasionally * Sequence Of Events P3 Likely I Avoidance or reduction of harm A1 Possible (e.g. low speed) A2 Impossible dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 14 / 19 Safety Hazards Mechanical hazards Electrical hazards Noise hazards I Acceleration/deceleration I Arc I Cavitation I Angular/rotating parts I Electromagnetic I Gas leaking at high speed I Crushing I Electrostatic I Manufacturing process I Cutting I Live parts I Whistling pneumatics I Elastic elements I Overload I Worn parts I Falling objects I Live parts under fault I … I Gravity condition Ergonomic hazards I High pressure I … I Access I Moving elements Thermal hazards I Design/location dials I Stored energy I Explosion I Effort I Vacuum I Flame I Flicker I … I High temperature I Mental over/underload I Low temperature I Posture dr. ir. G.A. Folkertsma, CEng I Radiation FMEA, RMF Autumn 2024 15 / 19 Safety Other risk aspects I cyber security I medical vs. non-medical I top-down vs bottom-up approach I trigger lists I non-intended use I failure modes (single or double fault) to hazards dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 16 / 19 Safety Exercise: robot digger hazard 1. Define one hazard that does not derive from malfunction. 2. Determine the I) severity II) exposure III) probability IV) avoidance. 3. Mitigate the hazard on three of these aspects. After 6 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 17 / 19 Conclusion Questions Question (1 min) What is the difference between RMA and FMEA? Question (1 min) Describe the difference between exposure, probability and avoidance. Question (2 min) What is the RPN and how do you use it? dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 18 / 19 Conclusion Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I RPN 3. Conceptual design & stakeholder analysis I Failure Mode and Effects Analysis (FMEA) 4. Scenarios & Use cases I Safety: Risk Management File 5. Context & external interfaces I Standards 6. Requirements 7. Preliminary design & Architecting Reading for next time: 8. Internal interfaces I ASE Ch 5 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng FMEA, RMF Autumn 2024 19 / 19  Risk management dr. ir. G.A. Folkertsma, CEng Autumn 2024 dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 1 / 24 Context Topics 1. Introduction to Systems Engineering 2. System life cycle & V-model I Types of risk 3. Conceptual design & stakeholder analysis I Risk identification and quantification 4. Scenarios & Use cases I Risk mitigation 5. Context & external interfaces 6. Requirements 7. Preliminary design & Architecting 8. Internal interfaces 9. Budgets & money 10. Decision-making 11. Risk management 12. FMEA, RMF 13. Detailed design & integration 14. Verification 15. Production, use & retirement 16. SEMP dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 2 / 24  Connecting My bike “I have an extra lock” “I have a spare bike” “I have a bike insurance” “I will take the risk.” dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 3 / 24 Connecting Revisit: Decision-making Question (1 min) What is a Pugh matrix? Question (1 min) What is scope creep? Question (2 min) Which step of the decision-making process can be done via design space exploration? dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 4 / 24 Connecting Insurance Some questions: 1. Who has a form of insurance? 2. Why do you have insurance? 3. How do you determine whether to get a certain insurance? dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 5 / 24 Connecting Prior knowledge I What do you already know about Risk management? I Where are we in the design process? Pre-acquisition Acquisition Utilisation Retirement Conceptual design Preliminary design Detailed design Production I Where are we in the V-model? problem spec/design real/verif solution dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 6 / 24 Connecting Risk Management “ The purpose of the Risk Management process is to identify, analyse, treat and monitor the risks continually. ” – INCOSE R =P ·C dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 7 / 24 Risk identification Risk types Other risk Project risks I Safety I Technical I SWOT I Cost I Market I Planning I Legal I Programme (external) I Environmental Course focus: technical, planning and safety. dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 8 / 24 Risk identification Risk relations dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 9 / 24 Risk identification Relation to stakeholders and requirements “ if 〈situation〉, then 〈consequence〉, for 〈stakeholder〉. ” – INCOSE on risk statements dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 10 / 24 Risk identification Quantifying risk I Risk Priority Number (next lecture) I Monetary cost I … “ Risk measurement is not an exact science. […] Low-likelihood/high-severity risks might require treatment […] regardless of the likelihood of occurrence. ” – INCOSE on measuring risk dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 11 / 24 Risk identification Exercise: robot digger risks 1. Identify three risks: a) one technical development risk; b) one technical product risk; c) one product safety risk. 2. For each, formulate clearly and concisely: a) the cause, b) the effect, and c) the likelihood. After 10 min, I will ask for some examples. dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 12 / 24 break Break 15 minutes’ break dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 13 / 24 Risk handling dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 14 / 24 Risk handling Options I) Avoid II) Contain III) Take IV) Delegate Question (3 min) If I am afraid my bike will be stolen, how do I handle that risk with these four options? dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 15 / 24 Risk handling Lowering risk I) Avoid II) Contain III) Take R =P ·C IV) Delegate Question (2 min) How do the four options lower the risk? dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 16 / 24 Risk handling External risks “ Typical issues are time-dependent processes, rigid sequence of activities, one dominant path for success, and little slack. ” – INCOSE How to mitigate? dr. ir. G.A. Folkertsma, CEng Risk management Autumn 2024 17 / 24 R