COSC-30013 Modeling and Simulations 35-53 PDF

A. Simulation Program Selection 1. Advances in the Computer Hardware a. Graphic Capabilities Advances, particularly in the areas of computing power and graphics displays, have had a tremendous impact on the acceptance of simulation modeling. b. Speed Incr...

A. Simulation Program Selection 1. Advances in the Computer Hardware a. Graphic Capabilities Advances, particularly in the areas of computing power and graphics displays, have had a tremendous impact on the acceptance of simulation modeling. b. Speed Increasing the speed tends to decrease the accuracy and conversely increasing accuracy deceases speed. To make the model real-time capable, maintain a balance between speed and accuracy. 2. Software Cost Cost helps to determine if your model is likely to cause an overrun when simulate it in real-time processor. 3. Developers’ Preferences To help decide which approach is more appropriate whether to use general programming language or simulation-specific software package. B. Model Translation Section Content 1. Getting Started The most difficult part of programming the simulation is getting started. At this point in the simulation program, a high-level flow chart should already exist. 2. Version Management To help assist the practitioner in creating and maintaining an organized program file system.  Project subdirectories  Saving simulation programs  File version management techniques  Backing up simulation project files 3. Programming Commenting As with any other computer program, a simulation program can benefit greatly from liberal commenting. Liberal commenting helps not only while you are developing the program but also when you attempt to look at and understand the program years from now. 4. Program Organization You must conduct an endless battle to keep the simulation program as organized as possible. 5. Mnemonic Naming Conversion The mnemonic naming convention should describe the exact nature of the component as well as what type of component it is. You will find that these mnemonic techniques will greatly speed the rate at which the program can be developed. 35  6. Advanced Program Planning Most simulation models usually end up being modified in some manner in order to represent different experimental alternatives. Because the model may eventually be recycled, it is almost always of benefit to attempt to build as much flexibility into the model as is reasonably possible. 7. Multiple Program Development Under some circumstances, you may need the assistance of other developers in order to complete the programming component of a simulation project. C. Ways to Organize Program 1. Graphic Spaghetti Code Spaghetti code was the result of the excess use of “goto” statements in a program. It is impossible to follow individual strands of spaghetti, it was impossible to follow spaghetti code that jumped in and out of this and that part of a program. 2. Subroutine Views In attempting to keep the program organized, you should keep in mind the different levels that can be used to view the program. You should attempt to keep all like components or components that are associated with each other in the same view. 3. Program Hot Keys Hot keys enable the practitioner to associate a particular view of the model with a key on the computer keyboard. So any time the practitioner wants to navigate to a particular area, all that is necessary is to press the key corresponding to that part of the model. 4. Mnemonic Name Conversion If you fail to use mnemonic techniques, you will be continuously forced to refer to other parts of the program. Activities: 1. How important are the factors in selecting program simulation? 2. How versioning program help you in simulation? 3. What is/are purpose/s of techniques to organize the program? 4. How each program organization techniques help you to simulate the model? ***** Note: Additional activities will be given by the faculty-in-charge ***** References: Chung, Christopher A., Simulation and Modeling Handbook: A Practical Approach, 2004, CRC Press https://www.mathworks.com 36  Lesson 7: Verification and Validation Overview: This lesson covers divide-and-conquer and subroutine view approaches, animation, writing to an output file, issues in validation, types of validation and validation data analysis process. Objectives:  To utilize the divide-and-conquer and subroutine approaches in simulation  To enumerate the animation process  To recognize how to write in an output file  To detail the issues in validation  To list the types of validation  To specify the process of data validation 37 A. What is verification? Verification is the process of ensuring that the simulation model operates as intended. This means that it has included all of the intended components in the model and that the model is actually able to run. Verification should be considered as a continuous process rather than a one-shot effort. In order to easily develop and debug the model, you should utilize a divide-and-conquer approach. This is the development of the model in small, easily debugged components rather than an attempt to create the model all at once B. Divide-and-Conquer Approach Simulation programs can benefit from the divide-and-conquer approach. This means that it should break the larger, more detailed system model into a smaller called subroutine view approach; simpler, perhaps higher-level model. The smaller, simpler model will be correspondingly easier to debug. Once the small, simple model includes all of the basic desired components, it can attempt to run the model. Any errors in syntax or variable naming can be more easily addressed. Once the model operates as intended, it can consider making a series of small enhancements to the simple model one by one. The enhancements can take the form of either:  Enhancements to the detail of the existing components of the initial model to represent the actual system  Expansion of the model to include other components that were not previously modeled but need to be modeled in order to represent the actual system C. Animation Animation is perhaps the most effective tool for performing basic verification. Being able to visualize what the program is doing makes it easier to detect errors in the program. Animation can be used for model verification in many different ways: Using different entity pictures for different types of entities For simulation models with different types of customers or different job priorities, the practitioner should employ different types of entity pictures. This can help insure that the different types of entities go to the appropriate part of the model. Following the entities through the system By following the animated entities through the model, it is possible to verify that the model logic is correct. The most common example of this situation is one in which to discover that a decision structure is improperly sending entities to various parts of the model. One common way that this situation manifests itself is when particular queues appear to become unreasonably overloaded. Another symptom is entities disappearing in particular parts of the model and reappearing in others. Changing entity pictures If the entity changes form as it proceeds through the model, the practitioner should also change the picture of the entity. For example, if a roll of raw material is cut into different sections by a machine, the entities that proceed from the machine should have a different entity picture than the roll of raw material. Thus, if an entity proceeds through the model with the wrong picture, it will be obvious that the entity somehow managed to bypass the mid-stream processes. 38  Displaying global variables or entity attribute values Another useful animation capability is the digital or analogue value display. This capability allows the practitioner to see either the instantaneous or the average value for any global variable or entity attribute. Displaying plots of global variables or entity attributes A digital or analogue animation display is limited to displaying a single value at any given moment. When the value changes rapidly, it is difficult to keep track of the past values. More importantly, the single value display cannot provide information on the value trends. When the plots display unusual or unrealistic trends, the practitioner should investigate the model logic. Displaying levels of system statistics Another type of animation that can help the practitioner debug a simulation model is the level. The level is analogous to gauge. If, for example, the practitioner wanted to know the running average utilization for a resource, a level could be used. The lower or empty level would correspond to 0 utilization. Conversely, the upper or full level would correspond to 1.0 or 100% utilization. If a level is utilized in this manner, it is important to remember that the level shows the average, not instantaneous, utilization level. It can be disconcerting to novice practitioners when it appears that a resource is being used, but the level indicates otherwise. D. Writing to an Output File Another potentially useful but more difficult to use verification tool is the use of output files. In contrast to the use of an animation display, output files provide a permanent record for the practitioner. These files may be written in a proprietary format or in ASCII text format. If the program writes files in a proprietary format, then the file will have to be exported by the practitioner into an ASCII file. On the other hand, if the file is in ASCII format, the file can be immediately and easily viewed using any text editor or word processor. 1. Event File List One type of file that the practitioner may use is an event list file. Here, entry is listed according to the system time. The amount of information that is written to the file for each event can make the output very difficult to decipher. This will result in information overload on the part of the practitioner. Because not every single detail and variable value change is necessarily of interest to the practitioner, most simulation software packages will allow you to designate the level of detail that is written to the file. 2. Variable or Attribute Output File This may not necessarily be interested in the change of the system event by event. Perhaps only the changes or final value of a variable or attribute are of specific interest to the practitioner in order to help debug the simulation model. In this case, it may elect to output only the variable or attribute of interest to the file. By looking at the variable or attribute value, this may ensure that the entities are flowing through the model properly. 39 E. Validation Validation is defined as the process of ensuring that a model represents reality at a given confidence level. This means that it will attempt to create a model that is a reasonable representation of the actual system. F. Issues in Validation 1. Assumptions Assumptions may be made because of lack of knowledge. These types of assumptions may be common when attempting to model a system that does not exist or a process that cannot be observed. Certain assumptions may have to be made with respect to the system components, interactions, and input data. Even if some data are available from designers or vendors, the practitioner will have to assume that the data are valid. 2. Simplifications Simplifications will be necessary in order to finish a simulation project in the allotted time. Others will be made because the internal detailed workings of the process are either too complex or thought to be insignificant. Avoiding simplifications caused by time constraints is easier said than done. 3. Oversights If there is any complexity to the system being modeled, it is extremely probable that is going to inadvertently overlook one or more critical system components. If in fact such a component does have a significant impact on the measures of performance, the model will not be able to be validated. Because if you are not aware of the potential problem in the first place, validation problems arising from oversights are much more difficult to handle. Other than conducting as detailed an orientation as possible, there are few defenses against the accidental omission of a particular event or process that may be part of the system. One thing that can do is maintain an ever-increasing checklist of lessons learned. 4. Limitations There are likely to be many limitations with respect to being able to model complex systems. The impact of each of these limitations on validating the model can vary significantly. Generally speaking, these limitations will not be under the direct control of the practitioner. As with model assumptions, it should maintain a list of limitations for the project presentation and report. Limitations may be caused by:  Practitioner  Modeling Software  Data G. Types of Validation 1. Face Validity Face validity is the continuous process of ensuring that the model, at least on the surface, represents reality. Face validity is normally performed with the assistance of domain experts. Face validity is a necessary but not individually sufficient condition for establishing complete model validity. This approach helps: 40   Instill a sense of ownership in the model  Prevent last minute “why didn’t you” question  Reduce the number of project progress inquiries 2. Statistical Validity Statistical validity involves comparison of the simulation model with the actual system. In statistical validity, some output measure of performance is collected. The same system loading conditions observed during the data collection process must be recreated in the simulation model. If statistical validity cannot be established, the model must be examined for flaws and corrected until statistical validity can be achieved. Statistical validity involves an objective and quantitative comparison between the actual system and the simulation model. If there is no statistically significant difference between the data sets, then the model is considered valid. This approach involves:  Validation data collection Validation data collection involves collecting data from both the actual system and the base simulation model that is designed to represent the actual system. Validation data may be based on either individual observations or summary statistics.  Validation data analysis process The validation data analysis process consists of first determining the appropriate statistical comparison of means test to execute. This is performed by determining whether or not one or both of the validation data sets are normal. Activities: 1. Differentiate verification and validation. 2. What are the two principle components of verification? Explain them. 3. How an output file be used to assist in the verification process? 4. Explain how the animation be used in the verification process. 5. How the issues affect the validation of model simulation? 6. Differentiate face validity and statistical validity. 7. What are some possible causes that may prevent a model from being valid? 8. What kind of information must be observed when validation data are being collected from the system? ***** Note: Additional activities will be given by the faculty-in-charge ***** 41 References: Chung, Christopher A., Simulation and Modeling Handbook: A Practical Approach, 2004, CRC Press Pegden, D.C., Shannon, R.E., and Sadowski, R.P. (1995), Introduction to Simulation Using Siman, 2nd ed., McGraw-Hill, New York Johnson, R.A., Freund, J.E., and Miller, I. (1999), Miller and Freund’s Probability and Statistics for Engineers, Pearson Education, New York Law, A.M. and Kelton, D.W. (2000), Simulation Modeling and Analysis, 3rd ed., McGraw-Hill, New York 42  Lesson 8: Experimental Design Overview: This lesson covers issue of simulation model experimental design, factors and levels and alternative experimental designs. Objectives:  To identify the issues on the experimental design  To enumerate the experimental designs  To differentiate factor from level 43 A. Factor and Level Factors can correspond to different types of resources in the system. Levels correspond to the number of each different type of resource to examine. The different possible factors and levels can rapidly increase. Factors are the different variables thought to have an effect on the output performance of the system. These variables are controllable that can vary the levels in both the actual system and the simulation models. B. Two Alternative Experimental Designs 1. Existing System If the actual system is currently in existence, the corresponding model will obviously be one of the other alternatives. 2. No Existing System The same factor and level dangers are present when one is modeling non-existing systems. The danger might actually be greater because even less is known about the alternative models in comparison to situations where at least one base model already exists. The two alternative experimental design approach should really be of limited interest to the practitioner. The reason for this is that the identification of additional models and the analysis of those models is minor in comparison to the overall effort required for the entire project. This means that with a little bit of extra effort, it can evaluate a number of different alternatives. The result will be a much more robust analysis of the system. To determine which models to analyze, it will have to formally design an experiment. In any two-alternative comparison, the statistical comparison techniques are relatively simple. These comparisons primarily involve some sort of t- test or rank sum test. C. One Factor Experimental Design It is efficient for determining if a variable has any effect. Results are easy to interpret and analyse. It is adequate for some theory testing and useful for applied comparisons. Often, we wish to investigate the effect of a factor on a response. We then carry out an experiment where the levels of the factor are varied. D. Two Factor Experimental Design An experimental design in which data is collected for all possible combinations of the levels of the two factors of interest. If equal sample sizes are taken for each of the possible factor combinations then the design is a balanced two factorial design. E. Multifactor Experimental Design One can investigate interactions. Adding factors decreases variability thus increasing statistical sensitivity. It increases generalizability without decreasing precision. In order to study the effects of two or more factors on a response variable, factorial design are usually used. By following these designs, all possible combinations of the level of the 44  factors are investigated. The factorial designs are ideal designs for studying the interaction effect between factors. F. 2k Experimental Designs Another commonly accepted means of reducing the number of alternatives is to reduce the number of levels in each factor to two. These levels are referred to as the low level and the high level. The use of only two levels results in 2 raised to the number of factor alternatives. This approach is frequently referred to as a 2k experimental design. In determining the alternatives for a 2k experimental design, the concept of a binary tree can be used. This consists of a series of branches progressing from left to right. Two branches are in each set of the series. The first level will have a minus and a plus branch. The second level will have a minus and a plus branch for the first level minus branch. Similarly, the second level will also have a minus and a plus branch from the first level plus branch. This means that the second level will have a total of four branches. The third level will have a minus and a plus branch coming out of each of the four branches from the second level for a total of eight different branches. Activities: 1. Give a simple system that will demonstrate the factors and levels. Explain. 2. Differentiate the two alternative experimental designs. 3. What are the characteristics of one factor, two factor and multifactor experimental designs? 4. Provide a system that will show the 2k experimental design. ***** Note: Additional activities will be given by the faculty-in-charge ***** References: Chung, Christopher A., Simulation and Modeling Handbook: A Practical Approach, 2004, CRC Press https://psychology.illinoisstate.edu https://math.montana,edu https://link.springer.com 45  Lesson 9: Analysis Overview: This lesson covers statistical analysis of terminating and nonterminating system. Objectives:  To differentiate the statistical analysis of terminating and nonterminating system  To utilize the statistical analysis in simulation 46 The analysis phase is the most statistically challenging project phase for the simulation. A review of basic statistics before we undertake the analysis phase may be beneficial. However, the ready availability of statistical functions in Excel and the statistical analysis features found in most dedicated simulation software packages help insulate from getting unnecessarily bogged down in this phase. A. Terminating System Analysis Analysis Process 1. Replication Analysis Replication analysis involves the determination of the number of simulation replications or runs that are required to analyze statistically the differences between the simulation models. The replication analysis process begins with selecting an initial number of replications. Summary statistics from this initial set of replications are then used to calculate whether or not additional replications are required at a particular level of confidence. If more replications are required, then the practitioner needs to run additional replications and recalculate the summary statistics and replication formulas for the process. 2. Production Simulation Runs Production simulation runs involve completing the number of simulation runs determined through the replication analysis. Once we know the standard error, we can use this information to calculate how many replications we need to run. However in order to do this, we must select a level of precision or error that is suitable for our study. There are two general approaches to determine objectively the level of precision that we are willing to accept. The first approach is based on an absolute comparison of the standard error to a particular tolerance level. The second approach is based on a relative value of the standard error in comparison to the sample mean. 3. Statistical Analysis of the Simulation Run Results The statistical analysis of the simulation runs will interpret the level of differences between the experimental design alternatives. In a very simple simulation project, it will have only two different models to compare. These will normally be the base model and the model with the alternative resource or operating policy. The statistical analysis involved in two-model comparisons is relatively simple. In a more complicated simulation project, the practitioner could easily have many more than two models to compare. With multiple models, it is possible to use the same general type of statistical analysis approaches as with only two models. However, more robust statistical methods are generally in order. To organize the statistical analysis process, we can categorize the analysis approaches into either: Simple two-model comparisons Three-model or more comparisons 47  4. Economic Analysis of Statistic Analysis The economic analysis puts the statistical analysis results in perspective from an engineering economy or business perspective. B. Nonterminating System Analysis Analysis Process 1. Starting Condition Starting conditions refer to simulation run initialization procedures. The system will first have an initial transient state before it becomes balanced and reaches the system’s steady state. Even though the initial transient state may not be of the greatest importance to the practitioner, it will still need to be modeled. The practitioner can choose either of two different approaches:  Begin with the system empty  Begin with the system loaded 2. Determining Steady State Determining the steady state involves eliminating the non-representative transient output performance data that are present at the beginning of the simulation run. This ensures that the remaining steady-state data are representative of the actual system performance. The statistical comparison between nonterminating systems must be conducted only with data from the models in steady state. So, it can conduct any statistical comparison, the initial transient state, if one exists, must be removed. During the simulation run, the initial transient state can be typically identified as a series of observations with continuously increasing output measures of performance values. For example, the initial entities in an unloaded system will have much shorter system times than entities when the system is loaded and operating in steady state. There are a number of ways to approximate when the output measure of performance exits the initial transient state and enters the steady state. These include: Graphic approach Linear regression 3. Addressing Autocorrelation Autocorrelation is the correlation between successive performance observations. This must be addressed so that the variance in the output data is not underestimated. This is the tendency of subsequent observations of output measures of performance to be related to each other. This occurs when, for example, the system time of one entity is correlated with the following entity. If the first entity experienced a high system time, the next entity may also experience a high system time. Similarly, if the first entity has a low system time, the next entity will also have a low system time. When this occurs, the system may be suffering from autocorrelation. 48  4. Length of Replication Because nonterminating systems use a single run for data analysis, we must determine how long the run should be. The length of the simulation must be long enough to remove the transient phase and have at least 10 batches of the required batch size. 5. Batching Method It is necessary to split the single long run into batches to facilitate the statistical comparison between nonterminating systems. Instead of running several replications of relatively short durations as with the terminating system model, we run the nonterminating system model for a single replication over a much longer time period and cut the run into separate batches. The batches are then considered as individual replications. The same comparison techniques can then be used for different alternatives. The steps in implementing this method are: Identify the non-significant correlation lag size Make a batch ten times the size of the lag Make the steady-state replication run length ten batches long Activities: 1. Name a terminating system. Analyse the system using the process. 2. Give a sample nonterminating system. Using the process, analyse the system. ***** Note: Additional activities will be given by the faculty-in-charge ***** References: Chung, Christopher A., Simulation and Modeling Handbook: A Practical Approach, 2004, CRC Press Johnson, R.A., Freund, J.E., and Miller, I. (1999), Miller and Freund’s Probability and Statistics for Engineers, Pearson Education, New York. Law, A.M. and Kelton, D.W. (2000), Simulation Modeling and Analysis, 3rd ed., McGraw-Hill, New York. 49  Lesson 10: Project Reports and Presentation Overview: This lesson covers written report guidelines, executive summaries, equations, screen captures and other graphics, presentation guidelines, presentation media, electronic presentation guidelines, electronic software presentation issues and actual presentation. Objectives:  To know the guidelines in presentation  To identify presentation media to be used  To construct an executive summary 50 A. Written Report Guidelines Generally, both the report and the presentation should follow the same format that was used as a guide for the simulation project. These include: Problem statement Project planning System definition Input data collection and analysis Model formulation Model translation Verification Validation Experimentation and analysis Recommendations and conclusions B. Executive Summaries It is a fact of simulation analysis that a good number of the individuals who need to be briefed on the project will not have as technical a background as most practitioners. Others will not necessarily be interested in the level of detail available in a comprehensive report. For these reasons, the practitioner should consider including an executive summary. The executive summary should not be more than two or three pages and should contain condensed descriptions from the report, including: Project objectives Results Recommendations and conclusions C. Equations A professionally prepared report should include the equations that were used during the analysis phase of the report. The existence of equation-editing software like that included in the Microsoft Office software suite make this previously painful process much less tedious. It is actually to the advantage of the practitioner to use an equation editor for the report because the equations can be then copied and pasted into the electronic presentation. Furthermore, once created all of the basic simulation equations, they can be reused in future simulation studies. D. Screen Captures and Graphics Another characteristic of a professional report is the inclusion of system photographs and simulation software screen captures. Photographs are invaluable in describing the operation of specific system processes. Simulation software screen captures can include the simulation model flow charts, the animation, and statistical plots or graphics. Photographs are usually imported into word processor documents, including those in Microsoft Word, by importing the graphic file. However, two distinct methods may be used to bring screen captures into the document. E. Presentation Guidelines Although the electronic presentation should follow the same basic simulation project process as the report, the practitioner will have to individually determine the level of 51  detail that should be provided during the presentation. Factors that will contribute to this decision include: Objective of the presentation Time for the presentation Technical level of the audience F. Presentation Media There are several choices of presentation media to select from. With the increasing availability of LCD computer projectors, the electronic presentation has become the preferred choice with major corporations and professional conferences. The electronic presentation holds the distinct advantage of making last minute corrections or updates possible. However, the use of LCD computer projectors does involve a few disadvantages including increased preparation time and computer compatibility issues. G. Electronic Presentation Software Issues The proper use of electronic presentation software can make a significant difference in the successful communication of the simulation project results. In this section, we discuss: Presentation masters Use of colors Use of multimedia effects Speaker’s notes Use of presentation handouts H. Actual Presentation It addresses issues associated with the actual presentation that will help the presenter deliver a professional standard presentation. These issues include: Rehearsal Dress Positioning Posture Presentation insurance Activities: 1. Are the written report guidelines needed in all reports? Explain. 2. How important is the executive summary? 3. How are you going to present equations in your written report? 4. What are the techniques to capture screen and graphics? Explain them. 5. What technique can you use to help avoid looking at the projected image on the screen? 6. Why is it necessary to allot an additional time for an electronic presentation? 52  7. Explain each issues in the actual presentation. How they are important to resolve them? ***** Note: Additional activities will be given by the faculty-in-charge ***** References: Chung, Christopher A., Simulation and Modeling Handbook: A Practical Approach, 2004, CRC Press 53