Manual on Training Needs Assessment PDF

Project  on  Improvement  of  Local  Administration  in  Cambodia Manual  on   Training   Needs  Assessment         What  is  Training  Needs  Assessment?   Five  Steps  of...

Project  on  Improvement  of  Local  Administration  in  Cambodia Manual  on   Training   Needs  Assessment         What  is  Training  Needs  Assessment?   Five  Steps  of  Training  Needs   Assessment   Step  1:  Identify  Problem  and  Needs   Step  2:  Determine  Design  of  Needs   Assessment   Step  3:  Collect  Data   Step  4:  Analyze  Data   Step  5:  Provide  Feedback          Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Table  of  Contents     Glossary........................................................................... 3 1. What is Training Needs Assessment?............................ 4 2. Five Steps of Traning Needs Assessment....................... 7 3. Step 1: Identify Problem and Needs............................... 8 4. Step 2: Determine Design of Needs Assessment............. 9 5. Step 3: Collect Data.................................................... 21 6. Step 4: Analyze Data................................................... 25 7. Step 5: Provide Feedback............................................ 36 References...................................................................... 37 Appendices   Appendix 1: Questionnaire for Training Needs Assessment Appendix 2: Guide Questions for Training Needs Assessment 2      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Glossary An observable behavior supported by specific knowledge, skills, and attitudes. Competency Each competency has a specific result or output. A procedure for organizing narrative and qualitative data into emerging themes Content Analysis and concepts. Usually associated with a quantitative form of analysis in which the themes are counted or measured. A cost-benefit analysis completed prior to conducting training. It is an estimate Feasibility of the cost of the training weighed against the possible benefits that could be Analysis achieved if training were conducted Also called performance analysis; identifies the difference between current Gap Analysis performance and the desired performance. The process of asking questions to experts or performers to identify training Interview needs. The process of identifying all the parts of a specific job; conducted before a task Job Analysis analysis. Learning Describes a specific behavior, conditions, level of achievement and is written Objectives from the learner’s point of view. Gathering of information about a specific work need that can be resolved by Needs training. The types of needs assessment include performance analysis, target Assessment population analysis, sorting training needs and wants, job analysis, and task analysis. Discovers training needs that are related to the organization’s work. Training is Needs versus linked to the final outcome and providing appropriate training will benefit the Wants Analysis individual as well as the organization. Performance Also known as gap analysis. Performance analysis looks at an official’s current Analysis performance and identifies whether the official is performing as desired Performance A difference with a negative connotation, implying that the official is not meeting Deficiency a known standard for performance. Project Team GDLA Task Force members and JICA experts. Target The individual or group involved in a needs assessment or training program. Population Task Analysis Finds the best method and sequence of steps to complete a specific task. Trainer A term used in a corporate setting for a teacher. Also instructor. Training Needs The method of determining if a training need exists and, if it does, what training Assessment is required to fill the gap. 3      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A       What  is  Training  Needs  Assessment?   1       Planning is the first step of the   training management cycle. At   the planning stage, the steps are   divided into two: i) Training   Needs Assessment (TNA) and ii)   Training Planning. This manual   takes up TNA.     Definition     “Training Needs Assessment” (TNA) is the method of determining if a training need exists and, if it does, what training is required to fill the gap. TNA seeks to identify accurately the levels of the present situation in the target surveys, interview, observation, secondary data and/or workshop. The gap between the present status and desired status may indicate problems that in turn can be translated into a training need. Training  Needs  =  Desired  Capability  –  Current  Capability  of  the  Participants   Training can reduce, if not eliminate, the gap, by equipping the participants with knowledge and skills and by encouraging them to build and enhance their capabilities. The data on the present status are vital to the evaluation or impact survey in the latter part of the training cycle. These shall serve as the baseline data. The following are some techniques for acquiring such data. These may be applied independently or in combination. 4      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A       TNA is also the process of collecting information about an expressed or implied organizational need that could be met by conducting training. The need can be a performance that does not meet the current standard. It means that there is a prescribed or best way of doing a task and that variance from it is creating a problem. The TNA process helps the trainer and the person requesting training to specify the training need or performance deficiency. Assessments can be formal (using survey and interview techniques) or informal (asking some questions of those involved). (Source:  Jean  Barbazette,  2006,  Training  Needs  Assessment:  Methods,  Tools  and  Techniques)   Why do we need training? Because training is a means to ensure that government officials have the knowledge and right skills to be able to do their work effectively and competently. Training may be needed when there is a gap between the desired performance, and the current performance, and the reason for that gap is lack of skill or knowledge. Training may only be able to resolve part of the problem. Thus we need to analyze the problem and find out whether training will be able to resolve it. If training is necessary, we also need to define the objective of the training and how it will help the staff member(s) become more effective. This process is called a Training Needs Assessment shown above or Training Needs Analysis. It is important to note that, despite many reasons to conduct training shown above, training may sometimes not be the only solution to a problem. There are many other means that impact on someone’s ability to do their work, as pointed out in the “Report on Training Needs Assessment” by PILAC. The following are other examples.  Lack of skills or knowledge, or experience  Not having the right equipment or resource  Not being encouraged by managers and colleagues to do the right thing  There are no standards or expectations that are set and communicated  Bad workplace morale or conditions (Source:  MOI/DOLA,  2004,  Training  Needs  Assessment)   5      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Why do we need a Training Needs Assessment? First, identify dissatisfaction with the current situation and desire for change as similarities among the requests. Each request implies that a gap or discrepancy exists between what is and what could be or should be. A learning or performance gap between the current and desired condition is called a need. TNA aims at the following situations.  Solving a current problem  Avoiding a past or current problem  Creating or taking advantage of a future opportunity  Providing learning, development or growth The purpose of TNA is to answer some familiar questions: why, who, how, what, and when. The following are descriptions of the questions and what analysis can be done to answer them. Figure  1:  The  Purpose  of  TNA   Why       conduct  the  training:  to  tie  the  performance  deficiency  to  a  working  need  and  be   sure  the  benefits  of  conducting  the  training  are  greater  than  the  problems  being   caused  by  the  performance  deficiency.  Conduct  two  types  of  analysis   to   answer   this  question:  (1)  needs  versus  wants  analysis  and  (2)  feasibility  analysis.   Who         is   involved   in   the   training:   involve   appropriate   parties   to   solve   the   deficiency.   Conduct  a  target  population  analysis  to  learn  as  much  as  possible  about  those   involved  in  the  deficiency  and  how  to  customize  a  training  program  to  capture   their  interest.   How       can   the   performance   deficiency   be   fixed:   training   can   fix   the   performance   deficiency  or  suggest  other  remediation  if  training  is  not  appropriate?  Conduct  a   performance  analysis  to  identify  what  skill  deficiency  is  to  be  fixed  by  a  training   remedy.     What       is   the   best   way   to   perform:   there   is   a   better   or   preferred   way   to   do   a   task   to   get   the   best   results.   Are   job   performance   standards   set   by   the   organization?   Are   there   governmental   regulations   to   consider   when   completing   the   task   in   a   required  manner?  Conduct  a  task  analysis  to  identify  the  best  way  to  perform.   When     will  training  take  place:  the  best  timing  to  deliver  training  because  attendance  at   training   can   be   impacted   by   work   cycles,   holidays,   and   so   forth.   Conduct   a   contextual  analysis  to  answer  logistics  questions.   (Source:  Jean  Barbazette,  2006,  Training  Needs  Assessment:  Methods,  Tools  and  Techniques)   6      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Five  Steps  of  Training  Needs  Assessment   2     Five Steps: The processes of Training Needs Assessment can be divided into five steps: i) identify problem and needs; ii) determine design of needs assessment; iii) collect data; iv) analyze data; and v) provide feedback.                       7      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     3   Step  1:  Identify  Problem  and  Needs   The first step in TNA is to identify problems and needs. Before TNA is conducted, it should be probed whether training is needed. In the public sector, it is important to identify organizational context in such aspects as policy, goal, roles and responsibilities. Realizing the policy direction of the organization, performance analysis known as “gap” analysis is conducted to look at an official’s current working performance and knowledge and identify whether an official is performing as desired based on given roles and responsibilities. Then the more explicit the standard for current performance and knowledge, the easier it will be to describe the gap in performance or knowledge deficiency. During the preliminary study of PILAC, the current problem was identified PILAC   as follows: “Training officials is urgently needed to implement D&D reform and improve local administration, because the efficient and effective implementation of the D&D Framework requires deep understanding of the Framework among public officials at the national, provincial, and district levels who are in charge of local administration.” After identifying problems and needs, set up overall objectives for a training course. In PILAC, the following objectives of the training course were already set up through discussion between JICA and the Ministry of Interior (MOI) prior to project implementation, i.e., during the preliminary study.  Public officials improve their understanding of D&D policy.  Public officials improve their skills to apply D&D policy to their daily duties and functions assigned. 8      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     4   Step  2:  Determine  Design  of  Needs  Analysis     The second step in TNA is to determine the following: i) target groups to be trained; ii) interviewees; iii) survey methods; iv) survey plan including schedule to be conducted TNA and persons in charge of TNA. Those items become the basis for a training course designer to either create a new training course, identify an existing one that can fulfill the need, or obtain one externally. Sometimes it may be impossible to fulfill the need, but that is not the decision of the person conducting TNA. Actually, knowing exactly what elements of information are required can serve as a guide –a road map- for your analysis. The survey must clearly define the target group of the training, i.e., target Target     population. Although no strict rules for defining exist, the target population Group   must be defined in line with the objectives of TNA. The survey should produce the following elements in its report: training subject(s); importance of the training; time requirements; current target group; potential target group; frequency of training; and required outputs of the training. In PILAC, target groups and the number to be trained were also determined PILAC   during the preliminary study of PILAC through discussion between MOI and JICA. Top managers in local administration of MOI and officials of the General Department of Local Administration (GDLA) were primarily identified as target group. During TNA, the selection criteria of trainees were to be set up. The details determined by MOI/JICA are as follows: 9      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A       Table  1:  Training  Course  of  PILAC  Phase  1   i Once a target group for the training, i.e., target population, is identified, Selection  of   interviewees for the survey are selected. It is likely that all the officials of the Interviewees   target group cannot be interviewed due to time constraints. Thus, sampling of the target population which will be addressed in the next section should be used. In TNA conducted in PILAC Phase 1, MOI officials who were involved in D&D PILAC   policy and local administration of GDLA and local administration in Phnom Penh Municipality and Kampong Cham Province from five model provinces were selected for the survey. The time factor was crucial in deciding how the target officials should be sampled in the five model provinces. The survey was also intended to compare the Municipality and the Province and see the differences and similarities between them that apply to all five model provinces and cities. 10      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Survey   Methods   The following figure indicates types of needs analysis and contents. It is suggested that those analyses be used in combination depending on the Types  of   Needs   needs, time availability and manpower. Assessment     Figure  2:  Types  of  Needs  Assessment   Types  of  Needs  Assessment     Type  of  Needs  Analysis           What  the  Analysis  Answers       Performance  analysis  or  gap  analysis    Is  this  issue  a  skill/knowledge  deficiency?      How  can  the  deficiency  be  addressed?      Is  training  the  appropriate  way  to  fix  this  deficiency?     Feasibility  analysis    Why  should  this  training  be  done?      Is  the  benefit  of  training  greater  than  the  cost  of  the   current  deficiency?     Needs  versus  wants  analysis      Why  should  this  training  be  done?      Is  the  deficiency  tied  to  a  need?     Goal  analysis      What  is  the  specific  behavior  improvement  behind  a   vague  desire?     Job/task  analysis      What  is  the  best  and  correct  way  to  do  this  work?      How  can  this  job  and  task  be  broken  down  into       teachable  parts?     Target  group  analysis      Who  is  the  trainee  for  this  training?      What  is  known  about  them  to  help  design  and   customize  this  training          What  other  groups  mgiht  benefit  from  training?     Contextual  analysis      When  will  the  training  be  presented?        What  are  the  other  requirements  to  deliver  the  training   successfully?                 (Source:  Jean  Barbazette,  2006,  Training  Needs  Assessment:  Methods,  Tools  and  Techniques)   11      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Data collection and analysis are essential parts of needs assessment. The Survey   Methods   following table describes the most commonly used methods of data collection. TNA is optimized when a combination of data collection methods is used to analyze quantitative and qualitative data. Regardless of which methods are used to collect and analyze data, it is important to consider the reliability, validity and trustworthiness of the data.   Table  2:  Survey  Methods     Method   Concept    Quantitative  research  method  commonly  employed  in  survey  research  to  ensure  that   each  interviewee  is  presented  with  the  same  questions  in  the  same  order  and  that   answers  can  be  reliably  aggregated  and  that  comparisons  can  be  made  with   Structured   confidence  between  sub-­‐groups  or  between  different  survey  periods.     Interview    Interviewers  read  the  questions  exactly  as  they  appear  on  the  survey  questionnaire.   The  choice  of  answers  to  the  questions  is  often  fixed  (close-­‐ended)  in  advance,  though   open-­‐ended  questions  can  also  be  included  within  a  structured  interview.    Unlike  the  structured  interview,  more  general  questions  or  topics.  Relevant  topics  are   initially  identified  and  the  possible  relationship  between  these  topics  and  the  issues   become  the  basis  for  more  specific  questions  which  do  not  need  to  be  prepared  in   Semi-­‐Structured   advance  allowing  both  the  interviewer  and  the  person  being  interviewed  the  flexibility   Interview   to  probe  for  details  or  discuss  issues.    New  questions  can  be  brought  up  during  the  interview  as  a  result  of  what  the   interviewee  says,  so  the  interview  flows  more  like  a  conversation.    Observation  of  working  environment  and  performance  of  officials  (office  materials,   Observation   communication  tool,  IT  system,  means  of  circulating  the  information)    A  questionnaire  is  a  survey  instrument  consisting  of  a  series  of  questions  and  other   Questionnaire   prompts  for  the  purpose  of  gathering  information  from  respondents.  They  are  often   Survey*   designed  for  statistical  analysis  of  the  responses.      Qualitative  research  method  whose  purpose  is  to  obtain  in-­‐depth  information  on   ideas  and  perceptions  of  a  group  and  also  to  be  more  than  a  question-­‐answer   Focus  Group   interaction.   discussion    A  relatively  small  meeting  (generally  six  to  twelve  participants)  convened  for  a  specific   purpose  under  the  direction  of  a  facilitator,  during  which  group  members  talk  freely   and  spontaneously  about  a  certain  topic.    An  educational  seminar  or  series  of  meetings  emphasizing  interaction  and  exchanged   of  information  among  a  usually  small  number  of  participants  developing  skill  or   Workshop   common  understanding  through  some  types  of  application      Discussion  on  verification  of  identified  staff  training  needs  in  the  returned  TNA   questionnaires  and  interview  results     *Details are shown below 12      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     The following table shows benefits and weaknesses of survey methods. Selection of the survey methods should be done according to availability of time and manpower for conducting the survey. It is recommended to combine several methods to quantitatively and qualitatively analyze the survey results.           Table  3:  Benefit  and  Weakness  of  Survey  Methods     METHOD   BENEFIT   WEAKNESS   WHEN  TO  USE   May  be  out  of  date   Factual  information   May  be  inaccurate  or   When  you  need  factual   Review  of   Objective   inconsistent   information  about   References   Can  collect  a  lot  if  you   Need  cooperation  of   performance   have  resources   others  to  obtain   information     May  not  get  important   Simple   Have  to  know  much   information   Quick   about  your  topic  first   Questionnaire   People  may  not  send   Easy     Combine  with  other   Survey   back  survey   Can  collate  a  lot  of   processes  to  encourage   May  be  hard  to   data   response   understand  responses   Obtain  information   Takes  time  of  yourself   about  attitudes   When  you  know  little   and  others   Obtain  a  lot  of   about  the  topic  or  area     More  difficult  to  organize   Interviews   qualitative  data   When  the  training  is   May  be  shy  to  respond   Can  have  greater   about  something   depending  on     understanding  of   complicated   interviewer   issues     Can  be  easy  and     People  may  be  shy  to  be   When  the  training  is   Focus   quick   honest  in  group   impacted  by  team  work     Group   Can  understand   People  may  dominate   When  there  is  not  much   Discussion   responses  more   discussion   time  for  other  methods   easily   Can  take  observer  a  lot  of   Does  not  interrupt   When  the  training  is   time   work   about  simple  skills     Observation   Need  time  to  collate   Can  be  more  reliable   When  you  know  about   Need  to  know  what  you   than  other  sources   the  topic  yourself   are  looking  for   (Source:  MOI/DOLA,  2004,  Training  Needs  Assessment,  Phnom  Penh)     13      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Along with the selection of interviewees, survey methods are also selected Sampling   considering availability of time and manpower. The following are sampling Methods   methods to be used for social survey. However, to make it simple, it is recommended that random sampling and stratified sampling methods be used for TNA in PILAC. Table  4:  Sampling  Methods   Method   Concept    Purest  form  of  probability  sampling.      Each  member  of  the  population  has  an  equal  and  known  chance  of   Random   being  selected.  When  there  are  very  large  populations,  it  is  often   sampling   difficult  or  impossible  to  identify  every  member  of  the  population,  so   the  pool  of  available  subjects  becomes  biased.    Often  used  instead  of  random  sampling.      It  is  also  called  an  Nth  name  selection  technique.  After  the  required   sample  size  has  been  calculated,  every  Nth  record  is  selected  from  a   Systematic   list  of  the  target  population  on  members.  As  long  as  the  list  does  not   sampling   contain  any  hidden  order,  this  sampling  method  is  as  good  as  the   random  sampling  method.  Its  only  advantage  over  the  random   sampling  technique  is  simplicity.      Commonly  used  method  that  is  superior  to  random  sampling  because   it  reduces  sampling  error.  A  stratum  is  a  subset  of  the  population  that   Stratified   shares  at  least  one  common  characteristic.  The  surveyors  identify  the   sampling   relevant  stratums  and  their  actual  representation  in  the  population.        Stratified  sampling  is  often  used  when  one  or  more  of  the  stratums  in   the  population  have  a  low  incidence  relative  to  the  other  stratums.   (Source:  Access  to  http://www.statpac.com/surveys/sampling.htm,  July  31,  2007)   14      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Questionnaire survey design proceeds in an orderly and specific manner. Process  of   Questionnaire   Each item in the flow chart shown below depends upon the successful Survey     completion of all the previous items. Therefore, it is important not to skip a single step. Notice that there are two feedback loops in the flow chart to allow revisions to the methodology and instruments. Figure  3:  Steps  for  Conducting  a  TNA         By using a systematic approach, you can ensure that gaps in performance are identified correctly. Usually only those gaps caused by lack of knowledge or skills can be improved through training. Performance deficiencies that occur because of lack of motivation, environmental problems, or systems issues require non-training interventions such as changes in the selection process, the performance appraisal process, or the reward system. 15      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     In creating questionnaires, it is important to consider the type, content, Creating  a   Questionnaire   wording, and order of the questions that they include. For the type of questions, there are two types of questions to be asked: Type  of   Questions   close-ended questions and open-ended questions. Closed-ended questions limit respondents' answers to the survey. The participants are allowed to choose from either a pre-existing set of dichotomous answers, such as yes/no, or multiple choice with an option for "other" to be filled in, or ranking scale response options. The most common of the ranking scale questions is called the scale question. This kind of question asks the respondents to look at a statement and then "rank" this statement according to the degree to which they agree (strongly agree; somewhat agree; no opinion; somewhat disagree; strongly disagree). Open-ended questions do not give respondents answers to choose from, but rather are phrased so that the respondents are encouraged to explain their answers and reactions to the question with a sentence, a paragraph, or even a page or more, depending on the survey. If you wish to find information on the same topic, but would like to find out what respondents would come up with on their own, you might choose an open-ended question like "What kinds of training do you want to attend?" rather than the scale question. However, keep in mind that you do not have to use close-ended or open-ended questions exclusively. Many researchers use a combination of closed and open questions; often researchers use close-ended questions at the beginning of their survey, and then allow for more expansive answers once the respondent has some background on the issue and is "warmed-up." When considering the content of your questionnaire, the most important Content  of   Questionnaire   consideration is whether the content of the questions will elicit the kinds of questions necessary to answer your initial research question. You can gauge the appropriateness of your questions by pre-testing your survey, but you should also consider the following questions as you create your initial questionnaire. 16      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Figure  4:  Wording  of  the  Questions   Wording  of    Does  your  choice  of  open  or  close-­‐ended  questions  lead  to  the  types  of  answers  you   the   Questions   would  like  to  get  from  your  respondents?    Is  every  question  in  your  survey  integral  to  your  intent?  Superfluous  questions  that   have  already  been  addressed  or  are  not  relevant  to  your  study  will  waste  the  time  of   both  the  respondents  and  the  researcher.    Does  one  topic  warrant  more  than  one  question?    Do  you  give  enough  prior  information/context  for  each  set  of  questions?  Sometimes   lead-­‐in  questions  are  useful  to  help  the  respondent  become  familiar  and  comfortable   with  the  topic.    Are  the  questions  both  general  enough  (they  are  both  standardized  and  relevant  to   your  entire  sample),  and  specific  enough  (avoid  vague  generalizations  and   ambiguousness)?    Is  each  question  as  succinct  as  it  can  be  without  leaving  out  essential  information?      Finally,  and  most  importantly,  write  a  survey  that  you  would  be  willing  to  answer   yourself,  and  be  polite,  courteous,  and  sensitive.  Thank  the  responder  for  participating   both  at  the  beginning  and  the  end  of  the  survey.     (Source:  Kavita  Gupta,  2007,  A  Practical  Guide  to  Needs  Assessment)       To write effective questions, researchers need to keep in mind these four   important techniques: directness, simplicity, specificity, and discreteness.     Figure  5:  Order  of  the  Questions   Order  of  the    Questions  should  be  written  in  a  straightforward,  direct  language  that  is  not  caught   Questions   up  in  complex  rhetoric  or  syntax,  or  in  a  discipline's  slang  or  lingo.  Questions  should   be  specifically  tailored  for  a  group  of  respondents.      Questions  should  be  kept  short  and  simple.  Respondents  should  not  be  expected  to   learn  new,  complex  information  in  order  to  answer  questions.      Specific   questions   are   for   the   most   part   better   than   general   ones.   Research   shows   that,   the   more   general   a   question   is,   the   wider   the   range   of   interpretation   among   respondents.   To   keep   specific   questions   brief,   researchers   can   sometimes   use   longer   introductions   that   make   the   context,   background,   and   purpose   of   the   survey   clear   so   that  this  information  is  not  necessary  to  include  in  the  actual  questions.      Avoid   questions   that   are   overly   personal   or   direct,   especially   when   dealing   with   sensitive  issues.       Kavita  Gupta,  2007,  A  Practical  Guide  to  Needs  Assessment)   (Source:     17          Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Although there are no general rules for ordering survey questions, below are Setting  Up  a   Questionnaire   a few suggestions that researchers can follow when setting up a questionnaire. Figure  6:  Points  of  Setting  Up  a  Questionnaire    Use  warm-­‐up  questions.  Easier  questions  will  ease  the  respondent  into  the  survey   and  will  set  the  tone  and  the  topic  of  the  survey.      Sensitive  questions  should  not  appear  at  the  beginning  of  the  survey.  Try  to  put   the  respondent  at  ease  before  addressing  uncomfortable  issues.  You  may  also   prepare  the  reader  for  these  sensitive  questions  with  some  sort  of  written  preface.      Consider  transition  questions  that  make  logical  links.      Try  not  to  mix  topics.  Topics  can  easily  be  placed  into  "sets"  of  questions.      Try  not  to  put  the  most  important  questions  last.  Respondents  may  become  bored   or  tired  before  they  get  to  the  end  of  the  survey.      Be  careful  with  contingency  questions  ("If  you  answered  yes  to  the  previous   question  .  .  .  etc.").      If  you  are  using  a  combination  of  open  and  close-­‐ended  questions,  try  not  to  start   your  survey  with  open-­‐ended  questions.  Respondents  will  be  more  likely  to  answer   the  survey  if  they  are  allowed  the  ease  of  closed-­‐questions  first.       (Source:  Kavita  Gupta,  2007,  A  Practical  Guide  to  Needs  Assessment)           18      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     Figure  7:  Types  of  Question-­‐and-­‐Answer  Format  for  Questionnaire        Objective Response: These are questions that can be answered with a simple yes or no   Example:       Q1)  Have  you  ever  attended  any  training  course  on  D&D?     _______Yes______  No      Multiple Choice: Each question provides several possible answers from which to choose Example:   Q2)  How  many  times  have  you  attended  training  course?     ____1  time,  _____2  times,  _____3  times,  _____4  times      Scale Response: Each question has a descriptive and/or numerical scale for responding Example:   Q3)  How  important  do  you  feel  it  is  for  newly  recruited  officials  to  participate  in  an   orientation  program?      Very  important      Somewhat  important    Neither  important  nor  unimportant      Somewhat  Unimportant      Not  important         In the process of planning the survey, a Survey Plan, which will be attached   Survey     to formal letter for interviewees, will be formulated and will be distributed to   Plan   concerned agencies and interviewees. The outline of the Survey Plan   includes the following: i) background; ii) objectives; iii) schedule; iv) survey team; and v) methodology (data collection and analysis). Once you have constructed a questionnaire, you will need to make a plan that outlines how and to whom you will administer it. A number of options are available in order to find a relevant sample group amongst your survey population. In addition, various considerations are involved with administering the survey itself. 19      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     In PILAC, the following methods were used for data collection. A stratified PILAC   sampling method was used by selecting respondents. The following analyses were used.  Review of Secondary Data/Information Review of i) existing data and information on the decentralization process, regulations (PRAKAS and LAW), and policy paper (Strategy on D&D) in the realms of training materials produced by the MOI; and ii) research papers produced by research institutes and other donors, especially UNDP and GTZ, on capacity development for the MOI officials and commune council members in the area of D&D.    Individual Survey (See Appendix 1) The TNA Questionnaire was designed to make it both simple and relatively quick to complete by providing boxes to tick in most categories rather than requiring written responses. Such design increased staff participation. The TNA Questionnaire covers the following: i) individual job description; ii) level of knowledge on D&D; iii) training attended; iv) training needs for specific knowledge and skills; and v) problems identified in terms of performing the duties more effectively. Filled copies of the questionnaire were collected after group interviews.  Group Discussion (See Appendix 2) In addition to the TNA Questionnaire, a total of 39 group discussions on a variety of training needs were held with officials. A document titled “Guide Questions” was prepared to obtain an organizational perspective on training requirements and verify the results of the TNA Questionnaire. The following were the main topics for discussion: i) tasks and jobs; ii) level of knowledge on D&D; iii) training sessions attended; iv) training need; and v) problems identified in terms of performing the work duties. 20      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     5   Step  3:  Collect  Data     The third step in TNA is to collect data through: i) reviewing documents on existing training (secondary data and information); and ii) conducting survey including interviews and observation at work. It is important to collect and review secondary data and information prior to Review   conducting interview surveys. It will lead GDLA Task Force members to Secondary   Data/   understanding and utilizing existing knowledge and experiences in the Information areas of D&D reform and local administration. Experiences of DOLA/MOI in particular, which has significant experiences engaged in capacity development at the commune level, can be referred to probe the applicability to PILAC. Based on the questionnaire for individual survey and guide question for group discussion developed during the preparation stage (step 2), pre-test of the questionnaires should be conducted with officials who will be the target population to see the applicability of the questions. If tested interviewees seem to have any difficulty in answering those questions, it should be noted those points and modified the questions to make it easier to answer. After testing, the GDLA Task Force should revise and formulate the questionnaires based on the results of the pre-test. The following five steps are to be taken in conducting a group discussion: i) Conduct   orientation on objectives and contents of the survey for interviewees; ii) Survey explaining questionnaires (individual survey); iii) conducting a group discussion with a guide question; iv) wrapping up the interview; and vi) modifying methods, process and questionnaires if needed. A proto-typed cycle of conducting a group discussion is shown the figure below. However, those cycle and process can be modified according to a survey plan. 21      Project  on  Improvement  of  Local  Administration  in  Cambodia  (PILAC)   Ministry  of  Interior  and  Japan  International  Cooperation  Agency     Manual  on  Training  Needs  Assessment   al  Department  of  Local  A     It is important to note that the survey team will: i) introduce yourself; ii) explain the objectives and contents of the survey and the reasons for selecting interviewees; iii) show appreciation for taking their time; iv) obtain permission to record the interview. In doing so, the interviewees can be prepared and feel more conformable to answer questions.   Figure  8:  Prototype  of  Process  of  Group  Discussion   Below are tips for developing interview questions.   Figure  9:  Tips  for  Developing  Interview  Questions      Place  easy-­‐to-­‐answer,  open-­‐ended  questions  at  the  beginning.  This  will  help  the   interviewee  to  begin  talking  and  can  help  to  develop  trust  and  rapport.    Place  important  questions  near  the  beginning  of  the  interview.    Ensure  that  each  question  matches  a  stated  objective.    Sequence  questions  from  general  to  specific.    Sequence  and  cluster  question  in  a  logical  order.    Ensure  that  questions  are  clear,  concise,  and  jargon-­‐free.    Be  sure  that  questions  are  appropriate  for  the  skill  and  experience  levels  of  the  target   audience.    Provide  adequate  space  between  questions  to  record  information.    Pilot-­‐test  interview  questions.    Make  appropriate  revisions.       (Source:   Kavita  Gupta,  2007,  A  Practical  Guide  to  Needs  Assessment)       22