Download ackland-j-dissertati.. - Department of Computer Science
Transcript
A training solution for practicing and return to practice NHS nurses Josh Ackland BSc (Hons) Computer Information Systems University of Bath May 2005 1 Declaration This dissertation is submitted to the University of Bath in accordance with the requirements of the degree of Bachelor of Science in the Department of computer Science. No portion of this work in this dissertation has been submitted in support of an application for any other degree of qualification of this or any other university if institution of learning. Except where acknowledged, it is the work of the author. Signed: …………. Abstract Research shows that nurses can find it very difficult to cope using the IT that has become a part of their daily life. The challenges are particularly difficult; they have irregular and part time schedules whilst IT and nursing knowledge is highly variable depending on their length of their break in service. This study looks into an appropriate approach to training return-topractice nurses in the use of IT, with the deliverables of a recommendations document and an e-learning system for use in the National Health Service (NHS). The interviews and walkthrough evaluations show that nurses found the e-learning system supported their learning and that peer support could be a valuable way of helping nurses in training. Acknowledgements I’d like to thank Leon Watts for being an excellent project supervisor, providing much time and giving relevant advice. Furthermore I would like to thank Julie Box my mother and Francis Ackland my step mother for their time in providing information needed for this project and arranging for nurses to take part in the study. Project details On the enclosed CD you will find • A PDF of this document • All system implementation files 2 Table of Contents 1 Introduction................................................................................................................ 5 1.1 Aim..................................................................................................................... 7 1.2 Objectives ........................................................................................................... 7 2 Theories & approaches to training ............................................................................ 8 2.1 Learning theories................................................................................................. 8 2.1.1 The learning cycle ......................................................................................... 10 2.1.2 Peer support and collaborative learning ......................................................... 11 2.2 Approaches to training and learning................................................................... 12 2.2.1 What is training? ........................................................................................... 12 2.2.2 Models of training......................................................................................... 13 3 E-learning & the future of learning ......................................................................... 15 3.1 Types of e-learning............................................................................................ 15 3.2 Peer support within e-learning ........................................................................... 17 3.3 Advantages and limitations of e-learning ........................................................... 17 3.4 The future of e-learning..................................................................................... 20 3.5 Blending: The future of learning strategies ........................................................ 20 3.6 Conclusion: ....................................................................................................... 21 4 The NHS and training .............................................................................................. 22 4.1 The History of the NHS..................................................................................... 22 4.2 Stakeholder centred requirements analysis ......................................................... 24 4.2.1 Interviews with stakeholders ......................................................................... 25 4.2.2 Cooperative evaluation of existing NHS systems ........................................... 26 4.3 The current NHS situation ................................................................................. 37 4.3.1 Statuary requirements.................................................................................... 37 4.3.2 How training is conducted............................................................................. 38 4.3.3 Advantages and disadvantages the NHS training strategy .............................. 40 4.4 Future NHS plans/e-learning ............................................................................. 41 4.5 Conclusion ........................................................................................................ 41 5 Requirements specification ...................................................................................... 43 5.1 Source of requirements...................................................................................... 44 5.2 Specifying the requirements .............................................................................. 44 5.3 System requirements ......................................................................................... 45 5.3.1 Functional requirements ................................................................................ 48 5.3.2 Non-functional requirements ......................................................................... 51 5.4 Conclusion to requirements ............................................................................... 53 6 System design ........................................................................................................... 54 6.1 Design decisions ............................................................................................... 54 6.2 Initial Prototype design and implementation ...................................................... 55 6.3 System architecture ........................................................................................... 57 6.4 Navigational design........................................................................................... 58 6.5 Database design................................................................................................. 59 3 6.6 Conclusion ........................................................................................................ 61 7 System Implementation............................................................................................ 62 7.1 Implementation language and system architecture.............................................. 62 7.2 System setup ..................................................................................................... 62 7.3 Approach to code implementation ..................................................................... 63 7.4 Setting up the interface ...................................................................................... 63 7.5 Implementing the navigation menu bars............................................................. 64 7.6 Setting up a login system................................................................................... 71 7.7 Setting up the lesson material. ........................................................................... 71 7.8 Setting up the peer support sections. .................................................................. 72 7.9 Setting up the tests............................................................................................. 72 7.10 Allowing the user to view all of their test results................................................ 73 7.11 Logout .............................................................................................................. 74 7.12 Coding of the admin section .............................................................................. 75 7.13 Administration of the tests................................................................................. 75 7.14 Conclusion to implementation ........................................................................... 78 8 Testing ...................................................................................................................... 79 8.1 Scenario testing ................................................................................................. 80 8.2 Pilot .................................................................................................................. 83 8.3 Final system ...................................................................................................... 84 8.4 Conclusion ........................................................................................................ 84 9 Evaluation of NITS system....................................................................................... 86 9.1 Think aloud protocol and cooperative evaluation ............................................... 86 9.2 Questionnaires................................................................................................... 87 9.3 Sample .............................................................................................................. 88 9.4 Method.............................................................................................................. 88 9.5 Results .............................................................................................................. 89 9.5.1 Think aloud with cooperative evaluation ....................................................... 89 9.5.2 Questionnaire and interviews......................................................................... 91 9.6 Conclusion ........................................................................................................ 91 10 Conclusions.......................................................................................................... 93 10.1 Future work....................................................................................................... 95 11 References:........................................................................................................... 98 12 Appendices..........................................................................................................100 12.1 Appendix A......................................................................................................100 12.2 Appendix B......................................................................................................104 12.3 Appendix C: Cooperative evaluations...............................................................109 12.4 Appendix D: Use cases.....................................................................................119 12.5 Appendix E: Questionnaire...............................................................................132 12.6 Appendix E: Test results...................................................................................138 12.7 Appendix G: Walkthrough summaries ..............................................................142 12.8 Appendix E: NHS IT training: Recommendations document.............................150 4 Chapter 1 1 Introduction The inspiration for this project sprang from an interest in the way people use Information Technology and an awareness of the nursing profession. Having an upbringing with family members involved in nursing there was anecdotal evidence of nurses struggling with the use of IT especially when studying for exams. These exams were typically for a qualification such as a diploma needed for continued professional development, often conducted at a university. The project follows evolutionary development lifecycle as identified in Sommerville (2001). Sommerville identifies that for small systems (less than 100,000 lines of code) this approach is normally the preferable. Further more it is preferred to sequential development models as it allows for feedback and requirements refining throughout the project. In this way nurses will be consulted throughout the design process, with prototypes used to gather information as to the usefulness of the system. This project examines the challenges posed to NHS (National Health Service) nurses by the use of IT and seeks to identify the characteristics of effective IT training support for practicing nurses. It will focus on return-to-practice nurses; those nurses who have been trained, left the profession and subsequently returned to practice. With the majority of nurses being female, career breaks are fairly regular for childcare etc. Whilst it is important to have an overview of how nurses are training within the NHS, ultimately a review of how return-topractice nurses are trained will be important. Return-to-practice nurse are likely to be more mature and therefore the use of IT can be even more difficult. In recent times the need for NHS professionals to be IT literate has increased, with IT becoming an integral part of nurse’s everyday lives. IT is needed for two main reasons, firstly that computers are now being used on wards for viewing patient records and obtaining results and secondly that nurses now take part in more training, taking courses, diplomas and degrees which often require the use of computers. The emergence of evidence-based practice (up-to-date knowledge for practice) has meant that training and exams have become a regular 5 part of nurse’s lives. Historically it was assumed nurses were qualified, but in today’s culture this needs to be certified. The integral nature of certification provides the focus of this project; how nurses can use IT to remain compliant with training regulations and specifically how to design systems that nurses can use and learn from. There is help available for nurses to improve their IT skills. Non-mandatory IT training courses are available, but these have to be taken in nurses own time. Nurses do not always see the value in taking these IT courses and therefore struggle using IT when it is required. One nurse used the term “nurse proof”, suggesting that any systems used by nurses, needs to be designed specifically for nurses to use. In addition, it seems that nurses are not always aware of the IT courses available, or are not willing to invest time in something that they see little value in. In fact, there are many reasons other than a lack of training why nurses may struggle using IT. Whilst the younger generation have grown up with technology, many nurses have not had any IT experience and others have had very little. Certainly more mature nurses have not learnt how to use IT at any point in their lives and therefore struggle to derive any benefit from its availability. These nurses have not had the chance to encounter IT whilst still in the main learning and qualification phase of their career. Nursing is generally a profession that spans a person’s life, therefore there are many experienced nurses who will not have any IT background, some younger nurses will, but the age of personnel is surely a factor in explaining the IT skills within the NHS. There are many questions that arise in this context; how do we help nurses train for these exams? What is the best way for them to learn? How can we help them use IT to help them perform their job? Are there systems or interfaces that they would find easy to use? At a high level, the question of the best way to train staff in general is important. Simonsen & Sein (2004) suggest that organisations don’t really look at their strategy when training staff. This helps explain why nurses may find IT difficult, as not much thought may have gone into how they should be trained in using IT in order to get the best results. Henry & Stone (1995) suggest that confidence is a major factor in the success of training. If the participant does not feel confident in what they are doing or feel inadequate, the learning can be a struggle. Although computer courses are available in the NHS, nurses have shied away from courses that require the use of computers, which suggests some are not confident with using the IT required. There is no doubt that IT can be a valuable resource for nurses to gain the information they want and help them in their jobs. Kazmer & Haythornwaite (2004) suggest that online training can be better for learning as people can learn in their own time and at their own speed. Nurses come from many different backgrounds and have “individual differences” in the way they learn and cope with new concepts. Some will find adapting to computer technology much easier than others; therefore a range of training media should be available. 6 1.1 Aim The aim of this project is to research an appropriate strategy for training return-to-practice NHS nurses to use IT. The main deliverable will be an online IT training system, with recommendations on how to help NHS staff to use IT effectively. 1.2 Objectives The objectives of this project are as follows • • • • To research training aspects of the problem domain in order to gain a better understanding of: Organisational approaches to training, including learning strategies and teaching media used. Online training systems – with particular reference to interface design To research the NHS training context including: NHS training requirements (including shifts in history) Current strategies for training nurses, with particular reference to IT training. Current media, methods and activities used for training nurses in IT. Existing IT systems used in the NHS. To formulate a viable approach for IT training Develop frameworks for helping with training nurses in the use of IT Design and implement a demonstration IT training system Use this system and test the way nurses interact with it Conclude with a set of recommendations on how to help nurses develop their IT knowledge and skills. 7 Chapter 2 2 Theories & approaches to training This project proposes a review of the training strategy for return-to-practice nurses within the NHS. This chapter looks into existing research on learning theories with a focus on collaborative learning strategies. Furthermore the section looks into training strategies and frameworks that can be used in organisations. 2.1 Learning theories Any attempt to aid people with learning should draw upon some basic knowledge of theories on learning. In this sub-section, theories on how people learn will be discussed with the aim of gaining information that can be used in a training solution to be developed for the NHS. Furthermore this information will be used to make recommendations to the NHS on how they impart knowledge to nurses. Learning can be defined as “a relatively permanent change in human capabilities that is not the result of the growth processes” (Noe, 2005). The outcome of learning can include verbal information, factual knowledge, intellectual reasoning, motor skills (shoot a gun and consistently hit a small object), attitudes and cognitive strategies. Reinforcement theories: Reinforcement theories suggest that the way learning is rewarded has an effect on the retention and application of this information. Positive reinforcement is defined as offering a positive outcome whereas negative reinforcement is the removal of a positive outcome. This could involve offering a bonus or promotion for employees excelling during a training course. Offering a reward or congratulations for learning can have a positive effect for some people, whereas a negative effect or the removal of a positive outcome can produce positive results in some situations. An example of this would be removing a bonus, or docking wages to ensure that exams are passed. Within the NHS this could involve having to spend time retaking a training course in a nurses own time in order to remain registered as a nurse. Although the overall picture of reward in a professional setting can be very complex, the benefits of success and the result of failure should be made clear. 8 Social learning theories: Social learning theory suggests that learning new skills comes from; 1. directly experiencing the consequences of using that behaviour or skill 2. the process of observing and seeing the consequences of the behaviour. Social learning theory suggests that learning is influenced by self efficacy (a person’s judgement about whether s/he can learn the knowledge and skills). Henry & Stone (1995) suggest that confidence is a major factor in the success of training. If the participant does not feel confident in what they are doing or feel inadequate, the learning can be a struggle. Learning strategies should aim to encourage and foster confidence to maximise self efficacy. Goal setting theory: Goal setting theory (Noe, 2005) suggests that behaviour results from a person’s conscious goals and intensions. Goals influence behaviour by directing energy and attention, sustaining effort over time and motivating the person to develop strategies for goal attainment. Goals only work if someone is committed to the goal. Strategies that are clear about the goal structure of learning should therefore help to focus energies of the learner. Need theories: Need theories help to explain the value that a person places on certain outcomes. A need is a deficiency that a person is experiencing at any point in time. Maslow (Grant 2005) developed a hierarchy of needs, with self actualisation, self esteem, social, safety and physical needs all needing attention in respective order. Self actualisation looks at personal development or progressing as a person, the trainee would need to feel that this training would result in the bettering of ones self. Self esteem is how a person feels about themselves, so the training should not make people feel inferior and mutual respect should be in a training environment. Social needs would suggest that e-learning should not be the only approach taken towards training as this would not cater for the social needs. Finally the trainee should feel safe and have their physical needs met (enough sleep, food, water etc). Need theories suggest that to motivate learning, trainers should identify trainee’s needs and communicate how the training program content will help them. Need theories intersect between social learning and goal setting but in addition, speak to career development. Expectancy theory: Expectancy theory suggests that a person’s behaviour is based on 3 factors: expectancy, instrumentality and valence. Beliefs about the link between trying to perform a behaviour and actually performing well are called expectancies. A belief that performing a given behaviour is associated with a particular outcome is called instrumentality. Valance is the value that a person places on an outcome. Again, this would suggest that trainers should make the aims and objectives of the training session clear to all trainees. The connection between actions taken during training and training outcomes should be clear. Adult learning theory: Adult learning theory provides some ideas on how to train adults. It suggests that adults have the need to know why they are learning something and need to be self directed. Adults bring more work related experiences into a learning situation and enter into a learning experience with a problem centred approach to learning. Adults are motivated to learn by both extrinsic and intrinsic motivators. The majority of learning theory has developed to account for learning in minors. Adults have an expectation of engagement with material and are socially 9 autonomous. Continuous professional development training strategies must capitalise on the majority of older learners. The above theories look at the best way to impart information to help someone learn and will help with designing and evaluating systems. Relating theories to the NHS will help to derive the relative advantages and disadvantages or these approaches and suggest possible gaps. 2.1.1 The learning cycle The section will look at how we learn or as Noe (2005) puts it; the learning cycle. This follows on from the last section about learning theories and goes into more depth about the physical way in which humans retain information. The aim of reviewing the way we learn is to uncover truths that can suggest effective ways of making learning systems help people to learn. It will help categorise different types of learning and the affect that age can have on a person’s ability to learn. Learning can be considered a dynamic cycle that involves four stages, concrete experience, reflective observation, abstract conceptualisation (how to solve the problem) and active experimentation (Noe 2005). Concrete experience suggests that we learn according to our own experiences, not necessarily by what others impart to us. For instance fire-fighters are normally very fire conscious, because of the experiences they have seen with fires. Reflective observation means that we need time alone to think and reflect on what we have seen, so that we can form our own ideas. Abstract conceptualisation looks at how we solve a problem and active experimentation suggest we learn by trying new things. It is important to note that age can have a significant impact on the way we learn. Firstly, mental capabilities decrease from the age of twenty until we are about seventy. Therefore it will be more difficult to learn new things from the age of 20. However experience can often compensate for this decrease in mental capabilities; the concrete experience starts to play a part. Noe (2005) categorises people born in different eras according to the educational experience they would have had in school. If you are born after 1980, you are likely to be optimistic, willing to learn and technology literate. This would make you more adept to learning new stills and therefore likely to adjust well to e-learning. Those born between 1961 and 1980 need feedback and flexibility and dislike close supervision, so reflective observation could be helpful. They also require a healthy work/home life balance. People born between 1945 and 1960 are often competitive and hard working and concerned about employee fairness. They would like to see people treated in the same way. People born between 1920 and 1944 are patriotic, loyal and have lots of knowledge. They like to share their experiences, so when training it is often helpful to draw out their personal experiences. It is important to note that these categories may be huge generalisations and that everyone is different, therefore these categories should not be strictly followed. Furthermore, employees need to know why they are learning, if they see no advantage to the training, they are likely to lack motivation and concentration. The objectives of the training should be clearly stated at the beginning of the training and feedback should be gathered as regards whether this is viewed as useful. 10 When learning, information must be committed to memory. Memory works by processing stimuli we perceive through our senses into short term memory. If the information is determined important it moves to long term memory where new interconnections are made between neurons or electrical connections in the brain. With this information, there are ways that a trainer can help employees store knowledge. By making trainees aware of how they are creating, processing and accessing memory and discussing appropriate learning styles, the trainer can help the trainee communicate how they would like to learn. In order to create long term memory, a concept map showing relationships amongst ideas can be created to help trainees visualise. Drawings and role plays can also work with some people. How we learn is very important to this project. Age is going to be a primary focus in this project and Noe's categories of different learners and the fact that mental capabilities deteriorate at the age of twenty are important factors. Return-to-practice nurses are often older, normally between the ages of thirty-five and fifty-five and may therefore struggle learning new skills. The above theories have shown that it may be best to draw on experience when training return-to-practice nurses. The physical ways in which we learn are important, this shows that using visualisations and role plays can help people commit things to memory. However, the example of workers at Siemens (discussed in the section below, is a very interesting study that is of interest here. The study has prompted research into collaborative learning and in particular, peer support. Communities of practice are invaluable in helping learning and are a very effective way for people to learn from each other. It will be essential to review how collaborative learning is supported within the NHS. 2.1.2 Peer support and collaborative learning A definition of peer support is as follows: “Peer support is social/emotional support, frequently coupled with material support - e.g., financial resources or housing - that is mutually offered or provided by persons” (UPenn collaborative on community integration website) This particular definition applied to peer support for people with disabilities, however it is also relevant for standard peer-to-peer support. It follows the idea of people helping each other when performing a task, developing synergies and therefore being more efficient. Collaborative learning is in contrast to the competitive process usually seen in a classroom (We 2002). Furthermore it is argued by Gokhale that collaborative learning can enhance both social and cognitive skills. Learning in communities of practice can help people “bounce ideas of each other. Siemens (Swart et al 2004) found that many of their employees were gathering around the coffee machines during the day and thought that productivity would increase if they removed the machine. However they found that calls to their helpdesk significantly increased and productivity was reduced. After further analysis they found that communities of practice had developed and people often discussed problems and worked out resolutions by the coffee machine. This is very important to learning and doesn’t always take part in a training 11 strategy, but for return-to-practice nurses, as adult learners, communities such as this are a great opportunity. 2.2 Approaches to training and learning There are many training techniques, methods and strategies each having a vital connection with the organisation for which the training is to be delivered. This section will look at training from a general organisational perspective, firstly describing what training is and why it is important, followed by looking into models developed for training people. 2.2.1 What is training? Noe (2005), describes training as a planned effort by a company to facilitate employees’ learning of job related competencies. These competencies include knowledge, skills or behaviours that are critical to successful job performance. Noe also describes learning as the acquisition of knowledge by individual employees or groups of employees who are willing to apply that knowledge to their job making decisions and accomplish tasks for an organisation. Training is costly, with $5 billion spent in the US on training alone (Noe, 2005). It is estimated that a lot of this money is wasted, due to companies’ lack of, or poor training strategies. However, in recent times, jobs have become more knowledge based, leading to increased training pressures. Knowledge workers are employees who contribute to the organisation not though manual labour but through what they know about customers or a specialist body of knowledge. Recent figures show that 85% of jobs in the US require extensive use of knowledge and one reason attributed to this is employee empowerment. Empowering means giving employees responsibility and authority to make decisions regarding all aspects of product development and customer service. This has led to training emerging as a vital aspect of any organisational strategy. Before the importance of training was fully realised, companies would often train employees without much thought going into a training strategy. Simonsen & Sein (2004) suggest that organisations don’t really look at their strategy when training staff. Nowadays training needs to be organised to link into company values, business strategy and organisational goals. With so much money being spent on training, organisations are realising the need to manage this training and ensure that the training links into the companies goals and meets the organisation's objectives. Nursing is a profession where training is more important than in most professions as the work that nurses do can save lives. If a salesman makes a mistake, it could lead to the downfall of a sale, but if a nurse makes a mistake, lives can be in jeopardy. As a professional government organisation, the NHS has to ensure that employees are trained to be able to do their job effectively. 12 2.2.2 Models of training The importance of strategic, ongoing training is clear. In this subsection, strategic models that training can take are described and evaluated. The aim of this is to develop knowledge on ways to train people and the advantages and disadvantages of these approaches which can then be applied to the NHS’ training strategy. Many models of training have been developed as companies aim to be more intelligent in the way they train staff. Faculty model: The faculty model (Noe, 2005) contains a training department headed by a director with a staff of experts with specialised knowledge of a particular area underneath him or her. An advantage of this is that trainers are experts in the area they train in (due to their specialised knowledge). A disadvantage of this model is that training may work, but not be linked into the organisations goals, people may be trained in tasks that don’t meet the organisation’s strategy, or they may not be trained in tasks needed to meet the strategy. Customer model: The customer model organises training by business function. This model overcomes the major weakness with the faculty model in that training is specifically designed to fit the needs of a particular business group. Training is integrated with other human resource management activities. However, the amount of time it takes for trainers to learn about the business function they serve can be lengthy. Once these training methods are developed it can be very difficult for the training director to oversee all training functions as the methods may be so diverse. Many of the trainers are likely to be experts in their field but not necessarily experts in training, which could lead to poor presentation skills. Matrix model: The matrix model involves trainers reporting to both a manager in the training department and a manager in a particular function. The trainer has the function of being both a training expert and a functional expert. So a sales trainer would report to two people, the marketing manager and the director of training. This ensures that training is linked to business strategy and that the trainer gains expertise in understanding a specific business function. However having two managers can lead to conflicts of interests in the training and add time demands on the trainers. Corporate university model: The corporate university model differs from other models in that the client group includes not only employees and managers but also stakeholders outside the organisation including colleges and universities. A wider range of programs and courses are offered as opposed to the other models mentioned. The university model centralises training to make use of best practices and enables the control of costs. Small and larges companies have started their own universities and have found the benefits include improved recruitment, increased revenues, reduced turnover, better employee advancement and a deeper bowl of talent. In order to be successful, the university must have a vision that links to the organisation’s strategy. 13 Virtual model: The virtual model assembles a collection of people from across an organisation into a training group, It operates according to three principles. Firstly, employees in the organisation have responsibility for learning. Secondly, the most effective learning takes place on the job, not in a classroom. Finally, the relationship between a manager and employee is critical if the training will lead to better job performance. A virtual training model is characterised by five competencies, strategic direction, product design, structural versatility, product delivery and accountability for the result. The most visible difference between a VTO (Virtual Training Organisation) and a traditional training model is its structure. The traditional training organisation tends to operate with a staff of trainers and administrators who can perform specific functions (silo approach). In a VTO all persons involved in the training process communicate and share resources. Conclusion: Models are used to ensure that the training is organised in a way that meets the organisations requirements. For instance, the virtual model is more likely to be used in larger, developed organisations. Referring to these models will help to identify what approach the NHS has taken and evaluate whether this is a sensible model to employ. However these models should be used with caution. Simonsen and Sein (2006) suggest that although a number of conceptual frameworks have been proposed in recent years, few have been evaluated empirically or put into practice. But these models will help to determine the suitability of this strategy from a high level and evaluate the advantages and disadvantages of this approach. They shall be drawn upon to propose an appropriate model for the training organisation of the NHS later in the project. Although some of the information discussed above on training strategies could be viewed as too general, it is important to develop knowledge on methods and strategies for training. This will enable informed recommendations and comments to be made on the NHS training strategy. 14 Chapter 3 3 E-learning & the future of learning In this section, e-learning is described, with its advantages and disadvantages critiqued. Elearning will be a primary focus in this project so this chapter should be important in deciding how to implement an e-learning system for the NHS. Methods of training and learning have developed as technological capacities have increased. Some definitions of e-learning are shown below: E-learning refers to instruction and delivery of training by computers online through the internet or the web (Noe, 2005). E-Learning refers to the use of internet technologies to deliver a broad array of solutions that enhance knowledge and performance (Rosenberg 2001). However, these definitions do not seem satisfactory, as they seem to narrow e-learning down to web use. Surely with the e in e-learning being electronic, e-learning can involve a range of electronic media such as DVD’s and CD’s. For the purpose of this project e-learning will be defined as the following; “E-learning refers to any training performed by electronic media.” E-learning was a major development in the way we learn and many companies have incorporated e-Learning into their training strategy. In the past, new technology for learning has often been a “fad” that comes and goes, so the question of why e-learning will be any different is a fair one (Rosenberg 2001). However with the scale that e-learning has now reached the likelihood of e-learning being a passing fad is diminishing. Large companies such as Cisco have found e-learning to be an essential part of their training program. The Internet has begun to radically change the teaching of adults in the US, who want to improve their skills or further their general education (Rosenberg 2001). “Technology has revolutionised business; now it must revolutionise learning” – Trace Urdan and Conelia Weggen W.R Hambrecht and Co (Rosenberg 2001). 3.1 Types of e-learning 15 There are many different types of e-learning systems which are discussed below. It will be important to have an understanding of these in order to evaluate the most suitable system to be implemented within the NHS and further understand the nature of e-learning systems. Expert systems (Rosenberg 2001) showed how to use technology for intellectual capital; they were a way of gaining information from a system. Expert systems have three elements: 1. A knowledge base that contains facts and figures and rules about a specific subject 2. A decision making capability that imitates an experts reasoning and draws conclusions based on facts and figures 3. A user interface that gathers and gives information. An example of an expert system is MYCIN (Shortcliffe and David 1975), which was used in hospitals and provided diagnosis based on the symptoms inputted. However, expert systems were very expensive and newer technologies have helped reduce the cost of systems whilst providing the users with many additional benefits. Groupware such as Lotus notes provides facilities for electronic meetings and allows people to collaborate on the same document. Groupware combines elements like e-mail, document management, databases and electronic bulletins. Groupware can allow digital collaboration; the use of technology to enhance and extend employees abilities to work together regardless of their geographic proximity. Tools such as Webex and Netmeeting can be used to share information over the Internet and are extensively used in geographically divided companies. Tools such as Webex allow for synchronous communication, where trainees, experts and learners interact with each in real time, in the same way they would in face-to-face classroom instruction. Groupware such as Lotus notes provides asynchronous communication methods and refers to non real-time interactions such as videos put on the web. As mentioned early, these types of collaborative tools are of interest in this project and will be discussed further in the peer support section. Computer Based Training (CBT) is an interactive training experience in which the computer provides stimulus for which the trainee must respond and the computer analyses and provides feedback. CBT training can be expensive to develop, but “off the shelf” CBT tools have also been created. For instance it is possible to purchase a “Performance management review” help system that responds to the stimulus and provides feedback to ensure wording in a performance review is professional. This kind of interactive system is again of importance in this project in that the need for computer analyses will be needed in any tests that are designed. In the introduction to this project, it was mentioned that staff are required to be certified. Therefore any e-learning system for the NHS should provide analysis of data to provide marks that will allow certification to be provided appropriately. Multimedia training combines audiovisual training methods with CBT. These programs integrate text, graphics, animation, audio and video and allow the trainee to interact with the content. Multimedia training was initially very popular, but can be expensive to develop and has often been replaced with online training techniques. Online techniques will likely be more suitable for use with nurses, as the ability to access online systems through any Internet enabled computer is an advantage. 16 The systems currently used by the NHS for e-learning will need to be evaluated. The above types of e-learning systems will help categorise these systems and allow the advantages and disadvantages to be ascertained. For instance, evaluating whether groupware is used in the NHS will help identify the rate at which collaborative learning is supported in the NHS. 3.2 Peer support within e-learning In this section, research will be documented on peer support within e-learning. A discussion on different types of e-learning has already been outlined and peer support could be incorporated into any of these types. The Siemens example discussed (Swart et al 2004) provided a very good example of the vital role that collaboration can play in today’s business environment with the increase in jobs requiring significant use of knowledge. Many tools have been created to provide this support such as e-mail, instant messages and forums. Companies have worked to develop tools that allow this communication and collaboration and the findings from these can be interesting. Reachout (Jacovi et al. 2003), developed at IBM Haifa research lab is an example of a tool aiming to support formal and informal interaction. It supports the sharing of ideas and fosters peer support in a very informal environment. Reachout provides an environment for posting questions to predefined interest groups and is very similar to a conference chat, although participants can see the whole transcript. The result was a tool that was successfully deployed and used, facilitating many discussions, but the most prominent feature was the peer support function. The above section has been useful in seeing the main form in which people choose to collaborate. It seems that often workers are happy to offer help to each other through electronic systems. In the experience of Reachout it seems that question and answer forums are an excellent way of doing this. 3.3 Advantages and limitations of e-learning There are many advantages claimed for e-Learning. The major benefits come from responsiveness, consistency, maintenance, self pacing, and location. Advantages: E-learning has enhanced business responsiveness (Rosenberg 2001) as lots of people can be trained quickly upon deployment of a system. In the past it would have taken lots of time to retrain the relevant people (sometimes everyone) in an organisation whereas nowadays this training can be rolled out quickly. This means that staff will have more knowledge about latest developments, which will increase the businesses responsiveness. Messages in training can remain consistent or be customised as required. In a situation where everyone needs the same instruction the training can be identical, something which would be very rare in classroom training, particularly when different trainers are used. However, different training programmes can be rolled out to different areas or departments if required, which gives the system flexibility. 17 If an e-learning system is well made, the data can be maintained easily, which means the content will be more timely and dependable. Training provided as a hardcopy cannot be updated easily, which means the data may not be as good as data on an e-learning system. Timely information can help companies adapt to business change and can provide companies with competitive advantage. E-learning systems allow the user to train in their own time and pace and can provide twenty-four-by-seven access. This is particularly useful as it is often difficult for people to make time to attend courses. This learning can sometimes take place anywhere there is an Internet connection and a computer. This training can take place in the home if required, which can encourage people to spend time on the training outside of work hours. Learner control refers to the ability for trainees to actively learn through self-pacing exercises, exploring links and other material. Learner control is a major advantage of e-learning systems. E-learning can reduce user ramp up time as people are generally able to use web technology these days. Standards of browsers mean that people are normally familiar with the Internet, no matter which machine they are on and understand the features of the Internet. This means that they do not have to learn anything other than how to navigate the e-learning systems user interface. E-learning can allow communities to gather and share information regardless of their geographic location. This can mean that after classroom training has ended, communities can stay in touch to share knowledge. Communities are a highly regarded way of sharing information, because people often work better in groups than they do individually. E-learning systems can provide forums for these e-learning communities to develop. The advantages of e-learning help support the case for developing a new e-learning system for the NHS and help evaluate the systems already in place. In order to have a balanced view of the impact of e-learning systems, the limitations of e-learning also need to be considered. These are strategic inflexibility, reliance on material without a human tutor, peer interaction and support and authenticity Disadvantages There are many reasons why e-learning will not work in an organisation. Many companies follow the idea of this ancient proverb “If we don’t change our direction, we’ll end up exactly where we are headed” (Rosenberg 2001). Companies are often reluctant to change a strategy, particularly if they do not see any major problems with it. With any technique, it is important that the content of the training is adequate; the best media for training is useless if the training material is poor. Repurposing refers to directly translating an instructor led face-to-face training program to an online format. If the training content is good initially this can be advantageous, but if the content is poor, the training will still be poor. “Don’t underestimate learner’s willingness to try e-learning but don’t think for a nanosecond that they will tolerate bad learning solution just because it is web deployed” (Rosenberg 2001). Companies can often focus too much on the technologies used for e18 learning and neglect the content of the training itself, which can lead to poor systems being developed. Research suggests that the reason why some employees fail to complete online learning and prefer instructor led face-to-face instruction over online learning is that they want to be able to learn and network with their peers (Noe, 2005). Effective online learning needs to connect trainees and facilitate interaction and sharing through the use of chat rooms, email, electronic bulletin boards and discussion groups. If e-learning lacks this, people will be more likely to favour other forms of learning. If the training isn’t authentic, i.e. if people cannot identify with the content and the system and relate it to experience, the learning experience will not be positive (Rosenberg 2001). If the program does not reflect what to do in an ideal world, users will be apprehensive about applying the learning to real life situations, which is the point of training in the first place! If the form of the training is better than the substance, i.e. the training just looks good; this will lead to failure. Lots of “fancy wizardly and latest gizmos” don’t add much to training, but take lots of time. This can be a costly lesson for organisations. It is also important to note that one size doesn’t fit all, people learn at different paces and therefore learning cannot be type casted. However, the development of systems to provide such flexibility can often put the costs to prohibitive levels. At some points the technology used can be a barrier to the success of the system. Many computer based training systems were developed, only to be made redundant by operating system and other technological changes. This problem is easing with the increase in interoperability and standards seen as important in new technological discoveries. In a similar vein, systems were often developed at great cost, only to be useless after fulfilling their initial purpose. The training may have been useful, but when asked a question a month or so in advance, it can often be difficult to go back and find this information again in the training, i.e. searching the system was a problem. Another disadvantage is that e-learning is often treated as a separate activity and not viewed as part of the overall training strategy. This means that learning isn’t reinforced after the initial e-learning session and leanings gained are not maintained. This can lead to people questioning the value of e-learning. Similarly if e-learning isn’t supported by managers, it will not be valued by employees. It is vital that e-learning receives high-level sponsorship if it is going to succeed. Managers will need to ensure that e-learning is part of the organisation’s culture, this is one the most important factors to e-learning being successful. Finally, a poor e-learning system can be very boring, the material may be uninteresting and the pages may just be filled with text. All training needs to be interesting for it to be successful and e-learning is no different in this regard. Conclusion: In summary, it is important to have a balanced view when implementing an e-learning system. Upon evaluation of e-learning it is clear that it can provide organisations and individuals with many benefits. However, the critical success factor of the training is the content within the system. It is essential to regularly collaborate with nurses throughout 19 development to ensure the content matches the requirements. It will also be vitally important to ensure that the e-learning system is integrated with the existing methods of training to ensure that it is not a separate activity. It will also be important to develop a system that helps the user learn, not a system to show of skills in programming languages. It seems that any development that is undertaken within this project will be a part of the project, not the whole project, as an in-depth look into the NHS’ training strategy and recommendations on how to improve this will be important. 3.4 The future of e-learning For the first time people have the equipment for e-learning available, they will therefore not have to purchase anything extra to take advantage of e-learning systems. However, there are a number of challenges to address if e-learning is to reach its true potential. E-learning will need to be sold to decision makers in organisations. Bonds of e-learning will need to be broken, with it seen as part of a strategy, with knowledge and performance management just as important. The speed at which e-learning systems are deployed must quicken, so that companies can take full advantage of a changing market. It is clear that the people element must be preserved within learning and e-learning must incorporate this. The convergence of e-learning and e-business will grow, with knowledge distribution becoming heavy business. E-learning will likely thrive in colleges and universities which will result in a decline in classroom usage and a redefinition of the role of the classroom. Performance certification will be likely to increase from a number of resources, with web certification becoming more common and recognised by businesses. Wireless devices will likely require knowledge management, the ability for people to learn on the move. Finally, we could see the end of “E”; it will not be differentiated from standard learning. Organisations will continue to look for better ways to train people and will therefore redefine their strategies to improve. It is clear that e-learning will not be a techno fad that will pass; it is something that will become fundamental to learning. However, e-learning will no longer be seen as a separate part of a training strategy, the next sub-section on blended learning will discuss this. 3.5 Blending: The future of learning strategies We have reviewed training strategies and methods, learning strategies and e-learning, which will now be tied together in this section; looking at blended learning. Clearly e-learning has been a significant step in the progression of training and learning. It is clear that with the increase of jobs requiring knowledge workers, training is going to continue to be important for any organisation. Most companies who need to train staff realise that one type of training isn’t enough, e.g. just e-Learning. Winston Churchill once said “personally, I’m always ready to learn, although I do not always like to be taught” (Noe 2000). This suggests that training should be mixed together and that e-learning can play an important part in any organisational strategy. The 20 above review of the advantages and disadvantages of e-learning clearly shows that a mix of techniques is required in order to combat some of the disadvantages of e-learning. In light of Winston Churchill’s comments it can be suggested that a mix of e-learning and classroom teaching can work well. However it has to be conceded that the classroom may not now be the default delivery system, there will be less teaching and more facilitating. The mix of training techniques can be referred to as blended learning (Noe, 2005). Blended learning combines online learning, face-to-face instruction, and other methods for distributed learning. Companies will use blended learning techniques as there is often insufficient bandwidth to do all learning over the web. Furthermore trainee preference is usually for faceto-face learning and employees are often unable to find unscheduled time during their workday. Blended learning will most likely be found in the NHS already. The main challenge will be the extent to which it is used, for instance if e-learning still plays a minor solitary part in the training or whether it is part of the overall training strategy. 3.6 Conclusion: This chapter has identified e-learning as something that will be in the future of the majority of learning and training strategies. Reviewing the advantages and disadvantages of e-learning will have an influence on any system that is built whereas the section on blended learning has significantly enhanced the scope of this project, with the focus being on providing recommendations on training issues rather than simply building a system. 21 Chapter 4: 4 The NHS and training To ground this project in NHS practice and the expectations it has of its nursing staff a review of the NHS and its training is undertaken. In the previous chapters a look into training from a general organisational perspective was reviewed. This chapter begins by looking into the NHS’ history, and then outlines how the subsequent information on NHS training strategy was gathered from stakeholders. It then looks into how training is currently conducted within the NHS and reviews the future of NHS training with particular reference to e-learning. Information in this chapter was extracted from three main sources, the information about the history of the NHS was gathered by using the NHS official website and the rest of the information came from interviews and evaluations of existing systems, which are documented in this chapter. 4.1 The History of the NHS The value of looking into the NHS’ history is that it highlights the nature of changes that have adapted the role of nurses. This is important in understanding the new pressures and responsibilities that have led to the increasing need for certification. The National Health Service (NHS) was established on the 5th of July, 1948, to provide healthcare for all citizens in the UK. The NHS brought hospital services, family practitioner services and community-based services into one organisation for the first time. Demand for NHS services was underestimated and the NHS was under funded and overly busy. GP’s performed the same job as they do today, referring patients where appropriate to hospitals or providing specialist treatment and prescribing medicines and drugs (NHS website). Between 1958 and 1967 the NHS was beginning to settle down and the treatment offered was getting better as improved drugs were being introduced. Heavy criticism of the NHS’ structure led to an restructuring; divisions were created with the aim of grouping medical staff by speciality. In 1968, optimism prevailed in the NHS, but this was progressively eroded as the full implications of cradle to grave care with increasingly sophisticated medical treatments 22 became clear. The decade between 1978 and 1987 was characterised by the growing acknowledgement that the NHS could not do everything required within its medical bounds. New technology was being introduced and new treatments were available, which were often more complex. This led to rising expectations of the health service, coupled with an increasingly elderly population that brought strain to the NHS. The change of government in 1979 did not bring an immediate change in the health service policy. By 1987 health authorities throughout the UK were in debt, waiting lists were increasing and hospital wards were being closed; despite evidence of higher spending and increases in staff numbers. The public and the health care professions were dissatisfied and the service was increasingly subjected to scrutiny in the media. A whitepaper entitled “the new NHS. Modern. Dependable” covering the period from 1988 till 1997 described a new approach based on six key principles. (Quoting from NHS official website, 7) • • • • • • To renew the NHS as a genuinely national service, offering fair access to consistently high quality, prompt and accessible services right across the country; To make the delivery of healthcare against these new national standards a matter of local responsibility, with local doctors and nurses in the driving seat in shaping services; To get the NHS to work in partnership, breaking down organisational barriers and forging stronger links with local authorities; To drive efficiency through a more rigorous approach to performance, cutting bureaucracy to maximise every pound spent in the NHS for the care of patients; To shift the focus onto quality of care so that excellence would be guaranteed to all patients, with quality the driving force for decision-making at every level of the service; To rebuild public confidence in the NHS as a public service, accountable to patients, open to the public and shaped by their views. The whitepaper was significant in writing the nurse into a central role in health care delivery. Critically it explains how quality of care coincides with the basic distinction between nurse and doctor medicine; care and cure. In March 1998, NHS Direct, the nurse-led health advice service, was launched to give people 24-hour health advice. It was the start of a growing range of convenient alternatives to traditional GP services, including the launch of NHS Walk-in Centres, which offer patient treatment and advice for a range of injuries and illnesses without the need to make an appointment. This further put nurses on the front line in these walk in centres, further adding to the responsibilities they have. The last decade has seen the IT age become a part of the NHS. The potential of Information Technology to transform the care of patients has been recognised in a paper published in September 1998, which introduced new strategic approaches to the use of IT within the NHS. 23 The publication of the 10-year NHS plan in July 2000 suggested plans to transform the NHS into a health service fit for the modern day. The NHS plan promised more hospitals, beds, doctors, nurses, shorter waiting times and cleaner wards. In December 2004 a new Knowledge and Skills Framework allowed staff to progress by taking on new responsibilities. This enables employers to design jobs around patient and staff needs, improving overall patient care and job satisfaction for nurses. The new system also introduced standard arrangements for hours, annual leave and overtime. The volume of work now placed on the NHS and the financial constraints they have will make it difficult to ensure that staff are trained sufficiently, as training is an expensive function and it is difficult to take nurses off the word and into the classroom. This brief overview of the NHS’ history has shown a few important changes. Firstly, the demand for NHS services has constantly increased and this has resulted in a redefinition of the role of nurses. Furthermore it is we can see that nurse’s roles have become more varied and autonomous. Nurses need to be highly skilled now, more so than they every have had to be before, for which the corollary is more training being required. 4.2 Stakeholder centred requirements analysis The NHS history has provided a backbone to the project and depended understanding of the challenges faced by the NHS. In the section, the processes of getting further information to develop understanding of the current NHS situation and the future of the NHS is discussed. Stakeholder centred requirement analysis will be used to gather this information. This project has clearly identified stakeholders namely training representatives and nurses. These stakeholders will be • • Nurses between the ages of 30 – 60 who have left and returned to the NHS at some point. Nurses from Basingstoke and Oxford will be used as representative samples. A training representative from the NHS. An NHS staff member who works in the training department (Donald Burrell centre) of the North Hampshire hospital will participate. The interviewee worked at the Basingstoke district North Hampshire hospital in the training department. In order to elicit the most accurate and detailed requirements it is essential to carry out a detailed study of the problem domain. Reviewing existing literature would not be sufficient in gathering requirements, although it helped to formulate requirements initially. These initial requirements can be found in Appendix B. However, following the literature review, requirements were gathered from other means and the source of the requirements will be stated for each functional requirement. Essentially, further requirements were derived from information that was gathered from interviews and evaluations of existing systems as described in this section. Requirements were also derived from informal discussions with 2 NHS nurses, both of whom had been with the NHS for over 10 years. Both had performed numerous roles, but at the time of this study, 24 one worked as a nurse in an oncology ward and the other worked as a nurse on a medical ward. Furthermore discussions with a project supervisor (see acknowledgements) were significant in suggesting ideas and providing advice on how to take the project forward. 4.2.1 Interviews with stakeholders This sub-section describes how interviews were used to analyse the project stakeholders. Through the use of questioning, interviews can be used to find useful information about the domain, namely training within the NHS. Structured interviews were conducted and notes from these interviews can be found in Appendix B. The structured interviews conducted are described below, namely an interview with an NHS nurse and an NHS training representative. The interview with an NHS nurse was conducted first due to the accessibility of this nurse. Furthermore the aim of this interview was to gain an overall understanding of training in the NHS, with this being supplemented subsequently by an interview with an NHS training representative. Interview with NHS nurse: The interviewee was a middle aged female who has been a nurse for 25 years, working for the NHS for 10 years. The nurse was a return-to-practice nurse who had been back in nursing for around 8 years. The nurse has recently completed a diploma at Southampton University. This interview provided a firm grounding on NHS policy, with information about NHS training policy and staff development being extracted. Because of the more formalised setting, answers provided were specific and detailed. However, some of the results on training were then contradicted by comments made by an NHS training representative. For instance, the NHS nurse commented that no IT training was available for IT help. Interview with NHS training representative: This formal interview was used to gather information about NHS IT training and any issues they have or requirements for improvements. As mentioned, this interview was used to supplement and confirm information received in the previous interview. This was the most difficult interview to set up due to the availability of the interviewee but was the most valuable in providing information specific to NHS training. The interview provided a more solid understanding of the training regulations for NHS staff and was very useful in determining the IT training offered. This interview discussed the ECDL (European computing drivers licence) and information was provided on the content of the ECDL. A sample of this information can be found in Appendix B. The outcome of the interview was that an electronic system following the format of the ECDL should be implemented. This would enable the ECDL to be taken by more staff and furthermore would provide users with all the benefits of using an e-learning system. The exact nature of what this system should do was discussed; this provided enough information to be able to document system requirements, as per the next chapter. 25 Conclusion: Following these structured interviews preliminary requirements for an e-learning system were derived. Further information on these interviews can be found in Appendix A. The nurse used in the initial interview was accessible and therefore requirements for the system and clarification could be obtained throughout the project in a number of unstructured interviews. This nurse was used throughout the project to provide needed information and clarification. 4.2.2 Cooperative evaluation of existing NHS systems This subsection documents two evaluations of e-learning IT training systems and a review of an e-learning system currently used within the NHS. The review of the online IT training systems will be useful in evaluating products that are similar to what will be implemented in this project. Furthermore the evaluation of the e-learning system used in the NHS is fundamental in ascertaining what systems are used by NHS staff and will help in understanding how best to design IT systems for nurses. It is good practice when developing systems for a specific set of users to look at the systems that they currently use. Assessing the good and bad points of existing software can help incorporate favourable features or eliminate undesirable features in the design of a new system. Due to the accessibility of nurses, nurses were only used in the evaluation of the NHS system. A novice computer user was used in the evaluation of existing online training systems. This user was inexperienced in the use of IT and therefore showed likeness in skills to that identified for nurses. 4.2.2.1 The DECIDE framework The DECIDE framework as identified by Basila et al was adopted for these evaluations (Preece 2002). This framework provides a clear structure for the evaluation with clear goals. The DECIDE framework follows six stages: Determining the goals Exploring the questions Choosing the evaluation paradigm Identify the practical issues Decide how to deal with the ethical issues Evaluate, interpret All three evaluations In this section followed the DECIDE framework. Determine the goals: Evaluate usability of systems identifying o Usability of login systems o Favourable navigational structures o Favourable designs 26 o Key usability issues o Use of standards o Constraints on the systems Evaluate the effectiveness of help and guidance within the system Explore the questions: In order to ensure that the user is on the correct track, specific questions will be asked to guide the analysis. These will include: • Whether the participant could login easily? • Could the participant identify links? • Could the participant navigate through to a particular section? • What issues did the participant encounter when using the system? • Is the participant happy using an online system? • Is the goal of the system clear? Choice of evaluation paradigm and techniques In order to evaluate systems questionnaires and interviews are not appropriate as the systems have to be evaluated for usability, and not seeing the participant use the system would not be practical. The decision was made to conduct a formal cooperative evaluation in an openended interactive style. This relaxed informal style of interaction would allow a participant to be evaluated whilst attempting to minimise the affect of demand characteristics. Identify practical issues: A major design problem with most evaluations is that the users own setting is not replicated. However, as the systems evaluated were online, these evaluations can be conducted at the participants own computer thus increasing the validity of the results. This does not mean that the participant will not be subject to demand characteristics as being evaluated can often change a participants behaviour, but it does help in reducing the effect. Furthermore due to the size of the study and the limitations on time only one participant will be used to review each the system. Therefore this small sample means that it will be difficult to generalise from the results obtained. Decide to deal with ethical issues Before the evaluation takes place the participant will be asked to sign a consent form stating that they were happy to be used in this project and the aims of the project will be stated. Furthermore, the form will make the participant aware that the data will be kept for no longer than necessary and that they can stop taking part in the study at any time. In addition, the participant will be verbally briefed as to the aim of the evaluation. It will be stressed that the aim is not to see how good their IT skills are, but to test the usability of the system being reviewed. At the end of the evaluation the participant will be debriefed. The participant will be thanked for taking part in the project and a summary of what was found will be discussed. Furthermore the relevance of these results to the project will be discussed. Evaluate, interpret and present data 27 The key is to evaluate the usability problems with the systems rather than the specific data used in the system. Therefore the results of the system will be discussed with the participant to ensure that they have been interpreted correctly. Due to the time constraints these evaluations will not be videoed, instead notes will be taken on what the user was doing and key issues will be discussed. 4.2.2.2 Systems/websites to be evaluated Evaluation of IT tutorials: As the system to be developed will be an IT learning system, some IT tutorials on the web will be evaluated. The tutorial systems to be evaluated are: • • http://www.comptechdoc.org/basic/basictut/ http://www.quia.com/mc/65620.html The basic tutorial on comptechdoc.org has been chosen because it was the first suitable tutorial found that contained subjects similar to the ECDL. Because of the simplicity of the comptechdoc website, quia.com was chosen as the second system to evaluate as it looked advanced and therefore differed in style from the comptechdoc website. Evaluation of existing NHS e-learning system: The nurse to participate in this evaluation has previously used the site when needing research to complete a diploma as part of her continued learning development. The walkthrough will be different to the evaluation of the IT tutorials as the participant has used the system before, therefore the aim is to replicate some of the problems previously experienced with using the site. 4.2.2.3 Evaluation 1: Comptechdoc.org using DECIDE framework. Description of comptechdoc This tutorial takes the user through some computing basics. Although unimaginative, navigation of the system is relatively simple with a scrollable frame on the left hand side allowing the different sections to be viewed. 28 Fig4.1: System homepage Method: In this evaluation the user was asked to complete the following tasks. A full review of this evaluation can be found in Appendix C. Navigate onto the homepage (the web address was provided on paper) View the information on the homepage Try to find tutorial information on the website about the Internet and read through it Try to find tutorial information on the website about files and read through it Explore the system and see if it meets expectations. Results of this evaluation: The participant commented that the site looked clear and it was easy to see what information was on the site. When asked about the scrollable navigational bar the participant commented that it was good that is stayed there as pages were navigated but she did not like having to scroll down to see all the menu items. Many of the comments made were specific to the information that was reviewed and will therefore not be documented here. However, when navigating from start to finish on this site the participant found it simple to go through each stage. The participant also commented on the fact that she was likely to become bored when using the system and that the site “did not look very interesting”. The participant also agreed that it would be difficult to know that she had learnt the information on offer and further admitted that a test providing feedback would be one way of doing this. Conclusion: The system is a very simple website to provide IT tuition. The navigation is clear and the website uncluttered. However, the site is uninteresting to use and provides no facility in which to test the users knowledge. The literature review identified that systems should be authentic to keep users attention and this system could not really be described as authentic, it 29 is fundamentally unoriginal. The most notable finding of this evaluation was that a clear navigational system is appreciated over high-tech graphics and varied use of colours. More specifically a navigational bar that doesn’t change from page to page is favourable as the user feels comfortable they can recover from a navigational error. 30 4.2.2.4 Evaluation 2: quia.com evaluation using DECIDE framework. Description of quia.com The homepage of the site provides a choice between entering an instructor zone and a student zone. Registration is needed to use the system and navigation of the large system is complex. Figure 4.2: System homepage A list of topic areas is shown on the site as can be seen at the bottom of Figure 8.3 and some of these topic areas relate to computing (when “and 100 other subject areas is pressed). The site includes tests such as found in figure 8.4. 31 Figure 4.3 Computer basics test: This is a simple program that asks the user to match devices to their descriptions. The following test was also included in the system, Figure 4.4 Computer and Internet Terminology test This test allowed players to compete against each other. The game asked each player to pick a category in which they would be asked a question. Upon answering the question correctly the user is notified of the success and when answering incorrectly the user is told the correct answer. 32 Figure 8.8: Getting an answer incorrect This system is an advanced system providing a variety of IT tests and games. Method: In this evaluation the user was simply asked to find some IT tutorials. A full review of this evaluation can be found in Appendix B. In order to ensure that the user is on the correct track, specific questions were asked to guide the analysis. These included: What are your impressions of the homepage? Do you think the navigational structure is clear? Did you expect the link to lead you to here? Is the system doing what you expect? What improvements do you think could be made? Do you think test games are useful? Is this site preferable to the last site evaluated? Furthermore when the user became stuck when navigating through the site a number of prompts were used in order to ensure that the user could continue. Results: The site suffered a number of usability issues. The participant found navigating through the site difficult often asking “Why have I got here”, or “how do I get back”. Furthermore it took the participant a while to get anywhere in the system and the participant had to retrace her steps a number of times. The games and tests described earlier were viewed as useful, although the participant commented these games could be viewed as “childish and “a waste of time”. However, these interactive tests seemed to be understood by the participant although some time was taken to wok out exactly how to use the tests. 33 When asked about the feedback provided when a question was answered correctly the participant commented that it is useful to know the correct answer when you have answered a question incorrectly. Furthermore, it was commented that the system was very difficult to navigate through and find information and this was easily observable. Many mistakes occurred when using the site and a few times prompts to help the participant continue were required. The participant commented that although the interactive games were good, with no information to read many of the answers provided were a “stab in the dark”. Conclusion: The following conclusions were made: • • • • Login systems should be clear and easy to use. Navigation through systems should be simple, menu bars should remain consistent and it should be clear how to get back to a previous page. Different formats of tests were difficult for novices. Novice computer users will likely prefer consistency. Feedback on the correct answer when questions are answered incorrectly is appreciated. 34 4.2.2.5 Evaluation 3: NHS database of journals: Description of nhs.dialog.com The following is a review of the NHS site designed to provide medical journals for NHS staff to aid learning. The site had been used by the participant previously when having to find journals to help in writing an essay to complete a diploma. The site requires the user to login using a username and password provided by the NHS. Once logged in the following page is displayed as shown in figure 4.5. Figure 4.5 Page viewed when login into the system: This homepage provides a range of databases which can be selected for searching journals. Furthermore the site contains tips on how to make the best of the search facility provided. When the participant had used the site previously, a number of usability issues were uncovered, resulting in the user not getting enough out of the system. Method: As the participant had used the system previously the aim was to replicate problems with using the system and ascertain why these problems occurred. The participant identified that she had tried to find journals on the effects of nursing patients in isolation with leukaemia.. This search was therefore replicated. Questions were asked as the participant used the system, such as: Did you find logging in intuitive? 35 Have you ever had any problems when logging in? Is the purpose of the system clear? Is it clear where to go to search for journals? Did the results from searches surprise you? If so, why? Results: The site suffered a number of usability issues. The participant commented that their was not a lot of information on the homepage of the system explaining what to do and that it was unclear where to search for journals. The participant found navigating through the site difficult. For instance when going through to try to search for journals the nurse navigated to a page which gave instructions on the best way to search rather than going onto the actual search page. After review of the homepage it was clear that this was documented, although the nurse did not read this information fully. Furthermore this page opens up a new window and doesn’t allow the user to get back to the previous page. This was a significant usability issue for the participant, adding time to the navigation of the site. When searching for information on isolation and leukaemia the participant identified that the problem when previously using the system was getting too many results back. After discussion of this problem it was identified that the participant had not been fully able to restrict the searches, and once the evaluator explained how to do this, the user was able to find the information she had aimed to. Conclusion: The participant and evaluator agreed that systems like this were valuable to use, but this particular system designed for nurses, suffers from a number of usability problems.. The use of explanatory text was often required but nurses often ignored this information. Furthermore, the help facility was not used by the nurse despite the problems that she had encountered using the system. Help and guidance should therefore be integrated into the system rather than separated. The problem in recovering from clicking the wrong button was notable with the participant spending much time trying to get back, especially when new windows opened up. It was decided that navigation should not take the user to windows outside the system. Furthermore, the evaluator and participant agreed that databases such as this should be demonstrated in a class to help nurses use the system. 4.2.2.6 Conclusion to cooperative evaluations A number of points have come out of the review of existing systems. Although many of the systems were quite different from what is likely to be developed, it helps to identify good and bad points from other systems, the knowledge of which can be incorporated into the system to be developed. The following points are therefore identified as recommendations on designing systems for nurses • Homepages should be less cluttered and show more information about how to use the system • Searching should produce fewer results, more specific • Buttons should be clearly labelled as regards what they do 36 • • Help should be integrated with the system as it is unlikely to be read when separated. New windows should not be opened automatically when using online systems The information from this analysis is now used to provide the information in the rest of this chapter. 4.3 The current NHS situation The review into the NHS history provided information on the organisation, but it is of paramount importance that a review of where the NHS is currently is conducted. This review will include information on what is expected of NHS staff, how the training is conducted, the aim and methods of training and the advantages and disadvantages of these techniques. 4.3.1 Statuary requirements “The Nursing and Midwifery Council (NMC) is an organisation set up by Parliament to protect the public by ensuring that nurses and midwives provide high standards of care to their patients and clients.” (Nursing and Midwifery Council 2002) They are a professional body which provides legislation for nurses, they require that nurses complete five study days every three years as a minimum and additions to this vary according to the role of the nurses. Some courses are mandatory such as basic life support and moving and handling. Further courses will be needed according to the job role the nurse is in. The minimum amount of training is not normally enough, with portfolios requiring updating each year. These portfolios can be audited at any time by the NMC so nurses need to ensure they are continually updated. A nurse needs to continually reflect on practice and keep abreast of current evidence as practice is continually changing. To work in the UK all nurses, midwives and specialist community public health nurses must register with the NMC. Practitioners have to renew their registration every three years. Requirements for return to practice nurses: (Nursing and Midwifery Council 2002) • • • If registration lapsed less than three months ago re-registration can be completed by using a “Notification of practice form”. If registration lapsed less more than three months ago the nurse will be required to apply for re-admission to the register. If registration lapsed less more than 5 years ago, and the nurse has not been registered with an overseas regulatory body during this time, the nurse will have to attend a Return to practice course before applying for readmission to the register. Furthermore, the amount of training required also depends on the type of nursing being conducted. If a nurse is moving ward or returning to a specific ward, then there are specific expectations about what training is required. 37 Essentially, the minimal training requirement for nurses is that they must know enough information in order to perform their job. However, in order to gain promotion or be highly successful in their area they must ensure they are continually updated by keeping a training portfolio together which can be reviewed at any time. 4.3.2 How training is conducted This subsection describes how training is currently conducted within the NHS. The NHS have different training strategies for the different grades of nurses, most training is encouraged rather than compulsory, but can ultimately effect the grade (rank) of the nurse. Trainee nurses need to have a diploma or a degree in order to become a registered nurse. This means working as a trainee nurse, studying and taking exams. Existing nurses are encouraged to update to a diploma or degree level as well as completing core courses which are proprietary. Nurses are asked to display evidence based practice; to study in their own time. This can include accessing literature online though databases or by reading journals such as the Nursing Times, or Nursing Standard. In the nurse’s area of practice they have to be confident of the reason they are doing things, they should not work blindly; this knowledge can come from research. 4.3.2.1 Aim of the training The aim of training in the NHS is ultimately to help nurses become better at their jobs and thereby to provide better quality of care. Training will also help nurses maintain compliance with legal requirements as regulations on training nurses is increased. The review of training strategies in chapter 2 identified that training should be linked to the values of the organisation. The NHS values (NHS website) can be found below: The NHS will provide a universal service for all based on clinical need, not ability to pay The NHS will provide a comprehensive range of services The NHS will shape its services around the needs and preferences of individual patients, their families and their careers The NHS will respond to the different needs of different populations The NHS will work continuously to improve quality services and to minimise errors The NHS will support and value its staff Public funds for healthcare will be devoted solely to NHS patients The NHS will work together with others to ensure a seamless service for patients The NHS will help to keep people healthy and work to reduce health inequalities The NHS will respect the confidentiality of individual patients and provide open access to information about services, treatment and performance Training should help nurses complete their jobs and although this is normally the case, this is not always true. An example provided was a training day set aside on NHS policies. Although a quick round up of NHS policies may have been useful, this went into too much detail and was not beneficial to the nurses “hands on work”. It is essential, with such 38 constraints on the amount of training days provided that training days are relevant to nurse’s everyday jobs. It is important to review the aim of training in this project, because any deliverables will need to fit in with the NHS’ aim for their training. When researching training strategies in organisations it was clear that training should fit in with the values held by the organisation. So whilst the values may not seem entirely relevant now, they should be considered with the development of any new system 4.3.2.2 Method of training In chapter two different methods of training available were reviewed and in chapter three, methods of e-learning were reviewed. In this section, the methods in which the NHS uses to train their staff will be reviewed. These include the following: 1. 2. 3. 4. 5. Self-directed learning Study days Collaborative learning Online resources Trainee nurse training/on the job learning Self Directed learning: The most common form of training seems to be self-directed learning; nurses ensuring they read and are up to speed on current practice. This self directed learning is performed by most nurses who want to improve their skills and allows them to keep up to date with the latest medical practices. This research can prove very useful for nurses in their daily job. For instance, the norm has always been that any wounds must be cleaned to stop infection. However, research has show that cleaning the wound can slow down the healing process. Nurses have challenged this on wards and now cleaning of the wound is only completed if it is really required. Most learning comes from personal research rather than directed learning. Most of this learning is completed in nurses own time and can include reading journals such as the Nursing Times and Nursing Standard. Study days: Study days have been discussed already, but here the focus will be on how they relate to the NHS’ training strategy. Although nurses have to attend five study days within a three year period, some will not do any for a year, ranging to a maximum of about ten training days per year. Some study days are general compulsory courses but others will be specialised to the job, for instance, a course on diabetes. Some nurses will not attend these courses, which will not give them an advantage and allow them to go up a grade and therefore attain more responsibility. Collaborative learning: In section 2.4 collaborative learning was highlighted as an important feature of any training strategy. It is important to investigate how collaborative learning is supported in the NHS and how this aligns to other methods of training. 39 During the study days discussed above, nurses did get the opportunity to work collaboratively. Due to nurses being at different stages of their careers it is not always the case that nurses will be on the same courses as their colleagues and therefore often do not know the other nurses. However, the study days were often interactive and working in groups was encouraged. The University of Southampton (where diplomas are taken for nurses in the Hampshire area) use a tool called “Blackboard” but most nurses were unable to understand how to use the tool. Blackboard can be a good way of encouraging people to collaborate; ask questions and share information, but it seems that the concept and how to use the tool was not explained appropriately to the nurses using it. Online resources: A review of an online NHS resource is undertaken earlier in this chapter. Trainee nurse training/on the job learning As with most jobs, the majority of the learning comes from doing the job. It was estimated that 75% of all learning comes from doing the job and the rest is a mix of some of the other training methods. This subsection has outlined the training methods used by the NHS and these will need to be taken into account when designing any new system or making recommendations on NHS strategy. 4.3.3 Advantages and disadvantages the NHS training strategy This section includes a brief summary of the advantages and disadvantages of the way nurses are trained in the NHS. There are clearly some advantages to the way nurses are trained. Firstly, it seems that nurses generally felt comfortable doing there day to day duties, so the medical training they receive seems to be serving its purpose. The study days are often interactive which can make them interesting and can be a nice break from day to day work. The training nurses go through can make the job “more interesting” and helps them to see “the wider picture”. This means they can see more than just inside their ward and the day to day tasks that they undertake, which can improve their morale, and also lead to improved behaviours. One disadvantage of the way nurses are trained is that a lot of training seems to be expected to be completed in nurses own time. As well as working unsociable hours, most nurses are females who have families, so fitting in time to do this is difficult. Some of the learning is completed during annual leave, so it doesn’t really fit into the pattern of work and can put strain and stress on the trainee, whilst in severe cases it could encourage the trainee not to do the training. Nurses have even gone sick to get work done to help with their diplomas, which is part of their nursing training. Financial constraints have led to a reduction in study days available for nurses. Having less training available is a huge disadvantage for any employee wishing to learn. Furthermore it seems that expenses are not catered for when a nurse is training. In most organisations, expenses are catered for, but in the NHS costs such as travel and parking are not. This can lead to nurses avoiding training or even feeling resentful of the situation and financial implications imposed. 40 In chapter 2 the different categories of learners was identified by Noe. Noe suggested that people born between 1945 and 1960 (which will include many return-to-practice nurses) seek fairness in learning. If their own time is consistently used for training and training costs are not catered for, these return-to-practice nurses are likely to view this as unfair; therefore the benefits of the training will be reduced. Most NHS staff are not experienced in the use of IT, so using e-learning systems can be difficult, certainly initially. When completing courses such as diplomas, even using tools such as Microsoft Word can be a struggle. Essays for a diploma have to be completed on a computer and need referencing, double line spacing and page numbers; all of which is a struggle for some nurses. Despite IT courses being available within NHS, most nurses have not taken advantage of this due to time limitations; with the likelihood that it would take place in their own time. Clearly the information on NHS specific training strategy above will be vitally important when completing this project. This information will assist in specifying the requirements for an e-learning system to be implemented for the NHS. 4.4 Future NHS plans/e-learning It is clear that evidence-based practice is and will continue to be an important aspect in the development of training. The exact future of NHS training is out of scope for this project, but it is likely that training regulations and professional bodies will continue to have high expectations. The NHS is also embarking on a few projects that will require their staff to be IT literate. Computers can already be found on wards and NHS libraries, but the NHS is taking further steps. The department of health outlines many improvements and spending on IT to aid nurses (Department of health website). Furthermore an interview with a nurse (documented earlier) suggested that in many hospitals moves have been taken to move nursing and medical notes onto IT systems so that records can be kept in a more organised and accessible fashion. A nurse interviewee suggested nurses were aware of these changes and some hospitals had already seen this implemented. In addition, it was identified that areas such as drug rounds may also become computerised in the future. It is clear from the above that IT is something that nurses will not be able to get away from as the NHS continues to invest and implement systems. 4.5 Conclusion This section supports the argument that e-learning could be a successful tool for training within the NHS. The numerous advantages that e-learning offers together with the challenges the NHS meets financially supports the idea that performance certification could become invaluable for use within the NHS. Any e-learning system deployed should surely provide a mechanism for testing, marking and storing results of examinations such that certification could be provided through the system. 41 This analysis of training within the NHS has been useful in reviewing information about training and learning which will be essential when designing a suitable system. Unfortunately the scope and size of the project is such that it will not be possible to provide a full training solution for the NHS. However, the scope of the project is sufficient that an e-learning system should not be the only deliverable from this project. The requirements specification of the project is discussed in the following section in which the system to be built will be discussed along with the project deliverables. 42 Chapter 5 5 Requirements specification The literature review and requirements analysis have changed the scope of the project significantly from the initial assessment. The project will focus on providing a general IT training solution for the NHS, rather than simply an online training system. Furthermore, peer support was also identified as something that could help nurses with learning and ensure that they helped each other to complete tasks. This chapter will look at where requirements are gathering from, how they should be specified before detailing the specifications. As stated at the beginning of this project: “The aim of this project is to research an appropriate strategy for training NHS staff to use IT. The main deliverable will be an online IT training system, with recommendations on how to help NHS staff to use IT effectively.” At a high level an IT e-learning system will be built for return-to-practice nurses to complete the ECDL. Upon building this e-learning system, studies into the usability of the system will be undertaken (chapter 7). Essentially these studies should provide the following information. • • • • • • If and how nurses navigate the NITS (NHS IT Training System) system * What features they liked and what features they disliked The general advantages and disadvantages of the NITS system The usefulness of peer support when using the NITS system Whether they managed to course errors Whether they needed help when using the system * The name NITS was chosen, an acronym for NHS, IT Training System. After these studies take place, questionnaires and interviews will be conducted to gain information on the user’s experiences with the system and information on their opinion on NHS training. This will specifically look at: 43 • • • • What IT training they have received in the past What IT systems they have used How they thought they got on with the NITS system Whether they liked using e-learning The results from these evaluations and questionnaires will be added to a recommendations document; a deliverable to the NHS. This document will go through • • • • 5.1 Recommendations on training strategy (suitable framework) Recommendations on IT training Recommendations on using the NITS system Conclusion and evaluation Source of requirements Fig 5.1 Source of requirements The following diagram summarises the information above, showing how the requirements were derived. Interviews with nurses Initial set of requirements Literature review Interviews with NHS training staff 5.2 Specifying the requirements The requirements documented below follow the standards set forth by Sommerville (2001) and Preece et al (2002) . They are split into functional and non functional requirements. Preece et al refined these classifications later on, in essence for this study; the following requirements categories will be used. • Functional requirements 44 • • • Non functional requirements Usability requirements Database requirements Usability requirements were added because of the importance of the usability of the system.. Database requirements were added to ensure the use of the database was clearly specified. 5.3 System requirements The requirements in this section go through the requirements for the online training system to be used by nurses. Further to methods used already to help derive these requirements, a number of use cases were completed, all of which can be found in Appendix D. Furthermore for the requirements elicitation, each requirement listed will name the source of the requirement and rationale as to why it was necessary. Use cases: The following scenarios were derived to have use cases documented and the same scenarios will be used later to test the system. These scenarios were chosen because they were the main scenarios identified that users would want to do. They also cover most of what theoretically could be done in the system to be developed. Standard user: • Register as a user of the system • Login to the system • Change your password • View lesson material • View peer support • Take a test • Receive the results of the rest • Submit feedback on the test and the lesson material (for others to use) • View all of your test results • Complete the final test • Logout Admin • Administer test questions • Review peer support of a particular lesson • Delete a test from a user’s record to all them to retake. Figures 5.2 and 5.3 are examples of these use cases. The use cases follow the standards set out by Cockburn (2001 p121). 45 Fig 5.2: An example use case for taking a test within the e-learning system. Take Test After reading a chapter the user will be required to take a test Used for nurses requesting IT training USE CASE #6 Context of Use Scope Level Primary Actor Stakeholders Interests User Goal level An NHS nurse and Stakeholder Interest An NHS nurse Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Learn how to use IT within there own time NHS training manager Wants trainees to learn how to use IT effectively. • The nurse is registered to use the system • The nurse is logged on • The nurse has read the material required and clicked on the appropriate test button. Test data recorded upon completion of a test Nurses receive certificate for passing a course…and feedback for future learning Nurse navigates to the appropriate test link Step 1 Action The system keeps session information and recognise the nurse. 2 Nurse presses the appropriate chapter 3 Nurse clicks on the test 4 Nurse completes test and submits data 5 Nurse is presented with marks and feedback on the results. Step Branching Action If the nurse does not pass the test, an alternative test must be provided. This will involve changing the test questions using the administration tool. Variations The nurse clicks on the test button without viewing the lesson data (they may have looked at this data previously. In this example for instance, is was easy to see that the nurse will need to be presented with results when submitting data for the test, rather than having to navigate to a separate page after data is submitted. 46 Fig 5.3: An example use case for logging into a system. USE CASE #2 Context of Use Scope Level Primary Actor Stakeholders and Interests Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Logging in The nurse will need to log in to progress from the homepage to the tutorials and tests. The whole NITS system. User Goal level A nurse. Stakeholder Interest A nurse Wishes to access the NITS system System administrator Only to allow access to authorised nurses • The nurse is not logged in • The nurse is registered to use the system • The nurse has navigated to the login page An unsuccessful login does not stop the user trying again. If login is authorized, the nurse is directed to the logged in homepage which enables the nurse to navigate to different the tutorials and tests. Nurse navigates to the login section of the NITS system Step Action 1 Nurse enters username 2 Nurse enters password 3 Nurse presses the ‘submit query’ button, 4 5 Step 3a 4a 4b The system validates the login The logged in homepage is displayed. Branching Action One or both fields are left blank 3a1 The ‘Log In’ page is redisplayed 3a2 The user is told why the login was unsuccessful. The username entered is not recognised by the system 4a1 The ‘Log In’ page is redisplayed 4a2 A message is displayed telling the Nurse that their login attempt was unsuccessful The given username exists in the database, but the password does not matched, 4b1 The ‘Log In’ page is redisplayed 4b2 A message is displayed telling the Nurse that their login attempt was unsuccessful Variations 1a None This use case shows exactly which error messages will be required in the login script. Use cases such as this did help in determining how things would work in the system, particularly what steps users would take and the variations and extensions that could take place. All 47 requirements gathering techniques have now been concluded and the requirements will be specified. 5.3.1 Functional requirements Definition: ‘The functional requirements are statements of services that the system should provide, how the system should react to particular inputs and how the system should behave in particular situations. In some cases, the functional requirements may also explicitly state what the system should not do.’ (Preece et al 2002) 1. The system homepage should provide information on the purpose of the system, how to login etc 1.1. All pages should follow interface requirements mentioned later. Rationale: The homepage is the most important page in most websites/online systems. The design should be clear so that nurses feel confident in using the system. Furthermore, the homepage should provide detailed information on all aspects of the system so that users are aware of what they should do. In chapter 2 it was identified that clear goals and professional relevance of the training is important, therefore the goals and relevance must be explained to the nurses. Source: This was an implicit requirement derived for most systems; however the point was highlighted when conducting heuristic evaluations on systems that clear explanations of system tasks are required. 2. The system should identify nurses who login. 2.1. New users should be able to register to use the system from the homepage 2.2. Users should be either standard or administrative users, giving them access to different parts of the system. 2.3. The system must authenticate the password, checking against the username before allowing access to the system. 2.4. The system should recognise the user after entry. 2.5. The username should be unique; therefore two people cannot use the same username. 2.6. The nurse should be able to change his/her password. 2.7. The user should use their e-mail address as their username. Rationale: The main use of the login system should be to identify users in order for them to take tests. In chapter 4 it was identified that nurses must keep up-to-date portfolios of training conducted, therefore the system must be able to recognise the user to provide test scores. Security is not the main concern for this project, as the NHS would probably want to use there own security system should the system be implemented. Therefore the login system is kept simple, with the key requirement being usability. Source: During development it became clear that entering personal information on each test should affect the usability of the system. With usability being identified as a key requirement in the literature review and in interviews, it was clear that the system should identify the user as they use the system. This prompted the use of a login system. 3. The system should have a sufficient tutorial session covering similar topics to that used for the European Computer Driving Licence 48 3.1. This will in the form of HTML 3.2. This will be in prototype form, simply to provide information that users can then be tested on Rationale: This data is essential in order for nurses to learn information so they can take the relevant tests. Ideally the actual content from the ECDL should be used. The data in the system should be authentic, see chapter 3 on e-learning which highlights that users will not put up with poor content; the data should be such that it keeps nurses attention. Source: This requirement was derived from interviews with NHS representatives in which the ECDL was regarded as the best training course to be put into an e-learning environment. 4. The system should have an admin section that enables a designated administrator section to change the questions and answers used in the tests. 4.1. The admin login section should be separate from the standard login section. 4.2. All questions, answers and required scores for each test should be editable by using the admin section. 4.3. The administrator should be able to view and delete users test scores. Rationale: This will ensure that the system remains data independent and theoretically could be used to teach other topic areas. This is because it is unlikely that a detailed knowledge of what nurses need to be trained in will be gained, therefore the ability to change this training material is important. Chapter 3 identified that e-learning systems should be easily maintained in order to stay up to date. As IT knowledge and techniques are evolving it is important that the data should be maintainable. Source: During interviews with NHS training representatives it became clear that it would be difficult to get specific content on training courses in order to put onto an elearning system. Therefore it was decided that data should remain independent from the system as much as possible. This was further enhanced by the review of e-learning systems suggesting that data should be maintainable. 5. The system should test the user on the information provided in each chapter 5.1. The system should provide the user with a series of multiple choice questions relevant to the chapter, which should be completed by the user. 5.2. A record of the results of the test for the user should be stored Rationale: The tests will be the main programming effort in developing the system, but is paramount to producing a realistic system for nurses to use. The tests should be data independent as the NHS may have other required areas for e-learning which the system could incorporate. Furthermore, chapter 2 identified that clear training goals should be identified. Having certification to aim for will help nurses focus their efforts towards the goal of certification. Source: Interviews with NHS training representatives concluded that the system should provide a mechanism for testing. 6. The system should score the user and give information on whether they have passed or failed according to a pass rate set within the system. 6.1. The user should gain information on what was the correct answer to all the questions 6.2. Upon passing, the users record should be updated to show they have passed 6.3. Upon passing all chapters a certificate should be available. 49 6.4. The user should only be able to take the test again after the administrator agrees and deletes their current test from the system. Rationale: It is imperative that this information is stored and that users are told whether they have passed or failed. Essentially, the results of these tests will determine whether the user passes of fails the course. The user should understand what the correct answer was so that they know for the future. Chapter 2 identified reinforcement theories, which suggest that reinforcement can affect the level to which learning takes place. Providing the nurse with the results of tests, meaning that they pass or fail and therefore their actions are reinforced or not. Source: During interviews it was identified that a test system should be developed. Having the users score data stored is an implicit requirement of a test system. The cooperative evaluations reinforced this and it was further identified that it is important for users to know when and why they get information incorrect. 7. The system should provide a help facility for people not needing to use the system 7.1. The help facility should answer common questions, and should be easily editable as more problem areas are discovered. Rationale: The system should be usable and therefore if someone is stuck they should be able to go somewhere for help. Chapter 2 suggested that tutors should be provided to help with this learning; it is therefore likely that the tutor could provide any help needed. Source: This was an implicit requirement for building an e-learning system. 8. The system should allow the user to leave comments and suggestions on the chapter for other users to view 8.1. After completing the test the user should have the opportunity to leave some comments and suggestions for others to use when they have completed a test. 8.2. After completing the test the user should have the opportunity to leave the administrators some feedback on improvements that should be made to the system. Rationale: The comments and suggestions will help determine if peer support will be a valuable tool in helping nurses to learn about IT. Nurse’s willingness to leave this information will be invaluable in determining the likelihood of the success of peer support. Peer support was identified as being important an important part of collaborative learning in chapter 2. Source: The literature review identified that the use of peer support can aid learning and help people solve problems. 9. The system should allow the user to view other peoples comments and suggestions on the chapter 9.1. When viewing lesson material, users should be able to view the peer support mentioned above, before and after taking the test. 9.2. The page that displays the information should provide the viewer with information on how to contact the person by e-mail in order to gain help. Rationale: One of the objectives of the project is to see the success of incorporating peer support data into an e-learning system. Nurses must have the ability to view any information that is left. As mentioned peer support was identified as aiding learning in chapter 2. Source: These are implicit requirements of a peer support section. As mentioned above, peer support was identified in the literature review as being important. 50 10. The system should provide a page in which the user can view all of their test answers and scores to date. 10.1. This should be viewable from the main system menu 10.2. The page should display scores and answers to tests taken. 10.3. When four tested have been completed the user should get a certificate Rationale: Users will want to view the tests they have taken and the marks they have received. Chapter 2 identifies reinforcement theories in which the user needs feedback and reinforcement to aid learning and this requirement will ensure that feedback is available to users. Source: Identified within the literature review. These functional requirements will help be used to build and design an e-learning system to train nurses. When testing, they will be referenced to check that every requirement derived for an e-learning system for nurses has been met. When looking at these requirements it is clear that the system is being built for nurses, with the emphasis being on usability. Usability and interface design is discussed later. 5.3.2 Non-functional requirements Definition: ‘The non-functional requirements are constraints on the services or functions offered by the system. They include timing constraints, constraints on the development process, standards, etc’. (Preece et al 2002) The non functional requirements in this section go look at usability, graphical user interface, and timing. Where stated, more detail can be found in Appendix B. 5.3.2.1 Usability requirements The system should be useable for novice computer users. Shneiderman’s user interface design principles found in Sommerville (2001) should be applied. Two of Shneiderman’s principals seem particularly relevant to the project, although the full list can be found in Appendix B: o Minimal surprise This means that the system should never react in a way, which will confuse the user. As discussed previously nurses can sometimes lack confidence using computers, so the system should not surprise them, it should do what they expect in a step by step manner. This therefore means that a Trade off between system feasibility and operation should favour operation. o User Guidance This is where the system gives the user helpful support while they are using the system. This includes properly worded error messages, using vocabulary that the nurse is familiar with to help them navigate through the system with ease. During cooperative evaluations documented in chapter 4 comments at to struggling 51 navigating through systems were made. Furthermore, nurses commented that the use of text to explain to the nurse what to do is required. Further to Shneiderman’s usability principals the following requirements have been derived: • • • • The system must not use technical jargon. Although nurses understood words like logged on Navigation should allow any system features to be accessed easily – see graphical user interface requirements. The system should perform quickly even on lower speed connections. The design of the system will be paramount to how usable the system is, for instance, navigation will be of primary importance. However, having textual instructions will help nurses navigate through the system and will add to usability. It is important to note that usability cannot be added on; it must be designed into the system (Sommerville 2001). 5.3.2.2 Graphical user interface requirements This leads on from the usability requirements in that the interface must be built specifically with usability in mind. A full list of graphical user interface requirements can be found in Appendix B, but some of the requirements that were felt to be vitally important, but also that may be specific to systems designed for nurses or other inexperienced users are documented below. For instance usability testing with nurses showed that nurses do not necessary notice when processing on a system is taking place and can click again or become impatient. It therefore seems sensible to ensure that processing time is minimal and state change is shown clearly upon completion. Navigation being simple is also very important as nurses are not experienced users and may therefore get lost in a system or become frustrated at an unclear system structure. Information from usability testing with nurses showed that nurses can get lost within systems and prefer to follow a logical progression in the system. I.e. “next page”. 5.3.2.3 Timing requirements The project must adhere to the project constraints placed on it by the computer science department. Therefore the project must be completed and handed in by the 8th May 2006. 52 5.4 Conclusion to requirements This completes the requirements section of this project, which covered chapters 4 and 5. The requirements analysis found in chapter 4 provided information that requirements could be based on. It provided an understanding of NHS requirements for training and structures currently in place. Chapter 5 then specified these requirements for the e-learning system to be developed for the NHS. During this requirements stage it was clear that interviews were crucial to the project. Whenever requirements were unclear the advantage of being able to consult NHS staff was paramount. Use cases were valuable in helping to visualise common scenarios in which the nurse would have a requirement to be met. However for the amount of documentation and time they took it may have been preferable to scale down the amount of detail in the use cases. For instance, the primary actor was nearly always the nurse using the system and documenting this each time became tedious. Heuristic evaluation with nurses using different tutorial systems was a valuable insight into the way that nurses use systems. When conducting evaluations it was interesting to see what nurses found difficult or not usable in systems, this helped contribute to many functional requirements and would be a technique that should be recommended to others looking to build a system specifically for nurses to use. The next step is therefore to design the project to meet the requirements set forth in this chapter. 53 Chapter 6 6 System design In order to ensure that the system meets the requirements and is usable for nurses, it is important that nurses are consulted throughout the design and implementation of the system. The system design stage covers all areas of designing the NITS system. With self efficacy/confidence being identified early on as a requirement for people using IT, the functions of the system will be designed transparently, so the inner workings of the system are not obvious. Rather the information they require will be presented clearly. 6.1 Design decisions It seemed unlikely that anyone could be interviewed in order to discuss architectures that should be used for implementation of a system. In addition this was outside the scope of the project in that it was unlikely that anything designed would actually be implemented within the NHS, the real aim of the study was to find out how NHS staff interacted with IT systems and to gain knowledge in order to make recommendations on how to help nurses use IT. Therefore the system to be designed and implemented would be a high fidelity prototype. This would be a system that could be adequately used by nurses to fulfil the requirements, but it would need further work in order to go live in the NHS. When designing this high level prototype, it is important to ensure that a simple test system, available anywhere is not built as this will add little value. The test system would have to be designed specifically with nurses in mind, incorporating what has been learnt about the way nurses work into the system. Standard usability features available in other systems will have to be incorporated, but the system must offer other features not always available on other systems, that nurses can use. The requirements stated that the system should cover training material provided on the ECDL. However, the realisation that the implementation of a course of this size is unrealistic in a university project, coupled with the inability to get the ECDL content meant that a few basic IT tests will be taken from websites that matched chapters that were already in the ECDL. In addition, the ECDL covers 8 chapters, whereas only 4 chapters will be designed 54 into the NITS system, as the additional development of lesson material that would not be viewed would add no value. See Appendix B for information on what is in the ECDL; the chosen chapters are circled. Although some time will be required to review the tutorial data and check it is appropriate, the content of the tutorials is not of paramount importance in this project, rather the way that the nurses used IT is of interest. 6.2 Initial Prototype design and implementation As discussed nurses often have busy lives and getting nurses time can be difficult, therefore it was essential to ensure that the NITS system was ready and tested when nurses used it. Therefore, in line with the systems lifecycle an initial prototype with a few inexperienced computer users was conducted, so they could use the system and offer feedback. This feedback will then be reviewed and changes will be incorporated into the final system. The main aim of this initial prototype was to check for errors that were not spotted by the developer. This prototype will represent the core functionality of the final system; essentially this will be designed without the administration section, so that questions and answers are hard coded. Furthermore only one chapter and test will be included in this prototype as opposed to the four that will be used in the actual implementation. The prototype will also not provide a peer-support facility. This will still allow the participant to interact with the system in the same way but will represent only core functionality. This prototype will be designed as a throw away probe with the purpose of assessing the design concept and seeing how users find accessing the system. This prototype will be used by one inexperienced computer user and a nurse. The nurse was used being the end user whilst the inexperience computer user was used in this evaluation due to the ease of accessibly of this person. Some screen shots of the prototype are shown below, but essentially this prototype looked the same as the main system. As the admin was the main functionality removed this did not affect the way the user interacted with the system. Figure 6.1: Page displayed when user logged in 55 Figure 6.2: Page displayed when user takes a test Again the DECIDE framework discussed earlier was used to evaluate this prototype. Determine the goals: Find any existing software errors Evaluate whether a help system would be used Evaluate if any additional features are required Evaluate issues with the usability of the system Explore the questions: These questions will include: • Were the results expected of actions expected? • Was the navigational structure clear? • Were standards used that the user did not understand? • Did you encounter any error messages? • Did you require help whilst using the system? • Did you click the help button on the menu bar? Choice of evaluation paradigm and techniques An informal evaluation technique in which the user is given some tasks and is asked to report back with feedback was used, due to the speed at which this prototype needed to be piloted. As the system was web based, these evaluations were conducted remotely, with MSN messenger being used to communicate. 56 Identify practical issues: The sample used in the study is small, with only two people being used. As discussed previously small samples can make it difficult to generalise from results. Furthermore one of the participants will not be a nurse, which is the end user group to be catered for. Furthermore due to the fact that the evaluation feedback was received over MSN, the participants were not observed whilst using the system. Therefore honesty will be relied upon when receiving feedback. Decide to deal with ethical issues Due to the informal nature of this study ethical issues will not be a major issue. The participants were briefed at the beginning of the study and could leave at any time. Evaluate, interpret and present data The key was to evaluate the usability problems with the systems rather than the specific data used in the system. Furthermore, this study was not meant to be a detailed and precise look into how the nurses used the system, it was used as a preliminary test. Findings on data will be presented by the participant to the evaluator. Method: The participants were asked to: 1. Register and login to the system 2. Read some tutorial information – 1 page 3. Answer questions in a test 4. View the results of the tests They were asked to provide feedback as to any issues they uncovered or any suggestions that they had. Results: No major issues were identified by either participant, the main feedback received was about he content of the information rather than the structure of the system itself. However, it was identified that the user did not know that they should register using their e-mail address as their username. This will cause problems when peer support information is used as the e-mail address should be used for nurses to contact each other. Therefore the registration page was updated to add information that their username should be their e-mail address. 6.3 System architecture From the requirements chapter it is possible to identify the main sections which are required in the system. The following give details of the overall systems architecture. • • Login – This is the first stage of the process in which the user will be identified as either an admin user or a standard user. Furthermore this login section relates to functional requirement 2 which states the requirements for logging in. Admin section – This is the second stage of the process for admin users who have logged in from the admin page. This relates to functional requirement 4. 57 • • • • Main menu – This is the second stage of the process for standard users who have logged in from the standard login page. Therefore this is the first screen that will be presented to users once they are authenticated. From this the nurses will be able to access the main features of the system as described in the navigational design section. Tutorial information – This will most likely be the third stage of the process dependant on the tasks standard users choose to undertake. This will include tutorial information on a number of basic computing topics. This relates to functional requirement 3. Test data – This would most likely be the fourth stage if the process. Nurses would answer electronic multiple choice questions that enable the nurse to select the correct answer. This relates to functional requirements 5 and 6. User results – This section would produce a stored record and display of the results of the tests taken so far. This relates to functional requirement 5. Figure 6.1: A high-level model of the systems architecture, detailing each of the sections and the interactions between them. Admin section Login Homepage 6.4 Lesson data Test data User results Navigational design This navigational design section follows on from the high level model mentioned previously by breaking the system down into states in which navigation between system components is defined. The navigation of the system will be complex due to the large amount of pages that will need to be incorporated. The following system states will be used: Loggedout: Non-logged in users should not be able to access the features of the system. In-fact the only areas they should be able to access are pages such as registration and login. Loggedin Users who have logged in are able to view all (except admin) areas of the system. This means that they should be able to access test material, lesson material and their individual test results. Adminlogin Admin users will have access to pages of the system purely designed for administration of the system. They will not be able to access the same areas as standard users and therefore will not view lesson material or take tests. 58 6.5 Database design The database section of the NITS system should adhere to the ACID properties as stated in Silbershatz (2002). Each table in the database should take into account each of the four properties associated with ACID in order to ensure there is no data inconsistency and that the actual data is unambiguous. The database design describes the ACID principals and shows how the database was designed to meet each of these criteria. With the system requiring the use of a database, standard practice would be to produce lots of UML database models. However, the database used will be very simple and it seems unnecessary to produce lots of diagrams. For instance an entity relationship diagram would be used to describer the relationship between the tables. However, as the database contained very few relationships and lots of entities, this would have added little value other than adding to the documentation. However, a schema of the database design can be found (Figure 6.2). A schema has been used as it shows very clearly all the entities to be in the tables of the database. There will be four different tests, so the test# table and the test#_answers table will become 8 tables in all (see implementation section). Normalisation Tables should be normalised to third normal form (A Simple Guide To Normal Forms Website). The table is in first normal form when all occurrences of a record type must contain the same number of fields. The table is in second normal form when a non-key field is a never a fact about a subset of a key. Third normal form is violated when a non-key field is a fact about another non-key field. The table is designed to be normalised into 3rd normal form as shown on the following page. 59 Figure 6.2: Initial Schema Diagram User Username Forename Surname Password test1_score test1_pass Admin Teststaken Test2_score Test2_pass Test3_score Test3_pass Test4_score Test4_pass Test5_score Test5_pass Test6_score Test6_pass Test7_score Test7_pass Test8_score Test8_pass Test#_answers Username q1 q2 q3 q4 q5 q6 q7 q8 q1fb q2fb q3fb q4fb q5fb q6fb q7fb q8fb Generalfb Adminimrovements 60 Test# Test#question1 Test#answer1 Test#answer2 Test#answer3 Test#answer4 Test#question2 Test#answer21 Test#answer22 Test#answer23 Test#answer24 Test#question3 Test#answer31 Test#answer32 Test#answer33 Test#answer34 Test#question4 Test#answer41 Test#answer42 Test#answer43 Test#answer44 Test#question5 Test#answer51 Test#answer52 Test#answer53 Test#answer54 Test#question6 Test#answer61 Test#answer62 Test#answer63 Test#answer64 Test#question7 Test#answer71 Test#answer72 Test#answer73 Test#answer74 Test#question8 Test#answer81 Test#answer82 Test#answer83 Test#answer84 Test#question1answer Test#question2answer Test#question3answer Test#question4answer Test#question5answer Test#question6answer Test#question7answer Test#question8answer passmark 6.6 Conclusion The design decisions in this section are important as they define the way the system will be built. Here it was decided that it was unrealistic to try and implement the full ECDL programme on an e-learning system. This decision was essentially taken due to the difficulty in attaining the ECDL content and the time constraints on the project. The prototype was useful in gaining some information about the system and spotting errors that could me made in development, although the emission of peer-support from this prototype was not a sensible decision as this section is was one of the key section in the NITS system. The system design is now complete and should enable implementation in a chosen language. The design of the system is such that it should meet all the requirements set forth in chapter five. 61 Chapter 7 7 System Implementation This section goes through the whole development process from the coding itself to the testing and pilot studies that eventually result in a final implementation. When the system differs from the requirements or design, an explanation for this will be clearly stated. The NITS system can be found at: http://www.oligarchic.co.uk/jackland/ A file listing can be found in the appendix, and relevant code can be found on the above site and on the attached CD. 7.1 Implementation language and system architecture The system was written with PHP and SQL embedded in HTML. The system needed to be web based so a web based language was required. Three languages came to mind, being PHP, ASP and ColdFusion. All three of these languages would have been suitable to meet the requirements but PHP was chosen because of the open source nature of the language and therefore is could most likely be implemented in the NHS. Furthermore, with previous coding experience in PHP, PHP was preferred to using ASP. ColdFusion was considered, with the coder having experience using it the idea was appealing, however the decision to stick to a well known open source language like PHP was taken. ColdFusion also needs dedicated ColdFusion servers, something that could not be facilitated. 7.2 System setup Finding available webspace was relatively straightforward and had PHP and SQL already available for use. Therefore development could take place directly onto this webserver throughout the project. Dreamweaver was used for writing code and was setup so that files could be uploaded to the server quickly. PHP MyAdmin was used to check the data directly in the database. 62 7.3 Approach to code implementation The coding of the system needed to be readable and efficient. Whilst coding, it was clear that there would be many lines of code, many of which would include the same information. For instance the main menu on the website stayed the same in the respective three states mentioned in the system design mentioned in section 4.14. Navigation was designed such that users who were logged in would see one menu, logged out users would see another again and finally logged in admin users would see a separate admin menu. These menus would be on many pages and during implementation it was decided to include .inc files for the menu bars, which cut down considerably on the amount of code on each script. The navigational design is discussed later. Furthermore, .inc files were used in other areas to cut down the amount of code in each file, test_results.php is a good example of this. Code that was constantly referenced in this way such as connecting to the database was also stored in a connect.inc file as shown below. Listing 7.1: Code for connecting to a database 1 2 3 4 5 6 7 8 9 10 <?php $db = mysql_connect('localhost', 'ol1garch_jack1an', 'password'); if (!$db) { die('Not connected : ' . mysql_error()); } $db_selected = mysql_select_db('ol1garch_jack1and', $db); if (!$db_selected) { die ('Can\'t use foo : ' . mysql_error()); } ?> Code was commented throughout the project, to the point that anyone with a basic understanding of PHP and the requirements of the system could understand all code in the system. A few scripts were not commented as they were exactly the same as other scripts bar the fact that they were for the second test rather than the first. 7.4 Setting up the interface The colours were chosen because they were neutral and not offensive, although the choice of these was largely down to personal preference. The fonts and other information needed for the interface were stored in a css file called projectstyle.css. This small file set the fonts, table fonts etc required so and this file ensured that the text was always the same on every page, meeting the user interface requirement of consistency. 63 Font: Comic sans was chosen mainly because it is easy to read, but it also seems a little less formal than the other fonts commonly used such as Arial. It was hoped that using a “fun” font would provide a calming effect for nurses using the system. 7.5 Implementing the navigation menu bars The requirements stated that the navigational menu bar should always be visible and allow users to access all system features they require. As mentioned in the navigational design (section 4.14) menu bars were implemented for logged out, logged in and administration. These menu bars were implemented using .inc files to save time changing code if the navigational design was changed. The following files provided this code. • • • Loggedout.inc Loggedin.inc Admin.inc Loggedout Non-logged in users can only access certain parts of the system. The file loggedout.inc provides the menu for this part. Figure 7.1: The navigational bar implanted for logged out users. Loggedin Users who have logged in are able to view all (except admin) areas of the system. The file loggedin.inc provides the menu for this part Figure 7.2: The navigational bar implanted for logged in users. Adminlogin Admin users have access to separate parts of the system and therefore cannot take tests. The file adminlogin provides the menu for this part of the system. Figure 7.3: The navigational bar implanted for logged in admin users. 64 Therefore users in any of these three areas can access any of the pages listed on the menu bar directly. The spider diagrams on the following pages explain the navigational flow through the NITS system. 65 Fig 7.4 Data flow for users not logged. When viewing the site at www.oligarchic.co.uk/jackland/ the default page is index.php, the homepage of the site which can be viewed when not logged onto the system. In essence, this page provides instruction and access to menu bars that enable you to register or login as a standard user. If logging in is successful the user is taken to either the logged in homepage or the admin homepage, of which the data flows can be viewed on the following pages. index.php (Main homepage) register.php Allows the user to register to use the system. system login.php Allows registered users to login. admin_register.php Allows the user to register as an administrator admin_login.php Allows registered admin users to login. Admin_login_result.php Authenticates the login Login_result.php Authenticates the login Admin_home.php Home.php 67 Fig 7.5 Data flow for users who have logged in. When logged into the site as a standard user, the main parts of the system can be accessed, such as taking tests, viewing test results, leaving peer support and feedback. Logging out then takes the user back to the area shown in figure 7.4. logout.php Can be accessed anywhere in the system Lesson#.php Includes any relevant material for the chapter home.php Welcoming page for user Lesson#.php Further material needed. Change_password.php Shows all test scores Lesson#.php FINAL LESSON material testresults.php Shows all test scores Test#.php Test relevant chapter peersupport#.php Further material needed. Feedback_result.php Feedback confirmed to user, feedback added to table. to Test#_result.php Test marked. Answers put into user table. Feedback left 68 help.php Shows all test scores Fig 7.6 Data flow for users who have logged into the administrators section When logged into the site as an administrator, the user has access to all administrator functions, but features for standard users (see figure 5.5) cannot be accessed. Logging out then takes the user back to the area shown in figure 7.4. logout.php Can be accessed anywhere in the system peersupport#.php Should they view and delete peer support? Admin_home.php Welcoming page for user Admin_test#.php Allows the administrator to update the questions – this is then shown in the relevant test Change_password.php Shows all test scores testresults.php Allows the administrator to delete a past result so t person can take test again Test#_result.php Test marked. Answers put into user table. Feedback left 69 7.6 Setting up a login system The login system described here was built to meet functional requirement 2 and match the design criteria set forth.. This identifies that a login system was essential in order to identify users. Through past experience the login system was set up using PHP sessions. The login script is designed with usability rather than security in mind, just passing the variables that you want to pass through the session. Initially it checks the user has entered a username and password and then checks the user database to see if these match what is currently stored. The following code then passes the username into the session. This was also used for other user information required. Listing 7.2: Passing session information for the login system. 1 2 3 $_SESSION['username'] = '$form_username'; $username = $_SESSION['username']; $_SESSION['username'] = $_POST['username']; The script was then set up to go to the homepage once logged in, with the following code being on every page to store session data. Listing 7.3: Starting sessions on each script. 1 2 3 4 5 6 7 session_start(); if (!isset($_SESSION['username'])) { echo "<bodyonload=\"parent.location='index.php'\">"; } session_cache_expire(330); ?> This meant that if the session didn’t pass a username, the system would go back to the index.php page. This would therefore ensure that authorised users could not access system features. 7.7 Setting up the lesson material. The lesson material had to be implemented to meet functional requirement 3 and fit in with system architecture. The requirements stated that ideally, the exact content of the ECDL would have been used in the tutorial material. However, the difficulty in getting this material from NHS training staff forced the decision of finding alternative data for the purpose of the study. The lesson material was relatively simple to setup, with HTML being the only required language. The material was taken from the following three sites and was not my own work. 71 1. http://www.comptechdoc.org/basic/basictut/ 2. http://www.imagescape.com/helpweb/welcome.html 3. http://www.baycongroup.com/word2003/word02.htm The chapters were split up into eight sections that would have a question asked in the test relevant to the respective question, so the material was split up accordingly. 7.8 Setting up the peer support sections. The peer support sections were designed to meet functional requirement 6. The peer support sections required the reporting of data from the database. These used some if statements as shown below to ensure that empty fields (when someone left no feedback) were not outputted. Listing 7.4: If statement used in peer support section 1 if ($row["q1fb"] != NULL) Therefore if the question was not left blank it would output the correct data. 7.9 Setting up the tests The tests had to be implemented to meet requirements 5 and 6 and fit in with system architecture. The tests became a significant part of the development of the system, mainly because of the admin feature which is discussed later. The test itself provided a form which submitted data to the test#_result page. The result page inserted this information into the database and displayed the users answer, the correct answer and then marked the user according to whether they had got the correct answer. The test scripts were the largest of the scripts used in the system, with effort needed to ensure that the correct scores were inputted into the user’s record. For instance, the following if statement was used to check if the answer selected was correct. Listing 7.5: If statement used in peer support section 1 2 3 4 5 6 7 8 9 10 11 //a1 was there answer and a2 was the correct answer. if ($a1 == $a2){ $a1a2 = correct; $correctcounter++; echo $a1a2; } else { echo Incorrect; $correctcounter; } 72 The correct counter was a counter that worked out how many questions the user had answered correctly. On the results page the user was also asked to provide peer support for other users and this would then be displayed for other users. The following if statement was then used to determine whether they have passed or failed. Listing 7.6: If statement used to see if the user passed the test 1 2 3 4 5 6 7 8 9 if ($testscore >= $passmark) { ?> Congratulations on passing the test. Please click <a href="lesson2.php">here</a> for the next lesson.<?php } else { ?> <h5>You have failed the test, please speak to a training representative</h5><?php } ?> 7.10 Allowing the user to view all of their test results This section had to be implemented to meet functional requirement 10 and fit in with system architecture. This required that whenever the user had completed a test that it would be shown on this page. This prompted the use of the teststaken counter, which checks how many tests the user has taken. The following switch statement was used to ensure that the right test results were displayed according to the amount of tests taken. 73 Listing 7.7: Switch statement used to output data on tests taken 1 2 3 4 5 6 7 8 9 10 11 12 14 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 $teststaken = $row4[7]; switch ($teststaken) { case 0; ?> <h5>Your have not taken any tests yet</h5> <?php break; case 1: include "table1.inc"; break; case 2: include "table1.inc"; include "table2.inc"; break; case 3: include "table1.inc"; include "table2.inc"; include "table3.inc"; break; case 4: include "table1.inc"; include "table2.inc"; include "table3.inc"; include "table4.inc"; ?> Congratulations on passing the course. Please click <a href="certificate.php">here</a> for a certificate. <?php break; } The .inc files simply included information that printed out the results of one of the tests. Upon completing the forth test, the user was able to obtain a certificate. This certificate can be found in the file certificate.php. This file is basic in that it simply passed the users details in order to print the certificate out. No thought went into the design of the certificate as this was unnecessary when considering the aims of the project. 7.11 Logout The logout file ensured that the user was logged out of the system and could only view the loggedout viewable pages as per the requirements. This included the following code that ensured that all session data was destroyed. 74 Listing 7.6: If statement used to see if the user passed the test 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 if(session_is_registered("Username")) { session_unset(); session_destroy(); ?> <h5>You have now successfully logged out.</h5> <?php } else { ?> <h5>You have now successfully logged out.</h5> <?php } ?> This covers the main coding required for the standard user features. 7.12 Coding of the admin section This subsection covers the coding required to create the admin section. The admin section had to be implemented to meet functional requirement 4 and fit in with system architecture. The admin section was included to ensure that the data stayed independent from the system. This means that technically, if the system was to be used to test another area, it could be adapted through the admin section. In practice this didn’t work as required, as it became clear that providing an admin section for the content would involve designing a content management system, a huge increase in the workload and out of the scope of the system. It was therefore decided to ensure that the tests could be changed through an administration tool. In order to meet the requirements and ensure that the admin section could not be accessed by standard users, a separate area was created for admin users, with its own login system. This was exactly the same as the standard login script, only in that it inserted a “y” into the admin column of the user table. The logon script would therefore only accept a user if they had y in the admin column, therefore ensuring that standard users could not log in. As administration of the tutorials was not required, the only administration was the administration of the tests and deleting test results. 7.13 Administration of the tests The administration of the tests was the significant coding part of the project. This involved a page which displayed all the information for editing and allowed all the data to be changed as shown below. 75 Figure 7.7: Snapshot of the administration system The coding a section of this is shown below Listing7.7: A sample of the code to provide the form for updating test questions and answers 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 <tr> <td>Enter answer 1 </td> <td><input type="text" size = "60" name="test1answer1" value=" <?php $a3 = $row[1]; $a4 = $row[40]; echo $a3;?>"></td> <td> <input type="radio" size = "60" name="q1" value =" <?php echo $a3;?>" <?php if($a3 == $a4) { echo "checked=\"true\""; } ?> </option> </td> </tr> 76 The updated data is then submitted in which a series of update queries ensure that this information is added into the database. Listing 7.8: Query used to update question data. 1 2 3 4 5 6 7 8 $query = <<<ENDOFSTRING UPDATE `test1` SET `test1question1` = '$form_test1question1', `test1answer1` = '$form_test1answer1', `test1answer2` = '$form_test1answer2', `test1answer3` = '$form_test1answer3', `test1answer4` = '$form_test1answer4'; ENDOFSTRING; On the same file, the user selects the correct result which is submitted and then updated in the table. Listing 7.8: Queries which ensure the data submitted is changed in the data 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 $query = <<<ENDOFSTRING UPDATE `test1` SET `test1question8` = '$form_test1question8', `test1answer81` = '$form_test1answer81', `test1answer82` = '$form_test1answer82', `test1answer83` = '$form_test1answer83', `test1answer84` = '$form_test1answer84'; ENDOFSTRING; $query = <<<ENDOFSTRING UPDATE `test1` SET `test1question1answer` = '$form_q1', `test1question2answer` = '$form_q2', `test1question3answer` = '$form_q3', `test1question4answer` = '$form_q4', `test1question5answer` = '$form_q5', `test1question6answer` = '$form_q6', `test1question7answer` = '$form_q7', `test1question8answer` = '$form_q8', `passmark` = '$form_passmark'; ENDOFSTRING; Administrators can then click on a link to take them to the page in which they can edit then changed data. 77 7.14 Conclusion to implementation Although most system functionality has been described above, it was impractical to go into great detail about the implementation of all system functionality. Furthermore in order to understand the system implementation further the coding of the system has been commented thoroughly to ensure that the workings of the system can be understood. At this stage the system has been built but may contain errors as it has not been fully tested. Therefore the implementation as described above is not the final implementation. This stage is not reached till the end of chapter 8. Essentially the coding of the system has met the requirements set forth and has been a considerable effort in this project. The following chapter is the testing chapter. 78 Chapter 8 8 Testing Testing for this project has taken many forms, the most formal method being scenario testing, of the same scenarios used for use cases. Essentially the system was tested for errors and to check that it met the functional requirements set forth in chapter 5. However, testing was performed throughout development and uncovered many errors. The content of the system was mainly written in HTML, so spell checks were performed on this data and furthermore it was proof read to ensure that it made sense and conveyed the correct information. Throughout development tests on links, menus etc were carried out and any errors were addressed. The main system testing was ensuring that the tests themselves worked correctly. The system was developed such that the 1st lesson and test were developed and then the subsequent tests were developed, once the first met all requirements set forth. This ensured that time was not wasted changing lots of different copies of code. An example of the type of testing carried out during development is the way that the admin section was tested. Sequential numbers were added as questions, so that errors could easily be spotted. If a number was missing, it was clear that the administration tool was not updating this field successfully. This is shown in figure 8.1. 79 Figure 8.1: Example of testing administration of tests designed. It was important to test the admin section early, as the admin provided the questions for the tests that would be checked later. Further functions such as logging in etc were informally tested as they were developed. 8.1 Scenario testing Formal scenario based testing was completed on all scenarios used in the use cases in the requirements gathering stage as documented in Appendix E. These scenarios were tested stage by stage, as shown in the appendix, but an example scenario test can be viewed in figure 8.2. 80 Figure 8.2: Testing the scenario of taking a test within the NITS system. Test No: Taking a test in the system Test Expected Result Click on test 1 Takes you to test1.php Complete test User is notified that the information is and submit data submitted, relevant data is inserted into the table test1_answers, which shows the answers given. Presented with System takes you to test1_result, marks and marking the results correctly and giving feedback on the the option of leaving peer support results. information Actual Result “ “ No variation “ “ No variation Passed? Passed Passed “ “ No variation Passed The following scenario’s produced errors. Password change – did not notify the user that their password had been changed. Solved by adding a simple echo statement Admin section – it was found that when changing an answer and the correct answer simultaneously, the correct answer submitted was not correct as it was set up to submit the old answer. Solved by adding files to ensure that this worked correctly. In order to solve the problem, it was decided that instead of changing the answers and the correct answers in the same script, two scripts would be used. One would be used to change the answers and the other used to select the final answer. This worked as follows: 81 Fig 8.3 : First page of administering questions. The data was editable so that questions could be changed. Following this the correct answers could be selected as shown below in figure 5.9 Figure 8.4: Changing the correct answers with the system 82 Previous to this, the answers and correct answers were all changed in one file, which caused this error. The conclusion of the scenario testing leads on to the pilot study. 8.2 Pilot This pilot study had no association with the prototype designed and implemented in chapter 5. This pilot study piloted the system developed as per the documentation above. Following the scenario based testing of the system, a few users with differing IT experience were asked to use the system to register, login and complete the tests. This was performed once in person, but two times over MSN, with the person providing feedback as they went through. The pilot uncovered a number of coding problems as discussed below. 1. The wording of some of the questions in the tests was unclear 2. When providing peer support, the user often struggled in determining what questions were asked in the first place. Therefore they didn’t leave peer support 3. Peer support was often mistaken for feedback on the system – i.e. to a system administrator 4. Peer support information didn’t appear correctly. During testing, leaving peer support involved leaving peer support on each question. However in reality, users simply wanted to leave feedback on the questions they got incorrect. When this happened the format of the data was incorrect. The solutions of these were implemented as below. 1. Questions were proofread and changed accordingly. These were then discussed with the participants in the pilot study. 2. The question was clearly marked on the peer support form. 3. A clear explanation of the use and point of peer support was added so that the user knew what to leave as peer support was added to before the user left feedback. 4. The following if statement was inserted into the peer support scripts. This checks whether feedback was left in any of the cells. If there is it prints the person’s details and then does a further if statement to see if there was data in each row, and output accordingly. This was completed for all areas were feedback could be left. 83 Listing 8.1: If statement to determine if users have left any feedback 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 if ($row["q1fb"] != NULL OR $row["q2fb"] != NULL OR $row["q3fb"] != NULL OR $row["q4fb"] != NULL OR $row["q5fb"] != NULL OR $row["q6fb"] != NULL OR $row["q7fb"] != NULL OR $row["q8fb"] != NULL) { ?> <table width="605" border="0"> <tr><td width="599"><strong><br>Name:</strong> <?php echo $row2["1"];?> <?php echo $row2["2"];?></td> </tr> <tr><td><strong>E-mail address:</strong> <?php echo $row2["0"]; ?> </td> </tr> <?php } if ($row["q1fb"] != NULL) {?> <tr><td><strong>Section 1 feedback </strong></td></tr> <tr><td> <?php echo $row["q1fb"]; }?></td></tr> </table> During this study it was realised that the prototype designed and evaluated in the design stage should have incorporated peer support, which may have eliminated the errors found above. This study was invaluable in spotting errors that were not noticed in system testing. A good example of this is problem 2 from the above list. During testing this was not noticed as the system did not contain an error, rather the tester relied on implicit knowledge of the requirements of the system and therefore had no problems. Only when piloting the system was this spotted. 8.3 Final system The final system incorporated the changes above and can be found at www.oligarchic.co.uk/jackland The changes only affected the PHP and HTML so the table design remained the same. 8.4 Conclusion The testing of the system was thorough in order to ensure no errors occurred whilst nurses used the system. Testing naturally took place during development and most errors were spotted here. Scenario testing applied scenarios used for use cases in the requirements section and ensured that the particular scenario did what was expected. Finally the pilot study 84 identified a few more errors that were not spotted earlier. This range of testing techniques provided a thorough review of the NITS system. 85 Chapter 9 9 Evaluation of NITS system. This chapter covers the process of evaluating the NITS system. The aim of this section is to evaluate whether the implementation of the system meets the requirements set forth for the project. At a higher level the project deliverables should meet the aims and objectives set forth in section 1 of this document. 9.1 Think aloud protocol and cooperative evaluation The aim of a study into the usability of the NITS system is to analyse how the nurses interacted with the system and to spot any potential usability problems. In an ideal scenario, ethnographic techniques would have been adopted to analyse users in their natural environment. This would enable the evaluator to spend time in the field with the end users, analysing exactly how they would use the system in the user’s natural environment. As discussed, getting hold of nurses was difficult enough. Furthermore, the literature review showed that nurses often completed training in their own time anyway, sometimes at their home. Therefore some of the nurses will be studied in their own home to increase the validity of the results. The tests into the usability of the system followed the think-aloud protocol and cooperative evaluation technique. Thinkaloud is an effective technique to ascertain what the user is thinking whilst they are using a system rather than simply observing what the user is doing and assuming their thought processes. The think aloud protocol usually involves a user going through a series of tasks and describing their thought processes to an evaluator. The evaluator is able to ask the participant questions about the system and discover the user's opinions and this normally plays an important role in ensuring that users thought processes are clearly articulated. Thinkaloud is extremely effective if evaluations are recorded as this provides lots of data for analysis. The cooperative evaluation technique was used in the requirements gathering stage to see the interactions nurses had with existing systems. Cooperative evaluation effectively involves the user in the pre-design process as feedback is obtained on the system and what they would like to see in a future system. 86 In order to conduct this study, nurses will be given the system and a brief introduction of the aims of the evaluation. In order to ensure the nurses are relaxed they will be assured that the aim of the study is not to see how high the results of the tests will be. Nurses will be told that the main aim of the study is to see how they interact with the system, rather than to test the suitability of the lesson material and test their IT knowledge. These studies will be essential in ascertaining the usefulness of the system developed as it will be the only chance to see how the nurses interact with the system designed for them. However it is also important to ensure that the evaluations are not too formalised, to ensure that nurses do not become nervous, feel like they are being tested and therefore be influenced by demand characteristics. 9.2 Questionnaires The main advantage of questionnaires, being that they can be used to gather information from a large amount of people, cannot be utilised in this project. However, a questionnaire will be designed as a sensible way of obtaining feedback from a nurse following taking part in a walkthrough mentioned above. It is likely that the user will have concentrated heavily on the lesson and test when using the NITS system and it therefore seemed inappropriate to conduct further tests and interviews on the same nurses afterwards. However, due to the limited availability of the nurses, the decision to hand out questionnaires was a compromise between getting information, and allowing the participant to determine the level of feedback they gave. A questionnaire was designed to get some formal information about the nurses learning experience and past experiences with IT. The questionnaire was mainly designed with multiple choice questions as it was felt that the nurses may be reluctant to spend lots of time on a questionnaire after going through a system walkthrough. However, on many occasions nurses are offered the chance to give reasons for their answer and it is hoped they would use this opportunity and provide further feedback. One of the main areas of information that the questionnaires will try to evaluate will be the usefulness of the peer support section. The walkthrough would help to determine whether they would use the peer support section, but it would not provide much information on how useful this section was if it was used and why they didn’t use it if they ignored it. The questionnaire will ask questions on the following subjects: • • • The nurses IT abilities and experience Their experience using the system developed. Their experience and opinions on NHS training The questionnaire designed can be found in Appendix E. The information from both the evaluations and questionnaires have been used to provide the recommendations found in the recommendations document (see Appendix F). The studies mentioned were designed in section 4.2. 87 9.3 Sample In total the final system was used by 7 nurses, 2 of whom used the system in its entirety, whilst the others completed just one of the 4 tests. 6 of these nurses were NHS staff, whereas one was a nurse who worked in a private nursing home but had previously worked for the NHS. The age of some of the nurses was unknown, but most were in their late forties. The studies were conducted in Basingstoke and Oxfordshire, two places where access to nurses were accessible. The fact that the nurses didn’t come from just one place is an advantage in that it allowed further generalisation of the results gathered. Two nurses, namely Francis Ackland and Julie Box completed all four tests and were asked to perform a number of activities such as changing their password and viewing their test results. The other 5 nurses completed one chapter and were kept for no longer than an hour. 9.4 Method The users were asked to perform a number of different activities as documented in the writeup of the walkthroughs, see Appendix G. One of these walkthroughs is summarised below: Walkthrough 2: Patricia Ash The study followed the method set out in section 9.1 The participant was given the following aims: • Register as a user • Login • Read and comprehend lesson 1 • Take the test • Change password • View all test results. The participant took 25 minutes to meet all the aims set. The participant past the test, getting 6 answers correct out of the 8 questions. In this walkthrough the peer support section was not viewed but the participant did decide to leave peer support information, but the information left was very specific. A number of points came from this walkthrough. Firstly it was clear that the participant was being influenced by being evaluated and admitted that in a real life setting may have acted differently. Furthermore the peer support section was misused, suggesting that more information on how to use this section should be provided. Whilst the cooperative evaluations with think aloud method was carefully thought-out, the walkthroughs were changed slightly throughout the evaluation. Two of the shorter studies were videoed and analysed more closely. The others could not be videoed due to resource and time constraints. 88 During the first study, the evaluator found that the participant was showing visible signs of being watched and felt very uncomfortable when reading through the data in the tutorial sessions, feeling the need to read the data quickly. The participant was also asking questions about the tutorial information and providing information on improvements to the information. Because of the friendly nature of the relationship between the evaluator and the participants, it seemed that the system was not being used as if the participant was not being evaluated. Therefore whilst it was very important for an evaluator to be there as the nurse went through the system to see how they interacted with it, it felt unnecessary to have an evaluator there whilst the participant read the HTML content. Therefore the participants were left to view the information and asked to call the evaluator when they had reviewed the content. Upon returning the evaluator asked some questions such as “if the content was clear” and “if they had any problems”. Whilst the participants took the test, they were observed and although the think aloud protocol method was being used, the evaluator did not ask questions when the test was being taken in order to ensure the participants did not become nervous. However after the participants pressed the submit query button, they were asked many questions like how they knew to only answer one question if they did etc. Mainly the participants would come back with specific information about the question the system asked and the content they had viewed rather than comments about the program structure. Essentially, the studies met the requirements set out in the design of these empirical studies. 9.5 Results There were some interesting and unexpected results from these studies. The empirical studies were used to find recommendations on how to improve the NITS system. The questionnaire and interviews were used to provide recommendations on how the NHS could improve its training strategy. 9.5.1 Think aloud with cooperative evaluation Below is a summary of the benefits and drawbacks of the NITS system and finally some recommendations on how it could be improved. The notes on a sample of these walkthroughs can be found in Appendix G. Benefits • • • • • • • Nurses were very positive about the use of e-learning. Nurses are able to study when they want to. Nurses don’t have to study with others and be embarrassed about their IT skills. Nurses can study in the comfort of their own home. Nurses are familiar with web technology. Nurses appreciate the need to learn about IT. Nurses read and left peer support information, it is likely that such a feature would be utilised. 89 Some of these benefits are benefits identified in the chapter 3 or using e-learning. The fact that nurses seem familiar with web technology and that they appreciate the need to learn about IT is very positive. Furthermore the fact that nurses used the peer support section, especially for leaving feedback was an unexpected result as this was optional. It was identified that the nurses didn’t think about whether to fill the data in, they almost saw this as a requirement of using the system. Drawbacks • • • • The system didn’t explain many features clearly enough. Peer support sometimes misused Admin login page can be mistaken for standard login page Submit query buttons confused user Some features such as peer support were sometimes misused by users who were not clear on exactly what kind of information should be left. They would often leave information on what they thought the answer was without giving an explanation of why, or would provide feedback for system administrators instead of peers support information. Furthermore one user tried to login into the administration section, not reading the label on the button. Recommendations • • • • • • • • • No user manual or help system required. More explanation of main features required Change “submit query” buttons to “submit” Provide ways of remembering the username and passwords Differentiate different pages from each other Move admin section away from homepage Navigation should be clearer – getting to the next logical step should be easy. Standards such as the use of radio buttons should be explained. Peer support should be used. The help button on the system led to a page which simply asked the user to inform the evaluator that they had required help. No participants clicked on this button and this supports information found in the cooperative evaluations conducted earlier which suggested that help should be built in with the system rather than separated. Furthermore it was identified that standards such as radio buttons should be explained as nurses may not be familiar with these. For a more detailed review and rationale for these recommendations please view the “Recommendations document”; Appendix 8.9. The walkthroughs were an essential part of this project and on reflection it seems the think aloud protocol and cooperative evaluation technique worked well in seeing how nurses interacted with the system. As mentioned it was important that the evaluator was flexible, if the design had been followed rigidly and the evaluator had stayed with the user all the time they were using the system the results could be open to demand characteristics and therefore may not be as trustworthy. 90 9.5.2 Questionnaire and interviews The following findings came from the questionnaires and interviews that were conducted after the empirical studies. Recommendations here mainly focused on NHS training, although some findings on how to improve the system were also found. One user suggested that the text in the tutorial section should be bulleted or broken up in some way as the information was difficult to read. When questioned about this the user suggested that she tended to skim the information so would have rather reviewed snappier clearer text. Through the amount of time others took in reviewing tutorial data it seems that many of the users would have skimmed the information rather than reading all the information in a paragraph. A number of recommendations to improve the NHS strategy were gained from the use of existing literature, questionnaires and interviews. A summary of these recommendations can be found below, as before more information can be found in the appendix G. Recommendations on NHS training strategy: • • • • • • • • Allow a training regime that means all training does not need to be conducted in a nurses own time Make training requirements clearer Provide accessible tutors for continued learning and development Use e-learning Make IT training compulsory Make high grade managers and nurses take the lead and complete IT training Use blended learning Provide avenues for peer support The use of questionnaires worked well in that it maximised the amount of information that was gained from the nurses. The information on these questionnaires on occasion was information that had not been gathered during the evaluation or through talking with the nurse. Interviews were used informally after the user had completed using the system and information was gathered from these that were of relevance. In fact it was from informal interviews such as this that the term “nurse proof” was expressed by one nurse, who commented that the term had been used by software developers for the NHS in the past. When interviews like this are kept informal and follow a natural conversation style it seemed that nurses are often willing to provide additional information. 9.6 Conclusion The NITS system did meet the requirements set forth for the project and followed design principals to ensure the system was usable. The think aloud protocol with cooperative evaluation was a suitable method for evaluating the nurses using the system and coupled with the questionnaires the results from the study were enhanced. The study could have benefited from more participants in order to be able to generalise from the results further and this will be touched on in the following chapter. 91 The results of using the NITS system were positive, largely down to its simplicity and nurses willingness to use e-learning systems. Furthermore, the study provided results that could be added to the recommendations document, something the study aimed to do. 92 Chapter 10 10 Conclusions In chapter 1 of this project, the rationale and the scope for the project were identified. The idea was that there were many issues with nurses using IT and that this was likely down to insufficient training. Following the literature review and requirements analysis it was confirmed that the focus of the project should be how to help nurses with the use of IT. The exact scope of the problem has always been difficult to specify given the interwoven nature of training strategy and training techniques. The original inspiration came from experience with family members who are nurses and struggled using IT, which was necessary to develop their career. In order to understand these problems research into training methods and strategies took place in the literature review and influenced the expected deliverables of the project such that a software artefact without any further deliverables didn’t seem as though it would be sufficient in addressing the problem. Information gathered by reviewing literature and conducting interviews suggested the high level problem is a direct result of fundamental issues within the NHS. NHS staff may comment that its all very well suggesting that nurses should receive more structure to their training, more e-learning and more IT training but if the financial resources are not available this cannot be done. Similarly if there is no one with he time in the NHS to really look at the issues identified, suggestions will go unheeded. The NHS may have other priorities, i.e. reducing waiting times, enhancing the level of care which is without question of paramount importance. This project cannot and has not attempted to solve any of the issues mentioned here, but what it has tried to achieve is two fold. First to provide recommendations on the NHS training strategy and second to construct a learning tool that could be incorporated into a blended learning strategy. it does try to do is provide some insight on how to help train nurses and how to make them more comfortable using IT. To do this has still taken a significant amount of research, software development and empirical studies in order to provide suggestions that are relevant and well-founded. 93 The preliminary research conducted in the literature review provided a largely theoretical grounding to the main issues relevant to the domain and further enhanced the justification of the basis of this study. The information in the literature review specific to the NHS such as its history, the training regulations etc (some of this information was gained from interviews), provided a factual backing for the issues that arise. The history seemed littered with stories about the NHS being overworked and under funded whilst the information on NHS training methods was helpful in understanding what happens in the NHS now. More information was gained on these topic areas as the project progressed, normally enhancing further the grounding for this project. Furthermore, the information on training in an organisation (section 2.3) helped to deepen knowledge on training techniques to enable qualified suggestions to be made about the NHS training strategy. For instance, information on collaborative learning first prompted the investigation into peer support for nurses using e-learning systems. The section on e-learning further helped identify good and poor design, therefore helping to design an e-learning solution. In addition discussions on the future of learning seemed to support the idea that elearning is not a fad and that the NHS would do well to ensure they use it. In addition, the section on blended learning (Section 2.5.4.1) was a big influence on the project in that it was again fundamentally clear that a simple software solution deliverable was not sufficient in helping nurses learn. Rather a range of methods should be used in training. The literature review coupled with interviews was paramount in deriving the requirements for the software solution. Usability was identified as a key requirement throughout this project. These standard usability requirements were supplemented to ensure that information gained in interviews and evaluations that show how nurses use IT was not ignored. During the requirements analysis stage a number of techniques were used, maybe the most important of which was the interview with an NHS training representative. This interview was paramount on getting a wider perspective of training within the NHS and understanding how nurses could be trained. It was this meeting that highlighted the need for an e-learning system for nurses to learn IT and furthermore highlighted that there were courses available for nurses to learn IT, but nurses often did not go. This further promoted the use of a recommendations document that would attempt to address the problem of why nurses may not attend this training. The evaluation of existing systems, although fairly brief was valuable in understanding how nurses interacted with IT systems. It was a good indication of the level of IT skills that nurses had and was useful in highlighting features that nurses used. It was during these evaluations that it was noticed that nurses tend to be thorough when filling out information, often including information on fields that are not mandatory. Therefore when evaluating whether nurses would leave peer support information for other nurses it was concluded that they were likely to do this. The NITS system was designed to meet requirements set forth and implemented in line with the requirements and design. The NITS system is an adequate e-learning system that could be used in the NHS. The recommendations on how to improve the NITS system as found in the recommendations document would allow for future improvements to the system. In addition many of these recommendations could be useful in providing help on designing new IT systems within the NHS or updating existing systems. 94 The project has enabled the successful development of an e-learning system, which fulfils the requirements of the nurses and training representatives in the NHS. The project has also established that peer support can be a valuable tool within an e-learning system used in the NHS and could help nurses help each other when training. The NITS system has been designed with learning and training theories reviewed in chapter 2 in mind. The use of the peer support section is used to support collaborative learning, providing the facility for nurses to pass on information and details for them to contact each other if required. Furthermore the tests can be used to attain goals (goal setting theory) and provide reinforcement (reinforcement theories). The project would have been of more value if further interviews could have been conducted with more members of the NHS training organisation. Only one interview with a training representative could be arranged and although a vast amount of information was gained it would have been advantageous to review the NITS system with the representative to help conclude on its usefulness. Furthermore, in the requirements stage a discussion on technologies to be used within the NHS, preferred security and login systems and even preferred design would have helped in ensuring NITS could be successfully implemented within the NHS. Due to the small size of the project there would always be further scope to do more, but with the time constraints placed on the project, the aims and objectives have been successfully met. When deriving the requirements only 2 nurses were consulted, due to availability of nurses. Although the system was successful in that the nurses studied were able to use it, more nurses should have been used throughout the design process. Usability problems such as the submit query buttons would have been eradicated if this could have happened. Furthermore. It would be difficult to generalise from the study as it stands now, if the sample could be increased, not just in the requirements stage but throughout the project it is likely that further value would be added. In addition, only one training representative from the NHS was used in the study. If more representatives from more hospitals could be studied this would be useful. Furthermore spending more time with them would no doubt have provided more valuable information. In fact, access to stakeholders throughout the project was a problem, if the study was enhanced, more time to be spent with key stakeholders would undoubtedly be necessary. 10.1 Future work When completing any project or developing any system it is important to consider the future. In a project setting, this will ensure that the results of the study live on and will still be relevant after project completion. In the area of systems development, considering the future ensures that the project will not become redundant in a few years. In addition, when completing a project in a short amount of time, there is often future development that could not be fully implemented or research areas identified. This project has been no different; there are many areas in which future work could be conducted. The following enhancements could be made to the NITS system: • Content management system for administration of content material 95 Rationale: The requirements stated that the system should be data independent and although the tests are such, content of tutorials still needs to be changed by changing the HTML files. Although this is possible, this isn’t true data independence. Therefore if more time could be spent on the project a content management system for the NITS systems training material would be beneficial. • Peer support section expanded Rationale: Although the system provides some basis for peer-support this is only implemented in a primitive form. Further usability testing and cooperative evaluation could be conducted to see how nurses would prefer to use peer support. The implementation of a forum in which users could chat, the use of instant messaging or furthermore, a site that details people who are doing the same course as you, providing contact details or a way to add them to your instant chat group would be very useful. Essentially, in order to maximise the benefits of peer-support, further development, research…and most likely, funding will be required. • More interesting lesson material Rationale: As mentioned in the requirements stage, the tutorial information was defined as out of scope for this project. However, as identified in the literature review, the information conveyed in teaching information is the most important factor in the success of training. Reference back to literature review - “Don’t underestimate learner’s willingness to try elearning but don’t think for a nanosecond that they will tolerate bad learning solution just because it is web deployed” (Rosenberg 2001). When enhancing the tutorial information on the NITS system, a detailed review into the exact use of IT for nurses in the NHS should be conducted and training should be developed to ensure that nurses can complete these tasks. The content must meet the requirements that the nurses have; currently it is unlikely that the NITS system does this. Furthermore, the way the training material is delivered could be more interesting and interactive. Although nurses identified that games may not be liked by nurses who will want to learn the material as quickly as possible, it may still be useful to include other forms of training, including flash or java web based games. Only one nurse suggested this wouldn’t be useful, others may disagree and the addition of other methods than simply reading the data may have a positive effect. In addition, the current usability problems found in the NITS system should be fixed. Submit query buttons should be changed to submit and navigation from page to page should be easier with the use of next and back buttons at the top and bottom of every page. Furthermore, session timeouts should clearly communicated to the user as to why they have been logged out…and warn them beforehand. The login system was not developed with security in mind and it is unlikely that such a system would be sufficient within the NHS. If nurses are to be accredited through the NITS system, additional security should be implemented and the NITS system should only be accessible to NHS staff, whether this is restricted to Intranet use, or whether registration to use the system should only take place on NHS premises. Otherwise, people not authorised to use the system may login and become accredited through the system. 96 Interviews suggested that remembering usernames and passwords for various systems became difficult for nurses. The NITS system tried to alleviate some of this concern by requiring the use of an e-mail address as the username. However, this caused difficulties when a nurse didn’t have an e-mail address. If peer support is going to be used within the NHS, it seems sensible that the NHS provide nurses with their own e-mail, which should be accessible away from the hospital if required. However, the NITS system did not provide any means of reminding the user of their username and password once registered. The use of validation questions, i.e. entering your email and being asked questions which you have already predefined the answers to...as used on www.ebuyer.co.uk./ would be useful for nurses. Furthermore one nurse said that a facility in which they could enter their username and have their password sent to a defined e-mail address would help. It also seems sensible that a nurse should be able to enter their full name and be reminded of the username they were using to help them remember their username. These kinds of additions would help the nurses use the system should they forget this information…which one nurse almost did in an hour long evaluation! The issue of NHS training strategy could not really be fully addressed due to the size of the project. A further review into this before developing or enhancing the NITS system would surely be of benefit. As cost has been identified as a major issue for the NHS, recommendations should be costed to help the NHS decipher which recommendations to incorporate. 97 11 References: A Simple Guide To Normal Forms in Relational Database Theory: Third normal form. Available from http://www.bkent.net/Doc/simple5.htm [Accessed April 2006] COCKBURN, A., 2001. Writing Effective Use Cases. Indianapolis, IN: Addison-Wesley. The 'one-column table' use case format, p.121 Department of health: DH:home The Department Of Health. Available from http://www.dh.gov.uk/Home/fs/en/ [Accessed 14 April 2006]. Grant, M. “Contemporary strategy analysis fifth edition, Blackwell publishing 2005 Henry, J & Stone, R. “Computer Self Efficacy and Outcome Expectancy: The Effects On The End User’s Job Satisfaction”, Computer Personnel – October 1995 Jacovi, M et al. “Why Do We ReachOut? Functions of a Semi-persistent Peer Support Tool”, IBM Research Lab in Haifa, Israel, 2003 Kazmer, M & Haythornwaite, C. “Multiple perspectives on Online Learning”, ACM 2004 NHS website: History of the NHS. NHS Connecting for Health. Available from: http://www.nhs.uk/England/AboutTheNhs/History/1998ToPresent.cmsx[Accessed Nov 2005] Noe. R A. “Conceptual Employee Training and Development Third edition”, McGraw-Hill International edition, 2005. Nursing and Midwifery Council: Code of Professional Conduct (2002) London (Nursing and Midwifery Council 2002) PREECE, J. et al. Interaction Design: Beyond Human Computer Interaction. New York: John Wiley & Sons, Inc. (2002) Rosenberg, M. “E-Learning, Strategies for delivering knowledge in the digital age”, McGraw-Hill 2001. Rosenberg, M. “E-Learning, Strategies for delivering knowledge in the digital age”, McGraw-Hill 2001. Simonsen & Sein, “Effective training applying frameworks to practice”. Sigmis 2006 Simonsen, M & Sein, M. “Conceptual Frameworks in Practice: Evaluating End-User Training Strategy in an Organisation”, Sigmis April 2004. 98 Sommerville. I “Software engineering sixth edition”, Addison Wesley, 2001 Swart et al, Human Resource Development Strategy and Tactics. Butterworth-Heinemann Ltd 2004 UPenn Collaborative on community integration: Community Integration – Peer Support. Available from:http://www.upennrrtc.org/issues/issue_peersupport.html [Accessed Feb 2006] We, T. “Collaborate to learn and learn to collaborate”, University of Tasmania 2002. Biliography Daeson, W “The Essence of Computing Project A Students Guide,” Prentice Hall, 2000. Dix ,A et al. Human Computer Interaction. Prentice Hall 2003 Dodero et al. “An experience on Students Participating in Blended vs. Online Styles of Learning” Universidad Carlos III de Madrid 2003 Jannvier, B. “HCI for web based development of interactive medical multimedia coursework, Human Communication” – Management Training, Allied Dunbar 1986 Shortcliffe T, David, R. “Some consideration for the implementation of a knowledge-based expert systems” Stanford University 1975 99 Chapter 12 12 Appendices 12.1 Appendix A The following are notes made on interviews conducted for this project: 12.1.1 Interview with NHS nurse Below are some notes made from an interview with an NHS staff member. The interviewee was a middle aged female who has been a nurse for 25 years, working for the NHS for 10 years. The nurse has recently completed her diploma at Southampton University. Thank you for agreeing to let me interview you. I am conducting a project into training for NHS staff for my computing degree and was hoping to ask you a few questions. These questions relate to the training NHS staff require for a given group of nurses and the methods of training available to them. The interview should last no more than 30 minutes. 1. How do you manage your continuing professional development? Firstly, there are five statutory development days over a three year period. Nurses normally try to go on other study days if they get the chance. It isn’t a case of just going to the training and that’s it, nurses are required to show what they have learnt by putting into practice and reflecting on it. How may you be asked to show you have put learning’s into practice? You have to sign a declaration to say you have done so many study days a year and that you have maintained a portfolio. The NNC can recall your portfolio at any time. Nurses are asked to display evidence based practice – study can include accessing literature online though databases. In your area of practice you have to know why you are doing things, do this by research. This has led to her challenging common practice they do everyday. 100 Also ready journals, library books, the nursing times etc. today nurses need a degree or diploma to become a registered nurse. Existing nurses are encouraged to update to a diploma or degree level as well as certain core courses which need to be completed. 2. How does this fit into your pattern of work? E.g. do you have days set aside for training? We get study days; some will not do any, ranging to a maximum of about 10 training days per year. However most learning is expected in your own time. Because of financial constraints study days are being cut back on as well. As well as working unsociable hours, most nurses are females who often have families, so fitting in time to do this is difficult. Some of the learning is completed during annual leave, so it doesn’t really fit into the pattern of work. She remarked that she has even gone sick to get work done to help with her diploma. 3. Do you use Collaborative learning in your current training methods? Often you are not on the same courses as colleagues and therefore you don’t know the other nurses. On the study days it was interactive, some self directed learning. University of Southampton do something called Blackboard but could never get the hang of it. Everyone on the course didn’t understand Blackboard; everyone got fed up and through it was a waste of time. No one has used it. 4. Does the learning you receive “on the job” fit in with the formal training you undertake? Not on the courses I’ve done. Some of the core courses for the diploma, but not all related to the job. For instance, one training day was on NHS policies. Although a quick round up of NHS policies may have been useful, this seemed overkill and wasn’t that beneficial to hands on work. There are certain documents to be aware of, but essentially you need to know the basics and learn the rest whilst doing it. Most nurses do most of their learning on the job. Research helps though, for example, nurses always used to clean a wound immediately, but following research read by nurses, many do not as cleaning the wound can slow down the healing process. Now cleaning of the wound is only done if it is really needed. Most learning comes from personal research rather than directed learning. However, most skills come from doing them 75%. 25% training course help, the rest is on the job. 5. What advantages are there with the way you are currently trained? Study days are interactive Makes the job more interesting to see the wider picture. 6. What disadvantages are there with the way you are currently trained? Pressure of work in own time Online learning can be difficult for novice computer users. Learning how to use Microsoft Word, how to number pages etc can be a struggle. Essays for 101 diploma have to be done on computer, referenced, double lined spaced. All of this was a struggle. Can be difficult to access the articles required. Spend most of the time trying to access articles needed for research on an online system. She attended a 1 hour training session on how to use databases….but when she got home she could remember all that was said and struggled using the system. Handouts were given, which were helpful, but the learning wasn’t followed through, forgot what was told. Handouts given in training. More drop in sessions for help were available, but she wasn’t aware of all of these – should have been documented. Essay writing skills are difficult to pick up when you haven’t done them in a while. People don’t remember passwords, more than one password, gets difficult. 7. Have you used any e-learning systems? If so, please describe them. To complete her diploma she accessed a database of journals – much like the ACM digital library. You logged in using a password, provided. You would then type what you were searching for, i.e. diabetes, then press search. As you want to search for diabetes elderly”, you then enter another criteria, “elderly”, then you do a combine search on these two sets of criteria. The way they worked and looked wasn’t familiar and she wasn’t sure of the difference between something like this and “Google”. Never used systems that mark you and give feedback etc. What would have helped your experience with e-learning? Working one to one with tutors so you feel able to ask questions and go at your own pace would have helped. It’s the kind of thing that you can’t just learn after one day, you need training each week, if only for a little while. E-mailing tutor did help and did get the opportunity to meet with the tutor, but during the time didn’t understand what was being said. Had to work out online training system yourself. The journals are in the library but it’s more difficult. Some of them you only get author and title but couldn’t read article, didn’t know what she was doing wrong. 12.1.2 Interview with NHS training representative: (Sample transcript, as this was an informal interview, documenting the conversation was difficult. Below are the main points that were taken from this interview) Sheila Thorne’s worked at the Basingstoke district hospital in the training department (Donald Burrell centre). The interview took place on January the 9th. The interview started with a briefing as to the purpose of the project being conducted, the aims of the interview etc. Upon hearing the project was looking at training offered by the NHS Sheila gave information on some training courses offered within the NHS. These included • How to write an essay • Study skills 102 • • The ECDL (European computing drivers licence) Literacy and numeracy The study skills course: Study skills is a programme that focuses on essay writing skills and the ability to critically read and assess. When asked about whether the course was successful Sheila responded that it was; people use the course and normally pass the module there they were aiming to. This had led to much positive feedback about the course and the students rise in confidence. IT training Again, this differs from hospital to hospital, but the Basingstoke district hospital use the ECDL (European computing drivers licence). Some hospitals may offer access to colleges in order to learn more on IT. The NHS are pressing forward with IT and it is a government standard that all hospitals should use IT. The library is available and computers can be used and booked at anytime. Sometimes the accessibility of computers on ward is scarce so this cannot be relied upon. Cost of training Sheila was not able to provide information on the cost of training but admitted there is a huge amount of money that goes into training. The hospitals library provides lots of IT services so often the true costs are hidden. In addition, as the project was IT related, Sheila mentioned that the library offered access to a database of medical journals for NHS staff to use. The question of what the regulations were for training in the hospital was asked: Nursing training requirements: Nurses typically will take between two and five study days per year although this is the way it works locally, regionally may be different. Nurses must complete these study days in order to re-register as a nurse and mist complete five study days in three years. This does not always involve sitting in a classroom, people can have the opportunity to do a variety of courses and do more courses than the minimum if the wish to. It was identified that an e-learning system to deliver the material in the ECDL would be useful. It was further suggested that this system should be on the internet and that the system should test the user’s knowledge on IT. The suggestion of providing a certificate upon successful completion of the ECDL would be appropriate, as they currently receive a certificate if they pass. This concluded the interview with the training representative. 103 12.2 Appendix B ****INITIAL REQUIREMENTS AT LITERATURE REVIEW STAGE**** Introduction to requirements Below are some provisional requirements for an e-learning system to be developed by the NHS. These requirements were determined from the information gained throughout this literature review and will develop significantly as more interviews are conducted. Functional requirements These are the functional requirements for the demonstration system. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. The system should have a sufficient tutorial session (link to non func 1) The system should provide a help facility (link to non func 1) The systems interface should be clear and uncluttered (link to non func 1) The system should provide the user with information and then ask questions on what has just been read (link to non func 2) The system should score the user and then provide feedback on question that were wrong (link to non func 2) The user should be able to print out the questions s/he answered incorrectly, with the correct answers (link to non func 2) The system should be searchable and allow use after testing is complete to check information. The information in the system should be able to be easily updated (link to non func 9) The system must link to other important resources The system should be web enabled – compatible with common browsers. The system should have sufficient documentation (link to non func 7) The system should be password protected and include relevant disclaimers. (link to func 6) The system should allow new users to register (link to non func 7) users should be able to work on the system in groups (link to func 8) The system (or training strategy) should provide a forum facility for the sharing of information The system should load quickly between screens (link to non func 10) 104 Non-functional requirements The system should adhere to Scheiderman's principals of user interface design: o User familiarity This means that all of the screens within the system should be similar to both each other and other web pages that the user may have been to in the past. This means that the layout of all of the screens should be roughly the same as each other. Also that certain “web” standards are upheld e.g. an underlined blue word is a hyperlink. This will help the system have a shallow learning curve for users. o Consistency This is fairly similar to the previous concept discussed above. This means that if an operation is activated by the user at one point in the system then it will be activated in a similar way elsewhere. o Minimal surprise This means that the system should never react in a way, which will confuse the user in any way. This will be achieved by following the two principles discussed above to make the system standardised. o User Guidance This is where the system gives the user helpful feedback while they are using the system. This includes properly worded error messages to help the user understand what a problem is and how they can prevent it a second time. o User diversity This means that there may be many different types of users accessing the system with different experiences and skills. Therefore the interfaces should accommodate for both experienced users and the completely unskilled user. The interface could either provide small logical steps to get to screens for novice users or more complicated search procedures to get to exactly the right data for the experienced users. o Recoverability This is set out later in this section under the reliability of the system. The system should be useable for novice computer users. Therefore, the following user interface design principals will be followed: User interface requirements: 1. Navigation time should be minimal. 1.1. Navigation should include explanatory text. 1.2. All symbols and text messages used should be appropriate for the task that will be carried out. 1.4. Each screen must be as organised as possible and not cluttered. 1.5. The language used must be simple and avoid jargon. Rationale - This is essential in allowing the nurses to find all information with the least amount of difficulty. 105 2. The application must have a user interface. 2.1. The user interface must follow the six principles of user interface design as described in Sommerville (2001) and are described in more depth in the nonfunctional requirements specification. Rationale – Usability is the key requirement for the system. 3. Colour must be used in a suitable style. 3.1. Use of colour will be limited and conservative. 3.2. Colour will be used to show a change in application status, e.g. rollover buttons 3.3. Colour will be used to support the task which nurses are trying to perform. 3.4. Colour coding will be used in a thoughtful and consistent way. 3.5. Thought will be placed when pairing colours. Rationale – The system should be attractive and comfortable to use for nurses. 5. The GUI should be as intuitive as possible. Rational - This is essential to allow nurses to find all information with the least amount of difficulty. 6. The application should reflect any change in state (e.g. a button being pressed) within 0.1 seconds. Rationale - This requirement stems from producing information for the nurses in the fastest time possible. Nurses can become impatient and click buttons again and again if they do not know that processing is taking place. 7. Help should be built in, not separate to the system Rationale – Nurses don’t seem to use the help systems available and in a smaller system are very unlikely to use it 8. Standards should be followed but also explained – i.e. menu buttons Rationale – Nurses will not have enough experience to recognise all standards. 9. Explanatory text should be abundant within the system Rationale – Nurses will not have enough experience using IT systems so functionality should be clearly explained. European computer driving licence and ECDL information can be found on subsequent pages 106 Figure 12.1: Leaflet for the ECDL 107 Figure 12.2: Leaflet explaining modules in the ECDL 108 12.3 Appendix C: Cooperative evaluations Think aloud and cooperative evaluation of the computertechdoc.org system The following is a review of the Basic Computer Tutorial, found on the following website. http://www.comptechdoc.org/basic/basictut/ Details of system: This tutorial is a web based tutorial that was found my searching the web for basic computing tutorials. The first thing to note is that the site looks quite data and uninteresting, the format of the data is such that it is not eye catching. Although unattractive, navigation of the system is relatively simple with a scrollable frame on the left hand side allowing the different sections to be viewed. Fig 12.3: System homepage In order to evaluate this system, the think aloud protocol with cooperative evaluation was used. These evaluations did not take a great deal of time as the requirement was to get an overview of how a nurses would approach certain tasks. The following tasks were set forth for the participant to perform: Enter the website address Reviews the data in the introduction Review the data on computer hardware Try navigating the system from start to finish. 109 The participant commented that the site looked clear and it was easy to see what information was on the site. When asked about the scrollable navigational bar the participant commented that it was good that is stayed there as pages were navigated but she did not like having to scroll down to see all the menu items. The user read the data in the introduction and then reviewed the information on computer hardware. Most of the comments made were specific to the information that was reviewed and will therefore not be documented here. However comments about what the adverts for C++ on the right hand side were and the confusion that led after clicking the adverts is an example. When navigating from start to finish on this site the participant found it simple to go through each stage. The participant also commented on the fact that she was likely to become bored when using the system and that the site did not look very interesting. The participant also agreed that it would be difficult to know that they had learnt the information on offer and further admitted that a test providing feedback would be one way of doing this. Conclusion: The system is clearly a very simple website to provide IT tuition. The navigation is clear and the website uncluttered. However the site is boring to use and provides no facility in which to test knowledge to ensure everything is being understood. This evaluation has been a very quick look at the usability of an e-learning website, the main finding being that a clear navigational system is appreciated over high tech graphics and varied use of colours. 110 Think aloud and cooperative evaluation of the http://www.quia.com/servlets/quia.web.QuiaWebManager The following is a review of the quia e-learning website (address above) Details of system: Whilst browsing for tutorials this site was found and some IT learning features used. The site was then bookmarked and a nurse was asked to use the system to find IT training information. The result of what would happen during this walkthrough was unclear as was the exact nature of the site. The homepage of the site gave the participant a choice between entering an instructor zone and a student zone and the participant double clicked on the enter student zone part. Commenting on the fact that they didn’t have a password and should they register it was suggested that they should. She clicked on the “create my free account” and completed this information. The username used was already taken so the participant chose one of the alternatives offered. The user successfully registered, although being prompted not to double click as processing may have problems! The screen below was shown. Figure 12.4: Screen shown once registered to use the Quia system. When asked if it was clear where to navigated to find the information the participant commented that it was not clear at all and did not know where to go. It was suggested that she should try navigating the system, She tried clicking on add class in which she was informed that she had not entered a code. She then tried clicking on the student zone section and was shown the same information again. The user commented that the system was frustrating and that she did not no where to go, she clicked on the homepage. After looking through she homepage and hovering over the subject buttons the participant clicked on “and 100 other study areas” as shown in figure 8.3. 111 Figure 12.5: System homepage This brought up a list of topic areas, with the participant clicking on computer applications. The user commented that she had finally found some computing information. The user logged in using the information that she had written down and viewed some training material. The system would not allow the user to use the information and therefore the user could not access the training material found. To assist the user the evaluator prompted her to click on top activities under shared activities in which there was a computer basics icon. The participant clicked on this and navigated to screens such as is found below. Figure 12.6 Computer basics test 112 Description of quiz: This is a simple program that asks you to match devices to their descriptions. Although useful, a nurse commented that these types of games may be viewed as a waste of time by nurses and that simple learning materials and answers may be more appropriate. The nurse was able to use the program to completion. The nurse also navigated to the test shown in figure 8.5t and managed complete the test although it took a while for the participant to understand exactly what the purpose of the test was. Figure 12.7: Computer and Internet Terminology test Test description: This test that allowed players to compete against each other. The game asked each player to pick a category in which they would be asked a question. The nurse clicked on 200 and the page was displayed as in figure 8.6. 113 Figure 12.8: Question answered of the participant: The nurse was able to answer the question correctly and the following screen was displayed. Figure 12.9: After answering the question correctly The nurse was prompted to click upon another question and answer it incorrectly. The following screen appeared. 114 Figure 12.10: Getting an answer incorrect When asked about the feedback provided in the question the user commented that it is useful to know the correct answer when you get a question wrong. The participant was asked to conclude on their learning experience. It was commented that the system was very difficult to go through and find information. Furthermore the nurse commented that although the interactive games were good, with no information to read many of the answers provided were a stab in the dark. 115 Think aloud and cooperative evaluation of the NHS site http://nhs.dialog.com/ The following is a review of the NHS site designed to provide medical journals for NHS staff to aid learning. The nurse who participated in this evaluation used the site when needing research to complete a diploma as part of her continued learning development. This review was conducted informally and in an interview style to gain as much information from past experience. The walkthrough was a little bit different as the nurse has used the system before, therefore the nurse aimed to replicate some of the problems she had previously experienced with using the site. As this was a while ago it was clear that not all of this would be remembered and therefore the following will not document all the actions that were completed when going though the system for the purpose of space! Review of the system: Firstly the login page could be accessed and there was login help available, although this took time when the nurse had forgotten her password o previous occasions. The login was only given to NHS staff, hence the nurse logged into the system. When logging in the following screen was shown Figure 12.11: Page viewed when logging into the system.: 116 The nurse commented that their was not a lot of information on the homepage of the system explaining what to do and that it was unclear where to search for journals. The nurse recalled that she had to look up isolated leukaemia when previously using the system. The evaluator asked the nurse to perform this search. The user clicked on easy search at the top in the blue link and got to this page: Figure 12.12: Page viewed when clicking on easy search. This was actually tips on how to search, which while useful was misunderstood for the actual search pages. After review of the homepage it was clear that this was documented, although the nurse did not read this information. Furthermore this page opens up a new window and doesn’t allow the user to get back to the previous page. This is a big usability issue and the user conceded that it took some time to find where to search, Furthermore in this evaluation it took a while and it was clear why the nurse would have had problems. After navigating back to the homepage the correct button for easy search was selected from the bottom of the page. Furthermore when selecting this easy search an error message came up asking to select a database. Essentially you must select a database and press easy search. This error message was helpful in determining the problem. Furthermore the evaluator commented on the use of the squares and asked what they meant. The nurse said that she 117 could use the squares to select databases and seemed familiar with the concept. Once the nurse clicked on all databases and searched, the following page was displayed Figure 12.13: Page viewed when clicking on the easy search button. Various things can be accessed from the drop down menu, a series of searches were tried to get the correct answer . The nurse typed in isolation leukaemia and got some results, 7 the first seven of which were not relevant. Overall the following observations were made. It was decided that system like this was valuable to use, but the system suffers from a number of usability problems as discovered using the system. The use of explanatory text is useful but it doesn’t not add much as the nurse often does not read it. Furthermore, the help facility was not used by the nurses despite her facing problems using the system. The problem in recovering from clicking the wrong button was notable with the participant spending much time trying to get back from a new windows opening up. It was decided that navigation should not take the user to windows outside the system. Furthermore it was decided that search facilities should be demonstrated in class to help the user get the most of them. 118 12.4 Appendix D: Use cases The following use cases are documented: Register as a user of the system Login to the system Change your password View lesson material View peer support Take a test Receive the results of the rest Submit feedback on the test and the lesson material (for others to use) View all of your test results Complete the final test Logout Admin Administer test questions Review peer support of a particular lesson Delete a test from a users record to all them to retake. 119 Register to use the NITS system The nurse will need to register with the system in order to be able to login. The whole NITS system. USE CASE #1 Context of Use Scope Level Primary Actor Stakeholders Interests User Goal level A nurse. and Stakeholder Interest A nurse System Wishes to be able to access the NITS system Needs to store information about registered users. Success Guarantees Trigger The nurse is not logged in The nurse is not registered to use the system The nurse has navigated to the register page An unsuccessful registration does not result in incorrect data being added to the system. Registration is completed and the nurse can then successfully login. Nurse navigates to the register section of the NITS system Description Step Preconditions Minimal Guarantees Extensions 1 2 3 Action Nurse enters forename Nurse enters surname Nurse enters username Nurse enters password Nurse presses the ‘Register button, 4 The system registers the data Step 3a Branching Action One or more fields are left blank 3a1 The ‘register page is redisplayed 3a2 The field(s) previously not filled-in are highlighted in red 3a3 A message is displayed at the top of the screen telling the Nurse that the registration was unsuccessful due to the highlighted field(s) being left blank 120 USE CASE #3 Context of Use Scope Level Primary Actor Stakeholders Interests Change password The nurse may want to change the password that s/he is currently using. The logged in part of the NITS system User Goal level A Nurse Interest and Stakeholder A nurse The System Administrator Preconditions Minimal Guarantees Success Guarantees Trigger Description Wishes to change his/her password To make sure the system allows the nurse to successfully change his/her password. The nurse is logged into the NITS system The nurse is notified if the password change has been successful or not and therefore knows what password to use. The nurse is informed that the password has been successfully changed. The Nurse clicks on the “Change Password” button Step Action 1 The Nurse clicks on the “Change Password” button 2 The system asked the nurse for their old password and their new password 3 The nurse clicks submit query 4 Extensions Step 2a The nurse is informed that his/her password has been successfully changed. The system fails to validate the old password 2a1. The user is informed that their old password does not match Variations: 2a None 121 USE CASE #4 Context of Use Scope Level Primary Actor Stakeholders and Interests Preconditions Minimal Guarantees Success Guarantees Trigger Description View lesson material. The nurse will need to view tutorial material that provides information to help with the tests. The tutorial part of the NITS system. User Goal level A nurse. Stakeholder Interest A nurse Wishes to view tutorial information Trainer Wants to ensure the information is correct. The nurse is logged in The nurse has navigated to the appropriate tutorial. The information is displayed to he user The information is displayed in an appropriate format Nurse navigates to the appropriate tutorial section of the NITS system Step Action 1 Nurse clicks on appropriate tutorial session 122 USE CASE #5 Context of Use Scope Level Primary Actor Stakeholders Interests View peer support When viewing lesson material the user can view peer support information left by other users. Relevant to each individual lesson User Goal level An NHS nurse Interest and Stakeholder An NHS nurse Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions To view the information left and gain help on understanding the information Nurse who left peers That their information is of use support The nurse is registered to use the system The nurse is logged on The nurse is viewing tutorial information. Empty peer support section is displayed Nurses can view all peer support information in a useful order. Nurse clicks on the peer support section. Step Action 1 The system keeps session information and recognizes users 2 Nurse presses the appropriate tutorial chapter 3 Nurse reviews data 4 Nurse clicks on the peer support section 5 Nurse is presented with information that other nurses have left regarding why they chose their answers. Step Branching Action None Variations The nurse does not view lesson material and navigate straight to peer support section 123 USE CASE #7 Context of Use Scope Level Primary Actor Stakeholders Interests Receive results The nurse has completed a test and wants to receive results on this. The logged in section of the NITS system. User Goal level A Nurse Interest and Stakeholder A nurse The training manager Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Wants to know if s/he has passed the test Wants to ensure that the nurse understands the correct result. The nurse has taken a test No data should be altered within the system Feedback should be appropriate The nurse should receive results to all questions with details of the correct result and scoring. The Nurse submits a test. Step Action 1 The Nurse submits a completed test 2 The system compares this to the correct data and provides feedback Step 2a None Variations: 2a None 124 USE CASE #8 Context of Use Scope Level Primary Actor Stakeholders Interests Submit peer-support The nurse wants to leave information relevant to the chapter to help colleagues completing the chapter The logged in section of the system User Goal level A Nurse Interest and Stakeholder A nurse Another nurse Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Wishes to peer-support Wants the nurse to leave feedback so s/he can view it The nurse has taken a test and is viewing the results page Peer support information is entered into the system Peer support information is entered into the system and the user is notified of this. The Nurse submits a test and starts entering peer support. Step Action 1 The Nurse submits a completed test 2 The system compares this to the correct data and provides feedback The nurse fills in peer support information The nurse submits the information Step 2a None Variations: 2a None 125 USE CASE #9 Context of Use Scope Level Primary Actor Stakeholders Interests View all test results The nurse will need to log out when wanting to leave the system The whole NITS system User Goal level A Nurse and Stakeholder Interest A nurse The training manager Wishes to leave view feedback on why they answered incorrectly Wants to ensure that the nurse understands why they were incorrect. Preconditions The nurse has taken a test Minimal Guarantees No data should be altered within the system Feedback should be appropriate The nurse should receive feedback on everything that s/he got wrong. The Nurse submits a test, or views the results page Success Guarantees Trigger Description Step 1 2 Action The Nurse submits a completed test The system compares this to the correct data and provides feedback Extensions Step 2a The system fails to validate the logout 2a1. All session data is destroyed 2a2. The browser informs the user of an error and redirects to the index.php page. 2a The nurse views the results page of the system and is provided with test scores and feedback on things s/he didn’t know. Variations: 126 USE CASE #10 Context of Use Scope Level Primary Actor Stakeholders Interests Complete all tests The nurse has completed all the tests in the system Logged in section of the NITS system User Goal level A Nurse Interest and Stakeholder Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions A nurse Wants to pass the course The training manager Wants to ensure nurses pass the course. The nurse has taken all tests The system stores information on all tests taken The nurse submits the final test and if passed is told they have completed the course and should be able to access a certificate. The Nurse submits information on the final test. Step Action 1 The Nurse submits the final test 2 The nurse passes the final test The nurse is informed they have passed and is awarded a certificate. Step None Variations: 2a The nurse fails the test and is told to speak to a training representative. 127 USE CASE #11 Context of Use Scope Level Primary Actor Stakeholders Interests Log Out The Nurse will need to log out when wanting to leave the system The whole Nurse System User Goal level A Nurse Interest and Stakeholder A Nurse The System Administrator Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Wishes to leave the Nurse system To make sure the system allows the Nurse to successfully log out of the system The Nurse is logged into the Nurse system Any transactions made whilst the Nurse was logged in shall endure. It will be clear whether logout was successful or not. The Nurse will not be able to access the system once logged out until they next log in If the log out is successful, the Nurse will be informed they have successfully logged out and presented with the log in screen. The Nurse clicks on the ‘Log Out’ button Step Action 1 The Nurse clicks on the ‘log out’ button 2 The system validates the logout and removes any access rights 3 The Nurse will be informed they are logged out and presented with the log in screen Step 2a The system fails to validate the logout 2a1. All session data is destroyed 2a2. The browser redisplays the login page Variations: 2a 2b The Nurse is idle for more than fifteen minutes. The system recognises that the Nurse has not performed any actions in more than five minutes, so the system automatically logs the Nurse out. The Nurse closes the web browser without logging out, the system will automatically log the Nurse out of the system. 128 USE CASE #12 Context of Use Scope Level Primary Actor Stakeholders Interests Administer test questions The administrator wants to set some test questions and answers. Admin section of the NITS system User Goal level Administrator Interest and Stakeholder Administrator Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions Wants to enter system questions and answers Logged in as administrator The tests in the system remain unchanged and the administrator is told that administration is unsuccessful. Questions and answers are changed successfully in the system. The administrator clicks on a test to administrate. Step Action 1 The administrator clicks on a test to administrate 2 The administrator changes the required questions and answers The administrator clicks submit. The administrator can then view the data to ensure it is correct Step None None Variations: 2a The administrator may choose not to view the data once it is changed and trust the system has executed the changes properly. 129 USE CASE #13 Context of Use Scope Level Primary Actor Stakeholders Interests Review feedback of a particular lesson The administrator can view feedback left from nurse son a particular lesson Admin section of the NITS system User Goal level A Nurse Interest and Stakeholder Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions A nurse Wants to pass feedback The system administrator Wants to view feedback Nurses have left feedback on the chapter The data is shown to the administrator The data is shown to the administrator The administrator clicks on view feedback for a chapter. Step Action 1 The administrator clicks on view feedback for a chapter. 2 Feedback is displayed Step None Variations: 2a No feedback has been left. The administrator is informed of this. 130 USE CASE #14 Context of Use Scope Level Primary Actor Stakeholders Interests Delete a test from a users record to all them to retake. A nurse needs results of a test to be deleted so s/he can take the test again Admin section of the NITS system User Goal level An administrator Interest and Stakeholder Preconditions Minimal Guarantees Success Guarantees Trigger Description Extensions A nurse Wants to take the test again The administrator Wants to delete the user’s record. The nurse has taken all tests. The system informs the user that the test data was not deleted. The test is deleted from the system and the administrator is notified of this. Clicks on the delete tests button Step Action 1 The administrator checks the box of the record they want to delete 2 The administrator clicks submit The administrator is informed that the test has been deleted. Step The administrator checks 2 or more boxes for deletion Variations: 2a None. 131 12.5 Appendix E: Questionnaire (see following pages) 132 Survey into the NITS system and NHS training Your name Age (not compulsory!) Amount of time in nursing Thank you for taking part in this study. Please answer these questions honestly about your learning, the results will help to make recommendations on improvements to NHS IT training in the future. The information will be used to produce a report and will not be kept for longer than necessary. Your names will remain anonymous. IT ability and experience 1. 2. How would you describe your IT abilities? o Expert o Proficient o Adequate o Poor o Very poor Have you received any prior IT training? o Yes o No Please give details: 3. Please give details of the IT systems you have used at work or to support your nursing 133 Your learning experience 4. 5. Overall how satisfied were you with your learning experience? o Extremely satisfied o Satisfied o Neutral o Not satisfied o Not at all satisfied. Did you find the NITS system intuitive? o Crystal clear o Clear o Fairly intuitive o Difficult to understand o Impossible to understand what’s going on Please give reasons for your answer: 6. Was the format of the information easy to read? o Crystal clear o Clear o Fairly clear o Difficult to read o Impossible read Please give suggestions on how it could be improved? 134 7. Did you find the peer support section useful? o Very useful o useful o It was worth looking at o I used it but it didn’t help o I didn’t use it Please give suggestions on how it could be improved/what would make you use it? 8. Would you have preferred to have learnt the information in a classroom? o Yes o No If yes, please give your reasons: 9. Please give the advantages you see with online training: 10. P lease give the disadvantages you see with online training: 135 NHS training 11. Are you satisfied with the level of training provided by the NHS? o Extremely satisfied o Satisfied o Adequate o Not satisfied o Not at all satisfied. If no, please give your reasons: 12. Are you satisfied with the level of IT training provided by the NHS? o Extremely satisfied o Satisfied o Adequate o Not satisfied o Not at all satisfied. Please give your reasons: 136 13. Do you think that online training systems could be a sensible way to administer training within the NHS? o Yes o No Please give your reasons: 14. Please give suggestions on how the NHS could improve its training regime: 15. Any other comments/suggestions: Thank you for taking part in this study and taking the time to complete the questionnaire. 137 12.6 Appendix E: Test results For confirmation, the following tests were used: Standard user 1. Register then login to the system 2. Change your password 3. View lesson material 4. View peer support 5. Take a test 6. Receive the results of the rest 7. Submit feedback on the test and the lesson material (for others to use) 8. View all of your test results 9. Complete the final test 10. Logout Test No 1: Register then login to the system Test Expected Result Click on Takes you to register.php register Complete all User is notified that the information is details and submitted, relevant data is inserted into the submit user table, which shows the information inserted. User clicks Takes you to login.php on login. User enters System logs the person in if the information username and is correct or provides an error message if the password and answer is incorrect. Upon submission user clicks submit is logged in and taken to page home.php. Actual Result No variation Passed? Passed No variation Passed No variation Passed No variation Passed **If any fields are left blank, the test is unsuccessful and the user has to enter the information Test No 2: Change your password Test Expected Result Click on change Takes you to change_password.php password Old password and User is notified that the password new password has changed entered, submit button clicked. Test No 3: View lesson material Test Expected Result Click on lesson 1 Takes you to lesson1.php Navigate through all Navigation is successful the pages in the test follows a sequential order 138 Actual Result No variation Screen blank Passed? Passed left Failed Actual Result No variation and No variation Passed? Passed Passed Test No 4: View peer support Test Expected Result Actual Result Click on “view others Takes you to peer_support.php No handy hints and tips” variation Is data readable Data should appear in the order of No the person who left it. variation Passed? Passed Passed Test No 5: Taking a test in the system See main report. Test No 6: Look at the results of the all tests taken (in this case all of them). Test Expected Result Actual Passed? Result Click test answers Takes you to test_results.php No Passed from anywhere once variation logged in have been selected. The page displays the In this case, four test results should No Passed results to all the tests be displayed, with the ability to variation taken so far view and print the certificate that is generated., Test No 7: Submit feedback on the lesson taken Test Expected Result Actual Result Click on test 1 Takes you to test1.php No variation Complete test and User is notified that the information No submit data is submitted, relevant data is variation inserted into the table test1_answers, which shows the answers given. Presented with marks System takes you to test1_result, No and feedback on the marking the results correctly and variation results. giving the option of leaving peer support information Passed? Passed Passed Passed Test No 8: View all of your test results Test Expected Result Passed? Click on test 1 Passed Complete test submit data Actual Result Takes you to test1.php No variation and User is notified that the information No is submitted, relevant data is variation inserted into the table test1_answers, which shows the 139 Passed answers given. Presented with marks System takes you to test1_result, No and feedback on the marking the results correctly and variation results. giving the option of leaving peer support information Test No 9: Complete the final test Test Expected Result Click on test 1 Takes you to test1.php Complete test and User is notified that the information submit data is submitted, relevant data is inserted into the table test1_answers, which shows the answers given. Presented with System takes you to test1_result, marks and feedback marking the results correctly and on the results. giving the option of leaving peer support information Test No 10: Logout Test Click on test 1 Complete test and submit data Expected Result Takes you to test1.php User is notified that the information is submitted, relevant data is inserted into the table test1_answers, which shows the answers given. Presented with System takes you to test1_result, marks and feedback marking the results correctly and on the results. giving the option of leaving peer support information Passed Actual Result No variation No variation Passed? Passed Passed No variation Passed Actual Result No variation No variation Passed? Passed Passed No variation Passed Actual Result No variation No variation Passed? Passed Passed No variation Passed Admin user: Test No 10: Administer test questions Test Expected Result Click on test 1 Takes you to admin_test1.php Submit information The admin_test1_result.php page is with question data shown as required The answer The admin_test11.php page is information is shown saying that the information updated and the has been updated and providing a submit button is link to be pressed to view the data pressed 140 Test No 11: Review feedback for a particular chapter Test Expected Result Actual Result Click on view Takes you to feedback.php. Data No variation feedback can be viewed. Test No 12: Delete a test from a users record to all them to retake. Test Expected Result Actual Result Click on user results Takes you to userresults.php No variation Navigate to Appropriate record can be found No variation appropriate record for deletion Check the box to System takes you to No variation delete the record userresults_result.php, informing and press submit the administrator that the record has been deleted. 141 Passed? Passed Passed? Passed Passed Passed 12.7 Appendix G: Walkthrough summaries Three walkthroughs have been summarised out of the seven carried out. The results of the other summaries are included in the project results. Walkthrough 1: Julie Ackland Aim: • Register as a user • Login • Read and comprehend lesson 1 • Log out • Log back in • Take test 1 • Read and comprehend lesson 2 • Take test 2 • Read and comprehend lesson 3 • Take test 3 • Look at the results of the test so far • Read and comprehend lesson 4 • Take test 4 Time taken: 40 minutes. • • • • • • • • • User read the homepage and clicked on the register button. Interestingly the evaluator was asked if the password needed to be the same as the password used for their e-mail. Registration was completed successfully. The user clicked on the login button on the homepage. The user commented that “submit query” was a funny expression because she just wanted to log in. Agreed that “submit” would be more sensible. The user seemed a little panicky and commented that upon clicking log in she had already forgotten her username. She also commented that this had been a problem with other systems, however, it didn’t take long to remember and she logged in successfully. When the user got to the homepage she didn’t look comfortable and said “haven’t I been here before”, presumably because it looked similar to the first page before she logged in. She also needed prompting the go the first lesson. The user asked if she needed to view the lesson material, or if she could simply take the test. The evaluator responded that this was up to her but it was advisable to look over the content. Lots of comments about the material were made, the user thought it was quite deep and suggested that it was a lot of information to take in. The evaluator responded that she should take as much time as she needs in order to feel comfortable. As further data was navigated, there was clearly noticeable tension “do I need to memorise all of this”. It seemed a general panic and poor self efficacy were displayed. 142 • • • • • • • • Many question about the content were asked, most were answered as the aim of this study was not to test the information in the tutorial. The user logged out and then tried to log back in. However, the user clicked on the admin login section and was confused as to why she had not been logged in. From this it seemed clear that she did not read all the information on the button. After prompting and clarification she logged in successfully.. Lesson Information seemed to be skimmed rather than read, maybe because she was conscious that she was being watched. The user clicked on the feedback and hints section after hovering over the “take test” button. She admitted that she would look at this to get hints. The nurse was confused by what the section headings alluded too. After viewing all the feedback, the user commented that she was “unsure” about how to get back to take the test. Without a reply she scrolled up the page and navigated to the test. The user asked whether they could select multiple answers or not, showing that they were not familiar with radio buttons. The user asked if she had to leave feedback. The evaluator commented that it was her choice, so she decided not to leave feedback. She commented that she could not remember the questions to give feedback anyway. The participant decided to take a break here. After discussion with the participant it was decided that the tests and interaction with the system would be reviewed, but that the evaluator would not observe as the participant views the test information. Viewing the lesson material took lots of time and the participant agreed that it was difficult to read with someone watching. • • • • The participant logged into the system to read session 2 and reported no problems reviewing the data. She then clicked on the peer support section suggesting maybe she should read it. When asked about the usefulness she commented that their was not much information in it (only a few dummy entries were added beforehand), but that it would be interesting to see what others put. She then clicked on test 2 to take the test. She passed the test and suggested that the peer-support information had been useful. The participant did not have any further time and the third chapter was taken up a few days later. • The participant logged into the system to read session 3 and reported no problems reviewing the data. • She then clicked on the peer support section again commenting that not a lot of information was available from there. • She then clicked on test 3 • She failed this test getting 5 out of 8. • Discussion over the large amount of material in this section was discussed with the recommendation that the amount of data should be cut down. The participant then took a short break before going onto the final chapter. 143 • • • • Upon returning the participant found that she had been logged out of the system. She suggested that the system should warn you of this and explain that it has happened as she was left a little confused. She logged in and read session 4 She chose not to view the peer support information commenting that the information seemed easier She took the test and passed, receiving 8 out of 8. The evaluator then debriefed the participant saying that in order to get a certificate she would need to retake test 3. As the participant wanted the certificate, the mark for the test failed was deleted, the participant took the test again and passed. She clicked on test results and then clicked to get the certificate. 144 Walkthrough 2: Patrisha Ash Aim: • • • • • • Register as a user Login Read and comprehend lesson 1 Take the test Change password View all test results. During this and subsequent walkthroughs, the user was watched when performing all actions apart from reading tutorial information. Time taken: 25 minutes. • • • • • • • • • • • • • • User clicked on the register button quickly Logged in, remembering username and password first time User clicked on lesson 1, the evaluator asked them to call me when they had finished reading the information. The evaluator came back to find the user taking the test. The evaluator asked if the peer support information had been viewed, the user responded no. Test taken quickly, some musings over the answers. Marks attained, user pleased with result. User looked over questions that she got wrong, commented to the evaluator on why they got them wrong User went to leave peer-support but left specific information that would be unlikely to help users. User entered the information, asked if this was the sort of thing wanted. Without wanting to influence the evaluator asked her to enter the information she would enter if he was not here User commented that if she was on her own she might be less likely to leave support information. User commented that she was unsure if people would be interested in her comments and reasoning’s on choosing a particular piece of information. the evaluator then asked the user to change her password o Quickly she clicked on the appropriate button and changed her password. The evaluator then asked the user to view the results of her test so far, which she did without deliberation. The evaluator then told the user that she had finished the test; she asked if she needed to logout, the evaluator replied yes so the user clicked the logout button. The evaluator commented that the user had completed the test very quickly. She commented that as she knew the results didn’t really matter she didn’t spend lots of time reading the information. She commented that if it had been a proper test, she may have taken longer answering the questions. She then discussed people she had worked with designing systems for staff and introduced the term “nurse proof”, which had been used as systems have to be designed specifically for nurses to be able to use them. 145 Walkthrough 3: Francis Ackland Aim: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Register as a user Login Read and comprehend lesson 1 Take the test Read and comprehend lesson 2 Take test two Change password View all test results. Read and comprehend lesson 3 Take test 3 Read and comprehend lesson 4 Take test 4 View test results. Time taken up to step 4: 1hr 15 minutes Time taken up to step 6 + 35mins Time taken up to step 10 + 1hr 47 Time taken up to step 13 + 54mins • • • • • • • • • • User registered and had to be prompted to use her e-mail address as he username. As she didn’t have an e-mail address, she entered a fake address. This will need to be addressed. User clicked on the top login button and entered her details to login to the system. User navigated to first chapter and started reading the information. At this point I left and asked the user to call me when she had read the information. She called me back and was reading the information on the peer-support section The evaluator asked how the material was, she commented that she could not take all the information in. She said that seeing other peoples comments was very useful, especially specific information that was given about firewalls, which helped clarify this information for her. The user then clicked on test 1. The test opened and she started answering the questions. She asked if she could choose more than one answers as she tried it, obviously realising that she couldn’t. Test was completed and answers submitted. User seemed to be a little nervous, breathing deeply and getting close to the screen to read the information. Results were scrutinised closely, the user commented that she couldn’t remember what the question was. The evaluator stepped in to help out. User left peer support just on the questions she got incorrect explaining her reasoning clearly. This information was unprompted and was exactly what was required from this section. User submitted the information. The user then took a break, leaving the system logged on. Notes end here as documenting was lengthy and added little value. 146 Summary of findings from further walkthroughs • Formatting of test results should be improved. • Next and back buttons should be at the top and bottom of scrollable pages Results of questionnaires. IT ability and experience How would you describe your IT abilities? Expert 0 Proficient 0 Adequate 1 Poor 3 Very poor 3 Have you received any prior IT training? Yes 2 No 5 Please give details: 1 user had completed the ECDL 1 user has taken even classes on computing skills at a local college Please give details of the IT systems you have used at work or to support your nursing Database of journals Your learning experience Overall how satisfied were you with your learning experience? Extremely satisfied 1 Satisfied 5 Neutral 1 Not satisfied 0 Not at all satisfied. 0 Did you find the NITS system intuitive? Crystal clear Clear Fairly intuitive Difficult to understand Impossible to understand what’s going on Please give reasons for your answer: It was kept simple which helped. 2 5 0 0 0 Was the format of the information easy to read? Crystal clear Clear Fairly clear 147 0 4 2 Difficult to read Impossible read Please give suggestions on how it could be improved? Bullet points would be nicer than lots of text More pictures Less information 1 0 Did you find the peer support section useful? Very useful 0 Useful 4 It was worth looking at 1 I used it but it didn’t help 1 I didn’t use it 1 Please give suggestions on how it could be improved/what would make you use it? • I didn’t really know what it was trying to do, what information should be added • I’m not sure that people will find my comments useful • I couldn’t remember the questions Would you have preferred to have learnt the information in a classroom? Yes 0 No 7 If yes, please give your reasons: Please give the advantages you see with online training: Can do it in your own time Can use it at home It makes me use a computer! You don’t have to travel Please give the disadvantages you see with online training: You have to use a computer It can be tedious NHS training Are you satisfied with the level of training provided by the NHS? Extremely satisfied 0 Satisfied 3 Adequate 3 Not satisfied 1 Not at all satisfied. 0 If no, please give your reasons: Training days are taken in our own time and normally aren’t very useful. 148 Are you satisfied with the level of IT training provided by the NHS? Extremely satisfied 1 Satisfied 1 Adequate 0 Not satisfied 4 Not at all satisfied. 1 Please give your reasons: The ECDL was very useful, it made this test easier! I haven’t taken any I didn’t know they offered IT training x 2 Do you think that online training systems could be a sensible way to administer training within the NHS? Yes 7 No 0 Please give your reasons: Advantages mentioned above Please give suggestions on how the NHS could improve its training regime: Provide more time to do training Pay us for it Any other comments/suggestions: 149 12.8 Appendix E: NHS IT training: Recommendations document 1. Introduction .................................................................................................................. 151 2. Document scope.......................................................................................................... 151 3. Recommendations on training strategy ............................................................... 151 3.1. Training framework .........................................................Error! Bookmark not defined. 3.2. Allow structure so all training doesn’t take place in nurses own time. ...................................................................................................................................... 152 3.3. Make training requirements clearer .................................................................. 153 3.4. Provide tutors to help ............................................................................................ 153 3.5. Use e-learning.......................................................................................................... 154 3.6. Conclusion ................................................................................................................. 154 4. Recommendations on IT training ........................................................................... 155 5. Recommendations on using the NITS system ................................................... 157 5.1. Benefits ...................................................................................................................... 157 6. Drawbacks..................................................................................................................... 158 6.1. Usermanual........................................................................Error! Bookmark not defined. 6.2. Enhancements.......................................................................................................... 161 6.3. Conclusion ................................................................................................................. 163 7. Conclusion ..................................................................................................................... 163 8. Evaluation...............................................................................Error! Bookmark not defined. 150 Introduction This document covers recommendations gathered from research into the best way to train NHS staff with the use of IT. The findings have come from empirical studies, documented in the overall project document, which can be found here: www.oligarchic.co.uk/jackland/project/ 1. Document scope This document provides a detailed account of the recommendations made following this project. The aim of this document is to compliment the NITS system, being the final deliverable to the NHS as recommendations on how to help nurses with the use of IT and how to continue to improve the training they receive. The document covers all the recommendations found in the project and should tie the NITS system together with the project research and findings. This document does not attempt to explain how the information used to make recommendations was gained and the techniques used, rather purely focuses on the results. Again, to find this information, please go to the project document: URL at the top of the page. 2. Recommendations on training strategy This section covers the general recommendations for the NHS on their training strategy, attempting to recommend a suitable framework for all future training, based on the information in the literature review. It is worthwhile noting that despite conducting a literature review, the exact 151 details of the NHS’ training strategy was not fully ascertained, as it was difficult getting time with personnel who could discuss this information. With the amount of time spent on this study, it would be unreasonable to assume a full strategy could be recommended and critiqued. Further research would need to be undertaken and far more time and experience would be required. However, in this document, a suitable framework and recommendations will be discussed based on the information known. Although I don’t expect this to offer a complete solution, the recommendations may be useful to NHS training representatives or software developers making a system for nurses use. Furthermore an outsider’s view on improvements to be made can often be useful for most organisations. 2.1. Allow structure so all training doesn’t take place in nurses own time. One of the key problems identified with nurses training is that the majority of the training needed to take place in the nurses own time. This has meant that nurses are sometimes unwilling to take the courses and will sometimes do the bare minimum required to succeed, this is particularly clear with IT training. This reluctance is understandable. Although other organisations may expect their staff to train within their own time, this is often accepted because of the pay rewards offered or the stature of the job. Nursing is widely regarded as an underpaid job and therefore asking for more work from nurses can be met with challenges. Furthermore, nurses are unlikely to have spare time at work to catch up with training, although this is more likely for nurses who work nights. Any training that requires the nurse to travel should surely reimburse the nurses with any costs associated with this. Nurses commented during 152 interviews that costs any training given to nurse should have all costs associated with training were not covered by the NHS. The emphasis seemed very much on nurses to ensure that they were upto-date with any training that needed to be conducted, no time was provided for paid training. This surely needs to be addressed in order to ensure that nurses do get suitable time at work to partake in the available and necessary training that is being offered. This seems to be starting to be addressed within the NHS, with the emphasis on evidence based practice and nurses proving they are qualified to do their job. 2.2. Make training requirements clearer Nurses seemed unclear on exactly what the requirements for training were and these standards seemed implicit rather than explicit. Most training seemed optional rather than compulsory and therefore nurses seemed a little unclear on what they should be training on. 2.3. Provide tutors to help Learning is always easier when you have a tutor to go to; tutors are used in all training institutions including University. In other organisations, managers are used as tutors in many situations, providing help and support for people when they require it. The literature review stressed the importance of self efficacy in learning. During the study it was clear that nurses sometimes lacked confidence when training. Nurses are often concerned about the repercussions of failing tests, having often not taken them in a while and have commented that they do not have support from tutors. 153 2.4. Use e-learning The response to e-learning from nurses has been particularly positive. The idea of learning in their own time and in their own setting seems very popular with nurses, especially due to the fact that many have children or duties in the home. If learning is to continue to be performed in nurses own time, increasing the use of learning is an excellent recommendation. The relative advantages and disadvantages of e-learning are discussed later in this document. However it is clear that the use of e-learning should be limited and supported by other means of training. Blended learning as discussed in the literature review should be used, so current training days offered could be supported by the use of e-learning and self directed learning. However, nurses who have never used a computer may object to being taught how to use one by using one. E-learning is not a substitute for classroom learning; rather it can be used to compliment it. 2.5. Conclusion Above there are many general recommendations on the NHS’ training strategy. It is clear that more structure should be provided for nurse training and a clear direction on the training that should take place. More time should be given to nurses for training, they should not be expected to do this training on their own time. This will enable more effective training as nurses will be more engaged with the training and willing to take part. It is clear that e-learning could be used to offer a range of training activities, to allow the nurses to study in their own time. However, this should be supplemented, other techniques should be used. 154 3. Recommendations on IT training The last section looked specifically at the training strategy used by the NHS for general training. This section will look specifically at IT training. Whist it is clear that there are training regimes within the NHS for IT; the following will include some recommendations on how to improve this training to help nurses cope with the IT that they use. It seems that although IT courses are available, as they are not compulsory, nurses do not see the value in attending these courses. However, it seems clear that most admit their skills in using IT are poor, furthermore IT is becoming more of a necessary skill for nurses. A simple solution would be to ensure that all nurses have to attend and pass an IT training course, possibly the ECDL. Furthermore, IT will only be taken seriously if high grade nurses and managers take it seriously. The example should be set from high and this will encourage more nurses to partake of this training. Self efficacy/confidence is something that has been highlighted as lacking when nurses use IT during this study. It seems that when using elearning, the nurse often wants to have someone else present, or someone to turn to should they require help. Tutors are known to enhance learning; this is supported by the use of blended learning. The response to the peer support section has been mixed, mainly due to confusion over the aims of this section. Peer-support is normally used by people before taking tests and it seems that nurses are willing to leave information for other nurses to use. Nurses are often friends with other 155 nurses and are therefore happy to provide information to help colleagues. Nurses are often interested in how their peers are doing on tests, so some forum or further way of enabling nurses to interact about their use of IT is likely to interest nurses. It is clear that there is a lot that can be done with IT training within the NHS. E-learning is a suitable and favourable way of delivering a lot of this information, but the use of training days in which nurses can work in teams could be useful to encourage peer support for nurses. Group work can often improve self efficacy. Although there are budgetary problems within NHS training, it is clear that nurses need more training than they are currently provided with the growing use of IT, or they should at least attend the IT training that is available. Helping nurses to help themselves use IT is a sensible way of encouraging this training with the financial restrictions the NHS has to work to. 156 4. Recommendations on using the NITS system THE NITS system is a high level prototype system for an e-learning system to be used within the NHS. These recommendations will cover how the system could be implemented if it were to be used by the NHS and recommendations on how to ensure that the e-learning system is beneficial for the user. A review of the benefits and limitations of the system will help in determining this. 4.1. Benefits One of the main benefits of the NITS system, and maybe an unexpected one is that the nurses liked e-learning and thought it was particularly useful for them. One nurse said that e-learning was “ideal” for an organisation such as the NHS, in which most training is conducted informally. One of the main benefits for nurses was that they can study in their own time. Because of the time constraints nurses have, the ability to do a short period of study when they want to, rather then dedicating lots of time to a particular course seems to be a favourable characteristic. Furthermore nurses can be a bit embarrassed about their IT skills and like being able to use a system on their own, in which they can go at their own pace. Learning the basics away from others can often make learners happier. Furthermore with the system being web enabled, nurses can complete the training in the comfort of their own home, something particularly useful for nurses who often have families to attend to. Another benefit of the system being web enabled is that nurses seemed familiar with the Internet and browser technology. Other systems may 157 involve more user “ramp up” time in which they need to become familiar with the interface. They were also familiar with phrases such as login and logout. They were also able to differentiate between what was a clickable item and what wasn’t; in essence nurses seemed to be relatively familiar with web standards. Nurses were happy to take part in IT training and seemed to view the need to learn the information one day as inevitable. As with any training, nurses need to be able to see that there is a point to learning this information and with the NITS system, they can see the point of learning the data in the system. As mentioned in the last section, the peer support seemed likely to be viewed, and the hints left were often helpful for users. Nurses were willing to use this and leave feedback; therefore the provision of some kind of peer support seemed like a worthwhile feature. 5. Drawbacks The NITS system was not entirely “nurse proof” and there are many recommendations on improvements, which can be found in the next section. It seems that some features needed more explanation as nurses seemed unclear about what to do. However, the study showed that nurses are likely to skim data presented to them, so this would need to be done in a concise way. It was very difficult to see how nurses would interact with the system if they were in their own setting and not taking part in the study. Nurses did not seem to take the tests that seriously and were happy to slim through the lesson material in order to take the tests. Nurses often commented 158 that if the test meant something they would have taken more time over it. Therefore the study may not be a good indication on how nurses learn. It was clear from the walkthroughs that nurses were acting as if they were being watched. It was also clear that nurses were aware of the time being spent, and didn’t want to spend large portions of their time on the study. When asked about skimming the information, the user commented that if it was an actual test, she would have spent far more time on it to ensure she got the correct answers. Despite usability testing in the pilot study, it seems there were a number of areas that nurses were not clear on. The kind of information to be left in peer-support was the main one, with nurses often not sure what to write and writing the wrong kind of information. Similarly when taking the test, the use of radio buttons was not self explanatory and needed more textual instruction. Furthermore having a login for admin users on the homepage left one user confused enough to not be able to login. As the admin logon page looked very similar to the standard login page, the nurse didn’t differentiate between the two sections. The use of “submit query” buttons confused one user and on reflection these buttons should be changed to “submit”. This would probably be true if the system was used by a standard user, but the problem was amplified when used by nurses. A drawback of the study into the use of the NITS system is that only a small sample of nurses were tested (NHS website). For more conclusive results are larger sample from a wider geographic area should be used. 159 5.1. User manual The decision not to use a user manual seems like a correct one as no one using the NITS system required help. Furthermore, no participants used clicked the button, so no one would have wanted a help system! Rather this kind of help should be 160 integrated into the system. 5.2. Enhancements The following are recommendations as to enhancements that should be made to the NITS system. A table with recommendations and rationale as to these recommendations is displayed below. No Recommendation Rationale 1 More explanation on the registration of Users didn’t know if they could the system, what to select as their create there own passwords or username and what is to be used as whether they had to be the the password should be given same as their e-mail address password. 2 Changing all “submit query” buttons to Users didn’t understand what a “submit” 3 query was. Ways of remembering usernames and Nurse passwords should be provided. • • (1) commented remembering that usernames often and Identifying questions: mothers passwords maiden name etc differentiate which passwords A button that can mail their should password to their e-mail system. be used can’t for each address 4 More information on the homepage, User thought the logged in clearly differentiating it from the pages homepage was the same as the when the user is not logged in. 5 homepage when not logged in. The homepage should contain more The user seemed confused by information about the purpose system and what she should do. 161 of what logged they in. should do Having when clear instructions on what is expected is required. 6 The material used is very important, Lots of comments were made should be at the right level for the about the material for learning, nurses. as this is not the focus of this project it will not be discussed here. “Lots of material made the user panic” “Do I need to memorise all this” 7 Admin logon should be moved away One user clicked on the admin from the links on the logged out logon instead of the standard homepage. logon and could not understand why it wouldn’t allow her to login. 8 Lesson information should be bulleted Users and broke up tended to skim the information rather than reading whole sentences. Questionnaire feedback suggested this. 9 “Sections” should be clearly identified One user didn’t leave feedback so that the user knows what they are and leaving feedback on. commented that they didn’t know which section they were leaving feedback on. 10 Getting to the next logical step of the One user commented that once system should be at the beginning and reading peer support, she didn’t end of each page know how to get to the test (The next logical test). 11 Tests using multiple choice questions When using radio buttons, the should clearly explain what is required nurse was unsure of whether 162 rather than relying on standards such she could choice one or two as radio buttons. 12 answers. Peer support should be encouraged Some nurses did not view this and sold more to the nurse 5.3. information. Conclusion The use of the NITS system was a relative success and with a few enhancements could successfully be implemented for use with nurses. Furthermore it is clear that systems for nurses should be designed specifically with the end user in mind, an off the shelf package for training is unlikely to meet requirements and be effective in training them. 6. Conclusion This document has provided recommendations for the NHS on training with particular reference to IT and e-learning. It is clear that NITS and elearning can have a role to play in training NHS staff and e-learning will likely become more popular as the NHS embraces the IT revolution. It is important to remember that nurses are a different user subset; they should not be treated in the same way as other users. In the foreseeable future, it is likely that nurses will struggle using IT, therefore patience is essential in helping nurses adapt to the challenges that IT presents. 163