Evaluation of Online Collaborative Technology Needs

Evaluation of Online Collaborative Technology Needs

Table of Contents

• List of Tables
• Executive Summary
• 1 Introduction
  • 1.1 Background
  • 1.2 Scope of Evaluation
  • 1.3 Needs Evaluation Questions
  • 1.4 Data Collection and Methodology
  • 1.5 Methodological Considerations
• 2 Evaluation Results
  • 2.1 Relevance
  • 2.2 Needs Assessment
• General Conclusion
• Recommendations
• Annex 1: Questionnaire Sent to Community of Practice Members
• Annex 2: Questionnaires Used for Interviews

List of Tables

• Table 2.1: Number of Communities and Members, 2005-2009
• Table 2.2: Four Categories of Virtual Communities of Practice
• Table 2.3: Satisfaction Among Users of Virtual Communities of Practice
• Table 2.4: Services Offered and Services Used, Based on the 36 MOUs
• Table 2.5: Statistics on Portal Use
• Table 2.6: Use of Tomoye Features by Community of Practice Members
• Table 2.7: Activities in Tomoye and New Activities
• Table 2.8: Bilingualism and Access from Outside the Office
• Table 2.9: Collaboration
• Table 2.10: Services and Training
• Table 2.11: Work Environment

Executive Summary

Background

The Canada School of Public Service (the School) conducted an evaluation of the needs of members of the virtual communities of practice that use the Tomoye groupware, one part of the School's Online Collaborative Technology Program. The main purpose of the evaluation was to determine the relevance of the program and to analyse the most frequently used features, the features that users would like to see included in the new online collaboration platform, and the options available to the School. The needs evaluation was intended to assist senior management in future decision-making with respect to the new technological platform and feature options.

The Online Collaborative Technology Program has several components, foremost among them the Centre of Expertise in Communities of Practice (the Centre). The mandate of the Centre, which was founded in 2006 in the Quebec Region, is to promote the creation of and provide support for communities of practice and learning in the federal Public Service by offering a range of services adapted to the needs of departmental clients. Concurrently, other areas within the School are involved in initiatives related to online learning, although the initiatives are not integrated and coordinated with one another.

The adoption of the new Integrated Learner Management System (ILMS) in 2008-2009 brought about a major change in the School's approach to online collaborative tools. As all users now have to use the new ILMS platform, the pilot project under which funding was provided for the Centre came to an end. Under these circumstances, and given that the original scope of and need for the evaluation had changed dramatically, it was decided to carry out a needs evaluation rather than a typical program evaluation.

The evaluation was carried out over 15 weeks between November 2009 and March 2010. Data from several different and complementary sources were compiled and analysed. Sources included official Government of Canada and School reports, administrative records from the Centre and the Learning Technologies Innovations group, a survey of Tomoye users and interviews with managers and employees with a role in the Online Collaborative Technology Program.

Findings

Relevance of the Program

The Online Collaborative Technology Program, as examined in this evaluation, is in keeping with the priorities of the Government of Canada. In 2006-2007, the government recognized the importance of adopting modern electronic platforms for online learning, networking and sharing of best practices. The School is meeting those priorities by developing initiatives related to collaborative learning, including the Centre and its portal. The Centre was created in response to a request from a departmental client and went on to develop a range of services to meet the growing need for collaborative learning. That need remained steady over the years and persists to this day, as witnessed by the growing number of communities of practice and the online activities of those communities.

Needs Assessment

Statistics on the use of the virtual communities of practice portal and data from the survey show which Tomoye features are used the most and the least. The first observation to emerge from the study is that the Tomoye platform is not used very much. This may be attributable to the fact that the use of collaborative tools in the government workplace is only just beginning. Those who use collaborative tools do so primarily to post knowledge objects (e.g. documents, links, best practices) or, to a lesser degree, participate in forums, read and download documents and information posted by members, collaborate with other members of the community of practice, and take part in learning activities. According to some survey respondents, the limitations of the portal are: lack of user friendliness; access and navigation problems; networking, because many people prefer forms other than collaborative tools; and search function.

The stated needs indicate that users would like the School's new platform to offer more activities that integrate new technologies, such as interactive chat, blogs, microblogs, wiki, synchronous tools, advanced search and automatic notification. Users would like a platform that is more user-friendly, bilingual and accessible outside the office. In addition to wanting new features, the majority of respondents indicated that they would like the new platform to allow them to go beyond the confines of virtual communities of practice and link with other public servants. Specifically, they would like to be linked with colleagues in other government departments and agencies that do the same type of work, expanding their network of contacts within their department and with public servants across the country. Most would like to continue receiving the support services and training that they currently receive regarding communities of practice and how to use the platform. The respondents also indicated that they need more time to participate actively in the virtual community of practice.

Options for Collaborative Learning

It should be noted that analysis of the collaborative learning options available to the School entails a great deal more than simply picking a technological platform and accompanying features. Choosing a platform requires broader consideration of many factors, starting with the School's vision of and approach to online learning. The process must also include the development of a governance model, a business model and the range of services that could be offered using the new collaborative learning platform.

Recommendations

These findings give rise to recommendations that will improve the School's online learning strategy and guide the School as it explores platform and feature options.

Recommendation 1: Consolidate the various collaborative learning initiatives under one organizational lead.

Recommendation 2: Define the School's orientation to collaborative learning.

Recommendation 3: Adopt a governance and business model for the collaborative learning tool.

Recommendation 4: Determine the relevance and mandate, if appropriate, of an experts group in collaborative learning to help course designers and users of the new technological platform.

Recommendation 5: Choose a flexible system with sufficient functionalities to respond to present and future client needs.

1. Introduction

The Online Collaborative Technology Program [footnote 1] has been in place at the School for several years. This program is the embodiment of the growing integration of new technologies into training delivery for Government of Canada employees. The newly created communities of practice portal is a good example, as the communities use collaborative learning tools in their online activities. Tools like these are increasingly popular in training programs, and the School has decided to standardize use of the technologies by adopting a single platform. The ILMS went into service in April 2010.

"It is a software application that transforms how the School interacts with learners and clients. It also allows us to better administer, manage and report on learning across the Government of Canada [...]. The I-LMS features innovative tools, electronic learning material and virtual places where people can meet, share and collaborate. For the learner, it also features technologies that allow learners to manage their own learning and performance needs - anywhere and anytime." [footnote 2]

Until recently, ILMS work has focused on the technical aspects of the platform in order to incorporate the content of the School's various systems and software, including Tomoye, the groupware used by the communities of practice portal. However, in January 2010, the School undertook a re-evaluation of its approaches to online learning in order to discuss issues related to the content to be integrated into the new platform.

This is the backdrop against which the evaluation of online collaboration needs was conducted. Senior management at the School wanted to understand the collaborative learning needs of members of communities of practice. This report presents the results of an evaluation of online collaborative learning needs carried out between November 2009 and February 2010.

The report is divided into several sections. The introduction presents the Online Collaborative Technology Program and various aspects of the evaluation methodology, such as the issues on which the approach was based, the sources used and the limits of those sources, and general considerations to be taken into account when reading the report. The second part presents the results of the evaluation, split into two main themes: the relevance of the program in relation to the priorities of the Government of Canada and the School, and the needs of the members of communities of practice in terms of the features they use and those they would like. The report ends with a general conclusion and appendices containing the survey and interview questionnaires.

1.1 Background

Communities of practice [footnote 3] have existed in the Quebec Region since the mid-1990s. In 2004, the School helped to set up the first virtual communities of practice. The following year, at the request of a client department, the Quebec Region adopted the Tomoye groupware to create a blended learning program which proposed, as a main strategy, the development of a virtual community of practice. The project's success led to requests from other departments, and the idea of founding a centre of expertise in communities of practice gradually gained momentum. In 2006, the Individual Learning Branch and the Regional Operations and Departmental Client Relations Division within the Registrar Branch of the School signed a Memorandum of Understanding, leading to the implementation of a three-year pilot project. The Centre was thus born in the Quebec Region.

The mandate of the Centre is to foster the creation and development of communities of practice and learning within the Public Service of Canada by providing a range of services to departmental clients. The Centre's main goal is to adopt innovative learning and knowledge transfer methods using various technological tools. More specifically, the Centre's role is to offer its expertise in creating virtual communities of practice and supporting them throughout the various phases of their existence. The Centre also supports the introduction and use of appropriate online collaborative tools, specifically through presentations on social networking methods and approaches. This expertise extends to approaches to learning and knowledge exchange and various technological tools.

Originally, this project was to pay for itself after three years through the recovery of its operating costs. The Centre received 100 percent funding the first year and 50 percent funding the second. In year three, operating revenue was expected to cover all the Centre's costs. However, the Centre did not become self-financing; it did not generate enough revenue to cover all its operating costs, including salaries, office space, research and development, and activities related to services. [footnote 4]

At the same time the Centre was being developed, the School's Individual Learning Branch launched other independent online learning initiatives. One of those initiatives was the Centre of Expertise in Blended Learning, [footnote 5] whose specific role was to guide and train program managers in adopting a blended learning approach. Another group, Learning Technology Innovations, [footnote 6] was created to research online learning and determine collaborative learning governance and policy. In addition, the Web 2.0 working group [footnote 7] was established at the beginning of the 2008-2009 fiscal year to develop a strategy for the School's use of information and communication technologies. The working group submitted its report in February 2009.

Other initiatives related to collaborative learning are being implemented by the School's Organizational Leadership and Innovation Branch. Some leadership training programs use Drupal or Joomla collaboration software, which is perceived to be better suited to their needs, instead of Tomoye, the groupware used by the Centre. In addition, the best practices research group studies collaborative learning solutions and works with other government-wide projects and initiatives, including GCpedia and GCConnex. [footnote 8] There is also the School's Wiki, which allows School employees to collaborate for the development of documents related to their activities and projects.

According to an inventory conducted in August 2008 by the Web 2.0 working group, a total of 41 initiatives used collaborative tools at the School and there were plans to start 27 more. [footnote 9]

At the start of the 2008-2009 fiscal year, the School decided to adopt a single platform for learning management (the ILMS) to replace all the systems in use. The decision included Tomoye and other software used for learning and social networking. A plan for moving the data in those systems to the ILMS was put in place. In short, the decision resulted in the end of the Centre in its current form.

The School also seized the opportunity to open discussion of the future of learning in the context of new technologies. In January 2010, a committee was established to develop the School's vision and role regarding online learning, including collaborative learning. The committee's mandate is to study the future of learning, to manage content, and to develop the School's business plan.

Several groups with the School are working on and re-examining collaborative learning and social networking. They have similar goals and work independently within their own branch.

1.2 Scope of Evaluation

The initial objective of this evaluation was to examine the results of the School's Online Collaborative Technologies Program. However, following the School's system migration to the ILMS, it became clear that it was no longer relevant to evaluate the results of a program which was in a transition phase. The evaluation committee therefore directed the Evaluation Division to evaluate the needs of communities of practice using Tomoye in order to identify the features they used, identify their future needs regarding online collaboration, and look at the School's options in this area. The purpose of this evaluation is therefore to inform strategic decision-making about the choice of a new technological platform and potential feature options.

1.3 Needs Evaluation Questions

The evaluation of needs will answer the following questions, in accordance with the Treasury Board Secretariat's Policy on Evaluation. [footnote 10]

1.3.1 Relevance of the Program

What are the links between the program objectives, the government's priorities and the strategic outcomes of the School?

Regarding the needs that the program was supposed to fulfil: to what extent do these needs continue to exist? Have they evolved since the Centre was set up?

1.3.2 Needs Assessment

What functionalities offered by the Tomoye groupware are used the most by members of Government of Canada online communities of practice?

What are the needs of members of online communities of practice?

What options are available to the School in terms of online collaboration?

What are the financial benefits of each of the options available?

1.4 Data Collection and Methodology

In accordance with the Policy on Evaluation, this needs evaluation used a variety of evidence.

1.4.1 Documentation and Administrative Files

Official documentation was consulted to respond to the questions regarding the relevance of the program. The evaluator scanned the documentation for priorities related to online collaborative technology and networking using those tools. The first sources consulted were official Government of Canada documents and reports, as well as reports from the Clerk of the Privy Council for fiscal years 2006-2007 to 2009-2010. The evaluator also consulted the School's reports on plans and priorities and departmental performance reports for the same period.

In addition, the Centre gave the evaluator full access to its records, including articles, reports and briefing documents on the Centre's activities; statistics on use of the Tomoye platform; surveys; needs studies; a summary of memoranda of agreement; and various documents pertaining to the migration to the ILMS.

The School's Learning Technology Innovations group shared its records, its documents, and reports pertaining to the consultations it undertook for the purpose of developing governance and business models.

The evaluator further reviewed numerous articles and studies on virtual communities of practice and on the evaluation of those communities.

1.4.2 Survey of the Needs of Members of Communities of Practice

A survey was sent to members of communities of practice using Tomoye. The purpose of the questionnaire was to determine the members' overall satisfaction with the virtual community of practice, the features currently used the most, and their needs in the context of implementation of a new technological platform. The questionnaire is reproduced in Appendix 1.

1.4.3 Interviews

Interviews were also conducted with managers and staff at the School who are directly or indirectly involved with the Online Collaborative Technology Program. The evaluator conducted semi-structured interviews using a series of open-ended questions. The questions are listed in Appendix 2.

1.5 Methodological Considerations

This section discusses the methodological considerations surrounding the needs evaluation and the sources used.

Mandate

This was not a "traditional" program evaluation as defined in the Treasury Board's Policy on Evaluation, which states that the purpose of an evaluation is to examine the relevance and performance (effectiveness, efficiency and economy) of a program. A different approach was taken for this evaluation. The evaluator began by preparing an evaluation framework that included questions on relevance and performance; however, before submitting it to the School's Evaluation Committee, program managers asked that the evaluation be deferred because the program was ending, as explained in a previous section. However, in light of ILMS's imminent implementation, the Evaluation Committee felt it would be appropriate to carry out a neutral needs analysis to identify suggestions that would be useful in choosing a new technological platform. This new direction for the evaluation was given a short deadline. The decision was made on November 18, 2009, and the report was to be completed by early March 2010.

Based on that decision, the evaluator developed a new evaluation framework that comprised two sets of questions: one on the relevance of the program in accordance with the Policy on Evaluation, and one on needs, an area not covered by the Policy. Given the new direction identified by the Evaluation Committee, there is, notably, no section on the efficiency of the program. Normally, evaluators look at a program from every angle, including economy. In this case, those questions were dropped in favour of the study of user needs, since this is what was deemed appropriate to meet the needs of the School in this instance.

Sources

The evaluator was able to find all the information required in the documentation and records consulted. The statistics from the Tomoye system, however, turned out to be limited and very general. There was no way to extract data from the system on use by fiscal year or any other period. Only snapshots on specific dates could be obtained. The only way to track online activities was to track the data manually, and even that tedious process had to be authorized by the communities of practice.

Some people were interviewed for the purpose of gathering information about the program itself, while others were asked to present the options available to the School. The interviews were conducted in a difficult context because of the transition period. There are many uncertainties and grey areas when it comes to the new platform, particularly in terms of the available options. The ILMS collaboration tools have less functionality than Tomoye. Moreover, the first option that takes into account the ILMS, Saba Social, is not yet available for beta testing or purchase. All interviewees know that the School has begun a re-examination of its vision and future position on collaborative learning and that the options have to go through the ILMS. It was therefore hard for the interviewees to suggest specific options outside that context.

The evaluation strategy relied on the survey of Tomoye users' overall satisfaction with virtual communities of practice, [footnote 11] but especially the most-used features and their needs regarding the new platform. The survey was sent to the members on December 22, 2009, based on the list used by the Centre when the migration from Tomoye to the ILMS was announced. 1,541 e-mails were sent, and 1,517 people actually received the survey. Of those, 202 completed and returned the questionnaire, which translates to a response rate of 13.3 percent. This is similar to the response rate in other surveys of virtual communities of practice. Based on generally accepted statistical practices, this response rate does not permit extrapolation of the results to the entire membership of the communities of practice. It should be noted, however, that the answers to the survey questions are quite similar to those found in other sources, such as statistics on the use of Tomoye and surveys and needs studies carried out by the Centre.

There are several factors that may account for the low response to the survey. The timing of the survey was not optimal given that the communities of practice had just learned about the change in platform from Tomoye to the ILMS. The community of practice coordinators were working with staff at the Centre to transfer data to the new platform, knowing that the new platform had fewer features than its predecessor. The activities of the virtual communities of practice were therefore at their lowest point, making people less inclined to take part in a survey. Further, the participation rate in the online activities of communities of practice was rather low, which also accounts for the lack of interest in completing the survey. From a different standpoint, collaborative learning is a relatively new practice in the Public Service of Canada; only a small number of employees use it in their day-to-day activities. This observation is borne out by the fact that, on average, 12.1 percent of the respondents answered "Not Applicable" to all the questions. Those respondents apparently did not use Tomoye enough to have a good understanding of the system.

Despite these limits, the data compiled and analysed in the course of this needs evaluation provide sufficient information to produce credible answers to the evaluation questions.

2. Evaluation Results

2.1 Relevance

What are the links between the program objectives, the government's priorities, and the strategic outcomes of the School?

Priorities of the Government of Canada and the Clerk of the Privy Council

Two main answers to this question are evident in official documents and reports: renewal of the Public Service and use of technologies in the workplace. These two themes are intrinsically linked because the aim of public service renewal is to attract a new generation of employees more familiar with new information technologies. The objective is therefore to develop technological platforms in order to attract and retain new generations of employees.

The Prime Minister of Canada took the first step in this area in November 2006 by creating the Advisory Committee on the Public Service. The Committee's mandate includes identifying the challenges facing the federal Public Service and advising the Prime Minister and the Clerk of the Privy Council. Two of the six challenges identified by the Prime Minister are relevant to this evaluation:

• the Public Service is undergoing significant demographic change, as the aging workforce is more prevalent in the Public Service than in other sectors; and
• like all employers, the Public Service is experiencing the changing nature of work as technology fundamentally alters how it conducts its business. [footnote 12]

Public service renewal is front and centre in the Clerk of the Privy Council's actions and messages. It also emphasizes the importance of sharing best practices. The following strategy is set out in the Fourteenth Annual Report of the Clerk of the Privy Council:

"Our short term action plan is to: [...] develop modern electronic platforms for on-line learning, networking and sharing best practices across the public service." [footnote 13]

In a January 2010 interview, the new Clerk discussed those challenges and made a clear link between the two themes.

"And, renewal is about more than renewing the workforce. We must also renew the workplace, which is an area that I intend to focus on more in my tenure as Clerk. This involves renewing our back office, fostering innovation and embracing collaborative technology such as Web 2.0 in our workplaces. If we are going to nurture the next generation of leaders, we must also look at our work environment. We must ask ourselves whether the workplace and workspaces we have to offer - some of which are antiquated and out of date - have what it takes not only to interest recent grads but also to ensure the Government of Canada, as an employer, remains competitive." [footnote 14]

Plans and Priorities of the School

The School quickly endorsed these priorities and adopted a specific plan for implementing them in the 2007-2008 fiscal year. [footnote 15] The School supports the idea that new information technologies change the way public servants work and learn. They broaden access to knowledge, encourage continuous learning and meet the needs and expectations of new generations of public service employees. The strategy consists of integrating technology into existing courses and programs and developing Web products and blended learning.

For the 2008-2009 fiscal year, the School adopted the key program sub-activity "Online Learning: Public Servants have Efficient Access to Online Learning Tools." To carry out this program activity, the School will

• support e-learning and blended learning products as well as implement and deliver new products in these formats;
• provide integrated learning solutions by implementing and maintaining departmental learning portals;
• undertake capacity planning and upgrades to the Campusdirect infrastructure;
• increase the capacity and features of the Campusdirect environment in order to support School operations;
• research, test and adopt new technologies in learning and research programs to promote greater access to and reach of School programs, and to enable greater interaction among participants;
• support the migration of existing classroom courses to blended learning solutions; and
• replace the Campusdirect platform with the new ILMS platform. [footnote 16]

A program activity was also adopted in the areas of leadership and exchange of best practices: "Leadership Community Building: Learning Opportunities Enhance Leaders' Contextual Knowledge". To carry out this activity, the key initiative in the leadership community development component consists of using technology to build and sustain a community of practices and networks for information sharing and the dissemination of best practices. [footnote 17]

The first official use of the name Online Collaborative Technology Program was in the 2009 2010 Report on Plans and Priorities. The report stated:

"Clients expect that the latest innovations and collaborative technologies will be available to promote interaction, learning, dialogue and knowledge exchange. These new technologies would also help to attract and retain the next generation of public servants who are accustomed to working and learning in such an environment.

The School is expanding its use of technologies such as Web conferencing, social networking, podcasting, virtual classroom and other interactive initiatives. This requires the continued development of expertise and service provision capacity in elearning. It is expected that 3,000 to 4,000 participants will use the School's online collaborative tools and vehicles, such as the Community of Practice Portal, in 2009 2010." [footnote 18]

Starting in the 2009-2010 fiscal year, the School will work specifically on the following activities:

• take advantage of the use of technologies, including blended solutions, and expand the use of self-evaluation and self-instruction tools to facilitate learning; and
• research, test and adopt relevant and innovative collaborative technologies to promote greater access to School programs and ensure future functionality with ILMS.

To summarize, the Online Collaborative Technology Program is in line with the priorities of both the Government of Canada and the School.

Regarding the needs that the program was supposed to fulfil: to what extent do they continue to exist? Have they evolved since the Centre was set up?

As stated in the background section above, the Online Collaborative Technology Program was implemented in 2005. The number of communities of practice subsequently increased every year, as did the number of members, as illustrated in Table 2.1 below. Of note, 2008 was a particularly good year for the Centre as 13 new virtual communities were created, which represents 46.4 percent of the total number of communities. In 2009, however, following the announcement about the migration to ILMS and the moratorium on the creation of new communities, there were only three additions. Most virtual communities of practice were willing to move their content to the ILMS, but some, including one of the largest communities in terms of number of members, decided to move forward with a different service provider, which would explain the difference of almost 800 between the total number of members in this table based on the 2009 statistics and the 1,541 members registered in December 2009.

Table 2.1: Number of Communities and Members, 2005-2009

These virtual communities of practice are divided into four different types depending on their objectives. [footnote 19] A community of practitioners is a group of people in a given field who work together to develop, exchange and innovate in their area of expertise. A community of learners is a group of people who come together to produce and exchange knowledge of a specific subject or to participate in a training activity that uses a blended learning approach. A community of interest is a group of people interested in discussing and exchanging information on a specific subject they have in common. A strategic community is formed within an organization and allows its members to pool their knowledge in order to meet strategic and organizational objectives.

A number of findings emerge from Table 2.2, which shows the number of communities by type and the number of communities from within the School. Communities of practitioners make up almost half the total number of virtual communities of practice and by and large come from other government departments and agencies. Communities of learners make up a third of the total number of communities and are in most cases used to support School training programs and courses. Communities of interest and strategic communities account for a relatively small proportion of the total virtual communities of practice portal.

Table 2.2: Four Categories of Virtual Communities of Practice

A series of survey questions dealt with the level of satisfaction among members of communities of practice using Tomoye. Table 2.3 provides details of the responses to the four questions asked. Half of the respondents said they were happy with their user experience as members of their virtual community of practice; however, a third said they were not. Further, the majority of respondents said that experienced and inexperienced members alike are able to share their knowledge and expertise within their virtual community. Only a third of the respondents said they agreed or agreed strongly with the statement that they develop standards, methods, new ideas and best practices. However, 40.6 percent neither agreed nor disagreed with that statement. According to 40.2 percent of the respondents, a virtual community is useful for networking. To summarize, the respondents appear to be relatively satisfied with their community and the key activities of sharing knowledge and developing ideas and networking.

Table 2.3: Satisfaction Among Users of Virtual Communities of Practice

Conclusion Regarding Program Relevance and Continued Need

In the 2006-2007 fiscal year, the Advisory Committee on the Public Service established by the Government of Canada and the Clerk of the Privy Council announced plans to implement modern electronic platforms for online learning, networking and sharing of best practices. The aim was to transfer knowledge from experienced public officials to new employees. The School responded to those priorities by developing a number of initiatives related to collaborative learning, including the communities of practice portal developed by the Centre. The Centre has gradually developed a range of services tailored to the needs its client departments, and the number of clients has increased rapidly from 2005 to 2009. The need for this program has remained steady throughout the years and still exists today.

2.2 Needs Assessment

What functionalities offered by the Tomoye groupware are used the most by members of Government of Canada online communities of practice?

One of the Centre's unique features is that it provides services to support and assist virtual communities of practice at different stages of their development. The Centre meets the needs expressed by communities of practice by studying clients' needs and suggesting an array of services in response. Virtual communities of practice choose the services that are in line with their objectives and budget, and their choice is then set out in a memorandum of understanding (MOU). Table 2.4 is a compilation of the needs covered by the 36 agreements the Centre signed with communities of practice. The most popular services are those referred to in at least half of the MOUs. Portal access, technical support, design, development and implementation, and training regarding communities of practice are the services most often requested by communities. At the other end of the spectrum, the least popular services - those referred to in fewer than a quarter of the agreements - are online facilitation and virtual classrooms.

Table 2.4: Services Offered and Services Used, Based on the 36 MOUs

Table 2.5 presents statistics on use of the portal by members of communities of practice from 2005 to May 7, 2009, in order to identify the most popular activities. This overview of how much the portal has been used since it was created was produced before coordinators of communities of practice were informed of the migration to the new platform. Subsequent to that announcement, activity on the site decreased, and the statistics on use have not changed a great deal since. This table gives an indication of the number of visits, responses in forums, and knowledge objects, which make up the key activities on the portal. This information is presented for each of the four types of communities and total activities since the portal opened. The data do not include the national community of learners, the only one which is open to all users and of which everyone automatically becomes a member when they subscribe to the communities of practice portal. Including that community would have considerably skewed the data in the table by inflating the number of members, which would have been doubled, and would have lowered the averages for portal activities.

According to the data in the table, two types of communities are more active than average: the communities of practitioners and strategic communities. On average, members of these communities visit the portal more, participate more in forums and post more knowledge objects than other members. There are several types of knowledge objects: learning activities, announcements, blogs, business cards, documents, events, multimedia files, books, best practices and Web sites. The table also shows that all members had connected to the portal on average almost 69 times since it was created. It should be noted, however, that some virtual communities of practice have been in existence since 2005, while others were created in early 2009. As well, posting knowledge objects is the most popular activity on the portal, far exceeding participation in discussion forums. On average, each member uploaded 3.72 items to the portal but participated in a forum only once.

Table 2.5: Statistics on Portal Use

The survey questionnaire sent to community of practice members was designed to determine which features are used most frequently in Tomoye. Table 2.6 presents that information as well as the frequency of use by members. For the purposes of this report, the choices for responses about frequency were divided into four main categories instead of the eight in the survey questionnaire. The first observation to be drawn from the data is that the respondents do not connect to the community of practice portal very often, as illustrated by the fact that 44 respondents (22 percent) open a session in the portal one or more times a week. By comparison, 82 respondents (41 percent) connect once a year and 21 (10.5 percent) never connect. In addition, approximately 36 percent of respondents read or download documents and information from the site only several times a year. A large majority never post documents, links to Web sites, or best practices and do not participate in the forum on their community's site. The same comment applies to creating and modifying business cards and finding an expert. Just under half of the respondents, however, have never collaborated with other members of their community, and the majority have never taken part in a training activity organized by the community of practice.

A survey of community of practice facilitators conducted by the Centre of Expertise [footnote 22] showed that 83.4 percent of facilitators acknowledge that members openly share information and resources. 75 percent of facilitators said they are disappointed with the low participation by members of the virtual community of practice.

These data need to be qualified, however. Several respondents accessed Tomoye as part of a training program at the School. They did not use the community of practice again once their program was done. In addition, several School employees registered in the portal, taking advantage of the fact that access was free, but were not part of a specific community of practice, thereby limiting their use of the features of the Tomoye platform.

It should also be noted that online collaboration in a government setting is relatively new. Consequently, there should be no expectation, in the short term, of extensive use of such tools in the workplace because of current work habits. This is a workplace culture that still needs to be developed.

The second observation to be drawn from Table 2.6 is that the data corroborate to some extent the general statistics presented above and also provide some clarification. The features used most often (by 30 percent or more of respondents) are opening a session in the portal, and reading and downloading documents and information posted by members. The features used occasionally (by between 10 percent and 20 percent of respondents) are adding documents, links or best practices, initiating and/or participating in a discussion, collaborating with other members of the community of practice, and taking part in a learning activity. These responses contradict somewhat the general statistics presented above, since the respondents said they participate in forums more often than they upload knowledge objects to the portal. Finally, the features that are rarely used (by fewer than 10 percent of respondents) are creating and modifying a person profile and finding an expert.

Table 2.6: Use of Tomoye Features by Community of Practice Member

Comments were included by 109 respondents in their response to the open-ended question "What are the most important aspects of your community of practice?" Of those, 18 (16.5 percent) said they had not used the portal enough to be able to comment. For the remainder (91 respondents or 83.5 percent), the most important aspects are sharing of knowledge and information, exchange of best practices and networking, both in the community of practice and in courses. Some of the comments received are:

"Sharing teaching materials. Being able to send a message to all the members when necessary/pertinent."

"Sharing of knowledge and best practices and enabling people to connect throughout Canada."

"Interdepartmental access to files/documents in a centralized location."

"The ability to search for and find resources easily. Access to a network of colleagues with varying levels of expertise."

"Teaching in departments can be very isolating, and since English teachers don't have a curriculum, the materials on Tomoye are extremely valuable. Otherwise, teachers would constantly be reinventing the wheel making up the same things."

69 respondents included comments in their response to the question "What aspects of your community of practice are least important?" Of those, 32 respondents (46.5 percent) identified the following aspects of the portal: problems with user friendliness, access and navigation (17 comments or 24.6 percent) topped the list, followed by networking (10 comments or 14.5 percent) and information searches (5 comments or 7.2 percent). 11 respondents (or 15.9 percent) said that the entire site is important, while 26 respondents (37.7 percent) said they had not used the site enough to be able to make any specific comments. Some of the comments received are:

"Current platform is not user friendly. It is messy, too many useless functions that causes confusion. The new platform should highlight the most used functions, and hide the least used functions in some sort of "preference" or "settings" section. Not everything has to be out in the open..."

"The pages are difficult and confusing to navigate. Emails are more effective."

"Personally, I do not find the interface easy to use. The navigation is not intuitive and even with some practice can be confusing. The software is not well integrated into desktop (e.g. as far as I know it does not interface with Outlook other than to send emails to it) so it essentially constitutes a new thing that I have to learn rather than integrating with what I already do."

What are the needs of members of online communities of practice?

A series of survey questions were asked in order to identify the online collaborative learning needs of members of communities of practice. Almost 60 percent of respondents said they would like the new collaboration platform to contain the same type of activities as Tomoye. Meanwhile, 53 percent said they would like the new platform to offer more activities, as illustrated in Table 2.7.

Table 2.7: Activities in Tomoye and New Activities

Asked about the new activities they would like to have, 40 of the 83 respondents indicated their expectations. Of those, 18 respondents (45 percent ) suggested integrating new technologies into the platform, such as interactive chat, blogs, microblogs, wiki, synchronous tools, advanced search and automatic notification.

"I would like to have access to a tool that maximizes the use of embedded videos, work in wiki, blogs and a Web 2.0 business platform. High-security access (https, if necessary) for our RCMP, National Defence and Revenue Canada clients and open to Apple technology. Bilingual Accessible GOL technology, which fosters openness and flexibility in design and administration."

"Better integration across activities in the CoP space (there is too-much Gopher-style drilling down and back). Better integration with my desktop or Outlook (including calendars). I am not sure what else it could offer. Document revision control might be nice. Video/data conferencing. Idea sharing with comments, votes and ranking? Not sure what is available."

Nine respondents (22.5 percent) would like the new platform to be more user-friendly and easier to use. They suggested reviewing the interface design and presenting the information in a better way, in other words making the platform more appealing to use than e-mail.

Eight respondents (20 percent) said they had not used Tomoye often enough to be able to make suggestions. Three respondents (7.5 percent) would like members to become more involved so that virtual communities of practice can operate more effectively. Finally, two respondents (5 percent) made comments unrelated to Tomoye activities.

It is important for a large majority of respondents to have a bilingual platform and bilingual documentation (116 respondents or 68.2 percent) and access from outside the office (148 respondents or 84.1 percent), as indicated in Table 2.8.

Table 2.8: Bilingualism and Access From Outside the Office

Table 2.9 shows the type of collaboration the respondents would like to have in the new platform. A large majority of respondents (151 or 85.3 percent) would like to use the new platform to contact colleagues in other government departments and agencies. A sizeable majority would like to expand their network of contacts within their own department (123 or 69.5 percent) and be able to contact other public servants across the country (112 or 63.6 percent).

These data indicate that the survey respondents want to move beyond the existing frameworks of the virtual community of practice and contact other colleagues, whether or not they do the same work.

Table 2.9: Collaboration

The vast majority of respondents want to continue receiving the support and training services they currently receive from the Centre, as indicated by the data in Table 2.10. More than 80 percent of the respondents would like training regarding the platform, access to support services for their community of practice and online help. In addition, 75 percent of the respondents would like training regarding collaborative learning and telephone access to technical support.

Table 2.10. Services and Training

Three questions dealt with the work environment, as indicated by Table 2.11. A large majority (137 respondents or 74.5 percent) would like to have more time to participate actively in the virtual community of practice. However, the respondents do not seem to require more support from their manager (56 respondents or 35.7 percent) or formal recognition of their participation (60 respondents or 36.4 percent).

Table 2.11: Work Environment

In all, 27 of 202 respondents made general comments in response to a question regarding formal recognition at the end of the survey. Of those, 17 (63 percent) suggested ways of improving the overall experience with virtual communities of practice. Six (22.2 percent) suggested making major improvements to the portal in order to make the site more user-friendly, changing the interface and resolving technical problems.

"I find this portal very non-user friendly, perhaps due to lack of knowledge and/or understanding/support on how to use the system."

"I hope that the interface of the new platform will be easier to use and navigate than Tomoye."

For other respondents (6 or 22.2 percent), lack of time during office hours is an obstacle to use of the portal.

"I have no problem with management, but our work prevents us from taking more time to communicate with other people within and/or outside our department/agency."

Five respondents (18.5 percent) would like to see this type of tool integrated more into their day-to-day work.

"It [the communities of practice portal] should be an integral part of the workplace - we need to look at the way we work and make better use of this type of tool so that there is no pile-on of work."

Finally, 10 respondents (37 percent) made general comments that went beyond the boundaries of this survey or said that they had not used the portal enough to be able to make comments.

Needs other than those of members of communities of practice in Tomoye

A survey of users of the government networking site GCConnex was conducted in October 2009. Two questions in particular touched on the themes addressed in this evaluation of needs. The first aimed to assess the respondents' expectations of the social networking site. In all, 258 comments described users' expectations of the site. The comments can be divided into four general categories: social networking drew 155 comments (60.1 percent), sharing of knowledge drew 77 (29.8 percent) and employment opportunity drew 14 (5.42 percent); 12 comments (4.7 percent) did not indicate any specific expectation of this type of tool. Social networking can be divided into several responses. Maintaining and expanding the network led the way with 59 comments (38.1 percent), followed by the creation of communities of practice between colleagues in the same field in other departments with 42 comments (27.1 percent), social networking in general with 31 comments (20.0 percent) and intergovernmental cooperation with 23 comments (14.8 percent).

Last fall, the Learning Technology Innovations team met with several groups at the School in order to identify their collaborative learning needs in the context of migration to the ILMS. These data were made available to the evaluator. The vast majority of the 15 groups met would like the best tool possible to support development of the group or community, but did not describe in detail the features they would like. Two groups provided details of the features. One wanted a platform that would make it possible to adapt the interface to its needs and incorporate such features as videos, photos and microblogs. The other would like to see mobile technology incorporated into the platform. To summarize, these groups would like a flexible platform that incorporates new technologies like other tools and social networking sites.

"I am looking for a platform adapted to the needs of my course and clients in their late 20s/early 30s (EX-4, -3). The participants are very familiar with new social networking tools. For purposes of the course, the networking has to be done and a community formed in the space of 12 months. I am looking for a Web site that allows me to combine the various features with other collaborative tools: photos of group meetings, documents, videos, chat, wiki, forums, etc. Every cohort has its own space where participants post documents. The goal is to promote dialogue between participants and between cohorts. There is also a common area that contains information about the program, the schedule of events and training phases. This is a quick and effective way of updating information." [footnote 23]

Conclusion Regarding the Needs Study

The survey carried out as part of this needs study has provided insights of the features used by community of practice members and their needs for the future collaborative learning platform. The survey data have been corroborated by other sources of information.

Based on the analysed data, the use of collaborative learning tools can be summarized as follows.

The features used frequently and occasionally are:

• posting knowledge objects (documents, links, best practices, etc.);
• participating in forums;
• reading and downloading documents and information posted by members;
• collaborating with other members of the community of practice; and
• taking part in learning activities.

The less valued features of the portal are:

• user-friendliness, access and navigation;
• networking, because many people prefer forms other than collaboration tools; and
• information search.

The needs identified in this evaluation report indicate that users would like the new platform to

• offer more activities by incorporating new technologies, such as interactive chat, blogs, microblogs, wiki, synchronous tools, advanced search and automatic notification;
• be more user-friendly and easy to use;
• be bilingual (interface and documentation); and
• be accessible from outside the office.

At the same time, the majority would like to

• step out of the restrictive frameworks of the virtual community of practice and contact other public servants;
• link colleagues doing the same job in all government departments and agencies;
• expand their network of contacts within their own department;
• continue receiving the current level of support and training services related to communities of practice and use of the platform; and
• have more time to participate actively in the virtual community of practice.

What options are available to the School in terms of online collaboration?

This section presents a summary of comments received through the interviews from employees of the School who work in the field of collaborative learning. It highlights the key issues that come into play when choosing a technological platform for collaborative learning. The choice is more complex than it seems at first glance because it entails a great deal more than simply comparing the features of different platforms. The final choice of a platform must be preceded by an examination of such factors as the direction of online learning, governance, the business model and the range of services to be included.

It should be noted that these factors are already being examined by the working group established in January 2010 to develop the School's vision and role in the area of online learning, including collaborative learning.

Online Learning

Many educational institutions and departmental training centres have begun using new learning technologies to deliver their courses and programs. The School, too, is interested in moving in this direction and has taken the first steps in charting a course. This process must be completed before technological tools can be chosen, as indicated by the following interview excerpt:

"The main challenge is determining where we want to go in terms of collaborative learning before we choose a technological platform. The choice is actually based on what we want to be as a training body; it is not just a matter or which software offers which features." [footnote 24]

As one respondent said, [footnote 25] successful migration to collaborative learning absolutely requires official and active support from senior management of the School in order to promote the use of the platform and encourage people to use new learning tools. It takes a great deal of time and effort to migrate to these new tools, as well as skilled people who are familiar with the technology.

Governance and the Business Model

The second task is to develop the governance model in order to establish the role, tasks and responsibilities of the individuals and teams involved in collaborative learning.

"The biggest challenge is establishing governance for this type of collaborative project, especially since there are many teams working on the same subject. This causes a loss of innovation potential. We have to make sure everyone is pulling in the same direction." [footnote 26]

A clear business model must also be developed for collaborative learning. The model has to specify the internal needs of the School and those of other departments.

In addition, the target audience for the new platform will have to be identified: communities of practice only, based on the current model, or all federal government employees, based on the GCpedia and GCConnex models.

Range of Services to be Provided

The School should also question the relevance of creating a group of experts in collaborative learning in order to manage collaboration tools. This includes the development of policies and standards for use of the technological platform, as well as the services and support that will have to be provided to learning specialists with the School and users of collaborative learning tools.

Collaborative Learning Technological Platform

The new platform will have to comply with Government of Canada standards and policies, in particular those applicable to information management, security, accessibility, official languages and environment. [footnote 27]

It should also be made clear that once the School has chosen a platform, the Information Management and Technology team will undertake a comprehensive study of the options based on stated collaborative learning needs. The study should compare the features of the various platforms and their compatibility with what is currently being used by the School.

"Once the needs have been clearly identified, the document "User Requirements" can be completed, and then the Information Management and Technology (IMT) group can study the various options and make suggestions to the School's management committee based on several scenarios. Following the needs study, IMT will make detailed recommendations for each available product: advantages and disadvantages of each platform, compatibility with technology used by the School, available staff, etc. IMT recommends that only one platform be used. However, the School may choose to use two or more, which would mean higher purchase and maintenance costs in terms of hours of work, number of employees, etc. The data storage group will also prepare a report, and after all the reports are analysed, a decision will be made." [footnote 28]

In view of the migration to ILMS and the School's desire to use a single integrated platform, the software options are limited. The first step will be to determine whether the collaborative tools included in ILMS (Saba Collaboration) are able to meet the needs of the School given its future orientation in this area. Saba will soon be releasing an advanced networking version, Saba Social. However, that software is still at the beta testing phase and is therefore not available on the market at this time. A team from the School will test Saba Social as soon as it becomes available.

The choice of a platform must also take into account the fact that users' needs change over time. Some users currently need only basic features, while others want to be able to use state of-the-art advanced features now. Public servants currently operate on the basis of a private knowledge model where each person has full control of all stages in the development of a document or learning product. This explains in part the limited use of collaborative platforms in the workplace. Adopting a model based on collaboration and sharing of knowledge in the workplace should increase the use of collaborative tools and thus create new needs among users. Even if users at first use only some of the features of the new platform, the platform must be technologically sophisticated enough to meet future needs.

"The choice of platform has to be made carefully, because one of the unique characteristics of social networks is their lack of structure. Some groups will use only one feature, while others will use several, depending on their needs. Those needs can also change over the years." [footnote 29]

Another factor that must be taken into consideration in choosing a new platform is the status and needs of the regions, as indicated by the following interview excerpt.

"For the regions, there are challenges that have to be taken into account in terms of features. Not all public servants live in major cities, so how can they be reached? We have to come up with new ways of delivering training to people, no matter where they live. Does the solution lie in the virtual classroom or online learning? This creates new challenges for facilitators, who have to learn how to use this type of tool." [footnote 30]

What are the financial benefits of each of the options available?

It is difficult to answer this question at this time because the School's main option, Saba Social, is not yet on the market and it is not known how much it will cost. Once the School has determined its approach to collaborative learning, its needs will have to be submitted to the Information Management and Technology group for a full comparative study.

Conclusion Regarding Collaborative Learning Options

Analysis of collaborative learning options entails a great deal more than simply choosing a technological platform. A number of issues have to be considered, as they will guide the decision. The School is currently studying the issues, one of which is its approach to online learning. A governance and business model can then be adopted, along with a range of services that could be offered with the new collaborative learning platform.

General Conclusion

Initially, this evaluation was supposed to examine the relevance and performance of the Online Collaborative Technology Program directed by the Centre, in accordance with the Policy on Evaluation. However, a number of factors led to a change in the evaluation strategy. The decision by the School to adopt a single technological platform, the ILMS, tops the list. That decision resulted in a complete overhaul of the activities and services of the Centre and the dismantling of the Centre, attributable primarily to the adoption of a new platform to host communities of practice. The pilot project that provided funding for the Centre also came to an end. However, the School's Evaluation Committee seized the opportunity to evaluate the collaborative learning needs of members of communities of practice in order to better plan the choice of a platform and the services to be provided instead of conducting a "traditional" program evaluation. At the same time, the School's Management Committee decided to set up a committee to develop the School's vision of and approach to collaborative learning.

The needs evaluation shows that the Online Collaborative Technology Program meets the government's priorities as stated by the Advisory Committee on the Public Service of Canada and developed by the Clerk of the Privy Council in 2006. In short, electronic tools for online learning, networking and sharing of best practices are desired so that the knowledge held by experienced employees is passed on to new employees, who are accustomed to using new technologies. The School met those priorities by adopting a blended learning approach characterized by a growing range of online courses and learning products and support for various projects that use new collaborative technologies. Foremost among them is the Centre, which used Tomoye software and offered a range of services to its members in order to meet the constantly growing needs of departmental clients.

However, this proliferation of projects in the area of online learning required significant general coordination. Several teams within the School were working independently in similar fields, which created confusion as to the roles and responsibilities of the teams, duplication of effort and loss of creativity.

Collaborative learning tools are a relatively new phenomenon in the federal Public Service, which is why the sources consulted during the course of this evaluation report that they are used very little. Use of these tools is a practice that needs to be encouraged and developed in the workplace. To date, the features used most frequently by members of the communities of practice are features related to the posting of knowledge objects such as documents, information and best practices. In the future, users expect the School's new technological platform to be more user-friendly and to offer features based on such leading-edge technologies as interactive chat, blogs, video, microblogs, wiki, advanced search and automatic notification. Many users would like the new platform to go beyond the confines of communities of practice and create forums with colleagues in other departments and agencies and with all government employees. Many would also like to be able to continue receiving services, training and online support like the range of services provided by the Centre.

A series of recommendations has been made to guide discussion of the options available to the School regarding a technological platform. The features of the new platform will certainly be important, but they must not be the only criterion. The School must clarify its position on online learning, develop a governance model and a business model, and determine the range of services to be offered in connection with the new platform.

Recommendations

The findings above give rise to recommendations that will improve the School's online learning strategy and guide the School as it explores platform and feature options.

Recommendation 1: Consolidate the various collaborative learning initiatives under one organizational lead.

Recommendation 2: Define the School's orientation to collaborative learning.

Recommendation 3: Adopt a governance and business model for the collaborative learning tool.

Recommendation 4: Determine the relevance and the mandate, if appropriate, of an experts group in collaborative learning to help course designers and users of the new technological platform.

Recommendation 5: Choose a flexible system with sufficient functionalities to respond to present and future client needs.

Annex 1: Questionnaire sent to community of practice members

Section 1: Demographics

Are you a member of more than one work-related online community of practice on Tomoye?

• Yes
• No

If yes, how many?

Name of your primary online community of practice on Tomoye.

How long (approximately) have you been a member of this community?

What is your role in your primary online community of practice?

• Facilitator/Coordinator
• Member
• Other

If other, please specify:

Which Department or Agency do you work for?

Section 2: Satisfaction

Please rate the following statements using the scale provided.

Section 3: Activities

Please indicate the frequency with which you carry out the activities listed. Rate all that apply.

Section 4: Members' Needs

Considering the move of your online community of practice to a new platform, what would you like as activities and services?

Annex 2: Questionnaires used for interviews

• Introduction: talk about the evaluation's goals and its new direction
  • Questions about the evaluation?
• What is your role in the online collaboration program?
  • Main tasks related to the program
  • How long have you been involved?
    • Brief background
• What are the Canada School of Public Service's options in terms of online collaboration?
  • Functionalities of the various online collaboration programs (Tomoye, Saba Social, Drupal, etc.)
  • Comparisons with other online collaboration initiatives of the Government of Canada
  • Governance
  • Services to be provided
  • Links with the Web 2.0 committee
  • Links with other committees
  • Do you have any documents to share in this regard?
• What are the financial advantages of each available option?
  • Cost to purchase the software?
  • How do we fund the purchase?
  • How do we fund the services?
  • Do you have any documentation in this regard?
• In your opinion, what are the main challenges for the Canada School of Public Service in terms of social collaboration and taking into account the migration to the ILMS?
  • What best practices should we retain regarding Tomoye?

1. In this report, the terms "online collaboration", "online learning" and "collaborative learning" are used interchangeably. (Return to source paragraph)
   
   
2. Canada School of Public Service. Information for: The Integrated Learning Management System (I-LMS) (Return to source paragraph)
   
   
3. A community of practice is defined as a process whereby a group of persons who have a common interest, activity or passion interact regularly to share their knowledge and practices. (Return to source paragraph)
   
   
4. Interviews with the Regional Director of the Quebec Region on February 19, 2010, and the Director General of Departmental Client Relations on January 22, 2010. (Return to source paragraph)
   
   
5. More information about the Centre of Expertise in Blended Learning can be found on the Canada School of Public Service's Wiki, which is available internally only (Return to source paragraph)
   
   
6. The group also has a space on the School's Wiki, which is available internally only. (Return to source paragraph)
   
   
7. The Web 2.0 working group also has a space on the Wiki, which is available internally only. (Return to source paragraph)
   
   
8. GCpedia is a collaborative work tool pilot for federal employees and GCConnex is the Government of Canada social networking pilot. (Return to source paragraph)
   
   
9. Web 2.0 Working Group. New Technologies Project Inventory, August 2008. (Return to source paragraph)
   
   
10. Treasury Board Secretariat of Canada. Policy on Evaluation. 2009. http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024. Last consulted on February 24, 2010. (Return to source paragraph)
    
    
11. Robert M. Verburg and J. H. Erik Andriessen. "The Assessment of Communities of Practice", in Knowledge and Process Management. Volume 13, Number 1, 2006. p. 18. The response rate attained by that team was 14 percent. (Return to source paragraph)
    
    
12. Prime Minister's Advisory Committee on the Public Service - Mandate and biographical notes. https://pm.gc.ca/en. Last consulted on February 16, 2010. (Return to source paragraph)
    
    
13. Kevin G. Lynch, Clerk of the Privy Council and Secretary to the Cabinet. Fourteenth Annual Report to the Prime Minister on the Public Service, for the year ending March 31, 2007, p. 25-26. http://publications.gc.ca/collections/Collection/CP1-1-2007E.pdf Last consulted on June 22, 2010. (Return to source paragraph)
    
    
14. Paul Crookall. "Trust, the antidote to risk aversion", IT in Canada. January 4, 2010, p. 2. https://canadiangovernmentexecutive.ca/trust-the-antidote-to-risk-aversion/. (Return to source paragraph)
    
    
15. Canada School of Public Service. 2008-2009 Report on Plans and Priorities, pp. 14-16. Amended on 2008-02-12. Plans and reports. Last consulted on February 2, 2010. (Return to source paragraph)
    
    
16. Ibid., pp. 22-23. (Return to source paragraph)
    
    
17. Canada School of Public Service. 2008-2009 Report on Plans and Priorities, pp. 14-16. (Return to source paragraph)
    
    
18. Canada School of Public Service. 2008-2009 Report on Plans and Priorities, pp. 14-16. Amended on 2008-02-12. Plans et rapports. Last consulted on February 2, 2010. (Return to source paragraph)
    
    
19. This typology is described in Hope Seidman et al. "From Theory to Practice: Communities of Practice across the Canadian Public Service", in Niki Lambropoulos and Margarida Romero (eds). Educational Social Software for Context-Aware Learning: Collaborative Methods and Human Interaction. Hershey, New York: IGI Global, 2010. pp. 193-195. (Return to source paragraph)
    
    
20. In this table and all following tables, "N" stands for the number of respondents. (Return to source paragraph)
    
    
21. N/A represents the percentage of respondents who indicated the question was not applicable to them. (Return to source paragraph)
    
    
22. Centre of Expertise in Communities of Practice. Questionnaire to evaluate the training needs of editors/facilitators/coordinators. Montreal, December 2009, pp. 2, 5. (Return to source paragraph)
    
    
23. Interview with a learning specialist from the Professional Centre for Leadership and Management Development, January 29, 2009. (Return to source paragraph)
    
    
24. Interview with a senior analyst and a knowledge management advisor from the organizational leadership and innovation group, January 25, 2010. (Return to source paragraph)
    
    
25. Interview with the Chief Librarian of the Canada School of Public Service, January 27, 2010. (Return to source paragraph)
    
    
26. Interview with the Director General, Functional Communities, January 26, 2010. (Return to source paragraph)
    
    
27. These policies can be found on the Treasury Board Secretariat of Canada's Web site. See in particular: Policy on Management of Information Technology, effective July 1, 2007. http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?evttoo=X&id=12755&section=text. (Return to source paragraph)
    
    
28. Interview with the Director of the Information Management and Technology group, January 25, 2010. (Return to source paragraph)
    
    
29. Interview with a senior analyst and a knowledge management advisor from the Organizational Leadership and Innovation Branch, January 25, 2010. (Return to source paragraph)
    
    
30. Interview with the Director of Departmental Client Relations, January 22, 2010. (Return to source paragraph)
    
    

Contact us Subscribe to the GCLearning newsletter

---

Features

Event: Recognizing the Resilience of Indigenous Women and the 2SLGBTQIA+ Community

(May 3, 2024)

Event: GBA Plus in Action: Respect, Relationships, and Intersectional Leadership in Gwaii Haanas

(May 6, 2024)

Event: Unlocking the Power of Scientific Evidence for Effective Policy-Making

(May 7, 2024)

Follow:

• Twitter
• YouTube
• LinkedIn
• GCLearning newsletter
• RSS