Health Canada
Symbol of the Government of Canada

Common menu bar links

Drugs and Health Products

Results of the Evaluation of Phase I of the Summary Basis of Decision Project

Help on accessing alternative formats, such as Portable Document Format (PDF), Microsoft Word and PowerPoint (PPT) files, can be obtained in the alternate format help section.

January 29, 2010

Notice

The Therapeutic Products Directorate (TPD) and the Biologics and Genetic Therapies Directorate (BGTD) are pleased to announce the release of the results of the evaluation of phase I of the Summary Basis of Decision (SBD) project.

The SBD initiative was developed to increase the transparency of the drug and medical device regulatory review processes and to provide Canadians with improved access to information about authorized products. Phase I of the project was launched on January 1, 2005. Notice of Decision (ND) and SBD documents were published for new drug submissions for new active substances and a subset of class IV medical devices authorized by TPD or BGTD after that date.

TPD and BGTD conducted an evaluation of phase I of the SBD project; the period evaluated was January 1, 2005 to September 30, 2008, during which time a total of 197 NDs and SBDs were published. The evaluation included an assessment of external stakeholders' satisfaction with the SBD project, via an evaluation workbook published on the Health Canada Web site in April/May 2009.

The evaluation report contains short-term recommendations that will be implemented immediately, as well as findings that will inform the development of phase II of the SBD project. The development of phase II of the SBD project will include its own action plan, analysis, and recommendations and will proceed through full internal and external consultations.

Questions or concerns related to the evaluation report should be directed to:

Office of Business Transformation
Therapeutic Products Directorate
Health Canada
Holland Cross, Tower A, Address Locator 3002C
11 Holland Avenue
Ottawa, Ontario
K1A 0K9

Telephone: 613-941-1248
Facsimile: 613-957-1483
E-mail: OBT_Enquiries@hc-sc.gc.ca

Table of Contents

Executive Summary

The Summary Basis of Decision initiative was developed to increase the transparency of the drug and medical device regulatory review processes and to provide Canadians with improved access to information about authorized products.

Phase I of the project was launched on January 1, 2005. Notice of Decision (ND) and SBD (SBD) documents were published for new drug submissions for new active substances and a subset of class IV medical devices authorized by the Therapeutic Products Directorate (TPD) or the Biologics and Genetic Therapies Directorate (BGTD) after that date.

An evaluation of phase I of the SBD project has been conducted by the Office of Business Transformation, TPD, responsible for the implementation of the project for both TPD and BGTD. The period evaluated was January 1, 2005 to September 30, 2008, during which time a total of 197 NDs and SBDs were published.

Data collection took place from Fall 2008 to Spring 2009. Internal data collection was conducted via a survey distributed to all TPD and BGTD staff, and interviews held with staff and managers of both Directorates. External data collection was conducted via a questionnaire distributed to drug submission sponsors and medical device manufacturers and an evaluation workbook posted on the Health Canada Web site. Also assessed were the transparency needs of Canadians, content of SBDs for drugs versus (vs.) the Product Monograph (PM), Web hits for SBD-related pages, performance vs. targets, resources, and other data.

The evaluation has produced short-term recommendations (that can be implemented immediately as directed by the SBD Unit), as well as findings that will inform the development of phase II of the SBD project. The latter will be assessed by the Bureau of Policy, Science and International Programs (BPSIP) as part of the process for development of phase II; this process will include its own recommendations and will proceed through full internal and external consultations.

Short-Term Recommendations

Communications

A regular listing of ND/SBD postings should be sent by e-mail, either to the area (bureau/office/centre) involved, or to the entire Directorate. In TPD, this could be combined with the weekly Good Review Practices News e-mails.

The Standard Operating Procedures (SOPs) should be widely communicated within TPD and BGTD. In TPD, they should be posted on the Good Review Practices Intranet site.

The information available to drug submission sponsors and medical device manufacturers should be improved. Information sessions could be held by the SBD Unit (perhaps in conjunction with industry associations), more information (such as SOPs) should be posted on the Health Canada Web site, and companies should be encouraged to contact their project managers and/or the SBD Unit for information. Until the SOPs are posted, the relevant SOP should be sent to individual sponsors/manufacturers with the draft ND.

Posting of Information with Notice of Decision/Summary Basis of Decision Documents

For medical devices, Operator's Manuals (OMs)/Package Inserts (PIs) should continue to be posted, but complex lengthy Instructions for Use (IFU) documents should not be published when provided.

The SOPs for the publication of NDs/SBDs for the three product lines (pharmaceuticals, biologics, and medical devices) should be posted on the SBD Web page.

Portions of the text of the SBD Foreword should be repeated on the SBD Web page itself, to provide readers with direct links to Fact Sheets on the review of drugs and medical devices in Canada, and relevant guidance documents.

Process for Producing Notice of Decision and Summary Basis of Decision Documents

Direct communication between the technical writer and reviewer should be increased; this includes involving the technical writer earlier in the review process. Work should continue to encourage meetings between the technical writer and the reviewer(s), at a minimum during review [for example (e.g.) the first team meeting] and after the reviewer has completed his/her assessment. A meeting should also take place when the first draft of the SBD has been prepared, to discuss the document.

Reviewers should ensure that their evaluation report clearly identifies their comments as well as important information supporting their decision that is to be summarized in the SBD. Reviews should accurately reflect the emphasis placed on certain areas of the evaluation and should include summaries and conclusions as well as a complete benefit/risk analysis. Reviews should also clearly identify where recommendations change as the review progresses (e.g. after the issuance of a clarifax or a request for additional information).

Review managers should be more knowledgeable of and involved in the ND/SBD process. They should recognize and allow the time required for reviewers to work on NDs and SBDs, and include this time in review workplans. In addition, review managers should recognize and acknowledge the importance of transparency, and have a positive attitude towards the SBD project.

Interactions with sponsors/manufacturers on individual documents should be fairly limited. The role of the sponsor vs. Health Canada should be clearly stated as part of the SBD initiative, and it should be clear that NDs/SBDs are Health Canada documents that will document both the positive and negative aspects of the regulator's assessment of the product.

Rationales should be provided to the drug submission sponsor/medical device manufacturer to explain why proposed revisions to an ND/SBD were accepted or rejected by Health Canada. Direct communication between technical writers and sponsors/manufacturers should be encouraged.

Sponsors and manufacturers of SBD-eligible products should be told when their product is authorized that draft NDs/SBDs will be provided for comment. In addition, as draft documents are being approved for sponsor comment, Health Canada should notify the sponsor to allow them to align their resources accordingly so they can more easily meet timelines.

The SBD Unit should continue work to reduce the impact of SBDs on the review population, including delivering information sessions on the project and the documents, giving advice/training on how reviewers can identify information for inclusion in SBDs, offering regular meetings between review teams and the technical writers, et cetera (etc.).

At this time, it is not recommended that targets related to NDs or SBDs be revised. This must be revisited during the development of phase II as a consequence of revisions to the scope of SBD-eligible products and submissions/applications, and changes to ND/SBD documents.

Resources

The SBD Unit should continue to regularly review the ND/SBD templates to update standard (already-translated) statements and continue to reduce translation costs.

Senior management within TPD, BGTD, and indeed HPFB, should re-iterate their commitment to transparency as a Departmental and Branch priority. This should be accompanied by increased scrutiny on areas consistently not meeting ND/SBD targets. On-going efforts to reduce the impact of SBDs on reviewers should continue.

TPD and BGTD staff should be asked to track time against an SBD-specific code in Timetrak with each draft ND/SBD sent by the technical writers.

As draft documents are being approved for sponsor comment, Health Canada should notify the sponsor to allow them to align their resources accordingly so they can more easily meet timelines.

Findings to Inform the Development of Phase II

Purpose of Notice of Decision and Summary Basis of Decision Documents

The focus of the SBD project should continue to be two-pronged: providing unbiased information to Canadians about the products reviewed and authorized by Health Canada, and improving the transparency of the review processes for drugs and medical devices. However, it is possible that the two goals may conflict (e.g. the former may require more concise information to be published while the latter may require more details to be published). In these cases, the goal of providing unbiased information to Canadians about products should take precedence, particularly given the current budget situation faced by the Health Products and Food Branch (HPFB).

Notice of Decision documents should be targeted to the 'general public' and SBDs to scientific or healthcare professionals. Revisions to both documents (including content, format, and language) should be made with these specific audiences in mind, rather than attempting to address both needs with both documents. This will be expanded upon throughout this report.

Consideration should be given to expanding the scope of SBD-eligible products to include more (or all) class IV medical devices (and potentially class III devices with novel technology), products eligible for review under the Priority Review and/or Notice of Compliance with Conditions Policies, and subsequent entry biologics. Submissions for generic drugs are not recommended for inclusion at this time.

Consideration should be given to expanding the scope of SBD-eligible submissions and applications to include supplemental submissions and applications for amendments for those drugs and medical devices (respectively) that are eligible for NDs/SBDs. It should be examined whether this is most feasibly done by posting new NDs and/or SBDs for subsequent submissions/applications or by updating NDs/SBDs published on first authorization.

Notices of Decision and/or SBDs should be updated (either via a revised or a new document) to describe whether and how conditions of the authorization are met, for those products authorized under the Notice of Compliance with Conditions Policy or Section 36(2) of the Medical Devices Regulations.

SBDs should be linked to the most recent PM, as well as to post-authorization activities, including risk communications, recalls, etc. found in other areas of the Health Canada Web site.

Consideration should be given to including within NDs/SBDs those indications that were modified or rejected by Health Canada during the review process for SBD-eligible submissions and applications. In addition, consideration should be given to expanding the scope of SBD-eligible submissions and applications to include those that were rejected by Health Canada, particularly for new uses for products already on the market.

Language, Structure, and Content of Notice of Decision and Summary Basis of Decision Documents

The focus of the SBD project should be to provide information not available elsewhere that would assist Canadians in making informed decisions about their health.

The format of the ND should be revised so that information can be more easily read and retrieved, via the addition of sub-headings or via a Question and Answer format. The ND should continue to focus on providing high-level information about the product and the efficacy of the product, and provide additional information about safety (e.g. serious adverse events).

A new document should be posted containing the information currently found in Section 1 of the SBD for drugs and medical devices (Product and Submission Information/Device and Application Information, respectively). This should be posted along with the ND.

The regulatory history of the submission/application should be expanded and easily retrievable in one section of the SBD, e.g. the reasons for a negative decision issued during the review, whether advice was solicited from a Scientific Advisory Committee and how it was assessed, etc.

Notice of Decision and Summary Basis of Decision documents should complement one another and provide information in a tiered approach. The documents should be appropriately linked so that readers can choose to access more or less information.

The SBD for drugs should be significantly shorter and focus on the clinical basis for the decision (notably safety and efficacy) as well as Health Canada's assessment of the risks and benefits of the product. Sections on the quality (with the exception of the assessment of adventitious agents), non-clinical, and clinical pharmacodynamics and pharmacokinetics portions of the submission should be replaced by brief, high-level statements assessing the adequacy of information filed. These sections should, however, highlight significant safety concerns with these portions of the submission that may be relevant to the clinical use of the product, as well as precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of these sections. The section on Risk/Benefit Assessment and Recommendation should be expanded and become more of the focus of the SBD.

The SBD for medical devices should also be shorter and again should focus on Health Canada's assessment of the risks and benefits of the product, primarily the clinical safety and effectiveness components of the application. Sections related to standards referenced, sterilization, manufacturing and quality control, preclinical studies, and software validation studies should be replaced by general high-level statements assessing the adequacy of information filed. These sections should, however, highlight significant safety concerns with these portions of the application that may be relevant to the clinical use of the product, as well as precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of these sections. The sections on Risk/Benefit Assessment and Recommendations should be expanded and become more of the focus of the SBD.

Communications

During the development of Phase II, options to improve the visibility of SBD-related documents on the Health Canada Web site should be explored. This includes exploring potential linkages with the Drug Product Database (DPD) Online, as well as potentially raising the profile of the SBD pages on the Health Products portions of the Web site. Linking the documents via plain-language explanations of their purpose should be considered (e.g. prefaced by statements such as 'If you'd like to know...').

Linkages should also be established between post-market Web pages and SBD-related pages. For example, if Health Canada has published a risk communication or recall regarding a product, this should be linked from the SBD page (and vice versa).

Consideration should also be given to a more user-friendly organization of SBD-related documents, particularly if documents are drafted for more products or for multiple submissions/applications for a product.

Once phase II has been developed and implemented, promotion activities should be undertaken to promote SBD-related documents and expand audience awareness. This could include partnering with healthcare professional and patient associations (e.g. the Canadian Pharmacists Association); media announcements; newspaper advertisements; brochures in healthcare practices, etc.

Posting of Information with Notice of Decision and Summary Basis of Decision Documents

For drugs (pharmaceuticals and biologics), the posting of Part III of the PM with the ND/SBD should be discontinued. Instead, consider ways in which NDs/SBDs for drugs can be linked to the most recent PM for the product, published as part of the DPD.

It is recommended that the area responsible for the regulation of medical devices publish current labelling information for licensed medical devices, similar to the publication of PMs for drugs.

Process for Producing Notice of Decision and Summary Basis of Decision Documents

It is not recommended that the role of technical writer be changed, or that NDs/SBDs be drafted by the review community.

Both Directorates should focus on earlier and more informal methods of dispute resolution for NDs and SBDs, including resolving issues of inaccuracies of data between the reviewer/technical writer and the sponsor within the ND/SBD process. Appeals should be limited to issues of interpretation of data, which will assist in allowing appeals to be resolved within target.

Resources

As discussed in Section 3 of this report, findings of this evaluation to streamline the ND/SBD documents would reduce the length and complexity of the documents, therefore reducing average translation costs. This would also reduce the review community's workload associated with reviewing individual documents.

Findings of this evaluation related to increasing the scope of products and submissions/applications eligible for SBDs should be examined in terms of their operational feasibility. This should include an assessment of the impact of proposed changes on technical writer staffing needs, the workload of the review, publications and Departmental Web communications communities, and translation costs associated with an increased number of documents.

1. Background

1.1. Summary Basis of Decision Project

The SBD project was initiated in 2003 as part of the transparency agenda of the Therapeutics Access Strategy (TAS). TAS was launched within the Health Products and Food Branch (HPFB) of Health Canada in direct response to the 2002 Speech from the Throne. Original goals of the SBD initiative were to increase the transparency of the drug and medical device regulatory review processes and to provide Canadians with improved access to information about authorized products. As a result of the SBD project, new documents (NDs and SBDs) were created and published on the Health Canada Web site.

An SBD outlines the scientific and benefit/risk-based considerations that factor into Health Canada's decision to grant market authorization for a drug or medical device. The document includes regulatory, safety, efficacy, and quality (chemistry and manufacturing) considerations and reflects the information within the regulatory review reports.

An ND is an approximately one-page summary outlining the authorization received and general information related to the drug or medical device. The ND is published independently after product authorization and is incorporated into the SBD as Section 2.

Guiding principles for the SBD Initiative were drafted when the project was being developed in 2003/04, and are attached as Appendix 1. These principles guided the development of the project, as well as its implementation and SBD-related decisions since the launch of phase I.

At the time the project was developed in 2003/04, three phases of the project were planned. Phase I would see documents published for authorized new drug submissions for new active substances (pharmaceuticals and biologics) and a subset of authorized class IV medical devices. Expansion of the project was planned for phase II, with documents being published for all authorized new drug submissions, abbreviated new drug submissions, and supplements, and an expanded set of authorized class IV medical device applications. Phase III was planned to explore the publication of documents related to negative decisions, consistent with international shifts and changing regulatory boundaries in terms of what is considered to be confidential business information.

Phase I of the SBD project was launched on January 1, 2005. Notice of Decision and SBD documents were published for new drug submissions for new active substances and a subset of class IV medical devices authorized by TPD and BGTD after that date. The subset of class IV medical device applications related to any one of the following: priority review applications, in vitro diagnostic devices for donor screening, cardiovascular devices with novel technology (endovascular stenting systems, carotid stenting systems, and left ventricular assist devices) and new indications for use for cardiovascular and neurological devices.

In phase I of the project, the SBDs have been 'frozen' to reflect only the information that supported the original decision to authorize the product. Post-authorization activities are not included in the documents, including subsequent submissions/applications filed for the product, risk communications issued, etc.

1.2. Publication of Notice of Decision and Summary Basis of Decision Documents

The first NDs were published on May 27, 2005 and the first SBDs on November 18, 2005. Table 1 outlines the number of NDs and SBDs that were published as of September 30, 2008.

Table 1: Notice of Decision and Summary Basis of Decision documents published by TPD and BGTD between January 1, 2005 and September 30, 2008
  Notice of Decisions Published Summary Basis of Decisions Published
Pharmaceuticals 61 51
Medical Devices 19 20
Biologics 24 22
Total 104 93

1.3. Evaluation of Phase I of the Summary Basis of Decision Project

Phase I was anticipated to proceed for approximately one year and then was to be followed by an evaluation and the development of phase II of the project. However, as seen by the dates of publication of the first documents above, implementation of the project was delayed while the necessary infrastructure, templates, and processes were put in place. As a result, an evaluation after one year was not appropriate. Resource concerns caused further delays and the evaluation began in the summer/fall of 2008.

The purpose of the evaluation was to determine whether the documents, and the processes by which they are drafted, meet the needs of staff and stakeholders. The Action Plan for the evaluation is on file.

The evaluation was conducted by the Office of Business Transformation (OBT), TPD, the office responsible for the implementation of phase I of the SBD project for both TPD and BGTD. The evaluation was carried out in conjunction with the Bureau of Policy, Science and International Programs (BPSIP), TPD with the assistance of the Office of Consumer and Public Involvement, HPFB.

The evaluation period included a total of 45 months, from January 1, 2005 to September 30, 2008. Data collection took place from Fall 2008 to Spring 2009. Internal data collection was conducted via a survey and interviews. External data collection was conducted via a questionnaire to drug submission sponsors and medical device manufacturers, and an evaluation workbook published on the Health Canada Web site. Other data were also collected and analysed by OBT and are described below.

The evaluation has produced short-term recommendations that can be implemented immediately as led by the SBD Unit, OBT, TPD. The report also describes a number of findings that will inform the development of phase II of the SBD project, which will be led by BPSIP, TPD. The findings from the evaluation report will be assessed by BPSIP as one element of the development process for phase II. The analysis conducted by BPSIP will result in proposals for phase II that will proceed through full internal and external consultations prior to implementation.

1.3.1. Internal Survey

An electronic survey was prepared and distributed to all TPD and BGTD staff on December 4 and 12, 2008, respectively. The closing date was originally December 31, 2008 but was extended to January 23, 2009. Response rates were similar for TPD and BGTD: 13% (78 out of approximately 615 staff) and 15% (55 responses out of approximately 365 staff) respectively. The majority of respondents were from the review community (55.8% in TPD and 41.8% in BGTD). Survey questions and results are on file.

1.3.2. Internal Interviews

To supplement the information obtained via the internal survey, targeted staff and managers were interviewed between December 2008 and February 2009. Staff and managers who had been involved with ND/SBD production were identified for each bureau, office, and centre in TPD and BGTD. These names were provided to Directors, who were asked to identify 2-4 people from their area to be interviewed. Interview questions and results are on file.

Overall, interviews were conducted with 28 staff and managers from TPD and 10 from BGTD. The majority of those interviewed were reviewers (40.5% or 15 interviewees); 27.0% (10 interviewees) were managers and 6.2% (6 interviewees) were project managers. Other interviewees included the SBD technical writers, publications coordinators, and staff who manage ND/SBD-related appeals. Most interviewees had worked on a total of 1-3 NDs/SBDs since the project's inception.

1.3.3. Questionnaire for Industry

A questionnaire was distributed to drug submission sponsors and medical device manufacturers who had been involved with an ND or SBD published for at least one product prior to September 30, 2008. The survey was distributed to 55 companies and 20 responses were received. The majority of respondents (60%) had been involved with SBDs for pharmaceutical products only; 20% were involved with both pharmaceuticals and biologics. Only one response from a medical device manufacturer was received. The majority of respondents had worked on 1-2 SBD documents, which is consistent with the diverse nature of products and companies involved. The questionnaire and results are on file.

1.3.4. Evaluation Workbook

An evaluation workbook was published on the Health Canada Web site on April 8, 2009 with a closing date of May 15, 2009. It was also published on the Health Canada Public Involvement Web page and the Government of Canada's Consulting with Canadians Web site. An e-mail invitation to complete the questionnaire was sent to almost 1500 Health Canada stakeholders, followed by an e-mail reminder. Forty distinct responses were received; of these, 47.5% were involved with the drug/medical device industry (drug submission sponsors, medical device manufacturers, consultants) and 32.5% identified themselves as patients.

The evaluation workbook questions and results are on file.

1.3.5. Other Data Collection

The Office of Business Transformation collected and analyzed data related to a number of other aspects of the SBD project, including:

  • media requests related to, or responded to with, SBDs;
  • requests filed under the Access to Information (ATI) Act;
  • assessment of the use of ND/SBD documents domestically and internationally;
  • assessment of transparency needs for drug- and medical device-related information, including a review of literature, public opinion surveys, etc.;
  • information available in other jurisdictions (United States Food and Drug Administration [FDA], European Medicines Agency [EMEA]) for drugs and medical devices;
  • content of ND/SBD for drugs vs. the PM;
  • Web site hits from January 1, 2007 - September 30, 2008 for SBD-related pages (data was not available for 2005 or 2006);
  • appeals filed related to NDs/SBDs between January 1, 2005 and September 30, 2008;
  • performance vs. targets for various stages of the ND/SBD process; and
  • resources spent internally (money and human resources).

The data and analyses are on file.

1.3.6. Limitations of Evaluation

As expected when the evaluation was planned, the evaluation has been limited by its ability to reach the 'general public'. As described above, the evaluation workbook was posted on the Health Canada Web site as well as a number of other consultation Web sites but the majority of respondents were already very involved with Health Canada.

In addition, a much smaller number of respondents (to internal and external components of the evaluation) were familiar with, and able to respond to, questions related to NDs/SBDs for medical devices than for drugs.

2. Purpose of Notice of Decision and Summary Basis of Decision Documents

2.1. Original Purpose

The original goals of the SBD initiative were to increase the transparency of the drug and medical device regulatory review processes, and to provide Canadians with improved access to information about authorized products. Results from various components of the evaluation provided information related to what the documents are actually used for.

Results of the evaluation workbook revealed that 20.8% of respondents accessed the ND, and 24.0% the SBD, in order to obtain information related to the regulatory review process. In addition, 41.7% of respondents accessed the ND, and 36.0% the SBD, to obtain information related to a specific drug or device.

The vast majority (87.5%) of respondents to the evaluation workbook stated that the ND is useful or somewhat useful in providing information about the product and/or the submission/application. In response to similar questions regarding the SBD, 82.5% of respondents said the SBD is useful or somewhat useful in providing information about the product and/or the submission/application, and 83.1% said the SBD is useful or somewhat useful in providing information about the drug or medical device review process. Fewer respondents (a total of 57.5%) said the SBD was useful or somewhat useful in helping them make informed treatment choices (for themselves or their patients), while a total of 37.5% of respondents said they did not use the document for this purpose, had not read SBDs, or did not respond.

In the internal survey, of those TPD/BGTD staff that refer external stakeholders to NDs/SBDs, most do so to give information related to a specific product or submission/application. About half as many do so to provide information on the product review process or Health Canada transparency initiatives. In TPD, most external stakeholders being referred to NDs/SBDs are industry, while in BGTD most are the general public.

Media requests to Health Canada related to SBDs during the evaluation period were assessed. When the SBD project was launched in 2005 there was some media interest, however most requests related to the data submitted and/or the review/approval process for specific high-interest products (e.g. breast implants, Gardasil vaccine). These requests were responded to at least in part by providing a link to the relevant ND/SBD.

2.2. Observed Uses

Various components of the evaluation provided insight into reasons NDs and SBDs are accessed by different audiences, beyond the original goals of the SBD initiative.

While not a stated purpose of the SBD project, during its development in 2003/04 it was predicted that the availability of detailed information in the SBD would reduce the number of requests filed under the ATI Act (at least for those specific products within the scope of phase I), or perhaps result in requests that were more specific. Consultations with the group responsible for ATI requests in HPFB have indicated that the number of ATI requests filed for drug- and device-specific documents has declined significantly since 2004/05 for a number of reasons and therefore it is impossible to specifically assess the impact of SBDs on ATI requests.
 
Internationally, it is increasingly being recognized that information made available by well-resourced regulatory authorities can facilitate the decision-making processes, and thereby enhance the regulatory capacity, of regulatory authorities that are less well-resourced. This will in turn improve world access to medicines particularly in impoverished areas. The World Health Organization recognizes that transparency is one of the issues necessary to promote good regulatory practices nationally and internationally.

Many internal responses to the survey and/or the interviews indicated that HPFB should discontinue the publication of NDs and SBDs altogether and instead focus resources on review. This was not an opinion expressed in responses to the sponsor questionnaire or the evaluation workbook, with the exception of one respondent who indicated that Health Canada should view this initiative as a success and move on to other projects as expansion would hold little value. Other internal respondents (to the survey and the interviews) indicated that SBDs should be discontinued and replaced with an expanded ND; this will be further explored in Section 3 of this report. However, Health Canada and HPFB have recently emphasized their respective commitments to the provision of information to assist Canadians in maintaining and improving their health, and in making healthy informed choices. Overall, the various components of the evaluation indicated that there is a need for accurate, unbiased information for Canadians - a gap that can and should be addressed by Health Canada.

2.3. Audience

When the SBD project was developed in 2004, the audience for NDs/SBDs was identified as the 'interested public'; e.g. healthcare professionals or patients with an interest in their care. It was recognized that the SBDs in particular were quite technical documents and would likely not be understood by the majority of the general public.

As part of the assessment of the transparency needs of Canadians, it is recognized that the audience to which Health Canada will target NDs/SBDs needs to be clarified, in order to target revisions to best meet the needs of readers. The assessment identified that there is a need for Health Canada information for two populations: the 'general public' and scientific or healthcare professionals. The two populations will have different specific information needs, levels of health literacy, etc. and should be considered separately.

2.4. Scope of Eligible Products and Submissions/Applications

As described in the Background section above, the scope of Phase I as it was originally established in 2004 includes new drug submissions for new active substances and a subset of class IV medical devices authorized by TPD and BGTD after January 1, 2005. The subset of class IV medical device applications related to any one of the following: priority review applications, in vitro diagnostic devices for donor screening, cardiovascular devices with novel technology (endovascular stenting systems, carotid stenting systems, and left ventricular assist devices) and new indications for use for cardiovascular and neurological devices.

In addition, NDs and SBDs have also been published for authorized breast implant applications, and the first-authorized subsequent entry biologic product. These were exceptions to phase I made in response to a need for transparency for these particular products of very high interest.

Notices of Decision and SBDs currently only contain information that supported the original decision; they are not updated to include post-authorization activity.

The majority of respondents for the internal survey (40.8% of those from TPD and 52.7% of those from BGTD) indicated this scope was satisfactory. More than one-third of respondents from each Directorate had no opinion, while 18.4% and 12.7% from TPD and BGTD, respectively, indicated the scope was too small.

In the evaluation workbook, 61.5% of respondents indicated that the scope of phase I was satisfactory, while 23.1% stated the scope was too small.

As explored further in Section 4 of this report, there is concern that due to the limited scope of phase I, the documents are not highly visible or well-known to Canadians. An expansion of the scope of products and/or submissions/applications eligible for SBDs would likely increase the visibility of the project and make Canadians more aware of their existence. However, the development of phase II will need to balance this against budgetary implications associated with an expansion of the scope of the project.

Responses to the internal survey and/or the evaluation workbook suggested the following additional medical devices for ND/SBD drafting: all class IV and class III device applications, and suggestions for specific types of class IV medical devices (e.g. trans-apical heart valves, more musculoskeletal devices). The following additional drugs were suggested for ND/SBD drafting: all new drug submissions (not just those for new active substances), subsequent entry biologics, and generics.

Given the current budget situation in HPFB, it may be prudent to focus on those products of highest interest such as those with novel technology, those eligible for priority review, and those authorized under new regulatory regimes such as subsequent entry biologics. There is likely less interest in product-specific information for generic products as they have been on the market for longer, albeit by a different company.

Both internally and externally, suggestions have repeatedly been made that HPFB should include post-authorization activity in the scope of NDs/SBDs for authorized products. This would include information on authorized supplemental submissions (drugs) or applications for amendments (medical devices) for new indications/uses.

There is significant concern that information available on the Health Canada Web site can be out-of-date and not reflect the current status or knowledge of a product. This is also a concern for drugs and medical devices authorized with conditions: while the conditions are described in the SBD, no further information is published to explain whether and how the company met those conditions. While there is benefit to making historical information available, in order to be most useful to patients and healthcare professionals, information must be current.

2.4.1. Publication of Information Related to Negative Decisions

As described above, NDs and SBDs are currently published for authorized products only. In addition, the documents currently contain information only about approved indications/uses and do not normally refer to indications modified or rejected during the review process. However, as described in the assessment of transparency needs of Canadians, a significant portion of prescriptions processed are for off-label (that is to say [i.e.] unapproved) use of a drug product; patients and even physicians may not be aware when a drug is being used off-label. In some cases, an indication may have been modified, or a submission/application for a new indication/use rejected by the regulator due to safety or efficacy concerns, but patients and physicians would have no way of knowing this. In addition, drug companies have been criticized for indirectly providing information related to, or otherwise promoting, off-label use of a product.

The publication of the reasons for modifying or rejecting an indication/use for a product otherwise on the market would therefore provide useful information to patients and healthcare professionals alike.

In addition, information on products not authorized for sale by Health Canada is not currently made available by the Department. Publication of this information would increase the transparency of departmental activities and decisions, and may provide useful information about related/similar products that are already on the market.

The international landscape has changed significantly in terms of the publication of negative decisions since the SBD project was developed in 2004. The EMEA now publishes considerable information about applications that have been refused by the Agency or withdrawn by the company. The United States FDA, with its recent transparency-related proposals, has indicated it may consider publishing information on rejected applications.

Internally and externally, it has been suggested that all negative decisions should be subject to NDs/SBDs. Given the current budget situation in HPFB as well as recent Branch and Departmental commitments to make more information available to assist Canadians in making healthy and informed decisions, it may be prudent to focus on publishing information that would have the greatest impact on the health of Canadians. The publication of information related to new indications/uses that were submitted and rejected for products already on the market would address concerns related to off-label use of authorized products, as well as provide patients and healthcare professionals with important information that could affect their safe use of a product.

2.5. Recommendations and Findings

2.5.1. Findings to Inform the Development of Phase II

The focus of the SBD project should continue to be two-pronged: providing unbiased information to Canadians about the products reviewed and authorized by Health Canada, and improving the transparency of the review processes for drugs and medical devices. However, it is possible that the two goals may conflict (e.g. the former may require more concise information to be published while the latter may require more details to be published). In these cases, the goal of providing unbiased information to Canadians about products should take precedence, particularly given the current budget situation faced by HPFB.

Notice of Decision documents should be targeted to the 'general public' and SBDs to scientific or healthcare professionals. Revisions to both documents (including content, format, and language) should be made with these specific audiences in mind, rather than attempting to address both needs with both documents. This will be expanded upon throughout this report.

Consideration should be given to expanding the scope of SBD-eligible products to include more (or all) class IV medical devices (and potentially class III devices with novel technology), products eligible for review under the Priority Review and/or Notice of Compliance with Conditions Policies, and subsequent entry biologics. Submissions for generic drugs are not recommended for inclusion at this time.

Consideration should be given to expanding the scope of SBD-eligible submissions and applications to include supplemental submissions and applications for amendments for those drugs and medical devices (respectively) that are eligible for NDs/SBDs. It should be examined whether this is most feasibly done by posting new NDs and/or SBDs for subsequent submissions/applications or by updating NDs/SBDs published on first authorization.

Notices of Decision and/or SBDs should be updated (either via a revised or a new document) to describe whether and how conditions of the authorization are met, for those products authorized under the Notice of Compliance with Conditions Policy or Section 36(2) of the Medical Devices Regulations.

SBDs should be linked to the most recent PM, as well as to post-authorization activities, including risk communications, recalls, etc. found in other areas of the Health Canada Web site.

Consideration should be given to including within NDs/SBDs those indications that were modified or rejected by Health Canada during the review process for SBD-eligible submissions and applications. In addition, consideration should be given to expanding the scope of SBD-eligible submissions and applications to include those that were rejected by Health Canada, particularly for new uses for products already on the market.

3. Language, Content, and Structure of Notice of Decision and Summary Basis of Decision Documents

The evaluation of phase I examined the language used in both NDs and SBDs, as well as the content and structure of both documents.

As described in the assessment of transparency needs of Canadians, it is important that patients and healthcare professionals not be overwhelmed with the amount of information available to them. In such cases, patients in particular are likely to become overloaded and either not read the information available, or read it but not retain it.

Responses from various internal and external components of the evaluation generally preferred that ND and SBD documents focus on Health Canada's assessment of the risks and benefits of a product, and not duplicate information found in the product labelling. In addition, Health Canada and HPFB have recently emphasized their respective commitments to the provision of information to assist Canadians in maintaining and improving their health, and in making healthy informed choices.

As raised in the 2004 Consultative Workshop on SBDs, and as described in the assessment of transparency needs of Canadians, a tiered approach to ND/SBD documents (with appropriate linkages) would allow readers to access more or less information as they so choose.

It is recognized that the SBD is written in technical language that is not likely to be understood by the average consumer. As discussed in Section 2 of this report, NDs should be targeted to the 'general public', and SBDs to the scientific or healthcare professional. This finding impacts proposed revisions to the language, content, and structure of the documents, as described below.

3.1. Language

Concerns were identified internally that the language of SBDs may be too technical to be understood by patients, or even some healthcare professionals. The evaluation workbook specifically asked whether respondents could understand the language used in NDs and SBDs. While 82.5% of respondents indicated they could understand the language in an ND, and 87.5% the language in an SBD, several acknowledged that this was due to their existing medical, scientific, or technical backgrounds. It should be noted that although 32.5% of respondents identified themselves as patients, many indicated they were also healthcare professionals. In addition, almost half of workbook respondents were from the drug/medical device industry.

Although revisions to the language of the ND may be considered to ensure that the NDs can be easily understood by the target audience (the 'general public'), this must be balanced by a need to be technically accurate in the descriptions of approved indications, contraindications, etc. Summary Basis of Decision documents targeted to scientific or healthcare professionals may retain their technical language.

3.2. Content/Structure

As discussed above, the ND for both drugs and medical devices should be targeted to the 'general public'; the SBD should complement the ND but provide more scientific details targeted towards scientific or healthcare professionals.

Although the current ND format is fairly straightforward, it could be improved so that information can be more easily read and retrieved. In addition, concerns have been raised that the NDs are focussed on efficacy and provide little information about significant safety concerns (e.g. serious adverse events).

In the SBD, Section 1 (Product and Submission Information or Device and Application Information for drugs or medical devices, respectively) currently provides general information on the product. Results of the internal and external components of the evaluation revealed that this section is quite useful in providing general information about the product that may not be easily retrievable elsewhere. There were several suggestions to add general information about the disease and mechanism of action, and to make this information available closer to product authorization (perhaps combined with the ND).

The regulatory history of the submission/application, including Section 4 of the SBD (Submission Milestones or Application Milestones for drugs or medical devices, respectively), was viewed favourably internally and externally. Information such as whether a negative decision was issued during review, whether advice was solicited from a Scientific Advisory Committee and how it was assessed, etc. was of particular interest to respondents as it is not available elsewhere. Currently this information may not be consistently included in the SBD, or easily retrievable in one section of the document.

Other sections of the SBD are specific to either drugs or medical devices and are discussed separately below. Results of questions asked during the internal interviews related to SBD content for drugs and medical devices were mixed, and often depended on the interviewee's assessment of the audience of the documents. Generally, if the intent of the document was to be transparent about the review process then all components currently found in the SBD were considered to be important. However, if the intent was to provide information to patients or healthcare professionals then some sections could be removed or de-emphasized. In addition, suggestions were received that would allow the documents to be simplified somewhat so that they are more focused and potentially less overwhelming.

3.2.1. Content of Summary Basis of Decision Documents - Drugs

The current Table of Contents for an SBD for a drug (pharmaceutical or biologic) is found in Appendix II.

Given the findings of this evaluation to focus on the provision of information to assist Canadians to make healthy choices, and to target SBDs to scientific or healthcare professionals, some sections of the SBD may be too detailed while others do not provide enough information.

Section 3.1 (Quality Basis for Decision) currently includes a number of general statements about the data that were reviewed and considered acceptable. While the inclusion of this section increases the transparency of the drug review process to some extent by describing the type of information that is reviewed, the level of detail is limited by the proprietary nature of much of the quality information. Internal and external components of the evaluation revealed that this section is generally not considered to provide information that would contribute to the safe use of the product, with the exception of the assessment of adventitious agents.

Various components of the evaluation found that Section 3.2 (Non-clinical Basis for Decision), and portions of Section 3.3 (Clinical Basis for Decision, particularly clinical pharmacodynamics and pharmacokinetics) were also less useful in terms of their ability to provide information for the use of the product. In addition, much of this information is available in the PM. However, Health Canada's assessment of the implications of the non-clinical studies to humans is of interest, as are the precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of the non-clinical package and pharmacology studies.

Both the internal and external components of the evaluation identified Sections 3.3.4 (Clinical Efficacy) and 3.3.5 (Clinical Safety) as important components of the SBD. However, much of this information is available in the PM, now available online through the DPD. The evaluation indicated that the content of these sections should focus on Health Canada's assessment of these portions of the drug submission, including information that led to the inclusion of precautions, warnings, or contraindications in the labelling. In addition, many of the respondents to the internal survey and interviews indicated that the SBD should provide reviewers the opportunity to communicate Health Canada concerns with the submission that were not incorporated into the labelling. Lastly, these sections could also include Health Canada's assessment of supportive studies and secondary endpoints not described in the PM, if they factored significantly into the decision to authorize the product.

Internal and external components of the evaluation identified Section 3.4 (Risk/Benefit Assessment and Recommendation) as one of the most important sections of the SBD. In addition, a key information gap identified in the assessment of transparency needs of Canadians is Health Canada's assessment of the risks and benefits of a product.

3.2.2. Content of Summary Basis of Decision Documents - Medical Devices

The current Table of Contents for an SBD for a medical device is found in Appendix III.

Given the findings of this report to focus on the provision of information to assist Canadians to make healthy choices, and to target SBDs to scientific or healthcare professionals, some sections of the SBD may be too detailed while others do not provide enough information.

Sections 3.1 (Introduction) and 3.2 (Device-Specific Detailed Information) were considered useful as they provide an overview of the device. However, at times they were found to be repetitive and could be combined into one introductory section.

Section 3.3 (Devices Containing Biological Material) was regarded as important for devices containing animal or human material, as measures to mitigate risk are discussed.

The following sub-sections of 3.4, Safety and Effectiveness: 3.4.1 (List of Standards), 3.4.2 (Method of Sterilization), 3.4.3 (Manufacturing and Quality Control), and 3.4.6 (Software Validation Studies) currently include a number of general statements about the data that were reviewed and considered acceptable. While the inclusion of these sections increases the transparency of the device review process to some extent by describing the type of information reviewed, the level of detail is limited by the proprietary nature of much of this information. Internal and external components of the evaluation revealed that this section is generally not considered to provide information that would contribute to the safe use of the product.

Sub-section 3.4.4, Preclinical Studies, was also found to be relatively less useful in terms of its ability to provide information for the use of the product. However, Health Canada's assessment of the implications of the preclinical studies to humans is of interest, as are the precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of the preclinical studies.

Section 3.4.5, Clinical Safety and Effectiveness, was identified internally and externally as an important section of the SBD. The evaluation indicated that the content of this section should focus on Health Canada's assessment of these portions of the application, including information that led to the inclusion of precautions, warnings, or contraindications in the labelling. In addition, many of the respondents to the internal survey and interviews indicated that the SBD should provide reviewers the opportunity to communicate Health Canada's concerns with the application that were not incorporated into the labelling. Lastly, these sections could also include Health Canada's assessment of supportive studies and secondary endpoints if they factored significantly into the decision to authorize the product.

Section 3.4.7 (Labelling) currently consists of a standard statement indicating that the labelling was acceptable. Any serious risks, warnings, precautions, or contraindications added to the labelling would be discussed in other sections therefore this standard statement is sufficient.

Internal and external components of the evaluation identified Sections 3.5 (Risk/Benefit Assessment) and 3.6 (Recommendation) as two of the most important sections of the SBD. In addition, a key information gap identified in the assessment of transparency needs of Canadians is Health Canada's assessment of the risks and benefits of a product.

3.3. Recommendations and Findings

3.3.1. Findings to Inform the Development of Phase II

The focus of the SBD project should be to provide information not available elsewhere that would assist Canadians in making informed decisions about their health.

The format of the ND should be revised so that information can be more easily read and retrieved, via the addition of sub-headings or via a Question and Answer format. The ND should continue to focus on providing high-level information about the product and the efficacy of the product, and provide additional information about safety (e.g. serious adverse events).

A new document should be posted containing the information currently found in Section 1 of the SBD for drugs and medical devices (Product and Submission Information/Device and Application Information, respectively). This should be posted along with the ND.

The regulatory history of the submission/application should be expanded and easily retrievable in one section of the SBD, e.g. the reasons for a negative decision issued during the review, whether advice was solicited from a Scientific Advisory Committee and how it was assessed, etc.

Notice of Decision and Summary Basis of Decision documents should complement one another and provide information in a tiered approach. The documents should be appropriately linked so that readers can choose to access more or less information.

The SBD for drugs should be significantly shorter and focus on the clinical basis for the decision (notably safety and efficacy) as well as Health Canada's assessment of the risks and benefits of the product. Sections on the quality (with the exception of the assessment of adventitious agents), non-clinical, and clinical pharmacodynamics and pharmacokinetics portions of the submission should be replaced by brief, high-level statements assessing the adequacy of information filed. These sections should, however, highlight significant safety concerns with these portions of the submission that may be relevant to the clinical use of the product, as well as precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of these sections. The section on Risk/Benefit Assessment and Recommendation should be expanded and become more of the focus of the SBD.

The SBD for medical devices should also be shorter and again should focus on Health Canada's assessment of the risks and benefits of the product, primarily the clinical safety and effectiveness components of the application. Sections related to standards referenced, sterilization, manufacturing and quality control, preclinical studies, and software validation studies should be replaced by general high-level statements assessing the adequacy of information filed. These sections should, however, highlight significant safety concerns with these portions of the application that may be relevant to the clinical use of the product, as well as precautions, warnings, or contraindications that were added to the labelling as a result of the assessment of these sections. The sections on Risk/Benefit Assessment and Recommendations should be expanded and become more of the focus of the SBD.

4. Communications

In order for the SBD project (or any transparency initiative) to be successful, information about the initiative must be well-communicated, both internally and externally.

4.1. Awareness and Accessibility

Notice of Decision and SBD documents are posted on the Health Canada Web site on the Drugs and Health Products pages (under Biologics, Radiopharmaceuticals and Genetic Therapies, Drug Products, and Medical Devices). Vanity URLs are in place at www.healthcanada.gc.ca/sbd and www.santecanada.gc.ca/smd. The SBD pages are divided into the following topics:

  • "Communications" (includes Notices, Templates, Frequently Asked Questions document, Fact Sheet, and Reader's Guides)
  • "Consultations Workshop" (includes documents related to the consultative workshop held during the project's development in June 2004)
  • "Pilot Exercises" (includes pilot SBDs drafted in 2004 for a drug product, a biologic, and a medical device)
  • "Summary Basis of Decision: Phase I" (includes ND and SBD documents for medical devices and drugs, which includes pharmaceuticals and biologics)

Notice of Decision and SBD documents are split into two topics: medical devices and drugs, which includes pharmaceuticals and biologics. All documents are organized by product name (alphabetical listing).

The assessment of transparency needs of Canadians clearly indicated that the Internet is a growing source of health-related information, and that the Government of Canada (in particular Health Canada) has a significant role to place in the provision of such information to Canadians. Given that the majority of Canadians (over 80%) report having Internet access at home, the focus of the ND/SBD initiative should continue to be the provision of online information. Various components of the SBD evaluation have assessed whether the documents are easily retrievable from their current location.

In the internal survey, approximately 40% of TPD and BGTD respondents indicated they had never looked for the ND/SBD documents online. This is not a surprising result as the documents are not explicitly drafted for internal users, who have access to much more detailed product- and submission/application-related information. Of those TPD and BGTD staff who have looked for them, the majority indicated they were difficult to find, or retrievable more easily using a search engine.

In responses to the evaluation workbook, 42.5% of respondents said ND/SBD documents were easy to find; this is likely indicative of the large percentage of respondents from the drug/device industry, who are more familiar with the Health Canada Web site than the general public. Only 15.0% said they were difficult to find, however an additional 22.5% said they could find them but only via a search engine. Comments made in response to other workbook questions indicated the documents should be more easily accessible (e.g. in more general/public areas of the Health Canada Web site) and that they should be linked using statements explaining the purpose of the documents, not just the title. Other respondents indicated that better links are needed between the ND/SBD documents, the NOC database, and the DPD online (particularly the PMs).

While questions about SBD-related communications were not specifically asked in the evaluation workbook, the low number (40) of responses received is a good indication that the external audience may not be aware of the documents. It was recognized at the outset of the evaluation that the so-called 'general public' would be the most difficult audience to contact and evaluate; in fact, a significant portion of respondents were from the drug/device industry, particularly those involved in ND/SBD production. Several respondents to the workbook expressed concern that the 'general public' is likely not aware of the existence of the project or the documents.

As phase I has been implemented, there has been growing concern internally that the scope of phase I may itself limit the awareness of the documents. While it is recognized that it was necessary to implement on a small scale initially, the fact that documents are only available for a small subset of marketed drugs and medical devices may itself have decreased the visibility of the project. If future phases result in an expansion of the scope of products and submissions/applications that are eligible for NDs/SBDs, this itself may make Canadians more aware of the documents. This will of course be assisted by making the documents more useful to Canadians, and more easily accessible.
Several internal interviewees indicated the Health Canada Web site is not user-friendly, and that they had received comments to that effect from external stakeholders with respect to SBD documents. There was also concern that potential readers of the documents may not be aware of their existence due to their location on the Health Canada Web site.

Unfortunately, TPD and BGTD are somewhat limited in the possible organizations of the Web site due to Treasury Board and Health Canada's requirements and guidelines, including Common Look and Feel 2.0. However, there may be some flexibility in terms of where the Web page can be placed (i.e. whether the profile can be raised so the documents are more easily retrievable from the Health Canada or the Drugs and Health Products Web pages) and from where it can be linked.

As additional documents are published to the Health Canada Web site, the current alphabetical listing by product will become more cumbersome. This is particularly a concern if, during phase II, documents are drafted for more products or for multiple submissions/applications for a product.

4.1.1. Linkages to and from Summary Basis of Decision Web Pages

Approximately 60% of Canadians have indicated that access to several sources of information or related services in one place contributes to a Web site's ease of use. Currently, SBD-related documents for drugs are also available via the Notice of Compliance (NOC) database at www.nocdatabase.ca. The documents have not been made available via the DPD Online as they are specific to decisions, not products. SBD-related documents for medical devices are available via the Medical Devices Active Licence Listing at www.mdall.ca.

The majority of respondents (25.0%) found out about ND/SBD documents because they are involved in ND/SBD production; 17.5% were made aware when they were sent the invitation or were otherwise asked to complete the workbook. Others (10.0% each) found the documents while searching for information about a drug or medical device, or by following a link on another Health Canada Web page. Improvements are needed to increase Canadians' ability to easily locate and retrieve the documents from the Health Canada Web site.

4.2. Web Hits on Summary Basis of Decision Web Pages

Web hits between January 1, 2007 and September 30, 2008 were analysed (data was not available for 2005 or 2006). It should be noted that it was not possible to distinguish between Web hits from computers internal vs. external to Health Canada.

The SBD home page was the most viewed page of all SBD-related pages, with 18,344 visits in the 21 month period reported, ranking it 87th among all Web pages in the Drugs and Health Products profile. The index of available NDs/SBDs for drugs (which includes pharmaceuticals and biologics) was the second-most viewed page, with 8,233 visits and a ranking of 224 among all Drugs and Health Products Web pages. The index of available NDs/SBDs for medical devices was the 11th most viewed SBD-related page, with 3,415 visits and a ranking of 730 among all Drugs and Health Products Web pages.

A total of 74 products had at least one document (ND or SBD, English or French) that received over 500 Web hits; 189 distinct documents were viewed over 500 times, out of a total of 394 ND/SBD documents (English or French) available on the Web site for viewing by September 30, 2008. The most popular documents visited were for the vaccine Gardasil (the English and French NDs and the English SBD ranked among the top 9 SBD-related sites visited), with a total of almost 17,000 Web hits for those three pages.

4.3. Internal Communications

The internal survey distributed to all TPD and BGTD staff explored the issue of internal communications, asking respondents how informed they considered themselves to be and what additional information could be provided. While some respondents did indicate they were well-informed, approximately one-third stated they received no communication about NDs, SBDs, or the SBD project. Those who are informed receive information from their manager, their project manager, or from information/training sessions provided by the SBD Unit.

When asked what additional information they would like to receive about NDs and/or SBDs, the most popular response was a desire to receive e-mail notification when an ND or SBD is posted. Others requested guidance on the location of NDs and SBDs on the Web site and on roles and responsibilities of those involved in the ND/SBD process.

4.4. Communications to Industry

When the SBD project was initiated, many submission sponsors and/or medical device manufacturers were provided with a copy of the internal SOP for ND/SBD production; this is no longer standard practice. In addition, several information sessions were given by the SBD Unit, OBT in conjunction with the Canadian Association for Pharmaceutical Regulatory Affairs; however, it is recognized that not all companies with which TPD and BGTD are involved had access to these sessions.

Drug submission sponsors and medical device manufacturers were asked what additional information they might wish to receive about NDs, SBDs, and the SBD project. Only 20% stated they had sufficient information; the majority indicated they would benefit from procedures and/or timelines, information sessions, and a contact person to whom questions could be directed. These responses were re-iterated by workbook respondents from the drug/device industry.

4.5. Recommendations and Findings

4.5.1. Short-Term Recommendations

A regular listing of ND/SBD postings should be sent by e-mail, either to the area (bureau/office/centre) involved, or to the entire Directorate. In TPD, this could be combined with the weekly Good Review Practices News e-mails.

The SOPs should be widely communicated within TPD and BGTD. In TPD, they should be posted on the Good Review Practices Intranet site.

The information available to drug submission sponsors and medical device manufacturers should be improved. Information sessions could be held by the SBD Unit (perhaps in conjunction with industry associations), more information (such as SOPs) should be posted on the Health Canada Web site, and companies should be encouraged to contact their project managers and/or the SBD Unit for information. Until the SOPs are posted, the relevant SOP should be sent to individual sponsors/manufacturers with the draft ND.

4.5.2. Findings to Inform the Development of Phase II

During the development of Phase II, options to improve the visibility of SBD-related documents on the Health Canada Web site should be explored. This includes exploring potential linkages with the DPD Online, as well as potentially raising the profile of the SBD pages on the Health Products portions of the Web site. Linking the documents via plain-language explanations of their purpose should be considered (e.g. prefaced by statements such as 'If you'd like to know...').

Linkages should also be established between post-market Web pages and SBD-related pages. For example, if Health Canada has published a risk communication or recall regarding a product, this should be linked from the SBD Web page (and vice versa).

Consideration should also be given to a more user-friendly organization of SBD-related documents, particularly if documents are drafted for more products or for multiple submissions/applications for a product.

Once phase II has been developed and implemented, promotion activities should be undertaken to promote SBD-related documents and expand audience awareness. This could include partnering with healthcare professional and patient associations (e.g. the Canadian Pharmacists Association); media announcements; newspaper advertisements; brochures in healthcare practices, etc.

5. Posting of Information with Notice of Decision and Summary Basis of Decision Documents

Summary Basis of Decision documents are currently accompanied by Part III of the PM (Consumer Information) (for drugs) or the Operator's Manual (OM)/Package Insert (PI) (for medical devices). This was originally designed so that NDs and SBDs were complemented by information in lay language about the product. Only the PM (or OM/PI) that was approved at the time of product authorization is posted, consistent with the approach to the NDs/SBDs: information is provided to reflect the time of product authorization and not post-authorization activity.

In addition, as described in Section 4 of this report, other documents are posted under the "Communications" topic on the SBD Web page. These include a Frequently Asked Questions document, a Fact Sheet (both about the SBD project) and Reader's Guides that are intended to describe the information found in each section of an SBD for drugs and medical devices.

5.1. Posting of Product Monograph Part III or Operator's Manual/Package Insert

The SBD initiative was developed to address a gap in information available from the regulator about authorized products. However, the availability of information related to drugs has changed significantly since the SBD project was developed in 2004, and information needs of the Canadian public may have changed as well.

In 2008, Health Canada began publishing on its Web site full PMs (authorized since 2004) for marketed products. Product Monographs are authored by the drug submission sponsor (in accordance with Health Canada guidelines), and provide additional information related to the product. However, as outlined in the assessment of transparency needs of Canadians, the public tends to distrust information provided by drug and medical device companies, while information provided by government Web sites is seen as highly trustworthy. Also of note, there is no similar information posted for medical devices.

Several components of the evaluation questioned whether the posting of PMs/OMs/PIs should be continued, particularly given the availability of current PMs on the Health Canada Web site for drugs. The internal survey did not result in a clear majority, although more respondents supported posting the information for medical devices than for drugs. The majority of survey respondents said no additional information should accompany the ND/SBD posting, though a few requested the full PM be posted (not just Part III). As discussed elsewhere, several suggested linking to other Health Canada databases, warnings/advisories, and/or recalls.

Respondents to the evaluation workbook also had mixed reactions to similar questions. While 55% of respondents said Part III of the PM should be posted with SBDs for drugs (pointing to the benefits of having the information available and easily accessible), it is not clear whether respondents understood that this information is likely no longer current when posted. Some respondents (22.5%) said this information should not be posted, as there is no reason to post out-of-date information when the current PM is available online. Several respondents suggested linking to the most current PM within the SBD, or having better links between the SBD, NOC database, and DPD Online.

The workbook responses were also much more supportive of posting the OM/PI for medical devices, with a large majority of those who responded to the question indicating this information should continue to be posted. Again, however, it is not clear whether respondents understood that this information is likely no longer current when posted - only one respondent indicated there should be a process to post revised OMs/PIs.

The posting of information to accompany medical device SBDs has become more complicated recently. Several Instructions for Use documents have recently been provided by medical device manufacturers in response to requests for OMs/PIs. Unfortunately, due to the nature of the medical devices involved, the IFUs are extremely lengthy, complex documents with many graphics. Due to the Government of Canada's Common Look and Feel 2.0 requirements, the SBD Unit has been unable to have these documents posted. In addition, it is questionable whether publishing these very complex documents would meet the original goal of publishing PM Part III and OMs/PIs: to make information available to consumers in lay language.

5.2. Posting of Additional Information

As described above, several communications products are posted on the SBD Web pages. These include a Frequently Asked Questions document, a Fact Sheet (both about the SBD project) and Reader's Guides that are intended to describe the information found in each section of an SBD for drugs and medical devices.

The evaluation workbook specifically asked whether respondents had read the Reader's Guides, and whether they were useful. A total of 42.5% of respondents indicated they had read the Reader's Guides. Of those, 25% said they were useful, specifying the language is easily understood and the documents provide a good overview of the process and an introduction to the SBD content. Others who indicated they were not useful stated they were already familiar with the information but that they might be useful to patients or healthcare professionals.

The evaluation workbook specifically asked whether any additional information should accompany the ND/SBD posting for a drug, and for a medical device. While the majority of respondents said no or did not respond, a few indicated study publications and post-marketing surveillance plans should be posted. As mentioned elsewhere and consistent to internal components of the evaluation, others stressed that linkages should be provided between NDs/SBDs and databases and risk communications posted elsewhere on the Health Canada Web site.

Evaluation workbook respondents were specifically asked what additional information would help in understanding ND/SBD documents. In response, the vast majority (total 80%) did not respond or said 'none'. The small number of suggestions received requested an overview of phases of drug development, data published in peer-reviewed journals or on Web sites, and expanding the Reader's Guides to explain SBD terminology such as 'what is an impurity'.

The Foreword of each SBD document refers readers not only to the Reader's Guides but also to existing Fact Sheets on the review of drugs and medical devices in Canada and relevant policies and guidance documents. However, these links should be included on the SBD Web pages.

5.3. Recommendations and Findings

5.3.1. Short-Term Recommendations

For medical devices, OMs/PIs should continue to be posted, but complex lengthy IFUs should not be published when provided.

The SOPs for the publication of NDs/SBDs for the three product lines (pharmaceuticals, biologics, and medical devices) should be posted on the SBD Web pages.

Portions of the text of the SBD Foreword should be repeated on the SBD Web page, to provide readers with direct links to Fact Sheets on the review of drugs and medical devices in Canada, and relevant guidance documents.

5.3.2. Findings to Inform the Development of Phase II

For drugs (pharmaceuticals and biologics), the posting of Part III of the PM with the ND/SBD should be discontinued. Instead, consider ways in which NDs/SBDs for drugs can be linked to the most recent PM for the product, published as part of the DPD.

It is recommended that the area responsible for the regulation of medical devices publish current labelling information for licensed medical devices, similar to the publication of PMs for drugs.

6. Process for Producing Notice of Decision and Summary Basis of Decision Documents

6.1. Current Process

Technical writers responsible for drafting NDs and SBDs are located in the SBD Unit, OBT and draft documents for both TPD and BGTD, for all three product lines (pharmaceuticals, medical devices, and biologics).

The process for drafting and publishing NDs and SBDs varies between the Directorates, and across product lines. Three SOPs (with accompanying checklists) are in place, one for TPD - medical devices, one for TPD - drugs, and one for BGTD. Overall responsibility for the project management of individual NDs/SBDs is different for each of the three areas: Technical writers for TPD-drugs, Business Transformation Officers for TPD-medical devices, and Regulatory Project Officers for BGTD. However, for each product line, technical writers, project managers, reviewers, managers, directors, and the respective director general all have responsibilities to move the individual documents forward to publication.

When reviewing an SBD-eligible drug submission or medical device application, reviewers are asked to clearly identify their comments within the review templates. They are not responsible for drafting the SBD but are asked to identify important information that will be summarized in the SBD by the technical writers.

The technical writer drafts the ND/SBD based on the reviews and the approved product labelling, and the draft is sent to the reviewers for comment. Discussion between the technical writers and the reviewers will result in a revised draft that is sent to the drug submission sponsor or medical device manufacturer for comment. Companies are asked to limit their comments to inaccuracies or potentially proprietary information, though issues of interpretation of data are frequently raised.

The company's comments are assessed by the technical writer with the input of the review team, and a final version is sent to the company for their information. If the sponsor/manufacturer continues to dispute the content of the ND/SBD, they may appeal to the Director General of the respective Directorate. The final version is sent for translation, then for publication on the Health Canada Web site.

6.2. Evaluation of Internal Process

As demonstrated via the internal interviews, most TPD and BGTD staff were satisfied with the current role of each of the groups involved in the internal ND/SBD process (technical writers, project managers, reviewers, review managers). Suggestions were made to tweak the internal process, many of which are described below in Short-Term Recommendations.

While the majority of reviewers interviewed indicated that ND/SBD drafts were of good quality, there were some concerns with the role of technical writers in the ND/SBD process, particularly related to information taken out of context or misinterpreted, how information was summarized, and whether the wide range of medical device technologies could be fully comprehended by technical writers who are not specialists in the individual areas. It was recognized that the quality of the NDs/SBDs is dependent on many things, including the quality of the review report, the skill of the technical writer, and the nature of the product and the submission/application.

These concerns can be mitigated with better communication between the technical writers and reviewers, and good documentation by the reviewers of their assessment of the product within their evaluation reports. Reviewers were generally not supportive of drafting the documents themselves, and they did not feel these concerns were insurmountable.

Reviewers and technical writers alike identified the delay between review completion and SBD drafting as a concern. Often, when the reviewer receives a draft SBD for comment, many months have passed after finalization of that review component. This requires the reviewer to spend time re-familiarizing him- or herself with the file in order to provide comments on the SBD draft, which in turn delays the file even further.

6.3. Evaluation of External Process

The review of documents by the sponsor/manufacturer was assessed via the questionnaire. Most respondents agreed that the timeframes for review of the ND and SBD are reasonable (60% and 55%, respectively). Twenty-five percent and 35% respectively indicated it can be difficult to meet timeframes, especially when the documents are distributed globally within the company for review.

There was general agreement at the internal interviews that sponsors should have the right to comment on the documents, and that sponsors have improved past NDs and SBDs by identifying errors or providing additional clarification. However, interaction between the sponsor and Health Canada should be rather limited (and not, for example, comparable to the negotiations that take place when finalizing the PM) and care should be taken so that the sponsor does not bias the document.

Drug submission sponsors and medical device manufacturers were asked about their issues related to the content of NDs and SBDs for their products, and how they were resolved. They indicated that their comments were normally related to correcting inaccuracies, clarifying information that was not reflective of final review conclusions (e.g. did not include information provided and reviewed via clarifax, etc.), and proprietary information. The majority (70%) of respondents were satisfied with the resolution of their issues, even if not all the changes were accepted by Health Canada.

Responses to the questionnaire demonstrated that sponsors/manufacturers are not consistently being provided with Health Canada's rationale for accepting or rejecting changes suggested by the company; however, when provided, the rationale greatly assists in the resolution of issues. Companies also indicated they would prefer to have direct interaction with either the technical writer or the reviewer to resolve issues.

6.3.1. Appeals by Industry

During the evaluation period (January 1, 2005 to September 30, 2008), four appeals were filed (2% of 197 documents published), all to the Director General of the Therapeutic Products Directorate and all related to SBDs for pharmaceuticals. No appeals were filed for biologics or medical devices. The appeals were filed by four different sponsors, and for two of the sponsors it was their first involvement with the publication of an SBD.

The sponsors for all four appeals requested the clarification or removal of information which was regarded as inaccurate and/or inconsistent with the PM and/or review conclusions. In three of the four cases, most of the sponsor's objections were accepted and revised text was provided by TPD or statements were removed. In the fourth appeal, the company stated that much of the information was confidential under the ATI Act. In this case most of the sponsor's objections were denied in order to maintain the integrity of the document and uphold the purpose of releasing the SBD. In addition, the SBD project is a transparency initiative and the sections of the ATI Act do not apply when no ATI request has been made.

In TPD, SBD-related appeals are administered by the Office of Science (OoS), BPSIP (the group responsible for administering reconsiderations of final decisions issued for drug submissions and medical device applications in TPD). In interviews with OoS staff involved with SBD appeals, all indicated that more issues (particularly those related to inaccuracies of data) should be resolved informally between the sponsor and the reviewer and/or the technical writer, obviating the need for an appeal.

Overall, the appeal process as it is currently administered is not considered to be a good use of TPD or HPFB resources. Most OoS staff indicated that the sponsor should continue to have the right to appeal but that the OoS role should be limited to appeals related to the interpretation of data. This is a direct contradiction of information currently found in the SOP for the publication of NDs and SBDs, which states that issues related to interpretation of data will not be considered. In practice, however, these types of issues are commonly raised by the sponsor and resolved informally by the technical writer and/or the review team, or in appeal.

Appeal decisions are targeted to be issued four weeks after filing by the sponsor/manufacturer. Of the four appeals filed within the evaluation period, none were completed on time; total appeal times for the four were 70, 88, 186, and 237 days. For the latter file, the final SBD was not posted until 380 days after appeal filing due to post-appeal revisions discussed between the sponsor and OoS. Delays were frequently encountered by the OoS due to the nature of the appeals. When appeals dealt with inaccuracies of data, the OoS was required to access the submission volumes and verify the information filed. This is a timely process, and as discussed above is not a good use of TPD or HPFB resources.

6.4. Performance vs. Targets

6.4.1. Evaluation of Performance vs. Current Targets

Each of the various steps of the ND and SBD processes has a target. Targets for the major steps for NDs and SBDs are described in Appendix IV. The overall target for publication of an ND is 6 weeks and for an SBD it is 20 weeks (assuming no appeal is filed).

The performance targets were set when the SBD initiative was launched, based on an estimate of time required. It was recognized at that time that the targets may need to be revised however significant revisions have been put on hold pending the results of this evaluation.

For those steps in the process completed prior to the end of the evaluation period (i.e. before September 30, 2008) it is clear that neither Directorate is meeting targets though there are differences between product lines.

Targets are very close to being met for the majority of medical device NDs and SBDs. The average times for drafting medical device NDs and SBDs need improvement (each is approximately 3 times over target); however the average times for assessment and approval of the draft by reviewers and managers, respectively, are very close to target. The average overall times for publication of NDs and SBDs are 7 weeks and 24 weeks, respectively.

Elsewhere in TPD, targets for drug NDs and SBDs are not being met consistently. Again, average drafting times for both NDs and SBDs are approximately 3 times over target. Average times for the assessment of a draft ND and SBD by reviewers are 5 and 7 times over target respectively, and average times for management approvals of NDs and SBDs are 17 and 22 days respectively. The overall times for publication of NDs and SBDs are approximately double the target (13 weeks and 41 weeks, respectively).

The Branch is also not consistently meeting targets for BGTD NDs and SBDs. Time for drafting NDs and SBDs during the evaluation period was 60 and 13 times over target, respectively, due in part to a prolonged period in 2005/06 where no technical writer was assigned to BGTD NDs/SBDs. Time for assessment of draft documents by reviewers is also delayed (6 and 8 times past target for NDs and SBDs, respectively) but approval times for each are less than in TPD-drugs (10 and 9 days, respectively). The overall average times for publication of NDs and SBDs in BGTD are 23 and 50 weeks, respectively. It should be noted that for products authorized by BGTD in 2007 and 2008, average times for almost all major steps have improved for both NDs and SBDs.

Reasons for the differences in performance across the three product lines are numerous, and range from the nature of the submissions and applications, to the priority placed on the SBD project within the respective areas, to the workload of the SBD Unit.

In TPD-medical devices, applications (and therefore reviews and review targets) are shorter; in addition, fewer products are SBD-eligible. This has made the workload for medical devices easier to manage than in other areas. In addition, the process is well-organized by the Business Transformation Officer who is responsible for overseeing the process for each ND and SBD. While the medical devices involved require the technical writers to familiarize themselves with a wide range of technologies, the SBDs themselves are much shorter on average and faster to draft and review.

In TPD-drugs, the process changed significantly during the evaluation period. Initially, Regulatory Project Managers were responsible for overseeing the process for each ND and SBD; this was changed in Fall 2007 and technical writers took over this responsibility. These responsibilities have added to the workload of the SBD Unit, with no additional resources. While the technical writers appreciate the ability to more closely manage the process, it is not clear that they have had greater success than did the RPMs at obtaining compliance with targets in the review and approval steps.

The SBD Unit has not been consistently fully staffed (at 3 technical writers) throughout the evaluation period. For approximately 20 of the 45 months, the Unit was staffed with only two technical writers instead of three, plus a manager assigned to the project part-time (who did not draft NDs or SBDs but took care of various operational issues). This had a particular impact on BGTD in 2005/06 when no technical writer was assigned to BGTD documents for 6 months. (As mentioned above, the performance for initial drafting steps for BGTD NDs and SBDs has improved dramatically in recent years.)

While the SBD project itself has been confirmed repeatedly as a priority at the Directorate and Branch levels, in truth it is one of many priorities for the review areas. Implementation came at a time when the Branch was very focussed on other very visible TAS initiatives, namely eliminating the backlog of submissions and applications within TPD and BGTD. Although steps have been taken to minimize the impact of the SBD project on reviewers, the nature of the documents and reviews mean that the review population's input is essential. There is intense pressure on review areas to complete reviews on time; SBDs unfortunately are not consistently seen as part of the submission or application but are viewed as extra workload.

In all areas, performance of drug submission sponsors and medical device manufacturers is quite good. While extensions are occasionally requested (and usually granted), most companies are able to provide comments within target.

6.4.2. Evaluation of Performance Targets

As part of the internal survey, participants were asked whether they thought that the performance targets for their steps in the ND/SBD process were reasonable. In both TPD and BGTD, respondents were fairly evenly divided between agreeing the targets were reasonable (43.9% and 47.8%, respectively) and stating the targets were do-able, if not for other priorities (41.5% and 43.5%, respectively). Few respondents from either Directorate indicated the targets were unreasonable.

The interview results were fairly consistent with the survey results. Most interviewees indicated that the targets were reasonable, but were not being met as SBDs are not considered a priority in reviewer workload, because there was too much work required, and/or because of a lack of available reviewers.

Reflective of the multiple priorities faced by the review areas, only 60% of internal interviewees indicated that SBDs are included in their workplan, or considered part of their duties (or part of the review); 40% said it was an added duty.

Several interviewees indicated that the targets for their step should be extended, or suggested that targets be set on a case-by-case basis depending on the file, the workload, and the quality of the draft ND/SBD. However, setting targets on a case-by-case basis is not practical: there would be no expectation internally or externally on when documents would make their way through the various steps of the process, or be published. In addition, it has been found that the sooner the reviewers receive the draft SBD after review completion, the less time it takes them to re-familiarize themselves with the file and provide comments. Delaying review targets would exacerbate the impact of the SBD project on the review community.

The SBD Unit has made a number of changes to the ND/SBD process to reduce the resource impact of NDs and SBDs on reviewers. These include delivering information sessions on the project and the documents, giving advice/training on how reviewers can identify information for inclusion in SBDs (which reduces the work necessary to draft the SBD, and to review the draft), offering regular meetings between review teams and the technical writers, etc. Work continues to implement these changes consistently across TPD and BGTD and the SBD Unit has seen some positive impacts.

Consideration of an extension to timeframes must be balanced by the need to make documents available as soon as possible after product authorization in order that they be useful to Canadians. Transparency is only as effective as it is timely, and documents published (as has been the case) a year or even two years after the product was authorized and marketed in Canada are not meeting either SBD objective: to increase the transparency of review processes and to provide information about authorized products.

Forty percent of respondents to the evaluation workbook indicated that NDs should be available sooner, while 30% stated they were available in an appropriate amount of time. When asked the same question about SBDs, 45% indicated they should be available sooner, while 25% stated they were available in an appropriate amount of time. Thirty percent of respondents to each question did not know as they had not been involved with, or accessed, the documents regularly.

6.5. Recommendations and Findings

6.5.1. Short-Term Recommendations

Direct communication between the technical writer and reviewer should be increased; this includes involving the technical writer earlier in the review process. Work should continue to encourage meetings between the technical writer and the reviewer(s), at a minimum during review (e.g. the first team meeting) and after the reviewer has completed his/her assessment. A meeting should also take place when the first draft of the SBD has been prepared, to discuss the document.

Reviewers should ensure that their evaluation report clearly identifies their comments as well as important information supporting their decision that is to be summarized in the SBD. Reviews should accurately reflect the emphasis placed on certain areas of the evaluation and should include summaries and conclusions as well as a complete benefit/risk analysis. Reviews should also clearly identify where recommendations change as the review progresses (e.g. after the issuance of a clarifax or a request for additional information).

Review managers should be more knowledgeable of and involved in the ND/SBD process. They should recognize and allow the time required for reviewers to work on NDs and SBDs, and include this time in review workplans. In addition, review managers should recognize and acknowledge the importance of transparency, and have a positive attitude towards the SBD project.

Interactions with sponsors/manufacturers on individual documents should be fairly limited. The role of the sponsor vs. Health Canada should be clearly stated as part of the SBD initiative, and it should be clear that NDs/SBDs are Health Canada documents that will document both the positive and negative aspects of the regulator's assessment of the product.

Rationales should be provided to the drug submission sponsor/medical device manufacturer to explain why proposed revisions to an ND/SBD were accepted or rejected by Health Canada. Direct communication between technical writers and sponsors/manufacturers should be encouraged.

Sponsors and manufacturers of SBD-eligible products should be told when their product is authorized that draft NDs/SBDs will be provided for comment. In addition, as draft documents are being approved for sponsor comment, Health Canada should notify the sponsor to allow them to align their resources accordingly so they can more easily meet timelines.

The SBD Unit should continue work to reduce the impact of SBDs on the review population, including delivering information sessions on the project and the documents, giving advice/training on how reviewers can identify information for inclusion in SBDs, offering regular meetings between review teams and the technical writers, etc.

At this time, it is not recommended that targets related to NDs or SBDs be revised. This must be revisited during the development of phase II as a consequence of revisions to the scope of SBD-eligible products and submissions/applications, and changes to ND/SBD documents.

6.5.2. Findings to Inform the Development of Phase II

It is not recommended that the role of technical writer be changed, or that NDs/SBDs be drafted by the review community.

Both Directorates should focus on earlier and more informal methods of dispute resolution for NDs and SBDs, including resolving issues of inaccuracies of data between the reviewer/technical writer and the sponsor within the ND/SBD process. Appeals should be limited to issues of interpretation of data, which will assist in allowing appeals to be resolved within target.

7. Resources

7.1. Translation Costs

As Government of Canada publications, NDs and SBDs are subject to the Official Languages Act and must be published in both English and French. Due to the technical nature of the documents (particularly SBDs), translation is costly. During the evaluation period, approximately $400,000 was spent translating ND and SBD documents (approximate average $107,000/year). Translation services are currently provided by the PWGSC's Translation Bureau at the 'ultra specialized translation' rate (the hourly rate increased annually to $83/hour at the end of the evaluation period).

Translation averages were compiled using data ending September 30, 2008 (consistent with other aspects of the SBD evaluation). Average translation costs are described in Table 2. The variation in average price is a reflection of the length and complexity of the documents: documents for medical devices are generally shorter, while those for biologics are generally both lengthier and more complex.

Table 2: Average translation costs for Notice of Decision and Summary Basis of Decision documents published by TPD and BGTD between January 1, 2005 and September 30, 2008
  Average Translation Cost: Notice of Decision (approximate) Average Translation Cost: Summary Basis of Decision (approximate)
Overall average $285 $3,230
Pharmaceuticals $285 $3,345
Medical Devices $270 $2,445
Biologics $300 $3,620

The SBD Unit has worked with the Translation Bureau to improve the quality of translation received. The SBD Unit also regularly reviews the templates for NDs and SBDs, identifying standard statements that can be used in most SBDs, particularly in the Quality sections where more consistent, general information is presented. These statements have already been translated; the templates are used by technical writers and the Translation Bureau to reduce the cost of translating individual documents.

Consistent with the findings of this evaluation discussed in Section 3 of this report, shorter, more streamlined SBDs would reduce translation costs incurred by HPFB; however, an increase in scope of eligible products and/or submissions/applications may increase costs.

7.2. Salary Costs

It has been very difficult to assess the resources expanded by the review areas in ND/SBD production during the evaluation period (January 1, 2005 to September 30, 2008). Unfortunately, time is not consistently tracked within the HPFB Timetrak system, nor is SBD time consistently attributed to SBDs.

The internal survey and interviews attempted to assess the average amount of time spent by TPD/BGTD staff on an ND and SBD. These averages were used, together with an average number of staff and managers typically involved in a file and average salaries for those positions, to estimate the internal resources spent on the SBD project between January 1, 2005 and September 30, 2008. Estimated salary costs of the areas with major responsibilities in the ND/SBD process (including SBD Unit, project managers, review staff, managers) are outlined in Table 3 below. Note that this does not include costs incurred by the Directors General, or the Publications Unit responsible for publishing the documents on the Web site. As indicated, these are very rough estimates. The actual internal resources used varied widely depending on the review area, the quality of the draft SBD provided, the availability of the original reviewers, and whether the review clearly identified information for the technical writers to summarize and include in the SBD.

Table 3: Estimated salary costs associated with the ND/SBD process during evaluation period
Role in Notice of Decision/Summary Basis of Decision Process Estimated Total Salary Costs for Therapeutic Products Directorate + Biologic and Genetic Therapies Directorate
(Jan. 1/05 - Sept. 30/08)
Estimated Average Salary Cost per Year
(Jan. 1/05 - Sept. 30/08)
Summary Basis of Decision Unit $878,000 $234,000
Project Managers $45,000 $12,000
Review1 $300,000 $80,000
Managers2 $60,000 $16,000
Total $1,283,000 $342,000

1 Total review costs for the Therapeutic Products Directorate are estimated at $110,000 and for the Biologic and Genetic Therapies Directorate at $90,000.
2 Total management costs for both the Therapeutic Products Directorate and the Biologic and Genetic Therapies Directorate are estimated at $30,000.

The SBD unit continues to implement measures described under Section 6 of this report to reduce the review population's workload associated with SBDs, and reduce the salary costs involved.

While findings related to SBD content described in Section 3 of this report would reduce the workload of reviewers for individual documents, the expansion of scope discussed in the same section would increase the number of documents being published.

7.3. Costs to Industry

Resources expended by drug submission sponsors and medical device manufacturers were assessed via the external questionnaire. Very different processes are followed by different companies, depending on their size and structure. Usually, a team approach is used for the review of documents. In some companies, this approach can include a multi-disciplinary team with scientific, legal, intellectual property, and marketing experts, as well as management. Some companies distribute the documents locally and others globally.

Most industry respondents take 0.5-5 days to review an ND and 4-12 days to review an SBD. Most agreed that the timeframes for review of NDs and SBDs are reasonable, though 25% and 35% indicated they can be difficult to meet for NDs and SBDs respectively, especially if their company uses a global approach to review the documents. Some industry respondents indicated that if they were given advance notice that the ND/SBD would be sent for comment, they could better prepare and more easily meet targets.

Of note, internal experience with NDs/SBDs has shown that, for most NDs/SBDs, industry is able to provide comments within the timeframes requested; requests for extensions are the exception rather than the norm. However, this is no indication of the burden on industry to meet such targets.

7.4. Recommendations and Findings

7.4.1. Short-Term Recommendations

The SBD Unit should continue to regularly review the ND/SBD templates to update standard (already-translated) statements and continue to reduce translation costs.

Senior management within TPD, BGTD, and indeed HPFB, should re-iterate their commitment to transparency as a Departmental and Branch priority. This should be accompanied by increased scrutiny on areas consistently not meeting ND/SBD targets. On-going efforts to reduce the impact of SBDs on reviewers should continue.

TPD and BGTD staff should be asked to track time against an SBD-specific code in Timetrak with each draft ND/SBD sent by the technical writers.

As draft documents are being approved for sponsor comment, Health Canada should notify the sponsor to allow them to align their resources accordingly so they can more easily meet timelines.

7.4.2. Findings to Inform the Development of Phase II

Findings of this evaluation, discussed in Section 3 of this report, to streamline the ND/SBD documents would reduce the length and complexity of the documents, therefore reducing average translation costs. This would also reduce the review community's workload associated with reviewing individual documents.

Findings of this evaluation related to increasing the scope of products and submissions/applications eligible for SBDs should be examined in terms of their operational feasibility. This should include an assessment of the impact of proposed changes on technical writer staffing needs, the workload of the review, publications and Departmental Web communications communities, and translation costs associated with an increased number of documents.

Appendix I - Summary Basis of Decision Guiding Principles

SBDs will:

  • Be written for all Canadians interested in the basis for Health Canada regulatory decisions on drugs and medical devices;
  • Be a factual and objective presentation of the scientific and regulatory basis for the decision;
  • Be an accurate reflection of the evaluation reports;
  • Be clear and concise;
  • Complement, not duplicate, the Product Monograph/Operator's Manual;
  • Be available in both official languages;
  • Be easily retrievable and available in a timely manner;
  • Be of standardized format and use standardized wording whenever possible;
  • Not be resource intensive for reviewers;
  • Disclose as much information as possible, respecting the boundaries of what is currently considered to be proprietary.

Appendix II - Current Table of Contents for Summary Basis of Decision Documents for Drugs (Pharmaceuticals or Biologics)

  1. Product and Submission Information
  2. Notice of Decision
  3. Scientific and Regulatory Basis for Decision
    • 3.1 Quality Basis for Decision
      • 3.1.1 Drug Substance (Medicinal Ingredient)
      • 3.1.2 Drug Product
      • 3.1.3 Facilities and Equipment
      • 3.1.4 Adventitious Agents Safety Evaluation
      • 3.1.5 Conclusion
    • 3.2 Non-clinical Basis for Decision
      • 3.2.1 Pharmacodynamics
      • 3.2.2 Pharmacokinetics
      • 3.2.3 Toxicology
      • 3.2.4 Summary and Conclusion
    • 3.3 Clinical Basis for Decision
      • 3.3.1 Human Pharmacology
      • 3.3.2 Pharmacodynamics
      • 3.3.3 Pharmacokinetics
      • 3.3.4 Clinical Efficacy
      • 3.3.5 Clinical Safety
      • 3.3.6 Additional Issues
    • 3.4 Benefit/Risk Assessment and Recommendation
      • 3.4.1 Benefit/Risk Assessment
      • 3.4.2 Recommendation
  4. Submission Milestones

Appendix III - Current Table of Contents for Summary Basis of Decision Documents for Medical Devices

  1. Product and Submission Information
  2. Notice of Decision
  3. Scientific and Regulatory Basis for Decision
    • 3.1. Introduction
    • 3.2. Device-Specific Detailed Information
    • 3.3. Devices Containing Biological Material
    • 3.4. Safety and Effectiveness
      • 3.4.1. List of Standards
      • 3.4.2. Method of Sterilization
      • 3.4.3. Manufacturing and Quality Control
      • 3.4.4. Preclinical Studies
      • 3.4.5. Clinical Effectiveness and Safety
      • 3.4.6. Software Validation Studies
      • 3.4.7. Labelling
    • 3.5. Benefit/Risk Assessment
    • 3.6. Recommendation
  4. Application Milestones

Appendix IV - Performance Targets for Major Steps in Notice of Decision and Summary Basis of Decision Processes

Major Step in Process* Target: Notice of Decision
(business days)
Target: Summary Basis of Decision
(business days)
Draft document 2 15
Peer-review of document 1 3
Reviewers comment on document 3 10
Company comments on document 5 15
Translation Bureau translates document 3 10
Total target 6 weeks 20 weeks

* Note many steps such as revising, formatting, publishing, and obtaining management approvals for documents are not delineated but are captured in total target.