GxP professionals understand the need for quality and quality system and we discuss quality with one another on a daily basis. But how do we measure it? How do we quantify our results? Once again we turn to Len Grunbaum and Emma Barsky, regular contributors to GxP Perspectives, for their insight on how to quantify quality for the development manufacture, and distribution of health products such as drugs, medical devices, and biologics.
Quantifying Quality
In its simplest form, the definition of “quality” is “how good something is.” But what exactly does this mean for the life science industry, whose frame of reference is defined by regulations which are often vague and which provide little or no guidance regarding how they should be implemented?
In light of this, we would like to offer some ideas regarding how to measure – quantify – how good your “quality” is in tangible and practical terms. We contend that such metrics are useful in order for company management to make sound decisions regarding whether and/or where the quality system (i.e., the operational infrastructure that promotes and facilitates “quality”) requires improvement. The following key indicators are not all-inclusive (nor are the items mutually exclusive), but they provide meaningful ways to assess your “quality”:
• Number of successful external and internal audits as a percentage of the total number of external and internal audits: the higher the percentage of successful external audits (e.g., by existing/potential clients, regulators), especially when you have a large number of them, the better your “quality.” Passing one audit with flying colors is great but passing multiple audits with few minor or no observations is way better. It not only sets a trend regarding “legitimate” quality but it also validates the company’s degree of quality from different perspectives. This scenario allows any company to claim that its quality system has withstood scrutiny from a variety of companies and/or regulatory agencies over a long period of time.
While “looking good” to the outsiders is great, “feeling good” about what is under the covers is even better. Therefore, if thorough internal audits do not find any issues that either directly (critical observation) or indirectly (major observation) impact subject/consumer safety and/or data/product integrity, then “quality” is inherent to the operations.
You may wonder how a subjective term like “success” is defined in this context. Fair question. A result of “no audit findings” (e.g., no FDA 483s, no audit observations) is the clearest measure of success. A relative handful of “minor/cosmetic” issues is not perfect but is certainly acceptable in this context. To the extent that the number of observations may be “critical” or “major,” as defined above, the audit will certainly be viewed as less successful or even unsuccessful. One should also remember that it is a common thing in the industry to consider a large number of minor observations as a major issue because this scenario gives an impression of a negative trend, the latter of which is not conducive to having quality operations.• Number of “directed” (i.e., “for cause”) audits as a percentage of total audits: because directed audits are performed to follow up on actual or perceived regulatory compliance problems, the higher the number of “directed” audits, the more questions will be raised about your “quality.” “Directed” audits could be external (i.e., performed by existing clients or regulatory authorities) or internal (i.e., performed by internal quality staff). The higher the number of problems confirmed, the weaker the quality system. Even if these types of audits indicate in general that there are no actual problems, or a minimal number of problems, a large number of such audits should prompt questions regarding why the perception exists that the degree of “quality” is such that an investigation is required.
• Number of investigations/CAPAs: an investigation is a formal and documented process performed to gather information (e.g., root cause, impact) regarding a specific problem encountered (e.g., a customer complaint, a missing controlled document) and which, depending on the outcome of the investigation, may lead to corrective and preventive actions. An “excessive” number (the definition of which is admittedly subjective in nature) of investigations, even if satisfactorily completed and closed, gives an impression that the underlying cause has never been properly identified and/or corrected.• Number of repeating issues as a percentage of the number of audits performed: repeating issues are symptomatic of a quality system that does not correct or otherwise effectively address problems. While isolated incidents are not necessarily a reflection on the company’s overall quality, incidents that span multiple project teams and/or departments and/or are observed more than once may be indicative of quality-related problems. It is very difficult to convince anyone of the quality of operations when problems that are systemic in nature become evident.
The higher the percentage of audits that contain repeating issues, the more likely that this may be viewed as 1) management indifference, 2) lack of management involvement, 3) inappropriateness of personnel qualifications and/or 4) inability/unwillingness to invest in “quality.”
• Number of business opportunities lost due to unsuccessful external audits as a percentage of the number of external audits: audits are sometimes performed as a basis for determining whether a business relationship should be consummated or continued (e.g., you will be chosen as a vendor/supplier, an existing relationship will be sustained) or expanded (e.g., a company will be awarded additional projects). Some life science companies (e.g. pharma, biotech) have to get clearance from the FDA prior to being able to market their product. Support companies (e.g., CROs, contract manufacturers) may have to undergo due diligence inspections to establish/maintain/enhance a business relationship. The higher the percentage of such opportunities lost (e.g., loss of a potential or existing client, project cancelled/not awarded, FDA did not grant an approval) because of poor audit results, as a percentage of external audits performed, the stronger the indication that your “quality” is dangerously weak. This, in turn, has a financial “bottom line” impact on the company: loss of business opportunities can also be translated into wasted R&D cost and/or lost anticipated revenue, both of which become a major risk to the company’s financial health.In addition to the items listed above, there is another important quantifiable component to “quality,” which is too often being overlooked or not being considered at all. This component is what we define as “the monetary expenditure associated with ‘quality.’” Namely, we are talking about an operationally quantifiable parameter – cost of establishing and maintaining “quality” operations. Most will argue that “quality” is very expensive no matter what. We firmly believe that it does not have to be that way if the underlying causes, which directly and unnecessarily contribute to the extra cost of doing business, are either eliminated or minimized. Here are a few examples to give you a flavor of what can contribute to increased costs when it comes to meeting the regulatory responsibility of instituting and sustaining “quality”:
• Regulatory compliance decisions that are not defined in writing and/or are not defensible.
• Cumbersome and inflexible procedures that require more resources than necessary to execute them without “procedural deviations.”
• Inefficient procedures that require the same activity to be done more than once in order to be in compliance.
• Ineffective procedures that do not reach the desired objective of being in compliance after the first execution.
• Unclear procedures that result in too many on-going corrections in order to inject “quality” into operations.
• Too many procedures that company staff must follow without any value added.
• Contradictory procedures that lead to generating Notes-To-File, CAPAs, deviations, investigations, etc. because compliance to one procedure results in non-compliance with one or more other procedures.
The above-listed activities not only translate into the need to spend more time and money in an attempt to have operational quality, but a number of these items translate into further quality-related costs to the company. Examples of the latter include, but are not necessarily limited to, taking the time to respond to observations or even worse yet, an FDA-483 or a Warning Letter. We think the point we are trying to make is clear…Our bottom line is that you can make both your QA and CFO happy by quantifying “quality” in terms that will be understood and appreciated by both. This means that sound decisions can be made regarding whether and/or where to apply precious company time and resources help ensure that your “quality” is as good as it can be without putting the business out of business.
Emma Barsky and Len Grunbaum
Partners of The Practical Solutions Group, LLC
609.683.0756
Practical Solutions
====
In News from FDA: Yet another weight loss danger in Japanese “rapid weight loss” pills. Read the story: foodconsumer.org
====
Join the GxP Perspectives Linkedin Group Here
Or get an email subscription (on the right sidebar)
====





Posted by GxP Perspectives 




































GxP Quality- Building a Culture of Compliance
January 24, 2011Building a Culture of Compliance
This will be the last post for two weeks as I go traveling for a bit. I wanted to let readers know of two important recent articles (with links at the bottom of the post.)
The first is about a report issued earlier this month by the British Academy of Medical Sciences. The report was commissioned by the British government to try to pare back some of the bureaucracy involved with clinical trials in the U.K. An article on the report by Nick Taylor in Outsourcing-Pharma there are some interesting revelations about the British regulatory agency, MHRA. I strongly recommend reading this article.
In addition, I would like to refer readers to last week’s FDA Matters for interesting issues of concern at FDA. Now, here are Len & Emma:
Guest Commentary by Emma Barsky & Len Grunbaum
Implementing a “Culture of Compliance”: Practical Steps
Establishing a “culture of compliance” is not a “paint-by-numbers” exercise; it must be injected into the DNA of a company. In our experience, many companies strive to create a “culture of compliance” but few approach it from the perspective of a mindset of “shared attitudes, values, goals and practices that characterizes an institution or organization,” which is the essence of a “culture.” The ideas that follow are intended to provide a roadmap and practical steps towards implementing a company-wide “culture of compliance” in the life science industry, across any GXP area.
1. Develop a quality policy statement and quality objectives
Developing a Quality Statement
2. Identify and document quality criteria
Company management should identify and document the criteria for “good quality practices.” These components may include, but may not necessarily be limited to, adherence to written policies and procedures, exercising good documentation practices (e.g. initialing and dating cross-outs, using a single line to for corrections), promptly bringing any deficiencies and/or deviations to the attention of company management, documenting unplanned deviations, providing explanations/justifications when planned deviations occur and correcting them in a timely fashion.
Defining Expectations for Quality
3. Establish a robust quality baseline
Through performing internal audits and assessing CAPAs, the company’s Quality function (e.g., Quality Assurance) should identify quality-related issues based on the criteria regarding the components of “good quality practices” outlined in item number 2.
While performing the internal audits and through capturing the findings in the CAPA system, the Quality function should focus on identifying trends and themes related to non-compliance. In order to achieve this objective, the company’s CAPA system should be such that it allows the company to collect information regarding appropriate metrics to assess the success of the “culture of compliance” initiative.
Lack of compliance should be tracked not only on an issue-by-issue basis, but also across departments, individuals and operations/processes. Collecting information on these items will help to identify the root cause of the issue, which may stem from a larger issue related, but not necessarily limited to,
1. Cumbersome and, therefore, ineffective processes
2. Lack of appropriate supervision
3. Procedures that lack clarity (e.g., poorly worded documentation; lack of guidance, contradictory information)
4. Lack of documented procedures
5. Lack of an individual’s attention to detail and common sense
6. Lack of effective training
The list of potential root causes of the deficiencies encountered is limitless, but one thing always remains clear: unless quality-related issues are addressed and remedied at the root cause level, the fix to compliance issues will not be permanent, nor will the company be able to create and maintain the “culture of compliance.”
4. Maintain a robust quality baseline
Every internal audit should focus on the effectiveness of the established CAPA system. Specifically, the Quality function should determine whether:
1. The nature and impact of the deficiency on data has been properly identified
2. Effective corrective and preventive actions have been implemented
3. Appropriate follow-ups have been performed
4. The system is robust enough to trend and track quality-related issues accurately and completely
5. The system is robust enough to identify weaknesses with processes, documentation practices, personnel, etc.
The company’s Quality function should investigate all instances where the above has not been achieved for improvements within the quality system. This includes the defined metrics of the CAPA system.
In conclusion, we would like to emphasize that many factors, including but not limited to, new staff, acquisitions/mergers, new lines of business, staff reductions and/or new management have an impact on the “culture of compliance.” Therefore, in order to be successful, a “culture of compliance” should be a “living” initiative, which is constantly assessed for its effectiveness in light of changes that every company experiences during the normal course of events.
Emma Barsky
Len Grunbaum
Partners
The Practical Solutions Group, LLC
=============
Read about the Academy of Medical Research Report by Nick Taylor in Outsourcing-Pharma
UPDATE: There is a very interesting Important Notice to IRBs that is on the FDA website. Sort of a Coast IRB redux.
=============
Please join GxP Perspectives on LinkedIn at:
GxP Perspectives LinkedIn Group
See the page Guest Commentaries (at the top of the blog) for the previous article by Len & Emma on Part 11 compliance.
On FDA’s Website there are Two New Warning Letters from FDA to Clinical Investigators that show the need to effectively respond to a Form FDA 483, Inspectional Observations, with a well thought out CAPA Plan.
Share this:
Like this: