To supplement the Qualio Validation Overview and Executing Qualio Validation articles, we’ve compiled a list of common questions and answers because we understand there is a lot to process and juggle during the validation process. If you don’t find what you are looking for after reviewing available articles, please reach out to our Support Team or your Customer Onboarding Manager; we are happy to assist.
This article is divided into sections to help you find the question and answer you are looking for:
New Validation Method
If Qualio tests the software as part of its own quality management activities, does that mean I don’t need to test it myself as part of our validation activities?
Qualio is a web-based, multi-tenant system that runs identically for all customers. It would therefore only be a waste of your time to repeat the functional tests already performed and documented by Qualio with tools like CircleCI and GitHub.
The only testing required by you comes from the specific system configurations you build into Qualio, and how they might impact your business processes and, by extension, the safety of your products and patients. Regulators like the FDA encourage a critical-thinking and risk-based approach to testing, so it’s appropriate to the business-specific risks you identify, and isn’t overly excessive or burdensome.
Why does Qualio require less rigorous assurance activity than other systems?
An eQMS is an example of low-risk, non-product software with no direct impact on patient safety, product quality, or the integrity of the data underpinning these areas.
Your auditors will expect your assurance activity to be focused proportionately based on the systems you’re using, and an eQMS naturally demands less rigorous assurance than a high-risk system for these reasons.
A Qualio competitor is offering IQ, OQ and PQ documentation as part of their validation approach. Why aren’t you? Are you less compliant?
No. Any modern software vendor providing IQ, OQ and PQ documentation as standard is simply sticking de rigeur to 20-year-old outdated validation practice. Both FDA CSA and ISPE GAMP guidance have now diverged from this approach, understanding that ‘linear’ documentation like IQs, OQs and PQs doesn’t reflect the non-linear, agile nature of modern software development.
Simply put: a vendor pushing this approach is only interested in appeasing customers worried about ticking old boxes – and not interested in offering a modern, streamlined and least burdensome approach for their customers.
Why is your process better than the old methodology of IQs, OQs and PQs?
The foundation of modern CSA is demonstrating that the system meets your requirements for intended use with a minimum of burden and effort. With that as the basis, you’re proving that the system meets the requirements defined during its inception and in subsequent updates.
If we map the traceability of requirements to functions to testing (which is what we used to do with IQs/OQs/PQs), we have proven our system is suitable for use leveraging our software development lifecycle process.
From there, you then configure the system to meet your needs and manage the configuration. Our partnership as your eQMS vendor ensures we are managing the system for you as the experts in QMS software.
Ditching time-heavy documents like IQs, OQs and PQs saves you time and lets you start extracting value from Qualio more quickly.
I’ve been used to IQs, OQs and PQs my whole career. Can you give those to me?
Qualio provides test scripts and documentation on requirements for the system, and you can format them as your company and your policies require.
We also provide a template for customers who require performance qualification. For example, customers may want to PQ their complaint workflow in Qualio’s Events area, because this activity can impact patient safety. However, as we’ve seen, customers are responsible for configuring, documenting and testing system workflows themselves.
I don’t believe regulators will accept this approach. How do you know they will?
Not only is the Qualio team highly experienced in modern quality and compliance, we aren’t afraid of listening to industry experts and third parties to keep our approach aligned with the latest trends and expectations.
We asked Sion Wyn, editor of GAMP 5, FDA advisor and a leading expert on computerized system compliance, to interrogate our software assurance approach and advise us on the direction of travel from the FDA and ISPE.
You can hear Sion’s thoughts on our assurance approach here.
And you can read our breakdown guide of all the threads of modern computerized system compliance, and why we do what we do, here.
How do I defend the validation approach if I accept your validation instead of doing it myself?
Risk assessment, plenty of documentation, and confidence in us as your QMS SaaS provider will help you here. You should qualify us as a supplier to your organization, and understand how we manage changes and validation on your behalf. You should understand the software capabilities, and document your “intended use”. If you accept our testing and validation as your own, you should review the process and the validation pack in detail and ensure you understand what we did. You should then have a statement associated with this documentation to clearly state that this meets your intended use of the system.
If you’re using any of the more highly configurable areas of the software (Events, Suppliers, Design Controls), a configuration document should be kept up to date. And lastly, formally documenting that you review our product change log (located within the application directly) on a periodic basis demonstrates that you’re keeping an eye on things, and reviewing with your company’s risk and intended use in mind.
What if we are a new customer and we completed validation recently?
You still have a validated instance of Qualio, but with the recent product updates in October we recommend you review the product updates and evaluate for impact to your organization. New feature evaluation will be ongoing throughout our partnership.
There are requirements in the QSR and ISO 13485 that are functionally executed in Qualio, such as document control and records management. It is these functions that are validated by us, the user. Are you saying that we do not need to do this any longer?
Correct. Customers do not need to additionally test software functionality that we have already tested.
ISO/TR 80002-2:2017 “Medical device software — Part 2: Validation of software for medical device quality systems”, has this guideline been considered by Qualio?
We have reviewed that guidance, yes. We have chosen to align with GAMP, and there is no conflict with that standard and GAMP.
Will you be updating your new "CSA" process to incorporate Part 11 Compliance in your validation package? Regulatory risk should be considered. Specifically ensuring the changes and how they are tracked in the Audit log is considered and confirmed with each update.
Yes, we have always had a part 11 assessment in the pack and we have updated that to improve the traceability into the actual system requirements.
I agree that your application does not have a direct impact on product safety, etc., but its functionality does have an impact on meeting customer imposed policy requirements to meet QSR, ISO 13485 requirements, e.g. stated time to fulfill a specific action.
Yes, that is correct. But this is driven in practice in how you manage your configuration and business processes. Repeating the functional testing for the software does not mitigate process risks, only controlling the processes appropriately does that.
Not a question - We recently had an ISO:13485 audit and we have a disagreement about the approach of software validation with the auditor. He expected us to perform a risk assessment and PQ for every single product update, which as you know happens almost daily with the release of Product Changes. He also did not like there was no 'version' number for the Qualio software.
Thanks for this feedback, it is very helpful. We do release a "version" with every deploy, and that version provides traceability on our side all the way back to the released piece of code.
There is zero value in a "PQ", unless you have determined that the process you're using Qualio to support can somehow cause a patient injury or product quality problem. To better align your assessment process with ours, I would recommend that you include in your re-val SOP, that bug fixes are not changes, they only return the software to currently expected behavior, and the vendor (Qualio) has tested the fix. Since your use of the system is focused on your configuration in your instance, changes that do not affect that configuration do not require any additional action on your part (or maybe just an internal SOP update).
Do you have any feedback with the new validation process from Auditors?
Not yet, still too new. Do keep us posted as you hear from them!
Have all of your Help materials been updated to reflect this new CSA model?
Yes, some are still in the works but for the most part, those materials have been released and can be accessed from the Help Center.
General Validation Questions
How will you support me if I’d like to do the validation myself instead of accepting your tests?
Qualio provides test scripts and documentation on requirements for the system, and you can format them as your company or your policies require.
How do I know I’m on the most current version of the validation?
The Validation Resources article lists all versions of Validation Packs along with publication date. Consult the list and compare against the Effective Date of your Qualio Validation Record.
How often do I need to revalidate?
As individual features are updated, the validation support documentation will be updated and customers will be notified based on our Customer Notification of Product Change policy.
Once notified, customers will need to determine if revalidation is necessary.
It is impossible to prescribe how often revalidation will be required. However, the Qualio Product and Quality teams are cognizant of the resources required to revalidate features and attempt to minimize and condense whenever possible.
What is Multi-Tenancy, and how does it apply to customers?
With the move to cloud computing, the idea of multi-tenancy was born. Multi-tenancy means there is only one application running, and all customers are using it. Customer data, however, is segregated logically in the software instead of physically with hardware; customers can, and should, continue to expect privacy of their data, though it does not physically reside under their control.
Multi-tenancy is how cloud software can be updated quickly and easily without requiring customers to upgrade hardware and mess with software updates. That means users get new features and bugs get fixed quickly, which leads to better product adoption and user satisfaction.
It also means that Qualio performs validation tests prior to releasing new features and any update, lifting the burden from our customers.
What does multi-Tenancy mean for a “sandbox” instance? How is this different from my production environment?
All customer instances, whether “production” or “sandbox” are using the same application and database. The actual data is segregated in software, so they are not “connected” to each other, just like any other customer account. Your sandbox can be validated the exact same way as your “production” instance, and other than any test data you create, the functionality will be exactly the same. This is most useful in developing new templates or workflows, as you can “experiment” in a sandbox without creating any data in your “production” account. You will have to recreate anything you want to use in production, as the instances are not connected.
Having a sandbox gives us the ability to leverage feature flags to turn on new features to test out and also for you to decide how you want to implement new features in your business, and update internal SOPs.
For updates to existing functionality, we will eventually “turn it on” for everyone, and the changes will be available in your “production” instance. Because of the nature of multi-tenancy and SaaS in general, there is not an option to not accept updates to functionality in most cases. Some features are permanently “feature flagged” however, so you can choose not to use those.
What if I’m using the API?
The API changes how data comes into our application, but not the behavior of the application itself. You should document the configuration of the API and any set up of the system on your side that you’ve tied to the API. The responsibility for risk assessment and validation of the API resides with you, the customer. The API artifacts are covered on our side in our automated testing routines.
What about features we don’t use? Do I have to validate those?
We use a SaaS standard approach called “feature flagging” (or an “on/off toggle” for a specific product feature). This means that some updates can happen to areas of the software that you may not use. There is no regulatory expectation that you validate areas of the application that you’re not using, or cannot even see because they’re behind a “flag”. This is important to understand, because you will see new “versions” happening in the product releases tab of our application, but this does not mean you’re no longer in a “validated state”. We make clear the “version” every time we do bug fixes. There is just one “version” of Qualio, and the releases and such you see there are minor in nature (unless we notify you otherwise).
Does the video evidence demonstrate each user role’s ability/permission?
Yes; the User Management feature Test Plan details testing of each role.
Where can I find the current software version number of Qualio?
The version number is available in-app, located in the product changes section. The number in the version is the date it was released. In the example below, it was released on December 22, 2021.
What SQA regulations do you follow related to FDA?
We follow Agile development methodology, use design control principles in development, and GAMP guidance for validation.
Have you identified any residual risks after testing?
No, the system risks are all mitigated. As the customer, you will incorporate your business risks, as you manage the content and user access for your instance.
To qualify you as a supplier in our QMS, is there a standard package of documents that you provide/make available to your customers?
Yes, we have updated the supplier qualification record, as well as made copies of our ISO certs available through the Help Center links.
My company uses software as part of the process for building patient-specific customized medical devices. We use various software packages (e.g. CAD programs) as part of the production process, but inspect, test and qualify every single product before it ships (as each product is unique). We have never had a non-conformance that was related to any of the software used in our process. What level of scrutiny do you think is appropriate for the software in this context and what pathway should we use to, for example, implement software updates? Thanks!
Sounds like you have it covered. Those tools (such as CAD) should be included in your supplier management program, with the risk assessment of the use of said tools driving the level of control. I have always treated those tools as "Class 2" - or indirect impact to my product - tools, then done a short, documented assessment of their capabilities based on our intended use. If they have direct impact, then I'd qualify more deeply, to include a full process FMEA supporting their use in my manufacturing processes.
How do we get access to the dropbox with qualio’s policy and other system information?
Available in the Help Center
How should I update “Risk Management SOP” and “E-Signature SOP” references?
You'll notice that the Validation Plan and Risk Assessment documents in the validation pack ask for references to specific SOPs. If these SOPs do not yet exist - no need to worry! You can remove the specific document ID callout in the validation pack documents. For example, in the Risk Assessment, you could say "The definitions from the Risk Management SOP are used to categorize the process failure risk."
These procedures do not need to be in place prior to completing the validation and can be developed later.
What should I do with the Validation Pack documents when I'm done?
Ultimately, you should be using Qualio to manage your document stack, including Validation records. These documents should be saved in Qualio and routed through the review/approval workflow to become an effective document. Then Smartlink to this document from the Validation Summary Report. These documents, combined with your audit trail, will serve as your proof of QMS validation. The Executing Qualio Validation article contains more details.
How should I notify the FDA of Electronic Signatures Use?
For 21 CFR Part 11 Compliance 11.100(c), persons using electronic signatures shall, prior to or at the time of such use, certify to the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures. The certification shall be submitted in paper form and signed with a traditional handwritten signature, to the FDA at the:
Office of Regional Operations (HFC-100)
5600 Fishers Lane
Rockville, MD 20857
This link includes two sample letter formats that can be provided to the FDA. This letter is NOT the same notification for signing up on the Electronic Signature Gateway (ESG), but just an informed consent that is established to equivocate the electronic signatures used in Qualio to traditional wet ink signatures of a paper-based system.
Why are you using Dropbox for test evidence and documentation vs. using Qualio and has it been qualified by Qualio as a Supplier?
Dropbox is an approved supplier for Qualio. Dropbox allows Qualio to add new folders for new version releases without needing to update all customer documentation for each change. Because Qualio cannot invite all our customers into our internal systems we are leveraging Dropbox as an easy way to share these records with our customers. Supplier qualification documentation is available upon request.
I have an ISO 13485 audit next week and we are in the process of implementing Qualio. My QUAL-VAL pack is dated 11/2021 and approved 1/2022. I intend to use this VAL pack as evidence for validation of qualio. Will this suffice?
We have released system updates since 1/2022, so I would definitely recommend that you address those updates in your validation records. It can be as simple as reviewing the documents we've provided, stating there is no impact of those changes to your intended use of the software, done!
If I add a new Document Template, Event Template or Training Plan should I update my QMS Platform Configuration Specification record?
We recommend keeping the configuration document current. But this is ultimately a customer’s responsibility and they can determine what triggers an update.
Who is responsible for creating the configuration document in Qualio? My COM or me?
Onboarding customers can leverage their COM to create this document and document a baseline configuration. But who does what onboarding tasks should always be a conversation between customers and their assigned COM.
If I’m not implementing all the features listed on the QMS Configuration template, should I remove those sections or mark it as N/A?
We recommend marking the section as N/A in order to preserve the details for potential future implementation of that feature.
What components/features are “configurable” in the strictest sense of the word? The configuration template seems to include all features.
Suppliers and Design Controls, with their integration abilities and policies that can be built into the system based on end user needs, would be considered configurable features. Nothing in the platform “backend” can be customized for individual customers by the customer or Qualio engineers. Documents, User Management, Training and Events features have been included on the configuration document for customer convenience as some customers have demonstrated an interest in documenting settings and templates.