Qualio is launching its first dedicated Analytics product. The initial product surfaces Events, Documents and Training data in a simple visual display, allowing quality managers to monitor the operational performance of their organizations and quickly identify areas of improvement. Note, future iterations of this will cover Suppliers and Design Controls data. Providing a graphic way to track and present quality operations progress is critical to the success of our customers.
Through Document Comment History, users will gain immediate access to the comment history on any document. This enables users to gain full context of the work completed within a document and provides key oversight of all commentary. This feature will save users time by eliminating manual follow-up with stakeholders.
With Suppliers Permissions capabilities, users can now grant or remove feature permission without contacting Qualio support, giving customers more control and transparency over who can use each part of the system and saving them time.
May ‘23 Launch Train Timeline
EVENT | DATE | DETAILS |
30-Day evaluation period begins. New features added to Sandboxes | May 16, 2023 |
|
Customer Webinar | Jun 7, 2023 | |
Released to Production | June 15, 2023 |
|
Be sure to check out the Q&A session from our customer webinar below.
New Features & Enhancements
Analytics
The Challenge: Many customers have cited their interest in using analytics graphs for management review and other quality status presentations. Without these common charts and graphs, Quality teams are forced to spend valuable time and resources to process data exports and prepare the visuals. Because of these cumbersome efforts, Quality teams often lack data to support their efforts in day-to-day conversations within their organization.
The Solution: We are excited to now offer customers an MVP (Version 1) of the Qualio Analytics dashboard. Quality role users will have access to the NEW Analytics tab from the top navigation bar. They can then select which page to view; for now, an Events page, Documents page and Training page is available, but in the future Suppliers, and Design Controls pages will be available.
Each feature’s page will have multiple sheets displaying different views of the data. The sheets will vary by feature depending on what views and data make sense for that feature. See below for details of the different sheets and graphs available for the first version of Events, Documents and Training Analytics.
Additionally, all graphs are interactive in that you can “drill down” into more precise data based on other criteria like users, status, type, and measurements of time like weeks or quarters. Data can also be exported to CSV.
Validation: This is a new feature addition to Qualio and does not change any existing configuration or functionality. Val-Pack documentation has been updated to reflect this enhancement.
Analytics: Documents
Availability: Documents analytics are available to all customers.
For details of the different Document Analytics pages, expand the pages below for visuals, descriptions and example questions that can be answered.
Status Overview
Status Overview
A breakdown of document statuses across all document versions during a given (yet editable) period of time. Filterable by tags. Get answers to questions like:
At a glance, how many documents are waiting for Approval?
How many documents were updated / approved in the last X months?
How many draft documents exist? Is it time for a clean-up?How many documents were retired last year?
What tags are used the most?
How many ‘Supplier’ tagged documents are effective vs. drafted, for review, and for approval?
How many documents have no tag?
Note: Hidden Groups / tags are currently displayed in Analytics.
Periodic Review
Periodic Review
Due and overdue periodic reviews over time, displayed as a total as well as by owner. Filtered by document type. Get answers to questions like:
Are we keeping up with scheduled periodic reviews?
Currently, how many periodic reviews can we expect for next year?
Are certain team members failing to complete periodic reviews?
Process Efficiency
Process Efficiency
Throughput
Why were 76 document versions reviewed and approved in March, when we usually review and approve less than 40/month? Were we doing some Spring Cleaning, preparing for an audit, etc.? And should we expect this size workload every March?
Cycle Time
Our current average to cycle a document from creation to approval is 26 days. If we implement the proposed changes to the laboratory facilities, I estimate next quarter's average will raise to 45 days with the subsequent quarter to 32 days.
Cycle Time by Duration
In the last 6 months, what document types did we process fastest?
Which document types are taking over 90 days to process? And do we need to consider ways to streamline and make them more efficient?
Cycle Time by User
If it isn't a specific document type taking a long time to process, is it a specific user or users? And if it is, can we adjust workload or reprioritize some of their responsibilities?
Trends
Trends
Trends by Document Versions Created
Our throughput was really high in March. If 76 documents were reviewed and approved, how many were created?
Trends by Document Type
Is this boost in Risk Management Files in March due to the document template revision or something else we need to investigate?
Open Actions
Open Actions
Open Actions by Action Owner
Who has documents to review and or approve right now?
Who is working on draft documents and how many?
Analytics: Events
Availability: Events analytics are available to all customers.
For details of the different Event Analytics pages, expand the pages below for visuals, descriptions and example questions that can be answered.
Status Overview
Status Overview
Overview of events open versus closed, displayed by product and root cause. Additional filters available include date range and event type. Get answers to questions like:
At a glance, how many events are open?
In the past year, what is the distribution of event types?
How many events were opened in X time period?
Which types of events were open in X time period?
Which product is producing the most event activity?
Have all issues related to specific root causes been processed?
What are the main root causes in X time period?
Based on event data, do we have more internal or external issues?
Due and Overdue
Due and Overdue
Events due and overdue, displayed over time as well as by owner. Additional filter: Event type. Get answers to questions like:
Are we seeing a trend in how long events are open? And do we need to investigate why?
Did we see a lag in closing events last year when we made personnel changes?
Do team members need to be retrained on how to manage and close events so that they complete Events on time?
Is an Event Type not working efficiently and needs to be revamped?
Process Efficiency
Process Efficiency
Events Throughput
Displays the number of completed events monthly to compare against the rolling average completion.
Event Cycle Time
Displays the average number of days it took to complete an event when the completion date falls in that month. In the example shown below, notice that January 2023 has an average completion time of 122.5 days, which means some events closed in January were created in September 2022 or earlier.
Event Cycle Time by Duration
Analyze different event types by how long they take to complete.
Event Cycle Time by Owner
An accountability report to display the average completion time of events by event owner.
Trends
Trends
View event data over-time to identify trends based on event creation, events related to products, or events related to root cause. Get answers to questions like:
What trends can we identify based on time, root cause, and product?
Are we seeing a downward trend in the number of events created year over year?
Are there growing problems with a specific product?
Is product XYZ performing better this year based on the number of quality events?
Overtime, do we see more failures based on equipment or personnel?
Open Actions
Open Actions
Accountability report of steps, tasks and events pending action, displayed by owners. Get answers to questions like:
What users aren’t approving Task Steps - holding up the Event process?
Can we / should we redistribute workload?
Analytics: Training
Availability: Training analytics are available to all customers.
For details of the different Training Analytics pages, expand the pages below for visuals, descriptions and example questions that can be answered.
Note: Customers may notice that the training metrics in the Training module (Training Overview) and the metrics in the Analytics module don’t always align for a given period, such as a 6 month period for example. This is because the Analytics data counts months as full months (example: all of January through all of June) whereas the Training module counts months as 30 days each. The Training module metrics/graph will be updated to match the Analytics data in future releases.
Status Overview
Status Overview
An overview of completed, due, and overdue trainings out of the total assigned trainings. It may be filtered by groups or document types.
Do groups with high Overdue percentages correlate to a specific manager or director that we can work with? Maybe attend a team meeting to discuss the importance of quality related training?
Note: Hidden Groups / tags are currently displayed in Analytics.
Percentage Completion by Trainee
Which users have the lowest completion percentages?
Which users need to be retrained on how to complete training?
Due and Overdue
Due and Overdue
Track due and overdue training over time and by trainee. It may be filtered by groups or document types. Get answers to questions like:
How overdue are the overdue training assignments?
How many due and overdue training assignments does a specific user group or trainee have?
Process Efficiency
Process Efficiency
Training Throughput
Displays the monthly rate of training completion to compare against the rolling average completion.
Training Cycle Time
Displays the average number of days it took to complete a training when the completion date falls in that month. In the example below, notice that January 2023 has an average completion time of 35 days, which means many trainings were assigned in December, but because they were completed in January, that is where the data point is displayed.
Did our April social campaign to complete training quickly work?
Cycle Time by Group
Which user groups are currently taking the longest to complete training?
Note: Hidden Groups / tags are currently displayed in Analytics.
Cycle Time by User
Trends
Trends
Training Assigned
Are we assigning too much training?
Trends by Document Type
Trends by Group
Note: Hidden Groups / tags are currently displayed in Analytics.
Document Comments
Comment History
Challenge: While working within a document, users’ comments on the document were not saved after comments were resolved, so users could not access them when reviewing documents. This prevented users from gaining the full context of document changes and approvals.
Solution: During the drafting and review of documents the comments form the conversation around the changes being made. Viewing those comments and threads during approval and even after approval gives context to the approvals as well as to other employee’s thought process behind changes within the content.
Comment History is specific to each document version. Access a version’s Comment History by clicking the More Actions button in the top right corner and selecting Comment History. Comment History can be accessed from Viewing, Suggesting, or Edit Mode. It is available for Quality users and the document owner from the Draft, For Review, For Approval or Effective document status.
Availability: Document Comment History is available to all customers.
Validation: This update will not affect how your system is configured, but it may impact your procedures and processes. Val-Pack documentation has been updated to reflect this enhancement.
Color code
Insertions | Green |
Deletions | Red |
Formatting | Blue |
Comments | Yellow |
User Permissions: Suppliers
Challenge: Within Qualio, users could not adjust user permissions for Supplier workflows without contacting customer support. This created manual work for users and prevented them from having full control over and transparency with Suppliers user permissions.
Solution: With the new user permissions capabilities for Suppliers, Qualio Administrators can now edit Supplier module access for individual users within their company without the need to contact customer support. This enables qualified users to control permissions at the exact moment they need it.
Details:
User permissions management is controlled on a per/user basis from the User Record which can be accessed from the Admin Settings page.
Only Administrators with enabled access to the Suppliers module can grant Supplier permissions to other users within their organization. Qualio Customer Success will grant initial Supplier permissions to selected users during Onboarding.
The Suppliers feature is not available to all customers. Only if a customer’s plan includes Suppliers will the Suppliers user permission be visible to Administrators.
All newly added users, regardless of user role, have no access to Suppliers. Administrators must adjust permissions manually for new users.
When a user changes roles, the Administrators are given the option to change the permissions to the default of the new role or keep the existing custom permissions/deviations in the user’s profile. This does not apply to deleted users.
Changes to a user’s supplier permissions are logged on the Audit Trail.
Availability: Supplier Permissions is available to customers whose plan includes Suppliers.
Validation: This update will not affect how your system is configured, but it may impact your procedures and processes and you will want to evaluate the newly available user permissions. Val-Pack documentation has been updated to reflect this enhancement.
Customer Webinar Recording and Q&A
If you weren’t able to join our customer webinar on June 7, 2023, you can view the recording here, or the slides here. If you have any questions after watching the recording please reach out to our Support team or your Customer Success Manager.
Analytics
Are Analytics only available to Quality users?
Yes, only Quality role users at the moment can view Analytics. Basic and Normal users will not be able to see analytics at this time. How does that work for your company? Let us know (via support chat enhancement suggestion), as we may consider additional permissions in the future.
How do I see the "Reports" and "Analytics" tabs? I do not see them. Will these "appear" automatically in the next release? Is there additional licensing or cost to have these new features? As a Qualio Administrator, how can access to these tabs and the information be shared and/or restricted?
You must be a Quality level user to see the Analytics tab. All Qualio packages/licenses will include the Analytics module for Quality role users. You can currently see these features in your sandbox and they will be added to your Production account on June 15.
If the Qualio User level has the “Quality” Role, will they have unrestricted access to see all details in the “Reports” and “Analytics” tabs?
No. Quality role users will have unrestricted access to the Analytics page. However, we will continue to honor the permissions and access restrictions (i.e. via tags, groups or Admin permission controls) in the modules themselves. So, while the Quality user will see the numbers in the graph, that does not mean that they will have access to the specific documents/events behind those numbers.
Could you maybe please advance a bit the content of the supplier analytics?
We are excited to bring Suppliers Analytics to customers in the coming months; we are planning to release Suppliers Analytics in the next launch train which will kick off in July. You will receive an in-product notification and email alert when Suppliers Analytics is available. You can also sign up to participate in the beta here.
Can we add a filter for “by owner” in the Status Overview section?
We can. I’ll take note of it for future improvements. Thank you!
Will we be able to pull data from previous years, or is it from present data onwards?
Yes, it contains historical data so feel free to adjust the dates to past years.
Can the colors/size of the graphs be modified?
At this time, the color scheme and size are fixed for all customers.
Is there a functionality for the system to send an automated report on a certain frequency (e.g. weekly/monthly)?
We do not yet have this functionality. We will add this request to our product feedback for Analytics.
Any way to click and auto-gen a reminder email to reviewers/approvers when item Due and overdue? Simpler, faster way to reach out to approvers on particular urgent items.
We do not have this functionality at this time but will capture it in our product feedback log for Analytics.
Are the metrics exportable?
You can export the data as CSV. The exported data will reflect your filtering. The data is exported as a table.
Can we print or export the graphs either as a whole or after filtering?
You can currently export the data as CSV. The exported data will reflect your filtering. The data is exported as a table, so you will not see the graphs in the exported data at this time. This is something we’re working to bring to customers in the future.
Can the graphs be exported to Excel or PowerPoint?
You can currently export the data as CSV. The exported data will reflect your filtering. The data is exported as a table, so you will not see the graphs in the exported data at this time. This is something we’re working to bring to customers in the future.
Document Analytics
Are you able to view the number of days it takes a document from Draft to Approved?
We don’t have that right now. Cycle Time shows time from creation to approval but we can consider that for future improvements.
In the future releases, will I be able to click on a bar graph (ie overdue document) for a particular person and I will be able to actually see what the document IDs are overdue?
That is not available at this time, but we will include this request in our product feedback log for Analytics.
Training Analytics
How can we see and show training compliance within a specified time period?
We have date range filters in all the Training Analytics tabs. Let us know if there is anything else needed.
In the Training event dashboard by trainee, you show what is due and overdue. Do you also have the data visual for "complete"?
You may go to the Process Efficiency tab under Training and look for the Throughput metric, which shows you completion activity over time.
Events Analytics
For Events, can we filter by dates?
Yes, you can filter by specific dates with the “Event created” filter on the right hand side of the page.
Is it possible to supplement Events with the option for "Failure Mode"? Or should this just be a "tag"? Similar to root cause, this would allow users to trend against the type of issue that is occurring.
Failure Mode is not currently a data field in our system so we cannot pull its data right now. In our following release (Launch Train 3) we will add Event tag filters, so if Failure Mode is tagged, you may be able to filter for it in the future.
Document Comments Q&A
For the comment history, is it for all documents/events or certain modules?
For documents. Events will be available at a later time as we implement the new editor there.
Are you working on or will be working on the cluttering of comments when there are a lot of comments by multiple reviewers?
This is available today, and comments are grouped into conversations.
When the reviewer has done their review and marked it as completed, we update the reviewer's comments sending the document for review again. The marked review completion is gone. It needs to restart the review with all reviewers that have already marked their review. Would the associated comment content show up in the approved document?
Yes, comment history will be available in the Approved document under More Actions > Comment History.
Will showing all comments show every comment including comments created when the document was in suggesting mode?
Yes, all comments made are included.
How to access yellow comments?
Comment history can be accessed by clicking into More Actions and selecting Comment History
For Audits, sometimes you're required to show the auditor a redline version of the document. Is there a way to see the comparisons (of current and previous version) without all the code in it? For example the version you would see when making suggestions before making the suggestions. Or is showing the comment history the only way to see it?
Currently this is only possible via the Comment History functionality. However, we are exploring better ways of tracking changes within the document editor. Stay tuned for more updates on this!
I am a quality user and do not see this in my documents. Is there something I need to do to access this feature?
The comment history functionality is available in your sandbox. It will be enabled in your production instance next week.
Other Questions
When is the expected release for the new editor in Events?
We are currently working on this feature and are hoping to release it to customers in the coming months. Please note: it will not be included in the next Launch Train.
What validation documents need to be updated for these changes?
You can see the most up-to-date validation pack here.
Any way to link up multiple CCs (doc release/revisions) to have a 'set' of changes identified for NPI or launch and a trigger/rule that prevents one doc from going from Approved to Effective unless certain conditions are met (i.e. sequencing effectivity)?
We are working on Bulk CC for documents right now. Stay tuned for release updates this year.
Can you add reviewers after the review starts?
This will be available as part of the Bulk CC work in Documents