Data Audit: Process & Purpose
Data ready:
Hospitals need to submit the data for the Sepsis project. Hospitals correct exception report data and resubmit the data to the portal. The data is ready for sampling for the audit. Note that this sampling is not the same as hospitals who elect to sample for quarterly data submission. All hospitals have a sample of their medical records selected for audit.
Sampling for audit:
The DOH selects only a small sample of adult and pediatric cases for audit each quarter. All hospitals have a few cases selected even if they are a small volume hospital. The lists of the cases that are selected for each hospital include patient’s information such as UPI, MRN, PCN, etc. therefore the lists are uploaded to a secured server, Dropzone. This is the same location where medical records are uploaded when ready for audit. Hospitals are provided a month timeframe to upload the required medical records to the Dropzone.
Checking Receipt of Medical Records:
Analysts will check the medical records after downloading the records from the Dropzone to see if the submitted records are for the correct patients. If non-clinical submission errors, such as wrong patient record submission are found, the analysts will send tickets through Zendesk to notify the hospital to resubmit the records. This step will repeat until all of the records are correct for a hospital, and then the records will progress to the clinical professional audit team for review.
Warning and Non-compliance:
If a hospital doesn’t submit the requested records by the due date that is specified in the letter, a warning email will be sent from IPRO to allow for the possibility that the hospital did not receive the first notification. The hospital is then given a five business days to upload the requested records to meet submission requirements. This warning email is the final request for the medical records submission. Failure to submit records before the deadline of the warning letter will result in a referral to the NYSDOH for the processing of a Statement of Deficiency. The non-compliance letter will be sent directly from NYDOH to the hospital CEO for hospitals that have failed to submit medical records before deadline defined in the notification.
Audit:
For each quarter, approximately 40-50 data elements from the Data dictionary will be selected for audit. This list rotates based on such factors as prior audit performance, importance to the process measure or Risk Adjusted Mortality (RAMR) measure. Clinicians will view the sampled cases for these data elements and compare them with the medical records to see if the data reported by hospital is consistent with the medical record documentation. Generally we ensure that all data elements will be audited for at least one quarter in each year.
Second notice and resubmit:
During the audit, some situations may cause the audit to move to pending status for some cases. For example, the record for the case is incomplete, such as missing medication sheets; the records for the patients are from a different time (we only audit records in a particular discharge time, but a patient may be admitted to a hospital multiple times). In this instance, second notice reports will be sent to the hospitals and another week will be given for the hospitals to submit the supplemental materials or correct requested records.
Close cases:
Reviewers will close the case upon audit completion. Internal quality review ensues, with physician and nurse supervisor over-read for a sample of audited cases to ensure audit accuracy. In addition, during the review, weekly auditor and clinical supervisor meetings are held to discuss unclear cases. After the case is closed and oversight review has concluded, cases move to analytic phase where global, hospital specific, data category, and individual data item analytics are calculated. Audit reports are generated and internal quality review is performed to ensure data analysis accuracy.
Audit reports:
Audit reports are provided to each hospital that show the consistency of auditor review and hospital data submission for the variables selected during the audit timeframe. Historically, the audit report has been provided in word and html snapshot format with excel spreadsheets; however, the report for 2017 and forward is presented in Tableau, which allows for excel extract in addition to visualization within Tableau. In the audit report, hospitals can compare their performance for each variable with New York statewide performance, and hospitals can also view the case-level detail for each variable in each case. Comments are also provided for the data elements if reviewers disagree with hospital reported data to assist hospitals to improve future accuracy.
Hospitals correct dataand freeze data:
Hospitals are expected to correct data as applicable in the database in IPRO’s server after receiving the audit report, which is important since the data will be used for the creation of publicly reported hospital-specific risk adjusted mortality rates and performance measures. The database will remain open for a month after the audit reports are released, after which the data will be frozen. After the data are frozen, no additional changes are permitted for the data for the calendar year.
Key words:
Zendesk: Zendesk is the help desk platform maintained by IPRO, where tickets can be sent to the sepsis team seeking help, such as changing users for Dropzone or Portal, questions about uploading records, etc. The sepsis team may also send tickets to hospitals for additional information for audit or other issues.
Unique Personal Identifier (UPI): A composite field comprised of portions of the patient last name, first name and social security number. For more information please refer to Data dictionary.
Medical Record Number (MRN): The number used by the hospital’s Medical Records Department to identify the patient’s permanent medical records file. This number is not the same as the Patient Control Number (PCN).
Patient Control Number (PCN): A patient’s unique number assigned by the provider to facilitate retrieval of individual financial and clinical records and posting of payment.
Dropzone (MailGate SC): Dropzone is a secured platform with restricted access. For each hospital, only two people can have permission to access Dropzone, and the hospital’s privacy officer must be notified every time their hospital requires changing users.