Checklists for Privacy Audits

Much of my work are privacy audits of information systems. These vary in nature and context from a quick check to a full dissection, and from client applications to service back-ends, internal systems, cloud deployment etc. Over the years we've made progress in standardising the review both in terms of process and in terms of content along with developing various tools and ontologies for information classification, requirements etc.

One of the most challenging tasks however has been to externalise this knowledge - when you've a small team knowledge is often implicit, but for many reasons including certification (ISO9000 etc) and training new staff documenting the procedures in a form where they can be repeated and understood by others without sacrificing the quality of the review we must externalise what is embedded in our collective team unconscious. Making this simple is remarkably hard - we've certainly been through our fair share of templates, documents, process diagram etc.

Areas such as air traffic control and piloting are well known for their command-response checklist techniques and we've taken much inspiration from these. However the key inspiration for our current ideas actually came from the WHO's Surgical Safety Checklist which aims to provide a simple, high-level checklist for surgical procedures without compromising the context in which the checklist is employed. This latter aspect rings especially true for us in that our context of review changes dramatically between audits as do individual surgeries. Here's our version:

We've remained true to the WHO's version (why change something that works?) with the three high-level stages, two of which emphases the necessity of getting the scope, context and reviews made properly. The middle section - the operation if you like - preceded by a time-out in which we ensure that each member of the privacy audit team works properly together and have a common understanding of what is required. Note the emphasis at the end of evaluating the review and the effectiveness of our procedures - this is critical in bettering how our work is executed.

An accompanying manual in the same style as the WHO's Surgical Safety Checklist Implementation Manual is under construction and in test. This we aim to address not just the process but the roles and responsibilities of the case coordinators, the "operating team's" (including individual members) roles, management and, importantly, the R&D or owner of the system being investigated. Never forget about the customer especially as privacy reviews tend to be quite deep and invasive both in time, information needs and recommendations, Many development teams do see such reviews as not effective use of their time and taking them away from their primary job of developing software.

Now there are many existing checklists for privacy - some detailed, some more process oriented, some technical, some more legal in nature - but consolidating and standardising without compromising the contexts in which these are executed is the issue. This I expect is the guiding principle behind the WHO's checklist which itself has been proven to work effectively.

It is my belief that the best practices from safety-critical system development need to be deployed in the systems we are building through development techniques, methods and processes. I consider good information system management (through privacy principles) to be safety-critical in many senses.

* * *

Compulsory Reading: Atul Gawande. The Checklist Manifesto.