Preparing your children for total surveillance

I noticed these in a book shop today:

What they do is project a "soothing picture" onto the wall or ceiling at night - ostensibly to help your child sleep. What struck me was the similarly in their design to a surveillance camera, so I guess there's no better time that early childhood to start preparing your child to be oblivious to such devices in everyday life.

Remember, total surveillance is for your safety and security...

Go figure...

Agile is dead

This is an excellent post, though some might consider it a rant, about how Agile wasn't really nor became what Agile should it should have...

Agile is Dead

"Are you an IT consultant or contractor? Agile Software Development work is dead. If you practice that, you are a doorstop. If you manage that way, you are a boat-anchor. The wave has ended, it is over, and if you went for the head-fake and bought certifications, you wasted some money. Soon recruiters will be putting your resume in the circular storage container. I have been warning you for some time, and the day is here. Hah, you should have listened. Move along."

But here's the best quote:

The moral of this part of the story is that if you produce some compact politicised ideology (manifesto) consisting of principles and rules there will be unintended consequences.  Success creates a religion or cult, and defeat is being ignored.  No such doctrine is perfect.  Thinking you will change the world with a manifesto is naive, and if you succeed you may not have improved the world.

To me, given the problems that Privacy by Design has caused with privacy officers believing in some mythical "compliance" without any consideration of the software engineering or cultural aspects of developing information systems, this sums it up quite nicely!

Brussels - something to think about

The Independent is running a story today:

Brussels attacks: Security officials accused of missing a string of opportunities to stop suicide bombers
Accomplices still on the run after day of conflicting reports and confusion

Paul Peachey, Kim Sengupta, Leo Cendrowicz, Laura Pitel

The following paragraphs are deeply troubling given the emphasis on "banning" encryption and the need for total surveillance:

The French Prime Minister, Manuel Valls, who laid a wreath at the underground station close to the European Commission headquarters where more than 20 people died, said that EU nations had to invest “massively” in their security systems. 

The most direct criticism came from Turkey, which has previously criticised France for what it said was a failure to heed a prior warning about one of the suicide attackers involved in last year’s attack in Paris in which 130 people were killed. 

Turkish officials have previously said that French authorities were warned twice by Turkey about one of the assailants in the attacks on Paris in November. A senior government source told The Independent: “We had warned France before the Paris attacks, now this. It’s ridiculous.” 

The two brothers had been known to police in Belgium for years, and operated in some of the marginalised communities in the capital that had avoided close attention from the intelligence agencies despite problems of jihadist recruitment and terrorist links. 

The Belgian federal prosecutor, Frederic van Leeuw, told reporters that the two brothers, Brussels-born Belgian citizens, had “extensive” criminal records but they were not related to terrorism.

So, ultimately the failure was both of communication between intelligence and police agencies *and* a failure to listen. Worse is that the terrorists involved were already known (last paragraph above). If the signals of possible trouble were not seen in the above then the problem certainly does not lie with extensive data collection. In fact the perpetrators actually gained privacy by effectively hiding in plain sight.

The trouble is that now politicians are in the "do something" mode of operation, where doing anything, regardless of effectiveness, is far better than actually thinking and doing the right thing.

I had the pleasure of speaking with some security experts in counter-terrorism a while back. They effectively said that politicians want more security just to be seen to be doing something - this is why we've ended up with airport security that concentrates on bottles of water but not on mitigating the real risks - queues, delays, bottlenecks. The question of profiling, as seen in Israeli aviation security is too much for the politicians to risk their careers on so everyone will suffer under increasingly intrusive and increasingly ineffective security.

Finally this quote by Simon Jenkins of The Guardian:

Those who live under freedom know it demands a price, which is a degree of risk. We pay the state to protect us – but calmly, without constant boasting or fearmongering. We know that, in reality, life in Britain has never been safer. That it suits some people to pretend otherwise does not alter the fact. 

In his admiral manual, Terrorism: How to Respond, the Belfast academic Richard English defines the threat to democracy as not the “limited danger” of death and destruction. It is the danger “of provoking ill-judged, extravagant and counterproductive state responses”.

NSA, BigData and Privacy

We all know that BigData is good and that more data is better. In fact if you could collect everything then you could potentially stop all crimes, stop terrorism, save the World, freedom, puppies...literally do anything and everything!

Except, as most organisations should have realised (ordinary businesses take note!!) that having huge amounts of data doesn't really help you if you have no idea of what you have, what it means and how to actually extract the data you want.

Its worse when you have so much that even running the queries that might extract the right piece of data becomes so complex that you may as well just give up.

Pity the NSA then:

http://www.zdnet.com/article/nsa-whistleblower-overwhelmed-with-data-ineffective/

NSA is so overwhelmed with data, it's no longer effective, says whistleblower

One of the agency's first whistleblowers says the NSA is taking in too much data for it to handle, which can have disastrous -- if not deadly -- consequences.

By Zack Whittaker for Zero Day | April 30, 2015 -- 14:29 GMT (15:29 BST) | Topic: Security

So, we have a paradox in the sense that the more data we get the better we can hide...

Its funny but about 2 years ago there was an idea called the "Slow Data Movement" whose aim was to save the World from BigData madness but concentrating on what you actually need...

In fact we've even heard the argument that mass surveillance is no where as effective as "good old fashioned police work".

In fact, it even seems that the recent attacks in Belgium relied upon unencrypted communications ... which should have been easily spotable, unless of course you've got politicians obsessed with the evils of encryption and too much data to even see the weak signals of ordinary, unencrypted data.

Scary...

Second International Workshop on Privacy Engineering (IWPE'16)

Second International Workshop on Privacy Engineering (IWPE'16)
Co-located with 37th IEEE Symposium on Security and Privacy

26 May 2016 - The Fairmont, San Jose, CA

************************************************************

IMPORTANT DATES

Deadline for abstract submission: 18 January 2016
Deadline for paper submission: 8 February 2016
Notification of acceptance: 22 February 2016
Accepted paper camera-ready: 3 March 2016 

************************************************************
 
We are pleased to invite you to participate in the Second International Workshop on Privacy Engineering (IWPE'16).

Privacy engineering research has never been a more timely endeavor. Ongoing news reports regarding global surveillance programs, massive personal data breaches in corporate databases, and notorious examples of personal tragedies due to privacy violations have intensified societal demands for privacy-friendly systems. In response, current legislative and standardization processes worldwide are seeking to strengthen individuals’ privacy by introducing legal and organizational frameworks that personal data collectors and processors must follow. As a result, engineers are increasingly expected to build and maintain systems that preserve privacy and comply with data protection standards in different ICT domains (such as health, energy, transportation, social computing, law enforcement, and public services) and on different infrastructures and architectures (such as cloud, grid, or mobile computing).

Although there is a consensus on the benefits of an engineering approach to privacy, few concrete proposals exist for models, methodologies, techniques and tools to support engineers and organizations in this endeavor. Work that focuses on helping organizations and software developers to identify and adopt appropriate privacy engineering methods, techniques and tools in their daily practices is also missing. Furthermore, it is difficult to systematically evaluate whether the systems developed using privacy engineering methodologies comply with legal frameworks, provide necessary technical assurances, and fulfill users’ privacy requirements.

Clearly, more research is needed in developing methods that can help translate legal and normative concepts, as well as user expectations, into systems requirements. There is also a growing need for techniques and tools to support organizations and engineers in developing and maintaining (socio-)technical systems that meet these requirements. In an effort to close the gaps in research, the topics of IWPE'16 include all aspects of privacy engineering, ranging from its theoretical foundations, engineering approaches and support infrastructures to its practical application in projects of different scales.

Specifically, we are seeking the following kinds of papers:

1.  technical solution papers that illustrate a novel formalism, method or other research finding with preliminary evaluation;
2.  experience and practice papers that describe a case study, challenge or lessons learned in a specific domain;
3.  early evaluations of tools and techniques that support engineering tasks in privacy requirements, design, implementation, testing, etc.;
4.  interdisciplinary studies or critical reviews of existing privacy engineering concepts, methods and frameworks; 
5.  vision papers that take a clear position informed by evidence based on a thorough literature review.

IWPE’16 welcomes papers that focus on novel solutions based on recent developments in privacy engineering. Topics of interest include, but are not limited to:

• Integrating law and policy compliance into the development process
• Privacy impact assessment during software development
• Privacy risk management models
• Privacy breach recovery methods
• Technical standards, heuristics and best practices for privacy engineering
• Privacy engineering in technical standards
• Privacy requirements elicitation and analysis methods
• User privacy and data protection requirements
• Management of privacy requirements with other system requirements
• Privacy requirements implementation
• Privacy engineering strategies and design patterns
• Privacy-preserving architectures
• Privacy engineering and databases
• Privacy engineering in the context of interaction design and usability
• Privacy testing and evaluation methods
• Validation and verification of privacy requirements
• Engineering of Privacy Enhancing Technologies (PETs)
• Integration of PETs into systems
• Models and approaches for the verification of privacy properties
• Tools and formal languages supporting privacy engineering
• Teaching and training privacy engineering
• Adaptations of privacy engineering into specific software development processes
• Pilots and real-world applications
• Evaluation of privacy engineering methods, technologies and tools
• Privacy engineering and accountability
• Organizational, legal, political and economic aspects of privacy engineering

This topic list is not meant to be exhaustive, as IWPE'16 is interested in all aspects of privacy engineering. However, to screen out off-topic papers early in the review process, we request authors to submit an abstract prior to their paper submission. Abstracts of papers without a clear application to privacy engineering will be considered outside the scope of this workshop and may be rejected.

************************************************************

PAPER FORMAT & SUBMISSION GUIDELINES

We solicit unpublished short position papers (up to 4 pages) and long papers reporting technical, research or industry experience (up to 8 pages) on all dimensions of the privacy engineering domain. Each paper, written in English, must follow IEEE Proceedings format. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will attend the workshop to present the paper.

Abstracts and papers must be submitted via EasyChair

All IWPE'16 Papers will be published in IEEE eXplore, which is indexed by EI Engineering Index, ISI Conference Proceedings Citation Index (CPCI-S), Scopus, etc.

Google Blogger and EU Cookie Laws

This is very kind of Google...providing you with an automatically generated privacy notice to European customers as detailed on the Blogger settings page:

This to me highlights a few problems with privacy laws and compliance:

• Firstly, you have to understand EU privacy laws
• You have to understand how to write such a notice
• You have to understand what systems such as Google Analytics etc actually collect and process.
• You might have to provide an opt-out mechanism such as Google's Analytics Opt-Out.

For 99.999% of bloggers (+/- a few %age points), I strongly doubt that any of this is understood or even known about at all.

So while Google might come in for some criticism for its dominance in the information gathering domain, they at least try to make things easier for their customers.

Then there's the EU Cookie Consent Kit which guides you through at least one part of the consent notice maze.

As an exercise, write a simple website...now work out what privacy notice you should display. Just to make it interesting, you are not allowed to have any contact with a privacy lawyer nor anyone who has a detailed knowledge about such things.

This quote by Einstein (often misattributed to Feynmann) sums up privacy laws and the average person writing a blog:

You do not really understand something unless you can explain it to your grandmother

Our privacy laws have become so complicated and often so misaligned with technology that they can not be easily understood by the average Internet user.

Internet Marketing (Humour)

People often ask privacy professionals how they lock down their PCs to prevent loss of their data, tracking etc, or whether they use Facebook, Twitter etc...well the truth is, privacy professionals tend to be quite selective on what they post, and in some cases, leave one or two browsers or PCs deliberately open for various reasons. One is to game advertisers, or maybe to examine what advertisers and marketers are actually doing.

One thing I have noticed is that certain retailers, for example Gigantti of Finland comes to mind, obviously pass my purchase details on to some marketer/advertiser. I don't ever remember being asked to opt-out of this, but, I do now get adverts for the things I've just bought. They could redirect their advertising budget and remove a few middle managers and save a pile of cash instead...

Then there's things like this:

I must admit I love these; I never click on them, but without such crap as this, the Internet would be a lot less fun...so let's start.

Top left...doctors are annoyed at a 53 yo mother because she's found a miracle cure to wrinkles. I'm actually more surprised that it isn't cosmetic companies who are annoyed - surely they're the ones who'll be put out of business. I think doctors (even cosmetic surgeons!) have much more important things to worry about. Then you have to ask, "Who is this woman?"  Surely if she's upset so many doctors and discovered a miracle cure for wrinkles why isn't she on magazines, TV or even Oprah?!

Top middle...so women don't want other diets, just a pill that is exceptionally powerful. I guess this is some kind of diet pill and again I'm sure dieting companies would be more than interested in this, but...On the other hand I'm not sure that most women want to go from being normal and healthy to a misproportioned anorexic.

Top right...same again, except a selfie-obsessed, European looking blonde (so it isn't just asian women who know about this) receives a malformed, badly photoshopped lower body by using some secret Asian fat burning trick...

Bottom left...SIPOO?!?! If there are millionaires in Sipoo with that kind of yacht then they're probably getting its wreckage salvaged from the islands in the archipelago after they've run aground. Monaco would have been better idea with that size of yacht and the climate better for all those trees and the swimming pool. Nice use of IP geo-location to personalise that advert to me; almost had me fooled for a moment.

Bottom middle...I have those vegetables in my fridge: broccoli and coriander...sorry, kale and cilantro. Another interesting medical claim and I'm left wondering how those vegetables target those specific areas of your body and how this hasn't been discovered before given that we humans do eat quite a variety of vegetables. I wonder what would happen if you would dilute these vegetables in a big vat of water, shake it, dilute it again, shake it and so on until only a trace of the memory of the vegetables is left?

Bottom right...this is easy for a privacy professional, the EU have already come to your rescue with the Right to be Forgotten. Though I guess if getting out of your Ferrari while posting for the waiting paparazzi is your thing, then the right to be forgotten is probably way down on your list of things to worry about. Unless of course there's that picture in Hello magazine of your looking frumpy and overweight...in which cases I can recommend a miracle pill and two vegetables to help, and if there's any left over skin after the diet, there's a 53yo mother you can talk to; assuming you can get past the rioting throngs of doctors baying for her blood...

Marketing and advertising with a touch of personalisation, the Internet wouldn't be the same without it :-)

Warning: Coffee Might Kill You

So, I found out recently that coffee might kill you. Seriously, it is a dangerous substance - almost as dangerous as a 110ml bottle of water is to your safety on an aircraft (but a 1 litre bottle of something alcoholic bought at duty free isn't) - and here's the proof (seen in a Starbucks in San Jose, CA):

Proposition 65 Warning Notice: "Coffee Might Kill You"

I'm not actually sure whether that has stopped anyone from buying coffee (or tea for that matter), ever. I suppose you could switch to that decaf muck, but you're probably going to die of something equally horrible then too.

I guess some lawyers and/or politicians need to cover their asses...

Privacy Awareness Training (more thoughts)

I had the pleasure of presenting at the IAPP's DPIntensive workshop in London this month. After my session I got to talk with many about how to move privacy forward beyond an insular group discussion properly towards the engineers whose job it is to build the systems that implement these privacy rules.

One thing that came up was the need for training and that privacy awareness training hasn't had the effect hoped for. Given that awareness training is exactly that, is it no surprise that once the, usually, one hour presentation on how we should all care about privacy is made nothing happens?

Primarily this is because awareness training is by its very nature very abstract at best and irrelevant at worst. Awareness training is also rarely followed up by more context relevant training, for example, for the software architects or programmers or marketers and so on.

There are various reasons for this, mainly, that to continue training in such a manner takes a great deal of effort to set up and comes with an interesting catch-22 problem: the privacy department/group/... probably doesn't have any engineers; which makes generating relevant training for engineers remarkably difficult.

Worse is that because of the current nature of privacy - it is primarily a legal discipline, albeit one trying to break through to engineering - very few engineers move towards or even into privacy.

One member of the audience at the DPIntensive workshop remarked on this stating that this was one of their biggest problems, especially as they had so much to learn from engineering.

The other major difficulty is that the structures that need to be put in place in order to translate between a legal discipline and an engineering one are undoubtedly complex. Consider a linguist trying to create a translation into an as yet not understood language: first one must understand the script, the syntactic structure and then the semantic ones - not to mention the whole problem of the pragmatic structures and idioms that exist before a degree of fluency is reached that makes translation or even basic conversation possible.

So, the problem with privacy awareness training is that it becomes almost impossible to follow up and continue beyond anything more than a broad, common denominator.

Such training however are fantastic for metrics ... make the training compulsory and you'll get 99% of the company taking the training - which normally lasts an hour, can be delivered by webcast or similar. Working with metrics and a delivery mechanism like that makes it an amazing vehicle for improving 'management' metrics. Which in this case are exactly the wrong metrics, at least from the point of view of the good of the company.

So next time you create a privacy awareness training consider :

• whether that training is aimed at a particular audience, or it is broad and generic
• how that training is to be followed up
• what effects do you expect to see
• measurement of must be made on what effects of the training actually went into practice

We can go further and ask what cultural changes happened due to the training, from the point of view of:

• the programmers
• the engineers
• the overall R&D
• the management
• the marketing department
• the legal department
• the privacy group

Unless all of the above can be answered then the privacy awareness training will have no overall or lasting effect.

Privacy Awareness Training

Awareness of the implications of information loss, data breach and privacy in general is well known, though interestingly rarely acted upon in general practice. Consider the situation I've just witnessed: someone talking loudly in a Skype conversation about some very personal details of his life, plus the odd snipped of financial information and a few names, in a coffee shop - and a relatively crowded coffee shop at that.

We actually spend a disproportionate amount of time worrying about technical solutions to privacy and yet while we are told every day about the dangers of using our technological goodies: phones, laptops, tablets etc, we seem almost oblivious about the data leakage we commit even without technology. Or maybe in the case of a video call in a crowded coffee shop, with the help of technology.

We probably panic more about sharing pictures on Facebook than exposing all our personal details in public in the manner above.

I'm also reminded of any case that happened in the UK. A man - a parent of a child at the school - was prevented taking his DSLR camera to a school sports day on grounds that he might accidentally take pictures of other children with the camera. As we all know, a big expensive camera takes good pictures. Ironically every other parent took a mobile phone - most with just as good picture resolution as the DSLR and most likely all capable of upload to various social media sites along with the all precious meta-data: location, time stamps etc. Just to add irony here he probably had a telephoto lens so that he could take a picture of just his child...you might like to compare the capabilities of a telephoto lens with that of a camera phone. Further irony comes from the fact that how many picture were uploaded with the childrens' details to social media during and after the event without the permission of all present.

Let's us not forget the fact that an internet connected (or should I say radio network connected) mobile device is already exposing much more information than a non-network connected DSLR camera every will or can. I note in Canon's latest models this however is changing...

Buy hey, let's not let common sense and knowledge get in the way of blind panic and misunderstandings.

Let's for a moment concentrate on opposite end of the privacy spectrum, that of the software engineer or programmer trying to construct a system that processes data. For the most part these engineers receive very little in the way of specific training on algorithms, techniques etc for privacy and information processing in general.

When did you last educate your programmers and engineers on the latest data processing or security techniques?

So this brings me to the state of privacy awareness training. Most companies now mandate this for their employees and mandated training normally has a very high view rate. What is less understood is the amount of understanding gained or relevance from this training. In fact privacy awareness training is rapidly becomming the new sexual harrasment training: watch this 1 hour video and reverse 100s of years of cultural indoctrination and be reborn into a new egalitarian society...wow!

And this is one of the problems of privacy awareness training, that a short, generic introduction to the dangers of information loss and privacy magically solves everything. I am sure that many of the companies that have suffered data breaches of late have such training in place. Even the NSA surely has such training, though the outcome after this education is now well known.

One could enter into a huge sociological and cultural discussion about this, but one thing has always struck me about privacy awareness training:

It caters for the lowest common denominator

Awareness training invariably tells me of the dangers of information breaches, maybe some interesting anecdotes about Target, AOL, NSA etc, the dangers of the internet etc.

It never tells me about programming techniques, system design techniques, practical methods of protecting my email, social media use, the differences between DSLRs and mobile phones etc.

In a nutshell, privacy awareness training is rarely, if ever, relevant to the audience. By making privacy awareness training so generic it actually never properly educates the audience about privacy and information security.

Constructing training that properly addresses each of its target audiences is hard and takes time - that is not to be denied - but we can not continue with generic, information content-less material that while is tells about privacy, it does not educate.

Anyway, the man opposite me is continuing with a call to his therapist/lover...he's been through rough time recently it seems, and it is good to talk...about your religious beliefs, former marriage, your views on men/women/relationships, your friends, your financial situation etc...

Privacy and Governance

Actually this is a presentation by Prof. Alastair Scotland of the UK's NHS on governance and, ultimately, patient safety. If you work in governance, privacy, software engineering and any management discipline related with these then this is one video you must watch today.

If it took the NHS 10 years to get this far, then in areas such as privacy which share many of the same characteristics of being a flawed system (in the general sense) with many safety-critical features, we've a huge amount of work to do outside of the mainly philosophical and legal debates about what privacy is.


Alastair Scotland.mp4 from Guy Murray on Vimeo.

An Access Control Paradox

The canonical case for data flow and privacy is some data collection from a set of identifiable individuals and generate insights (formerly called reports) about these. In order to protect privacy we will apply the necessary security and access controls and anonymisation of log files as necessary.

Let's consider the case where where generate a number of reports, and we'll order them according to some metric of their information content and specifically how easy or possible it is to re-identify the original sources.

Consider the system below, we collect from a user their user ID, device ID and location - this is some kind of tracking application, or for that matter, any kind of application we typically have on our mobile devices, eg: something for social media, photo sharing etc...

We've taken necessary precautions for privacy - we'll assume there's notice and consent given - in that the user's data is passed using a secure channel into our system. Process of this data takes place and we generate two reports:

1. The first containing specific data about the user
2. The second using some anonymous ID associated with certain event data for logging purposes only. This report is very obviously anonymous!

For additional security purposes we'll even restrict access to the former because it contains PII - but the second which is anonymous doesn't need such protection.

In many cases this is considered sufficient - we've the notice and consent and all necessary access controls and channel security. Protecting the report or file with the sensitive data in it is a given. But now the less sensitive data is often forgotten in all of this:

• How is the identifier generated?
• How granular is the time stamp?
• What does the "event" actually contain?
• Who has access?
• How is this all secured?

Is the identifier some compound of data, hashed and salted, for example:

salt = "thesystem";id = sha256( deviceId + userid + salt);

This would at least allow analysis over unique user+device combinations and the salt, if specific to this logfile or system, then restricts matching to this log file only. Assuming of course the salt isn't know outside of here. 

The timestamp is of less importance but if of very high granularity would prevent the sequencing of events.

The contents of the event are always interesting - what data is stored there? What needs to be and how? If this is some debug log then there's probably just as much here as there is in the report containing the PII. Often it might just be stack traces (with or without parameters), or memory dumps - both of which contain interesting data, even if it is just a pointer to where a weakness in the system might exist.

Now come the questions of who has access and how is this secured? Given that such a report has interesting content shouldn't this be as secure as the report containing specific and identifiable user data? If there's some shared common knowledge could rainbow tables of hashes etc be constructed?

Consider this situation:

Where two separate systems exist, but there exists a common path between these systems which can be exploited because access control wasn't considered necessary for such "low grade", non-personal data.

Any common path is the precursor to de-anonymisation of data.

This might seem to be a rather trivial situation, except that such shared access and common knowledge of things such as salts, keys etc exist in most companies, large and small. In the latter it is often hard to avoid. Mechanisms such as employee contracts and awareness training actually do very little to solve this problem as they aren't designed to address or even understand this problem.

And here lies the paradox of access control: while we guard reports, files, datasets containing PII, we fail to address the same when working with anonymous data - whatever anonymous means.

PbD, The Privacy Engineer's Manifesto and Privacy Engineering

Had quite a bit of time to rereview the relationship between the foundational principles of PbD, the excellent book Privacy Engineer's Manifesto and my Privacy Engineering book. To me this is how it looks and finally I think we're starting to see a proper balance between these.

The seven foundational principles of Privacy by Design are well known throughout the privacy community and together they stand as an ideal focus for the development of privacy over our information systems as the Agile Manifesto did for software development processes.

1.  Proactive not Reactive; Preventative not Remedial
2.  Privacy as the Default Setting
3.  Privacy Embedded into Design
4.  Full Functionality – Positive-Sum, not Zero-Sum
5.  End-to-End Security – Full Lifecycle Protection
6.  Visibility and Transparency – Keep it Open
7.  Respect for User Privacy – Keep it User-Centric

As time has shown misunderstanding and incorrectly applying the prinicples of the Agile Manifesto has lead to severe development problems and technical debt.


One only needs to look at the modern application of the term agile to understand that its original meaning in many cases has been lost; such is the danger facing the principles of Privacy By Design and even now statements such as 'We Follow PbD Princples' are abound without any underpinning or engineering understanding of those principles in either code or process.

To move forward we must precisely understand how these principles can be integrated not just in to policies, but engineering requirements, design requirements, test cases, software development processes, analysis tools, development tools and even the very psyche of software engineering. Efforts such as the Privacy Engineer's Manifesto take the first step in addressing these aspects and the relationship between PbD.

However working from a purely top-down perspective does not solve all problems, but one needs to work simultaneous bottom-up from basic engineering and deeper theoretical perspectives and ensure that both directions of thought complement, balance and produce a consistent whole. We take the bottom-up approach here and do not attempt to define precise processes but rather present ontologies, structures and tools which can be adapted as local development practices require and dictate.

Privacy Checklists and a Study in Ontario

Not that particular study in Ontario, but another in Ontario regarding the Surgical Checklist and its "ineffectiveness", which was rebutted by many including Atul Gwande which ended with this quote:

Perhaps, however, this study will prompt greater attention to a fundamentally important question for health care reform broadly: how you implement an even simple change in systems that reduces errors and mortality – like a checklist. For there is one thing we know for sure: if you don’t use it, it doesn’t work.

Relating this back to my experiences in deploying and using checklists for privacy is that THEY ARE NOT A TOOL FOR IMPROVING PRIVACY DIRECTLY but a TOOL for organising your workflows, your actions and ensuring that all members of a team are actively cross-checking each other; and even then this is just a small part of the overall effect. Let us for a moment rewrite Gwande's statement a little:

Perhaps, however, this study will prompt greater attention to a fundamentally important question for privacy engineering: how you implement an even simple change in systems that reduces errors and non compliance– like a checklist. For there is one thing we know for sure: if you don’t use it, it doesn’t work.

In the paper [1] (emphasis mine)

The checklist approach to privacy protection has been debated.[24] Checklists have become important safety elements in airplane and medical procedures and are quite common in security auditing. However, their utility for privacy remains questionable. It might be possible to design privacy checklists for frequent and standardized use cases, but the breadth of potential projects makes a standard checklist for everything an unlikely tool. 

[24]  Ian Oliver, “Safety Systems – Defining Moments” Available at http://ijosblog.blogspot.com/2013/07/systems-safety-defining-moments.html

Indeed the two paragraphs on checklists and privacy impact assessments fails to properly understand the former and compares it against the latter which is a different kind of tool altogether. In fact, a PIA should be done and this would be ensured or reminded by having it included as a point on a checklist for privacy.

Indeed no mention was made, nor has been made of any "standardised checklist". In fact there is a capitalised statement on the bottom of the checklist:

THIS CHECKLIST IS NOT INTENDED TO BE COMPREHENSIVE. ADDITIONS AND MODIFICATIONS TO FIT LOCAL PRACTICE ARE ENCOURAGED.

Which can be read about in this article published back in February.

The point here in both cases: surgical and privacy engineering is that the checklist needs to be accompanied by a procedural and "societal" change for it be successful. One only needs to read about Pronovost's work with a simple checklist and the changes surrounding that to understand how checklists work in practice - that and the other experiences presented in Gawande's excellent book on the subject: The Checklist Manifesto. Our experiences can be read about in the presentation Flying Planes, Surgery and Privacy.

* * *

References:

[1] Cavoukian, Cronk, Shapiro (2014) Privacy Engineering: Proactively Embedding Privacy, by Design

Structuring Privacy Requirements pt 1

One of the toughest problems I'm having to solve, not just for my book on privacy engineering, but in my daily job as well is to formulate a set of privacy requirements to the software engineers and the R&D teams.

Actually it isn't that the privacy requirements themselves are hard, we've plenty at the policy level and extrapolating these down into the more functional and security related requirements at the implementation level (ok, it is difficult but there are harder things in this area).

Placing all these in a structure that ties together the various classifications and aspects of information, data flow modelling, requirements and risk management has been interesting and fiendishly difficult to say the least. Getting this structure into a state that supports all of these and getting the semantics of the kinds of thing it is structuring is critical to understanding how all of this privacy engineering works.

We assume that we understand the classification systems, for example the almost traditional Secret-Confidential-Public style security classifications and the Location, Time etc classifications of information type; as well as the other aspects such as purpose, usage, provenance, identity etc. Each of these has its own set of classification elements, hierarchy and other ontological structure. For example for information type:

Information Type Hierarchy

We also assume we understand data flow modelling with its processes, flows, sources, sinks and logical partitionings. We can also already see the link between elements here (as shown below) and the classification systems above

Example Data Flow with Annotations from Previously Described Ontologies/Classification Systems

Now the structure of our requirements needs to take into consideration the various elements from the classification systems, the aspect of the requirement we want to describe (more on this below) and the required detail of requirement relevant to the stage in the software process. This gives us the structure below:

So if we wish to find the requirements for User's Home Address Policy for Local Storage then we take the point corresponding to those coordinates. If there happens to be nothing there then we can use the classification systems' hierarchies to look for the requirement corresponding to a parent; a "user's home address" is-a "Location":

So if we take the example data flow from earlier then for each of the flows, storages and processes we can construct a set of requirements simply by reading off the corresponding requirements from the structure above.

This leads to an interesting situations where it is possible to construct a set of requirements which are overconstraining. That is simply that we can not build a system that can support everything, for example, one data point might trigger a secret classification and key management, encrypted content etc.

We then need to weaken the requirements such that a system can be constructed according to some economic(!) model. As we weaken or retrench our requirements we are introducing risk into the system,

Aside: Retrenchment - here's a good reference: Banach, Poppleton. Controlling Control Systems: An Application of Evolving Retrenchment. ZB 2002:Formal Specification and Development in Z and BLecture Notes in Computer Science Volume 2272, 2002, pp 42-61.

This gives us our link to risk management and deciding whether each "weakness" we introduce is worth the risk. And as risk can be quantified and we can perform further tasks such as failure mode and effects analysis then we obtain a rigorous method to predict failures and make informed decisions about these. I'll describe more on this in later articles.

Airbus Aircraft Operational Philosophy

I'm reading the Airbus Flight Crew Training Manual for the A320/A321 aircraft at the moment. If anything, to get an understanding of how such things are written and presented. Aircraft are fairly complex systems and as I've already talked about in this blog, many ideas are directly applicable to privacy, security, software engineering etc if we only take time to learn.

Aside: If anyone does own an A320 or equivalent simulator and would like to offer me some time flying (simulated or otherwise) then YES PLEASE!!!.  OK, back to reality now...

A few things struck me as being particularly relevant such as The Operational Golden Rules:

1. The aircraft can be flown like any other aircraft
2. Fly, navigate, communicate - in that order
3. One head up at all times
4. Cross check the accuracy of the FMS
5. Know your FMA at all times 
6. When things don’t go as expected - take over 
7. Use the proper level of automation for the task
8. Practice task sharing and back-up each other

Nothing too surprising there, other than they're stating the obvious.

That's the great thing about the obvious, completely missable...

Note the emphasis on getting the job done without requiring anything other than basic aviation skills; or in our case basic software engineering skills. The emphasis on cross-checking, task sharing and delegation, the use of appropriate techniques and technologies and the note that when things don't go as expected - take over!

As an idea to start off...

1. There is nothing special about this system under development
2. Plan, code, test - in that order
3. Be aware of the wider engineering and requirements context
4. Cross check your system with its original goals
5. ...

The learnings here to our privacy engineering world are deep and profound. Just as an exercise to the reader, look at the procedures being followed for any engineering task and could these be placed into a simple set of operational golden rules as above? There's an interesting comparison waiting to be drawn here...that will have to wait for the moment.

Privacy Engineers and Privacy Lawyers

Probably on the of the best articles I've seen on the apparent dichotomy between privacy lawyers and the engineers who must build the systems:

Engineers and Lawyers in Privacy Protection: Can We All Just Get Along?

By Peter Swire,and Annie Antón January 13, 2014

Actually I'd add a 3rd group called privacy advocates who tend to side with the lawyers and believe that the engineers are there to do their bidding.

But let's take a specific example as mentioned in the text, that of data minimisation. Actually it turns out that [software] engineers rarely gather too much - it really isn't in our nature to overcomplicate the already complicated process of building information systems. Indeed one of the ways to upset engineers is to repeatedly tell them not to collect data they aren't collecting in the first place. Each new data point usually involves additional validation, verification, tests and obviously more code. More code => more bugs => more test => more time => more expense etc...

But we also see other problems such as the emergence of the privacy cabal - a group of predominantly lawyers and advocates who do not understand the discipline and complexities  of engineering. Again the article above quite well explains this. Indeed Jim Adler in his PII2012 talk called this the Privacy-Industrial Complex: a mechanism for churning our policies, guidelines, edicts on the topic of privacy but without any basis in engineering reality. The engineers role is reduced to a mere bystander, a group of people to be handed orders from the cabal and privacy priesthood on high.

When things go wrong, it is invariably the engineers' fault, while the priests of privacy claim that their policies conform to best practice and take solice in the Privacy by Design Commandments. Take Target for example, their POS system collected only necessary data - credit card numbers  - was this an example of applying Fair Information Policy Principles? Is collecting and processing credit card numbers necessary for processing credit card numbers? Do our privacy principles accurately capture subtle requirements such as caching data in memory, types of encryption algorithm required, the human-computer interface etc Aside: OK, Target failed in many respects and the results of that investigation are to be seen.

Requirements such as don't collect PII ... is an IP address PII? How do I stop collecting this when the very protocols of the internet require such addresses just as the postal system relies upon physical addresses.

The position of the emerging privacy engineer will become one of the most important positions in the field of privacy. A group of people who understand the fundamentals of information systems from their engineering to their mathematical foundataions is critically required. Ignore these foundations and privacy becomes an hand-waving, powerpoint generating inconvenience to be humored and tolerated rather than an integral part of the business ecosystem.

When lawyers and engineers work together AS EQUALS we get some truly AMAZING work done. It happens rarely and it takes a lot of work from both sides just to build a common framework of understanding. Engineers have the ability to properly deconstruct and understand the finer workings of privacy compliance - ignore this as privacy will remain as we described above: a tolerated inconvenience.

Child Protection Software

Something of a hiatus, been busy with the book, formalising a major company's privacy policies and a side order of Clojure programming (see next post). All-in-all, busy. However I just want to turn to some experiences with software for child proofing a computer, or a tablet if you prefer. Protecting children from all the nasties of the Internet seems to be quite a moral crusade by some. Massive investments in filtering and censorship software and schemes seem to take precedence over good old-fashioned education - you know, that thing parents are supposed to do?

I've been working through some of these systems for child protection, ostensibly running on Android. Generally they all suck badly making the device almost impossible to use and administer and causing more headaches for the parents...just a general observation from asking many on the subject, including a few child protection experts.

One thing particularly struck me about some of the software for this kind of protection, they all seem to be gathering an awful lot of information about the child, either through reporting back statistics (including location) or by acting as proxies to content. In the latter case you are also at the mercy of the protection software provider with regards to what they're filtering.

One system asked me to register with a huge amount of personal details: my details, partner details, email addresses, the childrens' details, addresses, dates of birth, gender, preferences, interests, hobbies etc (marketers' dream), and then informed my through an extraordinarily lengthy terms and condition and privacy policy that they would collect the location of the tablet/phone/computer each time the software was activated, shut down, used. Finally they topped this off with a statement about how some data would be `anonymously' used by (unnamed) 3rd party companies for providing a better service.

Suffice to say I didn't install it and reverted to supervision, education and being a parent.

Privacy, Evidence Trails and a Change in Terminology?

One of the main aspects of personal [information] privacy is that much of the topic is that other parties would not collect nor perform any analysis of your data. The trouble is that this argument is often made in isolation, in that it somewhat assumes that the acts we perform by computer exist in a place where we can hide. For example, what someone does behind closed doors usually remains private. But, if that act is made in a public place, say, in the middle of the street by default whatever is done is not private - even if we hoped no-one saw.

Anything and everything we do on the internet is in public by default. When we perform things in public, then other people may or will see, find out and perform their own analysis to form a profile of you.

Many privacy enhancing technologies are akin to standing in the middle of a busy street and shouting "don't look!". Even if everyone looks away, more often than not there is a whole raft of other evidence to show what you've been doing.

Admittedly most of the time nobody really cares nor are actually looking in the first place. Though as it has been found out recently (and this really isn't a surprise) that some such as the NSA and GCHQ are continually watching. Even the advertisers don't really care that much; their main interest is trying to categorise you to ship a generic advertisement - and advertisers are often really easy to game...

If we really do want privacy on the internet then rather than concentrating on how to be private (or pretending that we are), we need to concentrate on how to reduce the evidence trail that we leave. Such evidence is in the form of web logs, search queries, location traces from your navigator, tweets, Facebook postings etc.

Once we have understood what crumbs of evidence is being left, we can start exploring all the side avenues where data flows (leaks) and the points where data can be extracted surreptitiously. We can also examine what data we do want released, or have no choice about.

At this moment, I don't really see a good debate about this, at least not at a technical level though there are some great tools such as Ghostery that assist in this. Certainly there is little discussion at a fundamental level which would really help us define what privacy really is.

I personally tend to take the view at the moment that privacy might even be the wrong term, or at best, somewhat a misleading term.

On the internet every detail of what we do is potentially public and can be used for good as well as evil (whatever those terms actually mean), our job as privacy professionals is to make that journey as safe as possible, hence the use of the term "information safety" to better describe what we do.

Semiotic Analysis of a Privacy Policy

Pretty much every service has a privacy policy attached to it; these policies state the data collection, usage, purpose and expectations that the customer has to agree with before using that said service. But at another level they also attempt to signify (that's going to be a key word) that the consumer can trust the company providing the service at some level. Ok, so there's been huge amounts of press about certain social media and search service provides "abusing" this trust and so on, but we still use the services provided by those companies.

So this gets me thinking, when a privacy policy is written, could we analyse that text to understand better the motives and expectations from both the customer and the service provider perspective? Effectively can we make a semiotic analysis of a privacy policy.

What would we gain by this? It is imperative that any texts of this nature portray the right image to the consumer, thus this can be used in the drafting of such a text to ensure that this this right image is correctly portrayed. For example, the oft seen statement:

"Your privacy is important to us"

is a sign in the semiotic sense, and in this case probably an 'icon' in its near universal usage. Signs are a relationship between the 'object' and 'interpretant', respectively the subject matter at hand and the clarified meaning respectively.

Pierce's Semiotic Trangle

The object may be that we (the writer of the statement) are trying to state a matter of fact about how trust worthy we are, or at least we want to emphasise that we can be trusted.

The interpretant of this, if we are the customer, can of course vary from total trust to utter cynicism. I guess of late the latter interpretation tends to be true. Understanding the variation in interpretants is a clear method for understanding what is being conveyed by the policy itself and whether the right impression is being given to the consumer.

At a very granular level the whole policy itself is a sign and the very existance of that policy and its structure, is it long and full of legalese or short and simple? Then there's the content (as described above) which may or may not depend upon the size of the policy....as in the World's Worst Privacy Policy.

References:

Andy Greenberg (2012) The World's Worst Privacy Policy. Forbes 25 Jan 2012.

Aside:

Found this paper while researching for this: Philippe Codognet's THE SEMIOTICS OF THE WEB, it starts with a quote:

I am not sure that the web weaved by Persephone in this Orphic tale, cited in exergue of Michel Serres’ La communication , is what we are currently used to call the World Wide Web. Our computer web on the internet is nevertheless akin Persephone’s in its aims : representing and covering the entire universe. Our learned ignorance is conceiving an infinite virtual world whose center is everywhere and circumference nowhere ...

Must admit, I find that very, very nice. Best I've got is getting quotes about existential crises and cosmological structures in a paper about the Semantic Web with Ora Lassila.