Privacy Metrics

Along with a colleage - Dr. Yoan Miche - we presented a paper outlining ideas regarding using mutual information as a metric for establishing some form of  'legal compliance' for data sets. The work is far from complete and the mathematics is getting horrendous!

The paper entitled "On the Development of A Metric for Quality of Information Content over Anonymised Data-Sets" was presented at the excellent Quatic 2016 conference held in Lisbon, Sept 6-9, 2016.

We were also extremely fortunate in that a presented in our session didn't turn up and we were graciously given the full hour not just to present the paper but give a much fuller background and details of future work and the state of our current results.

Here are the slides:

On The Development of A Metric for Quality of Information Content over Anonymised Data Sets from Ian Oliver

Abstract:

We propose a framework for measuring the impact of data anonymisation and obfuscation in information theoretic and data mining terms. Privacy functions often hamper machine learning but obscuring the classification functions. We propose to
use Mutual Information over non-Euclidean spaces as a means of measuring the distortion induced by privacy function and following the same principle, we also propose to use Machine Learning techniques in order to quantify the impact of said obfuscation in terms of further data mining goals.

Citation:

Ian Oliver and Yoan Miche (2016) On the Development of A Metric for Quality of
Information Content over Anonymised Data-Sets. Quatic 2016, Lisbon, Portugal, Sept 6-9, 2016.

Information wants to be free...?

I both like and dislike the quote "information wants to be free", mainly because it opens up a very nice philosophical discussion on what 'free' means but also because - and this is part I hate - it is some damned meaningless without any grounding in any form of semantics; and we've seen this before!

For the first part, this statement treats information in an anthropomorphic manner. Is it really information itself that has the need to be free? Let's assume that it does, though in a very fairy tale like way, it seems to me.

So let's then look at the word 'free', which I assume does not mean 'free' as in 'without cost' in the sense that someone has to pay for it. Though this is a curious idea in that information is somehow prostituting itself and despite all attempts someone (the information's pimp perhaps?) insists on controlling things. I guess this is the idea that information is going through some pre-1960's sexual revolution...

Rather I think the word 'free' refers to 'freedom' albeit in a Western sense of the word. Think of the use of the concept freedom as used in the US Declaration of Independence.

"We hold these truths to be self-evident, that all information is created equal, that it is endowed by its Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness"

...or should that be Existence, Communication and Semantics perhaps....?

Let's stick with the word 'freedom' and its naive or common-sense meaning. What does it mean to be free? We can turn further to the the UN Declaration of Human Rights and the EU Charter of Fundamental Rights for further clarification, though here I'm sure we go into more of a legal-political debate more than anything. Evidently freedom means either the right to do or be something or the right to be protected from something.

So the question is, if information want to be free:

• What does information want the freedom to do/be?
• What does information want the freedom from?

Under the first question, the freedom to be 'free' as in 'without cost' certainly falls. What about the freedom to be private, or the freedom not to be abused - as in excessive privacy violations? Do we further need a notion of agency - does information have an owner or provenance?

Without answering those - I don't think I can give a definitive answer anyway - here's another thought. Given that matter = energy, isn't the use of the term information quite literally another way of saying 'humans' (or 'men' as in the Declaration of Independence). In which case the question 'information wants to be free' is just an expression of man's desire to define what freedom is - ostensibly in terms of freedom to do/be and freedom from.

And here comes the practical part, which freedoms to do/be or from do we allow or deny in order to be "free"?

NSA, BigData and Privacy

We all know that BigData is good and that more data is better. In fact if you could collect everything then you could potentially stop all crimes, stop terrorism, save the World, freedom, puppies...literally do anything and everything!

Except, as most organisations should have realised (ordinary businesses take note!!) that having huge amounts of data doesn't really help you if you have no idea of what you have, what it means and how to actually extract the data you want.

Its worse when you have so much that even running the queries that might extract the right piece of data becomes so complex that you may as well just give up.

Pity the NSA then:

http://www.zdnet.com/article/nsa-whistleblower-overwhelmed-with-data-ineffective/

NSA is so overwhelmed with data, it's no longer effective, says whistleblower

One of the agency's first whistleblowers says the NSA is taking in too much data for it to handle, which can have disastrous -- if not deadly -- consequences.

By Zack Whittaker for Zero Day | April 30, 2015 -- 14:29 GMT (15:29 BST) | Topic: Security

So, we have a paradox in the sense that the more data we get the better we can hide...

Its funny but about 2 years ago there was an idea called the "Slow Data Movement" whose aim was to save the World from BigData madness but concentrating on what you actually need...

In fact we've even heard the argument that mass surveillance is no where as effective as "good old fashioned police work".

In fact, it even seems that the recent attacks in Belgium relied upon unencrypted communications ... which should have been easily spotable, unless of course you've got politicians obsessed with the evils of encryption and too much data to even see the weak signals of ordinary, unencrypted data.

Scary...

Twitter Discussion on Privacy and Engineering

Related with the upcoming DSummit conference in Malmö in May I've been involved in a fascinating discussion on Twitter with some of the big privacy people there.

The main point being raised is the need for a proper dialog between engineers and lawyers. I think we've seen this before, but still it is not being properly addressed and until it is privacy will remain a compliance activity rooted in a tick-box mentality with dreadful repercussions.

One only needs to take a look at the potential penalties in the EU's GDPR ... a potential fine of 4% of global turnover for a privacy violation!

The crux of this is that if you want to construct systems with privacy as an aspect, it has to be a first class aspect of that system's design. That means privacy is under the collective responsibility of lawyers, engineers and management and not the sole preserve of any of these groups.

Belief in high-level privacy impact assessments and "compliance", and placing trust in a legalese privacy policy is woefully insufficient, not to mention from a business perspective one step short of insanity.

Unfortunately going beyond this is considered by some - and I've seen too many examples of this - to be difficult and unnecessary and that legal compliance - whatever that means - is enough...

As we move to a "BigData" future, the knowledge of basic data handling, quality and governance at both engineering and legal levels is critical - not just for privacy but for basic business reasons, including consumer trust and quality of product.

How to do this is not difficult, but it does require thinking and small, but extremely beneficial cultural change...

Agreed. Need to translate high level privacy principles & PbD, into specific engineering reqs @mdennedy @i_j_oliver https://t.co/a4hgcUhGnJ

— Privacy Matters (@PrivacyMatters) December 20, 2015

and here's a recommendation to get those principles into use:

@daraghobrien I found @i_j_oliver's and @mdennedy's books on privacy engineering of enormous value @ll1t

— Privacy Matters (@PrivacyMatters) December 21, 2015

You can start here:Privacy Engineering and A Privacy Engineer's Manifesto

Tea

I'm pretty sure this is just hot water...

"Ceci n'est pas te."

Maybe we need a better semantics for what tea (and coffee) actually are... and not this kind of "treachery"!

Maybe it is homeopathic tea?

On Being Privacy Risk Adverse

Being risk adverse in [IT] system development isn't always a bad idea - consider mainframe technologies which are constructed to avoid any kind of failure bringing the whole system down, or not using the latest, greatest JavaScript library for your mission-critical web development...

Risk management in privacy has come to the fore of late, especially the with publication of the NIST standard of risk management. So today's conversation about being risk adverse and how one assess risk in privacy was extremely interesting.

Consider this:

Collecting personal data (or PII) is a risky activity and therefore must be minimised as much as possible.

The definition of personal data is very weak, but it is always best to consider almost everything personal data in case it is cross-referenced with other data (which would make it personal data)

therefore

Don't collect anything. Ever.

While extreme, it shows how a misplaced understanding is many aspects, including what is risk and the nature of information (personal data) can lead to extreme situations and conclusions.

While NIST is absolutely correct in its assessment that we need proper risk management procedures, how these relate to requirements, information type and all of the other privacy ontological structure is as yet very, very weak.

In fact, terms such as personal data and PII do not come even close to being in any form usable for risk management - for this we need to go much deeper into the nature of information. For example, instead of "personal data" we could use classifications on information type and a mapping from different kinds of data (of these types) to risk metrics (note the plural). An overall risk value can then be more accurately calculated - or at least be calculated on the basic of what information we actually have.

You can read more about this approach to privacy engineering in the book: Privacy Engineering - a dataflow and ontological approach.

Google Blogger and EU Cookie Laws

This is very kind of Google...providing you with an automatically generated privacy notice to European customers as detailed on the Blogger settings page:

This to me highlights a few problems with privacy laws and compliance:

• Firstly, you have to understand EU privacy laws
• You have to understand how to write such a notice
• You have to understand what systems such as Google Analytics etc actually collect and process.
• You might have to provide an opt-out mechanism such as Google's Analytics Opt-Out.

For 99.999% of bloggers (+/- a few %age points), I strongly doubt that any of this is understood or even known about at all.

So while Google might come in for some criticism for its dominance in the information gathering domain, they at least try to make things easier for their customers.

Then there's the EU Cookie Consent Kit which guides you through at least one part of the consent notice maze.

As an exercise, write a simple website...now work out what privacy notice you should display. Just to make it interesting, you are not allowed to have any contact with a privacy lawyer nor anyone who has a detailed knowledge about such things.

This quote by Einstein (often misattributed to Feynmann) sums up privacy laws and the average person writing a blog:

You do not really understand something unless you can explain it to your grandmother

Our privacy laws have become so complicated and often so misaligned with technology that they can not be easily understood by the average Internet user.

UK Ontology Network 2015 Presentation

I really have been a bit lax here of late - not that I have little to write, but rather time (as always!) and the day job takes me away from privacy into new uncharted, more security research related areas - which I will admit is great fun!

Back in April I attending a fascinating little workshop run by the UK Ontology Network to talk about ontologies for privacy. What makes this workshop fun is the sheer amount of interaction between the participants: the distinction between presenter and audience is completely blurred. As a presenter you get a 5 minute slot then after a number of presenters on similar subjects, a 20 minute panel session where the audience really gets into the conversation.

I'm going to be unfair to all the presenters at UKON 2015 but I'm going to pick out a presentation by Adam Nogradi on Sparqlycode - a tool for semantically annotating source code and establishing compliance against, in this case, certain security guarantees. Of course, to move to a different area all you need is an ontology for your subject area, say, privacy, and you have a tool for privacy compliance...more on this later :-)

So here's my presentation based on the work in the book Privacy Engineering: A Dataflow and Ontological Approach.

Ontologies for Privacy from Ian Oliver

And the book containing a much more detailed description can be bought from Amazon, Barnes and Nobles etc etc.

An article on The Semantics of PII

A while back I wrote a short article for the IAPP's Privacy Tech Blog. With permission I'll reproduce it here for additional reference. Also,a tip of the hat for the administrator of the blog: Jedidah Bracy of the IAPP for his spell checking, grammar checking and editorial skills!

The Semantics of PII

Privacy Tech | Feb 26, 2015

Last year, Profs. Peter Swire and Annie Antón wrote a compelling piece in Privacy Perspectives about the need for privacy engineers and lawyers to get along. Establishing a common language in which to communicate will be essential to appropriately connect policy with technology.

It’s probably safe to say that the most common terms used in privacy are personally identifiable information (PII) and personal data, depending upon whether you come from a U.S. or European background. I think these terms are more or less self-explanatory.

But what do they really mean?

Take PII, for example. It means a chunk of data that reveals some knowledge about a person that can be unambiguously identified. Sounds more or less about right, doesn’t it? Is a computer's IP address personally identifiable? What if that IP address belongs to a router for a large, multinational corporation? Is it PII then? And what if it belongs to a family using multiple computers, tablets, phones or other devices?

We will soon find ourselves delving into the minutiae of meaning—the what-does-personal-really-mean type questions. Plus, we must ask what isinformation, and what does identifiable denote?

There is a whole area of linguistics, philosophy and mathematics—take your pick—that deals with the meaning of things, otherwise known as semantics, or even semiotics if you want the overall field.

Mathematicians took years to fully understand the semantics of even simple statements such as 1+1=2, which looks obvious until you try to explain what 1 is, what 2 is, what + means, what = means and then what it means to say 1+1. The English philosophers Bertrand Russell and Albert Whitehead spent most of their careers writing Principia Mathematica to answer this question, and after four editions and 300 pages of dense mathematics, they had an answer. That was, until a young German by the name of Kurt Gödel came along and shook mathematics to its foundations with an equally "trivial" result.

So if it took 300 pages by two of the brightest minds in mathematics to give us a semantics for 1+1=2, how many pages—and years of work—will it take to give "PII" a semantics?

Now here's an interesting point: The definition of PII that is used in contemporary privacy is perfectly well defined in the privacy-legal context. I can go to various legal documents and read a formal definition of what PII or personal data means. But as we move between disciplines—in our case from privacy-legal to privacy-engineering disciplines—these definitions no longer hold, or at the very least, they don't work well.

If we move to the other end of the scale from legal to mathematics, we find concepts such as information entropy, which provides a clear, unambiguous and precise definition of what information is as well as the identifiability of a data set with respect to some population and so on. Information entropy, however, is not an easy concept with which to work. We can state now that the legal definition of PII can be defined in terms of the mathematical definition; it's just that this is obscenely difficult to do.

Somewhere between these two extremes lies software engineering, the discipline that actually implements privacy law into our systems, in ostensibly mathematical (programming language) terms.

Software engineers, much to the chagrin of privacy lawyers, do not understand legal terms. Well, ok, they do to a point, but you try coding a statement such as "reasonable privacy" into C++ or Java!

Plus, privacy lawyers don't understand all the subtle ramifications of virtual machines, machine language, object orientation, distributed computing, network protocols, XML, RDF—the list goes on!—again, much to the chagrin of software engineers.

Yet, as we stated earlier, there is a relationship between the terms and language that privacy lawyers use and the terms and language that software engineers use. That link provides the translation mechanism that allows both groups not just to talk but to properly communicate with each other.

We can spend as much time as we’d like writing manifestos and principles, designing processes, inventing new job titles such as privacy officer, privacy compliance tsar, grand chief-overseer-of-the-worshipful-court-of-privacy-dudes and so on, but without grounding semantics into terms such as PII and personal data—terms that will allow us to translate between legal-speak and engineer-speak—all of this work will be in vain.

Automating Privacy Compliance

I was at the 2015 meeting of the UK Ontology Network in Leeds earlier this month where, amongst many, there was a presentation about a tool called Sparqlycode - which if you get the chance you must check out!

Anyway, Paul Worral of Interition Ltd wrote a very nice summary of my work:

Ian Oliver [Nokia Networks, Espoo, Finland] presented Ontologies for Privacy.  The whole idea behind Sparqlycode is to provide an information tier for software that enables it to be linked to knowledge about the business.  Ian's work is a perfect example of this. He demonstrates how high-level policies on Personally Identifiable Information should and can be directly related to the code responsible for adhering to them. Ian has authored a book on the subject, Privacy Engineering. I bought it and hope to have some examples of how it can be applied to Sparqlycode soon.

I look forward to seeing how this works out, but it certainly is in the direction of where I hoped. It'll be very interesting to see how this particular approach matches with a more data-flow based modelling approach.

At UKON2015 there was also an extremely interesting presentation about a tool called TawnyOWL for programmatically generating ontologies. Given that Clojure is my current language of choice this seems a perfect fit for the privacy ontologies themselves.

A Definition of PII and Personal Data

There's been an interesting discussion on Twitter about the terms PII and "personal data", classification of information and metrics.

Personally I think the terms "PII" and "personal data" are too broadly applied. Their definitions are poor at best; when did you last see a formal definition of these terms? Indeed classifying a data set as PII only comes about from the types of data inside that data set and by measuring the amount of identifiability of that set.

There now exists two problems in that a classification system underneath that of PII isn't well established in normal terminology. Secondly metrics for information content are very much defined in terms of information entropy.

Providing these underlying classifications is critical to better comprehending the data that we are dealing with. For example, consider the following diagram:

Given any set of data, each field can be mapped into one or more of the seven broad categories on the left - If we wanted we could create much more sophisticated ontologies to express this. Within each of these we can specialise more and this is somewhat represented as we move horizontally across the diagram.

Avoiding information entropy as much as possible, we can and have derived some form of metric to at least assess the risk of data being held or processed. A high 'score' means high risk and a high degree of reidentification is possible, while a low score the opposite - though not necessarily meaning that there is no risk. Each of the categories could be further weighted such as using location is twice as risky as financial data.

There could be and are some interesting relationships between the categories, for example, identifiers such as machine addresses (IPs) can be mapped into personal identifiers and locations - depending upon the use case.

I'm not going to go into a full formalisation of the function to calculate this, but a simple function which takes in a data set's fields and produces a value, say in the range 0 to 5 to state the risk of the data set might suffice. A second function to map that value to a set of requirements to handle that risk is the needed.

What about PII?  Well, to really establish this we should go into the contents of the data and the context in which that data exists. Another, rather brutal way, is to draw a boundary line across the above diagram such that things on the right-hand-side are potentially PII and those on the left not. This might then become a useful weighting metric, that if anything appears to the right of this line then the whole data set gets tagged with being potentially PII. I guess you could also become quite clever in using this division line to normalise the risk scoring across the various information classifications.

In summary, we can therefore give the term PII (or personal data) a definition in terms of what a data set contains rather than using it as a catch-all classification. This allows us then to have a proper discussion about risk and requirements.

References

Ian Oliver. Privacy Engineering: A Data Flow and Ontological Approach. ISBN 978-1497569713

On Finding Reasonable Measures To Bridge the Gap Between Privacy Engineers and Lawyers

This article originally appeared on the Privacy Association website on 29th July 2014.

On Finding Reasonable Measures To Bridge the Gap Between Privacy Engineers and Lawyers

Getting privacy lawyers and software engineers to work together to implement privacy is a perennial problem. Peter Swire, CIPP/US, and Annie Anton's article "Engineers and Lawyers in Privacy Protection: Can We All Just Get Along?" has explored this as has The Privacy Engineer's Manifesto, of which their article is part.

But here's the problem: We cannot keep addressing privacy from a top-down, legally driven perspective. No amount of additional processes and compliance checks is going to change the fact that software itself is so complex. Software engineering is often assumed to be the final stage and by some a mere consequence of many requirements from a large number of often conflicting sources.

Often privacy issues are “solved” by hashing an identifier, encrypting a communications link or anonymizing a data set, for some definition of anonymization. Many of these are piecemeal, ineffective, Band-Aid type solutions—the proverbial rearranging of deck chairs on the Titanic. Privacy Theater at its worst … or best.

So how do we address this?

First, maybe we should realize that we don't really understand each other’s disciplines. Very few lawyers are trained software engineers and vice-versa; therefore, constructing a lingua franca between these groups is part of that first step.

Often, understanding ideas that are simple in one domain do not translate to the other. For example, the use of the term “reasonable” means nothing in the software engineering domain. Plus, the simple act of “encryption” to a software engineer hides an enormous complexity of network protocols, key management, performance, encryption libraries and so on. Similarly, the now-ubiquitous use of “app” by lawyers to mean something that runs on your phone means a lot more to the software engineer.

What does “app” really mean to software engineers?

That little piece of code that downloads your news feed each morning and presents it in a friendly way—allowing you the scroll through using your touch screen device and maybe now and again presenting an advert because you didn't want to buy the paid version—is, in fact, not a tiny bit of code at all.

Effectively, what runs on your device is a multi-layered, complex interaction of hundreds of components, each sharing the device's memory, long-term storage, network components, display, keyboard, microphone, camera, speaker, GPS and so on. Each of those individual components interacts with layers underneath the interface, passing messages between components, scheduling when pieces of code must run and which piece of code gets the notification that you've just touched the screen as well as how and where data is stored.

When the app needs to get the next news item “from the web,” we have a huge interaction of network protocols that marshal the contents of the news feed, check its consistency, perform error-correction, marshal the individual segments of message from the network, manage addressing messages and internal format. There are protocols underneath this that decide on the routing between networks and those that control the electrical pulses over wires or antennae. This itself is repeated many, many times as messages are passed between routing points all over the Internet and includes cell towers, home wireless routers, undersea cables and satellite connections. Even the use of privacy-preserving technologies—encryption, for example—can be rendered virtually meaningless because underneath lies a veritable treasure trove of metadata. And that is just the networking component!

Let's for the moment look at the application's development itself, probably coded in some language such as Java, which itself is a clever beast that theoretically runs anywhere because it runs on top of a virtual machine and can be ported to almost any computing platform. The language contains many features to make a programmer's life easier by abstracting away complex details of the underlying mechanisms of how computers work. These are brought together through a vast library of publicly available libraries for a plethora of tasks—from generating random numbers to big data analytics—in only a 'few' lines of well-chosen code.

Even without diving deep into the code and components of which it’s made, similar complexity awaits in understanding where our content flows. Maybe your smartphone’s news app gets data from a news server and gives you the opportunity to share it through your favorite social network. Where does your data flow after these points? To advertisers, marketers, system administrators and so on? Through what components? And what kinds of processing? How about cross-referencing? Where is that data logged and who has access?

Constructing a full map or data flow of a typical software system—even a simple mobile app—becomes a spider web of interacting information flows, further complicated by layering over the logical and physical architectures, the geographical distribution and concepts such as controller and processor. Not to mention the actual content of the data, its provenance, the details of its collection, the risks, the links with policies and so on. And yet we still have not considered the security, performance and other functional and nonfunctional aspects!

Software engineers have enough of a problem managing this complexity without having to learn and comprehend legal language, too. Indeed, privacy won't get a foothold in software engineering until a path from “reasonable” can be traced through the myriad requirements to those millions of lines of code that actually implement “reasonable.”

Engineers love a challenge, but often when solutions are laid out before them in terms of policy and vague concepts—concepts which to us might be perfectly reasonable (there's that word again!)—then those engineers are just going to ignore or, worse, mis-implement those requirements. Not out of any malicious intent but because those requirements are almost meaningless in their domain. Even concepts such as data minimization can lose much if not all of their meaning as we move through the layers of code and interaction.

Life as a software engineer is hard: juggling a complex framework of requirements, ideas, systems, components and so on without interpreting what a privacy policy actually means across and inside the Internet-wide environment in which we work.

Software isn't a black box protected by a privacy policy and a suite of magic privacy-enhancing technologies but a veritable Pandora's Box of who-knows-what of “things.”

As privacy professionals, we should open that box often to fully comprehend what is really going on when we say “reasonable.” As software engineers, we'll more than make you welcome in our world, and we'd probably relish the chance to explore yours. But until we both get visibility in our respective domains and make the effort to understand each other's languages and how these relate to each other, this isn't going to happen—at least not in any “reasonable” way
.

How much data?!?!

I took part as one of the speakers in a presentation about analytics today; explaining how data is collected through instrumentation of applications, web pages etc, to an audience who are not familiar with the intricacies of data collection and analytics.

We had a brief discussion about identifiers and what identifiers actually are which was enlightening and hopefully will have prevented a few errors later on. This bears explaining briefly: an identifier is rarely a single field, but should be considered any one of the subsets of the whole record. There are caveats there of course, some fields can't be used as part of some compound identifier, but the point here was to emphasis that you need to examine the whole record not just individual fields in isolation.

The bulk of the talk however introduced from where data comes from. For example if we instrument an application such that a particular action is collected, then we're not just collecting an instance of that action but also whatever contextual data provided by the instrumentation and the data from the traffic or transport layer. This came as a surprise that there is so much information available via the transport/traffic layers:

Said meta-data includes location, device/application/session identifiers, browser and environment details and so on, and so on...

Furthermore data can be cross-referenced with other data after collection. A canonical example is geolocation over IP addresses to provide information about location. Consider the case where a user switches off the location services on his or her mobile device; location can still be inferred later in the analytics process to a surprisingly high-level of accuracy.

If data is collected over time, then even though we are not collecting specific latitude-longitude coordinates we are collecting data about movements of a single, unique human being; even though no `explicit' location collection seems to be being made. If you find that somewhat disturbing, consider what happens every time you pay with a credit card or use a store card.

Then of course there's the whole anonymisation process where once again we have to take into consideration not just what an identifier is, but the semantics of the data, the granularity etc. Only then can we obtain an anonymous data set. Such a data set can be shared publicly...or maybe not as we saw in a previous posting.  

Even when one starts tokenising and suppressing fields, the k-anonymity remains remarkably low, typically with more than 70% of the records remaining unique within that dataset. Arguments about the usefulness of k-anonymity notwithstanding - on the other hand it is one of the few privacy metrics we have,

So, the lesson here is rather simple, you're collected a massive amount more than you really think.

The next surprise was how tricky or "interesting" this becomes when developing a privacy policy that contains all the necessary details about data collection, meta-data collection, traffic data collection; and then the uses to which that data is put, whether it is primary or secondary collection and so on.

Surgical privacy: Information Handling in an Infectious Environment

What has privacy engineering, data flow modelling and analysis got to do with how infectious materials and the sterile field are handled in medical situations?  Are there things we can learn by exploiting by drawing an analogy between these seemingly different fields?

Surgical privacy: Information Handling in an Infectious Environment from Ian Oliver

We've discussed this subject earlier and a few links can be found here. Indeed privacy engineering has a lot to learn from analogous environments such as aviation, medicine, anaesthesia, chemical engineering and so on; the commonality here is that those environments understood they had to take a whole systems approach rather than relying upon a top-down driven approach or relying upon embedding the semantics of the area in one selected discipline.

Foundations of Privacy - Another Idea

This got triggered by a post on LinkedIn about what a degree in privacy might contain. I've certainly thought about this before, at least in terms of software engineering, and even have a whole course that could be taken over a semester ready to go.

Aside: CMU has the "World's First Privacy Engineering Course": a Master of Science in Information Technology—Privacy Engineering (MSIT-PE) degree. So, close, but a major university here in Finland turned down the chance to create something similar a few years back...

That aside, I've been wondering about how to present they various levels of things we need to consider to properly define privacy and put it on strong foundations. Though in the guise of information theory we already have this, though admittedly Shannon's seminal work from the 1930's is maybe a little too deep. On the other hand understanding concepts such as channels, entropy are fundamental building blocks, so maybe they should be there along with privacy law - now that would make some course!

Even just sketching out areas to present and what might be contained therein...how about this, even if a linear map from morality to mathematics is too constraining?

There are missing bits - we still have a  semantic gap between the "legal world" and the "engineering world"; parts that I'm hoping that things such as the many conferences, academic works and books such as the excellent Privacy Engineer's Manifesto and Privacy Engineering will play a role in defining. Maybe the semantic gap goes away once we start looking at this...is there even a semantic gap? 

However, imagine for a moment starting anywhere in this stack and working up and down and keeping everything linked together in the context of privacy and information security. Imagine seeing the link between EU privacy laws and type theory, or between the construction of policies and entropy, the algebra of HIPAA, a side course in homotopy type theory and privacy...maybe with that last one I'm getting carried away, but, this is exactly what we need to have in place.

Each layer provides the semantics to the layer above - what do our morals and ethics means in terms of formalised laws, what do laws mean in terms of policies, what do policies mean in terms of software engineering structures, and down to the core mathematics and algebras of information.

Privacy and privacy engineering in particular almost has everything: law, algebra, morals, ethics, semantics, policy, software, entropy, information, data, BigData, Semantic Web etc etc etc. Furthermore, we have links to areas such as security, cryptography, economic theory etc!

Aren't these the very things any practitioner of privacy (engineering) should know, or at least have knowledge of? Imagine if lawyers understood information theory and semantics, and, software engineers understood law? 

OK, so there might be various ways of putting this stack together, competing theories of privacy etc, but that would be the real beauty here - a complete theory of privacy from the core mathematics through physics, computation, type theory, software engineering, policies, law and even ethics and morals.

But again, no more naivety, no more terminological or ontological confusions, policies and laws being traceable right down to the computation structures and code. Quite a tall order, but such a course bringing all these together really would be wonderful...

And wouldn't that be something!

Legal Document Humour

I know software engineers and legal speak different languages, but...

Daniel Solove (2006) A Taxonomy of Privacy. University of Pennsylvania Law Review. 154(3)

Not quite as naive about privacy as before

A while back I wrote an article on how were we being naive about privacy in that we're all talking about the subject but no-one seems to be actually asking the question of what it is.

In order to answer this we've* taken an ontological approach by decomposing concepts into things such as information type, security class, jurisdiction, purpose, usage, provenance etc. All those concepts which make sense to the engineers who have to build information systems.

*Ora Lassila (he of RDF fame) has had a pretty big (huge!) hand in all of this too. Hey! We even got demonstration and prototype implementations working!

No work like this is even done in isolation - ontological approaches aren't new and certainly security, privacy, risk management etc have been tackled in one way or another - Solove, Schneier just to name two big names and a host of other researchers along too.

Now this is where I have a lot of hope: there is quite a bit a work in this area - that is, formalising concepts of privacy and in particular risk and risk avoidance in this ontological manner. There's even work on matching ontologies together. We start to see the real, fundamental structure of privacy and its conceptual basis.

What this means in the long term (and even the short!) is that we have a common terminological and semantic framework from lawyers to programmers coming into place.

We're missing some parts of course: how do all these ontologies fit together?  Can we unify the notions of consent used by lawyers with the [boolean] data types used by programmers?

"Your privacy is important to us"

bool optedIn = False  //sensible default

Actually we do in part - myself and Ora did develop quite a nice unification framework to link the ontologies together, link with the idea of information, link it with the notions of database table, CSV structures, classes etc; and even link it with how systems such as HADOOP process data.

So this gets me to a few places:

1. There is work on this being made - various groups are developing ontologies to express relevant concepts about information and aspects of information
2. Some groups are unifying those and drawing out subtle semantic differences
3. Some groups are applying these to more abstract areas such as the notions of consent and notice and how these may be made more meaningful to machines, and I hope humans too

We're starting to fill in the technical, fundamental details of what privacy is and now I find myself working with how we link these structures with the ideas of data flow modelling, risk management and software lifecycle requirements which requires all of the above.

References

Cena, Dokoohaki, Matskin. (2011) Forging Trust and Privacy with User Modeling Frameworks: An Ontological Analysis. STICS2011 (Draft)

Anya Kim and Jim Luo and Myong Kang (2005) Security Ontology for Annotating Resources. Research Lab, NRL Memorandum Report, pp51.

Kost, Freytag, Kargl, Kung. Privacy Verification using Ontologies

Golnaz Elahi, Eric Yu, Nicola Zannone (2009) A Modeling Ontology for Integrating Vulnerabilities into Security Requirements Conceptual Foundations?

Security and Privacy Ontology

Structuring Privacy Requirements pt 1

One of the toughest problems I'm having to solve, not just for my book on privacy engineering, but in my daily job as well is to formulate a set of privacy requirements to the software engineers and the R&D teams.

Actually it isn't that the privacy requirements themselves are hard, we've plenty at the policy level and extrapolating these down into the more functional and security related requirements at the implementation level (ok, it is difficult but there are harder things in this area).

Placing all these in a structure that ties together the various classifications and aspects of information, data flow modelling, requirements and risk management has been interesting and fiendishly difficult to say the least. Getting this structure into a state that supports all of these and getting the semantics of the kinds of thing it is structuring is critical to understanding how all of this privacy engineering works.

We assume that we understand the classification systems, for example the almost traditional Secret-Confidential-Public style security classifications and the Location, Time etc classifications of information type; as well as the other aspects such as purpose, usage, provenance, identity etc. Each of these has its own set of classification elements, hierarchy and other ontological structure. For example for information type:

Information Type Hierarchy

We also assume we understand data flow modelling with its processes, flows, sources, sinks and logical partitionings. We can also already see the link between elements here (as shown below) and the classification systems above

Example Data Flow with Annotations from Previously Described Ontologies/Classification Systems

Now the structure of our requirements needs to take into consideration the various elements from the classification systems, the aspect of the requirement we want to describe (more on this below) and the required detail of requirement relevant to the stage in the software process. This gives us the structure below:

So if we wish to find the requirements for User's Home Address Policy for Local Storage then we take the point corresponding to those coordinates. If there happens to be nothing there then we can use the classification systems' hierarchies to look for the requirement corresponding to a parent; a "user's home address" is-a "Location":

So if we take the example data flow from earlier then for each of the flows, storages and processes we can construct a set of requirements simply by reading off the corresponding requirements from the structure above.

This leads to an interesting situations where it is possible to construct a set of requirements which are overconstraining. That is simply that we can not build a system that can support everything, for example, one data point might trigger a secret classification and key management, encrypted content etc.

We then need to weaken the requirements such that a system can be constructed according to some economic(!) model. As we weaken or retrench our requirements we are introducing risk into the system,

Aside: Retrenchment - here's a good reference: Banach, Poppleton. Controlling Control Systems: An Application of Evolving Retrenchment. ZB 2002:Formal Specification and Development in Z and BLecture Notes in Computer Science Volume 2272, 2002, pp 42-61.

This gives us our link to risk management and deciding whether each "weakness" we introduce is worth the risk. And as risk can be quantified and we can perform further tasks such as failure mode and effects analysis then we obtain a rigorous method to predict failures and make informed decisions about these. I'll describe more on this in later articles.

Semantics Rant

Worthy of The Daily WTF, and a common occurence when machine types are mixed up with the everyday "types" of things.

Telephone numbers are NOT integers, and money amounts are NOT floating point numbers

Even though the machine types look like these an examination of the ITU-T E.123 document will more than convince that telephone numbers are not integers (or more specifically long unsigned integers). Further proof is that the mathematics operations of addition, subtraction are not defined. Indeed the only operation is equality between two numbers, plus a few other operations to extract different parts, eg: area code, country code etc.

Money amounts similarly: floating point numbers or even decimals have very different properties to the representation and storage of money amounts. In particular the problem of rounding (see also Superman 3 for a humorous take on this). What makes this worse is that support is varying, for example in SQL there are Currency and Money types but these are not supported across all RDBMs, or not provided at all. As Martin Fowler has pointed out, in object-oriented languages this is easily rectified with the provision of a class for money and himself provides the pattern.

Now I feel better...

On being formal, and possibly agile too...

Way back in my past I used to research formal methods for software engineering. Actually I still do, though now most of my time is actually using formal methods to make better software.

Formal methods are nothing more than a collection of languages and techniques for modifying and reasoning about things (models) written in those languages. Some of these techniques encompass how the process of building a system is made. Herein lies one of the first problems encountered by formal methods practitioners and that is the almost constant challenge from some, such as many in the agile community, that seem to be religiously against any form of modelling.

To those who believe that code is the only deliverable and the only thing that matters, well, C++, C, Fortran, Clojure etc are all formal languages, and you're probably using many of the techniques from formal methods right now as your write your code.

Language such as B, Alloy, Z, VDM etc do is provide a method of expressing a model without worrying about certain, awkward details of their implementation or execution.

Indeed what is happening here is that we have languages and techniques that allow you to concentrate on reasoning and thinking about the problem you are trying to solve without getting bogged down in the details of the final implementation.

If your first worry is the implementation language, or the operating system, or which libraries to use, etc, then you're most certainly not solving the problem.

At Nokia we had some very great successes using formal methods in an agile manner for the development of a semantic web infrastructure for the "Internet of Things". Concentrating on what the system had to do and then later worrying about how it was implemented meant that when it did come to the time we needed to architect the components and decide on specific implementation issues we already knew how the system was going to work, what tests we would need to run and what the expected answers were going to be.
Indeed, many of the tests were little more than checking that the code behaved the same as our earlier models - regression testing if you like.

This resulted in a huge decrease in the time spent in coding and the effective removal of nearly all (logic) bugs before even the beta releases. In fact most of the bugs turned out to by typos.

Furthermore, when it came to updating the software with additional features, instead of blindly bolting on a new use case we could reduce most of the new feature requests down to library or convenience functions over the core functionality rather than complicating the design with those "new" features.

This latter point is very important in that even though there is pressure to constantly add new features  (and the Pareto Principle applies here), most new features are really just convenience functions that already exist in the software.

I even remember one system where management demanded so many new features (all specified as their own use cases) that the system actually ended up implementing not only the same feature many times but features to disable the requested feature...

Ultimately formal methods is a discipline of thinking, rather than any technique to develop software. Just as much as Agile is a discipline of development.

To use formal methods does not mean any form of top-down or Waterfall development, it does not mean that one has to use refinement or a language like B or Z or VDM or Alloy etc. Just the simple act of writing a precondition to a function in C, or expressing a simple class diagram in UML, or ER diagram to explain a database schema (SQL or NoSQL) to demonstrate the workings of a system, or to clarify what something mean IS being formal.

The best agile developers I have seen all have formal methods backgrounds. The reasoning is that they already have the discipline and education and the tools to think and reason about their system, even if applied implicitly. Agile depends upon great communication between the developers and the customers and giving those customers exactly what they need in a manner that avoids technical debt (viz. situational awareness).

Whether we like it or not, great software engineering comes from understanding how our craft works at its most fundamental levels - imagine civil engineering without the mathematics of physics (a classic example), or even ballet without an understanding of human movement?

References:

[1] Ian Oliver Experiences of Formal Methods in 'Conventional' Software and Systems Design. FACS 2007 Christmas Workshop: Formal Methods in Industry. BCS London, UK, 17 December 2007 

[2]  Ian Oliver Experiences of Formal Methods in 'Conventional' Software and Systems Design

Formal Methods in Conventional Software Development Environments from Ian Oliver