Privacy Metrics

Along with a colleage - Dr. Yoan Miche - we presented a paper outlining ideas regarding using mutual information as a metric for establishing some form of  'legal compliance' for data sets. The work is far from complete and the mathematics is getting horrendous!

The paper entitled "On the Development of A Metric for Quality of Information Content over Anonymised Data-Sets" was presented at the excellent Quatic 2016 conference held in Lisbon, Sept 6-9, 2016.

We were also extremely fortunate in that a presented in our session didn't turn up and we were graciously given the full hour not just to present the paper but give a much fuller background and details of future work and the state of our current results.

Here are the slides:

On The Development of A Metric for Quality of Information Content over Anonymised Data Sets from Ian Oliver

Abstract:

We propose a framework for measuring the impact of data anonymisation and obfuscation in information theoretic and data mining terms. Privacy functions often hamper machine learning but obscuring the classification functions. We propose to
use Mutual Information over non-Euclidean spaces as a means of measuring the distortion induced by privacy function and following the same principle, we also propose to use Machine Learning techniques in order to quantify the impact of said obfuscation in terms of further data mining goals.

Citation:

Ian Oliver and Yoan Miche (2016) On the Development of A Metric for Quality of
Information Content over Anonymised Data-Sets. Quatic 2016, Lisbon, Portugal, Sept 6-9, 2016.

2nd IW5GS - Programme

The 2nd International Workshop on 5G Security

IW5GS2016

Xi'an, China

June 19, 2016

http://www.mobimedia.org/2016/show/program-final

IW5GS-01: (June 19, 2016, Sunday, 10: 30 – 12:30, Room B)

Session Chair: Valtteri Niemi (Email: valtteri.niemi@helsinki.fi)

Keynote 1: 5G Security for IoT

Speaker: Dr. Zhiyuan Hu, Nokia Shanghai Bell  (zhiyuan.hu@alcatel-sbell.com.cn)

Keynote 2: 5G Security: Forward Thinking

Speaker: Bo Zhang, Huawei (liufei19@huawei.com)

IW5GS-02: (June 19, 2016, Sunday, 14: 00 – 16:30, Room B)

Session Chair: Siddharth Prakash Rao (Siddharth.rao@aalto.fi); Ian Oliver (ian.oliver@nokia.com)

Paper 1: Protecting IMSI and User Privacy in 5G Networks

Karl Norrman, Elena Dubrova, Mats Näslund

Paper 2: Privacy of the Long-Term Identities in Cellular Networks

Philip Ginzboorg, Valtteri Niemi

Paper 3: Error-Correcting Message Authentication for 5G

Elena Dubrova, Mats Näslund, Göran Selander, Karl Norrman

Paper 4: Privacy in LTE networks

Siddharth Prakash Rao, Bhanu Teja Kotte, Silke Holtmanns

Paper 5: A Survey on Software-Defined Networking Security

Shanshan Bian, Peng Zhang, Zheng Yan

Paper 6: Designing Hybrid Cloud Computing Framework using OpenStack for Supporting Multimedia with Security and Privacy

Isaac Cushman, Lei Chen, Danda B. Rawat, and Nhien-An Le-Khac

International Workshop on 5G Security - Programme

The 1st IEEE International Workshop on 5G Security held in conjunction with IEEE TrustCom-15

There is a fast on-going change in the technical architectures and topologies of the Internet: in the near future 5G and next generation 4G/LTE network architectures will be based on or migrated to Software Defined Networking (SDN) and Network Functions Virtualization (NFV). These create new virtual network elements each affecting the logic of the network operation, traffic management and introducing new and novel security challenges. Aspects such as security of orchestration, management functionality as well as surveillance and privacy are brought to the fore. At the same time they introduce new ways of dealing with attack prevention, management and recovery.

The one-day workshop will consist of papers, presentations and demonstrations on the subject of advanced network security. While primarily related to 5G networks, experiences from 4G/LTE, 3G and earlier, including case studies on practicalities of known attacks and novel attack vectors will be considered for acceptance. An invited keynote speech will be given setting out the overall area of security in network development and operations.

IW5GS Programme
TrustCom 2015 Helsinki, Finland
Friday 21, August 2015

10h05-11h50 – Session I

• Ian Oliver, Silke Holtmanns, Workshop Opening
• Günther Horn and Peter Schneider, Towards 5G Security
• Siddharth Prakash Rao, Silke Holtmanns, Ian Oliver and Tuomas Aura. Unblocking stolen mobile devices using SS7-MAP vulnerabilities
• Vikramajeet Khatri and Joerg Abendroth, Mobile Guard Demo - Network Based Malware Detection

13h10-14h55 – Session II

• Ian Oliver, Aspects of 5G Security
• Nicolae Paladi and Christian Gehrmann. Towards Secure SDN-based Multi-tenant Virtualized Networks
• Elena Dubrova, Mats Näslund and Göran Selander. CRC-Based Message Authentication for 5G Mobile Technology
• Prajwol Kumar Nakarmi, Oscar Ohlsson and Michael Liljenstam. An Air Interface Signaling Protection Function for Mobile Networks: GSM Experiments and Beyond

15h15-17h00 – Session III

• Bengt Shalin, keynote talk
• Mingjun Wang and Zheng Yan. Security in D2D Communications: A Review
• Karl Norrman, Mats Näslund, Bengt Sahlin and Jari Arkko. A USIM compatible 5G AKA protocol with perfect forward secrecy
• Silke Holtmanns,Ian Oliver, Workshop Closing

Loyalty Cards

Tangentially related to privacy which I'll discuss later regarding information asymmetry, but first I wanted to comment on this article that appeared on LinkedIn: Business Travelers Are Saying "Buh-bye" to Loyalty Programs - Here's Why by Christopher Elliott, a "Reader advocate" for National Geographic Traveler

"After years of putting up with blackout dates, broken promises and bait-and-switch games, American travellers — particularly air travellers — are saying “Enough!”
They’re refusing to play the loyalty-program game, jettisoning blind brand allegiance in favor of a more pragmatic view of travel. Price and convenience are trumping mindless devotion to an airline, a car rental company or a hotel."

This applies equally to European travellers to I suspect. But let's look at the price-quality-customer service dimensions. I did as the article states a few years ago and switched allegiance from one to another, despite accumulating a sizeable amount of points and occasionally using them to upgrade it actually became too difficult to actually use those points.

I remember a few cases clearly. The first was trying to enter a business lounge at Helsinki Airport - according to the "loyalty" scheme terms and conditions I had enough flights to qualify for an upgrade to the next tier, thus allowing me access to the business lounge. Except that there was a mistranslation between the English translation and the "correct" terms and conditions. The airline customer service basically stated it wasn't their fault. Given that the lounge at the time was empty, but staffed by no less than three "customer representatives" make it all a little too surreal.

The second was at Heathrow when lounge entry was not just dependent on the card but on the airline that issued to card, despite both airlines being part of the same alliance. It worked like this: if you gained the second tier on airline A you could enter business lounges run by the alliance that A was a member of, except, if the lounges were at an airport where airline B, also of the same alliance, was the major carrier.

The third was when trying to book a flight with air miles. For my selected flight I had enough miles and successfully made the booking until it came to select my seats, where upon the whole process failed and I was unceremoniously ditched out of the booking process with a terse error about the flight no longer being available to points holders. It turns out this was an artefact of the internal IT systems and actually you had to call the customers service department to book the flights - after a long hold where you were constantly told that all of the customer service representatives were busy, but you really are valued by the company you're trying to spend you money with.

After a few flights across the Atlantic I switched to Lufthansa - Frankfurt and Munich are clean, efficient and as pleasant as airports go. Lufthansa may not have good economy seating on long haul but the crews are some of the most professional (and smiling!) I've had the pleasure to be on an aircraft with.

I actually have no idea actually how to redeem the points I've collected with Lufthansa and SAS - they'll probably expire or have expired without my knowledge without me ever receiving any bonus from my loyalty as a customer.

A few years ago a colleague of mine called a major Nordic airline in response to a request for customer feedback. It turns out that they couldn't care why he stopped flying with them. Surely with all the data analytics being applied to our data collected, the airline never stopped for a moment to question why customers suddenly stopped flying with them. 

Now due to primarily economic reasons, I fly with whoever gives me the best deal. Given that the terms and conditions are pretty much the same regardless whether you fly a national carrier or a "cheap" airline, price will always win. 

Given that many established, major airlines decided to compete with the cheap carriers by matching their standards and terms, it is no wonder that the loyalty of the customer to the airline has all but disappeared?

Then, we come to credit cards and store cards...will these go the same way as air miles as customers realise that the deal they get is barely worth the loyalty?

But as a final statement, given that this is all about customer loyalty - has anyone ever been contacted by an airline, shop or other about why they stopped using those products offered?

References:

"The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2001: Information for the Public", press release from the Royal Swedish Academy of Sciences, Nobel Foundation, nobelprize.org, October 2001

Surgical privacy: Information Handling in an Infectious Environment

What has privacy engineering, data flow modelling and analysis got to do with how infectious materials and the sterile field are handled in medical situations?  Are there things we can learn by exploiting by drawing an analogy between these seemingly different fields?

Surgical privacy: Information Handling in an Infectious Environment from Ian Oliver

We've discussed this subject earlier and a few links can be found here. Indeed privacy engineering has a lot to learn from analogous environments such as aviation, medicine, anaesthesia, chemical engineering and so on; the commonality here is that those environments understood they had to take a whole systems approach rather than relying upon a top-down driven approach or relying upon embedding the semantics of the area in one selected discipline.

Foundations of Privacy - Another Idea

This got triggered by a post on LinkedIn about what a degree in privacy might contain. I've certainly thought about this before, at least in terms of software engineering, and even have a whole course that could be taken over a semester ready to go.

Aside: CMU has the "World's First Privacy Engineering Course": a Master of Science in Information Technology—Privacy Engineering (MSIT-PE) degree. So, close, but a major university here in Finland turned down the chance to create something similar a few years back...

That aside, I've been wondering about how to present they various levels of things we need to consider to properly define privacy and put it on strong foundations. Though in the guise of information theory we already have this, though admittedly Shannon's seminal work from the 1930's is maybe a little too deep. On the other hand understanding concepts such as channels, entropy are fundamental building blocks, so maybe they should be there along with privacy law - now that would make some course!

Even just sketching out areas to present and what might be contained therein...how about this, even if a linear map from morality to mathematics is too constraining?

There are missing bits - we still have a  semantic gap between the "legal world" and the "engineering world"; parts that I'm hoping that things such as the many conferences, academic works and books such as the excellent Privacy Engineer's Manifesto and Privacy Engineering will play a role in defining. Maybe the semantic gap goes away once we start looking at this...is there even a semantic gap? 

However, imagine for a moment starting anywhere in this stack and working up and down and keeping everything linked together in the context of privacy and information security. Imagine seeing the link between EU privacy laws and type theory, or between the construction of policies and entropy, the algebra of HIPAA, a side course in homotopy type theory and privacy...maybe with that last one I'm getting carried away, but, this is exactly what we need to have in place.

Each layer provides the semantics to the layer above - what do our morals and ethics means in terms of formalised laws, what do laws mean in terms of policies, what do policies mean in terms of software engineering structures, and down to the core mathematics and algebras of information.

Privacy and privacy engineering in particular almost has everything: law, algebra, morals, ethics, semantics, policy, software, entropy, information, data, BigData, Semantic Web etc etc etc. Furthermore, we have links to areas such as security, cryptography, economic theory etc!

Aren't these the very things any practitioner of privacy (engineering) should know, or at least have knowledge of? Imagine if lawyers understood information theory and semantics, and, software engineers understood law? 

OK, so there might be various ways of putting this stack together, competing theories of privacy etc, but that would be the real beauty here - a complete theory of privacy from the core mathematics through physics, computation, type theory, software engineering, policies, law and even ethics and morals.

But again, no more naivety, no more terminological or ontological confusions, policies and laws being traceable right down to the computation structures and code. Quite a tall order, but such a course bringing all these together really would be wonderful...

And wouldn't that be something!

Classifying Data Flow Processes

In previous posts (here, here, here and especially here) we presented various aspects for classifying data nominally being sent over data flows or channels between processes. Now we turn to the processes themselves and tie the classification of channels together.

Consider the data flow shown here (from the earlier article on measurement, from where we take the colour scheme of the flows) where we show the movement of, say, location information:

Notice how data flows via a process called "Anonymisation" (whatever that word actually means!) to the advertiser. During the anonymisation the location data is cleaned to a degree that business or our privacy policy allows - such boundaries are very context dependent.

This gives the first type of process, one that reduces the information content.

The other type of processes we see are those that filter data and those that combine or cross-reference data.

Filtering is where data is extracted from an opaque or more complex set of information into something simpler. For example if we feed some content, eg: a picture, then we might have processes that filter out the location data from said pictures.

Cross-referencing means that two or more channels are combined to produce a third channel containing information from the first two. A good example of this is geo-locating IP addresses which takes in as one source an IP address and another a geographical location look-up table. Consider the example below:

which combines data from secondary and primary sources to produce reports for billing, business etc.

In the above case we probably wish to investigate the whole set of processes that are actually taking place and make a considerable amount of decomposition of the process.

When combined with the classifications on the channels, particularly information and security classes we can make some substantial reasoning. For example, if there is a mismatch in information content and/or security classifications then we have problems; similarly if some of these are transported over insecure media.

To summarise, in earlier articles we explained how data itself may be classified, and here how processes may be classified according to a simple scheme:

• Anonymising
• Filtering
• Cross-Referencing

In a later article I'll go more into the decomposition and refinement of channels and processes.

Measurement and Metrics for Information Privacy

We have already discussed an analogy for understanding and implicitly measuring the information content over a data channel. The idea that information is an "infectious agent" is quite a powerful analogy in the sense that it allows us better to understand the consequences of information processing and the distribution of that data better, viz:

• Any body of data which is containing certain amounts and kinds of sensitive data we can consider to be non-sterile
• Information which is truly anonymous is sterile
• Mixing two sets of information produces a single set of new information which is as at least as unclean as the dirtiest set of data mixed, and usually more so!
• The higher the security classification the dirtier the information

Now we can using our information classification earlier introduced we can further refine our understanding and get some kind of metric over the information content.

Ley us classify information content into seven basic categories: financial, health, location, personal, time, identifiers and content. Just knowing what kinds of data are present as we have already discussed gives us a mechanism to pinpoint where more investigation is required.

We can then go further an pick out particular data flows for further investigation and then map this to some metric of contamination:

For example, transporting location data has a particular set of concerns, enough to make any privacy professional nervous at the least! However if we examine a particular data flow or store we can evaluate what is really happening, for example, transmitting country level data is a lot less invasive than highly accurate latitude and longitude.

Now one might ask why not deal with the accurate data initially? The reasons are that we might not have access to that accurate, field-level data, we might not want to deal with the specifics at a given point in time, specific design decisions might not have been made etc.

Furthermore, for each of the seven categories we can give some "average" weighting and abstract away from specific details which might just complicate any discussion.

Because we have a measure, we can calculate and compare over that measurement. For example, if we have a data channel carrying a number of identifiers (eg: IP, DeviceID, UserID) we can take the maximum of these as being indicative of the sensitivity of the whole channel for that aspect.

We can compare two channels, or two design decisions, for example, a channel carrying an applicationID is less sensitive (or contaminated) than one carrying device identifiers.

We also can construct a vector over the whole channel composed out of the seven dimensionsb above to give a further way of comparing and reasoning about the level of contamination or sensitivity:

| (FIN=0,HLT=0,LOC=10,PER=8,TIM=3,ID=7,CONT=0) | 
   < 
| (FIN=3,HLT=2,LOC=4,PER=4,TIM=2,ID=9,CONT=2) |

for some numerical values gien to each category. Arriving at these values will be specific to a given wider context and then the weighting given to each, but there is one measure which can be used to ground all this, and that is of information entropy, or, how identifiying the contents are to a given, unique human being. A great example of this is given at the EFF's Panopticlick pages.

We've only spoken about a single data flow at the moment, however the typical scenario is for reasoning over longer flows, for example, we might have our infrastructure set up as below*

In this example we might look at all the instances where AppID and Location are specified together and use a colour coding such that:

• Black: unknown/irrelevant
• Red: high degree of contamination, both AppID and Location unhashed and accurate respectively
• Yellow: some degree of contamination, AppID may be hashed(+salt?) or Location at city level or better
• Green: AppID randomised over time, hashed, salted and Location at country level or better

Immediately readable from the above flow are our points of concern which need to be investigated, particular the flows from the analytics processing via the reports storage and to direct marketing. It is also easy to see that there might be a concern with the flow to business usages, what kinds of queries ensure that the flow here is less contaminated than the actual reports storage itself?

There are a number of points we have not yet discussed, such as that some kinds of data can be transformed into a different type. For example some kinds of content such as pictures inherently contain device identifiers, locations etc. Indeed the weighting for some categories such as content might be very much higher than that of identifiers for example - unless the investigation is made. Indeed it does become almost a trivial exercise for some to explicitly hide sensitive information inside opaque data such as images and not declare then when a privacy audit is made.

To summarise, we've a simple mechanism for evaluating information content over a system in quantative terms, complete with a refinement mechanism that allows us to work at many levels of abstraction depending upon situation and context. Indeed, what we are doing is explcitly formalising and externalising our skills when performing such evaluations, and through analogies such as "infection control" providing means for easing the education of professionals outside of the privacy area.

*This is not indicative of any system real, living or dead, but just an example configuration.

Diagrams Research

For a number of years I and some colleagues have worked closely with the University of Brighton's Visual Modelling Group using their work on diagrammatic methods of modelling and reasoning. One of the areas where we've had quite a nice success is in modelling aspects of information privacy [1] with some particularly useful and beautiful and natural representations of complex ideas and concepts.

Another area has been in the development of ontologies and classification systems - something quite critical in the area of information management and privacy. Some of this dates back to work we made with the M3 project and the whole idea of SmartSpaces incorporating the best of the Semantic Web, Big Data etc.



We've gained quite a considerable amount of value out of this relatively, simple industrial-academic partnership. A small amount of funding, no major dictatorial project plans but just letting the project and work develop naturally, or even if you like, in an agile manner, produces some excellent, useful and mutually beneficial results.

Indeed not having a project plan but just a clearly defined set of things that we need addresses and solved (or just tackled - many minds with differing points of view really does help!) means that both partners: the industrial and the academic, can get on with the work rather than battling an artificial project plan which becomes increasingly irrelevant and industrial focus and academic ideas change over time. Work continues with more ontology engineering in the OntoED project.

References:

1. I. Oliver, J. Howse, G. Stapleton. Protecting Privacy: Towards a Visual Framework for Handling End-User Data. IEEE Symposium on Visual Languages and Human-Centric Computing, San Jose, USA, IEEE, September, to appear, 2013.
2. I. Oliver, J. Howse, G. Stapleton, E. Nuutila, S. Torma. Visualising and Specifying Ontologies using Diagrammatic Logics. In proceedings of 5th Australasian Ontologies Workshop, Melboune, Australia, CRPIT vol. 112, December, pages 37-47, 2009. Awarded Best Paper
3. J. Howse, S. Schuman, G. Stapleton, I. Oliver. Diagrammatic Formal Specification of a Configuration Control Platform. 2009 Refinement Workshop, pages 87-104, ENTCS, November, 2009.
4. I. Oliver, J. Howse, G. Stapleton, E. Nuutila, S. Torma. A Proposed Diagrammatic Logic for Ontology Specification and Visualization. 8th International Semantic Web Conference (Posters and Demos), October, 2009.
5. J. Howse, G. Stapleton, I. Oliver. Visual Reasoning about Ontologies.International Semantic Web Conference, China, November, CEUR volume 658, pages 5-8, 2010.
6. P. Chapman, G. Stapleton, J. Howse, I. Oliver. Deriving Sound Inference Rules for Concept Diagrams. IEEE Symposium on Visual Languages and Human-Centric Computing, Pittsburgh, USA, IEEE, September, pages 87-94, 2011.
7. G. Stapleton, J. Howse, P. Chapman, I. Oliver, A. Delaney. What can Concept Diagrams Say? Accepted for 7th International Conference on the Theory and Application of Diagrams 2012, Springer, pages 291-293, 2012.
8. G. Stapleton, J. Howse, P. Chapman, A. Delaney, J. Burton, I. Oliver.Formalizing Concept Diagrams. 19th International Conference on Distributed Multimedia Systems, International Workshop on Visual Languages and Computing, Knowledge Systems Institute, to appear 2013.

Modelling Data Flow and Information Channels

Before we delve further into policies and more analysis of our models, I want to first take a small detour and look at the data channels in our models. We earlier explained that we could refine the various classifications on our channels down to fine grained rules, this is one kind of refinement. We can also refine the channel structure itself to make the various "conversations" between components clear.

Firstly, what is a channel? There's the mathematical explanation taken from [1]:

An information channel consists of an indexed family C = { f_i : A_i <-> C} i\in I of infomorphisms with a common codomain C, called the core of the channel.

Phew! Or a more everyday description that an information channel is the conversation between two elements such as persons, system components, applications, servers etc.

We also note that conversations tend to be directed in the direction that the information flows. We generally don't model the ack/nack type protocol communications.

Starting with our model from earlier:

We should and can refine this to explicitly distinguish the particular conversations that occur between the application and the back end.

While the two components communicate all this information, maybe even over the same implementation, we might wish to explicitly distinguish between the primary and secondary information flows. The reason could be due to differing consent mechanisms, or differing processing on the receiving end etc.

In the above we are explicitly denoting two different conversations. These conversations are logically separate and from the information usage point of view.

As we decompose our channels into the constituent, logically separate conversations we are also making decisions about how the system should keep apart the data transported over those conversations. Whether this translates into physical separation or however the logical differentiation is made is an architectural issue modelled and decided elsewhere.

As we shall see later when we decompose the processing elements in our data flow model we can track the flows and where those flows meet, diverge, cross-reference and infer points of possible contamination.


References:

[1] Barwise and Seligman. (1997) Information Flow. Cambridge University Press.

Information as an Infectious Agent

Operating theatres are split into two parts:

• the sterile field
• the non-sterile surroundings

Any non-sterile item entering the sterile field renders it non-sterile; and stringent efforts and protocols [1,2] are made to ensure that this does not happen.

The protocols above extend via simply analogy [3,4] to information handling and information privacy.

• Any body of data which is containing certain amounts and kinds of sensitive data we can consider to be non-sterile - assume for a moment that certain bits and bytes are infectious (great analogy!).
• Everyone working with information is required to remain sterile and uncontaminated.
• Information which is truly anonymous is sterile
• Mixing two sets of information produces a single set of new information which is as at least as unclean as the dirtiest set of data mixed, and usually more so!
• The higher the security classification the dirtier the information

We can extend this latter point to specific information types, eg: location, personal data, or certain kinds of usages and purposes, eg: data for advertising or secondary data and so on.

Extending our analogy further we can protect the sterile field in two ways:

• ensuring that everyone in contact with the sterile field is sterile
• ensuring that the equipment entering the sterile field is sterile

Basic data handling techniques are essential:

• If two sets of data are to be mixed then ensure that the mixing occurs not in-situ but by generating a third data set kept separate from the two input sets
• Data can be made more sterile by removing information content. But, be warned that certain kinds of obfuscation are not effective, eg: hashing or encryption of fields might just hide the information content of that field but not the information content of the whole data set [3]
• Keep sterile and non-sterile data-sets apart, physically if possible
• Ensure that sterile and non-sterile data-sets have differing access permissions. Ideally different sets of people with access
• Clean up after yourself: secure data deletion, overwriting of memory, cache purges etc.

From a personnel point, in surgery precautions are made through restricting the persons inside the sterile field and even outside of this, basic precautions are taken in terms of protective clothing etc. While surgical attire might be overkill for office environments, the analogy here is that personnel with access to data have received the correct training and are aware of what data they can and can not use for various purposes.

In a surgical environment, everything entering and leaving the sterile field is checked and recorded. In an information systems environment this means logging of access so that when a breach of the sterile field occurs the route of the pathogen and its nature can be effectively tracked and cleaned.

References

[1] Infection Control Today - August 1, 2003 : Guidelines for Maintaining the Sterile Field
[2] Infection Control Today - November 7, 2006 - Best Practices in Maintaining the Sterile Field

[3] What has surgery got to do with information privacy?

[4] Flying planes, Surgery and Privacy - the presentation

Analysing Data Flow Models

In the previous post we introduced a pattern for the data-flows in and out of an application such as those found on your mobile phone or tablet. In this posting I want to expand on this pattern and explore various annotations to help us reason about how our application treats the information flowing through it.

Let's first introduce an example application, a photo sharing application. This app allows you to select a photo on your device and upload it to a social media account. Here's what it looks like to the user on a mobile device with rounded corners (as opposed to square corners which none of them seem to have):

It looks innocent enough but as we know there are many threats to user privacy even in the most innocent looking of places. So let's model what is really happening behind the scenes. We understand so far a number of things: the user supplies content and credentials for the services, these are stored locally for convenience, the app adds meta-data to the picture before uploading and sends information about the app's behaviour to the inventor of the app. We might then construct the following model:

On each of the dataflows we have noted the kind of information transported over those channels and the mechanism of communication. We also note our presumed trust boundary.

What this shows clearly is where data is flowing and by what means. We have for now skipped over the specific or precise meanings of some things hoping that the terms we have used are self-explanatory.

But, now we have formally written we can focus the discussion on specific aspects of the application, for example:

• What mechanisms are being used to store the user ID and password in the "operating system"? Is this storage secure and sandboxed? I.e. how do we increase the area of trust boundary?
• Are the communication mechanisms from the app to the social media and inventor appropriate?
• What infrastructure information is implicitly included over these channels, for example, IP addresses, user agent strings [1] etc?
• Does the app have authorisation to the various channels?
• What is the granularity of the Location data over the various channels?
• What information and channels are considered primary and which secondary?
• Is the information flowing to the inventor appropriate and what is the content?
• What about the EXIF data embedded in the picture?

Generating a formal list of questions from the above is relatively easy and this is exactly how we proceed.

The next questions which follow are related to how we can reduce the amount of information without compromising the application functionality and business model? For example:

• can we reduce the granularity of the Location sent to the social media systems to, say, city level or country level?
• can we automatically remove EXIF data?
• do we allow the app to work if the operating system's location service is switched off or the user decides not to use this?

And so on...Finally  we get down to the consent related questions

• What does the user see upon first-time usage of the app? What do they have to agree to?
• Do we tell the user what underlying services such as GPS we're using as part of the application
• Secondary data collection opt-out
• For what reason is the data being collected over both primary and secondary channels

And so on again.

What we have done is set the scene, or, circumscribed what we need to investigate and decide upon. Indeed at some level we even have the means to measure the information content of the application and even the extent the consents by implication; and if we can measure then we have a formal mechanism to decide whether one design is "better" than another in terms of privacy.

In the following articles I'll discuss more about the classification mechanisms (information, security, usage, purpose, provenance) and other annotations along with the detailed implications of these.

Notes

[1] User agent strings are very interesting...just ask Panopticlick.

Where are all the Information Scientists?

What happened to all the scientists and engineers with a real deep understanding of what information is? We're now apparently firmly in the era of Big Data and yet most of the Big Data implementations seem to stop be building large Hadoop clusters and running modified versions of the canonical word counting programs over log files from Apache.

I had the pleasure of working with a group of people who were blessed with PhDs in mathematics, physics and statistics and yet they complained bitterly about the lack of good, interesting and relevant data in order to properly ply their skills and generate valuable insights.

I know of one person who is utilising some quite cool natural language processing over massive data sets to extract all kinds of context, and yet no-one seems to be interested in what this is achieving.

The current Gartner Hype Curve for Big Data has us on the edge of the precipice into the Trough of Disillusionment and once again when the negative press starts everyone will be surprised. Worse is that adopters of Big Data and analytics will not question why it went wrong but rather blame Big Data for not delivering.

One of the reasons why Big Data is heading for this "failure" is because we do not understand, or more accurately want to understand our data. We do not want to put the investment into understanding the subtle and complex interactions between the various aspects of our data, the classification structures, the processing etc.

Indeed I find much of the current trend to NoSQL as being an excuse for ignoring the very basics of understanding data. Apparently SQL is bad: I'm told that SQL means static table structures, fixed schema, inflexible query languages and all that relational theory stuff about data organisation is obsolete. and irrelevant. Multi terabyte "traditional" databases are uncool, cloudified Linux clusters in AWS running Hadoop and canonical word counting programs over web log files is COOL? At one level we've just made OS/360 and COBOL more complicated without doing much more than was possible 30 years ago.

Have we really forgotten how to manage and construct our data so that it becomes information? Or, as I also worry about, have we lost the ability to understand what our data could tell us and been blinded by the hype behind a technology? Not that it hasn't happened before of course.

Much of our emphasis is on building products and getting cool apps to users; sometimes we get data out of these apps either by primary or secondary means. Surprisingly little thought is placed on what data and how we could really understand that data and even less on the interactions between different datasets and what that could tell us.

Rather than understanding the context in which our information exists we've become obsessed with generating abstract counts of things which translate easily into bar charts on a PowerPoint slide. We're failing at extracting the deeper context.

It's fine to learn that an application is being used by one million users this week and two million the next, but does this really tell you about how people are using that app, who is using that app, why, where and what for? Even more importantly we don't even consider the inverse of the questions, why aren't people using my app?

Consider air miles or store cards - when was the last time an airline or supermarket contacted you to ask you why or why not you are flying/shopping with them? I can guarantee that the airline or supermarkets has number of how many things were bought or flights flown...hardly Big Data and deep analytics is it?

To solve this we really do need the return of the information scientist - someone who understands data, information and knowledge, who understands taxonomies, ontologies and the mathematical underpinnings of these. He or she also needs to know how to implement these, how best to organise a database, a data structure whether it be physically implemented in a triple store, relational database, flat log file etc...

For many this then rears the spectre of data governance and huge data management processes that slow the business down - you're not being agile are you then? Big Process is often the result of not understanding or not wanting to understand how to do things. If you have a number of apps and services wouldn't it be better to harmonise the information collection and ensure that the schemata and underlying semantics are consistent over all your collected data? Surely spending less time on data quality and consistency because quality and consistency are already inherent in your data is a much better use of your resources.

So, if you're having arguments over which NoSQL database to use, or whether triple stores and graph databases have the performance you require or even if Semantic Web is just an academic construct, then you're certainly not doing Big Data and the knowledge you will gain from the data you are capturing will only be supporting your business in the most ephemeral of ways.

Wikileaks, NSA and a crytpographic cat and mouse game

So we know that certain government agencies are collecting data and we also know that Wikileaks has at least 400Gb of encrypted data (aka. "insurance") now in the public domain.

It is encrypted and no-one knows what's there...right?

Schneier posted on his blog an article about the upcomming cryptopocalypse about an presentation from the Black Hat conference entitled The Factoring Dead:Preparing for the Cryptopocalypse.

Given that we are fairly confident that the NSA has an enormous amount of computing power available to it, but even then probably not enough for a brute force attack on the Wikileak's encrypted data by a few orders of magnitude, they do have some pretty competent mathematicians working in that field and a vast amount of data including a lot of unencrypted originals.

Now, as mathematics starts chipping away as the encryption algorithms themselves the amount of computation and time to decrypt without a key comes dramatically down (it might just take a few billions of years now).

What helps in these scenarios is if you have some knowledge about what the encrypted data contains and its encryption method. If the Wikileaks file contains what it is supposed to contain (whatever that is, but it probably implicates the USA) then we have a head-start towards the decryption. This isn't too far from what Alan Turing and the Bletchley code breakers did.

Furthermore, rarely in the past have we have huge amounts of data to work with...encryption algorithms generate data with extremely high entropy. The point now is that is there enough entropy in 400Gb of encrypted data? Do patterns start emerging at some point? Admittedly even 400Gb is not considered a large amount of data these days.

Furthermore the encryption algorithm used is one approved by the NSA. What if the NSA start matching already encrypted documents they hold against the Wikileaks data? Is there a weakness in the keys? Are there patterns in the encrypted data that are independent of the keys being used? Then there's the interesting idea of a blackdoor in the algorithm too.

So, (tin foil hats on please), one of the things that Wikileaks is then looking forward is accidental disclosure of some information in that release. This might suggest that either NSA has succesfulyl decrypted the data. I'm sure that the NSA aren't that stupid to tell us that but governments are large things and even despite apparently good security practices breaches happen either deliberately or accidentally.

From a game theoretic point of view the best I can come up with at the moment is that any release of information by the NSA will reveal subtle hints about what they can and can not do with regards to their knowledge of encryption algorithms.

Shares in popcorn and publishers of books on game theory just went through the roof. Ultimately it might just be that physics, mathematics and plain old information theory has the last laugh...

Data Collection

Anyone who cares about privacy will tell you that data collection is bad...REALLY BAD, yet without data collection none of the services we use and need would work (there's an economic argument there). Indeed most of the issues around data collection seems to be very emotive in nature and usually end up in the "data collection is bad/must be minimsed panic".

First to understand data collection we must understand from where data is collected. First of all we need to understand the relationship between a client and a server via some infrastructure. This is described in an earlier article which talks about primary and secondary data.

Generally the case we mainly worry about in privacy is where the client is a human interacting with an app or browser via some device, eg: a mobile phone.

The end-user (customer or human) will provide some information required to fulfill the functions of the service required, eg: posting a photograph to Flickr with some description and their login details.

So far this seems reasonable - afterall, a user is posting a photograph to their account with a description. The service might then perform an number of other tasks

• Set the current time and date of upload
• Extract EXIF information from the photograph, including (but not limited to):
• date and time of the shot
• location information
• camera information
• free text strings, eg: copyright/ownership information

All of the above comes under the notion of data collection and all of the above is information provided by the user of the service. One can argue that the user didn't know about the EXIF contents of the photograph and surprisingly few people actually realise how much information is embedded into the picture.

Further operations can be performed such as facial recognition and linking this to persons who have been tagged in other photographs etc. For example, this is a service provided by Facebook.

Additionally the service maybe collecting secondary information about the usage of the service, for example, how much time was spent using a particular page, what the UI/UX flow was and so on.

Then there's the information collected from the infrastructure. This includes information provided by the browser or app, typically in the form of browser identification strings, application identifiers, device identifiers, and from the service infrastructure including such information as source IP address, contents of the API call, time and date of interaction, error codes etc.

When we talk about privacy we typically end up focussing on the primary and secondary information sets. The former set to decide whether that information is necessary for the service to function and provide the facilities the user requires and the second set to decide whether this information needs to be explicitly collected or not. Rarely do we even investigate the information collected by the infrastructure which can be used both to recreate primary and secondary data sets. Indeed one of the major targets for any hacker are the infrastructure logs themselves.

When we talk about data collection we must consider the following data sets:

Primary, secondary and infrastructure are described as above with the addition that we split primary into explicit and implicit where the former denotes that information which is explicitly understood by the end-user and implicit including that information that might be hidden, eg: EXIF. Combined together we produce the Total Information Set which when linked together by simple association becomes the Total Deduced Information Set.

Another way of looking at this is in a more Venn diagrammatic manner:

Where the labels are abbreviations of the corresponding labels in the first diagram and colours for emphasis.

It must be noted that some intersections might not properly exist, for example the union of EP and Sec, and others only in certain contexts, for example, whether IP is a proper superset of EP or not.

Now we have explicitly modelled data collection, we can now start tackling questions such as what does minimisation of data collection actually mean and then start looking at what the processing and extraction functions over that data might actually look like.

Foundations of Applied Mathematics

Via the n-Category Cafe blog I noticed a real gem in the workshop on The Foundations of Applied Mathematics. Simplifying systems and trying to understand the foundations of things (information systems, software engineering, privacy) is of great interest to me.

For example, in my current area of information system privacy we have a huge problem in even contemplating what the foundation of the discipline looks like, let alone trying to formalise it (and it probably has a lot to do with entropy, semantics, measurement theory and information flow if you ask me - but before we get there we've a lot of other work to do).

The particular piece of work that interested me was by John Baez who has a history of work trying to unify various areas of mathematics ostensibly through category theory and his presentation entitled The Foundations of Applied Mathematics (slides) presents his work on applying these ideas to applied mathematics. This was also followed up by an interesting discussion on Google+.

The whole presentation and body of work starts to remind me of a few things

• layered architecture
• esoteric  models of understanding (- need a better way of explaining this!)
• visualisation (= diagrams)

Baez presents a model of four (for oversimplicity's sake as he describes in his presentation) layers linking science and engineering with applied mathematics with pure mathematics and finally grounding that in the foundations of mathematics. Immediately coming to mind here are the kinds of layered architecture we see in software engineering with each layer building upon the primitives provided by the immediate lower layer.  This is then followed by the question:

"Can developments in applied mathematics force changes in the foundations of mathematics?"

I'd have to answer 'yes' to this if only to assert the position that if the lower layers have not been correctly formulated or established and can not support the upper layers in their needs then the above is inevitable.

The reasoning for this comes from, at least in my chosen area of software engineering, is that layers [of software] are often constructed without any reference to any other layers, the operations and structures in those layers are not "atomic" or fundamental in nature with respect to that layer and the separation of concerns and layering of architecture is often a matter personal choice. Layers are often polluted with convenience functions and particular architectural choices, see Anti Patterns.

Another way of looking at this is via a famous xkcd cartoon on the relationship between Lisp, Diety and how it was really built:

Maybe it just is that the foundations aren't bad at all, just that we got the layering wrong. Maybe mathematics isn't layered at all...object oriented anyone?

Further in the Google+ discussion started by Baez, a posting by Carsten Führmann quoted Von Neumann:

"As a mathematical discipline travels far from its empirical source, or still more, if it is a second and third generation only indirectly inspired from ideas coming from 'reality', it is beset with very grave dangers...."

The quotation continues but the gist is that the further a discipline goes away from its foundations the more esoteric it becomes. I think in software engineering we see this quite clearly with things like use case modelling and agile methods. Not that there's much wrong with either but their current application is often far from the ideals proposed by their creators.These then get use to develop the architectural layers seen earlier.

In information system privacy we do not have a clear, coherent understanding of the foundations of the discipline and yet we are developing at very high levels of abstraction concepts, laws, ideas without understanding how these are to be architected into the technologies we are building. Contemplate the semantic gap between the European Privacy Directive and the actual C++ (or whatever) code that implements those. I'm not saying that laws, directives etc are wrong, just that even simple ideas such a Do Not Track will expose huge holes in our understanding of the discipline of privacy (and a lot of related subjects too). Actually I think that DNT won't work in its current form because too fee people do not want to address the formalisation of privacy but remain at some esoteric level far removed from what DNT is trying to address and that might have other implications and Von Neumannesque danger signals.

I won't discuss visualisation here, that's for another time, but just to reference the work at the University of Brighton on diagrammatic notation and also to refer to another point in Baez's presentation on the use of diagrams across disciplines (and layers of science) to express concepts to different audiences...presenting Euler diagrams to lawyers, marking people etc is an extremely interesting experience, especially when - as "non-mathematicians" - they start interacting using those notations and at some level we've "accidentally" unified the languages mathematicians and lawyers....

More Maxims for Privacy

I wrote some maxims for privacy a while back, but still I'm looking for that crystalisation of privacy - the "one slider" or "one sentence" that encompasses the foundations, or the basics of privacy.

For the most part privacy comes down to data collection and subsequent usage of that data - the rest is just additions to that. At least if we're concentrating on privacy and not the wider scheme of information management of which I believe privacy is just a sub-speciality, albeit a rather important one.

So, when dealing with information:

"If you don't have a use for it, 
don't collect it!"

For me this sums it up. It links data collection and usage in a way that clearly states that just collect what you need right now; don't even think about the future uses yet. Interestingly I also think that deliberately restricting the data collection right at the start to what you absolutely know you are going to use immediately forces you (the software/system developer) to better focus on the product at hand - "Slow Data" anyone?

In these agile development days however, it is often argued that we'll develop the usages later and collect everything now. How often does this really happen? And, if you really were agile then you'd construct your system initially to do the minimum it needs to and get that out to the customer for their appraisal. If that goes well (or not), then modify as necessary during later stages in your agile development process.

If you're not agile (which apparently is waterfall, though I'm not sure), then you should have worked out what you need and work from there; which again should be self-limiting on the data collection. Surely in a good, fully worked out design you wouldn't be collecting superfluous things?

So, that's it, the essence of privacy as a single sentence; the rest is just layers pertaining to things such as provenance, data retention, purpose, infrastructure etc - that's what makes good information management a whole discipline in itself, of which privacy is one small, but important part.

On The Naivety of Privacy

Recent events regarding privacy and the internet have left me wondering if we are being somewhat naïve. We are starting to see a slew of new laws, strategies and technologies for protecting our privacy in what is effectively a public space. The end-user however is not, as far as I can tell, really getting the benefit of this - indeed if anyone is it is the emerging privacy-industrial complex [1] as some have written.

It is utterly naïve to believe that laws, strategies, intentions, grand speeches, certifications, automatic filtering, classification iconography and so on make for better end-user privacy. The more we do this the more confused we become, and simultaneously we lose sight of what we're really trying to achieve. Spare a thought for the poor end-users.

There is a great deal that is misunderstood or not known by privacy advocates about how the internet, computers and information systems work - I fear in a lot of cases either some don't want to understand because it takes them outside of their comfort zone, or the semantic gap between the engineers and the legal/advocacy side is too great and that bridging this gap is extraordinarily difficult for both parties.

I worry about our lack of formality and discipline, possibly in equal quantities. We – the privacy community – lack these aspects to really understand and accept the fundamentals of our area and how to apply this to the information systems we are trying to protect. In some cases we are actively fighting against the need to scientifically ground our chosen area.

We must take a moment to think and understand what problem we are really trying to solve. The more philosophical works by Solove and Nissenbaum address the overall concept of privacy. I'm not sure that the implications of these are really understood. Part of the problem is that general theories of information [2] are very abstract and obtuse when compared with the legal views of the above authors, and we've done very little to tie these areas together to produce the necessary scientific formalisation of privacy we need.

As an example, the Privacy by Design (PbD) manifesto is being waived by many to be the commandments of privacy and following these magically solves everything. This only leads to “technical debt” and greater problems in the future. Often we find the engineers, R&D teams and the scientists excluded from, and outside of, this discussion.

I think we're missing the point what privacy really is and certainly we have little idea at this time how to effectively build information systems with inherent privacy [3] as a property of those systems. I have one initial conclusion:

WE HAVE NO UNDERLYING THEORY OF PRIVACY

We have no common definitions, common language, common semantics nor mappings between our individual worlds: legal, advocacy and engineering. Worse, in each of these worlds terminology and semantics are not always so well internally defined.

When an [software] engineer says “data collection”, "log" or "architecture", these do not mean the same to a lawyer or a consumer advocate. Indeed I don't think these terms semantically map even remotely cleanly – if at all - between these groups.

A set of PowerPoint slides with a strategy, a vision, a manifesto, good intentions, project plan or a classifications scheme mean very little and without some form of semantics are wasted, token efforts that only add to the complexity and confusion of a rapidly changing field.

We desperately need to address the problem that we must create a way of communicating amongst ourselves through which all of the internal factions within the privacy community can effectively understand each other's point of view. Only then might we even have a chance of realistically and effectively addressing the moving target of privacy issues facing end- users and businesses that rely so much on the interchange and analysis of information.

The problem with formally (or rigorously) defining anything is that it has the nasty tendency to expose holes and weaknesses in our thinking. Said holes and weaknesses are not entirely appreciated, especially when it challenges an established school of thought or a political or dogmatic balance [4].

The privacy community is constantly developing new laws and legal arguments, new sets of guidelines, manifestos and doom scenarios while the engineers are trying to address these often inconsistent and complex ideas through technical means. From the engineering perspective not only we are internally exposing flaws in database design, information system architecture and user experience but also the mismatch between engineering, legal, the world of the consumer advocate and ultimately a company's information strategy.

An information strategy needs to address everything from how the engineers develop software to how you want your company to be perceived by the consumer. How many information strategies actually address the role that information plays in a modern, global consumer ecosystem where the central concept is the collection and processing of user information? Of those, how many address the engineering and scientific levels of information?

We must take a serious retrenchment [5] step and look back at what we have created. Then we need ruthlessly carve away anything that does not either solve the communication issue within the privacy community or does not immediately serve the end-user. Reemphasizing the latter point, this explicitly means the end-user values, not what we as a privacy community might perceive to be valued by the end-user.

We must fully appreciate the close link between privacy and information, and that privacy is one of the most crosscutting of disciplines. Privacy is going to expose every single flaw in the way we collect, manage, process and use information from the user experience, as well as the application and services eco-system, and even the manner in which we conduct our system and software engineering processes and information governance. The need to get privacy right is critical not just for the existence of privacy as a technical discipline in its own right (alongside security, architecture, etc) but also for the consumer and the business.

The emphasis must be placed on the deep technical knowledge of experts in information systems – these must be the drivers and unifiers between the engineers, the lawyers, the advocates and ultimately the users. Without this deep, holistic, scientific and mathematical foundation we will not be able to sufficiently nor consistency address or govern any issues that arise in the construction of our information systems at any level of abstraction.

If the work we do in privacy does not have a scientific, consistent and formal underpinning [6] that brings together the engineers, lawyers and advocates then privacy is waste of time at best and deeply destructive to the information systems at worst.

Without this we are a disjointed community caring for ourselves and not the business or consumer and privacy becomes just a bureaucratic exercise to fulfill the notions of performing a process and metrics rendered as meaningful as random numbers.

* * *

PostScript:

Via Twitter I came across a talk given by Jean Bezivin entitled "Should we Resurrect Software Engineering?" presented at the Choose Forum in December 2012. Many of the things he presented are analogous to what is happening in privacy. He makes the point a number of times that we have never addressed the missing underlying theory of software engineering and how to really unify the various communities, fields and techniques in this area. Two points I particularly liked was that we concentrated on the solution (MDE) but never thought about the problem; the other point is the use Albert Camus' quote

<< Mal nommer les choses, c'est ajouterau malheur du monde >>

[To misname things is to add misery to the world]

A subtle hint to getting the fundamentals right: terminology and semantics!


Notes

[1] #pii2012: The Emergent Privacy-Industrial Complex

[2] Jerry Seligmann, Jon Barwise (1997) Information Flow. Cambridge University Press.

[3] I like the idea of privacy being an inherent construct in system design in much the same way that inherent safety emerged from chemical/industrial plant design

[4] A blog article discussing “mathematical catastrophes” – two that come to mind are Russel and Frege and also Russel and Gödel. Both related but the latter’s challenge to the mathematical school of thought was dramatic to say the least.

[5] A formal retrenchment step in that we not just start again but actively record what we’re backtracking on. Poppleton et.al. constructed a theory of retrenchment for software design using formal methods; the same principles apply here.

[6] If you’re still in doubt just remember that whatever decisions are made with respect to privacy, there’s a programmer writing formal, mathematical statements encoding this. Lessig’s Code is Law principle.