A collection of discussions, links, stories, news and whatever else I find interesting in the fields of computing, information, science, privacy, semantics, mathematics and so on...
Showing posts with label quality. Show all posts
Showing posts with label quality. Show all posts
Tuesday, 15 November 2016
Privacy Engineering for Today - DIMECC Presentation
Here is my presentation from the DIMECC 9th Annual Seminary on Business innovation in Finland.
Friday, 5 August 2016
Privacy Engineering Procedures and Ebola
A seemingly unlikely combination: privacy engineering and ebola, though I guess there are similarities by which viruses spread with how personal data spreads around a company - another time and another study I think.
OK, so what the zark do these things have in common - the answer is via a convoluted path and actually is more related to how we react to an incident: privacy or medical (and we're back to safety-critical systems again).
Bit of background first: I've been reading about Marburg and Ebola recently - both are fascinating (and frightening) themselves, but what is more interesting from a procedural point of view is how they were discovered, researched and ultimately how we as a species react to them.
Now the procedural stuff, the CDC have a response plan for Ebola entitled: Identify, Isolate and Inform: Emergency Department Evaluation and Management for Patients Under Investigation for Ebola Virus Disease.
Worth reading just to understand how the CDC explain how to plan your due diligence - something we're exceptionally bad at in privacy ... we just scream PIA and COMPLIANCE !
The point here is that if privacy engineering is to emerge as a discipline we need to address our culture in how we react to incidents and even react in general. Learning from a discipline that already has to face critical incidents is a good start.
OK, so what the zark do these things have in common - the answer is via a convoluted path and actually is more related to how we react to an incident: privacy or medical (and we're back to safety-critical systems again).
Bit of background first: I've been reading about Marburg and Ebola recently - both are fascinating (and frightening) themselves, but what is more interesting from a procedural point of view is how they were discovered, researched and ultimately how we as a species react to them.
 |
| Ebola (via Wikipedia and CDC) |
Worth reading just to understand how the CDC explain how to plan your due diligence - something we're exceptionally bad at in privacy ... we just scream PIA and COMPLIANCE !
The point here is that if privacy engineering is to emerge as a discipline we need to address our culture in how we react to incidents and even react in general. Learning from a discipline that already has to face critical incidents is a good start.
Wednesday, 8 June 2016
2nd IW5GS - Programme
The 2nd International Workshop on 5G Security
IW5GS2016
Xi'an, China
June 19, 2016
http://www.mobimedia.org/2016/show/program-final
IW5GS-01: (June 19, 2016, Sunday, 10: 30 – 12:30, Room B)
Session Chair: Valtteri Niemi (Email: valtteri.niemi@helsinki.fi)
Keynote 1: 5G Security for IoT
Speaker: Dr. Zhiyuan Hu, Nokia Shanghai Bell (zhiyuan.hu@alcatel-sbell.com.cn)
Keynote 2: 5G Security: Forward Thinking
Speaker: Bo Zhang, Huawei (liufei19@huawei.com)
IW5GS-02: (June 19, 2016, Sunday, 14: 00 – 16:30, Room B)
Session Chair: Siddharth Prakash Rao (Siddharth.rao@aalto.fi); Ian Oliver (ian.oliver@nokia.com)
Paper 1: Protecting IMSI and User Privacy in 5G Networks
Karl Norrman, Elena Dubrova, Mats Näslund
Paper 2: Privacy of the Long-Term Identities in Cellular Networks
Philip Ginzboorg, Valtteri Niemi
Paper 3: Error-Correcting Message Authentication for 5G
Elena Dubrova, Mats Näslund, Göran Selander, Karl Norrman
Paper 4: Privacy in LTE networks
Siddharth Prakash Rao, Bhanu Teja Kotte, Silke Holtmanns
Paper 5: A Survey on Software-Defined Networking Security
Shanshan Bian, Peng Zhang, Zheng Yan
Paper 6: Designing Hybrid Cloud Computing Framework using OpenStack for Supporting Multimedia with Security and Privacy
Isaac Cushman, Lei Chen, Danda B. Rawat, and Nhien-An Le-Khac
Thursday, 25 June 2015
What is quality?
Everyone talks about quality, whether it be a quality customer experience, or quality product, or even a quality process...still not sure what that is...but ask the question of "what is quality?" and you won't get a good answer.
There are two good places to start:
Interestingly, quality doesn't necessary mean expensive or the best - though being called the "best" occurs when you have quality.
For example, SAS actually have a "quality" long-haul economy product. This doesn't mean it is great, but they do their best to keep passengers (on an 11 hour flight) well fed and watered. Now just because I had a good experience and perceived their economy product to be of a high quality doesn't necessarily mean that they shouldn't improve on it. Trust me, there are LOTS of things SAS could do to improve on their economy product.
Airlines are abound with examples of quality: for example, Finnair vs Norwegian - the former just feels to be an expensive Ryanair but without the friendliness while the latter gives you what you need plus free Wifi (slow Wifi, but Wifi still). To me, Norwegian provides me with a quality product. Another example is Lufthansa: while their long-haul economy offering is poor from a seating point of view, their food (SAS take note, I get beer and wine for free with my meal!) and superb crews, plus the added advantage of using the A380 means I choose to fly Lufthansa when I can.
But today I came across something interesting while stopping at a cafe to buy ice-cream for my children. There is no doubt this particular cafe has good service, smiling and ample staff, and excellent ice-cream, though a bit pricey! But even with, I counted 3 members of staff and a manager present they couldn't actually keep the 10 or so tables clean. It really isn't pleasant to have to sit at a table with someone else's food still left there, or having to clean it up yourself.
While everything about this cafe says "we provide a quality product" this is let down by a very simple action of not cleaning up.
This got me thinking, especially when referring back to Pirsig's observation that even though you might have the most expensive and best built motorcycle, it only takes a single screw to become threaded, or the head to become ground away to reduce that most expensive of machines to being totally unfit for purpose - in Pirsig's case, unrepairable because of a single flaw in the cheapest, most innocuous of components.
Funny how a simple act such as clearing up a table can ruin a cafe's reputation; or, how a faulty IFE screen can ruin a flight. Sometimes it is just attitude that ruins everything...when a much respected company fails to ever ask of, or even respect even a semi-regular customer then the quality of the whole is reduce to nothing.
There are two good places to start:
- Robert Pirsig's Zen and the Art of Motorcycle Maintenance
- John Guaspari's "I know it when I see it"
Anyone working in "quality" should have not just read but completely internalised both books before uttering a further word on the subject.
Interestingly, quality doesn't necessary mean expensive or the best - though being called the "best" occurs when you have quality.
For example, SAS actually have a "quality" long-haul economy product. This doesn't mean it is great, but they do their best to keep passengers (on an 11 hour flight) well fed and watered. Now just because I had a good experience and perceived their economy product to be of a high quality doesn't necessarily mean that they shouldn't improve on it. Trust me, there are LOTS of things SAS could do to improve on their economy product.
Airlines are abound with examples of quality: for example, Finnair vs Norwegian - the former just feels to be an expensive Ryanair but without the friendliness while the latter gives you what you need plus free Wifi (slow Wifi, but Wifi still). To me, Norwegian provides me with a quality product. Another example is Lufthansa: while their long-haul economy offering is poor from a seating point of view, their food (SAS take note, I get beer and wine for free with my meal!) and superb crews, plus the added advantage of using the A380 means I choose to fly Lufthansa when I can.
But today I came across something interesting while stopping at a cafe to buy ice-cream for my children. There is no doubt this particular cafe has good service, smiling and ample staff, and excellent ice-cream, though a bit pricey! But even with, I counted 3 members of staff and a manager present they couldn't actually keep the 10 or so tables clean. It really isn't pleasant to have to sit at a table with someone else's food still left there, or having to clean it up yourself.
While everything about this cafe says "we provide a quality product" this is let down by a very simple action of not cleaning up.
This got me thinking, especially when referring back to Pirsig's observation that even though you might have the most expensive and best built motorcycle, it only takes a single screw to become threaded, or the head to become ground away to reduce that most expensive of machines to being totally unfit for purpose - in Pirsig's case, unrepairable because of a single flaw in the cheapest, most innocuous of components.
Funny how a simple act such as clearing up a table can ruin a cafe's reputation; or, how a faulty IFE screen can ruin a flight. Sometimes it is just attitude that ruins everything...when a much respected company fails to ever ask of, or even respect even a semi-regular customer then the quality of the whole is reduce to nothing.
Sunday, 22 March 2015
Slowing Down Software Development
Stephen Wilson in his blog post Programming is like Playwriting (23 Feb 2011) which recently resurfaced via a Twitter conversation makes a few interesting points about how we write software and how the tools and speed of development cause some very interesting quality problems.
Coding is fast and furious. In a single day, a programmer can create a system probably more complex than an airport that takes more than 10,000 person-years to build. And software development is tremendous creative fun. Let's be honest: it's why the majority of programmers chose their craft in the first place.
Actually I found this statement ironic, especially in light of the Denver Airport Baggage System - which itself became far more complex than the rest of the airport's operations.
So, picking out two salient points:
We took our time. I was concerned that the CASE tools we introduced in the mid 90s might make code rather too easy to trot out, so at the same time I set a new rule that developers had toturn their workstations off for a whole day once a week, and work with pen and paper.
I worked a long while back in software-hardware co-design, to best understand the difference consider these situations:
Software - compilation and testing phases
$ vi myProg.c
$ gcc myProc.c
$ ./a.out
repeat multiple times per minute/hour as necessary. The cost of compiling and editing is measured only in man hours.
Hardware - compilation and testing phases
- Send net list to TI,Phillips or whoever for ASIC manufacturer
- Pay $1,000,000
- Wait 3-6 months
- Receive single ASIC in post
- Test
Maybe the solution is that each compilation is charged per compilation? Actually I knew one developer that added sleep statements to his compilation scripts so that the act of compilation would become so 'expensive' that he spend much more time ensuring that the code worked before compilation.
My internal coding standard included a requirement that when starting a new module, developers write their comments before they write their code, and their comments had to describe ‘why’ not ‘what’. Code is all syntax; the meaning and intent of any software can only be found in the natural language comments.
Formal specification? Now whether you use B, Z, VDM or any of the other host of mathematical languages (and by the way, C, Java etc are mathematical languages in that sense) along with their tools and techniques is largely irrelevant, though for actually expressing the WHY and WHAT they are rather good at this!
We have had some excellent results regarding so called 'light-weight' usage of formal methods. The main learning however is not doing formal methods for the sake of doing formal methods but the fact that the communication and clarity of the requirements and subsequent code was much improved.
References:
[1] Ian Oliver Experiences of Formal Methods in 'Conventional' Software and Systems Design. FACS 2007 Christmas Workshop: Formal Methods in Industry. BCS London, UK, 17 December 2007
[2] Ian Oliver Experiences of Formal Methods in 'Conventional' Software and Systems Design
Monday, 26 January 2015
Kings Cross,Trains and a Swiss Cheese
New Year's Resolution: write more .... and .... 26 days later....finally got around to it. So after a prolonged break and spending most of it reading upon topics from organizational failures, safety, privacy and a touch of homomorphic encryption, I came around this:
which is an exceptionally well written summary of the problems at King's Cross Railway Station after Christmas 2014 where overrunning engineering works blocked all long distance and commuter rail services out of one of the London busiest termini.
What actually started out as a fantastic opportunity to perform engineering work over a rare, four day holiday became the proverbial nightmare as works overran by 24 hours. As a relatively simple study in the Swiss Cheese model it provides a wealth of issues and examples of how the holes in the Swiss Cheese lined up as a large number of minor problems coalesced into a "disaster".
Despite meticulous planning and well rehearsed contingency and emergency plans, and even pre-emptive measures such as providing extra machinery and fitters to correct problems on-site it eventually came down to a lack of drivers for freight trains that finally provided the final hole in the Safety Swiss Cheese.
An excellent example of how things go wrong despite detailed planning. Of course it is easy to judge with 20/20 hindsight and see clearly that a number of known failure points were obvious, but like many of these cases at each point the situational awareness was for many reasons lacking. Funny how the same aspects occur here in engineering as well as medicine, aviation and of course privacy.
which is an exceptionally well written summary of the problems at King's Cross Railway Station after Christmas 2014 where overrunning engineering works blocked all long distance and commuter rail services out of one of the London busiest termini.
What actually started out as a fantastic opportunity to perform engineering work over a rare, four day holiday became the proverbial nightmare as works overran by 24 hours. As a relatively simple study in the Swiss Cheese model it provides a wealth of issues and examples of how the holes in the Swiss Cheese lined up as a large number of minor problems coalesced into a "disaster".
Despite meticulous planning and well rehearsed contingency and emergency plans, and even pre-emptive measures such as providing extra machinery and fitters to correct problems on-site it eventually came down to a lack of drivers for freight trains that finally provided the final hole in the Safety Swiss Cheese.
An excellent example of how things go wrong despite detailed planning. Of course it is easy to judge with 20/20 hindsight and see clearly that a number of known failure points were obvious, but like many of these cases at each point the situational awareness was for many reasons lacking. Funny how the same aspects occur here in engineering as well as medicine, aviation and of course privacy.
Wednesday, 5 February 2014
Writing...
Despite writing supposedly being cathartic...ha!...writing anything, from an academic paper to a forthcoming book is not a linear stream of word but a moulding of multiple, parallel, concurrent streams of conciousness into something appearing a whole.
I thought of the analogy with moulding a piece of clay into a work of art...and it works until you realise that piece of clay is probably a piece of s*** that you're holding. At which point you throw it away, wash your hands and feel disgusted and what you've done...
...and then feel more disgusted as to the time you've wasted and the work you've put in and what you've thrown away...warts and all...
It is at that point that you realise that somewhere in that piece of p** was a nugget of gold. So it's on with the rubber gloves to fish out that "clay", find the piece of gold (which may have just been a flash of sunlight if the first place), clean it off and continue with the moulding of your writing until you've achieved something that you're happy to abandon in public view. Just hoping that you've managed to wash off all the smell...
Hemmingway got it right:
The first draft of anything is shit.
I thought of the analogy with moulding a piece of clay into a work of art...and it works until you realise that piece of clay is probably a piece of s*** that you're holding. At which point you throw it away, wash your hands and feel disgusted and what you've done...
...and then feel more disgusted as to the time you've wasted and the work you've put in and what you've thrown away...warts and all...
It is at that point that you realise that somewhere in that piece of p** was a nugget of gold. So it's on with the rubber gloves to fish out that "clay", find the piece of gold (which may have just been a flash of sunlight if the first place), clean it off and continue with the moulding of your writing until you've achieved something that you're happy to abandon in public view. Just hoping that you've managed to wash off all the smell...
Hemmingway got it right:
The first draft of anything is shit.
...anyway I'm off to read Pirsig and walk the dog...maybe simultaneously....
Tuesday, 19 November 2013
Losing Situational Awareness in Software Engineering
The cause of many aircraft accidents is
attributed to loss of situational awareness. The reasons for such a situation
are generally due to high workload, confusing or incorrect data and
misinterpretation of data by the pilot.
The common manifestation of this is mode
confusion where the pilot's expectation for a given situation (eg: approach)
does not match the actual situation. This is often seen (though not exclusively)
in highly automated environments, leading to the oft repeated anecdote:
A novice pilot will exclaim, "Hey! What's it doing now?", whereas an experienced pilot will exclaim, "Hey! It's doing that again!"
Aside: while this seems to be applied to
Airbus’ FBW models, the above can be traced back to the
American Airlines Childrenof the Magenta tutorial and specifically refers to the decidedly non-FBW Boeing
757/767 and Airbus A300 models, and not the modern computerized machines… [1]
This obviously isn't restricted to
aircraft but also to many other environments; for example, consider the
situation in a car when cruise control is deactivated due to an external
situation not perceived by the driver. Is the car slowing due to road
conditions, braking etc? Consider this in conjunction when you are expecting
the car to slow down where the deactivation has the same effect as expected;
this type of loss of situational awareness was seen in the Turkish 737 accident
at Schipol.
In an auditing environment where we obtain
a view horizontally across a company we too suffer from loss of situational
awareness. In agile, fast environments with many simultaneous projects, we
often fail to see the interactions between those projects.
Yet, understanding these links in
non-functional areas such as security and privacy is absolutely critical to a
clear and consistent application of policy and decisions.
A complicating factor we have seen is
that projects change names, components of projects and data-sets are reused
both dynamically and statically leading to identity confusion. Systems reside
locally, in the cloud and elsewhere in the Universe, terminology is stretched
to illogical extremes: big data and agile being two examples of this. Simplicity is considered a weakness and
complexity a sign of the hero developer and manager.
In systems with safetycritical properties heroes are a bad thing.
In today's so-called agile, uber-innovative,
risk-taking, fail fast, fail often, continuous deployment and development
environments we are missing the very basic and I guess old fashioned exercise
of communicating rigorously and simply what we're doing and reusing material
that already exists.
Fail often, fail fast, keep making the
same mistakes and grow that technical debt.
We need to build platforms around shared
data, not functionality overlapping, vertical components with the siloed data
mentality. This required formal communication and an emphasis on quality, not
on the quantity of rehashing buzzwords from the current zeitgeist.
In major construction projects there is
always a coordinator whose job it is to ensure that not only do individual
experts communicate (eg: the plumbers, the electricians etc) but that their
work is complimentary and that one group does not repeat the work or base that
another team has put in place.
If software engineers built a house, one
team would construct foundations by tearing down the walls that inconveniently stood
atop of already built foundations, while another would build windows and doors
while digging up the foundation as it is being constructed. A further team
charged with installing the plumbing and electrics would first endeavor to
invent copper, water and electricity...and all together losing awareness of the
overall situation.
OK, I'm being hard on my fellow software
engineers, but it is critical that we concentrate more on communication, common
goals, less "competition" [2] and ensuring that our fellow software
engineers are aware of each other's tasks.
As a final example, we will (and have!)
seen situations in big data where analytics is severely compromised because we
failed to communicate and decide upon common data models, common catalogs of
where and what data exists.
So, like the pilot faced with a situation
where he recognises he's losing his situational awareness, drop back down the
automation and resort to flying by the old-fashioned methods and raw data.
The next question is, how do you
recognise that you're losing situational awareness?
Wednesday, 4 September 2013
The Art of Writing Good Documentation: Teach, Don't Tell
Just come across this posting on how to write documentation, or more specifically, how to write GOOD documentation. It is by Steve Losh and called Teach, Don't Tell.
For example:
This is one of the main reasons why we gave up on brilliant languages, frameworks etc- for example, this is the main reason why we had to give up on Opa sadly.
This is always the case when the experts in a language, framework, library etc are those who are developing the langauge, framework or library. They are so consumed by developing they forget the people who want to develop with their creation.
Teach the developers how to do something, don't fob the developer off with function signatures, obscure examples etc. If a developer is asking naive and "stupid" questions then there's probably a damned good reason why.
Maybe reading Zen and the Art of Motorcycle Maintenance should be compulsory for all?
For example:
If you use many open source libraries you’ve undoubtedly encountered some whose README says something like “read the source”. Every time I see one, I die a little bit inside.
Source code is not documentation. Can you learn to be a guitarist by simply listening to a piece of music intently? Can you become a painter by visiting a lot of museums? Of course not!
This is one of the main reasons why we gave up on brilliant languages, frameworks etc- for example, this is the main reason why we had to give up on Opa sadly.
This is always the case when the experts in a language, framework, library etc are those who are developing the langauge, framework or library. They are so consumed by developing they forget the people who want to develop with their creation.
Teach the developers how to do something, don't fob the developer off with function signatures, obscure examples etc. If a developer is asking naive and "stupid" questions then there's probably a damned good reason why.
Maybe reading Zen and the Art of Motorcycle Maintenance should be compulsory for all?
Friday, 25 January 2013
On The Naivety of Privacy
Recent events regarding privacy and the internet have left me wondering if we are being somewhat naïve. We are starting to see a slew of new laws, strategies and technologies for protecting our privacy in what is effectively a public space. The end-user however is not, as far as I can tell, really getting the benefit of this - indeed if anyone is it is the emerging privacy-industrial complex [1] as some have written.
It is utterly naïve to believe that laws, strategies, intentions, grand speeches, certifications, automatic filtering, classification iconography and so on make for better end-user privacy. The more we do this the more confused we become, and simultaneously we lose sight of what we're really trying to achieve. Spare a thought for the poor end-users.
There is a great deal that is misunderstood or not known by privacy advocates about how the internet, computers and information systems work - I fear in a lot of cases either some don't want to understand because it takes them outside of their comfort zone, or the semantic gap between the engineers and the legal/advocacy side is too great and that bridging this gap is extraordinarily difficult for both parties.
I worry about our lack of formality and discipline, possibly in equal quantities. We – the privacy community – lack these aspects to really understand and accept the fundamentals of our area and how to apply this to the information systems we are trying to protect. In some cases we are actively fighting against the need to scientifically ground our chosen area.
We must take a moment to think and understand what problem we are really trying to solve. The more philosophical works by Solove and Nissenbaum address the overall concept of privacy. I'm not sure that the implications of these are really understood. Part of the problem is that general theories of information [2] are very abstract and obtuse when compared with the legal views of the above authors, and we've done very little to tie these areas together to produce the necessary scientific formalisation of privacy we need.
As an example, the Privacy by Design (PbD) manifesto is being waived by many to be the commandments of privacy and following these magically solves everything. This only leads to “technical debt” and greater problems in the future. Often we find the engineers, R&D teams and the scientists excluded from, and outside of, this discussion.
I think we're missing the point what privacy really is and certainly we have little idea at this time how to effectively build information systems with inherent privacy [3] as a property of those systems. I have one initial conclusion:
We have no common definitions, common language, common semantics nor mappings between our individual worlds: legal, advocacy and engineering. Worse, in each of these worlds terminology and semantics are not always so well internally defined.
When an [software] engineer says “data collection”, "log" or "architecture", these do not mean the same to a lawyer or a consumer advocate. Indeed I don't think these terms semantically map even remotely cleanly – if at all - between these groups.
A set of PowerPoint slides with a strategy, a vision, a manifesto, good intentions, project plan or a classifications scheme mean very little and without some form of semantics are wasted, token efforts that only add to the complexity and confusion of a rapidly changing field.
We desperately need to address the problem that we must create a way of communicating amongst ourselves through which all of the internal factions within the privacy community can effectively understand each other's point of view. Only then might we even have a chance of realistically and effectively addressing the moving target of privacy issues facing end- users and businesses that rely so much on the interchange and analysis of information.
The problem with formally (or rigorously) defining anything is that it has the nasty tendency to expose holes and weaknesses in our thinking. Said holes and weaknesses are not entirely appreciated, especially when it challenges an established school of thought or a political or dogmatic balance [4].
The privacy community is constantly developing new laws and legal arguments, new sets of guidelines, manifestos and doom scenarios while the engineers are trying to address these often inconsistent and complex ideas through technical means. From the engineering perspective not only we are internally exposing flaws in database design, information system architecture and user experience but also the mismatch between engineering, legal, the world of the consumer advocate and ultimately a company's information strategy.
An information strategy needs to address everything from how the engineers develop software to how you want your company to be perceived by the consumer. How many information strategies actually address the role that information plays in a modern, global consumer ecosystem where the central concept is the collection and processing of user information? Of those, how many address the engineering and scientific levels of information?
We must take a serious retrenchment [5] step and look back at what we have created. Then we need ruthlessly carve away anything that does not either solve the communication issue within the privacy community or does not immediately serve the end-user. Reemphasizing the latter point, this explicitly means the end-user values, not what we as a privacy community might perceive to be valued by the end-user.
We must fully appreciate the close link between privacy and information, and that privacy is one of the most crosscutting of disciplines. Privacy is going to expose every single flaw in the way we collect, manage, process and use information from the user experience, as well as the application and services eco-system, and even the manner in which we conduct our system and software engineering processes and information governance. The need to get privacy right is critical not just for the existence of privacy as a technical discipline in its own right (alongside security, architecture, etc) but also for the consumer and the business.
The emphasis must be placed on the deep technical knowledge of experts in information systems – these must be the drivers and unifiers between the engineers, the lawyers, the advocates and ultimately the users. Without this deep, holistic, scientific and mathematical foundation we will not be able to sufficiently nor consistency address or govern any issues that arise in the construction of our information systems at any level of abstraction.
If the work we do in privacy does not have a scientific, consistent and formal underpinning [6] that brings together the engineers, lawyers and advocates then privacy is waste of time at best and deeply destructive to the information systems at worst.
Without this we are a disjointed community caring for ourselves and not the business or consumer and privacy becomes just a bureaucratic exercise to fulfill the notions of performing a process and metrics rendered as meaningful as random numbers.
PostScript:
Via Twitter I came across a talk given by Jean Bezivin entitled "Should we Resurrect Software Engineering?" presented at the Choose Forum in December 2012. Many of the things he presented are analogous to what is happening in privacy. He makes the point a number of times that we have never addressed the missing underlying theory of software engineering and how to really unify the various communities, fields and techniques in this area. Two points I particularly liked was that we concentrated on the solution (MDE) but never thought about the problem; the other point is the use Albert Camus' quote
Notes
[1] #pii2012: The Emergent Privacy-Industrial Complex
[2] Jerry Seligmann, Jon Barwise (1997) Information Flow. Cambridge University Press.
[3] I like the idea of privacy being an inherent construct in system design in much the same way that inherent safety emerged from chemical/industrial plant design
[4] A blog article discussing “mathematical catastrophes” – two that come to mind are Russel and Frege and also Russel and Gödel. Both related but the latter’s challenge to the mathematical school of thought was dramatic to say the least.
[5] A formal retrenchment step in that we not just start again but actively record what we’re backtracking on. Poppleton et.al. constructed a theory of retrenchment for software design using formal methods; the same principles apply here.
[6] If you’re still in doubt just remember that whatever decisions are made with respect to privacy, there’s a programmer writing formal, mathematical statements encoding this. Lessig’s Code is Law principle.
It is utterly naïve to believe that laws, strategies, intentions, grand speeches, certifications, automatic filtering, classification iconography and so on make for better end-user privacy. The more we do this the more confused we become, and simultaneously we lose sight of what we're really trying to achieve. Spare a thought for the poor end-users.
There is a great deal that is misunderstood or not known by privacy advocates about how the internet, computers and information systems work - I fear in a lot of cases either some don't want to understand because it takes them outside of their comfort zone, or the semantic gap between the engineers and the legal/advocacy side is too great and that bridging this gap is extraordinarily difficult for both parties.
I worry about our lack of formality and discipline, possibly in equal quantities. We – the privacy community – lack these aspects to really understand and accept the fundamentals of our area and how to apply this to the information systems we are trying to protect. In some cases we are actively fighting against the need to scientifically ground our chosen area.
We must take a moment to think and understand what problem we are really trying to solve. The more philosophical works by Solove and Nissenbaum address the overall concept of privacy. I'm not sure that the implications of these are really understood. Part of the problem is that general theories of information [2] are very abstract and obtuse when compared with the legal views of the above authors, and we've done very little to tie these areas together to produce the necessary scientific formalisation of privacy we need.
As an example, the Privacy by Design (PbD) manifesto is being waived by many to be the commandments of privacy and following these magically solves everything. This only leads to “technical debt” and greater problems in the future. Often we find the engineers, R&D teams and the scientists excluded from, and outside of, this discussion.
I think we're missing the point what privacy really is and certainly we have little idea at this time how to effectively build information systems with inherent privacy [3] as a property of those systems. I have one initial conclusion:
WE HAVE NO UNDERLYING THEORY OF PRIVACY
We have no common definitions, common language, common semantics nor mappings between our individual worlds: legal, advocacy and engineering. Worse, in each of these worlds terminology and semantics are not always so well internally defined.
When an [software] engineer says “data collection”, "log" or "architecture", these do not mean the same to a lawyer or a consumer advocate. Indeed I don't think these terms semantically map even remotely cleanly – if at all - between these groups.
A set of PowerPoint slides with a strategy, a vision, a manifesto, good intentions, project plan or a classifications scheme mean very little and without some form of semantics are wasted, token efforts that only add to the complexity and confusion of a rapidly changing field.
We desperately need to address the problem that we must create a way of communicating amongst ourselves through which all of the internal factions within the privacy community can effectively understand each other's point of view. Only then might we even have a chance of realistically and effectively addressing the moving target of privacy issues facing end- users and businesses that rely so much on the interchange and analysis of information.
The problem with formally (or rigorously) defining anything is that it has the nasty tendency to expose holes and weaknesses in our thinking. Said holes and weaknesses are not entirely appreciated, especially when it challenges an established school of thought or a political or dogmatic balance [4].
The privacy community is constantly developing new laws and legal arguments, new sets of guidelines, manifestos and doom scenarios while the engineers are trying to address these often inconsistent and complex ideas through technical means. From the engineering perspective not only we are internally exposing flaws in database design, information system architecture and user experience but also the mismatch between engineering, legal, the world of the consumer advocate and ultimately a company's information strategy.
An information strategy needs to address everything from how the engineers develop software to how you want your company to be perceived by the consumer. How many information strategies actually address the role that information plays in a modern, global consumer ecosystem where the central concept is the collection and processing of user information? Of those, how many address the engineering and scientific levels of information?
We must take a serious retrenchment [5] step and look back at what we have created. Then we need ruthlessly carve away anything that does not either solve the communication issue within the privacy community or does not immediately serve the end-user. Reemphasizing the latter point, this explicitly means the end-user values, not what we as a privacy community might perceive to be valued by the end-user.
We must fully appreciate the close link between privacy and information, and that privacy is one of the most crosscutting of disciplines. Privacy is going to expose every single flaw in the way we collect, manage, process and use information from the user experience, as well as the application and services eco-system, and even the manner in which we conduct our system and software engineering processes and information governance. The need to get privacy right is critical not just for the existence of privacy as a technical discipline in its own right (alongside security, architecture, etc) but also for the consumer and the business.
The emphasis must be placed on the deep technical knowledge of experts in information systems – these must be the drivers and unifiers between the engineers, the lawyers, the advocates and ultimately the users. Without this deep, holistic, scientific and mathematical foundation we will not be able to sufficiently nor consistency address or govern any issues that arise in the construction of our information systems at any level of abstraction.
If the work we do in privacy does not have a scientific, consistent and formal underpinning [6] that brings together the engineers, lawyers and advocates then privacy is waste of time at best and deeply destructive to the information systems at worst.
Without this we are a disjointed community caring for ourselves and not the business or consumer and privacy becomes just a bureaucratic exercise to fulfill the notions of performing a process and metrics rendered as meaningful as random numbers.
* * *
PostScript:
Via Twitter I came across a talk given by Jean Bezivin entitled "Should we Resurrect Software Engineering?" presented at the Choose Forum in December 2012. Many of the things he presented are analogous to what is happening in privacy. He makes the point a number of times that we have never addressed the missing underlying theory of software engineering and how to really unify the various communities, fields and techniques in this area. Two points I particularly liked was that we concentrated on the solution (MDE) but never thought about the problem; the other point is the use Albert Camus' quote
<< Mal nommer les choses, c'est ajouterau malheur du monde >>
[To misname things is to add misery to the world]A subtle hint to getting the fundamentals right: terminology and semantics!
Notes
[1] #pii2012: The Emergent Privacy-Industrial Complex
[2] Jerry Seligmann, Jon Barwise (1997) Information Flow. Cambridge University Press.
[3] I like the idea of privacy being an inherent construct in system design in much the same way that inherent safety emerged from chemical/industrial plant design
[4] A blog article discussing “mathematical catastrophes” – two that come to mind are Russel and Frege and also Russel and Gödel. Both related but the latter’s challenge to the mathematical school of thought was dramatic to say the least.
[5] A formal retrenchment step in that we not just start again but actively record what we’re backtracking on. Poppleton et.al. constructed a theory of retrenchment for software design using formal methods; the same principles apply here.
[6] If you’re still in doubt just remember that whatever decisions are made with respect to privacy, there’s a programmer writing formal, mathematical statements encoding this. Lessig’s Code is Law principle.
Wednesday, 23 January 2013
Kubler-Ross and Getting Ideas Accepted
When discussing new or challenging ideas, or even anything that challenges or even questions the existing schools of thought (or business process!) there is often much "push-back" with responses such as "that'll never work", "impossible" etc...sometimes even when confronted with the evidence and demonstration.
Dealing with this is often soul destroying from the innovator's perspective and getting past this is 90% of the challenge of getting new ideas and view points accepted. So having a mechanism to understand the responses would be useful. I think the Kubler-Ross model might be useful here to examine people's responses.
The model itself was developed for psychologists to understand the process of grief. While the model has sparked some controversy, this does not detract from the basic principle of the model. The model consists of five sequential stages:
Let's say we have a process and metrics for some purpose - the process is complicated and dogmatic, the metrics measure some completion rate but not effort or compliance. A challenge to this might be met with the following responses:
This model must be used in an introspective and reflective manner to ensure that you as the originator and presenter of the idea do not fall into the trap of stages 1 and 2 yourself. Understanding your reactions in the above terms is very enlightening regarding your own behaviour.
If you do reach stage 3 in the discussions then this is time that you need to be absolutely sure in how your idea works, what the flaws are and how it integrates and improves what came previously. At this stage you have the chance to get everyone on board but after this however it is extremely difficult to turn people to your idea.
Stage 4 is depression all round, you will probably have accepted many changes to your idea and let go of some cherished ideas. Worse is that you've probably challenged the existing school and dogma to such a degree you are going to get a lot of "push back" on the ideas. In some respects this is where ideas do die either "naturally" or through "suicide" to use some dark terminology. To get through this stage you need to be the supporter of everyone. Indeed emphasis on the previous school of thought as being the catalyst to the newer ideas is critical to get through this; after all, wasn't it the previous systems that sparked the need for change in the first place?
Stage 5 is requires real leadership of the innovation and building of the team to carry this forward. Like it not, teamwork and ensuring that everyone, even the detractors, have a voice is critical. Sometimes your challenge might free some of the original detractors out of their earlier beliefs - this can come as quite a relief to these people and offer them badly needed, new challenges and purpose.
There are many more things one could write on this and there are many books and theories on how to manage innovation and invention elsewhere. The idea here was to relate some experiences with the Kubler-Ross model and understand things in that context, which personally I've found to be a very useful tool.
Dealing with this is often soul destroying from the innovator's perspective and getting past this is 90% of the challenge of getting new ideas and view points accepted. So having a mechanism to understand the responses would be useful. I think the Kubler-Ross model might be useful here to examine people's responses.
The model itself was developed for psychologists to understand the process of grief. While the model has sparked some controversy, this does not detract from the basic principle of the model. The model consists of five sequential stages:
- Denial - "we're fine", "everything works"
- Anger - "NO!"
- Bargaining - "Ok, so how do you fix this?"
- Depression - "Why bother...?", "Too difficult"
- Acceptance - "Let's do this!!!"
Let's say we have a process and metrics for some purpose - the process is complicated and dogmatic, the metrics measure some completion rate but not effort or compliance. A challenge to this might be met with the following responses:
- Denial - the process works! No-one has complained! We have metrics! We're CMM Level 3!
- Anger - Why are you complaining? We don't need to change!
- Bargaining - OK, we'll consider your ideas and changes but we're not promising anything. Can you come up with a project plan, budget, strategy, PowerPoints etc...?
- Depression - OK, there are problems, but we can't deal with them. It's too late and complex to change. Let's create a project plan, strategy and vision. How can we ever capture those metrics?
- Acceptance - You're right, let's run with this
This model must be used in an introspective and reflective manner to ensure that you as the originator and presenter of the idea do not fall into the trap of stages 1 and 2 yourself. Understanding your reactions in the above terms is very enlightening regarding your own behaviour.
If you do reach stage 3 in the discussions then this is time that you need to be absolutely sure in how your idea works, what the flaws are and how it integrates and improves what came previously. At this stage you have the chance to get everyone on board but after this however it is extremely difficult to turn people to your idea.
Stage 4 is depression all round, you will probably have accepted many changes to your idea and let go of some cherished ideas. Worse is that you've probably challenged the existing school and dogma to such a degree you are going to get a lot of "push back" on the ideas. In some respects this is where ideas do die either "naturally" or through "suicide" to use some dark terminology. To get through this stage you need to be the supporter of everyone. Indeed emphasis on the previous school of thought as being the catalyst to the newer ideas is critical to get through this; after all, wasn't it the previous systems that sparked the need for change in the first place?
Stage 5 is requires real leadership of the innovation and building of the team to carry this forward. Like it not, teamwork and ensuring that everyone, even the detractors, have a voice is critical. Sometimes your challenge might free some of the original detractors out of their earlier beliefs - this can come as quite a relief to these people and offer them badly needed, new challenges and purpose.
There are many more things one could write on this and there are many books and theories on how to manage innovation and invention elsewhere. The idea here was to relate some experiences with the Kubler-Ross model and understand things in that context, which personally I've found to be a very useful tool.
Tuesday, 31 January 2012
Extended CIMM (CMM)
A long time ago I used to be a CMM [3] auditor - it was fun at the time - but what was really interesting was the amount of effort spent on developing and desinging processes to satisfy the CMM audit rather than concentrating on the how the actual products (software and related services generally) was being made.
Those companies which we audited which came out as being CMM Level 2 generally had very expansive processes and many "quality" metrics - rarely these metrics actually had something to do with the product being produced. Even more rarely those companies knew how to act upon those relevant metrics.
Now while I tend to focus on the methods, techniques and metrics of the actually job of engineering and dismiss process - or at least overbearing process - CMM does have its place but does require the audit to be carried out by not only process people but real, live engineers too.
CMM has been however the butt of many jokes - the most famous of which was started by Anthony Finkelstein in his famous 1992 paper: A Software Process Immaturity Model [1]. This was later taken up by Schorsch [2] in 1996 and expanded into the "The Capability Im-Maturity Model (CIMM)".
To understand these a quick explanation of the CMM levels is required. CMM rates a company's processes in terms of their maturity ranked on a scale of 0 to 5:
Schorsch's expansion of this is:
NB: Finkelstein originally had levels 0 to -2 with the naming: 0 (foolish), -1 (stupid) and -2 (lunatic); the meanings are the same though.
Not content with faking their own performance, undermining organizations routinely work to downplay and sabotage the efforts of rival organizations, especially those successfully implementing processes common to CMM level 2 and higher. This is worst where company policy causes departments to compete for scarce resources, which are allocated to the loudest advocates.
After performing a number of review years ago this got me thinking at the time and I even wrote a short paper on this which will probably forever remain unpublished, namely as it is in some backup archive somewhere languishing on some slowly decaying magnetic media.
The CIMM (Capability Immaturity Model) is in the range [-3 to 5]:Integer. However I think it is more accurate to expand the CIMM levels into complex numbers, albeit integer ones. The two parts of a complex number are denoted real and imaginary - the meanings of those words fit nicely with auditing reality.
The eCIMM (Extended CIMM - my term!) level of a company can be expressed using both real and imaginary parts: the imaginary part, or what the company would like to believe, is what the auditors find out through the CMM audit, and the real part is actually what is happening inside the company. Just as CMM level 2 is the typically found, in the future this should be reported as something like eCIMM level -2+2i.
That is, the CMM auditors were presented with material showing that the company had spent much time on developing and documenting procesess and obtained a level 2 rating, but reality inside was that the processes exist but are routinely ignored and things fudged to make the organisation look good.
I guess that if we ever do find a CMM level 5 company, its eCIMM level would be 0+5i: a massive team of process people and one guy hacking (sorry, using agile methods) in the corner - and probably doing a good job of it too!
Now you could extend this by performing numerous audits and plotting the progress on and argand diagram - and you could probably come up with some interesting metrics and analysis of the distances between points - I guess it fits all the defintions of a metric and maybe might even be worth writing about.
Those companies which we audited which came out as being CMM Level 2 generally had very expansive processes and many "quality" metrics - rarely these metrics actually had something to do with the product being produced. Even more rarely those companies knew how to act upon those relevant metrics.
Now while I tend to focus on the methods, techniques and metrics of the actually job of engineering and dismiss process - or at least overbearing process - CMM does have its place but does require the audit to be carried out by not only process people but real, live engineers too.
CMM has been however the butt of many jokes - the most famous of which was started by Anthony Finkelstein in his famous 1992 paper: A Software Process Immaturity Model [1]. This was later taken up by Schorsch [2] in 1996 and expanded into the "The Capability Im-Maturity Model (CIMM)".
To understand these a quick explanation of the CMM levels is required. CMM rates a company's processes in terms of their maturity ranked on a scale of 0 to 5:
| Level | Name | Description of Maturity |
| 0 | Undefined | No audit or nothing at all |
| 1 | Initial | Chaotic, ad hoc and/or individual heroics |
| 2 | Repeatable | The process is at least documented sufficiently such that repeating the same steps may be attempted |
| 3 | Defined | The process is defined/confirmed as a standard business process |
| 4 | Managed | The process is quantitatively managed in accordance with agreed-upon metrics |
| 5 | Optimising | Process management includes deliberate process optimization/improvement |
Schorsch's expansion of this is:
| Level | Name | Description of Maturity |
| 0 | Negligent | The organization pays lip service, often with excessive fanfare, to implementing software engineering processes, but lacks the will to carry through the necessary effort. Whereas CMM level 1 assumes eventual success in producing software, CIMM level 0 organizations generally fail to produce any product, or do so by abandoning regular procedures in favor of crash programs. |
| -1 | Obstructive | Processes, however inappropriate and ineffective, are implemented with rigor and tend to obstruct work. Adherence to process is the measure of success in a Level -1 organization. Any actual creation of viable product is incidental. The quality of any product is not assessed, presumably on the assumption that if the proper process was followed, high quality is guaranteed. This is the most common level achieved by most organizations that pursue CMMI ratings. Paradoxically, Level -1 organizations believe fervently in following defined procedures, but lacking the will to measure the effectiveness of the procedures they rarely succeed at their basic task of creating software. Unfortunately, this behavior is inherent in the CMMI evaluation process. Since many government agencies will only award contracts over a certain dollar value to organizations that can pass a CMMI-3 or higher SCAMPI appraisal, management is more often than not willing to accept some inefficiencies in order to win these lucrative contracts. Government contracting models, in which organizations are paid not for the value of their products but by the number of hours spent building them, reward organizations for performing non-value-added activities related to CMMI compliance. Thus, government contractors with CMMI ratings will ultimately be more profitable than non-CMMI rated companies regardless of the quality of the software they produce. Whether CMMI processes provide any value in producing higher quality software is immaterial. |
| -2 | Contemptuous | While processes exist, they are routinely ignored by engineering staff and those charged with overseeing the processes are regarded with hostility. Measurements are fudged to make the organization look good. |
| -3 | Undermining | Not content with faking their own performance, undermining organizations routinely work to downplay and sabotage the efforts of rival organizations, especially those successfully implementing processes common to CMM level 2 and higher. This is worst where company policy causes departments to compete for scarce resources, which are allocated to the loudest advocates. |
NB: Finkelstein originally had levels 0 to -2 with the naming: 0 (foolish), -1 (stupid) and -2 (lunatic); the meanings are the same though.
Not content with faking their own performance, undermining organizations routinely work to downplay and sabotage the efforts of rival organizations, especially those successfully implementing processes common to CMM level 2 and higher. This is worst where company policy causes departments to compete for scarce resources, which are allocated to the loudest advocates.
After performing a number of review years ago this got me thinking at the time and I even wrote a short paper on this which will probably forever remain unpublished, namely as it is in some backup archive somewhere languishing on some slowly decaying magnetic media.
The CIMM (Capability Immaturity Model) is in the range [-3 to 5]:Integer. However I think it is more accurate to expand the CIMM levels into complex numbers, albeit integer ones. The two parts of a complex number are denoted real and imaginary - the meanings of those words fit nicely with auditing reality.
The eCIMM (Extended CIMM - my term!) level of a company can be expressed using both real and imaginary parts: the imaginary part, or what the company would like to believe, is what the auditors find out through the CMM audit, and the real part is actually what is happening inside the company. Just as CMM level 2 is the typically found, in the future this should be reported as something like eCIMM level -2+2i.
That is, the CMM auditors were presented with material showing that the company had spent much time on developing and documenting procesess and obtained a level 2 rating, but reality inside was that the processes exist but are routinely ignored and things fudged to make the organisation look good.
I guess that if we ever do find a CMM level 5 company, its eCIMM level would be 0+5i: a massive team of process people and one guy hacking (sorry, using agile methods) in the corner - and probably doing a good job of it too!
Now you could extend this by performing numerous audits and plotting the progress on and argand diagram - and you could probably come up with some interesting metrics and analysis of the distances between points - I guess it fits all the defintions of a metric and maybe might even be worth writing about.
- [1] A. Finkelstein, A Software Process Immaturity Model, SIGSOFT Software Engineering Notes, 1992.
- [2] Schorsch, "The Capability Im-Maturity Model (CIMM)", U.S. Air Force (CrossTalk Magazine), 1996.
- [3] Humphrey W.S.; Kitson D.H. & Kasse T.C. (1989); The State of Software Engineering Practice: a preliminary report; Proc. IEEE 11th International Conference on Software Engineering; pp 277-288, IEEE CS Press.
Postscript: at last I've written this down after years of not quite getting around to it...using complex numbers as CMM level indicators...
Subscribe to:
Posts (Atom)