New Year's Resolution: write more .... and .... 26 days later....finally got around to it. So after a prolonged break and spending most of it reading upon topics from organizational failures, safety, privacy and a touch of homomorphic encryption, I came around this:
which is an exceptionally well written summary of the problems at King's Cross Railway Station after Christmas 2014 where overrunning engineering works blocked all long distance and commuter rail services out of one of the London busiest termini.
What actually started out as a fantastic opportunity to perform engineering work over a rare, four day holiday became the proverbial nightmare as works overran by 24 hours. As a relatively simple study in the Swiss Cheese model it provides a wealth of issues and examples of how the holes in the Swiss Cheese lined up as a large number of minor problems coalesced into a "disaster".
Despite meticulous planning and well rehearsed contingency and emergency plans, and even pre-emptive measures such as providing extra machinery and fitters to correct problems on-site it eventually came down to a lack of drivers for freight trains that finally provided the final hole in the Safety Swiss Cheese.
An excellent example of how things go wrong despite detailed planning. Of course it is easy to judge with 20/20 hindsight and see clearly that a number of known failure points were obvious, but like many of these cases at each point the situational awareness was for many reasons lacking. Funny how the same aspects occur here in engineering as well as medicine, aviation and of course privacy.
A collection of discussions, links, stories, news and whatever else I find interesting in the fields of computing, information, science, privacy, semantics, mathematics and so on...
Showing posts with label Railways. Show all posts
Showing posts with label Railways. Show all posts
Monday, 26 January 2015
Monday, 5 August 2013
Understanding Software Engineering Accidents
The cause of 99.999...% of accidents is easy to ascertain, quite
simply it is the pilot's/driver's fault. In the cases of two recent
aviation and railway accidents (Asiana 214 and the Galicia's train crash) these were caused by the pilot and driver respectively....?
Finding the causes of an accident don't actually involve finding who is to blame but rather the whole context of what led up to the point where an accident was inevitable. And then even after that exploring what actually occurred during and after the accident.
The reasoning here is that if we concentrate on who to blame then we will miss all the circumstances that allowed that accident to occur in the first place. For example., in the case of the Galician train crash it has been ascertained that the driver was speeding, on the phone and had a history of breaking the rules. However this misses the more subtle questions of why did the train derail, why could the driver break the speed limit, what was the reasoning of the phone call, why did the carriages fail catastrophically after the derailment, did the safety systems work sufficiently, what state was the signalling in, did the driver override systems, was the driver sufficiently trained etc etc etc.
In other words, the whole context of the systems and organisation needs to be taken into account before appointing final blame; and even then very, very rarely is it single point of failure: the Swiss Cheese Model.
If we apply this model to software engineering and specifically accidents such as hacking and data breaches we become very aware of how many holes in our computing Swiss cheese we have.
Take a simple data breach where "hackers" have accessed a database on some server via an SQL injection via some web pages. If we apply our earlier thinking, it is obviously the fault of the 'stupid' system administrators who can't secure a system and the 'stupid' software engineers who can't write good code. Hindsight is great here isn't it?
To answer 'who to blame?' or better still 'why did things go wrong and how can we prevent this in the future?' we need to put ourselves, as other accident investigators do, in the position of those software engineers, system administrators, architects, hackers and managers AT THE POINT IN TIME WHERE THEY ACTED, and NEVER in hindsight.
Why did the trained, intelligent software engineer write code susceptible to SQLi in the first place?
Maybe they were under time pressure, no proper, formal design documentation, coding standards, never trained to spot those kinds of errors? Actually just stating that we have a failure of discipline is already pointing to wholesale failures across our whole organisation rather than just one individual.
Even if in the above case this was malice by the programmer, then why didn't the testing pick this up? Why was the code released without testing or checking? Now we have a failure elsewhere too.
Were there no procedures for this? Was the code signed-off by management with the risk in place? Now the net widens further across the organisation
and so on...
In aviation there is a term to explain most accidents: loss of situational awareness, which when explored invariably ends up with a multitude of 'wrong' choices being made over a longer period of time rather than just at those few critical minutes or hours in the cockpit.
I'm of the opinion that in software engineering that we almost always operate in a mode where we have no or little situational awareness. Our systems are complex, we lack formal models of our systems that clearly and concisely explain how the system works; indeed one of the maxims used by some practitioners of agile methods actively eschews the use of formality and modelling tools. Coupled with tight deadlines, a code-is-king mentality and rapidly and inconsistently changing requirements we have a fantastic recipe for disaster.
Bringing this back to an aviation analogy again, consider Turkish Airlines Flight 1951 which crashed as Schiphol in 2009. It was initially easy to blame the pilots for allowíng the plane to stall on final approach, but the whole accident investigation revealed deficiencies in the training, the approach procedures of Schiphol, a non-fault tolerant autothrottle and radar combination, a massively high workload situation for the pilots and ultimately a fault which manifested itself in precisely the behaviour that the pilots were requiring and expecting on their approach, that is the aircraft was losing speed.
As an exercise, how does the above accident map to what we experience every day in software engineering? Given high workloads, changing requirements, inconsistent planning and deadlines to get something (anything!) out that sort of works and we start getting answers to why intelligent administrators and programmers make mistakes.
Finding the causes of an accident don't actually involve finding who is to blame but rather the whole context of what led up to the point where an accident was inevitable. And then even after that exploring what actually occurred during and after the accident.
The reasoning here is that if we concentrate on who to blame then we will miss all the circumstances that allowed that accident to occur in the first place. For example., in the case of the Galician train crash it has been ascertained that the driver was speeding, on the phone and had a history of breaking the rules. However this misses the more subtle questions of why did the train derail, why could the driver break the speed limit, what was the reasoning of the phone call, why did the carriages fail catastrophically after the derailment, did the safety systems work sufficiently, what state was the signalling in, did the driver override systems, was the driver sufficiently trained etc etc etc.
In other words, the whole context of the systems and organisation needs to be taken into account before appointing final blame; and even then very, very rarely is it single point of failure: the Swiss Cheese Model.
If we apply this model to software engineering and specifically accidents such as hacking and data breaches we become very aware of how many holes in our computing Swiss cheese we have.
Take a simple data breach where "hackers" have accessed a database on some server via an SQL injection via some web pages. If we apply our earlier thinking, it is obviously the fault of the 'stupid' system administrators who can't secure a system and the 'stupid' software engineers who can't write good code. Hindsight is great here isn't it?
To answer 'who to blame?' or better still 'why did things go wrong and how can we prevent this in the future?' we need to put ourselves, as other accident investigators do, in the position of those software engineers, system administrators, architects, hackers and managers AT THE POINT IN TIME WHERE THEY ACTED, and NEVER in hindsight.
Why did the trained, intelligent software engineer write code susceptible to SQLi in the first place?
Maybe they were under time pressure, no proper, formal design documentation, coding standards, never trained to spot those kinds of errors? Actually just stating that we have a failure of discipline is already pointing to wholesale failures across our whole organisation rather than just one individual.
Even if in the above case this was malice by the programmer, then why didn't the testing pick this up? Why was the code released without testing or checking? Now we have a failure elsewhere too.
Were there no procedures for this? Was the code signed-off by management with the risk in place? Now the net widens further across the organisation
and so on...
In aviation there is a term to explain most accidents: loss of situational awareness, which when explored invariably ends up with a multitude of 'wrong' choices being made over a longer period of time rather than just at those few critical minutes or hours in the cockpit.
I'm of the opinion that in software engineering that we almost always operate in a mode where we have no or little situational awareness. Our systems are complex, we lack formal models of our systems that clearly and concisely explain how the system works; indeed one of the maxims used by some practitioners of agile methods actively eschews the use of formality and modelling tools. Coupled with tight deadlines, a code-is-king mentality and rapidly and inconsistently changing requirements we have a fantastic recipe for disaster.
Bringing this back to an aviation analogy again, consider Turkish Airlines Flight 1951 which crashed as Schiphol in 2009. It was initially easy to blame the pilots for allowíng the plane to stall on final approach, but the whole accident investigation revealed deficiencies in the training, the approach procedures of Schiphol, a non-fault tolerant autothrottle and radar combination, a massively high workload situation for the pilots and ultimately a fault which manifested itself in precisely the behaviour that the pilots were requiring and expecting on their approach, that is the aircraft was losing speed.
As an exercise, how does the above accident map to what we experience every day in software engineering? Given high workloads, changing requirements, inconsistent planning and deadlines to get something (anything!) out that sort of works and we start getting answers to why intelligent administrators and programmers make mistakes.
Wednesday, 1 February 2012
Tuesday, 18 October 2011
UK Railway Map
Last posting on railways today...a superb railway map of the UK (New Adlestrop Railway Atlas) showing all lines open and closed which together build a fascinating study of UK railway history. Here's a quick extract (which coincidentally corresponds with my postings on the L&TVJR here and here):
The designer (Richard Fairhurst) has a number of other maps available at his map website here.
The are some updates required - at least in Wales - eg: the Ebbw Vale line is now in passenger operation, the occasional passenger train runs on the Swansea District Line between Briton Ferry and Llangennech and the Vale of Glamorgan Line. However this hardly distracts from the quality of the map itself.
NB: the map is copyright Richard Fairhurst!
The designer (Richard Fairhurst) has a number of other maps available at his map website here.
The are some updates required - at least in Wales - eg: the Ebbw Vale line is now in passenger operation, the occasional passenger train runs on the Swansea District Line between Briton Ferry and Llangennech and the Vale of Glamorgan Line. However this hardly distracts from the quality of the map itself.
NB: the map is copyright Richard Fairhurst!
Llantrisant and Taff Vale on Facebook?
Someone (or something) has copied the Wikipedia articles that I mainly authored on the Llantriant and Taff Vale Junction Railway and made Facebook pages out of them...not quite sure why unless they're trying to get a following for the line and individual stations - not really sure how social networking really helps a disused railway with no chance of reopening (pity!)....
Some links to the Facebook pages:
I don't really mind, it is in the spirit of Wikiedia and the articles are according the creative commons licensing; nice to be sort of famous....anyway the Wikipedia articles are the authorative source along with Chapman's book on the line:
Some links to the Facebook pages:
I don't really mind, it is in the spirit of Wikiedia and the articles are according the creative commons licensing; nice to be sort of famous....anyway the Wikipedia articles are the authorative source along with Chapman's book on the line:
Chapman, C (1996) The Llantrisant Branches of the Taff Vale Railway. The Oakwood Press. ISBN 0 85361 4814Here's the original blog article from here linking to the Wikipedia articles...
Saturday, 3 September 2011
Severn Tunnel, 125 years
Severn Tunnel linking Wales to England is 125 years old
3 September 2011 Last updated at 08:35 GMTCarwin Jones
The Severn Tunnel rail link which joins south Wales and England, is 125 years old.
About 200 trains pass through this feat of Victorian engineering every day, but given its age, is it still fit for 21st Century use?
Thursday, 18 August 2011
"Siberia across your screen"
Interesting service from Russian Railways by way of the Railway Gazette:
Siberia across your screen
Siberia across your screen
22 February 2010, Railway Gazette
and the service provided by Google:RUSSIA: Google and Russian Railways have launched a ‘virtual’ Trans-Siberian journey from Moscow to Vladivostok.Anyone with a broadband internet connection will be able to make a 'journey' along the route starting from any city or station without leaving home.
Video footage totalling around 150 h was shot through a train window during daylight hours, enabling the viewer to watch almost 9 300 km of scenery pass across their screen. The journey can also be tracked using Google Maps, allowing the user to click on any point to see footage of that section of route.
Moscow-Vladivostok: virtual journey on Google Maps
The great Trans Siberian Railway, the pride of Russia, goes across two continents, 12 regions and 87 cities. The joint project of Google and the Russian Railways lets you take a trip along the famous route and see Baikal, Khekhtsirsky range, Barguzin mountains, Yenisei river and many other picturesque places of Russia without leaving your house. During the trip, you can enjoy Russian classic literature, brilliant images by photographer Anton Lange and fascinating stories about the most attractive sites on the route. Let's go!
Monday, 31 January 2011
Friday, 21 January 2011
Llantrisant and Taff Vale Junction Railway
Interesting article on Wikipedia (quite a lot of which I'm responsible for...especially the map) about the railway line between Pontypridd and Llantrisant as well as the "Barry Railway" branch:
Llantrisant and Taff Vale Junction Railway
Additionally some fantastic pictures of Tonteg Halt and Junction. This was the post 1930 Tonteg Halt station after track rationalisation between Tonteg and Treforest High Level and Llantwit Road, the former Tonteg Halt was further down Maes Mawr road.
Llantrisant and Taff Vale Junction Railway
Additionally some fantastic pictures of Tonteg Halt and Junction. This was the post 1930 Tonteg Halt station after track rationalisation between Tonteg and Treforest High Level and Llantwit Road, the former Tonteg Halt was further down Maes Mawr road.
Tuesday, 14 December 2010
Frequency Annotated Rail Map of France
Really interesting...would like to see something similar for Finland or the UK...
http://membres.multimania.fr/cartesferro/france/france_en.html
http://membres.multimania.fr/cartesferro/france/france_en.html
Sunday, 12 December 2010
Thursday, 30 September 2010
Receipt - Kuluttajavirasto
Good information about receipts and their validity for customers in Finland
Receipt - Kuluttajavirasto
Receipt - Kuluttajavirasto
Tuesday, 31 August 2010
Train timetables
A useful and interesting service from Finland's Ratahalintokeskus (RHK)...the ability to generate graphical timetables for any stretch of track and date in Finland (regular traffic only):
Säännöllisen liikenteen aikataulujen haku (Regular Traffic Timetable Search)
Säännöllisen liikenteen aikataulujen haku (Regular Traffic Timetable Search)
Sunday, 29 August 2010
Trains v People
In the "war between people and trains", trains inevitably win....occasionally even ghost trains win...especially when they turn out to be real trains.
Not sure if this guy qualifies for a Darwin Award, but....
Amateur NC ghost hunter looking for 'ghost train' hit and killed by real train
Not sure if this guy qualifies for a Darwin Award, but....
Amateur NC ghost hunter looking for 'ghost train' hit and killed by real train
By Jeff Rivenbark - email
STATESVILLE, NC (WBTV) - A man who was with about a dozen people who were looking for a legendary "ghost train" in Iredell County was hit by a locomotive and killed early Friday morning.
Saturday, 5 December 2009
Monday, 9 November 2009
Helsinki Metro - Disaster or Advertisment?
A major break in a water pipe flooded part of the Helsinki Metro at the Rautatientori metro station last night.
A wonderful photograph has appeared on the MTV3 Uutiset website of the situation at the platform level, it possibly describes what has actually happened.
(C)MTV3 Finland
The picture shows the escalators leading to the main railway station from the metro station platforms. On the floor is an advertisment for the film 2012 - one of the post 9/11, Global-Warming, Flu-panic genre modern disaster films...the text reads: "KOE LOPUN ALKU" - "experience the beginning of the end". The picture in the advertisment shows a statue being overwhelmed by a great flood....
...the irony!!
Now YLE Uutiset is running a story Helsingin aseman Kompassitorin halkeamista sortumavaara - Helsinki Station's Kompassitori Collapse Danger.
So...given a couple of days we're going to have the metro station and part of central Helsinki disappearing into a big hole (there'll be much celebration on the streets of Nikkilä, Östersundom and Södekulla !) and Helsinki finally will get two metro lines...
A wonderful photograph has appeared on the MTV3 Uutiset website of the situation at the platform level, it possibly describes what has actually happened.
(C)MTV3 Finland
The picture shows the escalators leading to the main railway station from the metro station platforms. On the floor is an advertisment for the film 2012 - one of the post 9/11, Global-Warming, Flu-panic genre modern disaster films...the text reads: "KOE LOPUN ALKU" - "experience the beginning of the end". The picture in the advertisment shows a statue being overwhelmed by a great flood....
...the irony!!
Now YLE Uutiset is running a story Helsingin aseman Kompassitorin halkeamista sortumavaara - Helsinki Station's Kompassitori Collapse Danger.
So...given a couple of days we're going to have the metro station and part of central Helsinki disappearing into a big hole (there'll be much celebration on the streets of Nikkilä, Östersundom and Södekulla !) and Helsinki finally will get two metro lines...
Subscribe to:
Posts (Atom)