UK’s National Health Services contractors pushed patient data to the cloud, for no good reason and without any patient permission & we’re applauded for it when they should have been fired

Commentary By Jarrett Neil Ridlinghafer –
“First off, this consulting company claims it took 24 hours to process a batch of data on their own servers that they were then able to process in the cloud in 30 minutes. They were then awarded a bunch of “kudos” and an award by the UK IT Industry whatever that is, and another by Computer Weekly both of which are laughable to the extreme….

The issues are much deeper than this one incident which however does point to the much larger problems if you have eyes and the experience to “see”. And that is the fact that even so called industry analysts who work for the tech magazines have bought into the cloud miracle hype hook line and sinker…. The awful reality is that practically all the young college graduates from the past 6 to 8 Years now are 100 percent behind the public cloud and if the opinion it is the “end all to be all” and “can do no wrong” because the truth is they have no other points of reference to know otherwise!

Here is a new rule of thumb, DO NOT TRUST ANY CLOUD ANALYST AGED UNDER 40 if you do, your blinding yourself to the truth and reality which is the wrong way to run your business I guarantee you.

I’m fortunate to have started my career when the Internet started almost exactly.. I started prior to the browser when the BBS was the big thing going and AOL was king of everything Internet. I was again fortunate to be hired at the company that started it all “Netscape Communications Corp” and to work with the man who everyone owes a great deal to today Marc Andreessen (for those so-called Analysts who don’t know who that is… Go look it up! 🙂 After Netscape where we all worked 5-6 Years 24x7x365 days a year to launch the “Internet Revolution” as it was coined I started my own broadband ISP out of my home and built my first Data-Center, call center and NOC all from Scratch and having never touched a commercial network device prior to that point, I was forced to learn everything on my own which I did and have been doing ever since. After over twenty Data-Centers designed and built from concept to production, and 25 plus years of hard work at age 53 I feel I as just barely far enough along in my experience to offer an educated opinion based on my own experiences and to put the title of “Technology Analyst” after my name. I’ve just scratched the surface of the knowledge to be learned with regards to the data infrastructure and the cloud in particular and so when I see people in their twenties with the term “Analyst” besides their name, I run the other direction and so should you, they were not even born when the first browser was invented, how much practical experience could that person actually have under their belt in order to provide objective analysis of anything?

So that my friends is issue number one. Go look and you will find dozens of what I call “kiddy” tech sites where their analysts are 17 and 21 year old still in school or working their first job, it’s a pretty scary thing to see the “blind leading the blind” that way and yet they have a following because they cater the the population on the net ignored by big corporations…. Which I think has been a huge mistake and this is the result.

The next and even bigger issue is one I’ve been attempting to teach people from recruiters to C-level execs about for over three years now, after I saw what was happening, and that is the blatant and truly dangerous lies of the marketing campaigns by the major Cloud Service Providers or “CSP”. It has left people believing a lie, pretty much everyone from young software developers to grizzled CEO’s have been blanketed by these false assertions that “the Cloud is THE safest place for your data and your entire business infrastructure” I cannot tell you how many people truly believe that nonsense and have to be re-educated to the facts and the truth and it’s not an easy thing to do sometimes while others they knew all along something smelled rotten but we’re not sure which was why they called someone like myself to help. For real life examples I spoke to a recruiter who said to me, after listening to my 30 minute rant about the old hype “I thought Amazon was the Cloud” indeed I’ve discovered since that many believe the word “Amazon” to be synonymous with “Cloud”, the scary thing to my mind and when I realized just how sinister and invasive this had become was when he told me that the CEO for the company he was recruiting, was the person who explained the cloud to him…. And indeed the reason we were having the conversation in the first place was the job description contained the third issue I’ve seen come from this “greed” my CSP’s, and that is the attempt to combine the infrastructure into software development!!! In my opinion this is the scariest result of anything they could have done, as I believe it is one of, if not the major root cause of the current “crisis” which we have in the entire industry (The entire Internet Industry my friends)

The truth is, if they’re internal infrastructure was taking that long then it was a design flaw and instead of being given an award they should have been instructed on how to build a proper analytics cluster and a private cloud, not, like so many others do, blame the systems they have and rush to the cloud with out ANY THOUGHT which is obviously what they did, as a simple brainstorm session should have immediately brought up the security and compliance concerns these two women eventually expressed themselves….

This should be a hard lesson and reality check for all those depending on consultants who either don’t have the experience or knowledge to know better, or to even figure out that “hey maybe we’re doing something wrong here, there’s no way it should be taking 24 hours to process this data….”

I advise all my clients and people I give speeches to to never jump into the cloud based on emotional responses to anything especially not pressure from external forces, that is exactly when mistakes occur. The cloud is not something to do frivolously or without deep thought and long planning, it REQUIRES careful planning and evaluation typically taking months, by an experienced infrastructure expert with a minimum of 15 years experience in Security & Compliance, Data-Center Infrastructure and Cloud Technology and is not something your 19 year old IT administrator should be your “expert” for, unless you care nothing about your business or customer data.”

Begin Article –

The UK’s National Health Service is at the centre of a fresh data privacy storm after a firm of technology consultants uploaded huge amounts of patient data to an offshore cloud in an effort to speed up an analytics processing job.

PA Consulting was contracted by the NHS to prove out a model for using hospital data to generate better patient outcomes for the UK health sector.

Under the agreement, PA purchased an archive of aggregated hospital activity across the UK, known as the Hospital Episode Statistics (HES).

The consulting firm installed the data on its hardware and uploaded it into a Microsoft SQL database on a 1TB server such that it could query the data to find out a range of information, such as admissions for particular conditions during the past decade.

PA’s promotional materials, published online [pdf], noted that running these queries on its own hardware “still took several hours.”

“With time and more storage space we could dramatically improve these times by normalising the tables and building suitable cubes. We could also load it into a more sophisticated analytics engine with dedicated hardware and software, but that would require significant capital investment.”

The consulting firm came up with a workaround: to upload HES to the Google Storage cloud and extract insights from it using Google’s BigQuery analytics web service. PA found queries that took all night on its internal servers were returned in 30 seconds using cloud services.

Within two weeks of using the Google tools, PA was able to produce interactive maps from the data, each of which took mere seconds to generate. PA estimated that each of these maps would previously have taken months to produce. 

The company felt that this proved out the power of using a cloud service for batch analytics jobs.

“The slowness of the current process severely limits the quality and number of interesting results,” PA’s brochure stated. “This means that analysis that should be highly influential in changing the way people allocate the NHS budget is not done at all or is too slow to have an effect.”  

The process could be improved without a large capital investment in technology, the firm noted.

PA Consulting has enjoyed kudos for its achievement, winning a UK IT Industry Award for best big data project and another from UK IT journalComputerWeekly.

But links to these awards have since been pulled from the PA website after Conservative MP Sarah Wollaston, a member of the UK’s Health Select Committee, posted a tweet questioning whether any patients had consented to having their data uploaded.