Posts From July, 2016

The History of Big Data 

Always interested in news and articles about Big Data I ran into the an article about the history of Big Data you simply don't want to miss.

It’s a quite interesting time line which will explain as well that actually Big Data has always been around in the world.

C 18,000 BCE

The earliest examples we have of humans storing and analyzing data are the tally sticks. The Ishango Bone was discovered in 1960 in what is now Uganda and is thought to be one of the earliest pieces of evidence of prehistoric data storage. Palaeolithic tribespeople would mark notches into sticks or bones, to keep track of trading activity or supplies. They would compare sticks and notches to carry out rudimentary calculations, enabling them to make predictions such as how long their food supplies would last.

C 2400 BCE

The abacus – the first dedicated device constructed specifically for performing calculations – comes into use in Babylon. The first libraries also appeared around this time, representing our first attempts at mass data storage.

300 BC - 48 AD

The Library of Alexandria is perhaps the largest collection of data in the ancient world, housing up to perhaps half a million scrolls and covering everything we had learned so far, about pretty much everything. Unfortunately, in 48AD it is thought to have been destroyed by the invading Romans, perhaps accidentally. Contrary to common myth, not everything was lost – significant parts of the library’s collections were moved to other buildings in the city, or stolen and dispersed throughout the ancient world.

C 100 – 200 AD

The Antikythera Mechanism, the earliest discovered mechanical computer, is produced, presumably by Greek scientists. It’s “CPU” consists of 30 interlocking bronze gears and it is thought to have been designed for astrological purposes and tracking the cycle of Olympic Games. Its design suggests it is probably an evolution of an earlier device – but these so far remain undiscovered.

The Emergence of Statistics

1663

In London, John Graunt carries out the first recorded experiment in statistical data analysis. By recording information about mortality, he theorized that he can design an early warning system for the bubonic plague ravaging Europe.

1865

The term “business intelligence” is used by Richard Millar Devens in his Encyclopaedia of Commercial and Business Anecdotes, describing how the banker Henry Furnese achieved an advantage over competitors by collecting and analyzing information relevant to his business activities in a structured manner. This is thought to be the first study of a business putting data analysis to use for commercial purposes.

1880

The US Census Bureau has a problem – it estimates that it will take it 8 years to crunch all the data collected in the 1880 census, and it is predicted that the data generated by the 1890 census will take over 10 years, meaning it will not even be ready to look at until it is outdated by the 1900 census. In 1881 a young engineer employed by the bureau – Herman Hollerith – produces what will become known as the Hollerith Tabulating Machine. Using punch cards, he reduces 10 years’ work to three months and achieves his place in history as the father of modern automated computation. The company he founds will go on to become known as IBM.

The Early Days of Modern Data Storage

1926

Interviewed by Colliers magazine, inventor Nikola Tesla states that when wireless technology is “perfectly applied the whole Earth will be converted into a huge brain, which in fact it is, all things being particles of a real and rhythmic whole … and the instruments through which we shall be able to do this will be amazingly simple compared to our present telephone. A man will be able to carry one in his vest pocket.”

1928

Fritz Pfleumer, a German-Austrian engineer, invents a method of storing information magnetically on tape. The principles he develops are still in use today, with the vast majority of digital data being stored magnetically on computer hard disks.

 

1944

Fremont Rider, librarian at Wesleyan University, Connecticut, US, published a paper titled The Scholar and the Future of the Research Library.

In one of the earliest attempts to quantify the amount of information being produced, he observes that in order to store all the academic and popular works of value being produced, American libraries would have to double their capacity every 16 years. This led him to speculate that the Yale Library, by 2040, will contain 200 million books spread over 6,000 miles of shelves.

The Beginnings of Business Intelligence

1958

IBM researcher Hans Peter Luhn defines Business Intelligence as “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.”

1962

The first steps are taken towards speech recognition, when IBM engineer William C Dersch presents the Shoebox Machine at the 1962 World Fair. It can interpret numbers and sixteen words spoken in the English language into digital information.

1964

An article in the New Statesman refers to the difficulty in managing the increasing amount of information becoming available.

The Start of Large Data Centers

1965

The US Government plans the world’s first data center to store 742 million tax returns and 175 million sets of fingerprints on magnetic tape.

1970

IBM mathematician Edgar F Codd presents his framework for a “relational database”. The model provides the framework that many modern data services use today, to store information in a hierarchical format, which can be accessed by anyone who knows what they are looking for. Prior to this accessing data from a computer’s memory banks usually required an expert.

1976

Material Requirements Planning (MRP) systems are becoming more commonly used across the business world, representing one of the first mainstream commercial uses of computers to speed up everyday processes and make efficiencies. Until now, most people have probably only seen them in research and development or academic settings.

1989

Possibly the first use of the term Big Data (without capitalization) in the way it is used today. International best-selling author Erik Larson pens an article for Harpers Magazine speculating on the origin of the junk mail he receives. He writes: “The keepers of big data say they are doing it for the consumer’s benefit. But data have a way of being used for purposes other originally intended.”

Additionally “business intelligence” – already a popular concept since the late 50s – sees a surge in popularity with newly emerging software and systems for analyzing commercial and operational performance.

The Internet Kicks Into Gear

1991

Computer scientist Tim Berners-Lee announced the birth of what would become the World Wide Web as we know it today. In a post in the Usenet group alt.hypertext he sets out the specifications for a worldwide, interconnected web of data, accessible to anyone from anywhere.

1996

According to R J T Morris and B J Truskowski in their 2003 book The Evolution of Storage Systems, this is the point where digital storage became more cost effective than paper.

1997

Michael Lesk publishes his paper How Much Information is there in the World? Theorizing that the existence of 12,000 petabytes is “perhaps not an unreasonable guess”. He also points out that even at this early point in its development, the web is increasing in size 10-fold each year. Much of this data, he points out, will never be seen by anyone and therefore yield no insight.

Google Search also debuts this year – and for the next 20 years (at least) its name will become shorthand for searching the internet for data.

Early Ideas of Big Data

1999

A couple of years later and the term Big Data appears in Visually Exploring Gigabyte Datasets in Real Time, published by the Association for Computing Machinery. Again the propensity for storing large amounts of data with no way of adequately analyzing it is lamented. The paper goes on to quote computing pioneer Richard W Hamming as saying: “The purpose of computing is insight, not numbers.”

Also possibly first use of the term “Internet of Things”, to describe the growing number of devices online and the potential for them to communicate with each other, often without a human “middle man”. The term is used as the title of a presentation given to Procter and Gamble by RFID pioneer Kevin Ashton.

2000

In How Much Information? Peter Lyman and Hal Varian (now chief economist at Google) attempted to quantify the amount of digital information in the world, and its rate of growth, for the first time. They concluded: “The world’s total yearly production of print, film, optical and magnetic content would require roughly 1.5 billion gigabytes of storage. This is the equivalent of 250 megabytes per person for each man, woman and child on Earth.”

2001

In his paper 3D Data Management: Controlling Data Volume, Velocity and Variety Doug Laney, analyst at Gartner, defines three of what will come to be the commonly-accepted characteristics of Big Data.

This year also see the first use of the term “software as a service” – a concept fundamental to many of the cloud-based applications which are industry-standard today – in the article Strategic Backgrounder: Software as a Service by the Software and Information Industry Association.

Web 2.0 Increases Data Volumes

2005

Commentators announce that we are witnessing the birth of “Web 2.0” – the user-generated web where the majority of content will be provided by users of services, rather than the service providers themselves. This is achieved through integration of traditional HTML-style web pages with vast back-end databases built on SQL. 5.5 million people are already using Facebook, launched a year earlier, to upload and share their own data with friends.

This year also sees the creation of Hadoop – the open source framework created specifically for storage and analysis of Big Data sets. Its flexibility makes it particularly useful for managing the unstructured data (voice, video, raw text etc) which we are increasingly generating and collecting.

Today’s Use of the Term ‘Big Data’ Emerges

2007

Wired brings the concept of Big Data to the masses with their article The End of Theory: The Data Deluge Makes the Scientific Model Obsolete.

2008

The world’s servers process 9.57 zettabytes (9.57 trillion gigabytes) of information – equivalent to 12 gigabytes of information per person, per day), according to the How Much Information? 2010 report. In International Production and Dissemination of Information, it is estimated that 14.7 exabytes of new information are produced this year.

2009

The average US company with over 1,000 employees is storing more than 200 terabytes of data according to the report Big Data: The Next Frontier for Innovation, Competition and Productivity by McKinsey Global Institute.

2010

Eric Schmidt, executive chairman of Google, tells a conference that as much data is now being created every two days, as was created from the beginning of human civilization to the year 2003.

2011

The McKinsey report states that by 2018 the US will face a shortfall of between 140,000 and 190,000 professional data scientists, and states that issues including privacy, security and intellectual property will have to be resolved before the full value of Big Data will be realised.

2014

The rise of the mobile machines – as for the first time, more people are using mobile devices to access digital data, than office or home computers. 88% of business executives surveyed by GE working with Accenture report that big data analytics is a top priority for their business.

Final Thought

What this teaches us is that Big Data is not a new or isolated phenomenon, but one that is part of a long evolution of capturing and using data. Like other key developments in data storage, data processing and the Internet, Big Data is just a further step that will bring change to the way we run business and society. At the same time it will lay the foundations on which many evolutions will be built.

 

Quite interesting isn’t it?

If you do want to know more what exactly Big Data can mean for your company simply sent an email to bigdata@ave.lu and we will be more than happy to explain how we could help you and your business.

Cheers,

Frits Steketee

 

Source: Bernard Marr

Posted by Frits Steketee Wednesday, July 27, 2016 4:16:00 PM

The 10 Benefits of working in the Cloud 

Around 20 years ago I was working at a commune in The Netherlands near The Hague.

A commune with a nice City Hall and around the 50.000 people living there.

Most of the employees working for the commune were located in the City Hall. They did have some external locations as well but mostly for the people working outside in the commune.

It was a time before the internet; yes that is how old I am…

It was also a time that companies rarely had access to decent networks, although this was one of the lucky ones and actually had one…

Well, I say “network”…

They actually had a network with one server. Backups where made with a “tapestreamer” – remember those…? -  connecting 4 departments (4 rooms) to each other just for the digital designs. 

Remember too – taking copies of the tape home or away from the office so that if the building burnt down we had a backup copy…

This was really something ahead of its time…

Rather than creating the Design Documents on our individual desktop we were able to store them on the server; and we all had access to the other documents needed for our project from the other departments.

This was really a great solution; for then !!!

As time progressed, so did the way that networks were installed and managed…

Networks became located on servers within air-conditioned rooms within a building, requiring a few people to run and maintain them. They would also send out messages instructing users to “log off” on a regular basis…

And even then – tapes and discs had to be copied and located safely in other buildings to comply with insurance and business continuity procedures…

All documents, internet & intranet site and some software applications were located on those servers; Office tools and software were installed locally on the desktop or laptops. 

In those day laptops needed to use docking stations in order to get a connection to the network.

In hindsight, the commune did have a quite impressive server room. Special floors, climate control, a halo system in case of fire, etc…

Pretty impressive but also pretty expensive.

Nowadays companies have the opportunity to change the way their network environment operates and move it into the Cloud.

Operating in the Cloud means you don’t need your own servers, you don’t need backup tapes and discs, you don’t need to log-off for maintenance – and you don’t need to have a copy of everything in your garage just in case the office burns down…

In the Cloud – you can access all your data all the time from anywhere – you don’t need local software, just internet connections and access codes and you are in business…

In fact, if you are not in the Cloud, you are missing out; if your competitors are there but you are not, you are definitely missing out..

The 10 benefits of Working in the Cloud

  1. Achieve economies of scale – increase volume output or productivity with fewer people.
  2. Reduce spending on technology infrastructure - maintain easy access to your information with minimal upfront spending. Pay as you go (weekly, quarterly or yearly), based on demand.
  3. Globalise your workforce - People worldwide can access the cloud, provided they have an Internet connection and access authority.
  4. Streamline processes - Get more work done in less time with less people.
  5. Reduce capital costs - There’s no need to spend big money on hardware, software or licensing fees.
  6. Improve accessibility - You have access anytime, anywhere, making your life so much easier!
  7. Monitor projects more effectively - Stay within budget and ahead of completion cycle times.
  8. Less personnel training is needed - It takes fewer people to do more work on a cloud, with a minimal learning curve on hardware and software issues.
  9. Minimize licensing new software - Stretch and grow without the need to buy expensive software licenses or programs. You are able to add or reduce the licenses as you please.
  10. Improve flexibility - You can change direction without serious “people” or “financial” issues at stake. People can work from home or whatever place with an internet connection

Interesting isn’t it?

So now you are thinking…

“Could this cost saving more accurate way of working be something for my company?”

The answer is pretty simple… “Yes, it can be absolutely something for your company”

Like to know more about having your network in the Cloud ?

Please feel free to contact me at fsteketee@ave.lu or give me a callon +352 621 356 601

Thanks in advance.

 

ps. pretty warm today, so sitting in the shadow in the garden working on my laptop. And having all my software and data in the Cloud it makes it much easier to do J

Posted by Frits Steketee Wednesday, July 27, 2016 12:30:00 PM