How big MNC’s like Google, Facebook, Instagram etc stores, manages and manipulate Thousands of Terabytes of data with High Speed and High Efficiency.

Big Data is today, the hottest buzzword around, and with the amount of data being generated every minute by consumers, or/and businesses worldwide, there is huge value to be found in Big Data analytics.

What is Big Data?

Big Data is a massive amount of data sets that cannot be stored, processed, or analyzed using traditional tools.

Today, there are millions of data sources that generate data at a very rapid rate. These data sources are present across the world. Some of the largest sources of data are social media platforms and networks. Let’s use Facebook as an example it generates more than 500 terabytes of data every day. This data includes pictures, videos, messages, and more.

Data also exists in different formats, like structured data, semi-structured data, and unstructured data. For example, in a regular Excel sheet, data is classified as structured data with a definite format. In contrast, emails fall under semi-structured, and your pictures and videos fall under unstructured data. All this data combined makes up Big Data. But, Big Data in its raw form is of no use.

What is Big Data Analytics?

Big Data analytics is a process used to extract meaningful insights, such as hidden patterns, unknown correlations, market trends, and customer preferences. Big Data analytics provides various advantages :it can be used for better decision making, preventing fraudulent activities, among other things.

How Does Big Data Work?

Collect. Collecting the raw data :transactions, logs, mobile devices and more ,is the first challenge many organizations face when dealing with big data. A good big data platform makes this step easier, allowing developers to ingest a wide variety of data(from structured to unstructured)at any speed from real-time to batch.

Store. Any big data platform needs a secure, scalable, and durable repository to store data prior or even after processing tasks. Depending on your specific requirements, you may also need temporary stores for data in-transit.

Process & Analyze. This is the step where data is transformed from its raw state into a consumable format usually by means of sorting, aggregating, joining and even performing more advanced functions and algorithms. The resulting data sets are then stored for further processing or made available for consumption via business intelligence and data visualization tools.

Consume & Visualize. Big data is all about getting high value, actionable insights from your data assets. Ideally, data is made available to stakeholders through self-service business intelligence and agile data visualization tools that allow for fast and easy exploration of datasets. Depending on the type of analytics, end-users may also consume the resulting data in the form of statistical “predictions” in the case of predictive analytics or recommended actions in the case of prescriptive analytics.

Advantages of Big Data Analytics

1. Risk Management

Use Case: Banco de Oro, a Philippine banking company, uses Big Data analytics to identify fraudulent activities and discrepancies. The organization leverages it to narrow down a list of suspects or root causes of problems.

2. Product Development and Innovations

Use Case: Rolls-Royce, one of the largest manufacturers of jet engines for airlines and armed forces across the globe, uses Big Data analytics to analyze how efficient the engine designs are and if there is any need for improvements.

3. Quicker and Better Decision Making Within Organizations

Use Case: Starbucks uses Big Data analytics to make strategic decisions. For example, the company leverages it to decide if a particular location would be suitable for a new outlet or not. They will analyze several different factors, such as population, demographics, accessibility of the location, and more.

4. Improve Customer Experience

Use Case: Delta Air Lines uses Big Data analysis to improve customer experiences. They monitor tweets to find out their customers’ experience regarding their journeys, delays, and so on. The airline identifies negative tweets and does what’s necessary to remedy the situation. By publicly addressing these issues and offering solutions, it helps the airline build good customer relations.

Big Data Industry Applications

Here are some of the sectors where Big Data is actively used:

• Ecommerce — Predicting customer trends and optimizing prices are a few of the ways e-commerce uses Big Data analytics
• Marketing — Big Data analytics helps to drive high ROI marketing campaigns, which result in improved sales
• Education — Used to develop new and improve existing courses based on market requirements
• Healthcare — With the help of a patient’s medical history, Big Data analytics is used to predict how likely they are to have health issues
• Media and entertainment — Used to understand the demand of shows, movies, songs, and more to deliver a personalized recommendation list to its users
• Banking — Customer income and spending patterns help to predict the likelihood of choosing various banking offers, like loans and credit cards
• Telecommunications — Used to forecast network capacity and improve customer experience
• Government — Big Data analytics helps governments in law enforcement, among other things

MNCs Use cases: Amazon

In a world where competition is intense, users will simply dump you, if your app slows down or freezes. So your ‘downtime’ has to be pretty much close to zero. For the user, whether they are in New York, or Tokyo, or Ankara or Mumbai, you have to be always up, and always running, 24 hours a day.

Amazon:

Amazon is better known to the vast majority of us as the world’s largest online retailer, but to the tech community it is also the equivalent of an electric utility. Both Instagram and Pinterest installed and ran their software on Amazon’s ‘cloud’ computing platform.

All their data is stored on servers and in data centres (essentially vast warehouses with hundreds if not thousands of servers loaded with hard disks)owned and operated by Amazon and rented to companies like Instagram and Pinterest by the hour.

But Amazon provides not just storage but applications that companies can run in the ‘cloud’ as well. It’s as if you had nothing but a keyboard, a screen, a mouse, and an internet connection but could run Windows and MS Office without noticing the difference. So ubiquitous has Amazon become, that it has been estimated that one of three internet users visits a site run off Amazon’s cloud service at least once every day.

“The cloud has enabled us to be more efficient, to try out new experiments at a very low cost, and enabled us to grow the site very dramatically while maintaining a very small team,” Pinterest operations engineer Ryan Park told a conference in New York last month.

“Imagine we were running our data centre, and we had to go through a process of capacity planning and ordering and racking hardware. It wouldn’t have been possible to scale fast enough,” he told the conference, according to Techworld.com.

What’s the key here though is not just the ability to rent hardware and software, rather than buy them, but the way Amazon prices that service, and what this enables companies like Pinterest to do. Customers pay only for what they use and by the hour. So, as Park pointed out, Pinterest pays about $52 an hour to Amazon during peak hours of the day, and about $15 during the night when traffic on the app is less (most of its customers are in the US).

There are other services that Pinterest uses that add up to some few hundred dollars per month, but even then, paying that much to service 18 million users isn’t too bad a deal. And the ability to scale up and scale down when needed enables Pinterest to try out new services easily and at low cost. On the downside of course, and just like an electricity grid, if Amazon’s cloud services were to be hit by technical snags, large chunks of the Net could go ‘dark’.

The other facet of this world of cheap, pay-for-gigabytes-you-need world is of course cloud storage for consumers. When Google launched its much anticipated 5-GB-free storage plan earlier this month :the G-Drive,it jumped onto an already crowded bandwagon occupied by the likes of Box, Dropbox, Mozy and others.