What is Big Data

Wikipedia: Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy often refers as predictive analytics.

Big Data Greatly Increase Capabilities and Operational Efficiency

The era of "Big Data" is clearly upon us as sales and marketing professionals. For those of you who've been checked out of Hotel Reality for a while, big data is a bit of a catch-all phrase and definitely one of the more hyped terms of the past couple of years.

Simply put, big data refers to a collection of data sets so large and complex that it becomes difficult to process using traditional data processing applications. Although estimates vary widely, research conducted by CSC estimates a 4,300 percent increase in annual data generation by 2020.

What is Big Data?


This ever-increasing quantity of data generates enormous challenges, but also generates significant business opportunities. Data processing is no longer the sole domain of relational databases. Unstructured data such as digital photos, videos and social media are growing at an even faster rate than structured data.

As a result, an entire new industry has formed around technologies aimed at storing, sorting, viewing and gleaning business insights from big data. Companies such as Hortonworks and Cloudera help businesses manage their data using a new open-source data framework called Hadoop. Companies like Tableau, Birst and Domo help their customers see and understand their data in profoundly new ways. Collectively, these companies have generated billions of dollars of financing from investors of all stripes interested in profiting from the explosion of big data.

OK, enough with the history and technology lessons. What does the advent of big data mean for the future of sales and marketing? Well, big data is the most transformative paradigm shift to hit sales and marketing teams since the advent of the telephone. That's because more than any other profession, salespeople have long relied on the art of the deal. Much like the baseball scouts in Michael Lewis' popular book "Moneyball: The Art Of Winning an Unfair Game", salespeople have historically relied on relationships and other soft factors to target and close business. Similarly, marketing professionals have made huge bets on brand advertising and air cover campaigns with little to no empirical data to support their spending.

As we enter the new year, it is time to start thinking differently. Big Data and predictive analytics technologies represent the opportunity to turn the table on the house. In other words, sales and marketing can finally become more about math than magic.

Five ways big data will rock our world in future time.

  1. Large enterprises will be the first to widely adopt big data and predictive analytics technologies, but small and medium businesses will get on board soon thereafter and will benefit even more.
  2. Marketing spend will become significantly more precise by leveraging insights from big data to accurately target prospects and deploy account-based marketing strategies.
  3. Salespeople will gradually adopt data-driven methodologies to target high-value prospects, keep existing customers on board, and expand existing opportunities.
  4. Sales forecasting accuracy will improved dramatically as sophisticated algorithms supplant "gut feel" as the weapon of choice for predicting sales.
  5. Real-time sales data visualization technologies will emerge, empowering sales managers to adjust battlefield tactics based on live data feeds.

Now before we conclude, would like to offer some color commentary on the list above. We believe large enterprises will be the first to adopt because their need to explore and gain insight from their enormous troves of data is profound. However, internal stove-pipes and varying data repositories will make the job difficult and time-consuming.

Small and midsize companies will adopt a bit later, but will benefit more quickly and with greater impact than their big brothers in large enterprises. Startups may benefit the most because they are being born in the era of big data and are building capabilities to leverage it into their business models. For example, take a look at how some companies are leveraging geo-spatial data and location intelligence to fuel their emerging empire.

Many salespeople won't jump on board quickly or easily. However, predictive sales intelligence will provide better results than intuition, and ultimately that will convert even the most reluctant salespeople. If better personal results won't do the trick, then improved forecasting and actionable real-time sales data will quickly convert sales leadership and in turn, their ground forces.

We guess in the end, if you are a sales or marketing professional in any industry we would suggest that you get smart about what's happening in big data and predictive analytics. A tsunami of data and potentially powerful business insights is heading your way. You must decide if you're going to ride the wave or search for higher ground.

What big data has brought to the privacy discussion?

Last fall, Facebook had an 18-minute outage, which cost them about $400,000 (or $22,000 per minute). Granted, this is just a drop in the bucket for Facebook, but if you include the revenue lost by all the businesses using Facebook Ads, that number gets a whole lot bigger — for just 18 minutes. There’s no question that advertising is big money, but behavioural online advertising is even bigger money — and companies like Facebook and Google get that.

But contrary to popular belief, when it comes to advertising and privacy, advertisers really don’t care about what we do or where we go. They only care about one thing: getting us to buy what they’re selling. And if that’s true, you can’t help but ask, “What’s the harm?” I mean, who doesn’t appreciate a relevant-to-you ad when you’re surfing for a certain item online or a coupon delivered to your mobile device when you’re near one of your favourite stores?

It seems like a benign trade-off: a little bit of my personal information in exchange for some helpful, free service that could save me some money. But here’s the rub: the information we freely share is not just used by these advertisers selling stuff to us. It’s also used by the Facebooks and Googles of the world, plus a whole lot of other data players for a myriad of reasons — none of which we really have any control over.

Privacy challenges in a big data world

The privacy (and related security) discussion is not new — to consumers, citizens, companies, or government agencies. What we’re seeing, though, is that this discussion is shifting from IT, development, and legal to the boardroom and our customers — not to mention our own dinner table.

Big data is a key contributor to this shift. We are all generating data at a phenomenal rate — a rate that currently exceeds our ability to properly capture, process, store, and analyze this data for any meaningful insight in a timely manner.

Make no mistake: we’re making significant progress with “big data” technologies, but we can’t depend on technology alone to address the challenges big data has brought to the privacy table. Let’s briefly consider a few of these challenges:

  • Right to privacy. Who owns our data and what are we or “they” entitled to do with it? What assumptions can we make about personal data we now share online?
  • The internet age. We live our lives in a public and digital square where any person, company, or agency around the world can watch us, whether we want them to or not.
  • Security. Between data breaches and aggressive hackers, will our data ever really be secure? As data continues to grow, so do the opportunities for data breaches.
  • Safety. Face it, we live in a dangerous world. How do we balance safety with privacy and security at the data level?
  • Trust. Trust is at the heart of the privacy issue and is the glue that is going to keep the data ecosystem together.
  • Ethics. Technology has leapfrogged ethics, bringing us to the age-old question of what we can do versus what we should do.
  • Context. What is contextually important to you may not be important to me. Let me give you an example: Google Maps. We might both believe it makes our lives easier, but when the street views of our homes show up, my kids show up in the picture and I tell all my Facebook friends – and you become outraged because your dog was in the shot.
  • No borders. Data, in and of itself, has no country, respects no law, and travels freely across borders. In the digital age, there are no geographical borders. And yet, most governments have attempted to put restrictions on how their citizens’ data is used.
  • Transparency. If important decisions are being made about us based on an algorithm and big data, we have a right to know how the algorithm works and what data is being used. It’s outrageous that many of the ways big data is being used is shrouded in secrecy.
  • Global differences. The internet is a big place, and treating privacy as a U.S. issue ignores the global reach of technology companies, and the long arm of government agencies. When we hear about foreign issues, we treat them like they're strange and far away, ignoring the fact that those issues can very quickly come home to roost.

As you can see, the big data privacy discussion is not just about behavioural advertising, as some would have you believe. Rather, it’s a much-needed, complex discussion about how we can balance privacy, security and safety in an increasingly transparent and dangerous world. Have that discussion before it’s too late.