Understanding the New Skills Needed to Exploit Big Data Potential

By | Big Data, Big Data Strategies, Data Collection, Data Integration, Data Management, Innovation, Profit | No Comments

McKinsey & Company estimates that the US faces a huge shortage of people who have the skills to understand and make decisions based on the analysis and modeling of Big Data*.

Plus, in today’s environment, companies are using less than one-tenth of one percent of the data they could be using.  This is primarily due to a lack of understanding about how to approach the monumental volume of data being generated every day.

Structured databases are fading away as the standard means to house business data, and storage is now a commodity, so that untold amounts of data can be retained for trending and analysis.  Further, with the advent of more and different data formats, including video files, web logs, and social networking, new capabilities in staff and skills are required to exploit Big Data’s potential.

New thinking is now required of your staff so your organization can move beyond structured / linear data fields to a more conceptual approach to database marketing.  As more data is created, newer ways to capture, process, and cultivate it to work for your businesses will be required.  That thinking requires a holistic view of how to cultivate the richness to be derived from social media sites like Twitter, Facebook, and LinkedIn.

New thinking will also be demanded of your staff regarding modeling / analytics.  In the legacy approach to data modeling, numerous tests on subsets of variables were run iteratively until a model emerged.  The processing power and storage capacity available today has the potential to reduce the cost of building models by a factor of 10 and, at the same time, improve the predictive power of the model.  All of this can be accomplished through the application of Big Data.

In order to optimize the full value of Big Data, you’ll need to ensure your staff has the skills required to collect, analyze and apply the new insights to your organization’s specific strategy.  Big Data sources require new thinking in how to evaluate new insights for the benefit of your business’ success.  In addition, the availability of new data formats requires new methods for processing and implementing the information.  However, when approached successfully, the new data environment allows for improved modeling and reduced cost, greatly contributing to your organization’s success.

*Big Data: The next frontier for innovation, competition, and productivity.

McKinsey Global Institute May 2011

VDC Blog - Skills

Leveraging Big Data for B2B Applications

By | Big Data, Big Data Strategies, Competitive Analysis Tool, Data Asset, Profit, Search Web, Social Web, Uncategorized, Web Crawling | No Comments

Much has been said about the power of digital data as applied to business-to-consumer (B2C) opportunities.  But what about business-to-business (B2B) opportunities?  Are you missing an important opportunity to integrate digital data into your marketing programs?  What are the techniques and methods for accessing and harnessing digital data for bottom-line improvement in B2B organizations?

With a new set of data comes a new dimension in opportunity.  The classic approach to obtaining data is through traditional information service providers.  Most often, the data is compiled from such sources as Yellow Pages®.  The challenge with these sets of data has been timeliness and comprehensiveness.  Now, however, there are new sources available, offering greater and more up-to-date quality, data.  They come in the form of such sources and methods as social networks, web crawling and monitoring, Internet browsing behavior, ad networks / exchanges, and crowdsourcing, to name a few.

New opportunities may be realized through the use of these new digital data assets, such as identifying what companies are in-market for your products and services, as well as identifying key decision makers in the companies important to your marketing efforts, with the budgets to buy your goods.  By obtaining complete and timely digital data, you’ll be better able to augment your marketing programs.

There are a variety of ways to obtain these new sets of data, and best delivery methods can be customized for your specific business needs.

For example, if a company selling cloud computing services wants tofind prospects for its services, Big Data can facilitate their identification and targeting.  Big Data techniques can monitor and analyze enormous volumes of publically-available information sources and glean from them the best prospects for a variety of IT services.  The applications are numerous and can support the following:

  • Discovering new companies in emerging markets based on their network activity
  • Tracking the installation of hardware and software via network usage/installation data
  • Prioritizing key customer accounts based on previously unknown network activity
  • Developing and discovering corporate linkage hierarchies by mapping shared network assets (such as use of a specific e-mail server common to multiple locations)
  • Mapping digital data assets to physical assets using DUNS Number

Clearly, the value of digital data isn’t specific to B2C companies; it certainly extends into the B2B sphere.  The key is in understanding the best ways to collect and use the data resources at your disposal.

The Opportunity for CMOs to Exploit “Big Data”

By | Ad Spend, Big Data, Big Data Strategies, CMO, Competitive Analysis Tool, Crowd Sourcing, Innovation, Profit, Search Web, Social Web, Web Crawling | No Comments

Everywhere we turn these days, “Big Data” is top of mind.  For CMOs, the hype of “Big Data” will soon become a reality. When I think about “Big Data” from the CMO perspective, I divide the world by data source:   the Search Web, the Social Web, Web Crawling, and Crowd Sourcing.

Then, for each of the categorical sources of data we, at VDC, are building an inventory of use cases. For the CMO, for example, for the Search Web, the sources of data include both first-party (e.g., Expedia,, etc. ) and third-party sources (BlueKai, eXelate, etc.). When we view these companies as data sources, we can begin to create valuable, often-transformational applications using these new sources of “Big Data.”

For instance, by observing and understanding the quantity and geography of searches over time, we can improve the efficiency of a media buy.  Example:  if I am the CMO of a hotel property and I am a national advertiser using television, I can adjust the timing and targeted market(s) using these new sources of data. In this illustration:

  • If search volume for hotel bookings is currently low in Chicago but high in LA, I can adjust my television ad spend accordingly.
  • If search volume for hotel bookings are off-the-charts high, I can not only optimize my ad spend by market, but I can increase my overall ad spend in those markets that have exceptionally high traffic to capture more share. Another significant benefit: I can also increase my room rates.

Since high search volume is a leading indicator of the future high level of booking, I have an opportunity to set my rates and maximize profits.

In addition to using search data as a leading indicator of future bookings by market, the CMO can also use the data as a competitive analysis tool. This search traffic can further be broken down by brand:

  • How does the directed search volume for my property compare to that for my competitors?
  • How does it breakdown by market?

This data is not only useful for competitive analysis purposes but can be used to adjust the advertising message by market – and to emphasize my brand’s advantages over the competition.

Of course, this approach does not only apply to television, it applies to all targeted media:  direct mail, e-mail, display, mobile, etc.

Think about the power of these new data sources. What better leading indicator of future demand for a product could there be than to understand how consumers are planning to buy the product in the future as represented by their search activity today?  This data surely will transform our marketing processes.

This description represents just one use case of using search data to optimize ad spend and yield superior management strategies. There are many other use cases under development for the search web and there are an equal number of use cases under development for the other “Big Data” sourcing categories:  the Social Web, Web Crawling, and Crowd Sourcing.

While this example was for hotel properties, this same use case applies not only for many other aspects of travel-related services but also applies equally well for an almost endless array of consumer-driven industries:  auto, financial services, healthcare, etc.

This simple example illustrates the transformational power of “Big Data.” The capability to extract these insights from “Big Data” are here today and the practical application of these concepts to address the CMO challenges of today is just around the corner. It is truly a very exciting time for those of us who work in the “Big Data” business.

– Dr. Charles Stryker