We all create data whether we know it or not

Big Data Blog

Online shopping, social media, internet searches – it all enters a huge pool of data that companies use to eventually monetize. If you’re thinking about a career in Big Data or even studying Computer Science and the technologies that compliment Big Data – please read on.

There are numerous sub fields under Big Data but today we’ll look at some of the areas we as consumers or us as engineers obsess about these days.


Companies that own this data (even PII) have the grave responsibility of creating privacy rules, settings, and what to do with the data over time. In addition protecting this data is a very important, yet ever changing challenge that we have seen explode in the news/media over the last few years. Securing data and ensuring it is used in the manner construed through a privacy policy will continue to be critical over time. Those that can bridge the gap between engineering and policy creation and security expertise can expect a pretty lengthy career in big data.


Are you ever angry that it takes your PC, tablet, or phone to load a site, complete a transaction, or install? Companies care deeply about latency and as the demand for more data on these devices- by more and more users increases, companies will invest in datacenters, engineers that support/service datacenters, distributed systems that process all of the data, and developers that play in between to cut down time between your device and the datacenter to ensure you always get the content you need in the fastest time possible. That was a mouthful but spot on in terms of the investment companies/engineers make to give you a fast answer when you search for something on bing.com.


We all turn to the web at some point when we need something (advice, shoes, a service). As we spend time on a search engine, shopping site, or new site you will notice fast that the ads and content being served up to you are eerily tailored to what you were searching for or perhaps searching for a few days ago. The next big obsession of big data companies will focus on your “next’ transaction. Suggestions, recommendations, “you might like”, or customers like you…all incorporate machine learning into big data sets to provide you with things that consumers like you should/will buy. Beyond shopping or transactions, task completion is the true name of this game and a large investment is being made in this space by most companies whether its mobile, ads, or shopping – a gold mine in terms of opportunities and a hot time to be innovative and to crack the code here.

Analysis – Exabyte, zettabyte, yottabyte??

If you have ever seen an actual needle in a haystack – that is cool. However the ‘big data’ haystack is vastly larger, deeper, and more complicated than you or I can imagine. If you will imagine click through logs of people who have clicked on ads in an online advertising network, the number of purchases/returns made on a popular online shopping site, or even page views of a popular gossip site there is someone there to crunch the data down to meaningful chunks of information that help advertisers/companies understand their audience/consumer better essentially through a series of research, flights, trials change the experience for you as the consumer – a better more personalized experience. These folks excel in statistical analysis, software development, and experimentation – again all of which will continue to be in great need as the world moves into zettabytes of data and eventually yottabytes.

If you’re passionate about one or more of these areas or just want to play with a huge set of training data, we encourage to apply today. You will gain more than a job – a career, great peers, and benefits come with your adventure too.


Winsome posted on 26 Sep 2014

Hello, I am a data analyst and I am interested in big data. I would love to learn more and welcome the opportunity to experiment with large sets of data. Where can I get started? Thanks.

Codess posted on 29 Sep 2014

If you are interested in learning more about Big Data, perhaps enter the Kaggle competitions and also check out KDNuggets.com. They are both a wealth of information that can help you learn more about the Data Science world and how you can gain more experience in that area. If in the future, you are looking for Data Analyst jobs at Microsoft-then have a look at our careers site and see if there are any roles that you think align with your skills/experience. Also feel free to send your CV to mlrecruiting@microsoft.com and let them know you were recommended by Codess.

Lavonne posted on 16 Mar 2015

I thought I'd have to read a book for a diersvocy like this!

Leave a comment