brenda m. michelson 's Library tagged bigdata →  View Popular, Search in Google

“Big Data needs its unit of human computational threshold so it appeals to the billions that can benefit from it. Me? I’m waiting for Big Data to become Tiny Insights. Tangible bites of intelligence that help me make better decisions and improve outcomes. Make no mistake: Tiny Insights doesn’t mean tiny value. Tiny insights inform massive decisions for business or important decisions for individuals.” — Sameer Patel

bigdata

"We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment.

It wasn't just that the improvement was marginal, but that Netflix's business had shifted and the way customers used its product, and the kinds of recommendations the company had done, had shifted too. Suddenly, the prize winning solution just wasn't that useful -- in part because many people were streaming videos rather than renting DVDs -- and it turns out that the recommendation for streaming videos is different than for rental viewing a few days later."

bigdata kaggle contest

An excellent piece in the NYT on all things big data.

"GOOD with numbers? Fascinated by data? The sound you hear is opportunity knocking..."

bigdata nytimes

FINALLY! --> Streaming data processing: "Over the next few years we'll see the adoption of scalable frameworks and platforms for handling streaming, or near real-time, analysis and processing. In the same way that Hadoop has been borne out of large-scale web applications, these platforms will be driven by the needs of large-scale location-aware mobile, social and sensor use.

For some applications, there just isn't enough storage in the world to store every piece of data your business might receive: at some point you need to make a decision to throw things away. Having streaming computation abilities enables you to analyze data or make decisions about discarding it without having to go through the store-compute loop of map/reduce.

Emerging contenders in the real-time framework category include Storm, from Twitter, and S4, from Yahoo."

bigdata streaming realtime

"I.B.M.’s Strategic Intellectual Property Insight Platform. Clearly, the Watson branding team didn’t work on this name.

But then again, this isn’t for television, where Watson performed. It is for major corporate customers, seeking competitive advantage. The technology, sold as a cloud-based service, is the result of several years of joint development between IBM Research and four companies — AstraZeneca, Bristol-Myers Squibb, DuPont and Pfizer.

The insight platform uses data mining, natural-language processing and analytics to pore through millions of patent filings and biomedical journals to look for chemical compounds used in drug discovery."

ibm watson pharma bigdata

"What if the Netflix Prize model of solving hard problems about big data sets using contests could be applied to all sorts of other things? In fact, a remarkable start-up called Kaggle is doing exactly that — and already seems to be making it work.

Kaggle has facilitated breakthroughs in NASA’s analysis of dark matter, improved Allstate’s actuarial methods, predicted many of the top finishers of the Eurovision Song Contest, and is currently hosting a $3 million prize to device ways to reduce unnecessary hospitalizations."

bigdata kaggle

Good report. Used a sliver in my Active Information post.

"In June 2011 the Economist Intelligence Unit conducted a global survey of 586 senior executives, sponsored by SAS, to look at the state of big data, along with the organisational characteristics of companies that are adept at extracting value from data. It also explores the most challenging aspects of data management..."

economist bigdata

"...Cowen & Co. analyst Peter Goldmacher. In a 75-page report, Goldmacher walks through the database landscape and concludes that the consensus view that the growth of data will boost traditional database vendors is dead wrong. Goldmacher said:

We believe the vast majority of data growth is coming in the form of data sets that are not well suited for traditional relational database vendors like Oracle. Not only is the data too unstructured and/or too voluminous for a traditional RDBMS, the software and hardware costs required to crunch through these new data sets using traditional RDBMS technology are prohibitive..."

bigdata