Rhm2ktmi's List: Data Markets

• There is an emerging type of service provider that leverages the business intelligence (BI) platform-as-a-service (PaaS) model to give clients rapid access to their own transaction data.
• It does so by acting as a trusted broker in both its own operational business model and in the accumulation, management, and analysis of data.
• The cloud-based model has changed BI economics. Reduced expenses and freedom from acquisition, management, and maintenance lower the barriers to entry, so small- and medium-sized businesses (SMBs) can now benefit from analytics.

1 more annotation...

• What databases in the public cloud offer is hugely compelling: Elastic scalability, universal network accessibility, integration with mobile platforms, pay-per-use efficiencies, avoidance of capital costs, and the ability to work with widely scattered structured and unstructured data (that is, big data).
• As a result, there's been a massive migration to cloud-based databases, aka data-as-a-service -- one that seems to have escaped notice perhaps because it seems such an obvious step to take. The cloud is just too cheap, too fast, and too compelling. We'll see this massive migration continue this year as well.
• • Many startup database technology vendors are providing single-purpose databases. They do one or two things really well, such as transaction processing or analytics. Typically, these startups have a data-as-a-service offering, in addition to an on-premises version, that makes it easier and cheaper to try it out -- and to keep using it once proven.
• Most enterprises are moving to the use of complex database tools, whether the databases reside on site or in the cloud. The data-as-a-service option lets them both test these out more easily and avoid the upfront capital costs and deployment effort, a fundamental advantage of any cloud service.
• The key initial driver is the move to single-purpose databases.

3 more annotations...

• In domains like financial services, the buying and selling of data is commonplace, and providers such as Bloomberg and Thomson Reuters are well-established. Elsewhere, too, there is recognition that data supports good decision making when creating new products and services, both for businesses and the individual consumer
• A wide range of companies exists in this space, and often there are more differences than similarities in the various products on offer.
• Some of these companies, like AggData or Datafiniti, act as aggregators, pulling data from numerous topic areas across the web (e.g., entertainment, health, legal) and offering it for download from a single place and in a consistent manner.
• Others, like Factual, go further, cleaning data and aggregating facts from across multiple data sets.
• Still others offer visualization tools to better understand data, as DataMarket does with its datamarket.com portal.
• The financial models for these companies are different, too. Some charge the data provider, others the data consumer, and others are exploring ways of charging both.

4 more annotations...

• Marketplaces are useful in three major ways
• First, they provide a point of discoverability and comparison for data, along with indicators of quality and scope
• Second, they handle the cleaning and formatting of the data, so it is ready for use (often 80% of the work in any data integration).
• Finally, marketplaces provide an economic model for broad access to data that would otherwise prove difficult to either publish or consume.
• According to founder Flip Kromer, Infochimps was created to give data life in the same way that code hosting projects such as SourceForge or GitHub give life to code.
• The biggest challenge in the two years Infochimps has been operating is that of bootstrapping: a data market needs both supply and demand.
• The vision is very similar to that of Infochimps, but in late 2010 Factual elected to concentrate on one area of the market: geographical and place data.
• Launched in 2010, Microsoft’s Windows Azure Data Marketplace sits alongside the company’s Applications marketplace as part of the Azure cloud platform
• Where the other three main data marketplaces put a strong focus on the developer and IT customers, DataMarket caters to the end-user as well
• Other data suppliers
• • Note: Kasabi closed down in 2012.
• Kasabi, currently in beta, is a marketplace that is distinctive for hosting all its data as Linked Data, accessible via web standards such as SPARQL and RDF.

10 more annotations...

• The dynamic Ethiopian Commodity Exchange (www.ecx.com.et) is further spreading its information feed, and now customers can access general information and their accounts through SMS and voice telephone (“Interactive Voice Receiver” or IVR) systems, according to an article in Ethiopia’s Fortune newspaper. This is an information feed, not the automated trading systems being installed by stockbrokers on the Nairobi Stock Exchangeas highlighted on 29 Dec on this blog, as ECX trading is still done on a physical trading floor.
   One key aim of the ECX is to help Ethiopian agriculture become more efficient and productive by letting farmers all over the vast country get current information on what’s going on in commodity markets. The mobile phone systems should ensure that information is cheaply and quickly available to a wide range of farmers.

• Wikidata, a centralized structured data repository for facts and Wikimedia’s first big new project in the last 7 years, is now feeding the foundation’s main project, Wikipedia.
• The Wikidata project was kicked off around a year ago by the German chapter of Wikimedia, which is still steering its gradual development. For Wikipedia, the advantage is simple and powerful — if there’s a central, machine-readable source for facts, such as the population of a city, then any update to that data can be instantly reflected across all the articles in which the facts are included.

• Infochimps’ latest move suggests that the company is accelerating its gradual transition from one business model to another. Their new offerings are intriguing and useful, but it would be a shame to lose this early – and rare – example of a good data market.

   

• After Kasabi’s loss last year, and with a notable lack of new entrants to the space, the pressure really is mounting on DataMarket, Factual, and the few remaining players to show real and lasting value.

   

• The Austin-based Infochimps initially developed its data system behind the Infochimps Platform as the foundation for its big data marketplace. With the Infochimps Platform, customers will be able to leverage a proven big data system to construct big data projects for less money and optimize

• A platform as a service (PaaS) model will heavily reduce costs by facilitating the creation of big data constructs while circumventing the need for in-house hardware and maintenance. Enterprises with the resources (or those with the need) to build and manage their own big data infrastructures can also utilize the Infochimps Platform in on-premise deployments.

   

• Deploying and managing big data systems such as Hadoop clusters is not easy work, but Infochimps wants to change that with its new Infochimps Platform offering. The Austin, Texas-based startup best known for its data marketplace service is now offering a cloud-based big data service that takes the pain out of managing Hadoop and scale-out database environments
• But the real secret sauce is Ironfan, a configuration-and-management tool that Infochimps built atop Opscode’s Chef software
• The open-source Chef software is widely used in cloud computing and other distributed environments because it makes infrastructure configuration and management so much easier, but it’s limited to a single computer at a time, Infochimps CSO Dhruv Bansal told me
• It’s what lets the company spin up clusters for Infochimps Platform customers in minutes rather than days or hours, which is the only way the company could offer big data infrastructure on demand
• For users who’d rather work on their own internal gear, however, Infochimps is open sourcing Ironfan. Kelly said the company is currently working with early users on some in-house deployments, including atop the OpenStack cloud computing platform
• A few providers already offer some form of Hadoop as a cloud-based service, including IBM and Amazon Web Services, and Microsoft is working with Hortonworks on a Hadoop distribution that can run on the Windows Azure cloud platform

4 more annotations...