dan mcquillan's List: social computing IS52026B - lecture 14

• These APIs were also one of the leaders in the Web 2.0 vanguard because they offered stunning visual proof that users can do great things with remixed data. Everyone started plotting their geographic data on their websites because the JavaScript code made it as easy as adding a few lines.
• The explosive growth and endless optimism came crashing to an end in October 2011, when Google started charging heavy users. Light users could still get the services for free, but everyone else was going to pay to support the big map in the sky. It wasn't as tragic as it might seem, but Google's decision was one of the first to signal the end of totally free era.
• While some groused, others saw opportunity. Before the price change, everyone was happy to let the big companies do the hard work because no one wanted to compete with a free service from Google. Why pay for what you could get for free?
• These new stacks use liberal amounts of open source software mixed with proprietary tools and services. You do what you want on your own and pay for what you can't do. Open source licenses [5] give you the control. While some will still grouse about not getting everything for free, there's no doubt the new marketplace offers many more options and opportunities than before.
• Working with these tools can be a bit more complex than working with a big provider like Google. Some of these companies make JavaScript tools [6] for displaying the maps, and others just deliver the raw tiles that the browsers use to assemble the maps. (All browser-based map tools break the maps up into pre-rendered squares or tiles that can be downloaded independently.) Working with the code means making decisions about how you want to assemble the pieces -- now within your control. You can stick with one simple library or combine someone else's library with tiles you produce yourself.
• The OpenStreetMap servers also distribute the raw tiles built from this data, but the project discourages any use that puts a strain on its equipment. The website points to a long list of companies [9] that turn the data into services that you can use.
• One of the most prominent OSGeo projects is OpenLayers [12], a good, general JavaScript library that can display any number of layers from all of the big map providers and many of the smaller ones. There are at least a dozen options for getting your tiles and the tools for interacting with them.
• TileMill
  If you're a Web programmer who loves maps, you'll find TileMill [15] to be the greatest combination since electricity met semiconductors. The tool combines open source data with a language for controlling how the data is rendered. When you're done, TileMill will spit out all of the tiles that your mapping application presents to the user.
• The core of the product is Carto, a language for describing how the fonts, lines, and colors are combined. It is, for most intents and purposes, a version of CSS. You design your maps with the same corner of your brain that you use to determine the look for your Web pages. If you want the major streets to be thicker, you bump up the number next to the CSS-like parameter "line-width."
• There are also a wide variety of open data sources to drive your fun. I started poking around and was soon putting labels on railroads all over the United States. The software pulled the numbers from a large collection of open data sets.
• TileMill offers a surprisingly large range of options to the map designer, and the tool makes it quite easy to produce a map with just a few clicks. The trouble is that it's quite hard to produce something beautiful. It might be cheaper to hire someone with real talent.
• Modest Maps [19] is a solid, basic API for putting up a map in a DIV. It's like Google Maps, but open source. You point the API at your DIV and choose a source of map tiles; Modest Maps pretty much does the rest.
• Modest Maps works well with the different tile providers, which often share the library themselves or use it as a building block. HTMAPL [20], a tool for writing your map code in HTML and jQuery templates instead of pure JavaScript and HTML, is one of these.
• All of these open source tools don't come out of nowhere. TileMill, for instance, was created by MapBox [30], a company that makes its money by selling downloads of the map tiles. You can fiddle with the data and make all of the pretty pictures for free, but once people look at your map tiles more than 3,000 times in a month, it's time to pay.

• The higher-priced plans offer more than just endless streams of map tiles. You get analytical charts that help you understand who is looking at which corner of the globe. MapBox has a cloud-based infrastructure that can optimize your maps to make sure they work as smoothly as possible in the mobile browsers.

  MapBox contributes a fair amount of open source code, including not only TileMill but also Wax, a tool that makes it a bit easier to use libraries like Modest Maps to embed MapBox tiles in your site. My favorite part of Wax is TileJSON [31], a basic format for wrapping up the data about the tiles.
• If you're just playing with maps, CloudMade is a good place to begin. CloudMade gives away some maps [33] it created from OpenStreetMap data. Like the data, these are protected by the Creative Commons-Share Alike license. If you want to make use of these tiles, CloudMade offers Leaflet [34], a full-featured library for putting maps in Web pages. It's small and optimized for adding extra layers and pins. Here's an example I made with Leaflet [35].
• What if you want to go deeper? If you're interested in connecting location with games, for instance, you'll want to visit the business end of CloudMade where they're pioneering this integration. One library, for instance, helps you figure out the weather and time of the person playing your game. The game can adjust itself using this data so that the action on the screen can take place in the dark if it's night or light if it's day. Or maybe you just want to serve up the opposite of the grim reality surrounding the player. If it's a cold and gray January day, you could offer tropical weather in your game.

• This collection of mapping tools and the companies behind them show the power of open source to unlock competition while encouraging cooperation. Everyone has an incentive to contribute data to the central pool kept by OpenStreetMap, yet everyone is also striving to outdo the next company with better code and prettier renderings.

• There's no doubt that Google is feeling some pressure from this ersatz collective. The company announced it was lowering its prices dramatically, knocking down fees almost 90 percent from $4 per 1,000 views to 50 cents per 1,000 views. This price will be billed only to sites that regularly exceed 25,000 views per day.

  While this is a dramatic change, I'm not certain it would be enough to turn me away from these open source alternatives. While there's no doubt that the major Web companies provide great tools, there's something wonderful about controlling your own destiny. The ability to customize my own map tiles is so dangerously seductive that I'm sure I'll lose too many hours fiddling with the colors and the fonts so the maps look just so. Sure it's work, but the power and control are hard to beat.

17 more annotations...

• Google is locked in a battle with the world's largest company, Apple, about who will control the future of mobile phones. Whereas Apple's strengths are in product design, supply chain management, and retail marketing, Google's most obvious realm of competitive advantage is in information. Geo data -- and the apps built to use it -- are where Google can win just by being Google.

• I was slated to meet with Gupta and the engineering ringleader on his team, former NASA engineer Michael Weiss-Malik, who'd spent his 20 percent time working on Google Mars, and Nick Volmar, an "operator" who actually massages map data. 

   

   "So you want to make a map," Weiss-Malik tells me as we sit down in front of a massive monitor. "There are a couple of steps. You acquire data through partners. You do a bunch of engineering on that data to get it into the right format and conflate it with other sources of data, and then you do a bunch of operations, which is what this tool is about, to hand massage the data. And out the other end pops something that is higher quality than the sum of its parts."
• On first inspection, this data looks great. The roads look like they are all there and you've got the freeways differentiated. This is a good map to the untrained eye. But let's look closer. There are issues where the digital data does not match the physical world. I've circled a few obvious ones below.
• And that's just from comparing the map to the satellite imagery. But there are also a variety of other tools at Google's disposal. One is bringing in data from other sources, say the US Geological Survey. But Google's Ground Truthers can also bring another exclusive asset to bear on the maps problem: the Street View cars' tracks and imagery.
• Let's step back a tiny bit to recall with wonderment the idea that a single company decided to drive cars with custom cameras over every road they could access. Google is up to five million miles driven now. Each drive generates two kinds of really useful data for mapping. One is the actual tracks the cars have taken; these are proof-positive that certain routes can be taken. The other are all the photos. And what's significant about the photographs in Street View is that Google can run algorithms that extract the traffic signs and can even paste them onto the deep map within their Atlas tool. So, for a particularly complicated intersection like this one in downtown San Francisco, that could look like this:

  • ...and wifi data!
• Google Street View wasn't built to create maps like this, but the geo team quickly realized that computer vision could get them incredible data for ground truthing their maps.
• Not to detour too much, but what you see above is just the beginning of how Google is going to use Street View imagery. Think of them as the early web crawlers (remember those?) going out in the world, looking for the words on pages. That's what Street View is doing. One of its first uses is finding street signs (and addresses) so that Google's maps can better understand the logic of human transportation systems. But as computer vision and OCR improve, any word that is visible from a road will become a part of Google's index of the physical world.

  • ... and facial recognition
• Later in the day, Google Maps VP Brian McClendon put it like this: "We can actually organize the world's physical written information if we can OCR it and place it," McClendon said. "We use that to create our maps right now by extracting street names and addresses, but there is a lot more there."
• "We already have what we call 'view codes' for 6 million businesses and 20 million addresses, where we know exactly what we're looking at," McClendon continued. "We're able to use logo matching and find out where are the Kentucky Fried Chicken signs ... We're able to identify and make a semantic understanding of all the pixels we've acquired. That's fundamental to what we do."
• For now, though, computer vision transforming Street View images directly into geo-understanding remains in the future. The best way to figure out if you can make a left turn at a particular intersection is still to have a person look at a sign -- whether that's a human driving or a human looking at an image generated by a Street View car.
• There is an analogy to be made to one of Google's other impressive projects: Google Translate. What looks like machine intelligence is actually only a recombination of human intelligence. Translate relies on massive bodies of text that have been translated into different languages by humans; it then is able to extract words and phrases that match up. The algorithms are not actually that complex, but they work because of the massive amounts of data (i.e. human intelligence) that go into the task on the front end.
• Google Maps has executed a similar operation. Humans are coding every bit of the logic of the road onto a representation of the world so that computers can simply duplicate (infinitely, instantly) the judgments that a person already made.
• He probably uses twenty-five keyboard shortcuts switching between types of data on the map and he shows the kind of twitchy speed that I associate with long-time designers working with Adobe products or professional Starcraft players. Volmar has clearly spent thousands of hours working with this data. Weiss-Malik told me that it takes hundreds of operators to map a country. (Rumor has it many of these people work in the Bangalore office, out of which Gupta was promoted.) 

• The sheer amount of human effort that goes into Google's maps is just mind-boggling. Every road that you see slightly askew in the top image has been hand-massaged by a human. The most telling moment for me came when we looked at couple of the several thousand user reports of problems with Google Maps that come in every day. The Geo team tries to address the majority of fixable problems within minutes. One complaint reported that Google did not show a new roundabout that had been built in a rural part of the country. The satellite imagery did not show the change, but a Street View car had recently driven down the street and its tracks showed the new road perfectly. 

  Volmar began to fix the map, quickly drawing the new road and connecting it to the existing infrastructure. In his haste (and perhaps with the added pressure of three people watching his every move), he did not draw a perfect circle of points. Weiss-Malik and I detoured into another conversation for a couple of minutes. By the time I looked back at the screen, Volmar had redrawn the circle with perfect precision and upgraded a few other things while he was at it. The actions were impressively automatic. This is an operation that promotes perfectionism.
• Now imagine doing this for every tile on Google's map in the United States and 30 other countries over the last four years. Every roundabout perfectly circular, every intersection with the correct logic. Every new development. Every one-way street. This is a task of a nearly unimaginable scale. This is not something you can put together with a few dozen smart engineers. 

  
  

  I came away convinced that the geographic data Google has assembled is not likely to be matched by any other company.
• The secret to this success isn't, as you might expect, Google's facility with data, but rather its willingness to commit humans to combining and cleaning data about the physical world.
• Geographic information systems represent a jump from paper maps like the abacus to the computer. "I honestly think we're seeing a more profound change, for map-making, than the switch from manuscript to print in the Renaissance," University of London cartographic historian Jerry Brotton told the Sydney Morning Herald. "That was huge. But this is bigger."

• Their MapMaker product makes that ambition clear. Project managed by Gupta during his time in India, it's the "bottom up" version of Ground Truth. It's a publicly accessible way to edit Google Maps by adding landmarks and data about your piece of the world. It's a way of sucking data out of human brains and onto the Internet. And it's a lot like Google's open competitor, Open Street Map, which has proven that it, too, can harness the crowd's intelligence.

16 more annotations...

• 
   We’re introducing a method that lets you opt out of having your wireless access point included in the Google Location Server. To opt out, visit your access point’s settings and change the wireless network name (or SSID) so that it ends with “_nomap”.

• There's also a map of just street signs; one of power lines; one of the stars overhead; one of autumn leaves; another of the underground gas, water and sewer systems. Each tells a different story about the neighborhood by isolating one of its life support systems, each answering the question "Where am I?" in an entirely different way. "I wanted the atlas to read almost like a novel," Wood says. Still unfinished, the maps are the subject of a 1998 interview that remains one of the radio show This American Life's more popular episodes.
• Wood minces no words about what's wrong with the picture provided by both our standard street atlas and its virtual counterparts: "Google Maps reduces the world to a bunch of automobile pipes. I don't want to think of my world only in terms of streets." Wood has spent much of his career as artist, writer and educator arguing that maps influence how we think about every aspect of our environment, from natural resources to economic development, from school assignment zones to voting precincts. "Show me anyone's life today, and it's shown on a map, organized by maps and constructed with maps."
• The importance of maps is certainly nothing new—even older than writing, maps have been instrumental in everything from laying out simple farming settlements to colonizing the New World. The most obvious recent change in mapping is in the technology. Over the past 30 years or so, software based on the geographic information system (GIS) model has become the standard in professional cartography, changing the kinds and quantity of data that can be made available by a single map interface. At the same time, the digital revolution has made it easier for more people to get involved in mapping through free programs like Google Earth. But most of what's distinctive about today's maps comes down to the same key factors as any form of communication: what they're being used for, and for whom they're being made.
• In Chapel Hill, an innovative group of mapmakers is exploring ways to make visual narratives out of the dynamic forces that shape a modern research university. The Counter Cartographies Collective (3Cs), a group operating out of UNC-Chapel Hill's Cultural Studies department, refuses to see mapping as a passive process. Says spokesperson Tim Stallman, "We're less interested in making maps that describe the world as it is and more interested in making maps that develop your understanding of the world."
• Since its inception in the spring of 2005, 3Cs has been using maps to show how the Triangle's major biotech and medical research centers and its universities are taking control of North Carolina's economy, and the effects of these changes on local communities.
• 3Cs' most notable mapping project so far is entitled disOrientation 2006, which they passed out to incoming first-year students at UNC. Really several maps in one, it displays traffic patterns, the most dangerous intersections for pedestrians, where to find the densest populations of Triangle researchers, local business information, where UNC faculty live compared with where workers live, where students are studying abroad, and more.
• The map's unifying concern is to figure out what globalization and the so-called "knowledge economy" look like from the bottom up rather than the top down. As 3Cs member Craig Dalton puts it, "Knowledge requires labor to be produced."
• "If you crunch out the statistics," Stallman says, "the top 1 percent of landowners in Durham County own 50 percent of the land ... and if you include Duke and the City of Durham, it jumps to 75-90 percent. The maps we're working on now are asking, 'Where are these properties, where are they concentrated, where most turnover in home sales is happening, where most new construction is happening,' things like that." According to Wood, an admirer of their work, 3Cs' maps accomplish "what so many [other] maps fail to do—to make a point."
• ince 2004, Duke sociologist Gary Gereffi has been working with his students on studying North Carolina and globalization, but with a very different scope and perspective from 3Cs. The starting point of his North Carolina in the Global Economy project is to recognize that the success of some N.C. industries (biotech) and the decline of others (manufacturing) can no longer be understood strictly on the state or national level. It's not just N.C. vs. other states, but N.C. vs. ... China.
• Since they're designed to follow the "life" of a product across local boundaries, Gereffi's theories are hard to represent geographically. In a sense, the "real" maps are provided by the flash-animated "value chains" hovering above the images of North Carolina. By showing where "value" is added in any given production process, regardless of location, they represent the ideal form of the global production process to which the state must adapt. It's a vision of the world designed to be sold to professional clients, who today range as far afield as Korean corporate boardrooms and as near as the N.C. Department of Commerce.
• hese days all major cities in North Carolina have a GIS division. GIS can integrate zoning, hydrography, parcel boundaries, police beats and almost everything else into a single mapping tool. GIS data is shared among individual agencies and standardized across the state, allowing for, among other things, assisting the Department of Social Services place foster children and shorter 911 and emergency response times.
• there really are contradictions between spatial maps, narrative storytelling and democracy, Gary Kueber of the popular Endangered Durham blog may be the most likely Triangle dweller to break them down. He started the blog in August '06 "really because there wasn't a go-to resource." The focus has been on preservation advocacy for Durham's underprotected historic buildings. Mapping Durham's architectural and social history is a closely related interest

• Duke's special collections and a few of Kueber's readers, to tell the story of the forgotten Lakewood Amusement Park and its transformation into Lakewood Shopping Center. Ghostly images of the park's many atractions (including a merry-go-round, roller coaster and pool) and the trolley system it was intended to popularize are given historical context by a written account, and located in Durham's contemporary strip-mall landscape with maps.

     

  Kueber wants to move on from the Google Maps application programming interface he's currently using to mark the sites from his blog posts to a Web 2.0-based approach that would allow for freer exchange and collaboration. "I'd like for there to be a temporal version of something like Google Earth," he says.

• "One of the things that I learned from working with kids in schools, if I only showed them census data, that wasn't so interesting for them," Abel says. "But if I showed them a map, it became a really exciting way of reaching the students."

     

  Blending all of these primary sources gives the sort of multifaceted view of history usually reserved for professional historians. Whether eighth graders or undergraduates, Abel's way of teaching history is to expose her students to as many primary sources as possible. It's even better if that experience can spread outside the school to the amateur researcher and concerned citizen.
• Abel hopes Google Earth will be able to provide a new dimension to understanding Durham. "I'd like to think of it as time travel," she says. Although on their way to having a historical Google Earth "layer" ready by fall, Abel and her assistants have been frustrated by the very thing they're trying to study: the haphazard way Durham's urban environment has changed over time.

• While we're waiting for various technical problems to be overcome, it's not clear that fancy technology of any kind is needed for making maps. Wood, who has seen mapping trends come and go, admits that user-friendly programs like Google Earth have changed things: "For people who have never thought about making maps, it's tons easier. There are people making maps today who never would've made maps 10 years ago."

     

  But he is less enthusiastic about the way mapping technology is being relied on in general. "I haven't seen any change in quality." He scoffs at new digital capabilities like representing complexity, or a new ability to display large quantities of data. "The goal of many of these GIS dreamers is to do precisely that, to encapsulate all that data in an icon, in a meaningful form. And that is so, so hard to do.

     
• "Most people think [maps] represent the world," Wood says. "My argument is maps propose the world, then bring it into being through enforcement policies." Although the technology is making it easier for non-specialists to interpret their environments, making a good map, Wood insists, "isn't a question of tools. It's a question of imagining how things could be visualized." There has never been a better time to experiment.

15 more annotations...