Home
Weiye Loh's Library tagged → View Popular, Search in Google
a selection of tools that we use the most and that we enjoy working with. We called it selection.datavisualization.ch. It includes libraries for plotting data on maps, frameworks for creating charts, graphs and diagrams and tools to simplify the handling of data. Even if you’re not into programming, you’ll find applications that can be used without writing one single line of code. We will keep this list as a living repository and add / remove things as technology develops.
How much authority should we give to such work in our policy decisions? The question is important because media reports often seem to assume that any result presented as “scientific” has a claim to our serious attention. But this is hardly a reasonable view
-
A rational assessment of a scientific result must first take account of the broader context of the particular science involved. Where does the result lie on the continuum from preliminary studies, designed to suggest further directions of research, to maximally supported conclusions of the science? In physics, for example, there is the difference between early calculations positing the Higgs boson and what we hope will soon be the final experimental proof that it actually exists. Scientists working in a discipline generally have a good sense of where a given piece of works stands in their discipline.
-
often, as I have pointed out for the case of biomedical research, popular reports often do not make clear the limited value of a journalistically exciting result. Good headlines can make for bad reporting.
- 7 more annotation(s)...
On May 8, The Asia Foundation launched a new data visualization tool to help policy-makers and citizens explore the findings of the new 2012 Malaysia Business Environment Index (BEI), the only diagnostic tool designed to measure the business-friendliness of local governments in the country. The BEI collects a significant amount of data through a survey of 635 small- and medium-sized firms in 11 city and municipal districts across Malaysia. The BEI ranks these 11 districts against each other based on a 100-point scale, and scores the relative performance of each district along nine sub-indices designed to capture key elements of the local business environment. The new visualization tool allows policy-makers and the public to quickly compare how different districts rank, and explore the research data in greater detail. In addition, the full BEI report is available for download, along with the data compiled from the surveys and subsequently used to design the visualization tool. Data visualization is a tool The Asia Foundation is using increasingly to help synthesize large data sets and make them more accessible for broader analysis
in order to build the disease networks of tomorrow, we will need to move beyond the current linear approaches to science and to how scientists work. We all like a good story that unfolds in a straightforward way, but the story of disease plays out across a poly-nodal information network, similar to what an air traffic controller might track in the skies above a major airport. Biomedical researchers’ “lock and key” and linear-pathway representations are incomplete, and should be supplemented with disease maps that can now be built using molecular data.
-
Cultural barriers are the real stumbling block. As humans, we are highly evolved to adjust to our surroundings: we tend to adapt to a culture, well-conceived or not, and lose sight of its failings.
-
We must also build the infrastructure and cultivate the relationships needed to share disease maps with basic researchers, practicing doctors, drug developers, and even the public at large. And that could prove to be even more difficult, because the current closed nature of the medical-information system and its self-directed incentive structure block such sharing. Patents, trademarks, and competition for resources (people, money, and accolades) seal off information and prevent molecular data from being analyzed and shared. Rewards in biomedical research go to “solo workers,” and do nothing to acknowledge the work that can be done only by multi-functional groups.
It is difficult to communicate medical risk to a large audience, especially when official recommendations conflict with emotional narratives. That is why, when the United States Preventive Services Task Force (USPSTF) in 2009 presented its guidelines for breast cancer screening, which recommended against routine screenings for asymptomatic women in their 40’s and biennial, rather than annual, mammograms for women over 50, the public responded with confused fury.
Illustration by Paul Lachine
CommentsThe key to understanding this response is to be found in the nebulous zone between mathematics and psychology. People’s discomfort with the findings stemmed largely from faulty intuition: if earlier and more frequent screening increases the likelihood of detecting a possibly fatal cancer, then more screening is always desirable. If more screening can detect breast cancer in asymptomatic women in their 40’s, wouldn’t it also detect cancer in women in their 30’s? And, if so, why not, reductio ad absurdum, begin monthly mammograms at age 15?
-
The USPSTF recently issued an even sharper warning about the prostate-specific antigen test for prostate cancer, after concluding that the test’s harms outweigh its benefits. Chest X-rays for lung cancer and Pap tests for cervical cancer have received similar, albeit less definitive, criticism.
CommentsView/Create comment on this paragraphThe next step in the reevaluation of cancer screening was taken last year, when researchers at the Dartmouth Institute for Health Policy announced that the costs of screening for breast cancer were often minimized, and that the benefits were much exaggerated. Indeed, even a mammogram (almost 40 million are given annually in the US) that detects a cancer does not necessarily save a life.
CommentsView/Create comment on this paragraphThe Dartmouth researchers found that, of the estimated 138,000 breast cancers detected annually in the US, the test did not help 120,000-134,000 of the afflicted women. The cancers either were growing so slowly that they did not pose a problem, or they would have been treated successfully if discovered clinically later (or they were so aggressive that little could be done).
I’m a Christian and ultimately come to Christ through faith. With climate change no faith is required. There is a large and growing body of evidence. The way nature works applies the same to Republican and Democrat, Christian and Muslim, animal, tree and stone. Why do people who profess to love and follow God roll their eyes? Luke 16:2 says “Man has been appointed as a steward for the management of God’s property, and ultimately he will give account for his stewardship.”
It’s a message that my father put succinctly: Actions have consequences.
-
During a 2007 homecoming banquet for Iraqi war vets I asked my personal hero, Senator John McCain, if he thought this could all be some cosmic coincidence. He rolled his eyes. “Paul, I just returned from the Yukon, where a village elder presented me with a tomahawk that had just melted out of the permafrost. The answer is no.”
How did so much of the Republican Party enter perpetual denial? We’ve turned climate science into a bizarre litmus test for conservatism. To pretend that heat-trapping gases can be waved away with a nod and a smirk is political fairytale. No harm. No foul. Keep drilling.
The meteorologist and energy entrepreneur Paul Douglas is keeping up his valuable effort to depoliticize the science pointing to a growing human influence on the climate. Last month, I noted a post in which he described the scientific case posed by the unabated emissions of greenhouse gases. He described himself as a “Republican deeply concerned about the environmental sacrifices some are asking us to make to keep our economy powered-up, long-term.”
Now he’s going further, taking his argument to the commerce-oriented Bloomberg Businessweek Web site in a piece titled “Climate Change Has Nothing to Do With Al Gore” (the first of a two-part post, Douglas says)
-
How did so much of the Republican Party enter perpetual denial? We’ve turned climate science into a bizarre litmus test for conservatism. To pretend that heat-trapping gases can be waved away with a nod and a smirk is political fairytale. No harm. No foul. Keep drilling.
-
I’m a Christian and ultimately come to Christ through faith. With climate change no faith is required. There is a large and growing body of evidence. The way nature works applies the same to Republican and Democrat, Christian and Muslim, animal, tree and stone. Why do people who profess to love and follow God roll their eyes? Luke 16:2 says “Man has been appointed as a steward for the management of God’s property, and ultimately he will give account for his stewardship.”
You’ll see various interpretations. Those concerned about global warming (including at least one study author) are stressing that a longer evolutionary timeline implies the bears’ adaptation to climate change in the past was a slow process (meaning the speed of change now poses new threats). Those questioning the vulnerability of this species to warming will point to its successful survival through two previous warm intervals between ice ages as evidence the bear can deal with reduced ice and other big environmental shifts. Finally, there are basic questions about the robustness of the conclusions, which are based on a new line of genetic analysis not previously applied to polar bears. [April 20, 7:22 a.m. | Insert | I think this work bolsters the view of scientists who've been calling for a conservation strategy for polar bears and other ice-dependent species focused on areas of the Arctic where sea ice is projected to endure well into this greenhouse-heated era. Watch this presentation.]
-
James Gorman of the science staff at The Times captures this complexity well in his news story:
The report comes to no conclusion about how sensitive the bears are to the current loss of the sea ice that they live on, and the evolutionary tale it presents can be read in different ways.
The findings challenge the idea that the bears adapted very quickly, but confirm that they have made it through warming periods and loss of sea ice before. It may have been touch and go for the bears, however, because the authors find evidence of evolutionary bottlenecks, probably during warm periods, when only small populations survived, even though warming was occurring much more slowly than it is now. [Read the rest.]
One of the most familiar memes we hear from the climate-change deniers is the phrase, “Global warming ended in 1998 and it’s been cooling since then.” You find something along these lines on most of the AGW denier books and websites, and it is repeated endlessly as if somehow repetition makes it more true. This is just like creationists who continually repeat the phony argument that “evolution violates the Second Law of Thermodynamics”, even though this is patently false. As has been pointed out many times, the Second Law only applies to closed systems. The earth is not a closed system since it receives energy from the sun. Yet in every creationist book and website and debate for many decades now you’ll hear them repeat it over and over again, since it sounds impressive to their scientifically unsophisticated audience and apparently they cannot understand why it’s wrong, or they don’t care as long as it suits their political agenda.
-
Picking 1998 as a starting point is a classic example of cherry-picking data to show what you want it to show, and a deliberate attempt to distort the actual record. As climatologists have known for years, 1998 was an exceptionally warm year due to a record El Niño, which boosted average global temperature way above the overall trend from the past few decades. During El Niño years, the marine circulation patterns release a lot of tropical heat from the oceans and raise overall global temperature for a short time. Likewise, 2008 was a La Niña year, and it was cooler than normal. These are part of the year-to-year “noise” in the system of global temperatures that is well known to scientists. Scientists nevertake a single year’s temperature and then connect it to another data point and claim it’s a “trend.” Instead, the only rigorous and scientifically defensible method is to look at the long-term trends in climate over decades and “smooth” the curve using rolling averages, so that a more statistically meaningful curve fit can be performed.
-
Cherry-picking can be played both ways. If I pick any year prior to 1998-2000 and connect with any data point from 2001 onwards, I get a warming trend. In fact, the only way a AGW denier could get their “no warming since 1998″ misrepresentation is to deliberately and consciously look at the curve, pick 1998 to start, and only compare it to 1999-2001. Any other long-term combination of the data shows warming. Thus, this distortion of the data that Will keeps repeating is not just a simple misreading of the facts. Since the meme is quoted from 2009, this means that the deniers were consciously and fraudulently trying to distort the data to suit their purposes. The fact that this lie keeps being perpetuated despite the fact that scientists have offered numerous corrections shows the AGW deniers have the same casual disregard for the truth that creationists do. Such practices demonstrate the absymal level of their scientific integrity, and speaks to the fact that AGW deniers are not climate scientists, but people with political agendas who cherry-pick data, quote-mine out of context, and use whatever lies and half-truths they need to support their cause. The parallel with creationists and other science deniers could not be any clearer.
It is clear to everybody that if you solve a crossword puzzle, then you must also have the ability to do so. But the reverse conclusion is rather inane. If you do not solve it, it does not follow that you cannot solve it at all, in the sense that you completely and utterly lack the natural prerequisites for doing so. It just means that you did not solve it yet. So if you learn the principles of crossword puzzles – and maybe some foreign words and the names of a couple of ancient Norse and Greek gods – you will probably be able to solve them the next time.
-
it is no wonder that extended school attendance leads to a higher “IQ score” and that the lack of schooling leads to a score at the level of the mentally retarded.
-
This is a fact that the IQ theorists do not really like: They are not only very well aware of the fact that you can easily improve your faculties for solving IQ tests and thus gain a higher score, it is also common knowledge to them that the average population in industrial countries with regular schooling, as already mentioned, has become no less than 30 percent better at filling out IQ tests in the course of the 20th century.
- 3 more annotation(s)...
The reason why even the most bewildered tourist can find their way around the tube network easily is that the map does away with geographical accuracy in favour of clarity. The map retains the general shape of the tube network, the way the lines connect, but it distorts the actual distances between stations and pretends that trains only run in straight lines, horizontally, vertically or inclined at 45 degree angles. That isn't how they run in reality, but it makes the map a lot easier to read. It's a topological map named after an area of maths, topology, which tries to understand objects in terms of their overall shape rather than their precise geometry. It's also known as rubber sheet geometry because you're allowed to stretch and squeeze shapes, as long as you don't tear them.
-
Maps are great because our brains are good at making sense of pictures. So representing data in a visual form is a good way of understanding it. The question is how.
-
in reality things are more complicated. You'll probably have thousands of books and customers. Each book now comes, not with a pair of numbers, but with a huge long list containing the rating of each customer or perhaps a blank if a specific customer hasn't rated the book. Now you can't simply plot the data and spot the pattern.
This is where topology comes to the rescue: it gives a neat way of turning shapes into networks. Suppose you've got a wobbly circle as in the figure below. You can cover it by overlapping regions and then draw a dot on a piece of paper for each region. You then connect dots corresponding to overlapping regions by an edge. The network doesn't retain the wobbliness of the shape, that information has been lost, but its topology, the fact that it's circular, is clearly visible. And the great thing is that it doesn't matter what kind of covering you use to make your network. As long as the regions are small enough — the resolution is high enough — the network will draw out the topology of the shape.
-
The intention is good: it is to promote crowd-sourcing of maps, to improve planning in disasters and to improve the planning, management and monitoring of public services. This is an important goal, which is now being made possible by new technologies and the spread of the internet. The deal is sufficiently important for World Bank Managing Director Caroline Anstey to write about it in the opinion pages of the New York Times:
Under the agreement, the bank and its development partners — developing country governments and U.N. agencies — will be able to access Google Map Maker’s global mapping platform, allowing the collection, viewing, search and free access to data of geoinformation in over 150 countries and 60 languages.
This is all consistent with an admirable push in the World Bank towards ‘democratising development‘, including becoming more open about its own activities and promoting open data. Indeed, this effort has come to be a defining achievement of Robert Zoellick period as World Bank President.
-
Where [the World Bank] once imposed prescriptions on the Third World, it now shares knowledge with respected clients from the new world. Where it once hoarded data, it now displays it on the web. … One decade ago, the Bank was routinely accused of indifference to the views of local people. Today Mr Zoellick talks of empowering the most humble netizen to provide feedback on projects.
- 5 more annotation(s)...
"18 of the 20 outlets were left of center. The only two that were not were the Washington Times and Fox News’ Special Report with Brit Hume.
Our findings, however, contradicted a few claims of conservatives. For instance, they showed that some mainstream news outlets are nearly perfectly centrist, albeit still left-leaning. Two were ABC’s Good Morning America and [PBS's] The Newshour with Jim Lehrer. Also, we found that many supposedly far-left news outlets were not that far left. For instance, we found that National Public Radio was no more liberal than the Washington Post, Time, or Newsweek. And we found that it was less liberal than the average speech by Senator Joe Lieberman...
Our study was denounced by hundreds, and maybe thousands, of left-wing blogs, including Media Matters, the Daily Kos, and the Huffington Post. At one point if you googled “crap UCLA study,” most of the first ten listings would refer to our study...
-
The chairs of the departments of sociology, religion, and German and Russian languages were especially angry, and they called it “offensive” and “scandalous.” One said “The study isn’t research. It’s agitprop for the conservative blogosphere.”
-
… In that lay part of my objection, and here I have to say that it’s not to your work qua research at all. Rather, its presentation on the website made a pretty categorical claim about bias that taps into a charged political environment. There are difficult issues that underpin the website headline, and your study is complex and sophisticated enough to treat many of them; far more subtle and nuanced than the journalistic reductio. There are of course issues outstanding or open to discussion (what’s included by way of news sources, whether conceptual categories like liberal and conservative have veridical legitimacy as identity markers, where and how one designates boundaries of same [i.e., you can call something X and cite as reason a widely accepted standard, but that in no way means that the thing really is X, or so a philosopher would say], how one categorizes constellations of dispositions, how one treats what Bakhtin called dialogism in discourse analysis, and so forth. …
- 1 more annotation(s)...
An unvarnished translation would be -- "This graph means virtually nothing."
In addition to inflation, patterns of population growth, nature of economic development, accumulation of wealth all play a role in how extreme events distant in time would lead to economic impacts had they occurred with the same underlying societal conditions. (To understand why inflation is important but less so than other adjustments, in this paper in PDF compare the inflation-adjusted hurricane losses in Figure 3 and normalized hurricane losses in Figure 4.) As I wrote in my initial post on this topic, for 1980 there is certainly 4 (and maybe 5 or more) other events that occurred in 1980 but would exceed a billion dollar threshold had they occurred in 2011. So by adding one event to 1980, NOAA has recognized the general problem, but has not come close to actually dealing with it.
-
NOAA has quietly added 19 new disasters to their tally since 1980, apparently having discovered a quantity called inflation. The modification of their tally is (it seems) in response to a blog critique which was followed up by a Washington Post blogger. The new "billion dollar disaster" figure is at the top of this post, with the 19 new additions from NOAA in red.
As we have seen before with the IPCC, its review of the literature somehow missed key articles that one of its authors (in this case Trenberth, the lead for the relevant chapter) found to be in conflict with his personal opinions, or in this case "shameful." Of course, there is a deeper backstory here involving a conflict between my co-author Chris Landsea and Trenberth in early 2005, prompting Landsea to resign from the IPCC.
So almost seven years after we first submitted our paper how does it hold up? Pretty well I think, on all counts. I would not change any of the conclusions above, nor would I change the reply to Anthes et al. Science changes and moves ahead, so any review will eventually become outdated, but ours was an accurate reflection of the state of science as of 2005.
-
This series of exchanges was not acknowledged by the IPCC even though it was all peer-reviewed and appeared in the leading journal of the American Meteorological Society. As we have seen before with the IPCC, its review of the literature somehow missed key articles that one of its authors (in this case Trenberth, the lead for the relevant chapter) found to be in conflict with his personal opinions, or in this case "shameful." Of course, there is a deeper backstory here involving a conflict between my co-author Chris Landsea and Trenberth in early 2005, prompting Landsea to resign from the IPCC.
-
energy is fundamental for humanity, not only because of its potentially negative externalities, but also given its economic relevance
-
Owing mainly to its environmentally negative externalities, an unregulated energy market is not a useful governing mechanism, because it is unable to internalize the environmental costs. It has been calculated that the most contaminating energy sources would have to pay a 70% tax to reflect their negative externalities.
- 6 more annotation(s)...
After generating considerable attention with a preview on Capitol Hill last spring, an independent team of scientists has formally released their analysis of the land surface temperature record. Led by Richard Muller, a physicist at the University of California, Berkeley, the Berkeley Earth Surface Temperature study takes a different and more comprehensive approach than earlier assessments, but reaches the same basic conclusion: global warming is happening. Nature examines how the new study differs from its predecessors.
-
Muller says he listened to the sceptics and decided that an independent analysis was in order. He and his team decided to tackle the temperature record independently, on the basis of first principles. They say their results line up with previously published studies and suggest that the average global land temperature has risen by roughly 0.9 °C since the 1950s.
Muller says he is surprised at how well the findings line up with previous analyses, which he takes as evidence that the various scientific teams working on these data did indeed go about their work "in a truly unbiased manner".
-
The Berkeley researchers developed their own statistical methods so that they could use data from virtually all of the temperature stations on land — some 39,000 in all — whereas the other research groups relied on subsets of data from several thousand sites to build their records. This meant that they also had to figure out ways to handle shorter temperature records from instruments or stations where the record was interrupted.
Muller and his team also used a different approach to analysing the data. Scientists working on the earlier studies adjusted raw data to account for differences in the time of day when readings were made, for example, or for higher temperatures caused by the urban heat island effect, in which cities tend to be warmer than natural landscapes. Muller says his team included the raw data in its analysis and then applied standard statistical techniques to remove outliers.
- 3 more annotation(s)...
On October 20th Nature News reported on a new analysis of land temperatures by an independent group. They found the same results as previous analyses – since 1950 the earth has warmed by about 0.9 C. The results have yet to be peer-reviewed, but already reports of their analysis are making some waves.
-
The analysis was designed to be what can be called a consensus study – an independent group is taking a thorough analysis of the data, accounting for prior criticisms, to arrive at a result that everyone can agree on. Prior to announcing the results, in fact, some global warming skeptics stated publicly that they welcome the independent analysis and would stand by the results. PZ Myers reports on Anthony Watts response – initially saying he would accept the study results, but now considering the study to be fatally flawed.
-
The point of a consensus study is to bring all sides of a scientific controversy together, account for all criticisms of existing data, and then try to specifically address those criticisms so that everyone can agree on the results.
- 6 more annotation(s)...
-
From the outset, these [climate] scientists also brought their preferred solutions to the table in US Congressional hearings and other policy forums, all bundled. The proposition that ‘science’ somehow dictated particular policy responses, encouraged – indeed instructed – those who found those particular strategies unattractive to argue about the science.36 So, a distinctive characteristic of the climate change debate has been of scientists claiming with the authority of their position that their results dictated particular policies; of policy makers claiming that their preferred choices were dictated by science, and both acting as if ‘science’ and ‘policy’ were simply and rigidly linked as if it were a matter of escaping from the path of an oncoming tornado.
-
Andrew Dessler, currently a minor celebrity in the blog battles between climate scientists and their skeptical opponents, explains that those who reject his views of the science are politically motivated:
Really? Does "climate change" call for a "government solution"? Or is it more complicated than that? And if the "science of climate change is a proxy for views on the role of government" (which I agree with), does this apply only to opponents to action?"People who discount the science of climate change don't do it because they've read the science," he says. "The science of climate change is a proxy for views on the role of government. From what I understand, Perry's position is that he doesn't want government to interfere in private lives or industry. That means climate change — which calls for a government solution; there's no way for the free market to address climate change by itself — that doesn't fit anywhere with his political values. So he shoots the messenger." - 6 more annotation(s)...
Google uses enough energy to continuously power 200,000 homes.
Google's many data centers around the world burn through 260 million watts—one quarter of the output of a nuclear power plant—the New York Times reports. The company had been cagey about revealing energy usage stats in the past, probably because it didn't want to reveal to competitors how quickly its data centers were growing. It's no longer a secret that Google needs a crazy amount of data centers to keep things running smoothly.
-
Google uses enough energy to continuously power 200,000 homes
Google's many data centers around the world burn through 260 million watts—one quarter of the output of a nuclear power plant—the New York Times reports. The company had been cagey about revealing energy usage stats in the past, probably because it didn't want to reveal to competitors how quickly its data centers were growing. It's no longer a secret that Google needs a crazy amount of data centers to keep things running smoothly.
-
Google accounts for roughly 0.013 percent of the world's energy use
Data centers in general are responsible for 1.3 percent of the world's electricity consumption, according to one estimate, and Google says it accounts for a mere one-hundredth of that statistic
- 3 more annotation(s)...
Selected Tags
Related Tags
Top Contributors
Groups interested in Data
Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »
Join Diigo