↳ Metrics

December 2nd, 2017

↳ Metrics




The gray box of XAI

A recent longform piece in the New York Times identifies the problem of explaining artificial intelligence. The stakes are high because of the European Union’s controversial and unclear “right-to-explanation” law, which will become active in May 2018.

“Instead of certainty and cause, A.I. works off probability and correlation. And yet A.I. must nonetheless conform to the society we’ve built — one in which decisions require explanations, whether in a court of law, in the way a business is run or in the advice our doctors give us. The disconnect between how we make decisions and how machines make them, and the fact that machines are making more and more decisions for us, has birthed a new push for transparency and a field of research called explainable A.I., or X.A.I. Its goal is to make machines able to account for the things they learn, in ways that we can understand. But that goal, of course, raises the fundamental question of whether the world a machine sees can be made to match our own.”

Full article by CLIFF KUANG here. This page provides a short overview of DARPA's XAI (Explainable Artificial Intelligence) program.

An interdisciplinary group addresses the problem:

"Contrary to popular wisdom of AI systems as indecipherable black boxes, we find that this level of explanation should often be technically feasible but may sometimes be practically onerous—there are certain aspects of explanation that may be simple for humans to provide but challenging for AI systems, and vice versa. As an interdisciplinary team of legal scholars, computer scientists, and cognitive scientists, we recommend that for the present, AI systems can and should be held to a similar standard of explanation as humans currently are; in the future we may wish to hold an AI to a different standard."

Full article by FINALE DOSHI-VELEZ et al. here. ht Margarita For the layperson, the most interesting part of the article may be its general overview of societal norms around explanation and explanation in the law.

Michael comments: Human cognitive systems have generated similar questions in vastly different contexts. The problem of chick-sexing (see Part 3) gave rise to a mini-literature within epistemology.

From Michael S. Moore’s book Law and Society: Rethinking the Relationship: “A full explanation in terms of reasons for action requires two premises: the major premise, specifying the agent’s desires (goals, objectives, moral beliefs, purposes, aims, wants, etc.), and the minor premise, specifying the agent’s factual beliefs about the situation he is in and his ability to achieve, through some particular action, the object of his desires.” Link. ht Margarita

  • A Medium post with an illustrated summary of some XAI techniques. Link.
⤷ Full Article

January 6th, 2018

Anonymous Power Game



Nominations from top economists, including selections by Raj Chetty, Sendhil Mullainathan, and Angus Deaton

One favorite from this excellent round-up is by Hulten and Nakamura on metrics, selected by Diane Coyle (we previously sent her Indigo Prize paper):

Accounting for Growth in the Age of the Internet: The Importance of Output-Saving Technical Change by Charles Hulten and Leonard Nakamura

Main finding: Living standards may be growing faster than GDP growth.
Nominating economist: Diane Coyle, University of Manchester
Specialization: Economic statistics and the digital economy
Why?: “This paper tries to formalize the intuition that there is a growing gap between the standard measure of GDP, capturing economic activity, and true economic welfare and to draw out some of the implications.”

Robert Allen's "Absolute Poverty: When Necessity Displaces Desire" is another metrics-related piece on the list.

Also noteworthy, on the future of work:

Valuing Alternative Work Arrangements by Alexandre Mas and Amanda Pallais

Main finding: The average worker does not value an Uber-like ability to set their own schedule.
Nominating economist: Emily Oster, Brown University
Specialization: Health economics and research methodology
Why? “This paper looks at a question increasingly important in the current labor market: How do people value flexible work arrangements? The authors have an incredibly neat approach, using actual worker hiring to generate causal estimates of how workers value various employment setups.”

Full piece by DAN KOPF here.

⤷ Full Article

April 21st, 2018




"Digital goods have created large gains in well-being that are missed by conventional measures of GDP and productivity"

A new paper by ERIK BRYNJOLFSSON et al. suggests using massive online choice experiments as a method to find the true impact of digital goods on well-being. The background section gives an example of the impact that is currently unmeasured:

“... [in] a number of sectors ... physical goods and services are being substituted with digital goods and services. An apropos example of such a transition good is an encyclopedia. Since the 2000s, people have increasingly flocked to Wikipedia to get information about a wide variety of topics updated in real time by volunteers. In 2012, Encyclopedia Britannica, which had been one of the most popular encyclopedias, ceased printing books after 244 years (Pepitone 2012). Wikipedia has over 60 times as many articles as Britannica had, and its accuracy has been found to be on par with Britannica (Giles 2005). Far more people use Wikipedia than ever used Britannica—demand and well-being have presumably increased substantially. But while the revenues from Britannica sales were counted in GDP statistics, Wikipedia has virtually no revenues and therefore doesn’t contribute anything to GDP other than a few minimal costs for running servers and related activities and some voluntary contributions to cover these costs…For such transition goods, consumer surplus increases as free access spurs demand, but revenue decreases as prices become zero. Hence GDP and consumer welfare actually move in opposite directions.”

One finding of note: “50% of the Facebook users in our sample would give up all access to Facebook for one month if we paid them about $50 or more.” Link to paper on NBER here. A free draft is available here.

⤷ Full Article

April 28th, 2018

The Inaccessible Rock



Canada calculates expanding Ontario's guaranteed income to the entire nation

Canada’s Parliamentary Budget Office looks at the cost of expanding the Ontario pilot nationwide. Full report here. ht Lauren

ANDREW COYNE of the NATIONAL POST summarizes the findings (all figures are in Canadian dollars):

“The results, speculative as they are, are intriguing. The PBO puts the cost of a nationwide rollout of the Ontario program, guaranteeing every adult of working age a minimum of 16,989 CAD annually (24,027 CAD for couples), less 50 per cent of earned income—there’d also be a supplement of up to 6,000 CAD for those with a disability—at 76.0 billion CAD.
“Even that number, eye-watering as it is (the entire federal budget, for reference, is 312 billion CAD), is a long way from the 500 billion CAD estimates bandied about in some quarters.
“But that’s just the gross figure. The PBO estimates the cost of current federal support programs for people on low-income (not counting children and the elderly, who already have their own guaranteed income programs) at $33 billion annually. Assuming a federal basic income replaced these leaves a net cost of 43 billion CAD. That’s still a lot—one seventh of current federal spending.”

Full article here.

  • A few features of the Ontario model differentiate it from a a prototypical (universal) basic income: 1) the Ontario pilot is not universal: only those “living on low income (under 34,000 per year if you’re single or under 48,000 per year if a couple”) are eligible, according to the Ontario government. 2) It functions as a negative income tax—50% of any earned income above a set threshold is subtracted from the benefit. 3) If implemented, this guaranteed basic income would "replace Ontario Works and the Ontario Disability Support Program." The PBO report uses similar parameters expanded to the federal level.
  • In an article in Fast Company from February on the pilot, Ontario premier Kathleen Wynne explains the thinking behind using basic income to replace other social assistance: “‘I’ve met lots of people on social assistance who give a lot to the community and I have often thought ‘why aren’t we paying you to do this?’ Wynne says. ‘I envision a world where we help people to do the work that they can do. By work, I mean involvement in human society. I hope that, as we go through this project, we will see how that will work better.’” Link.
  • A HuffPost article imagines other ways the guaranteed income might cost even less over time: "This raw cost estimate is a very simplified snapshot. It just models what the government would have to spend to deliver the basic income, if nothing else changed. But with a basic income, plenty would change. First, we could expect a steep drop in the poverty rate. And that, in turn, could mean big savings for governments, because poverty is a major expense—particularly when it comes to health care." Link.
⤷ Full Article

May 5th, 2018

Aesthetic Integration



Renewed interest in an old model 

Last week we linked to the widely publicized news that SENATOR KIRSTEN GILLIBRAND would be pushing legislation to reintroduce government-run commercial banking through the United States Postal Service.

Link to the announcement, and link to Gillibrand's Twitter thread on the plan.

In a 2014 article for the HARVARD LAW REVIEW, law professor and postal banking advocate MEHRSA BARADARAN describes the context that makes postal banking an appealing solution: 

“Credit unions, S&Ls, and Morris Banks were alternatives to mainstream banks, but they were all supported and subsidized by the federal government through targeted regulation and deposit insurance protection.

Banking forms homogenized in the 1970s and 1980s, leaving little room for variation in institutional or regulatory design. Eventually, each of these institutions drifted from their initial mission of serving the poor and began to look more like commercial banks, even competing with them for ever-shrinking profit margins.

The result now is essentially two forms of banks: regulated mainstream banks that seek maximum profit for their shareholders by serving the needs of the wealthy and middle class, and unregulated fringe banks that seek maximum profits for their shareholders by serving the banking and credit needs of the poor. What is missing from the American banking landscape for the first time in almost a century is a government-sponsored bank whose main purpose is to meet the needs of the poor."

⤷ Full Article

September 29th, 2018

Catastrophe Theory



Questioning the great transition into a "global middle class"

Economist STEVE KNAUSS, in a new paper published by the CANADIAN JOURNAL OF DEVELOPMENT STUDIES, examines the "myth" of the global middle class and the claim that the $2/day measurement tells us anything substantive about poverty and inequality around the world.

"On the defensive in recent years, advocates of globalization have taken to highlighting achievements in developing countries, where globalization has supposedly pulled the majority out of poverty and catapulted them into the swelling "global middle class" remaking our world. This article provides a critical look at this interpretation. Carefully reviewing the global income distribution data behind such claims, it presents original calculations that generate new stylized facts for the globalization era.

The global income distribution approach does potentially have much to offer in terms of revealing the complexity of these changes, but in order to do so, greater attention and resources should be devoted to deepening our knowledge of the socio-historical changes underpinning the new realities of class formation and how they relate to the observed changes in global incomes. Instead of, or in addition to, constructing groups according to income thresholds, or national/global based deciles, ventiles or percentiles, more research should start from the other end, identifying national and global groups based on similarities in class formation and then attempting to trace such trajectories through the global income distribution."

Link to the article, and link to an ungated manuscript version. Jason Hickel comments:

"The question is: does their new petty income from the informal sector compensate for their loss of rural land, livestock, etc? It is not clear that it does. Therefore, we cannot say that this is a straightforward narrative of 'progress'—at least not in all regions."

Link to Hickel's thread.

  • Development economist Morten Jerven with a 2010 paper diving into the metrics question in the context of poverty in Africa: "The article therefore concludes that it is futile to use GDP estimates to prove a link between income today and existence of pro-growth institutions in the past, and recommends a searching reconsideration of the almost exclusive use of GDP as a measure of relative development." Link.
⤷ Full Article

October 13th, 2018

Answer to the Question



What policy lessons can we derive from UBI experiments?

Political philosopher KARL WIDERQUIST of Georgetown has published a 92-page book examining historical and current basic income pilots, the difficulties of extrapolating from policy research to policy, and “the practical impossibility of testing UBI.”

In his introduction, Widerquist mentions that the challenges for translating research into policy stem not only from the science, but also from the audience’s moral preferences and judgments, which are particularly heightened in the basic income discourse:

“Except in the rare case where research definitively proves a policy fails to achieve its supporters’ goals, reasonable people can disagree whether the evidence indicates the policy works and should be introduced or whether that same evidence indicates the policy does not work and should be rejected. This problem greatly affects the UBI discussion because supporters and opponents tend to take very different moral positions. Many people, including many specialists, are less than fully aware of the extent to which their beliefs on policy issues are driven by empirical evidence about a policy’s effects or by controversial moral evaluation of those effects. For example, mainstream economic methodology incorporates a money-based version of utilitarianism. Non-money-based utilitarianism was the prevailing ethical framework when basic mainstream economic techniques were developed but it lost prominence decades ago.”

Widerquist also writes lucidly on considerations for how to communicate scientific caveats and takeaways. The full book is available here. ht Lauren who comments: "It’s incredibly difficult to test every aspect of many, many policies (including most that are currently at full national scale). Testing a given welfare policy arguably only has to get decision makers to a point where it can be determined that the policy substantially helps those who need it and doesn’t hurt anyone as a result."

  • Activist Stanislas Jourdan spoke at the European Parliament in September about a basic income for Europe. Video of the presentation is here; slides are here. On the financing question, Jourdan proposes VAT ("already the most harmonized tax at EU level, large and reliable tax base"), as well as a European Corporation Tax, carbon taxes, and "quantitative easing for the people."
 ubi metrics


Federal student aid pays for itself

⤷ Full Article

October 20th, 2018

Action Plan



Competing definitons of the term have vast policy implications

The formal definition of family is “blood, marriage, or adoption,” but that leaves out many possible arrangements, including families of unmarried people, foster children, co-ops, and, until 2015, gay partnerships. In the 1970s, family law became more open to “functional families” outside the formal definition, while zoning law kept to the strictly formal. Legal historian KATE REDBURN writes, “These contradictions leave critical family law doctrines unstable in thirty-two states.”

In a recent working paper, Redburn examines how these changes came to be, and looks more generally at how legal regimes exist within connected networks and influence each other despite traditional boundaries of scale (local, state, etc.) and subject (family law, zoning law):

“Viewed through a broader lens, this story might suggest lessons for law and social movements. While progressives oriented their campaigns at the state level, homeowners imbued local governance with conservative social politics in defense of their prejudices and property values. Neither movement, nor the judges adjudicating their case, nor the legislators revising state and local statutes, paid adequate attention to the interlocking nature of legal doctrines, rendering their movements less successful than they have previously appeared. Though we tend to think of legal fields as distinct regimes, ignoring the multifaceted ways that doctrines overlap, connect, and contradict each other can have perilous consequences. Their blind spot has has grown to encompass millions of Americans.”

Redburn’s case study provides ample evidence that micro-level legal conflicts can uphold and alter legal understandings:

“Motivated constituencies of voters and their elected representatives can produce legal change quite out of sync with social trends. Such was the case in the zoning definition of family in the late 1960s and early 1970s. Despite social change resulting in more functional families, protective homeowners and the conservative movement successfully shifted zoning law away from the functional family approach.”

⤷ Full Article