Measuring the Return on Investment (ROI) of College

http://www.payscale.com/college-education-value-2013

I think the study's methodology is weak and does not prove causation. For the "return" part of ROI, they look at the median pay of grads for 30 years of working, adjusted to 2013 dollars. They project out pay increases by looking at grads from 1983 to 2012 (i.e. the earlier grads are later in their careers, and forecast what recent grads will earn at the same age). But of course that is fraught with problems, because universities could change a lot in quality and student composition over 30 years. And the economy changes a lot over time, so an older person may see very different wage changes from 1990-2010 vs. a recent grad from 2010-2030 (especially for degrees that saw a lot of industry growth or contraction). Also, I am not sure if they were looking at total earnings or just base pay, because we know that some jobs may have lower base pay but make up for it with better benefits, equity, or pensions. Students who went on to graduate/professional education were excluded, but that could be a problem too since some universities have a high % of students with those career aspirations (and those jobs tend to pay higher).

Also, are we measuring the ROI of the school, or the ROI of the students? University is not the only factor determining a person's future salary. Some Stanford students would still make similar money if they attended higher or lower ranked schools because of their abilities and ambition. But that is what the study should try to answer - if Joe Smith went to Cal vs. Stanford vs. Caltech and studied the same subject and pursued the same career path, how would his pay differ? This would be expensive and maybe unfeasible, but to control for student quality variation, they should have looked at each decile of US high school students (by general aggregated admission criteria), mapped out which college they attended, and then looked at their earnings over the first 5 years after graduation (30 years is overkill, entails too much uncertainty, and recent grad wages are a good predictor of late career wages anyway). But still, variation in regional pay and career choices would still confound the results.

This info would be tough to obtain as well, but possibly a cleaner way to answer the question would be to query employers. Do their compensation departments place a premium on various schools for certain job roles? Same with grad schools - do they give bonus points to students who went to undergrad at certain places (and convert that to a $ value)? Also repaying student loans has an opportunity cost, and there is value in the networking and relationships that one obtains in college. So it would be nice if they could incorporate those factors into their model as well. What do you think?

-----------

In this case, though, it's a little hard to separate out the reputation effects vs. the quality of education vs. network effects.

I.e. do Stanford kids earn more money because we graduate with a good education or because of the Stanford name premium or because we just happened to make friends at a school where there is a lot of smart people.

If it's because of the quality of education, then you could say Stanford is worth it. If it's because of the network effects, then maybe yes maybe no. An expensive networking program for sure, but maybe not replicable by any other experience (and thus worth it). If it's the reputation effect, then Stanford should be concerned, as companies could potentially test for real skills some day, undercutting the reputational aspect of our degree.

------------

I agree that those factors are heavily linked in such a data set and it's pretty hard to draw any meaningful conclusions about ROI.

Another interesting question: are Stanford students successful because they went to Stanford, or is Stanford successful because of its students? I am sure it is symbiotic, but maybe that is why the top US schools have been locked in an "arms race" to recruit star profs, improve the student experience, etc. to woo the top talent. I guess they realize that a key to their success is the general quality of their "customer base" (but the spending is also partly irrational pissing contest). Though I am pretty sure that most students accepted at Stanford would become professionally successful no matter what university they attend (a testament to the merits of the admission process), even dropping out and going Zuck style. But if Stanford and say Ole Miss (sorry for offending any alum) do a "trading places" student body swap, I think that plenty of Ole Miss students would not be able to capitalize on the Stanford experience as effectively, leading to lower outcomes. And if the avg. quality of one's peers decreases, that may have negative effects on one's performance. But on the flipside, the sudden infusion of Stanford-caliber students to Ole Miss would have profound benefits for that institution on many levels.

So I guess college students should have much more bargaining power vis-a-vis university administrators. They could form a union haha and demand lower tuition or face a walk-out. But that threat is not credible, unless all of them can join startups the next year, or the Ivy League is able to take in the thousands of striking Stanford students en masse. Plus Stanford would replace them with eager "scab" students from US public schools and Asia - probably lower quality students, but "passable" to keep the place running. Therefore, it seems that the top schools are engaged in a certain degree of price fixing and cartelization (i.e. why is the tuition at Stanford pretty similar to that of Carnegie Mellon or Columbia, despite massive geographic and programmatic cost differences?). Unlike Wall St. and Si. Valley, they refuse get in a self-destructive price war, or steal away top talent from each other with cash/other incentives. Maybe that is illegal, but if absolute student body quality is the goal (as it pertains to national rankings or other metrics that administrator bonuses are based on), then it's worth it (and their endowments can justify it). But really, just imagine how that would affect recruiting if Stanford decided to slash its tuition by 1/3? 50% of undergrads are on some sort of financial aid anyway, so it wouldn't be such a shock to their finances. Well, no one ever suggested that higher education was an efficient market. :)

Phil Robertson of Duck Dynasty's comments

Apropos of this discussion, http://talkingpointsmemo.com/livewire/duck-dynasty-star-rants-against-homosexual-behavior-islamists-and-shintos-in-magazine-interview

It's just a little sad, in a way. They're some of the last people on the wrong side of history and are the last to realize it. I'd like to think that many similarly minded people are basically good and if they'd been raised in a different time they'd hold more considerate beliefs about others.

--------

http://abcnews.go.com/blogs/politics/2013/12/gop-politicians-defend-duck-dynasty-stars-right-to-anti-gay-comments/

The Robertson stuff is unfortunately not that shocking these days, though the fact that such a guy gets a TV show and GQ exclusive interview is. Some Republicans' "defense of his first amendment rights" really pisses me off. I understand that Duckman was within his rights to say those things, but the blatant politics and hypocrisy is unacceptable. Palin, Jindal, Cruz - as if they have any credibility anymore, but folks like them weren't defending Rev. Wright's first amendment rights at the time. In fact they were convicting Obama as un-American just for sitting in the same room with Wright. Where is their outrage about homophobia, racism, and almost comical ignorance - traits that should be un-American but are unfortunately celebrated/condoned at times for being "traditional values"?

Sure, I get it that any time Christianity seems under attack, they circle the wagons. But another important American value is not ramming your beliefs down everyone else's throats (despite FNC and the Tea Party's revisionist history to the contrary). I'm sorry that the mere existence of gays, Muslims, and ethnic minorities in "your country" makes you upset. Part of a free society is tolerating diversity and even tolerating people you can't stand. Like how thinking people have to tolerate the likes of Robertson, Palin, and Cruz. Otherwise they might as well stop hating Iran and move there, because that is the type of regime that they are promoting. Let's see if these GOP figures similarly come to the defense of Joe Biden the next time he mouths off, or god forbid an urban black celebrity. They are good Christians too, aren't they? They're just as much Americans as Robertson, right?

---------

In typical Palin-esque fashion, the momma grizzly admitted that she didn't even know what Robertson actually said (didn't read his statements from GQ) before opining on his free speech rights and the tyranny of political correctness/liberal intolerance. If only Robertson said something about Mein Kampf... would Palin have stood by her defense of his rights? 

http://news.yahoo.com/blogs/sideshow/sarah-palin-admits-not-having-read--duck-dynasty--interview-180236668.html

Of all companies, Cracker Barrel pulled its Duck Dynasty themed merchandise from its restaurants after the Robertson comments, but a day later reinstated them after outcry from customers. Only in Amurrrica. :P

On a related note, here is an interview from a Harvard neuroscientist about why the human brain may be hard-wired for narrow-minded moral judgment and us-vs-them tribalism.

http://www.kqed.org/a/forum/R201312240900

Silicon Valley as a preview of America's future, for better or worse

http://www.sfgate.com/technology/businessinsider/article/FEDERAL-JUDGE-NSA-Program-Exposed-By-Snowden-5069183.php

First of all, is Snowden the runaway favorite for person of the year 2013? I can't think of anyone else who affected US society more (and likely for the better). A Federal judge ruled that the arbitrary, blanket collection of Americans' phone metadata is likely unconstitutional (unreasonable search, violation of reasonable expectations of privacy) and ineffective (no clear evidence it was critical to preventing attacks). Of course the WH and NSA disagree.

----

http://www.npr.org/blogs/alltechconsidered/2013/12/16/251608120/exploring-economic-inequality-from-heart-of-the-tech-boom

NPR is starting a series on Si Valley as a microcosm for the changing US economy and society. We've addressed some of these issues before regarding wealth inequality and the Bay Area housing market. Apparently some protesters recently blocked a Google bus in SF, complaining that the private buses were using MUNI stops without paying the city any fees. It's clearly part of a bigger sentiment of resentment, and part of the general debate regarding who is taking more of their fair share from society. Should those being priced out of SF "get with the program" and develop the skills needed to be able to afford living there? Or should the tech elites come down from their privileged position in society and contribute their fair share to helping the less fortunate? Or both?

As a tech employee (peon status) and longtime CA resident, I am really torn by all of this. I don't want the Bay Area to become "Facebookville" or "Elysium" where droves of poor, bitter service workers support the aloof upper class minority. But then again, we depend on disruption, competition, and innovation to win in the market, or we will suffer the same fate as Detroit. I think we can innovate and win, but also spread the benefits around so that the entire region and populace is better off. Yes, the rich should be taxed more.

Disruption is a double-edged sword. A dedicated team works like hell to address an unmet need or overcome some entrenched inefficiency, so they and their investors can be paid handsomely for it. But the industry that they just rendered obsolete is made up of people with families that depend on them. Through no fault of their own, they were disrupted out of a paycheck. A generation ago, all signs pointed to machinist or travel agent being good, stable careers with great ROI for the required education. But innovation changed the equation faster than most could adapt.

Now car services like Uber are getting rave reviews from customers, media, and generating decent financials. But what if they render thousands of low-skilled immigrant cab drivers out of a job? What if the Google self-driving car renders the Uber drivers out of a job? In some cases it is a zero-sum game, even if the metrics suggest that society overall is "better off". Does one new billionaire created compensate for 1,000 families who just got the American Dream taken from them? Everyone seems to celebrate the "winners" without considering the collateral damage. Yes, we need innovation to succeed and even survive in the world. And those who succeed should be rewarded. But what about the everyday folks who, through no fault of their own, get suddenly shifted from middle class to destitution? Do the disruptors (and the traditional companies being replaced) have a social responsibility to help those workers transition to the new reality? Does the government? If so, then I think they should fund such programs from the disruptors' stock gains.

-----

Sort of related (in that it perpetuates the massive inequalities of the Bay Area) is education. M Night Shyamalan just wrote a major book on education (hopefully he doesn't write more and his natural follow-up suck strikes!), and it makes a lot of good points.

http://www.motherjones.com/kevin-drum/2013/12/m-night-shyamalan-steps-education-wars

Basically, he suggests that our education system is actually great - if you're a a white kid. Our white kids do just as well as the Nordic white kids or other European white kids. But if you're a minority (and, typically, poor), the education system sucks and really drags down our averages. There are a number of ways to address this, but:

1) They take money $$

2) And they mean de-emphasizing putting resources into already highly performing white schools

I've come increasingly to the conclusion that American politics is all about race. I'm sure many of you have seen the research that support for redistribution increases if it is perceived by the voter as going to "someone like them" - i.e. someone of the same race. 

Most people in the Bay Area don't consider themselves racist and it's probably true - in fact, the Bay Area is one of the most diverse areas of the country. But poors in the Bay Area are still "the other" - they didn't go to good schools, they struggle to get by on bad salaries, their unemployment rate is x3 higher than well-educated whites. Basically, their experience is completely alien to most of the tech gods in the Bay Area and their presence is invisible. No surprise then that the tech gods (which have actual power) use little of their capacity to help the poors.

You can imagine the national consternation and outrage by the well-educated press and electorate if the unemployment rate was 20% and transportation and housing ate up 60-70% of the budget of native-born, well educated Americans, yet this is exactly the situation facing a plurality of Bay Area residents - the poors and the immigrants. I guess it's only a crisis for those that are rich. If you're poor and face such circumstances, given the media and elite orientation, it's a statistic and at best regrettable. And the rich wonder why resentment is on the increase...

-----

Thx, J. I agree, but why are non-whites disproportionately poor? Mostly racism. The 1% vs. 99% has also been a pervasive, ubiquitous problem for centuries, but of course the power balance has varied due to historical conditions. As you said, now more than ever gov't is serving the interests of the 1% at the expense of the 99% - and that is a big effing problem that we have to tackle if we care about democracy. But even if we returned to the economic conditions of 1960 when wealth and political power were more evenly distributed, there is the underlying racial problem that Andrew mentioned. Say we repeal Citizens United, reformed the justice system, and made a bunch of other needed changes - it will still be harder for minorities to get ahead. Addressing one problem is not a distraction from the other. Both are critical, somewhat independent, and deserve to be called out separately. There are some solutions that can mitigate both issues (namely taxation and campaign/election reforms), so we could try to prioritize those.

According to this moron, the poor and middle class are doing better than ever, because we are enjoying a "golden age of TV!" So what if I lose my job and get evicted... I can still watch the latest episode of "Duck Dynasty" at the local library's PC lab. Forget the macro indicators, our amazing selection of digital entertainment options makes life more fun!

http://www.bloomberg.com/news/2013-12-02/who-needs-a-raise-when-you-have-tv-.html

------
 

Oh the drama of "bus-gate": http://blogs.kqed.org/newsfix/2013/12/09/google-bus-protest-most-entertaining-moment-was-staged/

Thanks, A. Good to see M. Night making a comeback of sorts, with more social value than "The Happening". Yeah it seems the path to inequality starts from grade school, and maybe even in the womb, as lower income (often minorities) generally don't have access to comparable health, nutrition, and child care as affluent whites. This could set back their development, so that even if they were lucky enough to attend America's best schools, maybe they wouldn't be fully able to take advantage of the opportunity.

It is really sad that the inequality phenomenon has such a racial component. But if a liberal dares to make that claim, you can imagine the reaction from the right (race-baiting, it's a war on white men, etc.). Roberts said that racism is over in America after all. It is especially disappointing that minorities have fallen further behind during the Obama presidency. It's not his fault and it could have been worse under a white GOP president, but Obama's heritage almost blinds America to the fact that tens of millions of minorities are not succeeding despite his meteoric, outlier success. After all, the biggest beneficiaries of US affirmative action historically have been middle class white women. The social programs that disproportionately helped the urban poor are being dismantled in the name of "budget compromise."

On a local level, I don't think the tech giants can fully celebrate their CSM and social impact, when East Palo Alto (10 miles away from FB and GOOG HQs) is still held back by the same problems it faced a decade ago. Richmond and the Tenderloin are a stone's throw away from Twitter and Salesforce. Sure the tech companies engage in their annual "day of caring" and donate millions to worthy causes, but those good works barely tip the scales vs. all the negative externalities they unintentially impose on the less privileged. I guess rather than facing the cognitive dissonance of the injustice/inequality all around them vs. their desire to make the world a better place (ostensibly for all), tech people would rather live insular lives and stick with their own kind (luxury, exclusive housing to the luxury, exclusive bus to the luxury, exclusive office to the luxury, exclusive bar, etc.). This of course is not exclusive to tech workers, and the other Bay Area professionals (doctors, lawyers, bankers) are just as susceptible, if not more. But that is the problem when achievement and career take up so much of our attention - there isn't much room left for people, especially those who are different than us.

I am not sure if it is a generational issue either. After the Baby Boomers exit the labor force, and the younger folks take the helm of the big companies, I really hope for a sea change. Douchebags aside, I think that many Bay Area yuppies would prefer a more diverse workforce and society. CA is great because it's not a stuck up East Coast cocktail party where your alma mater and family name come first. It's boring when you only work with clones of yourself who think the same way (also it's bad for business). The Valley has been successful in recruiting and naturalizing so much foreign talent, but many of them were elites from their native lands (and they tend to self-segregate in the US too). I just don't see black and Latino native residents get recruited in sufficient numbers any time soon.

Check out this slideshow from Business Insider about racial-economic segregation in SF-Si Valley: http://www.businessinsider.com/maps-show-racial-divide-in-silicon-valley-2013-9#

This issue kind of reminds me of the military crisis. When only poor people go off to war, society doesn't really notice, so our leaders may be more likely to abuse those soldiers. But in the case of WWII and Vietnam, when even some millionaire's kids and Ivy Leaguers were getting slaughtered in the field, then that changed the equation. As you said about Bay Area housing and transport costs, the privileged won't lift a finger until they feel similar pain as the poor. A more progressive income tax that funnels money to the working poor would be a start, or maybe higher property tax on richer, less diverse neighborhoods? But of course city leaders get elected by those neighborhoods, so it's a nonstarter. In France, they have a saying, "Vive l'impot!" Or a celebration of tax. It's not a dirty word like in the US. Taxes are a mark of honor, a patriotic duty to reinvest in the nation that you love. And those taxes should go to helping strengthen society from the bottom up, not further the social advantages of the elites or become giveaways to already rich special interests.

------

Part of the solutions that M. Night and other education scholars have recommended is longer school days, year round schooling, and early intervention programs like head start, universal Kindergarten, etc. Basically spending a lot of money to get poor/racial minorities with bad home environments and poor nutritional habits into a much healthier system. Of course, doing this takes a lot of money. However, school inequality is still a major issue so there are some resources that could be recaptured from rich white schools.

As you may remember from the George Packer/New Yorker article, even in the Bay Area, public schools such as Woodside and Paly have foundations that parents started to "adequately" fund their school - i.e. they have way more money than they need and marginal dollars there are not doing much good, while schools like East Paly are hurting pretty bad. And of course thanks to restrictive zoning regulations meaning no new housing ever (into which the poors/middle class might move), public schools in Woodside/Los Gatos/Atherton/Paly are only public in name only: effectively they are privatized for the rich white residents. But beyond that, it would take a massive infusion of resources into places like the LAUSD, New York Public Schools, etc. that voters haven't shown the will to spend.

I know you're fairly hopeful about the promise of the younger generation, but I'm more pessimistic. It's true that race will be less of an issue for our generation and some of the issues that the oldsters have with things like gay marriage will dissipate. However, I feel like our generation, while still well-intentioned, is even more clueless about the life and struggles of poor people than any previous generation. While J is right that the gap between the 1%ers and the rest has grown astronomically, I feel like the life experiences of the top 10% is becoming increasingly disconnected from the lives of the rest. Worldly, plugged in, well educated, financially comfortable kids on the Google buses have basically nothing at all in common with day laborers in the Mission. I don't mean that they actively seek to screw them but I think it's easy for many to not even people like that exist. The problems and life experiences of the bottom 90% have basically just don't show up on the radar of your average techie or higher.

And while the top 1% have arrogated for themselves increased power, the top 10% still can drive the political conversation in the media and online. If people don't see the problems of the rest of society or only are dimly aware of them in an abstract way, I'm not optimistic that much progress can be made.

I see where J is coming from on this - I think on matters of economic policy the 1% have the resources to really get whatever they want.

But I think one of the under-appreciated developments in American politics is that, in previous historical periods, when you had this unbalanced of an income, populists would rally the poorer classes to demand change. That's how they broke up the robber barons (see Teddy Roosevelt), and that's how the back was broken of European nobility.

What's happened this time is that many of the people that might have been expected to join this populist coalition against the rich and powerful are instead diverted by issues of "other" (i.e. race). How else do you explain the fact that West Virginia, one of the poorest states in the nation, voted overwhelmingly for Romney? Or Kentucky? They're afraid that moochers (of course, moochers from out groups; in group moochers are fine) and immigrants and gays are wrecking America.  These are precisely the people that Democrats ought to find common cause with (and previously did, against the Robber Barons and again during the Depression), yet our politics that are focused on race and religion and culture have them voting against their economic interests.

------

Thanks, A. It's ludicrous that there is any opposition to these youth programs, as the research indisputably shows that the ROI is huge. So the expenditures are actually cost saving measures in the long run. There should be no conservative opposition, but of course there is. Is it just a philosophical/political impasse (block ANY new gov't programs, even helpful ones), or is it the elites lashing out because they don't want poor, colored folks to succeed and compete on a more level playing field with their privileged kids? Sure the rich would rather have tax breaks now than more education spending, but as I said these programs save $ in the long run and make a stronger society, which will reduce the rich people's future tax burden and probably result in more capital gains for them, as these kids grow up on a better trajectory and become valuable consumers/workers.

The data suggests that Millennials are more tech savvy, more socially aware, and more into volunteerism than previous generations. As you said, they are worldly, and with the internet at their fingertips, they are on average much more cognizant of global social issues than their parents and grandparents at that age. So really there is no excuse for them to turn their back on social problems, even if they can't fully "relate" to the needy. But residential and workplace segregation is a terrible problem, because if you don't see diversity and the struggles of others each day, you are less likely to do anything about them. I believe that most young adults (and most of us on this email) would vote for policies and leaders who want to make our society fairer. The problem is no one is putting such debates up for a vote (probably because most of the political system is co-opted by the elites as J said). I think the youth are in the right place morally, but they just need a spark or a dynamic figure to lead them, like a modern day "Ask not what your country can do for you..." or "We hold these truths to be self evident..." We don't have that unfortunately. All our brightest young people are going into the private sector, where they can't "do good" because they are beholden to shareholders, and fighting for their lives against cutthroat competition. Only when they're super rich and semi-retired like Gates will they hopefully give back. Look at us; we have money and are well aware of the problems out there, but we don't have the time or courage to advocate for the causes (many self-imposed or social barriers stand in our way). We will vote the "right way" if the occasion arises, and maybe even give some money, but we aren't able to put in the hours (not to mention blood, sweat, and tears) to lead the populist charge as you said. I admire the few folks who do.

But maybe all is not lost and there are some leaders with potential. Here's a Fresh Air interview about Pope Francis, and of course the main theme of his papacy so far is social justice and critique of global wealth inequality (and the institutions that perpetuate it). Europe is a powder keg of discontent regarding gov't corruption, voiding the social contract, and widespread unemployment. I will give Obama credit too that he has raised the issues of race and inequality (maybe not by choice) more than any other president since LBJ. Maybe he hasn't moved the needle on the debate in America, but he isn't letting us sweep it under the rug. So we are seeing some top-down emphasis of the race and wealth issues, but unfortunately that hasn't manifested itself into new laws and reforms (the plutocrats can control lawmaking bodies a lot more than individual executives).

Re: the US history of populism, you are right that conditions were a lot different when labor was stronger and the working white poor were a major Dem constituency. But the party made a decision in the '60s to fight one injustice at the expense of the other. They fought against racial discrimination, which lost them the Southern white vote. And in order to make up for it, they had to go more corporate and turn their back on labor too. So the Dems are more elite than ever, even if the GOP is really struggling with key demos like single women and non-whites. But I don't think that racist whites in the US hate colored folks just because they bought into the economic propaganda of the elites. They had those prejudices already, and the elites just fueled the fire with scare tactics relating to black uprising/crime and immigrants taking jobs/services. We also have to address the non-economic drivers of modern racism (I am not quite sure what they are but I could speculate). I agree that the GOP is using religion as a wedge issue to keep its base from turning populist, which is of course ironic because Christian values are clearly misaligned with the modern GOP.

--------

As A pointed out, "But I think one of the under-appreciated developments in American politics is that, in previous historical periods, when you had this unbalanced of an income, populists would rally the poorer classes to demand change. ", I find it interesting that at that time, communication between groups was limited by lack of technology.  Now we have instant communication with anyone in the world.  You would think that it would be easy to put all of the pieces together for the oppressed to rise up.  But, these same lines of instant communication are also used to manipulate us in ways we'd never seen before, essentially keeping us divided, blaming one another for our lot in life instead of seeing the real picture.

-----------

http://www.insidepolitics.org/ps111/riseofinternet.html
http://gnovisjournal.org/2010/04/25/media-fragmentation-and-political-polarization-how-high-choice-media-environment-leads-great/

Thanks, L. Yeah media fragmentation in the US can be a big problem. Families don't tune into the same 3 info sources around the radio or TV set anymore. So of course FNC says that everything MSNBC says are lies, and vice versa (and CNN just talks about the Kardashians) - adding to distrust, misinformation, and most harmful... polarization. Plus we are blitzed with too much info now on our multiple devices (especially entertainment/time-wasting content), which distracts us from thinking, collaborating, and rising up together.

Twitter and Facebook helped the Arab Spring gain momentum, but sadly it can't really do the same for us (even though we invented those services). Well, when commercial crap like Bieber and Coke are the most liked/followed entities on social media, you know we have a problem. It seems US social media is (deliberately or not) functioning to prevent us from mass political debate and assembly, whereas it has the opposite effect in repressive/dysfunctional foreign countries (which is why their state security services try to block/monitor it)?

Black Friday game theory

Cont'd from previous posts...

But like it was said earlier it is the prisoner's dilemma.  If target is open starting 10pm thanksgiving day with deals and no one else is, then that pool of money is going to disproportionately go to target.  So then JCP says "we'll be open too!" and on and on.  So now everyone is open and everyone is luring people in the only way they know how.

But this is also one of those instances where people grumble about traffic while sitting in traffic.  They ARE traffic.  Complaining about black friday as americans rings hollow when more americans go for the sales than vote. 

-----

Thx, M. Re: the pris. dilemma... I think the comparison is not quite valid here for a couple reasons. First, some PD's can be defeated through communication and agreements. Maybe through retail trade groups or whatnot, the major players can promise that they will not open before time X. If certain companies violate, then they could be kicked out of the group and flamed by the rest. Also, not all retailers decided to play this game. Notable exceptions are Nordstrom, Costco, Radio Shack, and I think Kohl's. They know that there are just as many potential costs as there are benefits to opening early (as we have already discussed).

“There’s a PR benefit to holding out, just as there’s a PR benefit to opening early,” said Roger Beahm, a marketing professor at Wake Forest University. “We know that there is a consumer backlash to this.”

http://www.huffingtonpost.com/2013/11/13/costco-thanksgiving_n_4262774.html

Second, the PD assumes a "one turn" game with finality, and the players are "all-in" for the decision at hand (open early or not, collaborate or defect). But in reality, holiday shopping is a sequential game with multiple "turns" where players can react to each other's past actions. Holiday shopping is a relatively long war; even if you "lose" B.F. turn 1, you can win later turns, which may in fact help your overall performance. So that may change the payoffs and strategies away from a pure PD.

Some of us discussed that opening earlier for B.F. just shifts purchases ahead in time, and doesn't necessarily earn you more overall profit for the entire season. Forbes had a related comment about that:

A Black Friday spending analysis from [MasterCard] shows a whopping 70% of consumer spending happens at the first two stores a shopper visits.

http://www.forbes.com/sites/clareoconnor/2013/11/28/heres-the-real-reason-stores-are-open-for-black-friday-sales-on-thanksgiving-day/

I guess that makes sense, as shoppers get tired after a few hours of lines, crowds, and cold. Also the article says that consumers allocate 50% of their total holiday budget to B.F. (I assume they got this from exit surveys, which can be problematic, and a lot of customers don't even make a formal budget so it's hard to be sure). So if all that is true, it will incentivize retailers to keep one-upping each other on B.F. But again, thinking about this as a sequential game, during B.F. retailers are fighting for a big slice of the pie (maybe 50% of total holiday spending), but there is still another 50% after. So maybe it is more cost-effective to not "blow one's load" on the B.F. arms race, where competition is fierce, ROI could be lower, and there is more risk. Instead wait as the morons duke it out, and then go on a blitz to eat up all the post-B.F. dollars, after you rivals have depleted their resources. You know, all these lame business-is-war analogies. :)

Lastly, I agree with your point that shoppers bitch about the B.F. issues, yet they are contributing to them at the same time. Maybe many of them have unrealistic expectations that they will just stroll into an empty store at a time that suits them, and quickly get all their wish list items on sale. That is called the internet. :) Assuming that sites don't crash due to heightened traffic.

More on Black Friday

A friend just sent this: http://online.wsj.com/news/articles/SB10001424052702304281004579217863262940166

So folks are stressing and fighting over what is likely phantom discounts anyway.

After working at a discount e-commerce site, I know that some consumers care more about the "% discount", rather than the final price (or even the actual use of the product!). So they fall into the trap. It's probably better to know what max price you will pay for item X, and just pick the nicest one that qualifies. The MSRP may suggest relative quality, but that's about it. My econ prof said that $, not %, counts in the end; a cheap toy may be 80% off, but if that only means $3 more in your pocket, it's really not that impressive. And of course it depends if the full price is reasonable or artificially marked up as the article said. But the majority of consumers are not that econ-finance savvy, and probably don't consider these factors in the heat of the moment.

I just fear that as long as real wages stagnate, the wealth gap widens, and purchasing power wanes (due to higher housing, medical, education, & energy costs), people will find it harder and harder to maintain or increase their consumption (despite all the marketing messages and social pressure to do so). Robert Reich said that these trends have been around since the 1980s, but the US middle class made up for it with home equity and consumer credit (to the benefit of Wall St.). Now that loans are getting harder to come by, and of course curbing consumption is out of the question, the stakes become even higher to secure "deals". That could change the shopping psychology and make B.F. pandemonium and conflict more likely and serious going forward. It's like NFL stadium violence - is it any surprise that it is likely correlated with soaring ticket prices and overall "cost of fandom"? When you squeeze and raise the stakes on people, of course they are going to react badly when they feel like they are not getting their money's worth.

-----

Happy Thanksgiving dude :)

It's always kind of baffling to me seeing these kind of "is X good for society" emails, because ultimately there is no benevolent overlord responsible for arbitrating this kind of thing. It's just ... game theory, lots of individuals trying to maximize their own whatever (sometimes profit, sometimes happiness, sometimes self-satisfaction).

Black Friday is an interesting concept. My basic mental model is that initially it was just a naturally-big shopping day, and (given retailers' low margins) the first day that retailers showed a profit for the year. Then retailers started competing for shoppers, and because their main method of competition was "sales" ... here we are.

Obviously you know that sales and coupons were the old-school form of price discrimination, before the internet and quants like you figuring out how to get every dime out of the customer ;) But now it's just old-school prisoner's dilemma, with every retailer defecting and offering sales. This year they're especially screwed, because JCP is desperate to sell off the old Ron Johnson-era merch and is undercutting everyone. It's straight-up race to the bottom, where half these guys will be out of business in 3 years and the rest will be lucky to survive 10. 

They're not doing it because it's good for the country or the culture. They're doing it because, like a heroin addict that hasn't gotten a hit in a while, momma needs that holiday shopping boost, and they'll do *whatever* it takes to get it. Some trickery, some desperation, but this is what giant corporations look like when they die. Like those images of a dinosaur trapped in a tar pit, trying to get out, every struggle dragging it deeper into the black ooze.

What we're seeing is the end of retail, and this and the next few Black Fridays are going to be ground zero for that shit. Loot what you can :)

-----

Thx J, same to you.

Even though B.F. is not a top-down, command economy program, I still think it is valid to evaluate the consequences. If we don't do that, then how can we ever know if a change is needed? Yes, game theory can accurately describe and predict players' actions given a set of conditions, but to me that kind of abstracts the situation as if we were just "rats in a maze". B.F. is about more than transactions. I wanted to consider the human side and social costs, because it's easy to take them for granted when all we see are the cool items, big crowds, and blowout prices in the media. As you said, if these dinosaur retailers are caught in a suicide spiral race-to-the-bottom, then the poor decisions that contributed to that situation will eventually hurt millions of shareholders, employees, and society as a whole (less tax collected, more pressure on social services).

Negative industry trends aside, is B.F. the best method for buyers and sellers to achieve their seasonal goals? As recently as the 1990s, B.F. wasn't even the heaviest US shopping day (that was reserved for the Sat before Xmas, for gift procrastinators LOL). Some limits were still respected. But as you said, companies are locked in a fierce rivalry, and some "innovators" thought to exploit B.F. more. I wish customers would push back and say, "I don't care how big your sale is. The holidays are for spending time with loved ones and being good to one another, not fighting for a spot in line at 3AM just to get an Xbox. It's not right to encourage rabid consumerism and make your employees work these crazy hours either." If we collectively protest and boycott, change will come, like apartheid in SA and cigarettes in the US. Unfortunately there haven't been many big success stories of corporate boycotts recently (they defeated the labor movement, and they know how to diffuse grassroots protest too). But it's American tradition: the Boston Tea Party was a boycott of sorts by colonial merchants getting squeezed by the East India Company. So going back to B.F., unfortunately there won't be boycotts because some customers will likely "cross the picket line," lusting after deals.

I am not that familiar with the JCP case, but didn't Johnson try to make the company more upscale and resist coupons/discounts? Obviously that didn't work and they are now making a big reversal, but JCP is not a leading retailer (it's $12B annual US revenue is 16% the size of Target's). So I don't think they are driving B.F. trends, but they probably will be gone in 3 years as you said (Sears too). However, predictions of retail's demise could be exaggerated (or at least premature). In the Bay we are surrounded with innovation, disruption, and e-commerce, so that could bias our perspective, because US e-commerce is still only 6% of total retail sales in 2013 (and the biggest category is travel, because it is expensive and there isn't really a retail equivalent after the death of the travel agency). Goliath Amazon still has less revenue than Target. Shopping is still a communal, visceral experience that e-commerce and social media haven't fully supplanted yet (some fun, scientific reads on the subject if you're interested: link). Even if Amazon develops cost-effective same-day delivery, many people will still go to the big box or the mall because it offers a value proposition that online doesn't exactly match. Yes brick-and-mortar retail is inefficient with its labor, rent, and overhead, but customers still demand it.  

http://ycharts.com/indicators/ecommerce_sales_as_percent_retail_sales
http://www.statista.com/statistics/191145/travel-e-commerce-sales-in-the-united-states-since-2002/
http://trends.e-strategyblog.com/2013/04/25/american-retail-ecommerce-sales-by-product-category/10687

------

I'm sure some of you saw this too, but Black Friday isn't even a good predictor of holiday sales: 

http://www.theatlantic.com/business/archive/2013/11/the-economic-case-for-ignoring-black-friday/281926/

So the whole thing is kind of a waste of time from the perspective of the economy; people are going to spend what they spend over the holiday period, and black friday can only take from that pool, not get consumers to spend more.

------

Thx, A. I agree, B.F. just accelerates and concentrates purchases, but then many folks will run out of cash/energy afterward and not shop much more for the rest of the season (like a hangover). If companies want to keep pouring gasoline on the fire and make B.F. more and more of a ludicrous spectacle where the worst of human nature and American "culture" is on display, then I wish gov'ts would pass laws requiring retailers to fit the bill for the extra police, paramedics, and others who have to work overtime, manage the crowds, and respond to shopping-related incidents. Maybe that would change their P&L calculus. 

Is Black Friday even worth it to our society/economy?

http://news.yahoo.com/blogs/sideshow/-walmartfights-sparks-a-trend-on-twitter-193622540.html
http://en.wikipedia.org/wiki/Black_Friday_%28shopping%29

As we know, scheduled promotions like "Cash for Clunkers" many not generate new transactions, but merely shift them to a specific date (i.e. I would have bought a TV for Uncle Bob at some point before Xmas, but I might as well go on B.F.). Sure, while you are frantically trekking through Walmart, the mob mentality and "holiday spirit" make you less responsible with the wallet, so you might be inspired to get some other items (which is incremental revenue), but it's really hard for businesses to measure if those purchases are totally due to B.F. With the "arms race" of fierce retail competition, and marketers whipping consumers into a frenzy (and now mobile broadband and social media make it even easier to learn about "great deals"), companies have to take a loss or break even on certain products just to earn some eyeballs (they likely will have to pay more for ads due to more competition). Of course they may only have 5 super-cheap TVs per store, but clearly they are pricing competitive items lower than they'd ideally want to, and likely below what the customer's normal willingness-to-pay is (sans B.F. expectations).

That is a risk too - customers are expecting great deals, so if they are sold out by the time they get to the shelves (even for folks who waited for hours in the cold), that will cause a lot of anger, and the retailer runs the risk of creating a detractor who badmouths them on social media. And that's if they're lucky - shortages may lead to riots and deaths, as we've unfortunately seen in the past. Imagine all the man-hours of extra associates, police, and other emergency responders needed to handle the B.F. madness and possible incidents. Imagine the carbon footprint of all the traffic jams and the utilities from the extra hours that stores are open (and it's getting earlier and earlier - some stores opened at 8PM ON THANKSGIVING... by 2020 Black Friday will be on Labor Day). B.F. may not drive many purely incremental sales, but it sure as heck will create many incremental traffic accidents, injuries, etc. The employees that have to deal with the insanity may get overtime pay, but they also will experience elevated stress and possibly employer resentment, which are documented to affect workplace productivity, personal relationships, and health (not to mention all the stress and panic behind the scenes where non-store employees have to rush to meet the B.F. related logistics, marketing, product launches, and other deadlines). I am pretty sure some folks at Sony and Microsoft would have liked a few more weeks to iron out the bugs associated with their new gaming consoles, but they had to get them on the shelves in time for B.F. And as far as I know, no one is getting B.F. or Xmas bonuses anymore for their extra efforts.

On the consumer side, the post-Great-Recession US shopper spends on avg. $400 during B.F. weekend (with 200M+ unique shoppers). People may not behave very rationally during B.F. (understatement of the year), so they may make inefficient and imprudent purchasing decisions that will have negative consequences on their economic viability (and that of their dependents). There are socioeconomic costs associated with frivolous spending: credit default, back taxes, late rent payments, etc. Some opportunists make money off things like that, but there will be preventable economic losses too. Also, the more we spend on shopping, the less we have to donate to charities (or even buy more medicine and healthier food). The fatigue, emotions, and sleep deprivation associated with B.F. shopping will affect shopper health, relationships, and productivity for weeks - especially during the winter time of bad weather and virus spreads. Adding all this up, I can imagine that the social costs of B.F. could easily top $1B in the US. I am not sure how much incremental profit B.F. generates (for scale, wiki says WM gets $5B from B.F., and it is ~3% of all US retail sales), but clearly it should be discounted if we want to make a fair, holistic value calculation.

Even Cyber Monday could have social costs by reducing workplace productivity and tying up bandwidth that could make pageload/latency times worse and important/emergency messages slower.

"Toxic Hot Seat" about cigarettes, flame retardants, and death for profits

Toxic Hot Seat:

http://www.huffingtonpost.com/marcia-g-yerman/toxic-hot-seat-ignites-aw_b_4338572.html

This is a clear example of Pope Francis' principle of greed-driven "murder", for lack of a better term. Some companies and trade groups decided to protect their profits rather than do the right thing for their customers. And this has created enormous social costs for everyone.

The storyline:

- Since the '70s, 2-6K Americans died each year from residential fires.

- By far, the biggest driver of these tragedies are unmonitored cigarettes (yet another way they kill users and bystanders), and this trend persists in most nations.

- A self-extinguishing cigarette is affordable and easily implementable, but the tobacco industry resisted, and created a "fire marshal" advocacy org to convince leaders and public that the problem was the "fuel" (household flammable stuff), not the ignition.

- So the chemical industry jumped on this and developed "flame retardants" that could be sprayed on curtains, furniture cushions, etc. (this was during the era of DDT and such where toxicity testing was nonexistent).

- In 1975, a professional study was warped by some CA regulators and lobbyists to mandate all furniture sold in the state to contain fire retardants with no health risk conditions. Since CA was such a big market, and it was expensive to develop 2 versions of furniture, manufacturers decided to put retardants on everything, and the standard has stuck.

- Despite a huge "astroturf" (fake grassroots) chemical industry push, Maine was able to ban retardants in favor of safer alternatives. The lobbying machine has prevailed so far in CA, and Representative Leno has sponsored several similar bills to ban them, but they failed each time due to the irrational fire scare and industry advocacy. Changing the law is part of Gov. Brown's current agenda, and so far he was able to at least permit the sale of furniture that doesn't contain retardants. It's up to consumers to check whether the products they buy have them or not, but an outright ban like Maine is still elusive.

Commentary:

So what are the consequences of big tobacco's greed/negligence and big chemical's opportunism (now a $5B global industry)? We know that preventing/reducing fire severity is important and can save lives/money. But are these retardants the most cost-effective solution? Studies show that smoke alarms and sprinklers are much more effective than retardants - with nearly zero downside. Only the retardant manufacturer association has produced a few suspect studies to support the use of their product. But fires are scary and their arguments won the day for decades ("whatever it takes" to prevent fires!). Fire danger is a lot more tangible and acute than nearly invisible carcinogens that may need extended exposure to do measurable harm (but no less bodily harm than burns). So it's easy to fixate on the fire risk, which may trump other concerns. 

It's one thing if retardants are ~90% effective (at least as effective as condoms), but they're not. The flame retardant standards state that the material must be able to resist a "small flame" for 12 secs. While that could be effective in some situations, it is deficient in many tests and in the field, because a furniture's unprotected covering burns first, so by the time it hits the cushions (that contain the retardants), it's no longer a small flame and the retardants are no longer effective.

So they don't really make us safer, yet they are likely making us sicker. Retardants are known carcinogens and mutagens like thalates, bromides, and BPA (that the baby industry has been forced to remove from plastic products due to customer anger). Retardant-containing products still burn, so when firefighters have to respond, they are rushing into and inhaling the chemical soup. When SFFD personnel were tested, incidence of middle-age female breast cancer was 6X the population average, and incidence of cancer among retirees was also unusually high. Parents groups fought to get these chemicals banned in child pajamas (on toxic grounds) and won, yet the exact same chemicals are still present in child car seats, play pens, backpacks, furniture, etc. that kids touch and lick. They're still present in those products because it's expensive to mount a legal challenge, and chemicals get the benefit of the doubt.

Speaking of that, why do virtually all side effects, interactions, allowable dosages, etc. need to be thoroughly documented and scrutinized for a drug to reach the market, but chemicals are "innocent until proven guilty"? It's about the burden of proof. As depicted in films like "A Civil Action", it is very hard to prove that a specific chemical directly caused measurable harm over what could be years of exposure to many chemicals. Companies can just show that rats "survived" when exposed to the chemical under specific, arbitrary conditions - and that is supposed to prove that they are universally safe.

And when pro-business leaders continually weaken and defund the EPA and other regulators, that makes them look ineffective/unnecessary and strengthens the industry argument to just trust the companies, not delay new product approval with frivolous tests, and not stifle chemical innovation, which "creates jobs/revenue" and "makes our lives better". And this is not just about retardants, but literally thousands of chemicals that we don't even know we're exposed to each day. Like all those "dispersants" used by BP to make the Gulf spill look less nasty to the naked eye - we have no idea WTF they do to living tissues over time, and they will likely show up in seafood, breast milk, etc.

There is an underlying assumption that benevolent gov't is watching out for us, and products wouldn't be on the shelf unless they were totally safe (and Dow and Monsanto say you can trust them). Well after leaded gas, asbestos, CFCs, and the sad history of tobacco, we should know better. The only ones who are looking after public safety are scientists, survivor victims, and grassroots orgs. And politicians will only listen to them if voter anger outweighs industry lobbying dollars. But this is obviously short-sighted leadership. It makes no sense to threaten and poison future society's health and productivity for present-day industry profits. Taxpayers and gov't coffers will be burdened by the health costs of harmful chemicals, which means less money for other national priorities. Everyone loses but the companies and their stakeholders.

The same thing is playing out in places like China with pathetic regulation, but they recognize this and are trying to improve. Can you imagine the impact on their economy when their current population ages and disproportionately develops all sorts of illnesses from the ubiquitous pollution, toxins, etc.?

And it's not like these "innovative chemicals" are so critical to human survival. We can get along fine with many natural, renewable products like wool, plants, and wax. We don't need chemical X to make our jacket down 0.1% warmer but our kids 10% sicker. Someone has to say enough is enough.

Obamacare fiasco discussion

If you thought Obamacare was bad...

As usual, China is way worse.

http://m.npr.org/programs/all/2/242344329

----

Part of another conversation I had about the recent Obamacare troubles... would love to hear your thoughts if you have time.

http://talkingpointsmemo.com/dc/obamacare-fix-what-is-it-good-for

Yeah, I have been really saddened and disappointed by how the Obama admin. has rolled out heathcare.gov. When you know the GOP is looking for any little excuse to scuttle the program, you'd think they would be extra careful every step of the way! It seriously looks like it was put together in a week by interns. I know health care is complex as hell, with different laws in different states, so then maybe they should have fought harder for single payer from the start! And as you suggested, if this is a wacky attempt to convince America that single payer is the way to go, it probably won't worry either since trust in gov't is so low. Well, even if Obamacare fails and is repealed by President Christie (LOL), I think it got the health-industrial complex to pay attention and realize that Americans (and even some of our leaders) are tired of being gouged by a broken system that under-delivers on health outcomes for all the money it takes in. Companies/doctors are being more cautious and not just assuming they can raise prices, over-prescribe treatments, cherry pick "profitable" patients, and bilk Medicare and get away with it every time. So I guess that is some progress?

I agree with you about the misrepresentation. That is the problem when passing the bill becomes the goal rather than actually helping citizens. But unfortunately Obama has not been a very transparent president, despite his claims. I think that is a reflection of his and his admin's poor communication skills. While Obama is an inspirational campaign speaker on the abstract level, he is fairly poor in terms of leadership and on the practical policy level, as we saw during the Romney debates. He gets too academic and technical, and doesn't empathize with others. Most of us barely understand our own health care, much less gov't policy, and it can be an emotional issue. Obama relied too much on logic/macroeconomics, and didn't do enough to put people's fears and concerns to rest (especially in the face of the GOP lie machine). Unlike Clinton, Obama never "felt our pain." So there wasn't much trust from the get-go, which was reflected in Obamacare's persistent low approval numbers among the public (despite high public support for the individual parts of the law, like covering kids until 25, no rejections for pre-existing conditions, etc.). So if he communicated better from the start, I think the public could be a lot more forgiving of the glitches now. Customers mostly forgave the iPhone 4 for it's "death grip" dropped calls problem, because they loved the company and were addicted to the product. ;)

Speaking of the implementation, it is seriously his Bay of Pigs. Like Kennedy, I bet Obama was too trusting of his people that they would handle it properly, and wasn't skeptical enough. He did go on the national PR tour to drum up support and interest, but he didn't really manage expectations and prepare the country for what could go wrong. In 2013, there is just no excuse for a website that works so poorly. Testing and pilot sessions with real customers were severely lacking. I haven't had time to follow the media discussion, so I am not sure if the pundits and Sebelius already covered these points. But they had ample time to prepare for the website launch, so I don't know why their product is so crappy now.

In general, I think a website is supposed to launch small at first to mitigate risk (like 5% of expected peak traffic) and gradually ramp up as long as things are working. I am not sure, but did healthcare.gov suddenly turn on for the whole country all at once? That is a bad strategy. Maybe try it out with ISPs from a couple small states first, check the performance, collect feedback, and make necessary fixes before expanding the reach. It would have let them catch bugs (or maybe delay the 100% launch) before it became a national embarrassment with thousands of frustrated users. I heard an alternate rollout plan was first to make sure the site was stable and let users browse plans only (to make sure the info they got was accurate too!), then when registration/enrollment was ready and tested, they would add that functionality later. But Obama's people worried that users would browse plans, not like any, and never come back (if you are so worried about that, then maybe your product is not so appealing and you should fix that first!), so they wanted to require registration prior to browsing. There was also the political pressure to get as many people enrolled as possible during the initial launch period, so they required visitors to register up front. But that is bad for several reasons: web users often mess up the registration process, and if you mandate it at launch, you introduce a bottleneck and they will flood your customer support line all at once. Also registration is annoying, so that will cause many users to just drop off too (if they can even get the page to load). Plus let's remember that many uninsured Americans or those who buy individual plans could be lower income and less tech savvy, even if they have a helper to assist.

On the issue of the "deception" over keeping your old plan and old doctor, I have mixed feelings about it. On one hand, if Obama knows that a major argument against Obamacare is a supposed loss of freedom, then "forcing" people to change plans/doctors (even to ostensibly better ones) is risky and plays into the GOP hand. But if they need to eliminate ineligible plans to make the whole system work, then I understand, but that is a major program risk that should have been addressed years ago. The doctor-patient relationship is sensitive and almost sacred, so if Obamacare really does prevent some people from keeping their old doctors (even if they are crappy or charge too much), then that is unacceptable to me. I think you can convince people to let the gov't upgrade their coverage (but again, Obama's people did a terrible communication job), but change is hard for some folks and you can't just drop their coverage suddenly and tell them to choose a new plan on a buggy website. All this mess plays into the conservative narrative that gov't is totally meddling in our lives and inept in providing services (which is unfortunately true at times). The 'fix' from the first link above is too little, too late. Maybe a better plan would be what companies do for new hires. They default you into a basic health plan, and you have the option to pick a different one within 60 days. At least then everyone is sure they are covered from the start, and there is no rush/panic/anger. Maybe that is impossible nationally, since health plans and laws are different in each state (another issue we probably need to address)?

http://www.washingtonpost.com/blogs/plum-line/wp/2013/11/13/what-the-obamacare-enrollment-numbers-really-tell-us/?wprss=rss_homepage&clsrd

The SF Bay Area housing crisis

http://blogs.kqed.org/newsfix/2013/10/30/kqed-launches-priced-out-the-high-cost-of-housing-in-the-bay-area/

This is probably he most important social issue for the area. It's strange that SF has a proud history of inclusion (and a pretty good record in overall today), but this housing crisis is driven by exclusion and privilege, not necessarily "normal economics."

-----

Well if you average the entire Bay Area, places like Brentwood and Antioch will bring down the average. But SF is #3 priciest per sq. ft. in the US behind NYC and HNL. Versus the whole world, SF is not that bad, but real estate is really messed up in older, more cramped cities in the EU and Asia.

http://curbed.com/archives/2011/09/27/heres-a-chart-of-the-worlds-cities-by-price-per-square-foot.php

It's not even about rent vs. own. Renting is very pricey too, obviously. I think one aspect of the problem is that landlords are taking rent-controlled (or regular) units off the market in order to convert them to TIC/condos for more $, or sell the whole bldg to speculators because there is no much insane pent-up demand and high willingness to pay from the upper classes. So rental supply is going down, which causes rents to rise too.

----

If you hit Wikipedia for median income by city in ca you'll find that those areas are also quite rich.  Housing doesn't happen in a vacuum.  More a function of income and the real problem is income inequality

It also happens with respect to density. The more houses/apartments available = the cheaper the rent. Nimbys in the Bay Area would like to still have their .5 acre ranch houses on some of the most valuable land on earth, meaning that the poors have to live 2 hours commute from their jobs.

-----

Totally agree. The social justice element also comes into play that historically lower income, blue collar, often minority neighborhoods (Mission, Castro, etc.) used to be less desirable for yuppies and therefore more affordable and off the radar of speculators. But now they are hipster paradises, and with gentrification, the "traditional residents" can't even maintain their old lives and are priced out of the market. I guess that is like economic eminent domain.

Measures like rent control or mandated affordable housing are often economically inefficient and flawed (because they are written and implemented by flawed bureaucrats, like healthcare.gov), but sometime optimal economics must take a back seat to concerns about fairness and social values (see taxes, labor laws). I'd rather have confusing rent control laws and too high minimum wage than a total lack of regulation (at least imperfect laws are a springboard for iteration and improvement). Because as you said, with the huge inequality gap, what is the point for the poorest 60% to struggle so hard trying to fit into the rich man's world (yet not really be allowed to live in the rich man's world... I guess like the "Elysium" plot, which I did not see)? Most people with a shred of heart would probably agree that there is something inherently unjust with greedy politicians/companies and wealthy invaders "annexing" and developing suddenly desirable land, which leads to the economic eviction of the residents who were there before (through legal market mechanisms and the landlords/property owners). Pardon my melodrama, but it's scarily similar to the Trail of Tears.

I heard an interview about the subject a while back, and one new SF resident and tech employee said something like, "Assholes (his word) like me can afford $3K/1BR rent and are making it hard on the previous residents, but hey, I want to live here." At least he had clear eyes. It's just sad because everyone has one life to live, and we don't want to compromise or suffer if we don't have to - even at cost to others (esp. when others' suffering at our hands is mostly invisible). Sorry for stereotyping, but I think the Gen X-ers and Millennials especially (present company excluded) really fixate on what I like to call "life maximization." Similar to their work habits of optimizing, achieving, and disrupting everything, in life they want it their way and they want it all. Best job, best home, best network, best gear, best marathon time, best family... best, best, best. When that is not really the goal but the status quo expectation in SF/Si Valley (when excellent becomes average/normal, what do all the sub-excellent people do?), then that doesn't really encourage a culture of togetherness and SHARING. We don't need the best life... can't we just enjoy our regular life? Isn't enough enough? Can I give a little up and still live a plenty comfy life so that more needy people can get a break? People don't ask these questions of themselves enough (myself included).

Sharing is the key I think. This isn't my world, it's our world. There is PLENTY of food, money, and room for everyone if we share reasonably, but the problem is the fucking 1% and the institutions who advocate for them just don't want to (and don't have to). Americans and some other cultures really fixate on fencing off what's yours and amassing/diving the pie, so that breeds an adversarial, zero-sum attitude that conservatives really eat up. Therefore, it's especially sad when educated, young, open-minded "liberal" West Coast people do the same and may not even realize it. At least liberals don't bad-mouth and hate the people they marginalize, but that doesn't leave them in the clear (myself included).

The best teachers I have had in my life (not necessarily in school) really cared about expanding the pie rather than dividing it. How can we make everyone happier? Any asshole with a bit of smarts can compete and beat others. But a truly smart, wise person goes out of his/her way to cooperate, find ways to align incentives, and make everyone better off - not just him/herself. For all the smug talent and genius brains and big money in the Bay, that skill is scarily absent. Don't get me wrong, some people practice it faithfully and I am in awe of them, but it's not enough to stem the tide. That's why I probably will leave this place next year, as much as it makes me sad and for all I am leaving behind. I don't want my kid to grow up in such a culture. At least I will make room for another person/family to have my stressful job and little slice of condo.

-----

Sorry I realize that I was unclear about one point in my last rant (unclear about at least one point, haha).

The Google buses are often scapegoats but not really the problem. By no means am I placing the bulk of the blame on the young, techie hipsters invading the Bay. They are not really the 1% and not the real driver of the problem, they are just a "symptom". Except for Zuck who bought a whole block of SF for privacy haha.

As usual, the problem is the old rich Nimby fucks that A alluded to. They are the ones who gobbled up all the choice land decades ago, and who currently occupy all the seats of political power and business influence. They are the ones who invested/profited from the tech boom (more than the current workforce), and stand to profit from real estate development and appreciation. They took so much that the current hipsters are practically forced to gentrify the Mission and maybe Hunter's Point soon. I bet most hipsters would much rather live in Nob Hill or downtown Paly, but all the old rich pricks there are just cramping their style anyway (and even a Twitter engineer still can't afford a $5M Nob Hill pad).

As A alluded to, we are not in the '50s Levittowns anymore, or not even in the '90s nouveau-riche gated golf communities. But the assholes want to keep us in the past, because they like the status and lifestyle they have, and don't want to give up even a shred to others. The city of Paris (and many other ancient cities) is like a 10-layer cake. New generations tear down the old crap that isn't working anymore and rebuild to fit the needs of the current people. We aren't really doing that, because the old fossils control the bulldozers. So all we have is SF Elysium vs. Vallejo. 

But "the people" are fighting back, and at least voted down a luxury condo development on the Embarcadero. It's a small victory in a long war that the good guys will almost certainly lose. But we might as well smoke a blunt and celebrate while we can. :)

http://www.sfgate.com/bayarea/nevius/article/Following-the-bankroll-for-SF-s-Propositions-B-C-4961999.php
http://www.npr.org/templates/story/story.php?storyId=243537913&ft=1&f=2&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+NprProgramsATC+%28NPR+Programs%3A+All+Things+Considered%29&utm_content=Yahoo+Search+Results 

-----

i once heard someone say cities are either places to live or places people come to make money.  he was talking about SF and that we were at a crossroads.  it was when matt gonzalez was running for mayor against gavin Newsome.  The speaker was making the case that if Newsome wins, the city will lose its soul, it's artist community, it's diversity of culture and class.  that's what you're looking at here….plain and simple.  and, you know what?  the city sucks now.  It's filled with people who are in a hurry, who lay on their horns, who yell.  

Why a lot of published scientific research could be wrong

http://www.economist.com/blogs/graphicdetail/2013/10/daily-chart-2#

We've talked about this before, but this infographic presents it pretty cleverly.

----

First I would love to see a source on where he gets his numbers of false positives and false negatives.

He also oversimplifies the way experiments work.  For example with the higgs boson.  The experiments were run continuously until statistical significance  was seen.  That means the hypothesis was testing hundreds and hundreds of times.  The false negative/positive issue is resolved by virtue of the experimental setup.  

And on top of that, published work is not set on a pedestal because it is published.  Once it is published there is generally rigorous work to review and replicate the findings.  

So in short, he is wrong, and when he is right it doesn't matter because that is part of the process.  

-----

Those #s were just hypothetical based on a probability distrib, just to make his point. But sadly few scientists even consider type 1 and 2 errors, and just project confidence that their findings are bulletproof. In bio, when ppl try to reproduce expts, they can't about half the time, according to Science mag.

Yeah I think they are referring to distinct hypotheses, so a series of expts relating to the same hypoth. would probably be 1 data point only. There is also a risk associated with repeated "testing until significance", cuz the more measurements you make, the higher the chance you'll get some random signal and mistakenly use it to prove your hypoth. 

Actually you would be surprised how little rigor goes into the review process, I am sure S and A could comment. If the author is a big name with political clout, free pass. No one has the time or resources to verify and replicate work, it is the honor system. If another group wants to build on the research and don't get the same results, they may publish that, but may not to avoid controversy and be embarrassed if others don't buy it. 

But considering the incentive structure and human nature and the higher stakes of published results, it's a bad combo. Also the journal wants to publish big sexy results to make more money so there is COI.

----

re: "mistakenly use it to prove your hypoth" I think you don't understand what I'm describing.  The measurements means more averaging as I'm describing it.  And more averaging means LESS chance of random signal causing an error.  It is just a method for eliminating "noise" whatever that noise may be.  For the higgs boson it was a very specific energy level measured from the decay.  Every time they took another measurement, if the higgs existed the energy signature would be reinforced and other sources of energy negated, or the opposite if it did not exist.  And with some fancy math they get a confidence level based on number of trials and blah blah.  So not all experiments are susceptible in the same way.

Re COI and big names.  Yea.  Reinheart and Roghoff (sp?) is an example where more weight was given to big names.  But it is also an example of the fact that people do look at this stuff.  Though you could argue the damage was done by the time we figured out their conclusions weren't great.

In any case I'm not arguing that false positivies don't exist in academic journals.  I am arguing that this particular guy's math that proves it is false at worst or misleading at best.

----

I see, thanks M. In that case, the higgs boson expt was doing its own internal QA and replicating the experiment many times (something very lacking in bio and business, where many expts are underpowered). So if the variance is small and there is a strong signal from most of the trials, then that is a very convincing result.

At work we were struggling with a related issue. We usually don't approve changes to the website unless they pass a lot of statistical and sanity checks. But due to probability we know that we are likely rejecting some features that were truly positive (but we couldn't detect it, by chance or by poor design), and we are approving some features that are in fact neutral or even negative. That is the scary possibility. Sometime our approval criterion is "do no harm", as in if all the key metrics look to be within noise, and we can't reject the null hypothesis, then it's OK. But there is a slim chance that we are approving stuff that is actually very harmful. And if the company is approving 100's of features a year, it's likely some of them are harmful but mislabeled as positive or neutral. Though for mature businesses, where you are squeezing out basis points on the margin, the risk is probably not terrible. I guess there is a balancing act between scientific rigor and business expediency/strategy. But at least in e-commerce, an error isn't likely going to cost lives (unless it's Apple maps LOL). But for consumables, finance, or public works, it could be really bad. And they say we don't need regulation? :)

----

Yeah, it's too bad that knowledge of this sort of thing seems to only reinforce the worst of carpetbagging instincts: get in early, publish as much as you can whether of integrity or not, and then do everything in your power to ensure that no one who comes after you could possibly do it as its supposed to be done in the first place.

Maybe it's a cultural thing - like, perpetuating a hierarchical ponzi scheme requires the smartest of people to buy in to the stupidest of ideas (like blinders, narrowness, and petty infighting)...

PS:  Technology will (ultimately) solve all these problems as democratized access to information will lead to market forces being brought to bear on the most indefensible of regressive attitudes and mentalities (most of which are sustained on the basis of petty economics).

----

We were talking about this, and here were our general follow-up thoughts:

- The sci. method and the peer-review process are hundreds of years old and developed during times when methods were not so empirical & specialized, and there wasn't such $$ at stake for discoveries (for people like Euler and Carnot, I think reputation, knowledge, and prestige might have trumped the incentive to cheat, be negligent, and cut corners)

- Obviously we live in a new age now, and as C said the democratizing effects of technology could mitigate the problem (i.e. a grad student found Rogoff's errors when big time editorial staff and top peers didn't), but there are limits to that in hard science because of the cost and specialization of certain expts (though if you need a certain highly skilled postdoc to do a test a particular nuanced way to get a positive result, probably your result is not that robust)

- At least for bio, we thought to develop an independent, confidential auditing lab within the NIH to verify all published results. Journals and authors have to pay a fee to support it, and an article that passes the audit gets a certification that gives it more clout. The author's lab has to send all the materials over to the NIH (or let them use their equipment), with instructions, and the expert NIH staff have to reproduce the result within reasonable variance (they sign NDAs and no-competes so the author has no fear of being scooped). If they can't reproduce, then the paper can still be published at the journal's discretion, but without certification.

----

I would think an algorithm could be developed (or maybe already has?) that can blindly take in the statistical data from an experiment and verify the conclusions.  Rogoff was a case of bad math not bad data right?  That removes the requirement to reproduce experiments and leaves you with only the first two cases of lies, damned lies, and statistics.

The NIH thing is interesting but there is still money involved which is always problematic.  The NIH is not free from politics since that is its major source of funding.  And there are many many public institutions that don't share the data found through public dollar funded experiments already.  The democratization of technology doesn't help when all the research is behind a pay wall.

----

My comments are enclosed (in-line)...  Also, in case anyone didn't see the full article:  http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble

I would think an algorithm could be developed (or maybe already has?) that can blindly take in the statistical data from an experiment and verify the conclusions.  Rogoff was a case of bad math not bad data right?  That removes the requirement to reproduce experiments and leaves you with only the first two cases of lies, damned lies, and statistics. 

It's a good idea; one would think some of that would be built-in to the tools the scientists and researchers are using but clearly there is progress to be made...

The NIH thing is interesting but there is still money involved which is always problematic.  The NIH is not free from politics since that is its major source of funding.  

True... but federal institutions tend to be bulwarks of trust (certainly to a greater extent than a tobacco company or The Koch Brothers)...

And there are many many public institutions that don't share the data found through public dollar funded experiments already.  

This is also true... but changing.  Even institutions which are known for being benefactors of the public (e.g. U.C. Berkeley) are getting more formal about sharing research and ensuring access... so the change is bound to ripple through to other public institutions too...

The democratization of technology doesn't help when all the research is behind a pay wall.

Yeah, that is slow to change - but perhaps the most hopeful front.  The economics of information mean is such that librarians and libraries are being confronted by these questions so the clock is already ticking: if our engineering library at U.C. Berkeley is getting rid of its books for the value the physical space has to the college then its probably a matter of time until someone rationalizes our giving research to private journals so that they can rip off the campus with subscription fees that (over time) do not seem to be worth more than the promise of a new student.

-----

For Rogoff, I think it was a variety of issues, but some of it was excel formulas pointing to the wrong cells, as well as "improper" suppression of some data points. So that is mostly human error and poor judgment, which is hard for an algo to rectify.

An algo would be cool (like how the IRS algos randomly scan over people's returns and flag errors/possible issues), but then the scientific data has to be formatted and structured according to some standards so the algo can read it properly. And since each article kind of measures different things and employs different significance tests, it could be tricky. Hell, they can't even get healthcare.gov right using 50 contractors (maybe that is their problem!).

There are some free or lower cost internet journals out there... hopefully they can give the heavies a run for their money some day. But due to the prestige factor, no big names want to "slum it" with online journals. But I agree that those journals are ripping off institutions to give then "novel research" that is at least 30% useless and another 30% incorrect.

Jokes aside, we piss and moan about how bad public institutions are (and they are of course not free of corruption either... see the MMS), but as C said, they are our last resort against a totally for-profit world. We need to strengthen the public institutions so that they offer a legit alternative to the for-profits, and then with customer choice it will compel the for-profits to clean up and stop shafting us so much. I guess that's why the health industrial complex fought so hard against single payer, which is BY FAR the best feasible health system in the Western world, warts and all. But with all the dysfunction in Washington, furloughs, and a reduction in public worker compensation/respect, it makes it less likely that our best and brightest will want to go into public service and stay there long enough to make an impact.