Black Friday game theory

Cont'd from previous posts...

But like it was said earlier it is the prisoner's dilemma.  If target is open starting 10pm thanksgiving day with deals and no one else is, then that pool of money is going to disproportionately go to target.  So then JCP says "we'll be open too!" and on and on.  So now everyone is open and everyone is luring people in the only way they know how.

But this is also one of those instances where people grumble about traffic while sitting in traffic.  They ARE traffic.  Complaining about black friday as americans rings hollow when more americans go for the sales than vote. 

-----

Thx, M. Re: the pris. dilemma... I think the comparison is not quite valid here for a couple reasons. First, some PD's can be defeated through communication and agreements. Maybe through retail trade groups or whatnot, the major players can promise that they will not open before time X. If certain companies violate, then they could be kicked out of the group and flamed by the rest. Also, not all retailers decided to play this game. Notable exceptions are Nordstrom, Costco, Radio Shack, and I think Kohl's. They know that there are just as many potential costs as there are benefits to opening early (as we have already discussed).

“There’s a PR benefit to holding out, just as there’s a PR benefit to opening early,” said Roger Beahm, a marketing professor at Wake Forest University. “We know that there is a consumer backlash to this.”

http://www.huffingtonpost.com/2013/11/13/costco-thanksgiving_n_4262774.html

Second, the PD assumes a "one turn" game with finality, and the players are "all-in" for the decision at hand (open early or not, collaborate or defect). But in reality, holiday shopping is a sequential game with multiple "turns" where players can react to each other's past actions. Holiday shopping is a relatively long war; even if you "lose" B.F. turn 1, you can win later turns, which may in fact help your overall performance. So that may change the payoffs and strategies away from a pure PD.

Some of us discussed that opening earlier for B.F. just shifts purchases ahead in time, and doesn't necessarily earn you more overall profit for the entire season. Forbes had a related comment about that:

A Black Friday spending analysis from [MasterCard] shows a whopping 70% of consumer spending happens at the first two stores a shopper visits.

http://www.forbes.com/sites/clareoconnor/2013/11/28/heres-the-real-reason-stores-are-open-for-black-friday-sales-on-thanksgiving-day/

I guess that makes sense, as shoppers get tired after a few hours of lines, crowds, and cold. Also the article says that consumers allocate 50% of their total holiday budget to B.F. (I assume they got this from exit surveys, which can be problematic, and a lot of customers don't even make a formal budget so it's hard to be sure). So if all that is true, it will incentivize retailers to keep one-upping each other on B.F. But again, thinking about this as a sequential game, during B.F. retailers are fighting for a big slice of the pie (maybe 50% of total holiday spending), but there is still another 50% after. So maybe it is more cost-effective to not "blow one's load" on the B.F. arms race, where competition is fierce, ROI could be lower, and there is more risk. Instead wait as the morons duke it out, and then go on a blitz to eat up all the post-B.F. dollars, after you rivals have depleted their resources. You know, all these lame business-is-war analogies. :)

Lastly, I agree with your point that shoppers bitch about the B.F. issues, yet they are contributing to them at the same time. Maybe many of them have unrealistic expectations that they will just stroll into an empty store at a time that suits them, and quickly get all their wish list items on sale. That is called the internet. :) Assuming that sites don't crash due to heightened traffic.

More on Black Friday

A friend just sent this: http://online.wsj.com/news/articles/SB10001424052702304281004579217863262940166

So folks are stressing and fighting over what is likely phantom discounts anyway.

After working at a discount e-commerce site, I know that some consumers care more about the "% discount", rather than the final price (or even the actual use of the product!). So they fall into the trap. It's probably better to know what max price you will pay for item X, and just pick the nicest one that qualifies. The MSRP may suggest relative quality, but that's about it. My econ prof said that $, not %, counts in the end; a cheap toy may be 80% off, but if that only means $3 more in your pocket, it's really not that impressive. And of course it depends if the full price is reasonable or artificially marked up as the article said. But the majority of consumers are not that econ-finance savvy, and probably don't consider these factors in the heat of the moment.

I just fear that as long as real wages stagnate, the wealth gap widens, and purchasing power wanes (due to higher housing, medical, education, & energy costs), people will find it harder and harder to maintain or increase their consumption (despite all the marketing messages and social pressure to do so). Robert Reich said that these trends have been around since the 1980s, but the US middle class made up for it with home equity and consumer credit (to the benefit of Wall St.). Now that loans are getting harder to come by, and of course curbing consumption is out of the question, the stakes become even higher to secure "deals". That could change the shopping psychology and make B.F. pandemonium and conflict more likely and serious going forward. It's like NFL stadium violence - is it any surprise that it is likely correlated with soaring ticket prices and overall "cost of fandom"? When you squeeze and raise the stakes on people, of course they are going to react badly when they feel like they are not getting their money's worth.

-----

Happy Thanksgiving dude :)

It's always kind of baffling to me seeing these kind of "is X good for society" emails, because ultimately there is no benevolent overlord responsible for arbitrating this kind of thing. It's just ... game theory, lots of individuals trying to maximize their own whatever (sometimes profit, sometimes happiness, sometimes self-satisfaction).

Black Friday is an interesting concept. My basic mental model is that initially it was just a naturally-big shopping day, and (given retailers' low margins) the first day that retailers showed a profit for the year. Then retailers started competing for shoppers, and because their main method of competition was "sales" ... here we are.

Obviously you know that sales and coupons were the old-school form of price discrimination, before the internet and quants like you figuring out how to get every dime out of the customer ;) But now it's just old-school prisoner's dilemma, with every retailer defecting and offering sales. This year they're especially screwed, because JCP is desperate to sell off the old Ron Johnson-era merch and is undercutting everyone. It's straight-up race to the bottom, where half these guys will be out of business in 3 years and the rest will be lucky to survive 10. 

They're not doing it because it's good for the country or the culture. They're doing it because, like a heroin addict that hasn't gotten a hit in a while, momma needs that holiday shopping boost, and they'll do *whatever* it takes to get it. Some trickery, some desperation, but this is what giant corporations look like when they die. Like those images of a dinosaur trapped in a tar pit, trying to get out, every struggle dragging it deeper into the black ooze.

What we're seeing is the end of retail, and this and the next few Black Fridays are going to be ground zero for that shit. Loot what you can :)

-----

Thx J, same to you.

Even though B.F. is not a top-down, command economy program, I still think it is valid to evaluate the consequences. If we don't do that, then how can we ever know if a change is needed? Yes, game theory can accurately describe and predict players' actions given a set of conditions, but to me that kind of abstracts the situation as if we were just "rats in a maze". B.F. is about more than transactions. I wanted to consider the human side and social costs, because it's easy to take them for granted when all we see are the cool items, big crowds, and blowout prices in the media. As you said, if these dinosaur retailers are caught in a suicide spiral race-to-the-bottom, then the poor decisions that contributed to that situation will eventually hurt millions of shareholders, employees, and society as a whole (less tax collected, more pressure on social services).

Negative industry trends aside, is B.F. the best method for buyers and sellers to achieve their seasonal goals? As recently as the 1990s, B.F. wasn't even the heaviest US shopping day (that was reserved for the Sat before Xmas, for gift procrastinators LOL). Some limits were still respected. But as you said, companies are locked in a fierce rivalry, and some "innovators" thought to exploit B.F. more. I wish customers would push back and say, "I don't care how big your sale is. The holidays are for spending time with loved ones and being good to one another, not fighting for a spot in line at 3AM just to get an Xbox. It's not right to encourage rabid consumerism and make your employees work these crazy hours either." If we collectively protest and boycott, change will come, like apartheid in SA and cigarettes in the US. Unfortunately there haven't been many big success stories of corporate boycotts recently (they defeated the labor movement, and they know how to diffuse grassroots protest too). But it's American tradition: the Boston Tea Party was a boycott of sorts by colonial merchants getting squeezed by the East India Company. So going back to B.F., unfortunately there won't be boycotts because some customers will likely "cross the picket line," lusting after deals.

I am not that familiar with the JCP case, but didn't Johnson try to make the company more upscale and resist coupons/discounts? Obviously that didn't work and they are now making a big reversal, but JCP is not a leading retailer (it's $12B annual US revenue is 16% the size of Target's). So I don't think they are driving B.F. trends, but they probably will be gone in 3 years as you said (Sears too). However, predictions of retail's demise could be exaggerated (or at least premature). In the Bay we are surrounded with innovation, disruption, and e-commerce, so that could bias our perspective, because US e-commerce is still only 6% of total retail sales in 2013 (and the biggest category is travel, because it is expensive and there isn't really a retail equivalent after the death of the travel agency). Goliath Amazon still has less revenue than Target. Shopping is still a communal, visceral experience that e-commerce and social media haven't fully supplanted yet (some fun, scientific reads on the subject if you're interested: link). Even if Amazon develops cost-effective same-day delivery, many people will still go to the big box or the mall because it offers a value proposition that online doesn't exactly match. Yes brick-and-mortar retail is inefficient with its labor, rent, and overhead, but customers still demand it.  

http://ycharts.com/indicators/ecommerce_sales_as_percent_retail_sales
http://www.statista.com/statistics/191145/travel-e-commerce-sales-in-the-united-states-since-2002/
http://trends.e-strategyblog.com/2013/04/25/american-retail-ecommerce-sales-by-product-category/10687

------

I'm sure some of you saw this too, but Black Friday isn't even a good predictor of holiday sales: 

http://www.theatlantic.com/business/archive/2013/11/the-economic-case-for-ignoring-black-friday/281926/

So the whole thing is kind of a waste of time from the perspective of the economy; people are going to spend what they spend over the holiday period, and black friday can only take from that pool, not get consumers to spend more.

------

Thx, A. I agree, B.F. just accelerates and concentrates purchases, but then many folks will run out of cash/energy afterward and not shop much more for the rest of the season (like a hangover). If companies want to keep pouring gasoline on the fire and make B.F. more and more of a ludicrous spectacle where the worst of human nature and American "culture" is on display, then I wish gov'ts would pass laws requiring retailers to fit the bill for the extra police, paramedics, and others who have to work overtime, manage the crowds, and respond to shopping-related incidents. Maybe that would change their P&L calculus. 

Is Black Friday even worth it to our society/economy?

http://news.yahoo.com/blogs/sideshow/-walmartfights-sparks-a-trend-on-twitter-193622540.html
http://en.wikipedia.org/wiki/Black_Friday_%28shopping%29

As we know, scheduled promotions like "Cash for Clunkers" many not generate new transactions, but merely shift them to a specific date (i.e. I would have bought a TV for Uncle Bob at some point before Xmas, but I might as well go on B.F.). Sure, while you are frantically trekking through Walmart, the mob mentality and "holiday spirit" make you less responsible with the wallet, so you might be inspired to get some other items (which is incremental revenue), but it's really hard for businesses to measure if those purchases are totally due to B.F. With the "arms race" of fierce retail competition, and marketers whipping consumers into a frenzy (and now mobile broadband and social media make it even easier to learn about "great deals"), companies have to take a loss or break even on certain products just to earn some eyeballs (they likely will have to pay more for ads due to more competition). Of course they may only have 5 super-cheap TVs per store, but clearly they are pricing competitive items lower than they'd ideally want to, and likely below what the customer's normal willingness-to-pay is (sans B.F. expectations).

That is a risk too - customers are expecting great deals, so if they are sold out by the time they get to the shelves (even for folks who waited for hours in the cold), that will cause a lot of anger, and the retailer runs the risk of creating a detractor who badmouths them on social media. And that's if they're lucky - shortages may lead to riots and deaths, as we've unfortunately seen in the past. Imagine all the man-hours of extra associates, police, and other emergency responders needed to handle the B.F. madness and possible incidents. Imagine the carbon footprint of all the traffic jams and the utilities from the extra hours that stores are open (and it's getting earlier and earlier - some stores opened at 8PM ON THANKSGIVING... by 2020 Black Friday will be on Labor Day). B.F. may not drive many purely incremental sales, but it sure as heck will create many incremental traffic accidents, injuries, etc. The employees that have to deal with the insanity may get overtime pay, but they also will experience elevated stress and possibly employer resentment, which are documented to affect workplace productivity, personal relationships, and health (not to mention all the stress and panic behind the scenes where non-store employees have to rush to meet the B.F. related logistics, marketing, product launches, and other deadlines). I am pretty sure some folks at Sony and Microsoft would have liked a few more weeks to iron out the bugs associated with their new gaming consoles, but they had to get them on the shelves in time for B.F. And as far as I know, no one is getting B.F. or Xmas bonuses anymore for their extra efforts.

On the consumer side, the post-Great-Recession US shopper spends on avg. $400 during B.F. weekend (with 200M+ unique shoppers). People may not behave very rationally during B.F. (understatement of the year), so they may make inefficient and imprudent purchasing decisions that will have negative consequences on their economic viability (and that of their dependents). There are socioeconomic costs associated with frivolous spending: credit default, back taxes, late rent payments, etc. Some opportunists make money off things like that, but there will be preventable economic losses too. Also, the more we spend on shopping, the less we have to donate to charities (or even buy more medicine and healthier food). The fatigue, emotions, and sleep deprivation associated with B.F. shopping will affect shopper health, relationships, and productivity for weeks - especially during the winter time of bad weather and virus spreads. Adding all this up, I can imagine that the social costs of B.F. could easily top $1B in the US. I am not sure how much incremental profit B.F. generates (for scale, wiki says WM gets $5B from B.F., and it is ~3% of all US retail sales), but clearly it should be discounted if we want to make a fair, holistic value calculation.

Even Cyber Monday could have social costs by reducing workplace productivity and tying up bandwidth that could make pageload/latency times worse and important/emergency messages slower.

"Toxic Hot Seat" about cigarettes, flame retardants, and death for profits

Toxic Hot Seat:

http://www.huffingtonpost.com/marcia-g-yerman/toxic-hot-seat-ignites-aw_b_4338572.html

This is a clear example of Pope Francis' principle of greed-driven "murder", for lack of a better term. Some companies and trade groups decided to protect their profits rather than do the right thing for their customers. And this has created enormous social costs for everyone.

The storyline:

- Since the '70s, 2-6K Americans died each year from residential fires.

- By far, the biggest driver of these tragedies are unmonitored cigarettes (yet another way they kill users and bystanders), and this trend persists in most nations.

- A self-extinguishing cigarette is affordable and easily implementable, but the tobacco industry resisted, and created a "fire marshal" advocacy org to convince leaders and public that the problem was the "fuel" (household flammable stuff), not the ignition.

- So the chemical industry jumped on this and developed "flame retardants" that could be sprayed on curtains, furniture cushions, etc. (this was during the era of DDT and such where toxicity testing was nonexistent).

- In 1975, a professional study was warped by some CA regulators and lobbyists to mandate all furniture sold in the state to contain fire retardants with no health risk conditions. Since CA was such a big market, and it was expensive to develop 2 versions of furniture, manufacturers decided to put retardants on everything, and the standard has stuck.

- Despite a huge "astroturf" (fake grassroots) chemical industry push, Maine was able to ban retardants in favor of safer alternatives. The lobbying machine has prevailed so far in CA, and Representative Leno has sponsored several similar bills to ban them, but they failed each time due to the irrational fire scare and industry advocacy. Changing the law is part of Gov. Brown's current agenda, and so far he was able to at least permit the sale of furniture that doesn't contain retardants. It's up to consumers to check whether the products they buy have them or not, but an outright ban like Maine is still elusive.

Commentary:

So what are the consequences of big tobacco's greed/negligence and big chemical's opportunism (now a $5B global industry)? We know that preventing/reducing fire severity is important and can save lives/money. But are these retardants the most cost-effective solution? Studies show that smoke alarms and sprinklers are much more effective than retardants - with nearly zero downside. Only the retardant manufacturer association has produced a few suspect studies to support the use of their product. But fires are scary and their arguments won the day for decades ("whatever it takes" to prevent fires!). Fire danger is a lot more tangible and acute than nearly invisible carcinogens that may need extended exposure to do measurable harm (but no less bodily harm than burns). So it's easy to fixate on the fire risk, which may trump other concerns. 

It's one thing if retardants are ~90% effective (at least as effective as condoms), but they're not. The flame retardant standards state that the material must be able to resist a "small flame" for 12 secs. While that could be effective in some situations, it is deficient in many tests and in the field, because a furniture's unprotected covering burns first, so by the time it hits the cushions (that contain the retardants), it's no longer a small flame and the retardants are no longer effective.

So they don't really make us safer, yet they are likely making us sicker. Retardants are known carcinogens and mutagens like thalates, bromides, and BPA (that the baby industry has been forced to remove from plastic products due to customer anger). Retardant-containing products still burn, so when firefighters have to respond, they are rushing into and inhaling the chemical soup. When SFFD personnel were tested, incidence of middle-age female breast cancer was 6X the population average, and incidence of cancer among retirees was also unusually high. Parents groups fought to get these chemicals banned in child pajamas (on toxic grounds) and won, yet the exact same chemicals are still present in child car seats, play pens, backpacks, furniture, etc. that kids touch and lick. They're still present in those products because it's expensive to mount a legal challenge, and chemicals get the benefit of the doubt.

Speaking of that, why do virtually all side effects, interactions, allowable dosages, etc. need to be thoroughly documented and scrutinized for a drug to reach the market, but chemicals are "innocent until proven guilty"? It's about the burden of proof. As depicted in films like "A Civil Action", it is very hard to prove that a specific chemical directly caused measurable harm over what could be years of exposure to many chemicals. Companies can just show that rats "survived" when exposed to the chemical under specific, arbitrary conditions - and that is supposed to prove that they are universally safe.

And when pro-business leaders continually weaken and defund the EPA and other regulators, that makes them look ineffective/unnecessary and strengthens the industry argument to just trust the companies, not delay new product approval with frivolous tests, and not stifle chemical innovation, which "creates jobs/revenue" and "makes our lives better". And this is not just about retardants, but literally thousands of chemicals that we don't even know we're exposed to each day. Like all those "dispersants" used by BP to make the Gulf spill look less nasty to the naked eye - we have no idea WTF they do to living tissues over time, and they will likely show up in seafood, breast milk, etc.

There is an underlying assumption that benevolent gov't is watching out for us, and products wouldn't be on the shelf unless they were totally safe (and Dow and Monsanto say you can trust them). Well after leaded gas, asbestos, CFCs, and the sad history of tobacco, we should know better. The only ones who are looking after public safety are scientists, survivor victims, and grassroots orgs. And politicians will only listen to them if voter anger outweighs industry lobbying dollars. But this is obviously short-sighted leadership. It makes no sense to threaten and poison future society's health and productivity for present-day industry profits. Taxpayers and gov't coffers will be burdened by the health costs of harmful chemicals, which means less money for other national priorities. Everyone loses but the companies and their stakeholders.

The same thing is playing out in places like China with pathetic regulation, but they recognize this and are trying to improve. Can you imagine the impact on their economy when their current population ages and disproportionately develops all sorts of illnesses from the ubiquitous pollution, toxins, etc.?

And it's not like these "innovative chemicals" are so critical to human survival. We can get along fine with many natural, renewable products like wool, plants, and wax. We don't need chemical X to make our jacket down 0.1% warmer but our kids 10% sicker. Someone has to say enough is enough.

Obamacare fiasco discussion

If you thought Obamacare was bad...

As usual, China is way worse.

http://m.npr.org/programs/all/2/242344329

----

Part of another conversation I had about the recent Obamacare troubles... would love to hear your thoughts if you have time.

http://talkingpointsmemo.com/dc/obamacare-fix-what-is-it-good-for

Yeah, I have been really saddened and disappointed by how the Obama admin. has rolled out heathcare.gov. When you know the GOP is looking for any little excuse to scuttle the program, you'd think they would be extra careful every step of the way! It seriously looks like it was put together in a week by interns. I know health care is complex as hell, with different laws in different states, so then maybe they should have fought harder for single payer from the start! And as you suggested, if this is a wacky attempt to convince America that single payer is the way to go, it probably won't worry either since trust in gov't is so low. Well, even if Obamacare fails and is repealed by President Christie (LOL), I think it got the health-industrial complex to pay attention and realize that Americans (and even some of our leaders) are tired of being gouged by a broken system that under-delivers on health outcomes for all the money it takes in. Companies/doctors are being more cautious and not just assuming they can raise prices, over-prescribe treatments, cherry pick "profitable" patients, and bilk Medicare and get away with it every time. So I guess that is some progress?

I agree with you about the misrepresentation. That is the problem when passing the bill becomes the goal rather than actually helping citizens. But unfortunately Obama has not been a very transparent president, despite his claims. I think that is a reflection of his and his admin's poor communication skills. While Obama is an inspirational campaign speaker on the abstract level, he is fairly poor in terms of leadership and on the practical policy level, as we saw during the Romney debates. He gets too academic and technical, and doesn't empathize with others. Most of us barely understand our own health care, much less gov't policy, and it can be an emotional issue. Obama relied too much on logic/macroeconomics, and didn't do enough to put people's fears and concerns to rest (especially in the face of the GOP lie machine). Unlike Clinton, Obama never "felt our pain." So there wasn't much trust from the get-go, which was reflected in Obamacare's persistent low approval numbers among the public (despite high public support for the individual parts of the law, like covering kids until 25, no rejections for pre-existing conditions, etc.). So if he communicated better from the start, I think the public could be a lot more forgiving of the glitches now. Customers mostly forgave the iPhone 4 for it's "death grip" dropped calls problem, because they loved the company and were addicted to the product. ;)

Speaking of the implementation, it is seriously his Bay of Pigs. Like Kennedy, I bet Obama was too trusting of his people that they would handle it properly, and wasn't skeptical enough. He did go on the national PR tour to drum up support and interest, but he didn't really manage expectations and prepare the country for what could go wrong. In 2013, there is just no excuse for a website that works so poorly. Testing and pilot sessions with real customers were severely lacking. I haven't had time to follow the media discussion, so I am not sure if the pundits and Sebelius already covered these points. But they had ample time to prepare for the website launch, so I don't know why their product is so crappy now.

In general, I think a website is supposed to launch small at first to mitigate risk (like 5% of expected peak traffic) and gradually ramp up as long as things are working. I am not sure, but did healthcare.gov suddenly turn on for the whole country all at once? That is a bad strategy. Maybe try it out with ISPs from a couple small states first, check the performance, collect feedback, and make necessary fixes before expanding the reach. It would have let them catch bugs (or maybe delay the 100% launch) before it became a national embarrassment with thousands of frustrated users. I heard an alternate rollout plan was first to make sure the site was stable and let users browse plans only (to make sure the info they got was accurate too!), then when registration/enrollment was ready and tested, they would add that functionality later. But Obama's people worried that users would browse plans, not like any, and never come back (if you are so worried about that, then maybe your product is not so appealing and you should fix that first!), so they wanted to require registration prior to browsing. There was also the political pressure to get as many people enrolled as possible during the initial launch period, so they required visitors to register up front. But that is bad for several reasons: web users often mess up the registration process, and if you mandate it at launch, you introduce a bottleneck and they will flood your customer support line all at once. Also registration is annoying, so that will cause many users to just drop off too (if they can even get the page to load). Plus let's remember that many uninsured Americans or those who buy individual plans could be lower income and less tech savvy, even if they have a helper to assist.

On the issue of the "deception" over keeping your old plan and old doctor, I have mixed feelings about it. On one hand, if Obama knows that a major argument against Obamacare is a supposed loss of freedom, then "forcing" people to change plans/doctors (even to ostensibly better ones) is risky and plays into the GOP hand. But if they need to eliminate ineligible plans to make the whole system work, then I understand, but that is a major program risk that should have been addressed years ago. The doctor-patient relationship is sensitive and almost sacred, so if Obamacare really does prevent some people from keeping their old doctors (even if they are crappy or charge too much), then that is unacceptable to me. I think you can convince people to let the gov't upgrade their coverage (but again, Obama's people did a terrible communication job), but change is hard for some folks and you can't just drop their coverage suddenly and tell them to choose a new plan on a buggy website. All this mess plays into the conservative narrative that gov't is totally meddling in our lives and inept in providing services (which is unfortunately true at times). The 'fix' from the first link above is too little, too late. Maybe a better plan would be what companies do for new hires. They default you into a basic health plan, and you have the option to pick a different one within 60 days. At least then everyone is sure they are covered from the start, and there is no rush/panic/anger. Maybe that is impossible nationally, since health plans and laws are different in each state (another issue we probably need to address)?

http://www.washingtonpost.com/blogs/plum-line/wp/2013/11/13/what-the-obamacare-enrollment-numbers-really-tell-us/?wprss=rss_homepage&clsrd

The SF Bay Area housing crisis

http://blogs.kqed.org/newsfix/2013/10/30/kqed-launches-priced-out-the-high-cost-of-housing-in-the-bay-area/

This is probably he most important social issue for the area. It's strange that SF has a proud history of inclusion (and a pretty good record in overall today), but this housing crisis is driven by exclusion and privilege, not necessarily "normal economics."

-----

Well if you average the entire Bay Area, places like Brentwood and Antioch will bring down the average. But SF is #3 priciest per sq. ft. in the US behind NYC and HNL. Versus the whole world, SF is not that bad, but real estate is really messed up in older, more cramped cities in the EU and Asia.

http://curbed.com/archives/2011/09/27/heres-a-chart-of-the-worlds-cities-by-price-per-square-foot.php

It's not even about rent vs. own. Renting is very pricey too, obviously. I think one aspect of the problem is that landlords are taking rent-controlled (or regular) units off the market in order to convert them to TIC/condos for more $, or sell the whole bldg to speculators because there is no much insane pent-up demand and high willingness to pay from the upper classes. So rental supply is going down, which causes rents to rise too.

----

If you hit Wikipedia for median income by city in ca you'll find that those areas are also quite rich.  Housing doesn't happen in a vacuum.  More a function of income and the real problem is income inequality

It also happens with respect to density. The more houses/apartments available = the cheaper the rent. Nimbys in the Bay Area would like to still have their .5 acre ranch houses on some of the most valuable land on earth, meaning that the poors have to live 2 hours commute from their jobs.

-----

Totally agree. The social justice element also comes into play that historically lower income, blue collar, often minority neighborhoods (Mission, Castro, etc.) used to be less desirable for yuppies and therefore more affordable and off the radar of speculators. But now they are hipster paradises, and with gentrification, the "traditional residents" can't even maintain their old lives and are priced out of the market. I guess that is like economic eminent domain.

Measures like rent control or mandated affordable housing are often economically inefficient and flawed (because they are written and implemented by flawed bureaucrats, like healthcare.gov), but sometime optimal economics must take a back seat to concerns about fairness and social values (see taxes, labor laws). I'd rather have confusing rent control laws and too high minimum wage than a total lack of regulation (at least imperfect laws are a springboard for iteration and improvement). Because as you said, with the huge inequality gap, what is the point for the poorest 60% to struggle so hard trying to fit into the rich man's world (yet not really be allowed to live in the rich man's world... I guess like the "Elysium" plot, which I did not see)? Most people with a shred of heart would probably agree that there is something inherently unjust with greedy politicians/companies and wealthy invaders "annexing" and developing suddenly desirable land, which leads to the economic eviction of the residents who were there before (through legal market mechanisms and the landlords/property owners). Pardon my melodrama, but it's scarily similar to the Trail of Tears.

I heard an interview about the subject a while back, and one new SF resident and tech employee said something like, "Assholes (his word) like me can afford $3K/1BR rent and are making it hard on the previous residents, but hey, I want to live here." At least he had clear eyes. It's just sad because everyone has one life to live, and we don't want to compromise or suffer if we don't have to - even at cost to others (esp. when others' suffering at our hands is mostly invisible). Sorry for stereotyping, but I think the Gen X-ers and Millennials especially (present company excluded) really fixate on what I like to call "life maximization." Similar to their work habits of optimizing, achieving, and disrupting everything, in life they want it their way and they want it all. Best job, best home, best network, best gear, best marathon time, best family... best, best, best. When that is not really the goal but the status quo expectation in SF/Si Valley (when excellent becomes average/normal, what do all the sub-excellent people do?), then that doesn't really encourage a culture of togetherness and SHARING. We don't need the best life... can't we just enjoy our regular life? Isn't enough enough? Can I give a little up and still live a plenty comfy life so that more needy people can get a break? People don't ask these questions of themselves enough (myself included).

Sharing is the key I think. This isn't my world, it's our world. There is PLENTY of food, money, and room for everyone if we share reasonably, but the problem is the fucking 1% and the institutions who advocate for them just don't want to (and don't have to). Americans and some other cultures really fixate on fencing off what's yours and amassing/diving the pie, so that breeds an adversarial, zero-sum attitude that conservatives really eat up. Therefore, it's especially sad when educated, young, open-minded "liberal" West Coast people do the same and may not even realize it. At least liberals don't bad-mouth and hate the people they marginalize, but that doesn't leave them in the clear (myself included).

The best teachers I have had in my life (not necessarily in school) really cared about expanding the pie rather than dividing it. How can we make everyone happier? Any asshole with a bit of smarts can compete and beat others. But a truly smart, wise person goes out of his/her way to cooperate, find ways to align incentives, and make everyone better off - not just him/herself. For all the smug talent and genius brains and big money in the Bay, that skill is scarily absent. Don't get me wrong, some people practice it faithfully and I am in awe of them, but it's not enough to stem the tide. That's why I probably will leave this place next year, as much as it makes me sad and for all I am leaving behind. I don't want my kid to grow up in such a culture. At least I will make room for another person/family to have my stressful job and little slice of condo.

-----

Sorry I realize that I was unclear about one point in my last rant (unclear about at least one point, haha).

The Google buses are often scapegoats but not really the problem. By no means am I placing the bulk of the blame on the young, techie hipsters invading the Bay. They are not really the 1% and not the real driver of the problem, they are just a "symptom". Except for Zuck who bought a whole block of SF for privacy haha.

As usual, the problem is the old rich Nimby fucks that A alluded to. They are the ones who gobbled up all the choice land decades ago, and who currently occupy all the seats of political power and business influence. They are the ones who invested/profited from the tech boom (more than the current workforce), and stand to profit from real estate development and appreciation. They took so much that the current hipsters are practically forced to gentrify the Mission and maybe Hunter's Point soon. I bet most hipsters would much rather live in Nob Hill or downtown Paly, but all the old rich pricks there are just cramping their style anyway (and even a Twitter engineer still can't afford a $5M Nob Hill pad).

As A alluded to, we are not in the '50s Levittowns anymore, or not even in the '90s nouveau-riche gated golf communities. But the assholes want to keep us in the past, because they like the status and lifestyle they have, and don't want to give up even a shred to others. The city of Paris (and many other ancient cities) is like a 10-layer cake. New generations tear down the old crap that isn't working anymore and rebuild to fit the needs of the current people. We aren't really doing that, because the old fossils control the bulldozers. So all we have is SF Elysium vs. Vallejo. 

But "the people" are fighting back, and at least voted down a luxury condo development on the Embarcadero. It's a small victory in a long war that the good guys will almost certainly lose. But we might as well smoke a blunt and celebrate while we can. :)

http://www.sfgate.com/bayarea/nevius/article/Following-the-bankroll-for-SF-s-Propositions-B-C-4961999.php
http://www.npr.org/templates/story/story.php?storyId=243537913&ft=1&f=2&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+NprProgramsATC+%28NPR+Programs%3A+All+Things+Considered%29&utm_content=Yahoo+Search+Results 

-----

i once heard someone say cities are either places to live or places people come to make money.  he was talking about SF and that we were at a crossroads.  it was when matt gonzalez was running for mayor against gavin Newsome.  The speaker was making the case that if Newsome wins, the city will lose its soul, it's artist community, it's diversity of culture and class.  that's what you're looking at here….plain and simple.  and, you know what?  the city sucks now.  It's filled with people who are in a hurry, who lay on their horns, who yell.