More on Black Friday

A friend just sent this: http://online.wsj.com/news/articles/SB10001424052702304281004579217863262940166

So folks are stressing and fighting over what is likely phantom discounts anyway.

After working at a discount e-commerce site, I know that some consumers care more about the "% discount", rather than the final price (or even the actual use of the product!). So they fall into the trap. It's probably better to know what max price you will pay for item X, and just pick the nicest one that qualifies. The MSRP may suggest relative quality, but that's about it. My econ prof said that $, not %, counts in the end; a cheap toy may be 80% off, but if that only means $3 more in your pocket, it's really not that impressive. And of course it depends if the full price is reasonable or artificially marked up as the article said. But the majority of consumers are not that econ-finance savvy, and probably don't consider these factors in the heat of the moment.

I just fear that as long as real wages stagnate, the wealth gap widens, and purchasing power wanes (due to higher housing, medical, education, & energy costs), people will find it harder and harder to maintain or increase their consumption (despite all the marketing messages and social pressure to do so). Robert Reich said that these trends have been around since the 1980s, but the US middle class made up for it with home equity and consumer credit (to the benefit of Wall St.). Now that loans are getting harder to come by, and of course curbing consumption is out of the question, the stakes become even higher to secure "deals". That could change the shopping psychology and make B.F. pandemonium and conflict more likely and serious going forward. It's like NFL stadium violence - is it any surprise that it is likely correlated with soaring ticket prices and overall "cost of fandom"? When you squeeze and raise the stakes on people, of course they are going to react badly when they feel like they are not getting their money's worth.

-----

Happy Thanksgiving dude :)

It's always kind of baffling to me seeing these kind of "is X good for society" emails, because ultimately there is no benevolent overlord responsible for arbitrating this kind of thing. It's just ... game theory, lots of individuals trying to maximize their own whatever (sometimes profit, sometimes happiness, sometimes self-satisfaction).

Black Friday is an interesting concept. My basic mental model is that initially it was just a naturally-big shopping day, and (given retailers' low margins) the first day that retailers showed a profit for the year. Then retailers started competing for shoppers, and because their main method of competition was "sales" ... here we are.

Obviously you know that sales and coupons were the old-school form of price discrimination, before the internet and quants like you figuring out how to get every dime out of the customer ;) But now it's just old-school prisoner's dilemma, with every retailer defecting and offering sales. This year they're especially screwed, because JCP is desperate to sell off the old Ron Johnson-era merch and is undercutting everyone. It's straight-up race to the bottom, where half these guys will be out of business in 3 years and the rest will be lucky to survive 10. 

They're not doing it because it's good for the country or the culture. They're doing it because, like a heroin addict that hasn't gotten a hit in a while, momma needs that holiday shopping boost, and they'll do *whatever* it takes to get it. Some trickery, some desperation, but this is what giant corporations look like when they die. Like those images of a dinosaur trapped in a tar pit, trying to get out, every struggle dragging it deeper into the black ooze.

What we're seeing is the end of retail, and this and the next few Black Fridays are going to be ground zero for that shit. Loot what you can :)

-----

Thx J, same to you.

Even though B.F. is not a top-down, command economy program, I still think it is valid to evaluate the consequences. If we don't do that, then how can we ever know if a change is needed? Yes, game theory can accurately describe and predict players' actions given a set of conditions, but to me that kind of abstracts the situation as if we were just "rats in a maze". B.F. is about more than transactions. I wanted to consider the human side and social costs, because it's easy to take them for granted when all we see are the cool items, big crowds, and blowout prices in the media. As you said, if these dinosaur retailers are caught in a suicide spiral race-to-the-bottom, then the poor decisions that contributed to that situation will eventually hurt millions of shareholders, employees, and society as a whole (less tax collected, more pressure on social services).

Negative industry trends aside, is B.F. the best method for buyers and sellers to achieve their seasonal goals? As recently as the 1990s, B.F. wasn't even the heaviest US shopping day (that was reserved for the Sat before Xmas, for gift procrastinators LOL). Some limits were still respected. But as you said, companies are locked in a fierce rivalry, and some "innovators" thought to exploit B.F. more. I wish customers would push back and say, "I don't care how big your sale is. The holidays are for spending time with loved ones and being good to one another, not fighting for a spot in line at 3AM just to get an Xbox. It's not right to encourage rabid consumerism and make your employees work these crazy hours either." If we collectively protest and boycott, change will come, like apartheid in SA and cigarettes in the US. Unfortunately there haven't been many big success stories of corporate boycotts recently (they defeated the labor movement, and they know how to diffuse grassroots protest too). But it's American tradition: the Boston Tea Party was a boycott of sorts by colonial merchants getting squeezed by the East India Company. So going back to B.F., unfortunately there won't be boycotts because some customers will likely "cross the picket line," lusting after deals.

I am not that familiar with the JCP case, but didn't Johnson try to make the company more upscale and resist coupons/discounts? Obviously that didn't work and they are now making a big reversal, but JCP is not a leading retailer (it's $12B annual US revenue is 16% the size of Target's). So I don't think they are driving B.F. trends, but they probably will be gone in 3 years as you said (Sears too). However, predictions of retail's demise could be exaggerated (or at least premature). In the Bay we are surrounded with innovation, disruption, and e-commerce, so that could bias our perspective, because US e-commerce is still only 6% of total retail sales in 2013 (and the biggest category is travel, because it is expensive and there isn't really a retail equivalent after the death of the travel agency). Goliath Amazon still has less revenue than Target. Shopping is still a communal, visceral experience that e-commerce and social media haven't fully supplanted yet (some fun, scientific reads on the subject if you're interested: link). Even if Amazon develops cost-effective same-day delivery, many people will still go to the big box or the mall because it offers a value proposition that online doesn't exactly match. Yes brick-and-mortar retail is inefficient with its labor, rent, and overhead, but customers still demand it.  

http://ycharts.com/indicators/ecommerce_sales_as_percent_retail_sales
http://www.statista.com/statistics/191145/travel-e-commerce-sales-in-the-united-states-since-2002/
http://trends.e-strategyblog.com/2013/04/25/american-retail-ecommerce-sales-by-product-category/10687

------

I'm sure some of you saw this too, but Black Friday isn't even a good predictor of holiday sales: 

http://www.theatlantic.com/business/archive/2013/11/the-economic-case-for-ignoring-black-friday/281926/

So the whole thing is kind of a waste of time from the perspective of the economy; people are going to spend what they spend over the holiday period, and black friday can only take from that pool, not get consumers to spend more.

------

Thx, A. I agree, B.F. just accelerates and concentrates purchases, but then many folks will run out of cash/energy afterward and not shop much more for the rest of the season (like a hangover). If companies want to keep pouring gasoline on the fire and make B.F. more and more of a ludicrous spectacle where the worst of human nature and American "culture" is on display, then I wish gov'ts would pass laws requiring retailers to fit the bill for the extra police, paramedics, and others who have to work overtime, manage the crowds, and respond to shopping-related incidents. Maybe that would change their P&L calculus. 

Is Black Friday even worth it to our society/economy?

http://news.yahoo.com/blogs/sideshow/-walmartfights-sparks-a-trend-on-twitter-193622540.html
http://en.wikipedia.org/wiki/Black_Friday_%28shopping%29

As we know, scheduled promotions like "Cash for Clunkers" many not generate new transactions, but merely shift them to a specific date (i.e. I would have bought a TV for Uncle Bob at some point before Xmas, but I might as well go on B.F.). Sure, while you are frantically trekking through Walmart, the mob mentality and "holiday spirit" make you less responsible with the wallet, so you might be inspired to get some other items (which is incremental revenue), but it's really hard for businesses to measure if those purchases are totally due to B.F. With the "arms race" of fierce retail competition, and marketers whipping consumers into a frenzy (and now mobile broadband and social media make it even easier to learn about "great deals"), companies have to take a loss or break even on certain products just to earn some eyeballs (they likely will have to pay more for ads due to more competition). Of course they may only have 5 super-cheap TVs per store, but clearly they are pricing competitive items lower than they'd ideally want to, and likely below what the customer's normal willingness-to-pay is (sans B.F. expectations).

That is a risk too - customers are expecting great deals, so if they are sold out by the time they get to the shelves (even for folks who waited for hours in the cold), that will cause a lot of anger, and the retailer runs the risk of creating a detractor who badmouths them on social media. And that's if they're lucky - shortages may lead to riots and deaths, as we've unfortunately seen in the past. Imagine all the man-hours of extra associates, police, and other emergency responders needed to handle the B.F. madness and possible incidents. Imagine the carbon footprint of all the traffic jams and the utilities from the extra hours that stores are open (and it's getting earlier and earlier - some stores opened at 8PM ON THANKSGIVING... by 2020 Black Friday will be on Labor Day). B.F. may not drive many purely incremental sales, but it sure as heck will create many incremental traffic accidents, injuries, etc. The employees that have to deal with the insanity may get overtime pay, but they also will experience elevated stress and possibly employer resentment, which are documented to affect workplace productivity, personal relationships, and health (not to mention all the stress and panic behind the scenes where non-store employees have to rush to meet the B.F. related logistics, marketing, product launches, and other deadlines). I am pretty sure some folks at Sony and Microsoft would have liked a few more weeks to iron out the bugs associated with their new gaming consoles, but they had to get them on the shelves in time for B.F. And as far as I know, no one is getting B.F. or Xmas bonuses anymore for their extra efforts.

On the consumer side, the post-Great-Recession US shopper spends on avg. $400 during B.F. weekend (with 200M+ unique shoppers). People may not behave very rationally during B.F. (understatement of the year), so they may make inefficient and imprudent purchasing decisions that will have negative consequences on their economic viability (and that of their dependents). There are socioeconomic costs associated with frivolous spending: credit default, back taxes, late rent payments, etc. Some opportunists make money off things like that, but there will be preventable economic losses too. Also, the more we spend on shopping, the less we have to donate to charities (or even buy more medicine and healthier food). The fatigue, emotions, and sleep deprivation associated with B.F. shopping will affect shopper health, relationships, and productivity for weeks - especially during the winter time of bad weather and virus spreads. Adding all this up, I can imagine that the social costs of B.F. could easily top $1B in the US. I am not sure how much incremental profit B.F. generates (for scale, wiki says WM gets $5B from B.F., and it is ~3% of all US retail sales), but clearly it should be discounted if we want to make a fair, holistic value calculation.

Even Cyber Monday could have social costs by reducing workplace productivity and tying up bandwidth that could make pageload/latency times worse and important/emergency messages slower.

"Toxic Hot Seat" about cigarettes, flame retardants, and death for profits

Toxic Hot Seat:

http://www.huffingtonpost.com/marcia-g-yerman/toxic-hot-seat-ignites-aw_b_4338572.html

This is a clear example of Pope Francis' principle of greed-driven "murder", for lack of a better term. Some companies and trade groups decided to protect their profits rather than do the right thing for their customers. And this has created enormous social costs for everyone.

The storyline:

- Since the '70s, 2-6K Americans died each year from residential fires.

- By far, the biggest driver of these tragedies are unmonitored cigarettes (yet another way they kill users and bystanders), and this trend persists in most nations.

- A self-extinguishing cigarette is affordable and easily implementable, but the tobacco industry resisted, and created a "fire marshal" advocacy org to convince leaders and public that the problem was the "fuel" (household flammable stuff), not the ignition.

- So the chemical industry jumped on this and developed "flame retardants" that could be sprayed on curtains, furniture cushions, etc. (this was during the era of DDT and such where toxicity testing was nonexistent).

- In 1975, a professional study was warped by some CA regulators and lobbyists to mandate all furniture sold in the state to contain fire retardants with no health risk conditions. Since CA was such a big market, and it was expensive to develop 2 versions of furniture, manufacturers decided to put retardants on everything, and the standard has stuck.

- Despite a huge "astroturf" (fake grassroots) chemical industry push, Maine was able to ban retardants in favor of safer alternatives. The lobbying machine has prevailed so far in CA, and Representative Leno has sponsored several similar bills to ban them, but they failed each time due to the irrational fire scare and industry advocacy. Changing the law is part of Gov. Brown's current agenda, and so far he was able to at least permit the sale of furniture that doesn't contain retardants. It's up to consumers to check whether the products they buy have them or not, but an outright ban like Maine is still elusive.

Commentary:

So what are the consequences of big tobacco's greed/negligence and big chemical's opportunism (now a $5B global industry)? We know that preventing/reducing fire severity is important and can save lives/money. But are these retardants the most cost-effective solution? Studies show that smoke alarms and sprinklers are much more effective than retardants - with nearly zero downside. Only the retardant manufacturer association has produced a few suspect studies to support the use of their product. But fires are scary and their arguments won the day for decades ("whatever it takes" to prevent fires!). Fire danger is a lot more tangible and acute than nearly invisible carcinogens that may need extended exposure to do measurable harm (but no less bodily harm than burns). So it's easy to fixate on the fire risk, which may trump other concerns. 

It's one thing if retardants are ~90% effective (at least as effective as condoms), but they're not. The flame retardant standards state that the material must be able to resist a "small flame" for 12 secs. While that could be effective in some situations, it is deficient in many tests and in the field, because a furniture's unprotected covering burns first, so by the time it hits the cushions (that contain the retardants), it's no longer a small flame and the retardants are no longer effective.

So they don't really make us safer, yet they are likely making us sicker. Retardants are known carcinogens and mutagens like thalates, bromides, and BPA (that the baby industry has been forced to remove from plastic products due to customer anger). Retardant-containing products still burn, so when firefighters have to respond, they are rushing into and inhaling the chemical soup. When SFFD personnel were tested, incidence of middle-age female breast cancer was 6X the population average, and incidence of cancer among retirees was also unusually high. Parents groups fought to get these chemicals banned in child pajamas (on toxic grounds) and won, yet the exact same chemicals are still present in child car seats, play pens, backpacks, furniture, etc. that kids touch and lick. They're still present in those products because it's expensive to mount a legal challenge, and chemicals get the benefit of the doubt.

Speaking of that, why do virtually all side effects, interactions, allowable dosages, etc. need to be thoroughly documented and scrutinized for a drug to reach the market, but chemicals are "innocent until proven guilty"? It's about the burden of proof. As depicted in films like "A Civil Action", it is very hard to prove that a specific chemical directly caused measurable harm over what could be years of exposure to many chemicals. Companies can just show that rats "survived" when exposed to the chemical under specific, arbitrary conditions - and that is supposed to prove that they are universally safe.

And when pro-business leaders continually weaken and defund the EPA and other regulators, that makes them look ineffective/unnecessary and strengthens the industry argument to just trust the companies, not delay new product approval with frivolous tests, and not stifle chemical innovation, which "creates jobs/revenue" and "makes our lives better". And this is not just about retardants, but literally thousands of chemicals that we don't even know we're exposed to each day. Like all those "dispersants" used by BP to make the Gulf spill look less nasty to the naked eye - we have no idea WTF they do to living tissues over time, and they will likely show up in seafood, breast milk, etc.

There is an underlying assumption that benevolent gov't is watching out for us, and products wouldn't be on the shelf unless they were totally safe (and Dow and Monsanto say you can trust them). Well after leaded gas, asbestos, CFCs, and the sad history of tobacco, we should know better. The only ones who are looking after public safety are scientists, survivor victims, and grassroots orgs. And politicians will only listen to them if voter anger outweighs industry lobbying dollars. But this is obviously short-sighted leadership. It makes no sense to threaten and poison future society's health and productivity for present-day industry profits. Taxpayers and gov't coffers will be burdened by the health costs of harmful chemicals, which means less money for other national priorities. Everyone loses but the companies and their stakeholders.

The same thing is playing out in places like China with pathetic regulation, but they recognize this and are trying to improve. Can you imagine the impact on their economy when their current population ages and disproportionately develops all sorts of illnesses from the ubiquitous pollution, toxins, etc.?

And it's not like these "innovative chemicals" are so critical to human survival. We can get along fine with many natural, renewable products like wool, plants, and wax. We don't need chemical X to make our jacket down 0.1% warmer but our kids 10% sicker. Someone has to say enough is enough.

Obamacare fiasco discussion

If you thought Obamacare was bad...

As usual, China is way worse.

http://m.npr.org/programs/all/2/242344329

----

Part of another conversation I had about the recent Obamacare troubles... would love to hear your thoughts if you have time.

http://talkingpointsmemo.com/dc/obamacare-fix-what-is-it-good-for

Yeah, I have been really saddened and disappointed by how the Obama admin. has rolled out heathcare.gov. When you know the GOP is looking for any little excuse to scuttle the program, you'd think they would be extra careful every step of the way! It seriously looks like it was put together in a week by interns. I know health care is complex as hell, with different laws in different states, so then maybe they should have fought harder for single payer from the start! And as you suggested, if this is a wacky attempt to convince America that single payer is the way to go, it probably won't worry either since trust in gov't is so low. Well, even if Obamacare fails and is repealed by President Christie (LOL), I think it got the health-industrial complex to pay attention and realize that Americans (and even some of our leaders) are tired of being gouged by a broken system that under-delivers on health outcomes for all the money it takes in. Companies/doctors are being more cautious and not just assuming they can raise prices, over-prescribe treatments, cherry pick "profitable" patients, and bilk Medicare and get away with it every time. So I guess that is some progress?

I agree with you about the misrepresentation. That is the problem when passing the bill becomes the goal rather than actually helping citizens. But unfortunately Obama has not been a very transparent president, despite his claims. I think that is a reflection of his and his admin's poor communication skills. While Obama is an inspirational campaign speaker on the abstract level, he is fairly poor in terms of leadership and on the practical policy level, as we saw during the Romney debates. He gets too academic and technical, and doesn't empathize with others. Most of us barely understand our own health care, much less gov't policy, and it can be an emotional issue. Obama relied too much on logic/macroeconomics, and didn't do enough to put people's fears and concerns to rest (especially in the face of the GOP lie machine). Unlike Clinton, Obama never "felt our pain." So there wasn't much trust from the get-go, which was reflected in Obamacare's persistent low approval numbers among the public (despite high public support for the individual parts of the law, like covering kids until 25, no rejections for pre-existing conditions, etc.). So if he communicated better from the start, I think the public could be a lot more forgiving of the glitches now. Customers mostly forgave the iPhone 4 for it's "death grip" dropped calls problem, because they loved the company and were addicted to the product. ;)

Speaking of the implementation, it is seriously his Bay of Pigs. Like Kennedy, I bet Obama was too trusting of his people that they would handle it properly, and wasn't skeptical enough. He did go on the national PR tour to drum up support and interest, but he didn't really manage expectations and prepare the country for what could go wrong. In 2013, there is just no excuse for a website that works so poorly. Testing and pilot sessions with real customers were severely lacking. I haven't had time to follow the media discussion, so I am not sure if the pundits and Sebelius already covered these points. But they had ample time to prepare for the website launch, so I don't know why their product is so crappy now.

In general, I think a website is supposed to launch small at first to mitigate risk (like 5% of expected peak traffic) and gradually ramp up as long as things are working. I am not sure, but did healthcare.gov suddenly turn on for the whole country all at once? That is a bad strategy. Maybe try it out with ISPs from a couple small states first, check the performance, collect feedback, and make necessary fixes before expanding the reach. It would have let them catch bugs (or maybe delay the 100% launch) before it became a national embarrassment with thousands of frustrated users. I heard an alternate rollout plan was first to make sure the site was stable and let users browse plans only (to make sure the info they got was accurate too!), then when registration/enrollment was ready and tested, they would add that functionality later. But Obama's people worried that users would browse plans, not like any, and never come back (if you are so worried about that, then maybe your product is not so appealing and you should fix that first!), so they wanted to require registration prior to browsing. There was also the political pressure to get as many people enrolled as possible during the initial launch period, so they required visitors to register up front. But that is bad for several reasons: web users often mess up the registration process, and if you mandate it at launch, you introduce a bottleneck and they will flood your customer support line all at once. Also registration is annoying, so that will cause many users to just drop off too (if they can even get the page to load). Plus let's remember that many uninsured Americans or those who buy individual plans could be lower income and less tech savvy, even if they have a helper to assist.

On the issue of the "deception" over keeping your old plan and old doctor, I have mixed feelings about it. On one hand, if Obama knows that a major argument against Obamacare is a supposed loss of freedom, then "forcing" people to change plans/doctors (even to ostensibly better ones) is risky and plays into the GOP hand. But if they need to eliminate ineligible plans to make the whole system work, then I understand, but that is a major program risk that should have been addressed years ago. The doctor-patient relationship is sensitive and almost sacred, so if Obamacare really does prevent some people from keeping their old doctors (even if they are crappy or charge too much), then that is unacceptable to me. I think you can convince people to let the gov't upgrade their coverage (but again, Obama's people did a terrible communication job), but change is hard for some folks and you can't just drop their coverage suddenly and tell them to choose a new plan on a buggy website. All this mess plays into the conservative narrative that gov't is totally meddling in our lives and inept in providing services (which is unfortunately true at times). The 'fix' from the first link above is too little, too late. Maybe a better plan would be what companies do for new hires. They default you into a basic health plan, and you have the option to pick a different one within 60 days. At least then everyone is sure they are covered from the start, and there is no rush/panic/anger. Maybe that is impossible nationally, since health plans and laws are different in each state (another issue we probably need to address)?

http://www.washingtonpost.com/blogs/plum-line/wp/2013/11/13/what-the-obamacare-enrollment-numbers-really-tell-us/?wprss=rss_homepage&clsrd

The SF Bay Area housing crisis

http://blogs.kqed.org/newsfix/2013/10/30/kqed-launches-priced-out-the-high-cost-of-housing-in-the-bay-area/

This is probably he most important social issue for the area. It's strange that SF has a proud history of inclusion (and a pretty good record in overall today), but this housing crisis is driven by exclusion and privilege, not necessarily "normal economics."

-----

Well if you average the entire Bay Area, places like Brentwood and Antioch will bring down the average. But SF is #3 priciest per sq. ft. in the US behind NYC and HNL. Versus the whole world, SF is not that bad, but real estate is really messed up in older, more cramped cities in the EU and Asia.

http://curbed.com/archives/2011/09/27/heres-a-chart-of-the-worlds-cities-by-price-per-square-foot.php

It's not even about rent vs. own. Renting is very pricey too, obviously. I think one aspect of the problem is that landlords are taking rent-controlled (or regular) units off the market in order to convert them to TIC/condos for more $, or sell the whole bldg to speculators because there is no much insane pent-up demand and high willingness to pay from the upper classes. So rental supply is going down, which causes rents to rise too.

----

If you hit Wikipedia for median income by city in ca you'll find that those areas are also quite rich.  Housing doesn't happen in a vacuum.  More a function of income and the real problem is income inequality

It also happens with respect to density. The more houses/apartments available = the cheaper the rent. Nimbys in the Bay Area would like to still have their .5 acre ranch houses on some of the most valuable land on earth, meaning that the poors have to live 2 hours commute from their jobs.

-----

Totally agree. The social justice element also comes into play that historically lower income, blue collar, often minority neighborhoods (Mission, Castro, etc.) used to be less desirable for yuppies and therefore more affordable and off the radar of speculators. But now they are hipster paradises, and with gentrification, the "traditional residents" can't even maintain their old lives and are priced out of the market. I guess that is like economic eminent domain.

Measures like rent control or mandated affordable housing are often economically inefficient and flawed (because they are written and implemented by flawed bureaucrats, like healthcare.gov), but sometime optimal economics must take a back seat to concerns about fairness and social values (see taxes, labor laws). I'd rather have confusing rent control laws and too high minimum wage than a total lack of regulation (at least imperfect laws are a springboard for iteration and improvement). Because as you said, with the huge inequality gap, what is the point for the poorest 60% to struggle so hard trying to fit into the rich man's world (yet not really be allowed to live in the rich man's world... I guess like the "Elysium" plot, which I did not see)? Most people with a shred of heart would probably agree that there is something inherently unjust with greedy politicians/companies and wealthy invaders "annexing" and developing suddenly desirable land, which leads to the economic eviction of the residents who were there before (through legal market mechanisms and the landlords/property owners). Pardon my melodrama, but it's scarily similar to the Trail of Tears.

I heard an interview about the subject a while back, and one new SF resident and tech employee said something like, "Assholes (his word) like me can afford $3K/1BR rent and are making it hard on the previous residents, but hey, I want to live here." At least he had clear eyes. It's just sad because everyone has one life to live, and we don't want to compromise or suffer if we don't have to - even at cost to others (esp. when others' suffering at our hands is mostly invisible). Sorry for stereotyping, but I think the Gen X-ers and Millennials especially (present company excluded) really fixate on what I like to call "life maximization." Similar to their work habits of optimizing, achieving, and disrupting everything, in life they want it their way and they want it all. Best job, best home, best network, best gear, best marathon time, best family... best, best, best. When that is not really the goal but the status quo expectation in SF/Si Valley (when excellent becomes average/normal, what do all the sub-excellent people do?), then that doesn't really encourage a culture of togetherness and SHARING. We don't need the best life... can't we just enjoy our regular life? Isn't enough enough? Can I give a little up and still live a plenty comfy life so that more needy people can get a break? People don't ask these questions of themselves enough (myself included).

Sharing is the key I think. This isn't my world, it's our world. There is PLENTY of food, money, and room for everyone if we share reasonably, but the problem is the fucking 1% and the institutions who advocate for them just don't want to (and don't have to). Americans and some other cultures really fixate on fencing off what's yours and amassing/diving the pie, so that breeds an adversarial, zero-sum attitude that conservatives really eat up. Therefore, it's especially sad when educated, young, open-minded "liberal" West Coast people do the same and may not even realize it. At least liberals don't bad-mouth and hate the people they marginalize, but that doesn't leave them in the clear (myself included).

The best teachers I have had in my life (not necessarily in school) really cared about expanding the pie rather than dividing it. How can we make everyone happier? Any asshole with a bit of smarts can compete and beat others. But a truly smart, wise person goes out of his/her way to cooperate, find ways to align incentives, and make everyone better off - not just him/herself. For all the smug talent and genius brains and big money in the Bay, that skill is scarily absent. Don't get me wrong, some people practice it faithfully and I am in awe of them, but it's not enough to stem the tide. That's why I probably will leave this place next year, as much as it makes me sad and for all I am leaving behind. I don't want my kid to grow up in such a culture. At least I will make room for another person/family to have my stressful job and little slice of condo.

-----

Sorry I realize that I was unclear about one point in my last rant (unclear about at least one point, haha).

The Google buses are often scapegoats but not really the problem. By no means am I placing the bulk of the blame on the young, techie hipsters invading the Bay. They are not really the 1% and not the real driver of the problem, they are just a "symptom". Except for Zuck who bought a whole block of SF for privacy haha.

As usual, the problem is the old rich Nimby fucks that A alluded to. They are the ones who gobbled up all the choice land decades ago, and who currently occupy all the seats of political power and business influence. They are the ones who invested/profited from the tech boom (more than the current workforce), and stand to profit from real estate development and appreciation. They took so much that the current hipsters are practically forced to gentrify the Mission and maybe Hunter's Point soon. I bet most hipsters would much rather live in Nob Hill or downtown Paly, but all the old rich pricks there are just cramping their style anyway (and even a Twitter engineer still can't afford a $5M Nob Hill pad).

As A alluded to, we are not in the '50s Levittowns anymore, or not even in the '90s nouveau-riche gated golf communities. But the assholes want to keep us in the past, because they like the status and lifestyle they have, and don't want to give up even a shred to others. The city of Paris (and many other ancient cities) is like a 10-layer cake. New generations tear down the old crap that isn't working anymore and rebuild to fit the needs of the current people. We aren't really doing that, because the old fossils control the bulldozers. So all we have is SF Elysium vs. Vallejo. 

But "the people" are fighting back, and at least voted down a luxury condo development on the Embarcadero. It's a small victory in a long war that the good guys will almost certainly lose. But we might as well smoke a blunt and celebrate while we can. :)

http://www.sfgate.com/bayarea/nevius/article/Following-the-bankroll-for-SF-s-Propositions-B-C-4961999.php
http://www.npr.org/templates/story/story.php?storyId=243537913&ft=1&f=2&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+NprProgramsATC+%28NPR+Programs%3A+All+Things+Considered%29&utm_content=Yahoo+Search+Results 

-----

i once heard someone say cities are either places to live or places people come to make money.  he was talking about SF and that we were at a crossroads.  it was when matt gonzalez was running for mayor against gavin Newsome.  The speaker was making the case that if Newsome wins, the city will lose its soul, it's artist community, it's diversity of culture and class.  that's what you're looking at here….plain and simple.  and, you know what?  the city sucks now.  It's filled with people who are in a hurry, who lay on their horns, who yell.  

Why a lot of published scientific research could be wrong

http://www.economist.com/blogs/graphicdetail/2013/10/daily-chart-2#

We've talked about this before, but this infographic presents it pretty cleverly.

----

First I would love to see a source on where he gets his numbers of false positives and false negatives.

He also oversimplifies the way experiments work.  For example with the higgs boson.  The experiments were run continuously until statistical significance  was seen.  That means the hypothesis was testing hundreds and hundreds of times.  The false negative/positive issue is resolved by virtue of the experimental setup.  

And on top of that, published work is not set on a pedestal because it is published.  Once it is published there is generally rigorous work to review and replicate the findings.  

So in short, he is wrong, and when he is right it doesn't matter because that is part of the process.  

-----

Those #s were just hypothetical based on a probability distrib, just to make his point. But sadly few scientists even consider type 1 and 2 errors, and just project confidence that their findings are bulletproof. In bio, when ppl try to reproduce expts, they can't about half the time, according to Science mag.

Yeah I think they are referring to distinct hypotheses, so a series of expts relating to the same hypoth. would probably be 1 data point only. There is also a risk associated with repeated "testing until significance", cuz the more measurements you make, the higher the chance you'll get some random signal and mistakenly use it to prove your hypoth. 

Actually you would be surprised how little rigor goes into the review process, I am sure S and A could comment. If the author is a big name with political clout, free pass. No one has the time or resources to verify and replicate work, it is the honor system. If another group wants to build on the research and don't get the same results, they may publish that, but may not to avoid controversy and be embarrassed if others don't buy it. 

But considering the incentive structure and human nature and the higher stakes of published results, it's a bad combo. Also the journal wants to publish big sexy results to make more money so there is COI.

----

re: "mistakenly use it to prove your hypoth" I think you don't understand what I'm describing.  The measurements means more averaging as I'm describing it.  And more averaging means LESS chance of random signal causing an error.  It is just a method for eliminating "noise" whatever that noise may be.  For the higgs boson it was a very specific energy level measured from the decay.  Every time they took another measurement, if the higgs existed the energy signature would be reinforced and other sources of energy negated, or the opposite if it did not exist.  And with some fancy math they get a confidence level based on number of trials and blah blah.  So not all experiments are susceptible in the same way.

Re COI and big names.  Yea.  Reinheart and Roghoff (sp?) is an example where more weight was given to big names.  But it is also an example of the fact that people do look at this stuff.  Though you could argue the damage was done by the time we figured out their conclusions weren't great.

In any case I'm not arguing that false positivies don't exist in academic journals.  I am arguing that this particular guy's math that proves it is false at worst or misleading at best.

----

I see, thanks M. In that case, the higgs boson expt was doing its own internal QA and replicating the experiment many times (something very lacking in bio and business, where many expts are underpowered). So if the variance is small and there is a strong signal from most of the trials, then that is a very convincing result.

At work we were struggling with a related issue. We usually don't approve changes to the website unless they pass a lot of statistical and sanity checks. But due to probability we know that we are likely rejecting some features that were truly positive (but we couldn't detect it, by chance or by poor design), and we are approving some features that are in fact neutral or even negative. That is the scary possibility. Sometime our approval criterion is "do no harm", as in if all the key metrics look to be within noise, and we can't reject the null hypothesis, then it's OK. But there is a slim chance that we are approving stuff that is actually very harmful. And if the company is approving 100's of features a year, it's likely some of them are harmful but mislabeled as positive or neutral. Though for mature businesses, where you are squeezing out basis points on the margin, the risk is probably not terrible. I guess there is a balancing act between scientific rigor and business expediency/strategy. But at least in e-commerce, an error isn't likely going to cost lives (unless it's Apple maps LOL). But for consumables, finance, or public works, it could be really bad. And they say we don't need regulation? :)

----

Yeah, it's too bad that knowledge of this sort of thing seems to only reinforce the worst of carpetbagging instincts: get in early, publish as much as you can whether of integrity or not, and then do everything in your power to ensure that no one who comes after you could possibly do it as its supposed to be done in the first place.

Maybe it's a cultural thing - like, perpetuating a hierarchical ponzi scheme requires the smartest of people to buy in to the stupidest of ideas (like blinders, narrowness, and petty infighting)...

PS:  Technology will (ultimately) solve all these problems as democratized access to information will lead to market forces being brought to bear on the most indefensible of regressive attitudes and mentalities (most of which are sustained on the basis of petty economics).

----

We were talking about this, and here were our general follow-up thoughts:

- The sci. method and the peer-review process are hundreds of years old and developed during times when methods were not so empirical & specialized, and there wasn't such $$ at stake for discoveries (for people like Euler and Carnot, I think reputation, knowledge, and prestige might have trumped the incentive to cheat, be negligent, and cut corners)

- Obviously we live in a new age now, and as C said the democratizing effects of technology could mitigate the problem (i.e. a grad student found Rogoff's errors when big time editorial staff and top peers didn't), but there are limits to that in hard science because of the cost and specialization of certain expts (though if you need a certain highly skilled postdoc to do a test a particular nuanced way to get a positive result, probably your result is not that robust)

- At least for bio, we thought to develop an independent, confidential auditing lab within the NIH to verify all published results. Journals and authors have to pay a fee to support it, and an article that passes the audit gets a certification that gives it more clout. The author's lab has to send all the materials over to the NIH (or let them use their equipment), with instructions, and the expert NIH staff have to reproduce the result within reasonable variance (they sign NDAs and no-competes so the author has no fear of being scooped). If they can't reproduce, then the paper can still be published at the journal's discretion, but without certification.

----

I would think an algorithm could be developed (or maybe already has?) that can blindly take in the statistical data from an experiment and verify the conclusions.  Rogoff was a case of bad math not bad data right?  That removes the requirement to reproduce experiments and leaves you with only the first two cases of lies, damned lies, and statistics.

The NIH thing is interesting but there is still money involved which is always problematic.  The NIH is not free from politics since that is its major source of funding.  And there are many many public institutions that don't share the data found through public dollar funded experiments already.  The democratization of technology doesn't help when all the research is behind a pay wall.

----

My comments are enclosed (in-line)...  Also, in case anyone didn't see the full article:  http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble

I would think an algorithm could be developed (or maybe already has?) that can blindly take in the statistical data from an experiment and verify the conclusions.  Rogoff was a case of bad math not bad data right?  That removes the requirement to reproduce experiments and leaves you with only the first two cases of lies, damned lies, and statistics. 

It's a good idea; one would think some of that would be built-in to the tools the scientists and researchers are using but clearly there is progress to be made...

The NIH thing is interesting but there is still money involved which is always problematic.  The NIH is not free from politics since that is its major source of funding.  

True... but federal institutions tend to be bulwarks of trust (certainly to a greater extent than a tobacco company or The Koch Brothers)...

And there are many many public institutions that don't share the data found through public dollar funded experiments already.  

This is also true... but changing.  Even institutions which are known for being benefactors of the public (e.g. U.C. Berkeley) are getting more formal about sharing research and ensuring access... so the change is bound to ripple through to other public institutions too...

The democratization of technology doesn't help when all the research is behind a pay wall.

Yeah, that is slow to change - but perhaps the most hopeful front.  The economics of information mean is such that librarians and libraries are being confronted by these questions so the clock is already ticking: if our engineering library at U.C. Berkeley is getting rid of its books for the value the physical space has to the college then its probably a matter of time until someone rationalizes our giving research to private journals so that they can rip off the campus with subscription fees that (over time) do not seem to be worth more than the promise of a new student.

-----

For Rogoff, I think it was a variety of issues, but some of it was excel formulas pointing to the wrong cells, as well as "improper" suppression of some data points. So that is mostly human error and poor judgment, which is hard for an algo to rectify.

An algo would be cool (like how the IRS algos randomly scan over people's returns and flag errors/possible issues), but then the scientific data has to be formatted and structured according to some standards so the algo can read it properly. And since each article kind of measures different things and employs different significance tests, it could be tricky. Hell, they can't even get healthcare.gov right using 50 contractors (maybe that is their problem!).

There are some free or lower cost internet journals out there... hopefully they can give the heavies a run for their money some day. But due to the prestige factor, no big names want to "slum it" with online journals. But I agree that those journals are ripping off institutions to give then "novel research" that is at least 30% useless and another 30% incorrect.

Jokes aside, we piss and moan about how bad public institutions are (and they are of course not free of corruption either... see the MMS), but as C said, they are our last resort against a totally for-profit world. We need to strengthen the public institutions so that they offer a legit alternative to the for-profits, and then with customer choice it will compel the for-profits to clean up and stop shafting us so much. I guess that's why the health industrial complex fought so hard against single payer, which is BY FAR the best feasible health system in the Western world, warts and all. But with all the dysfunction in Washington, furloughs, and a reduction in public worker compensation/respect, it makes it less likely that our best and brightest will want to go into public service and stay there long enough to make an impact. 

How the Dulles brothers made Kissinger and Cheney look like peaceniks

http://www.npr.org/2013/10/16/234752747/meet-the-brothers-who-shaped-u-s-policy-inside-and-out

We don't hear much about these two (apart from the DC airport name), but their beliefs and actions literally shaped the Cold War, US global ambitions, and we are still dealing with the negative repercussions today.

Ike appointed John F. Dulles as his Sec. State and Allen Dulles as the CIA head. They were both corporate lawyers who represented, among others, United Fruit and the financiers of the Anglo-Persian Oil Company. Enough said. Ike basically gave them carte blanche authority to make enemies and "intervene" in any foreign country they saw fit, and they didn't hesitate: Guatemala, Iran, Congo, Cuba, Indonesia, and Vietnam ... basically our greatest hits. They were especially adamant about Iran, because under no circumstances could they tolerate a leader nationalizing oil reserves for the people, and renegotiating terms on foreign companies. Imagine the shockwaves to big business, and what message that would send to other countries.

Unlike Kennedy's "team of rivals" model, it was very dangerous for Ike to appoint two like-minded, corporate shill, biological brothers into the complementary (and sometime necessarily adversarial) positions of State and CIA. One side shapes foreign policy and strategy, and the other carries it out covertly. Or one side tries to make peace, while the other sows the seeds of war. It's playing with fire. Now they were basically one org, without any sanity checks or devil's advocate to say, "What happens after we overthrow Lumumba? (answer: civil war and genocide for decades)," or "Maybe we can't win in Vietnam."  I guess the Dulles' and their Cold War were enough to make General Ike warn the nation about the military-industrial complex upon his retirement.

The Dulleses believed that the US, its market economy, and its multinational corporations were exceptional and entitled to dominate the globe (because we were the good guys). Anyone who opposed these interests (regardless of the reason, like the crazy notions of self determination and liberty) were automatically evil monsters. It's OK for George Washington, but not for Third World leaders (esp. when they are sitting on natural resources). Clearly enemy #1 was communism, and by extension populist nationalism, because it went against the model of private capital might makes right, and neo-colonialism/imperialism. Any attempt to regulate or take power/profit away from US companies was as much of a threat to US security as nuclear missiles.

Apart from the direct and indirect violence & collateral damage (5M total deaths at least), militancy/extremism, and harm to our global reputation in most of the world, here is the Dulles' biggest legacy and curse on humanity: a pervasive culture of rejecting peace talks and the undermining of democracy.

After Stalin died, the USSR wanted to improve relations with the West. It was a great opportunity, but Dulles was the only major foreign leader to reject this overture (and pressured our allies to do the same), because he thought the Russians were too evil to talk to, and preferred that our conflict continue. Later, both sides' nuclear brinksmanship almost ended the world. During the North-South Vietnam diplomatic talks after the French withdrawal, most European powers felt that the party was over and they should learn to live with Ho Chi Minh. Dulles was the only one who felt that Ho could be beaten militarily if we stuck it out, and he persuaded Ike and Kennedy. We know how that turned out. But you can see this US hubris and misguided thinking carrying over to our negotiations (or lack theoreof) with Cuba, Iran, and North Korea over the years. We have certain beliefs about other people, and our stubbornness ends up prolonging/worsening conflicts and missing out on opportunities.

Why was the US so successful in overthrowing the democratic regimes in Iran, Guatemala, and Congo in the '50s? The author argues that it was because those societies (and their idealistic, civilian, center-left leaders - a.k.a. died-in-the-wool commies) were fairly open and democratic, thereby allowing foreign agitators access to covertly undermine the gov't and eventually launch a coup d'etat (with a pro-US dictator waiting in the wings). Later revolutionaries learned from those cases, and ruled more like repressive tyrants in order to defend against a possible US overthrow. Cuba, North Vietnam, the Islamic Republic of Iran, North Korea, etc. all became more paranoid and less democratic after the Dulles' time. It could just be coincidence, but at least in the case of Cuba, Che and Castro were quoted that they would not repeat the mistakes of Arbenz in Guatemala. They would repress and spy on their own people to make sure the CIA wasn't gaining a foothold.

So even though the US portrayed itself as the global example and defender of democracy, in fact its foreign policy under the Dulleses was terribly undemocratic, and motivated many others around the world to choose brutality over democracy too (tragically, during a time of global change that we will never get back, when we really could have forged tremendous social progress and peace). And the human race may never fully recover from the legacies of those dark times. To me, that is a much greater failure, security threat, and shame on the US than letting a few undeveloped countries elect a socialist leader, or try to nationalize their resources.  

With friends like these...

http://www.npr.org/2013/09/25/225858836/u-s-soldier-crusading-for-afghan-interpreter-who-saved-his-life

Maybe you have heard about this story. The US sometime has the rep for "burning" foreign allies during the Cold War when it made political sense. But in the asymmetrical War on Terror, we have the resource advantage and there's no reason to turn our backs on the people who risked everything to help us - because they actually believed that we would make their countries better.

We're not talking about hundreds of thousands of refugees like Syria here (though it would be principled to do more so poor Jordan and Turkey don't have to bear the whole burden). I doubt the total is higher than a thousand Afghan and Iraqi translators and their families. In some cases, they directly saved American lives, which is more than most US citizens can say about patriotism.

A few years ago, Congress passed a law to grant visas to these folks because they will be in mortal danger once US forces leave. This isn't exaggerated speculation, as many of them have already received death threats (it's no secret that they are "infidel sympathizers" as they stood with US forces in public view). And we know that Taliban and AQ in Iraq types know how to make people disappear in the night.

But years later, only 12% of approved Afghans and 22% of Iraqis have made it to the safety of US shores, mostly because of DHS and State Dept immigration BS that these folks could be potential security risks. Yes it is true that there has been plenty of "green on blue" violence in Afghanistan where infiltrators masked as allies have killed UN forces. But those were fresh recruits in a warzone without much vetting. It's another story for an infiltrator to masquerade as an embedded translator, risk life and limb for years, and then travel to America just hurt us. If any of you know immigrants, you have probably heard stories about what a pain in the ass the background check is. Just imagine how it is for people from unstable Muslim nations. In some cases, the Taliban found out that some translators were getting out, so they may have made false "anonymous tips" to the US embassy that those people were actually threats. And all of a sudden their visas are on hold again.

The problem with bureaucrats is they fixate on the letter of the law rather than the intent and the bigger picture. I am sure none of them wants to be responsible for giving a terrorist a visa, but is that worth condemning our friends? At least transport them to Gitmo or a midwest jail until they are fully cleared. But we can't just leave them behind. And we wonder why they hate us?

Similar to the story of Sydney Shanberg, one soldier (Capt. Zeller) has tried tirelessly to get his translator Shinwari out of Afghanistan, after that man saved his life in the field and they became "brothers". He went to the media, wrote Congressmen, and just when it looked like it was going to work out, more visa delays. Now Zeller has lost contact with Shinwari, and he hopes he just went into hiding with his family rather than something worse. 

I hope Pope Francis delivers more hope and change than Obama

Not sure if you've been following some of the recent comments from Pope Francis, but he is definitely a better "Maverick" than McCain and Palin. I know there are many different levels of belief on this email list (each no better than the rest), so I don't mean to get too deep in the religion weeds here. But as a lifelong confused/jaded Catholic, I mostly just want to comment on the new pope's effectiveness vis-a-vis the core beliefs of the church, and what his org needs to do to be more successful in the 21st Century.

http://news.yahoo.com/pope-says-church-cannot-obsessed-gays-contraception-abortion-163220900.html

Yes, Christianity has to be about more than quasi-arbitrary rules... all the things we are not supposed to do (and imposing that on others who are doing no harm). That is Taliban style. Faith isn't about prohibition and guilt - it's about being happy and doing good. And what is the point to fight for the welfare of unborn fetuses when you will turn your back on them after they are born into bad circumstances? Many Christians do act like Christ, but many more are astoundingly silent or complicit regarding greed (Francis spoke out), violence, and basically any other bigotry and injustice that doesn't directly affect them. But that is the strange marriage of Christianity and western free market & social conservatism - for them to pretend to be compatible, something's gotta give (and it's usually an abdication of core Christian values).

Francis is like the polar opposite of Benedict (the least inspiration pope since the Nazi appeasers IMO), and is more in tune with reality and modernity (and Jesus' teachings). I think some hardliners are seriously going to plot to kill him (it has likely happened before). Francis is saying/doing all the right things considering the constraints of his office, except really coming clean and contrite about the sex abuse scandals and the inferior role of women in the church. Maybe he's working up to that. But so far he has chosen to live like a common man (as opposed to Benedict's Prada slippers), literally washed the feet of his flock, and held a peace vigil and fasting for the people of Syria.

Other Christian denominations are leaps and bounds ahead of the Vatican on gender and sexuality tolerance, but on the record Francis said that he is not opposed to gay priests, and has no right to judge anyone who practices goodwill and searches for the Lord. Let's see if he actually implements this in the church. This contrasts Benedict's statement in 2005 that gays would not be welcomed in the priesthood (but bishops who cover up sex abuse are fine?), and that homosexuality is an evil disorder. Supposedly God loves all his children, so why would he create and curse some to lead "evil" lives just so self-righteous bigots can denounce them to feel better about themselves? Well, why would God curse innocent children to be born into starvation, war zones, and AIDS (and bless others with safety, riches, and opportunity)... but I guess we'll never have an answer for that.

He's also said that atheists can go to heaven just like believers. Doing good is the criterion, not some ceremonial label. Clearly stuff like this has ruffled some feathers among the stuffy establishment types. But the College of Cardinals picked this man to lead (and came to their decision rather quickly compared to Benedict). It's not like Francis was always a rebel, or he would have never risen in the church's ranks. As recently as 2010 Francis spoke out against gay adoption in his native Argentina, but apparently he has changed his views for the greater needs of the office (some might think divine inspiration, or just sensible rebranding?). Maybe the Cardinals realized that the church has pretty much been failing in every way, especially with the young, vulnerable, and poor of the world (the most blessed people according to Jesus). So a big change was needed, like Obama-sized change. I just hope at the end of his papacy, we have more to show for it than a health care policy mess and drone strikes.

So far Francis has been mostly talk, though that's kind of his job. He isn't the head of a state (well, not a very big one), so he can't really change our lives as much as civic leaders. He does have a lot of financial and human resources at his disposal that he could deploy. But in general he is there to inspire. And it seems he is doing a good job of that, especially in the "growth markets" that the church will depend on in the future. He's the first non-European born pope in centuries (though he is ethnically Italian), and he is a huge hit among the young of the developing world. During the Brazil protests, he mingled with the crowds and spoke about inequality (Brazil is the largest Catholic country in the world and also has one of the biggest wealth gaps). While the silver-haired septuagenarians in the west are the church's core following (and cash cow), everyone knows that their time is short (and their kids and grandkids are not going to church anymore). Maybe the pope's message doesn't resonate with them, and he may make them uncomfortable (not necessarily a bad thing for a leader to do at times). But where are they going to go... join the LDS? :) Maybe Francis could get some of them to rethink their antiquated and frankly un-Christian beliefs. Ironically or maybe not, most of this email has nothing to do with religion. It's about getting people to consider the benefits of not being intolerant, selfish pricks. If Francis can move the needle a bit there, then he should be canonized.  

Like the GOP, there is a struggle for the soul and future of the church. The hateful, egocentric, self-serving, and power-hungry factions of both the GOP and the Catholic Church may hold disproportionate sway now, but it can't last or their future is doomed. Just get back to basics: why does the church exist? Not to look after itself, conceal crimes/corruption, and act superior judging others. It should help people through education and compassion. I think Francis realizes that the bigger the leader, the more of a servant he or she must be. A servant, not a king. Any leader could benefit from that lesson, from POTUS to parent.

How to find out who has a g0v't s3cur1ty cl3aranc3

http://www.marketplace.org/topics/tech/how-one-hack-got-engineers-security-clearances

Sorry for the funky email subject... was trying to prevent PRISM from detecting keywords and flagging us! Not sure if I am being paranoid or facetious.

Remember that old movie "Sneakers" (kind of put Bay Area tech culture on the map for Hollywood)? Robert Redford led a team of hackers who are paid to try to infiltrate an org's security, in order to expose vulnerabilities. Well that job is alive and well today, but with fewer shotguns. The pros don't waste their time with high-tech hacking, they just employ the oldest confidence schemes in the book! No point taking the Rube Goldberg path when marks will give you want you want for free. But the whole point of a gov. sec. clearance is to not broadcast to the world that you have one. How to out the engineers and companies who do?

It's much easier to convince someone on the inside to give you access rather than trying to break in yourself and risk getting pinched. There's always human psychology, and the gov't contractor-tech geek types aren't exactly known for their street smarts and social skills. So these "good hackers" went on LinkedIn, and used pretty simple tricks to reel in folks with gov't clearances to improperly expose themselves (not Anthony Wiener style fortunately).

They played to their impulses and egos: impress someone who claims to be a recruiter with your super-cool classified projects, and come to the aid of an attractive damsel-in-distress who wants to get into the NatSec industry. Kind of scary that it was this easy though. Clearly these orgs need to give more security training, especially when social media are involved. Or maybe they should snoop on their own workers first before sifting through the general population.

Snowden, Syria, and Obama

http://www.pbs.org/newshour/bb/government_programs/july-dec13/surveillance_09-06.html
http://www.nytimes.com/2013/09/06/us/nsa-foils-much-internet-encryption.html?hp&_r=0

Snowden leaked that the NSA gave companies court orders to fork over their security keys so that they could access confidential financial and personal info. The made the companies sign gag orders so they couldn't complain, and we are talking about the major players like MSFT and GOOG (in fact Google is now suing the gov't for the right to tell the public about how they were forced to give access to user info). Also, the NSA just blatantly cracked other firms' encryption. Ironically the US Gov't has recently complained about China doing that to our companies. The NSA hasn't denied anything, and said that the disclosure has hurt national security. As we discussed before in the original Snowden story, maybe this invasive action is tolerable when there is a court-issued warrant (from a real court with real oversight, not some shadowy FISA BS), but they can't just arbitrarily cast a dragnet on everyone.

http://www.pbs.org/newshour/bb/politics/july-dec13/shieldsbrooks_09-06.html

Was it a mistake for Obama to involve Congress in the Syria decision? Apart from the hard core hawks like McCain, many in the GOP have incentive to make Obama look ineffective on Syria. House leaders "support" Obama's plan, but won't round up votes for him. Congressional Dems now feel pressured to go to bat for the president, and if the Syria op goes bad, it may cost them in the midterms. The majority of Americans are apathetic or against military escalation in Syria, as are the majority of the world's people (or so it seems after the G20 summit). So it's unlikely that Obama will even be able to assemble a coalition at home and abroad to match the support for the Iraq invasion, as sad as that is (Saddam: imaginary WMDs, Assad: real WMDs and he used them). So now will Obama "go it alone" (+France), which is the type of unilateral action that he denounced when the Bushies did it. Or when is it OK to go it alone if you know you're right? Well, if you are right, it shouldn't be too hard to persuade/bribe/intimidate others into tacit approval or full support. And for all of Obama's big talk and int'l adoration, his administration has actually been pretty horrible on diplomacy. Talk is cheap now that Assad has deployed WMD. The trick is you talk to him BEFOREHAND, when better options still existed, stakes were lower, and emotions were cooler. But we tend to dither until problems get out of hand. Maybe that is also a side effect of Iraq's failed pre-emptive war strategy and resultant anti-Americanism, now we are skeptical/hesitant to swiftly engage anyone, even if it's not militarily.

I have mixed feelings about a military strike, unless we want to do it purely as formality to show that there are *some* consequences to using WMD (just lob some missiles at easy targets and that's it, no escalation plans, no what-ifs). But like Iraq, right now we don't really have an exit strategy, or any strategy, or even agreed-upon objectives (regime change, reduce Assad's ability to use WMD again, protect civilians? Not necessarily mutually exclusive). That's why it's so frustrating to see big Dems like Pelosi rubber-stamping this. Another problem is Iran has called for terror groups to strike at US targets and affiliates if we do anything to Syria, so we have to weigh the risk-benefits (and sorry Syrian victims, unfortunately your lives are less valuable to us).

Russia has moved some warships into the Eastern Med. I also don't understand that... why does Russia care so much what happens to Assad? I know he is a big trading partner and buys a lot of weapons from them (and is serving as a distraction for the US to prevent our "hegemony" in the Mideast), but is it worth losing face if Assad ends up going madman and doing something worse? According to Putin, it's not just him and China though, as India, Brazil, Argentina, and Italy have also opposed military action.

http://www.cnn.com/2013/08/29/world/meast/syria-iran-china-russia-supporters/index.html

But as usual, Syria is really a proxy battleground and the underlying issues are much bigger. Syria's people are unfortunately a test case to evaluate what are the int'l norms for violating sovereignty for humanitarian, political, or security reasons. Supposedly we're going to have to face similar issues with Iran as it gets closer to a functional bomb, and as the people demand more freedom from the theocracy and may revolt. What the interested parties do vis-a-vis Syria is signaling to the other parties on what will happen should Iran go down a similar path. Unfortunately the US is not sending out great signals at the moment: dithering on justification and appropriate response to WMD usage and other atrocities. Plus all this pre-warning and debate is giving Assad plenty of time to prepare and hide or shield his military assets, as well as giving Iran's proxies preparation for potential retaliations. The trick was to diffuse Syria way before we ever got to this point, but that ship has sailed long ago. You think Kennedy and Khruschev could have resolved the CMC if troops had already killed each other?

FUBAR....

--------

I was listening to someone on NPR about the Russian perspective and it was really interesting.  Aside from the fact that Russia has a fair amount of Russian interests at stake in Syria, they view the crisis much more on the side of Assad than they do the rebels because of their past with Chechnya.  A sub-unit of their country, Islamic terrorism, etc.  So for the ruskies this looks a lot more like a conflict they are familiar with and don't want to see the "terrorists" win.

I'm not sure why Obama is giving this to the congress.  I think, as a rule, it should.  But in his own speech at the G20 he said that these types of conflicts don't poll well.  If Rwanda were today it wouldn't poll well.  So given that he knows it is generally unpopular, why put it to a vote?  Just so posterity can be SURE that his action was against the will of the people?  

Also an onion article, I paraphrase:  "Obama debating to either proceed with missile strikes, or to strike with missiles".  It doesn't seem like there are any alternatives being considered, perhaps rightly so, but it doesn't help the idea that the congress is controlling this mess.  If they vote 'no' then what happens?

The fact that Snowden is not considered a whistle blower is ridiculous.  I don't see how anyone can, with a straight face, say that we the people don't have a right to this kind of information.  The fact that there are secret courts, massive spying networks, privacy tools currently available have been backdoor'ed or defeated entirely, is insane.  And the response is "why doesn't he let a court decide if he is really a whistle blower?"  As if history is full of examples of prosecuted whistle blowers who had nothing bad happen to them.  Get real.

--------

Thx, M.

I 100% agree with you on Snowden. The "official" definition of whistleblower is BS, modified by the people who stand to lose from whistleblowers. And as you said, the whole "give yourself up and you will get your chance to be heard" has probably never worked out. This NSA garbage is blatant abuse of power, and the only difference with Nixon is that the NSA hasn't been caught obstructing justice yet (because they have courts that exist to protect them, unlike Nixon). Another side effect of the terror-industrial complex. But I think the bigger tragedy is that democratically minded, progressive, and smart leaders like Obama and Feinstein are not only failing to hold the NSA accountable, but actively defending them. There was a scene from this season of Sorkin's "The Newsroom", ironically about investigating a wild claim that the US used WMD on Pakistani "terrorists". The producer was frustrated that his peers couldn't fathom this president's capacity for criminality because they liked and admired him so damn much. To me, THAT is Obama's greatest sin (not the fact that he is a Kenyan born Muslim).

Ya I can understand that Russian perspective, and China may have it too regarding their internal dissidents. But unless they really consider unarmed women and kids as "terrorists" who deserve to be gassed (assuming the evidence is solid and not the BS that Colin Powell presented to the UN), I think they are kind of full of it. Remember when Russia invaded Georgia, because Georgia was trying to put down a rebellion in South Ossetia that was trying to unify with Russia? So in that crude analogy, Georgia = Syria, Ossetian rebels = Al-Qaeda linked Syrian rebels, and Russia = the US ("peacekeepers"). So it was OK for Russia to invade then, but they oppose the US+France doing it now? There are plenty of reasons to protest Obama's plan, but Russia doesn't really have the cred to say anything.

A friend sent this interesting read on Syria (below). It is basically saying that the worst thing Obama can do is pledge a "limited" response to Assad's war crime (my bad, as that was what I suggested with some ceremonial Tomahawk strikes). That way Assad knows exactly what the cost of using WMD is, and he can make the rational choice as to when it will be worth it to use them. Uncertainty breeds fear and caution (and sometime clarity), and that is what we want to attack him with. It was no accident that Saddam and Osama looked pretty sickly and pathetic when we found them. Hiding for your life and never knowing when SEALs are going to bust down your door takes a helluva toll on a person. It might even motivate them to capitulate. So the author is saying that we shouldn't reveal our hand. We should build up massively on Syria's borders, run some major military exercises and flex our muscles, and that will scare the crap out of Assad. Maybe Obama should lose the cool veneer and act a bit unstable for the cameras. Dictators fear cowboys like Bush more than vulcans like Obama. Idiots and bullies think that violence solves problems. Clever leaders know that there are many other levers of persuasion.

And what about cyber warfare? We crashed Iran's centrifuges and we spy on our own innocent civilians. Why can't we shut down Assad's military IT and wipe our his finances? Heck we could make his warplanes fire on each other instead of strafing towns.

Syria and Byzantine Strategy

September 4, 2013 | 0900 GMT

Stratfor

In March 1984, I was reporting from the Hawizeh Marshes in southern Iraq near the Iranian border. The Iran-Iraq War was in its fourth year, and the Iranians had just launched a massive infantry attack, which the Iraqis repelled with poison gas. I beheld hundreds of young, dead Iranian soldiers, piled up and floating in the marshes, like dolls without a scar on any of them. An Iraqi officer poked one of the bodies with his walking stick and told me, "This is what happens to the enemies of Saddam [Hussein]." Of course, the Iranians were hostile troops invading Iraqi territory; not civilians. But Saddam got around to killing women and children, too, with chemical weapons. In March 1988, he gassed roughly 5,000 Kurds to death. As a British reporter with me in the Hawizeh Marshes had quipped, "You could fit the human rights of Iraq on the head of a pin, and still have room for the human rights of Iran."
The reaction of the Reagan administration to the gassing to death of thousands of Kurdish civilians by Saddam was to keep supporting him through the end of his war with Iran. The United States was then in the midst of a Cold War with the Soviet Union, and as late as mid-1989 it wouldn't be apparent that this twilight struggle would end so suddenly and so victoriously. Thus, with hundreds of thousands of American servicemen occupied in Europe and northeast Asia, using Saddam's Iraq as a proxy against Ayatollah Khomeini's Iran made perfect sense.
The United States has values, but as a great power it also has interests. Ronald Reagan may have spoken the rousing language of universal freedom, but his grand strategy was all of a piece. And that meant picking and choosing his burdens wisely. As a result, Saddam's genocide against the Kurds, featuring chemical weapons, was overlooked.
In fact, the 1980-1988 Iran-Iraq War, coterminous with the life of the Reagan administration, was a boon to it. By tying down two large and radical states in the heart of the Middle East, the war severely reduced the trouble that each on its own would certainly have caused the region for almost a decade. This gave Reagan an added measure of leeway in order to keep his focus on Europe and the Soviets -- and on hurting the Soviets in Afghanistan. To wit, only two years after the Iran-Iraq War ended, Saddam invaded Kuwait. Peace between Iran and Iraq was arguably no blessing to the United States and the West.
Likewise, it might be argued that the Syrian civil war, now well into its second year, has carried strategic benefits to the West. The analyst Edward N. Luttwak, writing recently in The New York Times, has pointed out that continued fighting in Syria is preferable to either of the two sides winning outright. If President Bashar al Assad's forces were to win, then the Iranians and the Russians would enjoy a much stronger position in the Levant than before the war. If the rebels were to win, it is entirely possible that Sunni jihadists, with ties to transnational terrorism, will have a staging post by the Mediterranean similar to what they had in Taliban-controlled Afghanistan until 2001, and also similar to what they currently have in Libya. So rather than entertain either of those two possibilities, it is better that the war continue.
Of course, all of this is quite cold-blooded. The Iran-Iraq War took the lives of over a million people. The Syrian civil war has so far claimed reportedly 110,000 lives. Even the celebrated realist of the mid-20th century, Hans Morgenthau of the University of Chicago, proclaimed the existence of a universal moral conscience, which sees war as a "natural catastrophe." And it is this very conscience that ultimately limits war's occurrence. That is what makes foreign policy so hard. If it were simply a matter of pursuing a state's naked interests, then there would be few contradictions between desires and actions. If it were simply a matter of defending human rights, there would similarly be fewer hard choices. But foreign policy is both. And because voters will only sustain losses to a nation's treasure when serious interests are threatened, interests often take precedence over values. Thus, awful compromises are countenanced.
Making this worse is the element of uncertainty. The more numerous the classified briefings a leader receives about a complex and dangerous foreign place, the more he may realize how little the intelligence community actually knows. This is not a criticism of the intelligence community, but an acknowledgment of complexity, especially when it concerns a profusion of armed and secretive groups, and an array of hard-to-quantify cultural factors. What option do I pursue? And even if I make the correct choice, how sure can I be of the consequences? And even if I can be sure of the consequences -- which is doubtful -- is it worth diverting me from other necessary matters, both foreign and domestic, for perhaps weeks or even months?
Luttwak himself offers partial relief to such enigmas through a meticulous and erudite study of one of the greatest survival strategies in history. In The Grand Strategy of the Byzantine Empire (2009), he demonstrates the properties by which Byzantium, despite a threatened geographic position, survived for a thousand years after the fall of Rome. This Byzantine strategy, in its own prodigiously varied and often unconscious way, mirrored Morgenthau's realism, laced as it is with humanism.
The Byzantines, Luttwak writes, relied continuously on every method of deterrence. "They routinely paid off their enemies….using all possible tools of persuasion to recruit allies, fragment hostile alliances, subvert unfriendly rulers…" He goes on: "For the Romans…as for most great powers until modern days, military force was the primary tool of statecraft, with persuasion a secondary complement. For the Byzantine Empire it was mostly the other way around. Indeed, the shift of emphasis from force to diplomacy is one way of differentiating Rome from Byzantium…" In other words, "Avoid war by every possible means in all possible circumstances, but always act as if it might start at any time [his italics]." The Byzantines bribed, connived, dissembled and so forth, and as a consequence survived for centuries on end and fought less wars than they would have otherwise.
The lesson: be devious rather than bloody. President Barack Obama's mistake is not his hesitancy about entering the Syrian mess; but announcing to the Syrians that his military strike, if it occurs, will be "narrow" and "limited." Never tell your adversary what you're not going to do! Let your adversary stay awake all night, worrying about the extent of a military strike! Unless Obama is being deliberately deceptive about his war aims, then some of the public statements from the administration have been naïve in the extreme.
A Byzantine strategy, refitted to the postmodern age, would maintain the requisite military force in the eastern Mediterranean, combined with only vague presidential statements about the degree to which such force might or might not be used. It would feature robust, secret and ongoing diplomacy with the Russians and the Iranians, aware always of their interests both regionally and globally, and always open to deals and horse-trades with them. The goal would be to engineer a stalemate-of-sorts in Syria rather than necessarily remove al Assad. Reducing the intensity of fighting would thus constitute a morality in and of itself, even as it would keep either side from winning outright. For if the regime suddenly crumbled, violence might only escalate, and al Qaeda might even find a sanctuary close to Israel and Jordan.
Such a strategy might satisfy relatively few of the cognoscenti. Though, the American public -- which has a more profound, albeit badly articulated sense of national survival -- will surely tolerate it. The Congressional debate that preceded the Iraq War did not save President George W. Bush from obloquy when that war went badly. The lack of such a debate would not hurt Obama were he to successfully execute the methods described in Luttwak's book.