Hacker News from Y Combinator

Syndicate content
Links for the intellectually curious, ranked by readers. // via fulltextrssfeed.com
Updated: 6 hours 58 min ago

Grow Your Own Type System

27 November 2015 - 10:16pm
tomprimozic/type-systems · GitHub

Skip to content

Something went wrong with that request. Please try again.

You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.

Telegram stalking

27 November 2015 - 7:53pm
Stalking anyone on Telegram

According to the Telegram CEO Pavel Durov, earlier this year the Telegram app was delivering 2 billion messages daily and had 62 million monthly active users. Possibly some of that size can be attributed to Telegram's focus on security and privacy. The Telegram FAQ focuses quite a bit on that and for instance states:

After having read that I was surprised to see the amount of metadata received from my contacts. Most of the metadata is not directly visible in the web and mobile clients, but using a third party client such as vysheng's CLI client any received metadata is displayed:

The Telegram android app sends a notification to all contacts when it becomes or stops being the "foreground" app on the device. Using that information alone it's at times easy to make guesses about who's talking to who if you have several contacts in common with a "victim". An "attacker" will sometimes see the victim and another contact taking turns going active/inactive as they pass messages back and forth.

Add anyone to your Telegram contacts

The metadata may be a considerable problem in itself but what makes it worse is that Telegram does not require contacts to mutually agree that they should be connected! As long as an attacker know the phone number of the victim and add it to the android contacts the victim will show up as a Telegram contact and the attacker will automatically subscribe to the victim's metadata. As a bonus the victim will not be notified in any way and the attacker will not show up among the victim's Telegram contacts.

Putting it together

The attacker then takes guesses at what Telegram contacts the victim may have. Adding them as android contacts makes them automatic Telegram contacts and the metadata analysis can begin. Using the metadata the attacker may have a good chance of figuring out who the victim is communicating with and when. That's quite problematic for an app focusing on protecting your conversations from snooping third parties ...

See you on Signal!

/Ola Flisbäck

Stalking anyone on Telegram

According to the Telegram CEO Pavel Durov, earlier this year the Telegram app was delivering 2 billion messages daily and had 62 million monthly active users. Possibly some of that size can be attributed to Telegram's focus on security and privacy. The Telegram FAQ focuses quite a bit on that and for instance states:

After having read that I was surprised to see the amount of metadata received from my contacts. Most of the metadata is not directly visible in the web and mobile clients, but using a third party client such as vysheng's CLI client any received metadata is displayed:

The Telegram android app sends a notification to all contacts when it becomes or stops being the "foreground" app on the device. Using that information alone it's at times easy to make guesses about who's talking to who if you have several contacts in common with a "victim". An "attacker" will sometimes see the victim and another contact taking turns going active/inactive as they pass messages back and forth.

Add anyone to your Telegram contacts

The metadata may be a considerable problem in itself but what makes it worse is that Telegram does not require contacts to mutually agree that they should be connected! As long as an attacker know the phone number of the victim and add it to the android contacts the victim will show up as a Telegram contact and the attacker will automatically subscribe to the victim's metadata. As a bonus the victim will not be notified in any way and the attacker will not show up among the victim's Telegram contacts.

Putting it together

The attacker then takes guesses at what Telegram contacts the victim may have. Adding them as android contacts makes them automatic Telegram contacts and the metadata analysis can begin. Using the metadata the attacker may have a good chance of figuring out who the victim is communicating with and when. That's quite problematic for an app focusing on protecting your conversations from snooping third parties ...

See you on Signal!

/Ola Flisbäck

Pirated Courses on Udemy

27 November 2015 - 4:56pm
How Udemy Is Profiting From Piracy

I watched a little drama unfold over the last few days involving Troy Hunt, a security specialist and a “tall, fine-looking Australian” according to some mutual friends:

Classic. This stuff happens. Especially in the video training world. When I ran Tekpub it would take (typically) a day or two (sometimes just a few hours) for the torrents to hit. It’s a price of doing business.

But I had never seen a business actually profiting from piracy. Not until Troy mentioned Udemy. And then I wondered: is any of my stuff up there?

So headed over and even as I entered the search term for the first video I could think of (a MeteorJS course I did for Pluralsight) I knew what I would find:

Yep. That’s mine.

It was submitted by “Robert C”. How clever:

I started to dig a little more but then stopped. I know what’s in there. It took me all of three seconds to find the first title that popped in my head.

A Very Shady Business

As I mention, piracy happens. But I’ve never seen it rewarded so openly… so brazenly. Yes I’m sure there are plenty of good courses at Udemy, but there are clearly a HUGE NUMBER of pirated ones.

The question is: does Udemy know this?

The answer: Of course they do. Why do I say that? Because I filled this little box with four letter words:

Udemy’s way of dealing with copyright problems.

The fun thing is you have to be logged in to report abuse. Isn’t that neat? And sleazy?

Can’t think of a reason to have a box like this next to the video itself unless you know copyright infringement is part of your business plan.

How do these people go to work every day and feel good about it?

I think this is probably their only measure of control in place. It has to be. My videos are watermarked and I mention Pluralsight throughout. Anyone doing any kind of reasonable copyright checks would see that.

TheNextWeb wrote about Troy’s story today but clearly — the story is bigger than this. Udemy is full of pirated material and it clearly has no process for checking copyright.

I think that sucks. I think it sucks because I work really hard on these videos and someone just stole it and is selling it on Udemy… and Udemy not only let them do this, they’re encouraging others to do the same.

So hey! Video Pirates of the world! You don’t need to just give your pirated shit away on BitTorrent — take it to Udemy and make some money.

I sincerely hope Udemy is flooded with pirated content. Lawsuit fodder.

Let the games begin.

0 A.D.: A free, open-source game of ancient warfare

27 November 2015 - 2:10pm

Wildfire Games, an international group of volunteer game developers, proudly announces the release of “0 A.D. Alpha 19 Syllepsis”, the nineteenth alpha version of 0 A.D., a free, open-source game of ancient warfare. This alpha features building and siege engine capture, a new pathfinder, visual replay and more!

Easy Download and Install

Download and installation instructions are available for Windows, Linux and Mac OS X. 0 A.D. is free of charge and always will be. Although you might find some people selling copies of 0 A.D., either over the internet or on physical media, you will always have the option to download 0 A.D. completely gratis, directly from the developers.

Moreover, you can redistribute the game and modify it freely as long as you abide by the GPL. You can even use parts of the art and sound for your own projects as long as you abide by CC BY-SA. No “freemium” model, no in-game advertising, no catch.

New Gameplay Features in This Release
  • Building and Siege Engine Capture: Non-siege units can now capture buildings and siege engines. When you hover your mouse over an entity that can be captured, two crossed flags appear so you can issue a “capture” order. If you do, you will start gaining some of that entity’s “capture points” at a certain rate. (On screen, you will see your color taking up a greater proportion of the width of the entity’s capture bar.) Once you gain all of these points, you capture the entity, and it is yours! (Note: You can still specify a regular attack on buildings and siege engines by holding CTRL when right-clicking on a target.) There is currently no animation for the units which are capturing, that will be added in a later version.
  • New Victory Modes: “Conquest Structures” (destroy or capture all enemy structures to win) and “Conquest Units” (destroy all enemy units to win).
  • Ceasefire Game Mode: The game can be set so that all players are completely unable to attack their enemies for a predetermined time at the start of a game.
  • Attack Coordination: Players can request allies (including bots) to attack a specific enemy by clicking a button next to the player name in the diplomacy window. Also, Petra AI now supports attack coordination.
  • Petra AI now warns its allies when it needs a tribute and lets them know when it advances to a new phase.
  • The Ptolemaic lighthouse now has its special feature implemented: It reveals the shore on the entire map.
  • New Skirmish Maps: Tuscan Acropolis (for 4 players; map preview), Northern Island (for 2 players; map preview), and Alpine Mountains (for 3 players; map preview).
Graphics and User Interface
  • Increased Maximum Map Height: The engine now supports an eight times greater range of terrain heights, allowing for the creation of maps with more diverse and impressive landscapes.
  • Visual Replay: Re-run a game and understand what took place in real time.
  • Aura Visualization: Units affected by an aura are now marked with an icon when the aura giver is selected.
  • New animals: New mastiff and wolfhound units have been added as well as a new rhinoceros.
  • The Roman units now have voices in Latin. Voice actors and people with knowledge in ancient languages are invited to contribute more voices to the game!
Under the Hood
  • New Pathfinder: The pathfinder is the component of the game engine that picks a route for a unit to move along from its current location to its target location, so that it does not collide with other units or with structures or with impassable terrain. The new pathfinder improves performance, but at the same time, it also introduces some new bugs. We have done as much as we could to ensure it was bug-free, but please report anything unusual!
  • XML Validation: In 0 A.D., the behavior of units, buildings and other world objects is defined by their components, such as cost, health and more. All of these are described in XML files. The “grammar” of the XML files is now checked for correctness before being used by the game engine, which helps prevent technical problems.
  • The generic Hellenic and Celtic factions have been removed.
  • Linux users, please be advised that SDL2 is now enabled by default on Linux.
Getting Support

Please see the “Get Support” page on our website to find ways to get help from the active and friendly 0 A.D. community.

We are well aware there is some room for improvement in 0 A.D. Some known issues are: Lag, visual glitches or textures not loaded, missing animations and more. When you provide feedback, we would ask you to focus on some of the other points that could be improved. Thanks.

Please Contribute!

We are seeking volunteer contributors in programming, art, sound, documentation and more. Programmers are especially welcome and can get started immediately.

Interested? Log onto #0ad-dev on QuakeNet on IRC and meet the developers. Also, you are invited to register on our forums and start participating!

Why “Syllepsis”?

We name our releases according to development status (“Alpha” or “Beta”), successive release number (1, 2, 3, …) and a word relating to the ancient world, in alphabetical order (“Argonaut” for A, “Bellerophon” for B, …).

“Syllepsis” relates to 0 A.D. as it means “capture” in Greek, and refers to the building and siege engine capture feature implemented in this version (Thanks, bouke!).

For the next alpha, we welcome fan suggestions for words relating to the ancient world beginning with the letter T. Keep it original and related to the time frame portrayed in 0 A.D. (appx. 500 BC – 1 BC)!

Subscribe to 0 A.D. Development News

Contact info for press, bloggers, etc.: aviv@wildfireATHENSgames.com without the capitalized name of a Greek city.

Paris Attacks Plot Was Hatched in Plain Sight

27 November 2015 - 1:02pm
Updated Nov. 27, 2015 4:18 a.m. ET

BOBIGNY, France—Three days before the attacks that ripped through Paris, Djazira Boulanger handed the keys to her row house, across the street from a kindergarten, to a guest who had booked it over the website Homelidays.com. His name was Brahim Abdeslam.

She didn’t know that Mr. Abdeslam was a central figure in plotting the deadly assault. As Ms. Boulanger tended to her two young children at home, authorities say Mr. Abdeslam and a band of cohorts were down the street preparing weapons for an assault on the Stade de France and Paris’s nightlife district.

“Did I suspect something was wrong? Not at all,” Ms. Boulanger said.

A day after he checked in, Mr. Abdeslam’s younger brother, Salah, pulled up to the roadside hotel Appart’City on the southern outskirts of Paris, according to staff, to claim reservations he made on Booking.com—also under his own name. The rooms were for another set of gunmen in the attacks: those assigned to mow down spectators inside the Bataclan concert hall.

Prosecutors suspect the brothers were preparing the logistics for Abdelhamid Abaaoud, the alleged architect of the massacres, to arrive in Paris and swiftly mount one of the deadliest terror attacks in French history. Brahim would later blow himself up during the attacks, while Salah is now the target of an international manhunt.

Mr. Abaaoud was the kind of adversary France had dreaded since the Syrian conflict began drawing European nationals in droves. Mr. Abaaoud—who would die several days after the Paris attacks in a police raid—drew on his experience as a battlefield logistical officer in Syria to launch a guerrilla-style ambush on unarmed civilians in the French capital.

The account emerging from French officials, witnesses and those who interacted with the suspected terrorists shows how the operation hinged on Mr. Abaaoud’s ability to use the tools of everyday modern life to lay the groundwork for the massacre. The ease with which he and his teams moved—all while avoiding detection by France’s security apparatus—suggests the challenges in identifying would-be terrorists and preventing further attacks in the fluid, digital and transnational world of today, especially when they are European citizens.

The array of car rentals, cellphones and online lodging reservations allowed Mr. Abaaoud to organize his militants as separate cells to ensure the plot wouldn’t unravel if one of the teams was compromised. Likewise, Mr. Abaaoud exploited Europe’s porous border system, sneaking stadium bombers into the continent amid the crush of Syrian refugees washing over Greece and tapping European nationals who could wield their own passports to move freely about the region.

Mr. Abaaoud was a native of Molenbeek, a heavily Muslim, working-class neighborhood of Brussels. In 2010, he and Salah Abdeslam, who had lived a few blocks away, had been convicted of breaking into a garage. The men served a prison sentence together.

By 2013, Mr. Abaaoud had become a more observant Muslim, growing out his beard, said Alexandre Chateau, his lawyer. The next January, he took off to northern Syria with his 13-year-old brother, Younes, according to his parents’ lawyer. German authorities flagged Mr. Abaaoud’s departure at Cologne-Bonn airport for Turkey, Europe’s gateway to Syria, because he was on an EU watch list. The entry, however, didn’t direct authorities to detain him.

In Syria, Mr. Abaaoud rose quickly as a recruiter of European fighters, according to French officials. He also honed a reputation for logistical prowess as the Islamic State official in charge of supplies for fighters in operations in Syria’s oil-rich province of Deir Ezzour, according to an Islamic State fighter who met Mr. Abaaoud in that role. He was in charge of procuring weapons and transport for front-line fighters, the fighter said.

In January 2015, Mr. Abaaoud surfaced in Athens, where he made a flurry of phone calls to Belgium, according to people familiar with the matter. In a purported interview with Islamic State’s in-house magazine Dabiq in February, Mr. Abaaoud said he was stockpiling a cache of automatic weapons at the time. Investigators suspect the purpose of the weaponry was to arm the crew of operatives he was assembling to carry out attacks on Europe.

Salah Abdeslam, Mr. Abaaoud’s acquaintance, handled logistics, traveling to the Italian port of Bari in early August where he and another man took a ferry to Patras, Greece, Italian officials said. “We are talking about citizens with regular European passports and with the right to travel freely,” Italian Interior Minister Angelino Alfano said.

Mr. Abaaoud also tapped two French nationals who had both recently spent time in Syria: Samy Amimour, a former Paris bus driver, and Omar Ismail Mostefai, a petty criminal who had been on a watch list for radicalization since 2010. Both were assigned to shoot up the Bataclan concert hall.

For the planned suicide bombing at the Stade de France, Mr. Abaaoud turned to Bilal Hadfi, a French national who had run off to fight in Syria as a teenager. Mr. Hadfi had since returned to Europe without telling his parents, who wondered if he had died fighting in Syria.

The two other stadium bombers arrived in Europe taking a more clandestine route. On Oct. 3, two men arrived on the Greek island of Leros, blending in with the scores of refugees that were washing up on Greece’s shores. One of the men carried a phony passport bearing the name Ahmad Almohammad. Officials haven’t identified the two men.

Days earlier, the Abdeslam brothers had sold Les Beguines, a bar in Molenbeek known for brawling and drug use, according to public records.

The brothers, acting as the group’s bank, started spending on logistics in Brussels and Paris. With more than nine people involved in the operation, they faced a transportation and housing challenge. In addition to a Seat-brand car, Salah Abdeslam rented a Volkswagen VLKAY -1.01 % and a Renault from two different rental agencies in Brussels. Rental companies in Belgium don’t vet clients as long as their driver’s license, government identification and credit cards are valid.

As the terrorists came together, the Abdeslam brothers arranged lodgings in the dilapidated outskirts of Paris. The brothers shuttled back and forth across a Franco-Belgian border that, under European Union treaties, is little more than a line on the map.

On Nov. 10, they arrived at Ms. Boulanger’s row house in the northeastern suburb of Bobigny, a 20-minute drive from the heart of Paris. The place came equipped with bunk-beds that easily accommodated the six operatives planning to attack the Stade de France and the capital’s busy nightlife district.

Across town, the Appart’City hotel was well-suited to allow the Bataclan team to move about without bringing attention to themselves. On Nov. 11, Salah Abdeslam checked into rooms 311 and 312 at the end of a hallway at the Appart’City hotel, where clients have access to a secondary stairwell that leads directly to a parking lot without ever passing the front desk. The two-star hotel doesn’t require guests to register their cars to use the parking lot. Nor does it have security cameras.

Salah Abdeslam didn’t stay in France for long. He raced back to Belgium to collect the attackers, according to video footage of him at a gas station outside Paris and, hours later, in Brussels. In the predawn hours of Nov. 12, a convoy of three cars left Brussels, setting a course for Paris.

On Friday, Nov. 13, Mr. Abaaoud’s terror cells launched the attacks. At 7:40 p.m. Messrs. Mostefai, Amimour and a third unidentified man steered the Volkswagen hatchback out of the Appart’City bound for the Bataclan concert hall.

On the other side of Paris, the men in Bobigny piled into two cars with different destinations. The black Seat, according to the Paris prosecutor, was carrying Mr. Abaaoud. The hatchback left the suburb at 8:38 p.m., ferrying the Belgian Islamic State operative and two other men to an area filled with restaurants and bars in the 10th and 11th districts of Paris, where they began a shooting spree.

The Renault wasn’t going nearly as far. Investigators believe Salah Abdeslam drove a team of three suicide bombers to the gates of a packed Stade de France, where French President François Hollande and 80,000 other fans were sitting down to a match between the French and German national soccer teams.

As explosions and gunfire began to ring out across the city, the Volkswagen crept up to the Bataclan. Before storming in, one militant sent a text message that investigators would later recover from a phone discarded near the theater. “We’re going for it,” the message read. “It’s starting.”

An hour after Mr. Abaaoud finished shooting up restaurants, he emerged from a metro station in the 12th district, according to data police pulled from his cellphone. He headed west toward the sound of sirens, his path zigzagging as he returned to the scene of his crimes.

For two hours after the massacre ended, prosecutors say, Mr. Abaaoud surveyed his handiwork, at one point blending in with panicked crowds and bloodied victims streaming from the Bataclan. Then, at 12:28 a.m., he went dark.

Write to Stacy Meichtry at stacy.meichtry@wsj.com and Joshua Robinson at joshua.robinson@wsj.com

Pg_paxos: High Availability Data Replication for PostgreSQL

27 November 2015 - 12:43pm

The Paxos algorithm is a powerful building block for building highly available distributed systems. Paxos can be seen as a function paxos(k,v) that returns the same value on all servers in a group for a certain key (k), and the value is one of the inputs (v). Paxos is most commonly used to implement log replication through a technique called Multi-Paxos. In Multi-Paxos, nodes call paxos using the indexes in a log as keys and state changes (e.g. 'set primary to node2') as values. Using this technique, data can be kept consistent and available across multiple servers, even if some of them fail.

We recently started exploring Paxos as a technique for doing automated fail-over of PostgreSQL servers and wrote a simple implementation of Paxos and Multi-Paxos including basic support for membership changes in about 1000 lines of PL/pgSQL. We found PL/pgSQL to be an excellent tool for implementing Paxos, because the necessary transactional semantics are a natural part of the language. Other consensus algorithms would have been harder to implement in this way, because they rely on timers and other background tasks. 

The pg_paxos extension demonstrates how the Paxos functions can be used for fault-tolerant, consistent table replication in PostgreSQL. While a replicated table has high write and read latencies due to the network round-trips involved, it can be very useful for applications such as automated fail-over. We plan to add optimisations such as the ones applied in Google Megastore to reduce the overhead and develop the extension further to be able to replace components like etcd or Zookeeper.

A Simulation in Emoji

27 November 2015 - 12:22pm

HTTP/1.1 200 OK Server: GitHub.com Date: Sat, 28 Nov 2015 04:21:53 GMT Content-Type: text/html; charset=utf-8 Content-Length: 3164 Last-Modified: Wed, 25 Nov 2015 22:21:49 GMT Access-Control-Allow-Origin: * Expires: Sat, 28 Nov 2015 04:31:53 GMT Cache-Control: max-age=600 Accept-Ranges: bytes X-GitHub-Request-Id: 42E42DB4:26B0:1A7CF233:56592BE1

How the Atari ST Almost Had Real Unix

27 November 2015 - 12:04pm


It was 4:00 in the afternoon, and time to walk to the dump, out past the sewage plant, and talk about hard problems.

I’ve had some of my best ideas while out walking. I’m 6’4″ with long legs, and I walk fast. I find when my legs are occupied I can let stuff sift through my head and sort of bounce around until they fall into place. One must leave the cell phone and music player and other distractions behind or you won’t get work done; just get out there and let the ground travel past your feet, and things get solved.

That, or just go out for a walk with cow-orkers and enjoy the day, and bullshit about stuff.

The latest problem I was working out was how to run Unix on the Atari ST. The Tramiels had somehow wrangled a license for AT&T’s SVR-something-or-other version of Unix (might have been SVR3, but this was in the bad old days when AT&T was actively fucking up Unix, and it could have been just about any version, including SVR666). The license was for a mind boggling, nay, jaw-dropping ten bucks a seat. The problem was that the ST didn’t have any kind of memory management hardware, just a raw CPU flinging real addresses at naked DRAM, and the machine’s cheap-ass vanilla 68000 was incapable of recovering from a fault unless you cheated.

[What’s that about Linux? Dear child, Linus was probably not out of whatever they use for high school in Finland. All we had in the market was 4.2bsd running on Vaxen, 68K-based Suns with a ton of hardware to work around the 68000 limitation re faulting, and a whole running field of wannabes that would sink without a trace in five years. Oh, and some screwed up AT&T workstations with monochrome graphics and UIs that curdled your eyeballs and left you wishing AT&T had simply stuck with making phones.]

The hardware folks were convinced that grafting an MMU into the ST was impossible; in theory you could still run something like Unix, but with no memory protection and no way to easily grow and shrink a process’s address space a straight-forward port of Unix would be glacial and prone to crashing really badly. The hardware guys were mostly right; the 68000 wasn’t capable of handling a page fault (it didn’t save enough information on its exception frames to restart all cases of a faulted instruction). Motorola didn’t offer an MMU chip anyway (the 68020 didn’t exist yet, and the sticker shock of its optional external MMU meant that only Apple folks could afford it, and it was still optional on most Macs for several years). Furthermore, the memory system of the ST wouldn’t tolerate the delays that a traditional MMU would incur; the ST’s DRAMs were being thrashed ten or fifteeen nanoseconds under spec (“You have to understand,” said our hardware guys, “DRAMs are really analog devices,” and I’m sure a DRAM designer somewhere felt cold and shivery all of a sudden, and didn’t know why).

To run Unix effectively we needed some hardware that was very fast, that was simple enough to put into a minor spin of the ST’s memory controller with little project risk, and that would still provide some kind of memory relocation and protection. The ability to have separate address spaces to isolate processes would be good, too.

“If you can come up with something that takes about a gate delay, I’ll put it in,” said John, the memory controller guy. He seemed dubious, but willing to listen.

I went for a bunch of walks.

– – – –

In the early 80s, eastern Sunnyvale bordered southern San Fransisco Bay with a landfill hill (a large, long mound maybe a hundred feet high), and a sewage treatment plant just beyond. Beyond these were settling ponds for the sewage, separated by a large number of wandering dikes upon which were set miles upon miles of paths for walking. I never exhausted the paths. It was easy to get your heart pumping and your legs swinging and let your head fly off into some tough technical nut. I never really noticed any smell; maybe once or twice. The winter rains washed the stink out of the air.

There were birds out there by the thousands, and any number of rodents. I saw an enormous heron once and realized why my parents had been so excited to see them nest in a marsh we’d lived near in Ohio.

We could also get a good view of planes at Moffet Field. Occasionally a U2 would take off, shaking the ground slightly as it roared into the stratosphere to look (we were told) for pot fields in northern California, saving the world for democracy.

Then the path would loop back, and I’d bounce some ideas off of people. Eventually we got it.

– – – –

The MMUs I knew about did page table walks of a multi-level tree; those multiple indirections implied complex, stateful and slow machinery. There was no room in the ST’s memory controller for the caches required to make a table-based system perform reasonably, even if the gate count of table-lookup hardware had been possible. The ST was no VAX. We had to pay dearly for chip area, schedules were tight, and DRAM timing was even tighter. Nobody wanted to pay for a feature they’d never use.

Non-MMU-based systems used base-and-bounds; a favorite technique in mainframes and minis from the 60s and 70s. We could get protection by checking user accesses against limit registers, a pretty cheap operation, but that wouldn’t get you relocation. To do that you had to muck with the address bits, and do an addition.

The problem was, there wasn’t time to do an addition with the necessary carry-propagation on every single address issue, not to mention the gate count.

So how does a typical Unix process grow? The text and data are at the bottom of the address space and don’t move; the bss follows those, and grows up via the “brk.” The stack grows down. That’s it. Very simple, very hippy 70s.

So imagine something really minimal, like replacing a bunch of address lines with a chosen constant value for user-mode accesses. Leave everything untouched for supervisor accesses. That’s it, that’s your total relocation. It’s really simple to implement in hardware, just a latch and some muxes to choose which address lines go through and which get replaced.

For address space growth you have another register, a mask or a bit count, that checks to see if some number of the issued upper address bits are either all zero or all one. You start the stack at 0xfffffffe and grow down. You start the bss low and grow up. A variable number of bits in the middle of each address are simply substituted. If the upper N bits aren’t all 0000…00 or 11111…11 then you generate a fault.

Now you have a system that both relocates addresses and handles growth in two directions in powers of two. You use throwaway instructions to do stack growth probing (dud sequences that don’t need to be restarted if they fault), and that needs a little compiler work, but it’s not too bad. Processes are tiled in memory at power-of-two addresses, so there’s more physical copying going on than you probably like when stuff grows, but again, it’s not too bad. Welcome to the world of doesn’t-totally-suck, and probably runs rings around a PDP-11 despite its limitations. AT&T SVR-N didn’t have paging anyway (like I said, they should have stuck with phones).

– – – –

John Horton, the memory chip guy, actually did this hardware for the Mega ST; I don’t know if it’s documented, or if he had to sneak it in or not. I do know that it was not used for Unix in my time at Atari; the deal with AT&T expired before the hardware existed, and frankly, supporting Unix probably would have been a massive effort, and one that the Atari software group would have been unable to adequately support. I vaguely recall some Unix-a-like environments for the ST, but I pretty much lost interest in the ST after I left Atari in 1987.

I recall talking about this scheme with John Gilmore, who took a kind interest and asked some good probing questions. We had some great conversations at some otherwise strained hacker dinners in Berkeley. (I’ll talk about South Bay versus North Bay geeks some other time…).

Incremental Query Processing on Big Data Streams

27 November 2015 - 11:25am
403 Forbidden

Sadly, your client "FTRF: Friendly robot/1.3" violates the automated access guidelines posted at arxiv.org, and is consequently excluded.

If you believe this determination to be in error, see http://arxiv.org/denied.html for additional information.

The 68000 Wars, Part 4: Rock Lobster

27 November 2015 - 10:34am

In the years following Jack Tramiel’s departure, Commodore suffered from a severe leadership deficit. The succession of men who came and went from the company’s executive suites with dizzying regularity often meant well, were often likable enough in their way. Yet they were also weak-willed men who offered only timid, conventional ideas whilst living in perpetual terror of the real boss of the show, Commodore’s dilettantish chairman of the board and interfering largest stockholder Irving Gould.

The exception that proves the rule of atrocious management is Thomas Rattigan, the man who during his brief tenure saved Commodore and in the process the Commodore Amiga from an early death. Rattigan wasn’t, mind you, a visionary; he never got the time to demonstrate such qualities even if he did happen to possess them. His wasn’t any great technical mind, nor was he an intrinsic fan of computers as an end unto themselves; in common with a rather distressing number of industry executives of the time, Rattigan, like Apple’s John Sculley a veteran of Pepsi Cola, seemed to take a perverse pride in his computer illiteracy, saying he “never got beyond the slide rule” and not even bothering to place a computer on his desk. He may not have even been a terribly nice guy; the thousands of employees he laid off, among them virtually the entire team that had once been Amiga, Incorporated, certainly aren’t likely to invite him to dinner anytime soon. No, Rattigan was simply competent, and carried along with that competence a certain courage of his own convictions. That was more than enough to make him stand out from his immediate predecessor and his many successors like the Beatles at a battle of the bands.

Thomas Rattigan

Rattigan was appointed President and Chief Operating Officer of Commodore International on December 2, 1985, and Chief Executive Officer on April 1, 1986, succeeding the feckless former steel executive Marshall Smith, whose own hapless tenure would serve as a blueprint for most of the Commodore leaders not named Rattigan who would follow. After replacing Tramiel in February of 1984, Smith had fiddled while Commodore burned, going from the billion-dollar face of home computing in North America to the business pages’ favorite source of schadenfreude, hemorrhaging money and living under the shadow of a gleeful deathwatch. The stock had dropped from almost $65 per share at the peak of Tramiel’s reign to less than $5 per share at the nadir of Smith’s. It was Rattigan, in one of his last acts before assuming the mantle of CEO as well as president, who negotiated the last-ditch $135 million loan package that gave Commodore — in other words, Rattigan himself — a lease on life of about one year to turn things around.

Some of the changes that Rattigan enacted to effect that turnaround were as inevitable as they were distressing: the waves of layoffs and cutbacks that had already begun under Smith’s reign continued for some time. Unlike Smith, however, Rattigan understood that he couldn’t cost-reduce Commodore back to profitablity.

The methods that Rattigan used to implement triage on the profit side of the ledger sheet were unsexy but surprisingly effective. One was entry into the burgeoning market for commodity-priced PC clones, hardware that could be thrown together quickly using off-the-shelf components and sold at a reasonable profit. Commodore’s line of PC clones would never do much of anything in North America — the nameplate was too associated with cheap, chirpy home computers for any corporate purchasing manager to glance at it twice — but it did do quite well in Europe; in some European countries, especially West Germany, the Commodore brand remained as respectable as any other.

Rattigan’s other revenue-boosting move was even more simple and even more effective. Commodore’s engineers had been working on a new version of the 64. Dubbed internally the 64CR, for “Cost Reduced,” it was built around a redesigned circuit board that better integrated many of the chips and circuitry using the latest production processes, resulting in a substantial reduction in the cost of production. The chassis and case were also simplified — for example, to use only two instead of many different types of screws. While they were at it, Commodore dramatically changed the look of the machine and most of its common peripherals to match that of the newer Commodore 128, thus creating a uniform appearance across their 8-bit line. As Rattigan said, “I think you’ve got to give people the opportunity not to have a black monitor, a green CPU, and a red disk drive.”

Commodore 64C

All of which was very practical and commonsensical. Looking at the new machine, however, Rattigan saw an opportunity to do something Commodore had never done before: to raise its price, and thereby to recoup some desperately needed profit margin. This really was a revolutionary thought for Commodore. Ever since releasing their VIC-20 model that had created the home-computer segment in North America, Commodore had competed almost entirely on the basis of offering more machine for less money than the other folks — an approach that did much to create the low-rent image that would dog the brand for the rest of the company’s life. Commodore had always kept their profit margins razor thin in comparison to the rest of the industry, trusting that they would, as the old saying/joke goes, “make it up in volume.” Now, though, Rattigan realized that the 64 had much more than price alone going for it. Almost everyone buying a 64 in 1986 was motivated largely by the platform’s peerless selection of games. Most, he theorized, would be willing to pay a little more than what Commodore was currently charging to gain access to that library. Thus when Commodore announced the facelifted 64 — now rechristened simply the 64C for obvious reasons — they also announced a 20 percent bump in its wholesale price. To ease some of the pain, they would bundle with it something called GEOS, an independently developed graphically-oriented operating environment that claimed to turn the humble 64 into a mini-Macintosh. (It didn’t really, of course, but it was a noble, impressive effort for a machine with a 1 MHz 8-bit processor and 64 K of memory.) Anyone who’s been around manufacturing at all will understood just what a huge difference a price increase of that magnitude, combined with a substantial reduction in manufacturing cost, would mean to Commodore’s bottom line if customers did indeed prove willing to continue buying the new model in roughly the same numbers as the old. Thankfully, Rattigan’s instincts proved correct. The 64C picked up right where the older model had left off, a brisk — and vastly more profitable — seller.

Sometimes, then, the simplest fixes really are the most effective. Taken together with the cost-cutting, these two measures returned Commodore to modest profitability well before Rattigan’s one-year deadline expired. Entering 1987, the company looked to be in relatively good shape for the short term. Yet questions still swirled around its long-term future. If Commodore didn’t want to accept the depressing fate of becoming strictly a maker of PC clones for the European market, they needed a successful platform of their own that could become the successor to the 64, which was proving longer lived than anyone had ever predicted but couldn’t go on forever. That successor had to the Amiga. And therein lay problems.

The Amiga was in a sadly moribund state by the beginning of 1987. The gala Lincoln Center debut was now eighteen months in the past, but it felt like an eternity. The excitement with which the press had first greeted the new machine had long since been replaced by narratives of failure and marketing ineptitude. Commodore had stopped production of the Amiga in mid-1986 after making just 140,000 machines, yet was still able to fill the trickle of new orders from warehouse stock. Sure, some pretty good games had been made for the Amiga, at least one of which was genuinely groundbreaking, but with numbers like those how long would that continue? Already Electronic Arts had quietly sidled away from their early declarations that together they and the Amiga would “revolutionize the home-computer industry,” turning their focus back to other, more plebeian platforms where they could actually sell enough games to make it worth their while. Ditto big players in business and productivity software like Borland, Ashton-Tate, and WordPerfect. The industry at large, it seemed, was just about ready to put a fork in Commodore’s erstwhile dream machine.

The Amiga’s most obvious failing was one of marketplace positioning. Really, just who was this machine for? There were two obvious markets: homes, where it would make the best games machine the world had yet seen; and the offices of creative professionals who could make use of its unprecedented multimedia capabilities. Yet the original Amiga model had managed to miss both targets in some fairly fundamental ways. Svelte and sexy as it was, it lacked the internal expansion slots and big power supply necessary to easily outfit it with the hard drives, memory expansions, accelerator cards, and genlocks demanded by the professionals. Meanwhile its price of almost $2000 for a reasonably complete, usable system was far too high for the home market that had so embraced the Commodore 64. Throw in horrid Commodore marketing that ignored both applications in favor of positioning the Amiga as some sort of challenger to the PC-clone business standard, and it was remarkable that the Amiga had sold as well as it had.

If there was a bright spot, it was that the Amiga’s obvious failing had an equally obvious solution: not one but two new models, each perfectly suited for — and, hopefully, marketed toward — one of its two logical customer bases. Rattigan, industry neophyte though he was, saw this reasoning as clearly as anyone, and pushed his engineers to deliver both new machines as quickly as possible. They were officially announced via a low-key, closed-door presentation to select members of the press at the January 1987 Winter Consumer Electronics Show. The two new models would entirely replace the original, now retroactively dubbed the Amiga 1000. The Amiga 2000 would be the big, professional-level machine, with a full 1MB of memory standard — four times that of the 1000 — and all the slots and expansion possibilities a programmer, artist, or video-production specialist — or, for that matter, a game developer — could possibly want.

Amiga 2000

But it was the Amiga 500 that would become the most successful Amiga model ever released, as well as the heart of its legacy as a gaming platform. Designed primarily by George Robbins and Bob Welland at Commodore’s West Chester, Pennsylvania, headquarters — the slowly evaporating original Los Gatos Amiga team had little to do with either of the new models — the 500 was code-named “Rock Lobster” during development after the B-52’s song (reason enough to love it right there if you ask me). Key to the work was a re-engineering of Agnus, the most complex of the Amiga’s custom chips, to make it smaller, simpler, and cheaper to manufacture; the end result was known as “Fat Agnus.” That accomplished, Robbins and Welland managed to stuff the contents of the 1000’s case into in an all-in-one design that looked like a bulbous, overgrown Commodore 128.

Amiga 500

The Amiga 500 wasn’t, especially in contrast to the 1000, going to win any beauty contests, but it got the job done. There was a disk drive built into the side of the case, a “trap door” underneath to easily increase memory from the standard 512 K to 1 MB, and an expansion port in lieu of the Amiga 2000’s slots that let the user add peripherals the old-fashioned Commodore way, by daisy-chaining them across the desktop. Best of all, a usable system could be had for around $1000, still a stratum or two above the likes of a 64 or 128 but nowhere near so out of the reach of the enthusiastic gamer or home hacker as had been the first Amiga. Compromised in some ways though it may have been from an engineering standpoint, enough to prompt a chorus of criticism from the old Los Gatos Amigans, the Amiga 500 was a brilliant little machine from a strategic standpoint, the smartest single move the post-Tramiel Commodore would ever make outside of electing to buy Amiga, Incorporated, in the first place.

But unfortunately, this was still Irving Gould’s Commodore, a company that seldom failed to follow every good decision with several bad ones. Amiga circles and the trade press at large were buzzing with anticipation for the not-yet-released new models, which were justifiably expected to change everything, when word hit the business press on April 23 that Thomas Rattigan had been unceremoniously fired. Like the firing of Jack Tramiel three years before when things were going so very well, it made and makes little sense. Gould would later say that Rattigan had been fired for “disobeying the chairman of the board” — i.e., him — and for “gross disregard of his duties,” but refused to get any more specific. Insiders muttered that Rattigan’s chief sin was that of being too good at his job, that the good press his decisions had been receiving had left Gould jealous. Just a couple of weeks before Rattigan’s firing, Commodore’s official magazine had published a lengthy interview with him, complete with his photo on the cover. To this Gould was reported to have taken grave exception. Yet Rattigan hardly comes across as a prima donna or a self-aggrandizer therein. On the contrary, he sounds serious, thoughtful, grounded, and very candid, explicitly rejecting the role of “media celebrity” enjoyed by Apple’s John Sculley, his former colleague at Pepsi: “When you have lost something in the range of $270 million in five quarters, I don’t think it’s time to be a media celebrity. I think it’s time to get back to your knitting and figure out how you’re going to get the company making money.” Nor does he overstate the extent of Commodore’s turnaround, much less take full credit for it, characterizing it as “tremendous improvement, but not an acceptable performance.” It seems hard to believe that Gould could be petty enough to object to such an interview as this one. But at least one more piece of circumstantial evidence exists that he did: Commodore Magazine‘s longtime editor Diane LeBold was forced out of the company on Rattigan’s heels, along with other real or perceived Rattigan loyalists. It made for one hell of a way to run a company.

True to form of being less of a pushover than Gould’s other executive lapdogs, Rattigan soon filed suit against Commodore for $9 million, for terminating his five-year employment contract four years early for no good reason. Commodore promptly counter-sued for $24 million, the whole ugly episode overshadowing the actual arrival of the Amiga 500 and 2000 in stores. After some five years of court battles, Rattigan would finally be awarded his $9 million — yes, every bit of it — just at a time when everything was starting to go sideways for Commodore and they could least afford to pay him.

With Rattigan now out of the picture — Gould had had him escorted off the campus by security guards, no less — Gould announced that he would be taking personal charge of day-to-day operations, a move that filled no one at the company other than his hand-picked circle of sycophants with any joy. But then, for Gould day-to-day oversight meant something different that it did for most people. He continued to live the lifestyle of the jet-setting super-rich, traveling the world — reportedly largely to dodge taxes — and conducting business, to whatever extent he did, by phone. Thus Commodore was not only under a cloud of rumor and gossip at this critical moment when these two critical new machines were being introduced, but they were also leaderless, their executive wings gutted and reeling from Gould’s purge and their ostensible new master who knew where. There was, needless to say, not much in the way of concerted promotion or messaging as the months marched on toward Christmas 1987, the big test of the Amiga 500.

While it didn’t abjectly fail that test, it didn’t really skate through with honors either. On the one hand, Amigas were selling again, and in better numbers than ever before. The narrative of the Amiga as a flop that was soon to be an orphan began to fade, and companies like Electronic Arts began to return to the platform, if not always as a target for first-run games at least as a consistent target for ports. WordPerfect even ported their industry-standard word processor to the Amiga. But on the other hand, the Amiga certainly wasn’t going to become a household name like the 64 anytime soon at this rate. In addition to the nearly complete lack of Commodore advertising, distribution remained a huge problem. Many people who might have found the Amiga very interesting literally never knew it existed, never saw an advertisement and never saw it in a store. Jack Tramiel’s decision to dump the 64 into mass-market channels like Sears and Toys “R” Us had been a breaking of his own word and a flagrant betrayal of his loyal dealers from which Commodore’s reputation had never entirely recovered. Yet it had also been key to the machine’s success; the 64 was available absolutely everywhere during its heyday, an inescapable presence to tempt plenty of people who would never think to walk into a dedicated computer store. Now, though, having laboriously and with very mixed results struggled to rebuild the dealer network that Tramiel had demolished, Commodore refused to do the same with the Amiga 500, even after some of those dealers had started to whisper through back channels that, really, it might be okay to offer some 500s through the mass market in the name of increasing brand awareness and corralling some new users who would quite likely end up coming to them for further hardware, software, and support anyway. But it didn’t happen, not in 1987, 1988, or the bulk of 1989.

The Amiga thus came to occupy an odd position on the American computing scene of the late 1980s, not quite a failure but never quite a full-fledged success either. Always the bridesmaid, never the bride; the talented actor never quite able to find her breakout role; pick your metaphor. Commodore blundered along, going through more of Irving Gould’s sock-puppet executives in the process. Max Toy, unfortunately named in light of the image that Commodore was still trying to shake, took over in October of 1987, to be replaced by Harold Copperman in July of 1989. Meanwhile the two Amiga models settled fairly comfortably into their roles.

Video production became the 2000’s particular strong suit. Amigas were soon regular workhorses on television series like Amazing Stories, Max Headroom, Lingo; on films like Prince of Darkness, Not Quite Human, Into the Homeland; on lots of commercials. If most of this stuff wasn’t exactly the pinnacle of cinematic art, it was certainly more Hollywood work than any other consumer-grade PC was getting. More important, and more inspiring, were the 2000s that found homes in small local newsrooms, on cable-access shows, and in small one- or two-person video-production studios. Just as the Macintosh had helped to democratize the means of production on paper via desktop publishing, the Amiga was now doing the same for the medium of video, complete with a new buzzword for the age: “desktop video.”

The strong suit of the Amiga 500, of course, was games. At first blush, the Amiga might seem a hard sell to game publishers. Even in 1988, after the 500 and 2000 had had some time to turn things around for the platform, a hit Amiga game might sell only 20,000 copies, a major blockbuster by the platform’s terms 50,000. The installed base still wasn’t big enough to support much bigger numbers than these. An only modestly successful MS-DOS game, by contrast, might sell 50,000 copies, while some titles had reportedly hit 500,000 copies on the Commodore 64 alone. Yet, despite the raw numbers, many publishers discovered that the Amiga carried with it a sort of halo effect. Everyone seriously into computer games knew which platform had the best graphics and sound, which platform had the best games, even if some were reluctant to admit it openly. Publishers found that an Amiga game down-ported to other platforms carried with it a certain cachet inherited from its original version. Cinemaware, the premiere Amiga game developer and later publisher in North America, used the Amiga’s halo effect to particularly good commercial effect. All of their big releases were born, bred, and released first on the Amiga. They found that it made good commercial sense to do so, even if they ultimately sold far more copies to MS-DOS and Commodore 64 owners. While it’s true that Cinemaware could never have survived if the Amiga had been the only platform for which they made games, neither could they have made a name for themselves in the first place if the Amiga versions of their games hadn’t existed. Some of the same triangulations held sway, albeit to a lesser extent, among other publishers.

All told, the last three years of the 1980s were, relatively speaking, the best the Amiga would ever enjoy in North America. By the end of that period, with the 64 at last fading into obsolescence, the Amiga could boast of being the number two platform, behind only MS-DOS, for computer games in North America — a distant second, granted, but second nevertheless — while Commodore stood as the number three maker of PCs in North America in terms of units sold, behind only IBM and Apple. And Commodore was actually making money for most of this period, which was by no means always such a sure thing in other periods. But perhaps more important than numbers and marketshare was the sense of optimism. Every month seemed to bring some breakthrough program or technology, while every Christmas brought the hope that this would be the one where the Amiga finally broke into the public consciousness in a big way. To continue to be an Amiga loyalist in later years would require one to embrace Murphy’s Law as a life’s creed if one didn’t want to be positively smothered under all of the constant disappointments and broken promises that could make the platform seem cursed by some malicious higher power. But in these early, innocent times everything still seemed so possible, if only there would come the right advertising campaign, the right change in management at Commodore, the right hardware improvements.

But, ah, Commodore’s management… there lay the rub, even during these good years. Amiga owners watched with concern and then alarm as Apple and the makers of MS-DOS machines alike steadily improved their offerings whilst Commodore did nothing. In 1987, Apple debuted the Macintosh II, their first color model, with a palette of millions of colors to the Amiga’s 4096 and a hot new 16 MHz 68020 CPU inside. Yes, it cost several times the price of even the professional-grade Amiga 2000, and yes, 68020 or no, the Amiga could still smoke it for many animation tasks thanks to its custom chips. But then, even Apple’s prices always came down over time, and everyone knew that their hardware would only continue to improve. That same year, IBM introduced their new PS/2 line, and with it the new VGA graphics standard with about 262,000 colors on offer. More caveats applied, as Amiga fans were all too quick to point out, but the fact remained that the competition was improving by leaps and bounds while Commodore remained wedded to the same core chipset that they had purchased back in 1984. The Amiga 1000 had been a generation ahead of anything else on the market at the time of its release, but, unfortunately, generations aren’t so long in the world of computers. Gould and his cronies seemed unconcerned about or, still more damningly, blissfully unaware of the competition that was beginning to match and surpass the Amiga in various ways. In 1989, IBM spent 10.9 percent of their gross revenue on R&D, Apple 6.7 percent. And Commodore? 1.7 percent. The one area where Commodore did rank among the biggest spenders in the industry was in executive compensation, particularly the salary of one Irving Gould.

For the 1989 Christmas season, Commodore launched what would prove to be their first and last major mainstream advertising campaign for the Amiga 500. The $20 million campaign featured television spots produced by no less leading lights than Steven Spielberg’s Amblin Entertainment and George Lucas’s Lucasfilm. The slogan was “Amiga: The Computer for the Creative Mind.” The most lavish of the spots featured cameos by a baffling grab bag of minor celebrities, including Tommy Lasorda, Tip O’Neill, the Pointer Sisters, Burt Bacharach, Little Richard, and astronauts Buzz Aldrin, Gordon Cooper, and Scott Carpenter. Commodore’s advertising agency announced confidently that 92 percent of Americans would see an Amiga commercial an average of twenty times during November and December. Commodore would even beginning selling 500s through mass-market merchandisers at last, albeit in a limited way, going through Sears and Service Merchandise alone. The campaign was hyped in the Amiga press as a last all-out effort to make that ever-elusive big breakthrough in North America. Sure, it was something they really needed to have done back in 1987, when the 500 first debuted, but at least they were doing it now. That was something, right? Right? In the end, it proved a heartbreaker of the sort with which Amiga fans would grow all too familiar over the years to come: it had no appreciable effect whatsoever. And with that Commodore slipped out of the mainstream American consciousness along with the decade with which their computers would always be identified.

The next year the first of a new generation of unprecedentedly ambitious games arrived — games like Wing Commander, Ultima VI, Railroad Tycoon — that looked, sounded, and played better on MS-DOS machines than they did on Amigas, thanks to the ever-improving graphics cards, sound cards, and new 32-bit 80386 processors in those heretofore bland beige boxes. Cinemaware that same year released Wings, the last of their big Amiga showcases, and then quietly died. The Amiga’s halo effect was no more. Just like that, an era ended.

And yet… well, here’s where things get a little confusing. As the Amiga was drying up as a gaming platform in North America, it was in many ways just getting started in Europe, with most of the classics still to come. Let’s rewind and try to understand how this parallel history came to be.

Commodore had always been extremely strong in Europe, going all the way back to their days as a maker of calculators. Their first full-fledged computer, the PET, had been little more than a blip on the radar in North America in comparison to its competitors the Radio Shack TRS-80 and the Apple II, yet it had fostered a successful and respected line of business computers across the pond. Commodore’s most consistently strong markets then would also proof the strongest of their twilight years: Britain and, especially, West Germany. Both operations were granted much more autonomy than the North American operation, and were staffed by smart people who were much better at selling Commodore’s American computers than Commodore’s Americans were. Germans in particular developed a special affinity for the Commodore brand, one that was virtually free of the home-computer/business-computer dichotomy that Commodore twisted themselves into knots trying to navigate in the United States. In Germany a good home computer was simply a good home computer, and if the same company happened to offer a good business computer, well, that was fine too.

When Commodore’s European leadership looked to the new Amiga 500, they saw a machine sufficient to make the traditional videogame demographic of teenage boys, who were currently snatching up Commodore 64s and Sinclair Spectrums, positively salivate. They unapologetically marketed it on that basis. Knowing what their buyers really wanted the machine for, they quite early on took to bundling together special packages, usually just in time for Christmas, that combined a 500 with a few of the latest hot games. A particular home run was 1989’s so-called “Batman Pack,” which included the game based on the hit Batman movie, a fresh new arcade conversion called The New Zealand Story, the graphically stunning casual flight simulator F/A-18 Interceptor, and, since this was an Amiga after all, the platform’s signature application, Deluxe Paint II. Deluxe Paint aside, there was no talk of video production or productivity of any other stripe, no mention of the Amiga’s groundbreaking multitasking operating system, no naval-gazing discussions of the platform’s place in the multimedia zeitgeist. Teenage boys didn’t want any of that. What they wanted was great games with great graphics, and that’s exactly what Commodore’s European operations gave them. You were just buying a fun computer, a game machine, so you didn’t need to go through a dealer. From the beginning, the Amiga 500 was widely available at all of the glossy electronics stores on European High Streets. The West German operation went even further: they started selling Amigas through grocery stores. Buy an Amiga 500, hook it up to a television, pop in a disk, turn it on, and start playing.

The British and especially the Germans took to the Amiga 500 in numbers that Commodore’s North American executives could only dream of. By early 1988, Commodore could announce that they had sold 500,000 Amigas worldwide, a strong turnaround for a marquee that had been all but left for dead a year before. A rather astonishing 200,000 of those machines, 40 percent of the total, had been sold in West Germany alone; Britain accounted for another 70,000. Even now, with the Christmas season behind them, Commodore West Germany was selling a steady 4000 Amiga 500s every week. A few months later came the simultaneously impressive and depressing news that the total market for Amiga hardware and software in West Germany (population 60 million) was now worth more than that for the United States (population 240 million). And where Germany led, the rest of Europe followed. Eighteen months later, with worldwide Amiga sales closing in on 1.5 million, it was the number one gaming computer in Europe, a position it would continue to enjoy for several years to come. Just about to begin its fade from prominence as a game machine in the United States, in Europe the Amiga’s best years and best games were still in front of it, as bedroom coders learned to coax performance out of the hardware of which its designers could hardly have conceived. The old Boing demo, once so stunning that crowds of trade-show attendees had peaked suspiciously under tables looking for the workstation-class machine that was really generating that animation, already looked singularly unimpressive. The story of the Amiga 500 in Europe was, in other words, the story of the Commodore 64 happening all over again. Commodore was now making the vast majority of their money in Europe, the North American operation a perpetual weak sister.

When journalists for the Amiga trade press in North America visited Europe, they were astounded. Here was the mainstream saturation that they had only been able to fantasize about back home. A report from a correspondent visiting a typical department store in Cologne must have read to American readers like a dispatch from Wonderland.

I came across a computerized book listing that was running on an Amiga 500. As I approached the computer department, I was greeted by a stack of Amiga 500s. I could not believe the assortment of Amiga titles on the book rack (hardcover ones, too!). I found two aisles full of Amiga software, consisting mostly of games. The Amiga selection was more than that of any other computer.

In a sense, it was just a reversion to the status quo. After all, prior to the introduction of the VIC-20 in 1981, Commodore’s income had been similarly unevenly distributed between the continents. Seen in this light, it’s the high times of the 1980s that are the anomaly, when American buyers flocked to the VIC-20 and the 64 and for a time made what had always been fundamentally a European brand — although, paradoxically, a European brand engineered and steered from the United States — into an intercontinental phenomenon. Not that that was of much comfort to the succession of executives who came and went from Irving Gould’s hotseat, fired one after another for their failure to make North American sales look as good as European sales.

But I did promise you 68000 Wars in the title of this article, didn’t it? So where, you might well be wondering, was the Amiga’s arch-rival the Atari ST in all this? Well, in North America it was a fairly negligible factor, although Atari would continue to sell their machine there almost as long as Commodore would the Amiga. The hype around the ST had dissipated quickly with the revelation in late 1986 that Atari really wasn’t selling anywhere near as many of them as Jack Tramiel liked to let on, and the Amiga 500, so obviously audiovisually superior and now much closer in price, soon proved a deadly foe indeed. The ST retained its small legion of loyal users: desktop publishers unwilling to splurge on a Macintosh, who took full advantage of its rock-solid monochrome high-resolution screen and Atari’s inexpensive laser printer, thereby truly making the ST live up to its old “Jackintoch” nickname; musicians, both amateur and professional, who loved its built-in MIDI port; programmers and hardware hackers who favored its simple, straightforward design over the Amiga’s more baroque approach; people who needed lots and lots of memory for one reason for another, on which terms Atari always offered the best deal in town (they released 2 MB and 4 MB ST models as early as 1987, when figures like that were all but inconceivable); inveterate Commodore haters and/or Atari lovers who bought it for the badge on its front. Still, there was little doubt which platform had won the 68000 Wars in North America. In the wake of the Amiga 500’s release, Atari began increasingly to turn to other ways of making money: buying the Federated chain of consumer-electronics stores; capitalizing on nostalgia for the glory days of the Atari VCS by continuing to sell both the old hardware and the cartridges to run on it; wresting away from Epyx a handheld gaming machine, the first of its kind, that was ironically designed by members of the old Amiga, Incorporated, team. When all else failed, there was always Jack Tramiel’s old hobby of lawsuits, of which he launched quite a few, most notably against the former owners of Federated for overstating their company’s value and against the new kid on the block in console gaming, Nintendo, for their alleged anti-competitive practices.

In Europe, the ST also came out second best to the Amiga, but the race was a much closer one for a while. Along with their love for all things Commodore, Germans found that they could also make room in their hearts for the Atari ST. It found a home in many German markets it never came close to cracking in the United States, being regarded as a perfectly respectable business computer there for quite some time. It also continued to do fairly well with gamers, thanks to Atari’s pricing strategies that always seemed to keep its low-end model just that little bit cheaper than the Amiga 500, enough to be a difference-maker for some buyers. When the Amiga became the biggest gaming computer in Europe, it was the Atari ST that slipped into the second spot. It would take the much more expensive MS-DOS machines some years yet to overtake these two 68000-based rivals. The economic chaos brought on by the reunification of West and East Germany, which caused many consumers there to tighten their wallets, only helped their cause, as did the millions of new price-conscious buyers who were suddenly scrambling for a piece of that Western computing action following the fall of the Iron Curtain.

The story of the Amiga, and to some extent also that of the ST, is often framed as a narrative of frustration, of brilliance that never got its due. There are some good reasons for that, but it can also be a myopic, America-centric view, ignoring as it does a veritable generation of youngsters on the other side of the Atlantic who grew up knowing these two platforms very well indeed. When I was writing my book about the Amiga, I couldn’t help but note the markedly different responses I got from friends in Europe and the United States when I told them about the project. Most Americans have no idea what an Amiga is (“Omega?”); most Europeans of a certain age remember it all too well, flashing me smiles redolent of nostalgia for afternoons spent before the television with their mates, when the summer seemed endless and the possibilities limitless. Instead of lamenting might-have-beens too much more, I expect to spend quite some articles over the next few years talking about the Amiga’s innovations and successes — and, yes, I’ll also have more to say about the Amiga’s perpetually overlooked little frenemy the Atari ST as well. Whether you grew up with one of these machines or you too aren’t quite sure yet what to make of this whole “Omega” thing, I hope you’ll stick around. Some amazing stuff is in store.

(Sources: Invaluable as always for these articles was Brian Bagnall’s book On the Edge: The Spectacular Rise and Fall of Commodore. I can’t wait for the better, longer version. The long-running “Roomers” column in Amazing Computing is my go-to source for a month-by-month chronology of developments on the Amiga scene, and the source of most of the nit-picky factoids in this article. The issues of Amazing used include: March 1987, June 1987, July 1987, August 1987, October 1987, November 1987, December 1987, February 1988, April 1988, May 1988, June 1988, July 1988, August 1988, September 1988, November 1988, December 1988, January 1989, February 1989, March 1989, April 1989, May 1989, June 1989, July 1989, August 1989, September 1989, October 1989, November 1989, December 1989, January 1990, April 1990, May 1990, June 1990, February 1991, March 1991, April 1991, May 1991, December 1991. Commodore Magazine‘s fateful interview with Thomas Rattigan appeared in the May 1987 issue. Other sources include Retro Gamer 39 and of course my own book The Future Was Here. Hey, it’s not every day a writer gets to cite himself…)

Employee Equity – Sam Altman

27 November 2015 - 12:35am

Startup employees often do not get treated very well when it comes to stock compensation.   New ideas float around occasionally, but lawyers are usually averse to trying new things, and investors don’t feel that they have enough incentive to try something new for employees.

There are four major problems: 

1) Employees usually don’t get enough stock. 

2) If an employee leaves the company, he or she often can’t afford to exercise and pay taxes on their options. 

3) Employee options sometimes get unfavorable tax treatment.

4) Employees usually don’t have enough information about the stock or options. 

Here are some proposed solutions:

1) Startups should give employees more stock.  Value is created over many, many years.  Founders certainly deserve a huge premium for starting the earliest, but probably not 100 or 200x what employee number 5 gets.  Additionally, companies can now get more done with less people.

It’s very difficult to put precise numbers on this because the specifics of every situation matter so much.  Perhaps the best way to think about it is to try to come up with a total compensation package with the same expected value (using the company valuation of the last round, or a best-efforts guess if it’s been a long time since the round) as the employee would get at a big company like Google.  As an extremely rough stab at actual numbers, I think a company ought to be giving at least 10% in total to the first 10 employees, 5% to the next 20, and 5% to the next 50.  In practice, the optimal numbers may be much higher.

One problem is that startups try to have very small option pools after their A rounds, because the dilution only comes from the founders and not the investors in most A-round term sheets.  The right thing to do would be to increase the size of the option pool post-A round, but unfortunately this rarely happens—no one wants to dilute themselves more, and this leads to short-sighted stinginess much of the time.

Option pools are complete fiction; boards can increase them whenever they want.  It should never be used as a reason for not making a grant.

2) Most employees only have 90 days after they leave a job to exercise their options.  Unfortunately, this requires money to cover the strike price and the tax bill due for the year of exercise (which is calculated on the difference between the strike and the current FMV).  This is often more cash than an employee has, and so the employee often has to choose between walking away from vested options he or she can’t afford to exercise, or being locked into staying at the company.  It’s a particularly bad situation when an employee gets terminated.

This doesn’t seem fair.  The best solution I have heard is from Adam D’Angelo at Quora.  The idea is to grant options that are exercisable for 10 years from the grant date, which should cover nearly all cases (i.e. the company will probably either go public, get acquired, or die in that time frame, and so either the employee will have the liquidity to exercise or it won’t matter.)  There are some tricky issues around this—for example, the options will automatically convert from ISOs to NSOs 3 months after employment terminates (if applicable) but it’s still far better than just losing the assets.  I think this is a policy all startups should adopt.

As an aside, some companies now write in a repurchase right on vested shares at the current common price when an employee leaves.  It’s fine if the company wants to offer to repurchase the shares, but it’s horrible for the company to be able to demand this.

3) Tax treatment on ISOs sounds good, but there’s an issue with AMT (Alternative Minimum Tax) and employees often end up paying more tax than they were expecting.   NSO gains are taxed as ordinary income.  RSU gains are also taxed as ordinary income.

Tax optimization is a second-order issue, and for an immediate solution, I think extending exercise windows to 10 years is the most important thing to do.  But longer-term, we should figure out a way for employees to be taxed on their stock compensation the same way as founders (whether or not capital gains should be taxed less than ordinary income is a separate discussion, but in any case I think the tax treatment should be the same). 

I think this may be doable.  Ideally, employees would just get restricted stock (not RSUs), and then when they sold it’d be taxed as long-term capital gains.  The problem is that as the company grows, the stock has a non-trivial present value, and if an employee were granted stock then they would then owe immediate tax on the value of the grant.

I think there are a lot of ways to fix this.  The easiest would be if the IRS would agree to not tax illiquid private stock until it gets sold, and then tax the gain from the basis as long-term capital gains and the original value as ordinary income.

Another might be to create a new class of employee stock.  Today, in an early-stage company, common shares are usually worth much less than preferred shares.  It might be possible to create a class of shares with less rights than common and thus worth even less.  The idea would be to convert these shares into common on an acquisition or IPO, but before that, they would be non-transferable and have no value.  If it were possible to create a class of stock that the IRS agreed had next to zero value, it might be possible to grant employees this sort of stock, have them owe a tiny bit of tax on it now, and then have normal long-term capital gains treatment years later when the startup goes public. 

4) Most startups do a bad job of helping employees think about the value of their options.  At a minimum, any startup should tell a prospective employee what percentage of the company the equity grant represents (number of shares is meaningless).  Some startups are very hesitant to do this—they don’t want to disclose the number of shares outstanding.  Employees should demand to know what percentage of the fully-diluted shares their stock options represent, and be very suspect of any startup that won’t tell them.

A specific question worth asking is some version of “so if I have 0.5% of company and it gets acquired tomorrow for $100 million dollars, will I get $500,000?”  There are many ways for this not to be the  case—there can be a huge liquidation preference, for example—that most employees don’t know to think about.  So it’s worth asking about a specific scenario.

While you’re asking questions, another good one to ask is “how much money did the company lose last month, and how much is in the bank?”  This is better than asking how much runway the company has, because most founders calculate that off of a plan that assumes revenue growth which does not always materialize.

I have two other thoughts about stock-based compensation at startups.

First, I think employee stock and options should usually not be transferrable.  It causes considerable problems for companies when employees sell their stock or options, or pledge them against a loan, or design any other transaction where they agree to potentially let someone else have their shares or proceeds from their shares in the future in exchange for money today.

I think it’s fair that if founders sell stock, they should offer an opportunity to employees that have been at the company for more than a certain number of years to sell some portion of their shares.  And some companies offer an employee liquidity program even when the founders don’t sell any shares themselves.  But otherwise, I think it’s reasonable for employees to wait for an acquisition or IPO.

Second, I think it’s time to consider other vesting ideas.  The standard at startups is 4 years with a 1-year cliff.  So you get 25% of your options after you’ve been there for a year, and 62.5% if you leave after 2.5 years.

It’s possible that 4 years is now too short—companies are often worth more than they were 10 years ago, but they take longer to reach liquidity.  I’ve seen some startups offer 5 or 6 year vesting schedules.  To compensate for this, they offer above-market grants.

Another structure I’ve seen is back-weighted vesting.  For example, 10% of the grant vests after the first year, and then 20%, 30%, 40% in the following years.  Again, the startups I know that do this tie it to above-market grants, and I think it helps them select for employees that really believe in the company and want to be there for a long time. (Also, companies that have vesting schedules like this usually do it for founders too.)

Finally, a third structure I’ve seen is a new way of thinking about refresher grants.  For a company using options, it’s nice to grant employees options early while the strike is low.  It’s possible to give “forward-dated grants”—i.e., you can give a high-performing employee a refresher grant today where 1/3 of it starts vesting immediately and the other 2/3 starts vesting when their initial grant is fully vested.  This guarantees them a low-strike price and presumably a relatively large grant in a few years.  Dustin Moskovitz at Asana does something like this, and I think it makes a lot of sense.

These are just a few of the ideas I’ve seen about new ideas for employee vesting.  But I think they’re worth considering—the default 4-year grant does not seem to be the best option.

Essential Reading List for Getting Started with Service Workers

27 November 2015 - 12:27am

After our conversation with Tal Ater on episode #183 about the offline revolution and his experience with Service Workers, we realized there is a lot of mystery around this new browser feature. So we’ve put together this essential reading list for those of you looking for the best resources out there on ServiceWorker.

Service workers essentially act as proxy servers that sit between web applications, and the browser and network. They are intended to (amongst other things) enable the creation of effective offline experiences, intercepting network requests and taking appropriate action based on whether the network is available and updated assets reside on the server. They will also allow access to push notifications and background sync APIs. – #

Introduction to Service Workers

This was written early in the life of ServiceWorker from Matt Gaunt. Still relevant.

Rich offline experiences, periodic background syncs, push notifications— functionality that would normally require a native application—are coming to the web. Service workers provide the technical foundation that all these features will rely on.

Instant Loading Web Apps With An Application Shell Architecture

Addy Osmani & Matt Gaunt team up to share the low-down on using ServiceWorker to cache your UI shell and make repeat visits load instantly.

…gain substantial performance benefits thanks to intelligent service worker caching of your UI shell for repeat visits.

Cache-limiting in Service Workers

Jeremy Keith shared what he learned bloodying his knuckles with cache-limiting. Be sure to check out the ServiceWorker code he uses to speed up his blog by storing assets in a cache.

I’ve got a function in there called stashInCache that takes a few arguments: which cache to use, the maximum number of items that should be in there, the request (URL), and the response…

ServiceWorker Cookbook

You can also contribute to this growing cookbook on GitHub.

The Service Worker Cookbook is a collection of working, practical examples of using service workers in modern web apps.

Offline Recipes for Service Workers

David Walsh breaks down his recipe to create an app that can be used both online and offline with the help of a ServiceWorker.

The Offline Status recipe illustrates how to use a service worker to cache a known asset list and then notify the user that they may now go offline and use the app.

Service Workers: Dynamic Responsive Images using WebP Images

Dean Hume shows off code and commentary on how he uses Service Workers to intercept requests passed to the server to support additional image formats like WebP, and Internet Explorer’s JPEGXR.

Service Workers open up a world of endless possibilities and this example could be extended to include other image formats and even caching. There’s no reason why we can’t present our users with fast web pages right now!

Service Workers Spec

This is the official spec published on GitHub but Alex Russell. The published version can found here, while the nightly version can be found here.

Service Worker API

This is an alternate, but not the canonical, spec on ServiceWorker. It’s a little easier to read through, but YMMV.

Is ServiceWorker Ready?

Safari has given no public commitment, while Edge is under consideration, but showing positive signals. Like any good resource, it’s on GitHub.

Yes, we have a follow-up post ready to post sharing our “Essential Watch List” on Service Workers so stay tuned. Subscribe to Changelog Weekly or follow us on Twitter to be notified.

If you know of an article or resource that needs to be listed here, let us know by opening an issue on GitHub.

Have comments? Send a tweet to @Changelog on Twitter.

Subscribe to Changelog Weekly – our weekly email covering everything that hits our open source radar.

GitHub: Support https on GitHub pages with custom domains

26 November 2015 - 11:54pm
Github Pages: Let's Encrypt! · GitHub

Skip to content

Create a gist now

Instantly share code, notes, and snippets.

Something went wrong with that request. Please try again.

You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.

Asia Struggles for a Solution to Its ‘Missing Women’ Problem

26 November 2015 - 6:14pm

SEOUL—A cultural preference for male children has cost Asia dearly. Count up all the girls who were never born because of selective abortion, victims of infanticide and females who died from neglect and there are upwards of 100 million women missing on the continent today by some estimates.

Not just a human-rights catastrophe, it is also a looming demographic disaster. With Asian birthrates already plummeting, that is tens of millions of women who will never be mothers. The economic and social impact on some of the world’s largest countries is incalculable.

For decades, South Korea was Exhibit A in this depressing trend. By 1990, as medical advances made prenatal sex selection routine, the ratio of male-to-female babies soared in South Korea to the world’s highest, at 116.5 males for every 100 females.

Then something unusual happened. South Korea did a U-turn.

In one generation, South Korea has gone from a society where sons are prized to one where daughters are just as eagerly received. Turbocharged industrialization, urbanization and education, along with a feminist revolt, wiped out centuries-old practices in which a son was essential to inherit property, worship ancestors, care for parents and continue the family lineage.

By last year, the country’s gender ratio had recovered to a normal 105 males born for every 100 females.

The dramatic transformation offers important lessons for Asian giants China and India—one-third of humanity that continues to give birth to significantly more males.

In these countries that together hold 2.7 billion people, vast numbers of men won’t be able to find brides in the coming decades, obliterating universal marriage, the underpinning of socioeconomic organization for centuries.

Having so many unattached, alienated men could have potentially devastating economic and social consequences. Some social scientists fear that incidences of rape and bride trafficking in parts of Asia could be early effects of the skewed sex ratios here.

One study by Lena Edlund at Columbia University and others suggested a causal link between a more masculine sex ratio and crime, analyzing province level crime data in China to show that a single point rise in the sex ratio of men aged 16 to 25 raised property and violent crime by between 5% and 6%. China has seen a dramatic increase in crime between 1992 and 2004, and the Columbia study attributed as much as one third of that rise to the increase in the maleness of the young adult population. The theory, in part: Unmarried men are more likely to commit crimes than married men, especially as they try to accumulate assets to compete for scarce brides.

“I don’t see any good consequences—I see only misery and desolation,” says Christophe Z. Guilmoto, a demographer at the French Research Institute for Development in Paris.

The intellectual debate over this misalignment has a long history. In 1990, Amartya Sen, the Nobel Prize-winning Indian economist, shocked the world with an article in the New York Review of Books that estimated there were 100 million missing women because of discrimination.

Since then, demographers have devised more precise methods of calculating missing women in each country. They factor in the toll caused by malnutrition and poor medical care and, more significantly, also the numbers lost to abortions of female fetuses, a problem recognized in the years after Dr. Sen wrote his article.

In June of 2015, the Population Council, a New York City-based research organization, published a study saying there were 88 million missing women world-wide in 1990, when Mr. Sen wrote his article, and that there were 126 million missing in 2010, roughly half of them attributable to prenatal sex selection. Of those, more than 112 million were in Asia.

The paper, by John Bongaarts, a distinguished scholar at the Population Council, and Dr. Guilmoto, projects an increase to 150 million missing by 2035 and then a slight decrease to 142 million by 2050.

Across Asia, the effects are only beginning to be felt as the first generation born with skewed sex ratios in the 1980s and 1990s reaches marriageable age.

In Korea, nearly 10% of marriages in 2005 were made up of a Korean husband and a foreign wife, mostly from China, Vietnam and the Philippines.

But as the marriage squeeze worsens in the next few decades, no amount of bride migration will be able to stem the shortfall, demographers say.

The problem will “intensify radically in the next five years and become explosive in the next decade,” says Nicholas Eberstadt, a political economist and demographer at the American Enterprise Institute in Washington who studies sex ratios.

If the masculine sex ratios remain as high, in China, there would be as many as 186 single men for every 100 single women hoping to marry by midcentury, according to Dr. Guilmoto, since unmarried men from one year join the next year’s group seeking wives. By 2060 in India, the peak could be higher: 191 men for each 100 women.

Somvir Sain, 38, a driver for a politician in Haryana, a north Indian state with one of the country’s highest sex ratios, has been looking for a wife for 12 years, but in vain. Neither he nor his three brothers have found brides, he says. He has even traveled to several nearby poorer states to try to buy a bride, a common practice, but couldn’t settle on a price and arrangement, he says. Prices ranged from about $400 to $1,500, he says. He offered the lower end of the range but no family accepted. Most wanted him to stay for a few weeks to get to know them, and he says he didn’t have enough vacation time.

“I feel badly for my mother who has to do all of the housework and wash all of my clothes at her age because I can’t find a wife,” he says. “I feel sad thinking of never having children and my family not continuing after me.”

At the same time, educated, affluent women in Asian nations such as South Korea, Singapore and Japan choose to take what is being called a “flight from marriage,” by remaining single or in unofficial relationships. In much of Asia, the social stigma remains strong against having children outside of marriage, and so staying single almost always means staying childless.

Both China and India have acknowledged the need for action. India has strengthened laws against sex detection of fetuses and enacted legislation aimed at gender equality. In China, the government in 2000 began developing a nationwide program called Care for Girls Campaign. It offered financial incentives to couples with daughters and clamped down on sex detection of fetuses during the second pregnancy, in which female feticide was more common, according to demographers.

Just last month, China announced it was abandoning its one-child policy, believed to be a significant contributor to skewed ratios.

There are some hopeful signs. In China, the ratio of male to female babies, rising in three decades to 120 males for every 100 female babies in 2008, has plateaued or declined slightly since. The sex ratio was 115.9 last year, the lowest it has been since 2001.

It is the same with Indian states with the worst sex ratios. Haryana’s sex ratio of 122 boys for each 100 girls in 2001 fell to about 120 in 2011, the most recent census. India calculates sex ratios from birth to 6 years old.

Still, bringing about change is “a slower process in China. It’s even slower in India,” notes Monica Das Gupta, an ex-demographer at the World Bank now at the University of Maryland. Both countries are much larger and more diverse than South Korea.

Even if the sex ratio at birth were to return to normal, hundreds of millions of Asian men will still remain single in the ensuing decades. More than 21% of Chinese men would still be unmarried at 50 in 2070, while in India the number would be nearly 15%, Dr. Guilmoto estimates.

For many decades, South Korea’s demographic trajectory was similarly grim. A military dictatorship that ruled from 1961 to 1987 preaching economic development helped transform the nation from mostly rural and agrarian to mostly urban and industrial. Fertility levels plunged and education levels soared.

Even so, Confucian cultural practices, codified into law, dictated that the eldest son inherited most of the family property, worshiped ancestors and continued the family lineage. This meant that even the newly educated, smaller urban families still felt the need to have a son.

So they rapidly adopted ultrasound imaging when it became widely available here in the 1980s, to detect the sex of the fetus—and aborted females.

“When a patient knew she was going to have a second girl, she cried,” says Dr. Kim Ahm, a 60-year-old gynecologist at Asan Medical Center. “When you told her she was having a third daughter, the patient really panicked.”

These women went on to abort the fetuses, hoping for a son instead, Dr. Kim said.

Abortions of female fetuses became particularly commonplace if families already had one or two daughters. By 1990, the sex ratio for a third child had risen to 193 boys for each 100 girls.

By then, Korea had held its first democratic elections. Free from military dictatorship, the feminist rebellion, already brewing, was unleashed.

The government strengthened a medical law banning the sex detection of a fetus by yanking the medical license of offending doctors. In the early 1990s, the government added a jail sentence for errant doctors.

After reaching a high of 116.5 males born for every 100 females in 1990, the sex ratios began declining, although they were still well above normal.

Feminist groups worked the media, the legislature and the courts, demanding gender equality.

KoEun Kwang-soon, a Korean traditional-medicine doctor, says she was radicalized seeing the roots of the sex ratio-problem in her practice, where she was asked “too many times to make medicines that help a patient conceive a boy.”

In 1997, Ms. KoEun helped launch a campaign for allowing families to use either the mother’s or the father’s family name, instead of always the father’s. The campaign had more symbolic than practical impact, driving home the idea that girls were important too.

“People thought we were a bunch of crazy women,” she says.

The next year, she helped found the Citizens Association Working to Abolish Hojuje, the practice of fathers being considered the legal head of the family.

Legislators seemed supportive of abolishing the practice in private, but backed off from moving publicly because of “opposition in their constituencies,” says Kwak Bae-hee, president of the Korea Legal Aid Center for Family Relations.

Her group turned to the courts. In 2005, Korea’s Constitutional Court ruled Hojuje violated the constitution granting all citizens dignity and equality. Soon after, the court ruled that families didn’t have to take the father’s name.

With the end of Hojuje, “sons were no longer key to the family lineage continuing,” says Lee Ki-soon, the director general at the women’s policy bureau of the Ministry of Gender Equality and Family.

In South Korea, the sex ratio had fallen to 110 males for every 100 female babies by 2005. Five years later, it dipped to 107, and last year, for the first time in recent history, it reached the natural level of 105. That is on par with the ratio in the U.S. and Europe, where the slight excess of boys is believed to be nature’s way of accommodating their higher fragility.

It isn’t that the sexes are economically equal. Even though Koreans elected their first woman president, corporate and political leadership is still predominantly male. South Korea ranked 125th out of 142 countries for income equality between men and women, according to the Gender Gap Index 2014 published by the World Economic Forum.

But for women and men starting families, the idea that it is an imperative to have a son has changed.

“I see no one crying these days about the baby being a girl,” says Dr. Kim, the gynecologist.

Where more than 40% of mothers said in a government survey in 1991 that they should bear a son, only about 8% did so in a 2012 questionnaire.

That change is reflected in the experiences of 64-year-old Lim Ki-ouk and her daughter, both the eldest children in their families, both university lecturers and both mothers who gave birth only to daughters.

With each of her four daughters, the mother says, she felt she had again “failed to continue the family line.” By contrast, when her daughter gave birth to three girls, a cranky uncle-in-law who dared suggest she keep trying for a son was scolded into apologizing.

“That’s how much South Korea has changed,” the daughter, Ko Bo-min, 38, said in a recent interview, squeezing the hand of her mother, seated across the table at a trendy cafe in Seoul.

Ms. Ko works as a college lecturer and researcher. Her husband doesn’t expect her to carry the double load of child-raising and work—he agreed they should pay for a baby-sitter who also helps with housekeeping.

Her mother interrupts. There was no such husband support when she worked as a university lecturer while raising her four daughters. Life was difficult, she says, because child care and housework also fell to her.

The Korean government has played a role in this transformation. It subsidizes the cost of child care at home, paying up to $177 a month for the first five years of a child’s life.

The government also offers incentives to companies for having paternity-leave policies that are more generous than maternity benefits. Local governments are organizing cooking classes and housekeeping courses for dads.

In this new era, Mrs. Lim, who once saw herself as a failure, now is envied. Quoting a new Korean saying that having two daughters gets you a gold medal, she gestures toward her orange-beaded neckline, smiles and says: “Now my friends say: ‘You get the diamond medal.’”

Illustrations: Jimmy Turrell

The Secret Ingredient in Orange Juice

26 November 2015 - 3:05pm
  • Awesome. The one thing I let my kids have as a treat because it’s real juice. *sigh* Shady, shady, shady…

    • I know, me too. I thought Orange Juice, the expensive Simply Orange is what we buy, was the one OK thing to let them have!

    • I don’t understand someone would give their kids all that sugar in the first place! Not good…rots the teeth (especially the acid in oranges), overloads the pancreas, no nutritional value, generally unhealthy. Some fresh, whole berries with raw cream…now that’s GOOD, and healthy!

      • Many people just don’t “get” how much sugar is in juice. It’s astronomical in the “grand spectrum” of things. They also don’t get that the pulp of fruit slows down the rate of absorption of the sugars. Many others just plain don’t care that they are eating nothing but junk and chemicals. Their motto is “If it tastes good, eat it” and “I want to enjoy my food”. Alright then.

      • I don’t understand why someone would look down on another person for giving their child an occasional glass of juice. Why would you give your child a glass of juice? Because the child enjoys juice. Because it’s recommended by pediatricians to help a child rehydrate during illness. Because mom says so. Seriously, of all the parenting choices you can pick on – juice needs to be a whole heck of a lot lower on the list. My kids get juice once in a blue moon. I prefer them to drink water but they like juice, their dad likes juice, and I like juice. So every now and then, we all have juice with dinner instead of water. We drink juice for the same reason we very occasionally have cake. Because we like it.

        • I agree! I hate how people shame others for food choices. Nobody is taking into consideration that healthy, whole foods cost more. Not everyone can afford to buy all of these organic whole foods. Obviously indulging a child in copious amounts of juice is going to have negative side effects, but a small amount occasionally isn’t going to kill them. People are freaking out as if you’re pouring radioactive waste into a sippy cup for your two year old.

      • Most people do not have a cow in the back yard, so we cannot get raw cream,and raw dairy is largely ILLEGAL in the US. Most of us do not have berry patches either.
        Personally, make my own juice, with a press, mostly organic veg, some fruit. I don’t eat any refined sugar, and only the occasional low glycemic fruit.
        But seriously judging others for the occasional glass of orange juice,when they are trying to to what is best, and suggesting they make an unrealistic substitution is silly.

    • That is the same reason I buy… bought simply Orange, I thought I was getting something good. The stuff in the freezer is going on Craigslist now. Bummer.

    • Nice article, thanks.

      I forget that alot of people aren’t aware of how “flavors” are made in mainstream food.

      Yeah, we never touch that stuff. Local, organic, grass-fed, pesticide-free…anything else isn’t really what you think it is.

    • Yeah, it would be nice if the labels were HONEST, not just sticking to the letter of some law written in twisty lawyer language but actually being fully honest and up front about the contents.

      Of course, if that happened more people would begin to ask for something better and they would lose market share and the government would lose the “value added” taxes they probably get for the additions and changes to the juice, so our regulatory bodies have little incentive to stop the fraud.

      I just try to stick with whole foods, organic when possible, locally grown when possible, and in general attempting to get nothing but the real thing. NOT easy!

    • Oh lovely…I’m going to assume(which I usually don’t do)that this also applies to grapefruit juice.I love grapefruit juice,too bad it’s not really real,tho…smh.If I could afford a juicer or one of those Vitamix thingies that have been mentioned…

    • My god, don’t you just sound so arrogant? The rest of the population aren’t idiots, you know.

    • Love the way yahoo recycles 3 year old articles and puts them up in the headlines. 43 years old and been drinking orange juice for almost as many, just like everybody else I know. Pretty sure this isn’t a new process, because I’ve been drinking it year round since I was a kid. Anything in excess can kill you. If you want to live longer..stop smoking!

  • OMG

  • Well, I stopped drinking juice long ago anyway. Fresh local fruit in season is what I do and here in south Florida that’s so easy. But this is something horrible that i never even knew before!

  • Always new to stay away from concentrates. Now I know why. Thanks for the literature.

    • Veronica, Now you know why to stay away from concentrates?? But this article is about ‘Not from concentrate’ juice

  • Awesome article, thanks for posting!

  • @Madeline — We’re in the same boat as you. We haven’t been juice drinkers in years, but I still thought that 100% juice was at least REAL, even if it wasn’t a whole food.

  • Veronica, I think this is all orange juice, not just the concentrates.

  • @Veronica — This is about juice specifically advertised as being made “not from concentrates.”

    • Forget about the orange juice and eat the orange! You not only get a REAL orange favor but you get fiber as well.

  • Does this include organic brands as well? My husband is pretty dead set in his ways. I could set my watch to him. He has to have his glass of orange juice with breakfast.

  • Thanks for the post. It helps point out the considerable degree to which we humans are enormously gullible and vulnerable. Would it be too paranoid to consider the notion that perhaps the grocery store is not our friend?

    Ciao, Pavil

  • I see. Well, I don’t buy either kind b/c i assumed they were all pasteurized. I don’t have a heavy duty squeezer, so we buy the fresh one from the grocery store. We even try not to drink too much of that due to sugar content and opt to make NTs “Orangina”.

  • What about organic orange juice? Like 365 brand from Whole Foods?

  • We drink no juice either, sometimes treat ourselves to locally pressed fresh apple cider in the fall, but this just blew my mind!

  • Very interesting, I always wondered why the juice in cartons lacked flavor. I no longer buy fruit juice. I will occasionally make my own orange juice in my Vitamix, which is just throwing a whole peeled orange in there and blending it up so I’m still getting the whole fruit. I add a carrot also just for even more nutrition. It is delicious!

  • I’m so glad we stopped drinking store juices a while ago.

  • Uhg, not my orange juice, one of my children not that long ago asked why it always taste the same, now we know

  • @Elizabeth — I just answered a similar question in the comments on the actual post. The short answer is: if it’s available year-round, then it HAS to be processed this way. There’s no other way to keep unconcentrated orange juice without it spoiling besides de-aeration, and de-aerated juice needs to have flavoring added back into it in order for it to be palatable.

  • I’ve heard that there are companies in Florida who squeeze them fresh and freeze them right away, and then they ship them to you frozen. Do you know of any that are good and trustworthy?

  • I’ve just ordered a juicer a few days ago. The more I read the more I am glad that we have the option to make our own!

  • Alot of juice makers put things in it to keep it from separating. I still think that fresh-squeezed juice tastes better.

  • I don’t drink a lot of juice, but I do occasionally. Whenever possible I drink fresh squeezed because it’s just so much yummier.

    However, my parents cannot seem to accept the advice I give them on food and beverages, so they still buy a lot of juice (and a million other things I don’t approve of). I have at least convinced Mom to buy not from concentrate, all natural, often organic.

    The other day, I saw a bottle of that new Trop 50 orange juice, the one that says “50% fewer Calories and 50% less Sugar,” in her refrigerator. So I asked her how “all-natural” juice could be made to have less sugar or calories… and why was it a big deal since I’m assuming most people aren’t getting a huge portion of their calories from orange juice. Of course, she couldn’t answer those questions.

    So I looked on the back of the bottle. The Trop 50 has 50 calories and 4% of the “recommended daily allowance” of carbs (sugar). So, inversely, regular orange juice would only have 100 calories and 8% RDA of carbs. Which I feel is perfectly fine for a juice you might have a glass or two of a day. SO much better than soda anyway. But that still didn’t answer the question of HOW they made it less.

    Answer: It’s literally watered down orange juice. The first ingredient is filtered water, then not from concentrate orange juice. Once again, we see the power of marketing. People are literally paying a company to water down their orange juice for them. And not paying half price either.

    The only good thing that comes of incidents like these is that my Mom is starting to believe me.

    • Oh, Leah! That’s so so tragic. I’m such an obsessive label reader that I find it hard to remember that most people aren’t and will (sadly) fall for this sort of marketing gimmick.

    • Oh my gosh… it’s amazing how sure they must be that people DON’T read labels! Because surely anyone reading that label would say what you said, Leah: “I’m paying them to water down my orange juice.” Incredible!

    • Leah, they do the same thing in juices marketed to kids, especially toddlers. I remember reading the label of one of those “healthy” juice boxes, and #1 ingredient is filtered water, then it had fruit and vegetable juices. Then they charge you more for it because it’s healthier. Mott’s for Tots makes no pretense – they tell you straight up it is diluted juice. I really don’t get why people will willingly pay more money for less juice!

    • Leah, Don’t be fooled by paying for “all natural” it is a marketing ploy, and means ABSOLUTELY NOTHING! If you want to be safe, only buy certified organic. Also don’t buy “made with organic ingredients” They can add two organic ingredients, and tons of chemicals. You have to become a smart consumer to avoid the carefully placed wording used by the food industry to trick consumers into thinking they are paying for something they are not getting. Vote with your dollars and support certified organic companies.

    • The more I see articles like this one the happier I am. I have always been health conscious, but I think I have put too much trust in the food labels. I find I am now reading every line on the label in the store. Natural flavor is another scary chemical additive that is essentially just as bad for you as artificial flavor. They all come from one lab in New Jersey and even though the original flavor is natural, they tweak it with chemicals to get the flavor you want. The scariest thing is that since natural flavor is new to the market there aren’t any regulations on it, so in some cases they may be even more dangerous than artificial flavors. WRITE YOUR FOOD COMPANIES! Question the ingredients on the label, it’s the only way we got anywhere with the organic food standards we have today. BECOME MORE AWARE OF WHAT YOU ARE INGESTING, DON’T LET BIG CORPORATIONS DECEIVE YOU INTO EATING WHAT THEY WANT YOU TO EAT. Make your own choices and speak out!

      • Nowadays, chemicals are found in pretty much all kinds of food. Panicking and saying orange juice is unhealthy just because of chemicals doesn’t really sound reliable. OJ has lots of vitamin C and keeps people from getting ill. Can’t see what’s so unhealthy about that. Please explain to me what exactly these “chemicals” do to our body. Then I might be listening.

    • I thought similarly when they started selling all those 100-calorie snack packs of various junk foods. So they give you less food, more packaging, and charge you more for it?

    • It is worse than what you are saying. The Trop 50 and other “light” juices are not simply watered down. In order to fool you into not noticing they are watered down they sweeten them back up with artificial sweeteners (not to mention the flavor packs and fragrance). I remember when they first came out. I was in college on a choir tour doing a home-stay in Tex. My host family had this half juice on the table for me to drink. I reached for it thinking it was my beloved Trop. pure and premium NEVER from concentrate only to realize that the packaging was a little different. I looked at the ingredients and was appalled! I asked my host-mom (a nurse!) if she thought this was actually healthier than real 100% juice. She simply said, “Of course. It has half the sugar and half the calories.” Boo medical community. Boo, advertising conspiracy. Boo, government (non)watchdogs. Boo, Boo, Boo. I am very sad. And yes I do both love and hate you for writing this article!

  • Well, that explains why I always feel the need to water down my juice, 50:50. Thanks for the info.

  • I peel oranges and put them in the vita mix. OJ !ta da! and yes it tastes different every time.

  • YUK!!!

  • I’m beginning to realize why my daughter seems to react to every eating plan except very basic paleo. Even the most “natural” packaged foods have hidden poisons, and sensitive systems are affected more than others.

    • Poisons is a little harsh. “Chemicals” can be good or bad, depending on what you are talking about. Water is a chemical. Just because it sounds bad, doesn’t mean it is!

    • I think it’s crazy how people pay attention to how “natural” their foods are, but don’t question how completely unnatural it is to have to follow a “food plan”. Eat what God’s given, plain and simple. He never gave us food to figure out which “plan” would work for our body. Just insane.

      • Michelle, when God created man, there was no death in the world, so man did not kill and eat animals. Once sin was introduced, death was also, and man changed his diet. Before Adam sinned, all food was perfect for his body. Now, however, our food choices are very different, and everyone’s bodies are different. Some NEED the flesh proteins, others NEED the carbs (like vegetarians). If we were still perfect like Adam was when he was created, this would not be a problem.

      • Michelle, if you are trying to maintain a caloric deficit (for weight loss) and your body is used to consuming 3000+ calories a day (which is really easy to do just eating whatever you feel like, in the land of excess) then yes, you do need a food plan of some sort at least in the beginning. (Even if that “plan” is just writing down everything you eat and keeping track of calories) Once you’ve retrained yourself to consume less and exercise portion control, life gets easier. I have found this to be true even though I should already know what I’m doing, having done it before. The natural “plan” that humans followed for millions of years involved not having unlimited access to highly processed food.

      • If people only ate foods “provided” by God, as opposed to those provided by Monsanto, General Mills, Campbell’s and other factories, no one would need a food plan.

        My grandmother lived to be over 100 and was never “good” nutritionally (she was addicted to chocolate and ate POUNDS of it weekly the last few decades of her life with never a sign of diabetes).

        But when her mother was pregnant with her, and when she was growing up, when her body was built, there simply WERE no GMOs, no HFCS, MSG, no processed vegetable oils, no “flavor packs”.

        All there was… was food. That’s what she ate, that’s what her mom ate, etc. Regular food… meat, vegetables, fruit, eggs, dairy, ACTUAL whole grains (and if not whole, ground right before using them).

        Food comes from farms or from the wild, not from factories. The vast majority of junk in grocery stores is NOT food.

      • I remember the age old saying “God helps those who help themselves”.

  • It’s articles like these that make me think every product should have a QR code on them that links to a document that fully details the processing steps that went into it. I think a lot of people would stop eating some packaged products if they knew how much was done to their food. “Ammonia bathed chicken parts “, anyone?

  • wow ! So much for that , what in the usa is not processed ?!?

  • It is becoming glaringly obvious anymore that there is absolutely nothing… yes, I said nothing worth spending my money on at the grocery stores!

  • I am just getting into trying to eat REAL food but this just disgusts me …

  • How sad! What about other juices? Pineapple, apple, etc? Are all juices made this way?

    • Andrea, I don’t know. Most of the information in this post originated from the book Squeezed: What You Don’t Know About Orange Juice, so it all pertains to how orange juice is processed. I don’t know how other juices are processed, only that after this I wouldn’t be surprised to discover something equally as disturbing.

    • I’m not sure the science behind making other juices, but I can tell you our experience from making our own juice. If we don’t drink the fresh juice right away & put it in the refrigerator, it starts to break down, separate and tastes different later. I would imagine all juices would need this type of process described to prevent this from happening on the store shelves.

  • Wow, I had NO idea about this! Fascinating! And disturbing! ; ) Thanks so much for spreading the information, and for the wonderful suggested alternatives at the end of your article. Much appreciation!

  • No more juice for me! I am going to join those who drink water with their meals and in between meals.

  • All and only fresh home squeezed here. My now 5-year old has known for years that the stuff in the carton doesn’t even taste like real orange juice!

  • My mom always said juice as expensive junk we couldn’t afford when we begged for it. I am in my late 40’s-guess my mom was a pretty smart shopper even then.

  • I wonder how “juice-tarians” will take this news. They may come at you with torches and pitchforks! LOL

  • huh, odd…I cannot repost this article to facebook, nor the one about the arsenic in chicken. Are you being sensored?! I actually love the boldness of these articles, please keep it up! But I’d love to be able to repost!!!!

    • Oh that’s weird. Have you tried using the Facebook share button on this page, under the “Sharing is Rebellious” tag line?

  • My question is why hasn’t the issue of quality control ( meaning were mix all of the oranges together to get one even flavor) hasn’t been brought up? Because you could pose this same exact question to all liquids and even foods such as milk, beer, wine, etc..
    Having the juice is such large vats that is required for large scale production is also going to yield a much more consistent flavor. Because when you think about it, even WITH adding in the extra chemical flavorings you will still get a slightly different end tasting product depending on the oranges that are used. And FTR I do notice a difference in cartons from time to time, especially the ones that have added calcium and vit. D, even thought we haven’t bought much juice at all in ages. Just my .02
    (P.S. If anyone has a Wegmans near them they carry fresh squeezed un-pasteurized OJ and it’s divine! Taste just like an orange. It’s pricey but you should really only be drinking juice for a special occasion anyways so I think it’s ok. I sure hope that doesn’t have the flavor packs added! )

    • While it’s true that the mixing of liquids in giant vats does make the flavor more homogenous, I still think the point stands, particularly when we talk about drinking orange juice when oranges are well out of season. “Fresh” juice made with out of season oranges shipped from Brazil or kept in cold storage for months on end ought to taste different than juice made “fresh” from perfectly ripe oranges, regardless of whether it’s in a big vat or not.

      Also, supermarket milk & beer are also made using recipes. (Did you know that “whole” milk in the supermarket isn’t even whole? It’s skim milk plus 3.75% milk fat.)

  • I’d love to know what they put in the juice to keep it from separating.

  • Excellent! I used to buy juice for my husband when we were first married. Eventually I got out of the habit of buying it at all.
    If we’ve done one thing right with our toddler – he’s only ever had a steady diet of fresh well water (filtered) and raw milk from local cows.
    Thanks again, kept your post going…

  • Thanks for the article. It explains a lot.
    I don’t normally drink fruit juice but had some orange juice while I was traveling in Europe and it tasted nothing like the OJ in the USA. Now I know why.

    Thanks for posting this.

  • …what exactly is the big deal? Do these flavor packets have any qualities known to be detrimental to human health?

    • The bigger question is, do they have anything in them to promote human health, you know, like all the advertisements say they do…

    • Well if the flavor packs were detrimental to health that would not stop the manufacturers from lying or ignoring the science – and doing it with total acceptance from the federal government.

    • J – I’m with you on this. People read ‘additives’ or ‘chemicals’, and everyone starts panicking.
      I would be more worried about the high amount of sugar in most juices.

    • @J Well, if we are not told they are there- that’s a big red flag for me.

    • I think the “big deal” is that some people try to avoid any proceessed foods, or any additives (some for allergic reasons, some for other reasons) and the fact that such a highly processed ingredient can hide in a “natural” product is infuriating.

      I hardly drink juice so it doesn’t bother me that much, but if I was avoiding all chemical additives for an allergy I can understand being very upset that my “all-natural orange juice” had chemical additives.

      • I understand people’s concern but the key fact to keep in mind here is that those chemicals (ethyl butyrate, terpenes, etc) are just the volatile flavor chemicals which were already present in the juice. My dad works in the flavor industry and makes these sort of things regularly. You’re not drinking anything that you wouldn’t already be getting when you juice an orange, that’s why the juice companies don’t have to list it on the label.

        If you don’t want to drink juice that’s great, it’s calorie-dense and nutrient-poor. But don’t go assuming that all chemicals are automatically bad for you. If it weren’t for those chemicals an orange wouldn’t taste like an orange when you bit into it.

        • Hannah, I appreciate your comment. I think what bugs me the most about the flavor industry is that they manufacture flavor for otherwise flavorless or unpalatable foods. I think if a food needs to have synthetic flavors added to it for us to enjoy it, then we ought to question whether or not it’s actually good for us and worth eating. It’s not so much that I think the flavors are unnaturally engineered chemicals (although sometimes, as with MSG, there is cause for concern).

      • Oh, and for the record “natural” on a food label does not have a clearly defined FDA definition. The FDA allows it to be used on foods with no added color, artificial flavors, or synthetic substances. “Natural flavors” means they were derived from plant (or animal I guess) sources and not synthesized, but the end product is the same chemical. This is not the dictionary definition of natural! Something to keep in mind if you’re concerned about the sources of your food.

    • My concern is more along the lines of deceitful product labeling. If we don’t even know what is in factory-made orange juice, how are we to comprehend what is in other food products?

    • The big deal is they’re LYING. The whole point of labelling laws is so I can DECIDE what I want to eat.

  • No. Way.
    And commenter named “J”–who was it that hired you to write that anyway? Use some common sense people.

    • Lori,

      Is it truly impossible for someone to have a differing opinion from you without being paid by a large corporation?

  • Well Lori, it certainly wasn’t the organic farmer who seems to be paying you.

  • I don’t buy OJ often and now I won’t buy it at all. I don’t think the fiber in fruits and veggies are at all good for people so I will use my juicer for make fresh juice.

  • Still a million times better than a glass of coke.

    There’s plenty of juice brands that are nice and thick with the pulp of the orange still in there, organic and healthy.. they might be more expensive though, and in the health food section. These juices are the real deal – they have shorter expiry dates than the supermarket brands, and you feel good after drinking a glass.

    Good point about the high dose of sugars we get from juice though. Probably best in moderation, or diluted with more water.

  • Yes! You’re right – I hate you! I could have done without knowing this. Is there a single food sold in the stores today that you can trust to know whats in it???

  • Hi,

    I’m a brazilian. I’d like to say that i like the taste of the Tropicana’s Orange Juice, but just like Minute Maid , i can tell that is not 100% natural from the first swallow.

    They don’t taste anything like our freshly squeezed orange juice that we are used to here.

    PS – Juices really are not the best way of consuming fruits. Just , as anything else, don’t drink too much of them through the day. Although it will always be healthier than any soda.

  • Sickening! Normally we are a no juice family; however, the past few months I have been craving grapefruit juice–so I caved and have been drinking some of that. I too always water it down, but how yuck!

  • I just love this stuff. Thank you Kristen!

  • This is interesting to know. I guess I’m in the minority here though, in that it doesn’t actually bother me, rather I see it as rather ingenious.

    I ask for orange juice out of season and this is how they get it to me. It generally tastes good and is made from natural (not that one would want to rely on the `naturalistic fallacy’) ingredients.

    You might argue that orange juice, or orange juice out of season, is a bad idea, but that’s a separate debate to the question of “is it OK to process OJ like this”.

  • Why do you advice against using all orange juice from the store? What about the orange juice from the cooled section that’s fresh, has fruit fibers in it and is only good for a couple of days? I don’t think it has gone through the process you describe.

  • Yeah, but…Why are you getting upset about these substances? Because they’re “chemically manipulated”? Wow, what a loaded phrase! Why do people assume “chemicals” are automatically unhealthy? It sounds like these are substances that have been used as flavorings for years, so there have to have been lots of studies about how safe they are. The FDA FORCES companies to do LOTS of food safety tests.

    If you’re upset about misleading labeling, that’s fine. If you want people to “get out of the juice habit”, there’s nothing wrong with that. But by trying to get people upset about “chemicals”, you’re telling them not to trust science.

  • “…depending on the oranges you made it from.” This sentence ends with a preposition, which can turn audiences against your credibility. Granted, not many, but in my opinion it isn’t worth it.

  • Wow. I’m put off drinking OJ now!

  • Milk or water for us (or homemade iced tea, in the summer). I rarely buy fruit juice, because I know it’s really just sugar without the good stuff! I’d much rather buy whole fruit. Now I have another reason to do so.

    @Lori (Laurel of Leaves)– I think J had an honest question. It’s unkind to assume that someone is “being paid” just because he doesn’t immediately agree with you.

    • You do realize that the milk that you buy in the store is also very different than what comes out of the cow, right? It doesn’t taste or smell near what it originally does, but if it did I doubt your children would drink it.

      Sometimes a little bit of processing actually makes a product better to consume.

      • More palatable, perhaps. Better is subjective in this case, and it is far from being “a little” processing, isn’t it?

      • Not all milk is processed. Real, raw milk comes out of the cow and goes into the jug. And my children do drink it.

        And by the way, it tastes a million times better than pasteurized homogenized milk.

      • OMG. Where do you get that idea? Raw milk fresh from the grass fed cow and cold chilled is the best tasting thing ever! More kids would drink milk if they had access to good milk and not that store bought crap.

      • Ummm, we DO buy milk as it comes from the cow, not as it comes from the grocery store.

        Your notions about what is palatable are… very inaccurate.

        It is like saying Cremora actually tastes better than cream! Or margarine is really yummier than butter!

        It’s just not so.

  • A really awesome researched article!
    I firmly believe that 99% of the man-made things around us, actually do us more harm then good.
    A remeber reading an article on harmful effects of soap and why we really dont need it.

    All a business cares about is money. They just dont give a damn to the slow poison they are feeding to kids. And ofcourse it cant happen without the goverment’s silent approval.
    Not sure when this endless hunger for money is going to end.

    apologies for the sad tone of the comment.
    now i am heading off to reading the other posts on ur site.

    thanks for giving me one more weapon to add to my list, which i will use whenever i will tell someone why using things directly from nature is a good thing

  • Why am I drinking juice? Because it tastes good, and gives me water and sugar when I feel the need for it.

    And because even if it does not have fibers, it does have vitamin C (I know that Vitamin C is slightly below the “not worth it” bar when it comes to supplements but having a normal supply is still good.)

    The thing with the flavour packs is weird because of the amount of work that goes into it when compared to just making the juice. I don’t really think they’re dangerous, though.

  • I heard about this initially on CBC radio when they were interviewing the author of “Squeezed: What You Don’t Know About Orange Juice”. Very interesting but not very surprising. We don’t drink juice normally but my husband does occasionally bring home the so-called fresh 100% juice. Sometimes I bring home the real thing from our local farmer’s market for a special treat, but it’s so expensive it’s not likely we’ll over do it!

  • Wow. I decided never to give my daughter juice for as long as we could get away with it. We are going on three years and most people think it’s weird. I must admit my reasons were simpler, but now I feel even better about my decision. Thanks!

  • Weird, I knew about this in bits and pieces but never put it together. My mom’s parents were orange pickers, and they worked from late November through February. I recall asking my grandmother how we could get orange juice year round and she said they either froze it or vacuum packed it. Vacuum packing is de-aeration.

    My mom worked at a Donald Duck orange juice plant in Florida in the 1970’s. Her job was to skim the rats off the top of the vats of juice that were being reconstituted. I never really considered how the juice was processed that it would need to be reconstituted.

  • So this who big point of this article is that the major orange juice companies have lied to us?

    I mean it’s extremely interesting that these juice companies manufacture the flavor, but how does that affect ones health? The articled ended basically saying that if your trying to be healthy juice is a bad choice anyways (all that liquid sugar your consuming).

    Is this really going to affect the juice that people buy? Its still the same juice, same nutritional content, same flavor (as you are used to) but a different process in which you assumed it was made.

    Maybe its because I’m not a health food advocate, but I just don’t understand the shock people are going through.

    If we found that this process affected peoples health, then that would be something to be shocked about.

  • Excellent article. My wife is from India and she won’t drink any packaged juices. According to Ayurveda the fruit starts to lose it’s good qualities 15min after being cut. So if we have juice we juice it ourselves ($60 juicer, works great). And we don’t keep any foods in the refrigator to eat the next day. Cook it and eat it fresh. We do have some breads and some frozen soya, but that’s it.

  • OMG! Its a CHEMICAL!?!?!??!!?

    Who cares about the flavor packs or manipulation? Is there any *evidence* to believe that this stuff is harmful?

  • That is very creepy and unpleasant. I shall stick with beer and water.

  • Thank you…we were just about to buy “the good stuff.” I think our new good stuff will be organic fresh fruit juiced in our juicer.

  • Yes, yes I can believe that. I get that response from much of my family when I point out the dangers of the food they eat.

  • Unfortunately, I see that often But… you have to press forward. Maybe this will plant the seed in their head… that nagging, irritating voice that you just want to squash that keeps preaching the news… and someday… someday, they’ll listen.

  • That’s so weird, bc yesterday as I read the article I was thinking how not everyone would care, only those who are concerned about the misleading information and have realized how it HAS had an effect on health. How do you prove what’s surrounding us ( sickness, cancer, unexplainable autoimmune disorders )
    Food definitley has been turned into a drug. With all the addictive nonfood ingredients. You go to a crack addict tell them it’ll cause death, they go back to it anyway. The ones who *really* want to quit hear reason. Also, like a smoker, inhalation of smoke is damaging, depletes of water, inhibits mineral absorption, do they care? Meh, when its convenient.

  • Thanks for the encouragement! I’m easily riled up when I see how our culture is perpetuating the most vast nutritional experiment ever conducted in human history, yet demanding that the burden of proof be placed on the traditionalists (who have the weight of history behind them) and not the innovators (who have only just started reaping what they’ve sown in terms of diseases of civilization).

  • Yikes! I just got home from store where i purchased oj! Then read this article!

  • I was just saying this today about how there are people who simple do not want to be bothered with the facts. I feel that we are all beings that must do our part and help one another when we can. But first and foremost I believe we must take care of ourselves and then we can help those that we are able to. So it is my stance that for those who do not care or want to know the facts – fine – but do not block or interfere with the information and messages of people that do want this material. I realized how frustrating it has become in my own life and career in healthcare when people want to debate. i am not here to debate. I am here to teach; as I learn I teach. We are ever changing and so is information. We must simply remain conscious that is all. Keep this information alive and flowing because we DO need it.

  • It is infuriating. I have so many major issues with the FDA and what they get away with not disclosing vs the way they handle real food. (did you see the article about walnuts being classified as a drug?)

  • I had been buying the “freshly squeezed” brand, because it is promoted in the SCD as not containing additional sugar. I tell you, some days I want to give up and just stop eating.

  • When I met my boyfriend, he didn’t know/didn’t care. 3 years later, when I gave him the recap of this article yesterday, he was extremely grossed out.

  • I don’t find that surprising at all – I think it’s the vast majority of people who don’t care to research or believe any of this information. And they are the same people who will tell you their health is seriously failing and they have no idea why, “probably my terrible genetics” they say. lol

  • @Debbie — You remind me of a friend of mine who has 5 kids and has to feed a family of 7 on about $500/month. They make an annual grass-fed beef order, buy raw milk, and buy staples (like wheat & coconut oil) in bulk buying clubs. BUT they buy everything else at the grocery store and try to make the best decisions they can with what the store carries. Sometimes this friend would walk into the store to buy groceries… and walk out with nothing an hour later in frustration. She had to send her husband to buy their essentials because she was too avid a label reader and too knowledgeable about how food is processed to feel good about ANY of her choices.

  • awesome article! thanks for sharing!

  • It’s sad. My brother-in-law is like that. Always poo-pooed anything I had to say on health matters, and now he’s diabetic and I’m not. And he’s likely to stay that way, because he believes the lies the American Diabetic says about “managing” care. Forget managing it; cure it like so many others have!

  • Thanks for this thread! I was a bit low yesterday after seeing the comments made regarding “blueberries” in processed food items from another forum. They were verging on violent, as if they wanted to tear the heads off of anyone who chooses to discern ethics from complacency. I was just an observer of comments and chose not to participate, however, I am reminded once again that indeed the only thing we can do is provide the info and set the example. There will always be someone who’s ears are primed to hear and eyes are made wide open. They’ve travelled a journey that rendered them able to learn and that’s why we’re here. I am so thankful for this forum, for the real food bloggers, the holistic medicine forums, the politics of food/med bloggers….This is actually an isolating life to lead for many since having an entire family on board may be rare but as long as I have a community to retreat too, one that educates responsibly and lives by example, that’s enough to keep me going. Keep up the amazing work, even amongst the negatives.

  • We expect people to have a rational response to truth, it’s only logical. But when confronted with truth, often the response is emotionally based.

  • You can only help those that want to help themselves.

  • It is so frustrating! We don’t have access to a variety of organic fruits to do our own juicing! Mainly strawberries and bananas….

  • I am the one that gets laughed at as I try to make changes to our diet from processed to whole, organic. When I bring up things they roll their eyes.

  • I know, it’s baffling to me. I just dont get why you need a study to tell you that drinking perfume is not ok. Bizarre.

  • Even aside from the “flavor packs” I had no idea that they store the juice in vats for up to a YEAR! That grosses me out, I haven’t been a juice consumer for at least 5 years but it still makes me feel ill. How do you call something fresh when it’s been stored for a year?!

  • we are definitely making changes in our home as to what products we buy/use! If anyone is interested I can talk to them about being greener & safer! My website is StartYourPlanToday.com
    And Keria is right…everyone makes their own choices, you can only do so much!

  • I learned about one of these plants that make flavors and the chemicals used to make them, its really scary stuff! I am glad we have a juicer it may be messy but at least we know what we’re consuming.

  • Interesting information, but in reality, most Americans will not benefit from it. We are a family of 6, four kids in college (all of them working part-time to help with tuition). My husband managed to keep his job by accepting a transfer to a city 53 miles away. His salary was trimmed to 39,00 per year; I recently became unemployed. Organic vegetables and fruits, dairy products and meats, not to mention grains, are WAY beyond our budget. We have no suitable space for gardening and livestock is not allowed by local zoning ordinances. Rather than spend about 5/6 of our income on food that’s “better” for us (yes I priced it out), we choose to pay our small mortgage, wear clothing, keep our two older cars running and pay our utility bills.Yes, I shop at the evil, evil local grocer and no, I am not ashamed to do so.

    • You’re right, healthier food is expensive, and it certainly seemed even more when I was at college and working part-time like your kids.

      We’re caught in a vicious loop where more of us would buy better food if only we could afford it, but the food won’t get affordable until more of us buy it. I’d suggest looking at some of the recipes on this blog. Kombucha certainly looks fascinating.

      Also, since you seem to have put some thought into this and run the numbers, try and see if changing one or two things might be feasible. Can you change only your milk? Or cut down on fizzy drinks? Every small step helps, and a diet with only one or two healthy components is still a couple of steps ahead of one with none at all.

    • Uh, it’s cheaper to NOT buy fruit juice and to drink tap water instead. Not really sure what you are arguing about.

    • The food is not “better” it’s BETTER. I agree with Croc, you have thought it out so you should find ways to x out small grocery store items that you can live without or replace at a local farmer’s market or health food store. It really isn’t so hard, even when you have a family. And no one is saying grocery stores are evil and you should even be ashamed to go to them. We have to eat, we have to survive, we do what we have to do to make that happen. But making a decision to do that the best way possible is what counts. Kristen also has a blog on if you want to start eating better and you’re new to it all. She emphasizes baby steps. You should check it out, it helped me a lot.

  • By “single setting” I think you mean “single sitting”

  • I find this shocking because I always thought the flavor was of freshly squeezed oranges, not a “flavor pack” that is added to emulate the original flavor of oranges; it essentially is an artificial flavor. Realizing that orange juice is no longer authentic has made me reconsider my OJD, obsessive juice disorder, Florida’s Natural with pulp

  • Crash your app in style (Golang)

    26 November 2015 - 12:11pm
    maruel/panicparse · GitHub

    Skip to content

    Something went wrong with that request. Please try again.

    You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.

    HTML5 Custom Elements: Contentious Bits

    26 November 2015 - 11:32am
    Custom Elements: Contentious Bits · w3c/webcomponents Wiki · GitHub

    Skip to content

    Something went wrong with that request. Please try again.

    You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.

    MongooseIM version 1.6: Riak, DevOps and more

    26 November 2015 - 11:27am

    MongooseIM 1.6: Riak, DevOps love, and so much more! November 25, 2015 by Nicolas Vérité and Joseph Yasemides

    Most of the development work around MongooseIM is done by working closely with our clients. Many of the new features in MongooseIM 1.6 are the result of building and testing around particular client requirements. We are particularly excited about this release, as it means the beginning of an improved team dynamic, new high-value features, and many improvements.

    What’s new in MongooseIM 1.6 Riak KV

    MongooseIM 1.6 introduces support for Riak - the scalable, fault-tolerant, distributed database written in Erlang. For this release, four modules can be configured with Riak, but more are on their way.

    Currently, MongooseIM supports Riak as a storage backend for:

    • Authentication
    • Message Archive Management (MAM, XEP-0313) for one-to-one conversations
    • vCards (XEP-0054)
    • Private XML storage (XEP-0049)

    This brings more flexibility in terms of database choice for any infrastructure, as you are free to choose between a RDBMS or NOSQL data store.

    Powerful metrics, DevOps love

    Version 1.6 offers very powerful metrics and monitoring infrastructure. For the underlying instrumentation, MongooseIM relies on Exometer. It reports OS-level and Erlang VM metrics, in addition to business metrics. You can then push those metrics to any ingesting and graphing system, such as Grafana or Kibana.

    This greatly improves DevOps' visibility for managing systems, no matter the size of the installation. Other departments will benefit from having more data to dig into (big data, analytics), thus better understanding the end-users and eventually discovering new opportunities to improve UX, core competencies, and selling points.

    Additionally, we have improved how session data from the DB is cleaned up when a node goes down. It eases the support for more database backends in the future, and it brings better cluster handling for DevOps.

    Also, it is now possible to change log-level dynamically with a custom log path. For DevOps, once again, it helps consolidated logging for easier deployment, administration, and analysis.

    Get your own easy-to deploy Docker image of MongooseIM. As it is still experimental and will improve over time, please handle it with care: https://hub.docker.com/r/mongooseim/mongooseim-docker/. Give it a test drive, come back to us with feedback or questions and tell us and what you'd like to see in the future. Feel free to fork it from: https://github.com/ppikula/mongooseim-docker.

    Additional high value improvements

    Extensive technical investment means we can continue to deliver a better MongooseIM with the Open Source community: we made changes to the core and integrated our test suite, the first of its kind, in the hope of seeing even more Open Source contribution.

    Selected improvements:

    • Supported XEPs are now documented on https://github.com/esl/MongooseIM
    • Substantial refactoring: authentication mechanism, c2s, simplified MAM
    • Erlang/OTP 18 support
    • Better RFC & XEP conformance

    We encourage everybody to review the release notes on GitHub.

    Coming up next

    MongooseIM 1.6 will continue to bring many benefits such as: extensive and powerful business metrics, flexibility for DevOps, a rock-solid code base and stability for server and backend developers, more conformance for client developers, and extensive testing for all.

    1.6.x maintenance series

    We will be working on a maintenance version, 1.6.1 (and perhaps a 1.6.2 later), with more complete Riak support, and of course the usual bug fixes, optimisations, and general improvements.

    1.7 and subsequent releases

    The development cycle for version 1.7 has already begun! We will focus on cloud, mobile, and testing. That is all we can share for now, but feel free to tell us what you think. We will also put more effort and commitment into making release schedules more predictable - in fact we already are.

    Invitations Be the first to get informed!

    We have set up a new public mailing-list for all announcements of major events happening on the MongooseIM front. Expect one or two emails per month, the archives are free and open. We highly encourage you to subscribe here: https://groups.google.com/d/forum/mongooseim-announce

    Click on the blue button "Join group":

    Then click in "Email delivery preference" on "Notify me for every new message":


    We received some very valuable contributions over the last months, and we would like to thank all of the developers who took part in the delivery of version 1.6: @rgrinberg, @vooolll, @syhpoon, @mweibel, @Stelminator, @larshesel, @ruanpienaar,@aszlig, @jonathanve, @gmodarelli.

    We hope to see everyone contributing again in the coming months - you have all the power to closely participate in this fully open source project.

    Influence our roadmap!

    We encourage you to comment below this blog post, tell us what you think about this release, and where we should go in the future. You can also give feedback on GitHub through “Issues” - we are keen on gathering common problems and goals to provide solutions.

    We believe the best thing to do is fork the project, and make a pull requests when you are happy. We will discuss with you, and include that into a milestone, entering our fully automated testing process.

    Go back to the blog Share Download

    Please enable JavaScript to view the comments powered by Disqus.

    I may be the only evil (bit) user on the internet

    26 November 2015 - 9:26am

    Almost every year a joke RFC is made on April 1st (these have caught on so well, that it’s now common to see more than one of these every year), or April Fool’s Day, and some of them are pretty great. Here are some of my favourites:

    • SONET to Sonnet Translation RFC1605
    • Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0) RFC2324
    • UTF-9 and UTF-18 RFC4042
    • TCP Option to Denote Packet Mood RFC8541
    • The Null Packet RFC6592
    • Scenic Routing for IPv6 RFC7511

    Though you can find a full list here: link

    However my favourite one so far is RFC3514 or the “The Security Flag in the IPv4 Header”

    One of the things that make RFC like the RFC3514 and RFC2324 (and others) great is that they are actually implementable, as in the case of the “Evil bit”. There is nothing stopping you from actually doing it.

    Curious by this as of late, I wondered if anyone out there actually sets the bit. It isn’t a very well acknowledged RFC: even though it’s under the “Informational” category (as are all April Fool’s RFCs) it’s not even recognized by Wireshark and other systems

    Some would say that Wireshark isn’t RFC3514 aware :) (Though someone tried to make it link)

    It’s even harder to look for these kind of packet in tools like tcpdump, you have to resort to a rather unobvious filter of ip[6:1] & 0x80 = 0x80

    In FreeBSD someone actually did the work and made it a option for your IP stack link however the patch was only in FreeBSD for around 24 hours before being reverted link

    I have not been able to find any attempt at having this in the Linux kernel, so I made a patch link for it. Very rudimentary, forces you to always have the evil bit enabled on your outgoing packets (good enough for myself).

    ben@metropolis:~$ uname -v #110+evil SMP Sat Oct 24 19:08:01 BST 2015

    Now that I have my home desktop recompiled with my new “evil” kernel, I am now sending almost all of my outbound packets with the evil bit set.

    Am I alone in this?

    In the RFC it mentions the following:

    Devices such as firewalls MUST drop all inbound packets that have the evil bit set. Packets with the evil bit off MUST NOT be dropped. Dropped packets SHOULD be noted in the appropriate MIB variable.

    After spending 2 weeks browsing normally with my hacked kernel, I found only one site that I could not access only from my desktop. freedesktop.org appears to have rules set to drop evil bit packets. freedesktop.org is in the IP space of Portland University, so after even more digging I found that all of the Portland University address space drops evil bit packets!

    This inspired me to go searching for sites that also had done this in the alexa 100k (at the time of writing, freedesktop ranks 35,426 on there link). After doing scans if I could connect to port 80 from a PC that had no evil bit kernel, and a normal one on the same network, I found the following list of domains that only failed on my evil bit computer, some of the interesting domains are:


    • cf.ac.uk
    • pdx.edu
    • skku.edu
    • shmtu.edu.cn
    • tc.edu.tw
    • cardiff.ac.uk
    • upf.edu
    • uq.edu.au
    • usc.edu
    • chapman.edu
    • kctcs.edu
    • lonestar.edu
    • missouristate.edu

    Banks/money handling companies:

    • citruspay.com
    • rbs.co.uk
    • rbsdigital.com
    • natwest.com

    Software Sites/Companies:

    • kaspersky.co.jp
    • kaspersky.com
    • kaspersky.ru
    • teamviewer.com
    • caniuse.com
    • unicode.org

    You can find the full list of top alexa 20k domains to look at here: link

    I highly suspect the thing that links all of these sites together is a common appliance, since most of these sites are not under a security service, with the one exception of ddos-guard.net who seem to filter it on all inbound to their network. None of the listed sites above (other than the full list) use ddos-guard.net.

    So now we know that sites target this bit to block, but the real question is why? Is it that someone didn’t see the date of the RFC, maybe sarcasm doesn’t translate very well, possibly someone in the real world actually sent the evil bit when doing evil things, and cause some products to target it?

    Time for some security by obscurity

    Now, it’s fairly obvious at this point that this is a bit that isn’t used that much. However lucky for you we can use that to lock down a port so it is only visible to “evil” people, meaning that most (if not all I suspect) scanners will never see this port, but people using the kernel patch can see it perfectly fine and without IP blocking.

    Unfortunately iptables itself does not have a way to target the IP flags on a packet, however there is a useful u32 module for iptables link that you can use to target random bits in packets for your own use. The following IPTable chain whitelists people who use the evil bit to access a port:

    iptables -N evil iptables -I INPUT -p tcp --destination-port <port> -j evil iptables -m u32 --u32 "3&0x80>>7=1" -A evil -j ACCEPT iptables -A evil -j DROP Obligatory tester

    Please load javascript!


    If you see a green “Evil is set” thing above ( aka this ), do let me know on either my email, [email protected] or tweeting me. Do let me know what equipment you also use.

    If you happen to run one of the sites I have mentioned that drop “evil” traffic, please also let me know what firewall/router setup you use, I would love to know what drops the evil bit by default.

    Iron law of oligarchy

    26 November 2015 - 6:41am

    This article may contain improper references to self-published sources. Please help improve it by removing references to unreliable sources, where they are used inappropriately. (May 2015)

    The iron law of oligarchy is a political theory, first developed by the German sociologist Robert Michels in his 1911 book, Political Parties.[1] It claims that rule by an elite, or oligarchy, is inevitable as an "iron law" within any democratic organization as part of the "tactical and technical necessities" of organization.[1]

    Michels theory states that all complex organizations, regardless of how democratic they are when started, eventually develop into oligarchies. Michels observed that since no sufficiently large and complex organization can function purely as a direct democracy; power within an organization will always get delegated to individuals within that group, elected or otherwise.

    Using anecdotes from political parties and trade unions struggling to operate democratically to build his argument in 1911, Michels addressed the application of this law to representative democracy, and stated: "Who says organization, says oligarchy."[1] He went on to state that "Historical evolution mocks all the prophylactic measures that have been adopted for the prevention of oligarchy."[1]

    According to Michels all organizations eventually come to be run by a "leadership class", who often function as paid administrators, executives, spokespersons, political strategists, organizers, etc. for the organization. Far from being "servants of the masses", Michels argues this "leadership class" will inevitably grow to dominate the organization's power structures rather than its membership. By controlling who has access to information, those in power can centralize their power successfully, often with little accountability, due to the apathy, indifference and non-participation most rank-and-file members have in relation to their organization's decision-making processes. Michels argues that democratic attempts to hold leadership positions accountable are prone to fail, since with power comes the ability to reward loyalty, the ability to control information about the organization, and the ability to control what procedures the organization follows when making decisions. All of these mechanisms can be used to strongly influence the outcome of any decisions made 'democratically' by members.[2]

    Michels stated that the official goal of representative democracy of eliminating elite rule was impossible, that representative democracy is a façade legitimizing the rule of a particular elite, and that elite rule, that he refers to as oligarchy, is inevitable.[1] Later Michels migrated to Italy and joined Benito Mussolini's Fascist Party, as he believed this was the next legitimate step of modern societies. The thesis became popular once more in post-war America with the publication of Union Democracy: The Internal Politics of the International Typographical Union (1956) and during the red scare brought about by McCarthyism.



    In 1911 Robert Michels argued that paradoxically the socialist parties of Europe, despite their democratic ideology and provisions for mass participation, seemed to be dominated by their leaders just like traditional conservative parties. Michels' conclusion was that the problem lay in the very nature of organizations. The more liberal and democratic modern era allowed the formation of organizations with innovative and revolutionary goals, but as such organizations become more complex, they became less and less democratic and revolutionary. Michels formulated the "Iron Law of Oligarchy": "Who says organization, says oligarchy."[3][4]

    At the time Michels formulated his Law, he was an anarcho-syndicalist.[4] He later became an important ideologue of Benito Mussolini's fascist regime in Italy, teaching economics at the University of Perugia.[5][6]


    Michels stressed several factors that underlie the Iron Law of Oligarchy. Darcy K. Leach summarized them briefly as: "Bureaucracy happens. If bureaucracy happens, power rises. Power corrupts."[4] Any large organization, Michels pointed out, has to create a bureaucracy in order to maintain its efficiency as it becomes larger—many decisions have to be made daily that cannot be made by large numbers of disorganized people. For the organization to function effectively, centralization has to occur and power will end up in the hands of a few. Those few—the oligarchy—will use all means necessary to preserve and further increase their power.[3][4]

    This process is further compounded as delegation is necessary in any large organization, as thousands—sometimes hundreds of thousands—of members cannot make decisions via participatory democracy. This has been dictated by the lack of technological means for large numbers of people to meet and debate, and also by matters related to crowd psychology, as Michels argued that people feel a need to be led. Delegation, however, leads to specialization—to the development of knowledge bases, skills and resources among a leadership—which further alienates the leadership from the rank and file and entrenches the leadership in office.

    Bureaucratization and specialization are the driving processes behind the Iron Law. They result in the rise of a group of professional administrators in a hierarchical organization, which in turn leads to the rationalization and routinization of authority and decision making, a process described first and perhaps best by Max Weber, later by John Kenneth Galbraith, and to a lesser and more cynical extent by the Peter Principle.

    Bureaucracy by design leads to centralization of power by the leaders. Leaders also have control over sanctions and rewards. They tend to promote those who share their opinions, which inevitably leads to self-perpetuating oligarchy.[3] People achieve leadership positions because they have above-average political skill (see charismatic authority). As they advance in their careers, their power and prestige increases. Leaders control the information that flows down the channels of communication, censoring what they do not want the rank-and-file to know. Leaders will also dedicate significant resources to persuade the rank-and-file of the rightness of their views.[3] This is compatible with most societies: people are taught to obey those in positions of authority. Therefore, the rank and file show little initiative, and wait for the leaders to exercise their judgment and issue directives to follow.


    The "iron law of oligarchy" states that all forms of organization, regardless of how democratic they may be at the start, will eventually and inevitably develop oligarchic tendencies, thus making true democracy practically and theoretically impossible, especially in large groups and complex organizations. The relative structural fluidity in a small-scale democracy succumbs to "social viscosity" in a large-scale organization. According to the "iron law," democracy and large-scale organization are incompatible.

    Examples and exceptions[edit]

    An example that Michels used in his book was Germany's Social Democratic Party.[4]

    The size and complexity of a group or organization is important to the Iron Law as well. During the 1970s and early 1980s, the Green Party of Germany made a conscious effort to break the Iron Law.[7] Anyone could be or could remove a party official. There were no permanent offices or officers. Even the smallest, most routine decisions could be put up for discussion and to a vote. When the party was small, these anti-oligarchic measures enjoyed some success. But as the organization grew larger and the party became more successful, the need to effectively compete in elections, raise funds, run large rallies and demonstrations and work with other political parties once elected, led the Greens to adopt more conventional structures and practices.

    Labour Unions and Lipset's Union Democracy[edit]

    One of the best known exceptions to the iron law of oligarchy was the now defunct International Typographical Union, described by Seymour Martin Lipset in his 1956 book, Union Democracy.[8] Lipset suggests a number of factors that existed in the ITU that are allegedly responsible for countering this tendency toward bureaucratic oligarchy. The first and perhaps most important has to do with the way the union was founded. Unlike many other unions (e.g., the CIO's United Steel Workers of America (USWA), and numerous other craft unions) which were organized from the top down, the ITU had a number of large, strong, local unions who valued their autonomy, which existed long before the international was formed. This local autonomy was strengthened by the economy of the printing industry which operated in largely local and regional markets, with little competition from other geographical areas. Large locals continued to jealously guard this autonomy against encroachments by international officers. Second, the existence of factions helped place a check on the oligarchic tendencies that existed at the national headquarters. Leaders that are unchecked tend to develop larger salaries and more sumptuous lifestyles, making them unwilling to go back to their previous jobs. But with a powerful out faction ready to expose profligacy, no leaders dared take overly generous personal remuneration. These two factors were compelling in the ITU case.

    Lipset and his collaborators also cite a number of other factors which are specific to craft unions in general and the printing crafts in particular, including the homogeneity of the membership, with respect to their work and lifestyles, their identification with their craft, their more middle class lifestyle and pay. For this latter point he draws upon Aristotle who argued that a democratic polity was most likely where there was a large, stable middle class, and the extremes of wealth and poverty were not great. Finally, the authors note the irregular work hours which led shopmates to spend more of their leisure time together. These latter factors are less persuasive, since they do not apply to many industrial forms of organization, where the greatest amount of trade union democracy has developed in recent times.[9]

    University Student Unions[edit]

    Titus Gregory uses Michels "iron law" to describe how the democratic centralist structure of the Canadian Federation of Students, consisting of individual student unions, encourages oligarchy.

    Titus Gregory argues that University Students' union today "exhibit both oligarchical and democratic tendencies." Unlike unions they have an ideologically diverse membership, and frequently have competitive democratic elections covered by independent campus media who guard their independence. These factors are strongly democratizing influences, creating conditions similar to those described by Lipset about the ITU. However, Gregory argues student unions can also be highly undemocratic and oligarchical as a result of the transient membership of the students involved. Every year between one quarter and one half of the membership turns over, and Gregory argues this creates a situation where elected student leaders become dependent on student union staff for institutional memory and guidance. Since many students' union extract compulsory fees from their transient membership, and many smaller colleges and/or commuter campus can extract this money with little accountability, oligarchical behaviour becomes encouraged. For example, Gregory points out how often student union election rules "operate under tyrannical rules and regulations" that are used frequently by those in power to disqualify or exclude would-be election challengers. Gregory concludes that students' unions can "resist the iron law of oligarchy" if they have "an engaged student community", an "independent student media," a "strong tradition of freedom of information," and an "unbiased elections authority" capable of administrating elections fairly. [10]

    See also[edit]
    1. ^ a b c d e James L. Hyland. Democratic theory: the philosophical foundations. Manchester, England, UK; New York, New York, USA: Manchester University Press ND, 1995. p. 247.
    2. ^ Robert Michels, Political Parties: A Sociological Study of the Oligarchical Tendencies of Modern Democracy, 1915, trans. Eden and Cedar Paul (Kitchener, Ontario: Batoche Books, 2001), 241, http://socserv2.socsci.mcmaster.ca/~econ/ugcm/3ll3/michels/polipart.pdf.
    3. ^ a b c d Frank W. Elwell, Max Weber's Home Page "A site for undergraduates" at Rogers State University]. Last accessed on 27 May 2006
    4. ^ a b c d e Darcy K. Leach, The Iron Law of What Again? Conceptualizing Oligarchy Across Organizational Forms, Sociological Theory, Volume 23, Number 3, September 2005 , pp. 312-337(26). IngentaConnect
    5. ^ Nicos P. Mouzelis (1968). Organisation and bureaucracy: an analysis of modern theories. Transaction Publishers. p. 29. ISBN 978-0-202-30078-8
    6. ^ Gerald Friedman (2007). Reigniting the labor movement: restoring means to ends in a democratic labor movement. Psychology Press. p. 53. ISBN 978-0-415-77071-2
    7. ^ . Red Pepper. 2014-06-26. Retrieved 2014-07-10. 
    8. ^ Citation Classics Commentary on Union Democracy PDF (254 KB), Seymour Martin Lipset, 20/1988. Last accessed on 16 September 2006
    9. ^ . Ou.edu. 1920-06-12. Retrieved 2014-07-10. 
    10. ^ (PDF). March 2010. p. 115. Retrieved 2015-08-27. 
    • Michels, Robert. 1915. Political Parties: A Sociological Study of the Oligarchical Tendencies of Modern Democracy. Translated into English by Eden Paul and Cedar Paul. New York: The Free Press. From the 1911 German source.
    • Robert Michels und das eherne Gesetz der Oligarchie by Gustav Wagner in "Wer wählt, hat seine Stimme abgegeben" Graswurzel Revolution pp. 28
    External links[edit]