Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Tell HN: Paywalls with workarounds are OK; paywall complaints are off topic
176 points by dang on Sept 6, 2015 | hide | past | favorite | 160 comments
Publications like NYT, WSJ, the Economist, and the New Yorker have paywalls that leave ways for readers to work around them. Such stories are OK to post to Hacker News. Yes, it sucks, but losing that many substantive articles would suck worse. In the future, when someone doesn't understand this, please direct them to this thread or to HN's FAQ [1], which now makes this explicit.

Complaints about paywalls are off topic, so please don't post them. The spirit of HN is to discuss specific articles and avoid generic rehashing. Arguments about The Paywall Question are all the same. For an example of what we want to avoid, see [2]. For more on our thinking, see [3].

It's ok to ask how to read an article or to help other users by sharing a workaround. But please do this without going on about paywalls. Focus on the content.

1. https://news.ycombinator.com/newsfaq.html

2. https://news.ycombinator.com/item?id=10178012

3. https://hn.algolia.com/?query=by:dang%20paywall&sort=byDate&...



Is it OK to mention it when it appears that a poster is spamming paywalled content for a specific site? For example, this guy [1] stopped posting altogether almost immediately after I politely pointed out the sheer volume of paywalled WSJ articles he was posting [2]. It isn't inconceivable that this account (and many others) were created solely to spam paywalled content to HN. If we are forbidden from mentioning such things, I think we are going to have a problem around here with many more attempts by paywalled sites to exploit HN.

[1] https://news.ycombinator.com/submitted?id=abetaha

[2] https://news.ycombinator.com/item?id=9641922


Sure, that's such a special case that I don't see why it would be a problem. You can also email us (hn@ycombinator.com) when an account is using HN solely to promote a particular site. If we can email them, we often ask people not to do that. And in egregious cases we take away submission privileges.


People out to promote one specific site can happen regardless of if the site has a paywall. Someone could always promote one specific ad supported site as well.


This will get worse. Ad revenue per click for news sites is down so much that most non-paywalled news sites are now utter crap.

If you're posting a story which began with a press release, it's better to find the original press release (probably on PR Newswire) and link to that. At least you can read the hype before it was munged by some minimum-wage Demand Media employee.


It'll be interesting where it goes next. The content on most news sites is complete trash, and they are obviously getting desperate to make money in any way possible.


> paywalls that leave ways for readers to work around them

I would call pasting-the-URL-into-Google-Search less of a intentional workaround and more of a trick to take advantage of the websites' compliance with Google rules.

Not every HN reader would know to do that, or look in the comments for that "workaround."


> Not every HN reader would know to do that.

That's right, so it's ok for people to ask and share how to read an article in the comments. There shouldn't need to be more than one or two comments about this, and it helps everyone focus on the content.

What's off-topic is the generic tangent of paywall complaining.


So all I need to do is read every comment, then I can try to find out how to read the article?


If an article is from a brand-new source that uses a kind of paywall you've never encountered before, then hopefully someone will comment with a workaround. If not, you can ask. Maybe you could even subscribe to publications you like, read regularly, and want to support.

Since almost all paywalled articles are from the WSJ, the Economist, or the NYT, this shouldn't happen to you very often.


Eventually you learn these things, from living life (including reading HN for awhile). And if you can't read an article because there's no workaround, or because you don't know of a workaround:

- Just don't read the article.

- Subscribe. If you can't/won't afford it, then see above, or see below.

- Search for other sources of the information. And post them, it adds to the discussion. Most articles worth taking up space, particularly on paywalled sites, are worth that space in other venues. Almost nothing is exclusive, not after a day anyway.

In the WSJ case, I've noticed that yahoo often prints the article verbatim.


Personally, I just google the key words in the headline and find it. I meant for others who have not had the benefit of experience.

Yes, over time you learn things.


That's one option. You can also google the title or URL of the article (this is the most common workaround); or you can search the comments for the word 'paywall'; or you can purchase a membership or subscription for the paywalled site; or you can skip reading the article.


So all I need to do is try every possible option? And even then it may fail (scientific journals, newspaper archives, etc)?

The links are just huge wastes of time. A prominent tag attached to the article would be ok, but in the absence of any other feature to avoid these time sinks, it makes sense to flag the articles to save others from additional wastage.


The links are a waste of time to you. Other people have useful subscriptions or know the work-arounds and those links are useful to them.

Of the flood of links posted to /newest the paywalled links are nowhere near the most problematic.


I'm curious what people think are the most problematic links?


Or we can stop showing paywalled links to people and not support this crap as a community.


It's either this or ads, and ads don't work anymore because people use adblock.


Most people don't use adblock and the ones that do weren't clicking on ads to start with.


This site has a graph that say 55% of visitors to gaming sites use adblock http://contently.com/strategist/2015/07/10/why-adblockers-sh... . I would assume hacker news visitors would have a similar number. That would be an interesting thing to measure. Someone who gets to the front page of HN should measure what percent of people with a HN referer block ads.

And many ads are paid per view, not per click.


I don't agree with the moderation decision, but given what this site is 'supposed' to be I suggest you step up your game.


In practice, this is memoized per site. Few HN readers don't know to open incognito windows for NYT pieces.


Chrome also allows cookie blocking, no need to interrupt your browsing workflow.


Good point. We don't use such techniques when moderating because it seems important not to insulate ourselves from the typical reader experience.


If these paywall sites are being officially supported, why not reroute users directly to the google search for these articles?


I've thought about trying to solve this problem with software, but it feels like a line we probably shouldn't cross. Hence the current answer: it's fine for users to help each other read articles. That seems unimpeachable, whereas having HN officially undermine paywalls seems like a Schrödinger can of worms if not a classical one.


I disagree.

The news sites have made the economic calculation that allowing access to traffic from content aggregators like Google (which is the price of being discoverable by Google) is worthwhile.

The idea that only sufficiently large aggregators/traffic sources should get a special pass seems preposterous; anyone trying to enforce would be engaged in downright anticompetitive behavior.

The cat is already dead, can we please open the box & acknowledge the source of the foul smell?


Maybe you can automatically put paywall bypass instructions in the "TEXT" portion of the URL submissions?

edit: The auto-generated bypass instructions will get the top-sorted/top-comment favoritism that we normally try to avoid from users.

If sites don't want people to bypass paywalls, then they would not allow "special" ways to bypass paywalls. The fact that some paywalls have special referrer bypass rules reeks of financially motivated favoritism and entrenched interests preventing competition; the next search engine startup to be created is going to have a rough time of it.


That sounds like it would be a worse experience for readers who are paying subscribers of the article's source.


Because it doesn’t work everywhere.


Since I can't be the only one looking for things to break, I'll ask: What if I complain about a paywall and provide a mirror at the same time?


Then I suppose your remark would be partly on and partly off topic.


Mirrors and workarounds should be in the post to begin with. No link (not just on HN, on the internet) should lead to paywalled content without clear indication. Providing a mirror or at least spelling out the best workarounds (Yes every.single.time) should be mandatory.

Edit: I understand it could be controversial for HN to officially push links that pierce paywalls. But a tag users can avoid wasting time would be appropriate, just like a link to a discussion of common workarounds.


Would we also need to carve out an exception for the typically vapid "announcement" articles advertising paywalled academic journal papers?

The announcement-mills (phys.org comes to mind but there are plenty of others including nature.com itself) are not really "original" sources, the papers are, but such announcement-advertisement articles are submitted regularly.

Finding the freely available pre-print and/or author provided copies without resorting to (ahem) other workarounds is a pain but useful.


I'm reluctant to say that paywalls with no workaround should be banned outright, but obviously they're not covered by the "ok" policy.

Sometimes people post these and others respond with links to freely available versions, or articles about the work. In such cases we're happy to update the URLs.

We're not happy about announcement mills either (and those sites are penalized on HN), but that's arguably a separate problem.


My concern is with the interaction of the "original source" rule and the "ok for paywalls with workarounds" rule preventing most articles on new research. The "original source" papers are often locked behind paywalls (or embargoes) when the insipid announcement mill advertisement articles start appearing. Even supposedly reputable official university press sites are thoroughly guilty these overtly promotional teaser articles. The trouble is, lightweight advertisement articles are often the only things we can (legally) access when the "news" first becomes public.

As much as I hate to admit it, the sad state of suckage for announcement mills (including university press sites) actually does have some minor advantages; which would you be more inclined to read and up-vote?

"Astronomers detect furthest galaxy yet with Keck telescope"

or

"Lyman-Alpha Emission From A Luminous Z=8.68 Galaxy: Implications For Galaxies As Tracers Of Cosmic Reionization"

Non-Astronomers would be lucky if they understand the details presented in just the abstract of the paper, and I say this as a non-astronomer who does _NOT_ understand all of said details. Reading original source papers takes far more effort than reading lightweight announcements, and this gets to the fundamental question of, "What do we want HN to be?"

The status quo of interested HN users finding and comment-linking to the original source papers (if available) on the puff-piece stories is a lot of manual work and some stuff gets missed, but it really does tend to work out reasonably well. If we forbid paywalls without workarounds and require original sources, then we will miss out on a lot of great new research. Besides infringement, there is no easy answer for this situation.


I’d like to complain: For me, from Germany, the Paywall workarounds for WSJ do NOT work at all. This means I can’t read those articles, and have to use proxies to do so. This is not acceptable.

EDIT: One solution would be to use a link to a webcache, or screenshot, waybackmachine, or similar


When you say "This is not acceptable", what do you mean? If a link is accessible from Germany without a proxy but not the US, would that be acceptable? What if it's accessible to you in Germany and me in the US without a proxy, but not to someone in China unless they use a proxy?

So maybe it's OK to require a proxy from China? But what if a user in North Korea can't access a proxy? Should that link also be prohibited on HN? How is HN supposed to know what consists of an acceptable workaround, and what does not? Are the standards different for different countries?

It seems simpler to assume that the articles that get upvotes have people who want to read them, and that if people want to read the article everyone is raving about, they will find a way. It's Hacker News after all.


If a link is not available to everyone, link to archive.org, or archive.is, or Google webcache.

This means everyone can read it.


People do that in the threads, which is helpful. It's usually better not to supplant the canonical URL of a story in its link.

This problem will get a lot easier when we have a way to group the related URLs for a story. That's something we are eventually going to work on. Indeed, I wonder if it couldn't turn into a broader solution to the paywall question.


The Google scholar approach might work as a format. Official link as the hypertext, publicly accessible link (if available) in the right column. https://scholar.google.com/scholar?cites=1273530321270058317...


Same here in Spain, I think it's geoIP based.


A: This is something pretty terrible and should be in violation of search engine rules. So perhaps file a complaint.

B: Even Google gets on the geo-IP bus and will outright 404 pages in some zones with zero indication that the URL was ever OK. Recent example was the Google Solar Roof thing - in some countries, it was just a 404, nothing else.


Maybe there should be some flag about GeoIP'd posts?

Or just outright banning them?


Same here in Brazil.


Long ago in the mists of time, some Usenet groups had a wonderful policy of self-moderation. If you wanted to post something, you had to figure out how to add an "Approved" header to your own post. A hassle the first couple times you had to do it, but great for overall signal-to-noise after that: http://meatballwiki.org/wiki/AltHackers

I'm occasionally tempted to think that HN should in the same direction: no links whatsoever, everything is plain text. You want to read the article, you cut-and-paste. Or write your own browser extension, or whistle it into a cell-phone or something. Terrible for rapid reading, but would definitely cut down on the complaining about paywall tags. One-click links probably violate some Amazon patent anyway.

Taking it a step farther, all submissions must be done rot13. If you can't figure out how to translate a link to rot13 (or install an appropriate browser extension), maybe you shouldn't be posting here. Not because you are inherently unworthy, but because you haven't bothered to read and follow the instructions. The instructions could be given on the bottom of the guidelines page, and all improperly formatted submissions could redirect to the guidelines: https://news.ycombinator.com/newsguidelines.html

I feel like it's either that, or more all-Erlang days:

  You can help the spike subside by making HN look extra 
  boring. For the next couple days it would be better to have 
  posts about the innards of Erlang than women who create 
  sites to get hired by Twitter.
pg, https://news.ycombinator.com/item?id=512145

And while you are at it, get off my lawn. :)


Gubhtu V ybir lbhe cubar juvfgyvat cebcbfny, n orggre vqrn zvtug or gb unir n phgbhg bs na naguebcbzbecuvp pnegbba punenpgre jvgu uvf unaq uryq bhg ng n cnegvphyne urvtug juvyr ubyqvat n fvta fnlvat, "Lbh zhfg or guvf vagryyrpghnyyl vagrerfgrq gb rawbl guvf fvgr."


If the workarounds are so commonly accepted, can you start automagically editing the links to be the google/url redirect bypass then?


Or add a [bypass paywall] link, like there used to be a [scribd] link? ;).



Even just a [paywall] like you do [pdf] to warn people not to bother clicking it would help greatly.


[pdf] is added by software.


Yup. I thought about exactly the same reasons you described there; that's why I put a ";)".

But while we're at it, how about a [cache] link? More often than not we end up overloading the submitted site and there's always a comment with Google Cache link.


I would very much like to see a small tag indicating paywalled content. It is easy to tell that NYT, WSJ etc are paywalled, and some even allow free viewing, however scientific papers and academia seem to have a higher proportion of such sites. Since this community is pretty STEM centic, a lot of papers, journals, and smaller subscription sites are posted here. If users could simply "tag as paywalled" it would be a timesaver and a rather nice feature. Thanks.


HN generally eschews tags, and I fear having to maintain this (edit: I mean manually). But we'll think about it.


Thanks, really appreciate you giving it consideration would be awesome. Cheers. Also, couldn't it just be as simple as adding a "tag as paywalled" button next to the flag option. Then, if 3 people, (or a reputation heuristic meets a threshold) a small wall icon appears after the article. It doesn't have to crank up gravity or affect the post.

Thank you for your contributions to this site.


It could be an optional tag like [dead] or [flagged].


But those are automated. I've added "manually" above to clarify why I fear this.


Would it be practical to introduce a tag system which lets each user manage their own list?


You can do this yourself with a userscript


This would be my preferred option.

I've got a specific use case that's not applicable to most HN readers - here in .au Popular Science links are useless - they do a geo redirect based on your ip address, which redirects me to the homepage of the .com.au version of their site, which in general doesn't even have the original article available...


I'm not sure how configurable userscripts are, but I'd love to add this.

I don't really want to write RES for HackerNews, but it's an interesting project for the last week of my holidays.


Working on this now, no ETA though.


Done. It's terrible. In its own repo so I can have pull requests that add extra sites.

Hit the "Raw" button on https://github.com/voltagex/hackernews-paywalltag/blob/maste... - linking directly is not a good user experience for anyone using GreaseMonkey


I wrote a user script so you don't have to.


I like this idea. Or maybe a checkbox at submission - "this is paywalled" - and have HN offer a link to the Google result for the title of the page instead of a direct link if the box is checked.


This would be perfect. Anyone else remember those "[scribd]" links that used to be next to PDF submissions? There could be a "[google]" link to the Google redirect page (google.com/url?…).


Submitters don't always know if something is paywalled


Done: https://github.com/voltagex/hackernews-paywalltag/blob/maste... - click "Raw" and install it with GreaseMonkey


Yeah, I'd like to see this as well.


It would be also nice if HK incorporated a filter, so people can choose if they want to see posts to paywalled articles or not.


Just click on the link and you'll find out if it is paywalled or not.

Why should someone else spend all the effort to code up a solution to your problem when you believe it is beneath you to click on a link and find out for yourself?


I, and many others it seems, would appreciate it. I simply asked if s/he would consider it as a feature request because although a small gripe, it is quite annoying. It wouldn't be as bothersome if it were not for mobile. Waiting 5+ seconds for several JS libraries to get served to the client, then after about a paragraph of reading a paralyzing modal covers the entire screen asking to subscribe to read the full article. Not so much well known "reputable" sites like wsj, nyt, exonomist, but others. There is limited popup blocking and thus sometimes the page needs to be manually shut off, so one must then find the link and try to find a non-paywalled version in google, possibly repeating that process instead of knowingly bypassing the article for the comments and following the top comment of a helpful community member.

It was a polite feature request, asked civilly.


Complaining about paywalls on HN is strong irony. Paywalls are erected around websites intended to make money.

A significant portion of the HN community are specifically building websites intended to make money. Perhaps the majority in the past, before the Elves left the forest.

What's special about news sites, that compels people to complain about them popping up on HN? If it's really a bad thing, then shouldn't we be complaining about non-news sites that make money (or are trying to?).

Isn't every YC company trying to make money, and charging for what their website offers?

Sheesh.


I have no problem with paywalls. In my opinion, that's how sites ought to fund themselves. Ads and tracking suck. So yes, I subscribe to sites that I read frequently. But I'm not going to subscribe to everything that strikes my fancy on HN.

So anyway, what I complain about are sites that load the paywall at first visit. If the site allows five free articles per month, why not just wait until the sixth?


I'd like to think the comments in this thread don't represent HN.

Should we also stop what's you favourite book threads since they are mostly paywalled or ask if we are allowed to link to the torrents?


Paywalls don't make as much sense on the web because of hyperlinking. I don't read articles from just one source (like a newspaper) I'm reading tiny bits from hundreds of different sources in a day.

I'm not sure the best way to monetize that but a paywall on a single domain sort of misses how 99% of the audience actually uses the site.


To extend the irony further, it's products like the Economist that are consistently of a better quality (than non paywalled publications), it seems. I'm no libertarian, but I have yet to find a news publication better than the Economist.


It's not really ironic though. This is a site for discussing articles. Having to pay to play really ruins the experience and excludes a lot of valuable discussion.

Nobody cares about sites trying to make money. What people complain about is when it means an HN reader has to directly pay to participate.


> Having to pay to play really ruins the experience

Indeed it would, which is why it's not happening. That's already clear from the title: "workarounds" means the only price is a bit of annoyance.


Under what circumstances will this decision be reconsidered? It would be very bad to adopt a permanent policy that prevented any discussion of itself, since there would be no way to correct any such policy if it turned out to be a bad one.


If it has bad consequences we'll reconsider it, like any decision.


This might be the right thread to ask this: the usual workaround for WSJ of googling for the article never works for me. Neither do the google-redirecting links people post. Anybody else have this problem?


You should install refcontrol, it is a firefox extension. It allow you to create rules for your Referer header. Make a rule so that wsj.com receives the referer www.google.com.


I should add that this invariably happens when I don't have the time to debug things. Then nothing works: clearing cookies, going incognito, switching browsers. But of course when I go looking for examples right now everything seems ok.

I'll keep refcontrol in mind next time it happens, thanks. I'm not sure why setting the header this way would work when clicking directly on the link or the google search page doesn't, but heck I have no model for this behavior anymore.


Perhaps add [paywalled] to the post title, link it to the FAQ entry and expand latter with a list of common workarounds?