I refer to this answer from Too much version control and bug tracking overhead per change? for which the question itself was actually the subject of a previous meta question. Quoted here in the unlikely event of changes:

Heavy processes are common, unfortunately. Some people - especially management - religiously imagine that processes produce products. So they overdo the processes and forget that it's really a handful of hard-working, smart people who actually create the products. For upper management, it's frightening to even think that their business is in the hands of few geeks, and so the close their eyes from the reality and think of their dear "process" instead, which gives them the illusion of control.

That's why agile startups with a handful of good engineers can beat big, established corporations, whose workers spend 95 % of their energy on process and reporting.

This answer is making grand assertions left and right without a single shred of evidence to back any of them up. Paraphrased from my comments:

  • Which people are "some people"?
  • What evidence is there for managers believing it more often than anyone else? Why would this be the case?
  • Why liken it to a "religious belief"? Is there no factual/logical evidence in favour of the belief? How can we be sure? Have those people been questioned about their belief?
  • Who claimed/claims that "processes produce products" and what was the context of that claim? If the answer, as I suspect, is "nobody", then why mention this at all?
  • What does "overdo the process" mean? At what point does a process become "overdone"?
  • Is it really true that the "business is in the hands of a few geeks?" Is this true in the 85% of programming jobs that are not in the software publishing industry, which is presumably where most of these heavy processes are being instituted?
  • What does the phrase "close their eyes from the reality" mean? Do these people presumably have no accountability to their own managers, shareholders, board of directors, etc.? How would they justify this?
  • What does "illusion of control" mean? How is it different from actual control? What provides the illusion and how is it supposed to keep people fooled?
  • Although the answer technically only says that "agile startups" can beat "big established corporations", it's misleadingly worded as to imply that this is routine. To what degree does this actually happen and is there evidence that it's any more than chance, as opposed to better management?
  • Where was the 95% figure derived from? Was this measured somewhere?

As much as I hate to pick on one person/post, this isn't information, it's entertainment. It's mindless, populist drivel clearly written with one purpose: to provoke an emotional response and farm upvotes.

And in this community, it works. Every single time. With 84 upvotes and just 1 downvote (mine, obviously), this is clearly not only the best answer, but many times better than any of the other answers, which the author stubbornly uses as a justification for why it doesn't need to be improved (can't blame him, I guess, I wouldn't look a gift horse in the mouth either).

But the most irritating part is the most recent comment:

This forum is not a scientific paper. Nobody, not you nor me, is going to provide an explanation for every single sentence he writes.

Call me old fashioned, but I thought that the whole point of the Subjective Question Guidelines was to elicit answers that would Back It Up:

Back It Up! means that your answers must be based on either:

  • Something that happened to you personally
  • Something you can back up with a reference

I realize that we're not Wikipedia. We're not even Skeptics.SE. We can afford to be a little more lax.

But who is correct, him or me? Do we/should we have no standards whatsoever for evidence, or are answers here supposed to be more than just statements of opinion?

share
    
wow - and i thought i was pedantic! ;-) –  Steven A. Lowe Jul 23 '11 at 22:20
1  
Regarding your quoted passage: his grandiose claims only seem grandiose to those who have never worked in such a company. That said, the 95% claim does seem a bit hyperbolic. –  Robert Harvey Jul 25 '11 at 4:25
2  
@Robert: Sure, that's the representativeness heuristic at work. I'll buy that some companies are like that (well, hell, it's already implicit in the question), but it all goes downhill from there, attributing it without qualification to inept management and painting a picture of this supposedly common/popular archetype. We don't know how common or accurate that picture really is. I'd rather hear about his own experiences with these management types. That would have been a great answer. –  Aaronaught Jul 25 '11 at 13:30
add comment

8 Answers

It depends on what you think Programmers.SE is for.

If it's a site for expert programmers who are interested in conceptual questions on software development, then answers had better be backed up by references. Even for the kind of subjective questions that make up the meat of the site, some kind of justification should be provided — real experts don't just say “yes” or “no”, they can explain why.

If Programmers.SE is Stack Overflow's toilet bowl, then answers aren't for reading, you don't know where they've been (or rather you do!), so who cares.

share
add comment

In all areas of life extraordinary claims should require not just evidence but extraordinary evidence.

If this doesn't happen, this shouldn't actually be a problem that needs a specific solution. You'd hope that an answer that makes such claims without backing them up is a poor answer and should be treated as such by those voting and commenting, at least that's the theory.

But the nature of PSE and the audience it has means it doesn't always work that way. There are some answers which get away with some fairly outrageous claims because the answer presents a broadly popular perspective. Suggest that someone quit a shitty job and you can get away with almost anything else in the same post. Similarly blame management for getting in your way and the detail will likely be ignored.

While that's annoying it's not actually that much of a problem. For the most part I'd look to read not just the top answer, but the top three or so answers. Generally where you get one answer playing to the crowd, somewhere close to it you get a contrary view point.

And it's important not to discard the populist viewpoint. It may not be what you want to do but you should use it to inform your understanding of the world and the culture you work in. One place I work lived by the phrase "the perception is reality" - essentially saying that even if something wasn't true, if enough people believed it, it was something you had to deal with and I think that's the case with these answers.

Read all the answers, use your own judgement to work out where you think the truth lies but try and learn from all the views that have been put forward as they will all tell you something, even if it's not exactly what the poster was trying to say.

(I speak as someone who is most proud of some of my answers that have got next to no up votes...)

EDIT (from a comments discussion with Aaronaught):

So just to be clear from the outset - I wish all posts drew on research and personal experience and provided links to reputable sources where the claims warranted them. To me that's a good post and what we should aspire to.

But what we have is a situation where some popular answers don't meet that standard, and a question about what that means and whether it's a big deal.

To me the nature of subjective questions means that the most useful "answer" isn't a single answer (whether the top voted or not) but is the amalgamation of all the great information contained in all the answers that have been provided. Subjective questions by their nature don't have a single answer so for me what the upvotes are saying is the community indicating "here is something you should have a think about while you're working out how to handle the specific situation you find yourself in".

In that context a glib unsupported popular answer with a mass of up votes is useful information. Sure it would be more useful if it was better supported but if you get 40+ people saying "I agree with this" it is absolutely something you should consider even if it's categorically wrong.

The reason for that is that right and wrong are not the only thing that matters. If you're in a shitty job and you post about it you will get an answer with a bunch of votes saying "Quit now!" sure as the sun will rise tomorrow morning. While on one level that's a bad answer it gives you useful information - that you work in an industry where a significant number of people believe that that is an acceptable approach to the problem. While I and many others would argue it's not the most constructive approach that doesn't change the reality that many people believe this and act on this and it is something that you will see and have to deal with in the industry.

I think what I'm saying (because I'm kind of thinking aloud here) is a significant number of up votes are evidence in themselves. Not perhaps of factual correctness, but of a perception that is frequently so strong and so prevalent that it has to be understood and considered. As programmers we sometimes forget that factual correctness isn't everything and that situations are more complex than that, that the softer and more emotive side of things is equally important and that any answer which ignores that may be factually correct but isn't the whole story.

Obviously this requires the reader to apply some thought to the answers being provided and understand a wider context but I hope and believe that the people we're talking to are capable of doing that. If they're not then I suspect there's not much we can do for them anyway.

So I'm not saying we should encourage bad answers, just that very popular bad answers provide useful information we can learn from and that we shouldn't be so quick to dismiss.

share
3  
Good points, and worth an upvote in itself - however, my problem with this is that we as a community are allowing (and in fact helping) these popular-but-wrong or popular-but-unsupported ideas to become self-reinforcing by promoting them to the top of the heap. The more people read these convenient ideas, the more popular they become - and the more they get voted up, the more people will read them. –  Aaronaught Jul 19 '11 at 14:29
3  
@Aaronaught - I'm wary about saying that a lot of these things are out and out wrong. The nature of the site is that we're talking about subjective things and even where research exists it's usually patchy or can be reasonably argued to be only indirectly relevant. Personally I'm happy just making sure that contrary well argued answers exist and are available to people. The first step in changing people's views (if indeed they're wrong and we're right) is in making it clear that there is another view. –  Jon Hopkins Jul 19 '11 at 15:26
4  
That's why I added the qualifier "...or popular-but-unsupported". They may very well be right, but that's not the point; the purpose of this site is to educate, not discuss. Pointing out that some particular popular view exists may in and of itself be educational, but again, this isn't intended to be a site for opinion polling, where votes are used as a measurement of the popularity of some particular viewpoint or product. Votes are supposed to indicate the quality or usefulness of an answer, and without evidence, both are pretty low. –  Aaronaught Jul 19 '11 at 15:48
2  
@Aaronaught - As you say though, seeing that point and the lack of support usually provided with it is educational in itself. It may not be high quality but it can still be useful, on one hand those open to that idea can take from it that the expressed viewpoint is widely held (and therefore likely "socially" acceptable), on the other those who are more sceptical might see the lack of support and judge it accordingly. I'm not saying it's ideal, just that perhaps it's as big an issue as it might look. –  Jon Hopkins Jul 19 '11 at 16:05
4  
You're talking about what we can learn in a personal sense, and that's great, but on Stack Exchange you are writing to (and voting on behalf of) the whole entire internet, and the nuances are going to be lost on those people. We should be trying to educate them better, not slinging easy answers at them. –  Aaronaught Jul 19 '11 at 16:23
2  
Aaronaught - I'm talking about what anyone reading the answers and applying the smallest amount of thought can learn - we are no different to anyone else reading it. As I say, I'm not saying it's ideal, I'd rather every answer was comprehensive and well sourced, but I think that as a side effect of having mass involvement, it's not that bad. –  Jon Hopkins Jul 19 '11 at 16:39
3  
I will add to your edit that what is right or what is wrong is also highly subjective. The proof is all the discussions here on meta. And yes, upvotes are a pretty good indication that an opinion or experience is worth reading. –  user2567 Jul 20 '11 at 9:27
1  
@Aaronaught: I find your statement of "those people in the entire internet whom we should be trying to educate" quite outrageous. Who are "we" (whoever that means) to decide what's good for "those people in the entire internet" and what's not? I would rather believe in presenting different views of things - no matter how contradictory - because that's the way things usually are: simple right/wrong rarely exists, especially in subjective matters. –  Joonas Pulakka Jul 20 '11 at 10:26
3  
@Jon Hopkins: most of us agrees with the fact answers are better backed with references. The real problem is believing we can do anything about it. We can't. And the best proof is the huge number of meta posts talking about the problem, the actions taken and the poor results. –  user2567 Jul 20 '11 at 11:16
1  
@Pierre we also have a bigger community to share the responsibility. My point mostly is that saying "we can't change it" is an unfortunate attitide when we certainly can do something about it. –  Anna Lear Jul 20 '11 at 15:45
1  
@Pierre: You keep referring to "references" but I've stated several times that nobody (including myself) is expecting to see the kinds of citations that would be expected on Wikipedia or Skeptics.SE. Personal experiences are fine too. An opinion without any explanation or justification is, as Anna says, just an empty platitude, and not worth the electricity and bandwidth used to post or display it. To say that one can't do anything about it is just silly; if you have an opinion, you should know where it came from. If you don't even know where it came from, you should refrain from posting. –  Aaronaught Jul 20 '11 at 20:52
3  
@Pierre: If by "the world" you mean "people who answer questions on P.SE" then I ask: Why not? That's like saying we can't ask people to write on-topic or comprehensible questions. We can and do; it's just a question of making sure we have the right tools and policies in place. Wikipedia has far more stringent standards and yet they manage to pull it off. You have to cultivate the right sort of culture, and if not now, then when? –  Aaronaught Jul 21 '11 at 14:23
2  
@Pierre - That's like saying you'll never get rid of all criminals to there's no point in fighting crime. I don't expect everyone to provide perfect answers but that doesn't mean that there's no point in encouraging people to do so through votes, comments and editing things to improve them. I'm not trying to create a perfect world, just one that's slightly better than an acceptance of mediocrity. –  Jon Hopkins Jul 21 '11 at 14:56
2  
@Pierre - I'm not clear how continually posting variations on the same suggestion (basically more freedom) on meta time and time again despite it being declined every time is actually helping. The powers that be have made their feelings known and explained why they don't want to go down this route. How does repetition help? –  Jon Hopkins Jul 21 '11 at 22:17
2  
@Pierre: Why do you say that everything so far has failed? Mostly, in the past, we've focused heavily on questions, and the quality of questions has gone up. It's far from perfect, but I'd say that it's up to an acceptable level, for now. It's really only during the past month or two that answer quality has come under the gun. I'm sure that we can improve those too, if enough of us make a concerted effort. –  Aaronaught Jul 22 '11 at 12:43
show 24 more comments

The short answer is "yes", but that's only (as you say) for significant claims.

Anybody can make a claim in an question or answer, but without any evidence to back it up, but a good question or answer will have that claim backed up with links, personal details etc. That's not to say that a question or answer without back up is a bad question.

You're right to say we don't need (nor should we insist on) the same level of detail as other sites, but if you are quoting figures then this must be backed up.

share
    
I agree with figures. We can't tell 95% if it's not 95% (according to a study). That's why it's preferable to use most or few or may or IMHO words here. –  user2567 Jul 19 '11 at 12:47
1  
I think citing numerical figures is kind of a low bar, don't you? A huge generalization of how managers think and how businesses produce value is a pretty significant claim, isn't it? –  Aaronaught Jul 19 '11 at 13:19
1  
@Pierre - I agree but 95% has become a figure of speech meaning "a significant majority". 99% and 98% are used in the same way. It's not meant mathematically and I think there is some onus on the reader to consider whether it's meant in that context. –  Jon Hopkins Jul 19 '11 at 16:10
1  
@Jon Hopkins: +1 for pointing language related expression that may be misinterpreted (as I probably did). –  user2567 Jul 19 '11 at 16:11
2  
@Jon: "A significant majority" is still a significant claim. Figure or no figure, it demands evidence. –  Aaronaught Jul 19 '11 at 16:24
1  
@Aaronaught: How about the phrase "Developers are notoriously anti-management"? How would you back that up? It's a significant claim, isn't it? Or is it so obvious that it doesn't need to be backed up? I think the significance of a claims depends a lot on who you ask. Is it reasonably possible to back up everything that someone out there may consider significant? If not, then who decides what's significant and what's not? –  Joonas Pulakka Jul 20 '11 at 7:48
    
@Aaronaught: I extracted the above comment into a separate answer. –  Joonas Pulakka Jul 20 '11 at 8:08
add comment

I've read through all of this discussion, and the one thing that keeps popping out at me is the disconnect between both sides on who has the burden of proof.

Self-evidence

The crux of the argument in support of these types of answers, as I understand it, is that they aren't providing anything that necessitates detailed evidence, and such a requirement would be onerous on otherwise self-evident (or commonly-held) ideas.

Something that's self-evident is, by definition, something any reasonable person would know. By providing answers that are self-evident, you don't provide any expert value.

Take the classic example of someone coming to the IT guy for help with a computer issue they're having, and the IT guy responding, "Did you plug it in?" It's self-evident that a computer must be plugged in for it to work, but the exchange speaks to anyone who's had to deal with really trivial computer questions; it's functionally equivalent to saying "You're wasting my time by asking me this question." It's been lampooned in many a comedy sketch because it's exactly the type of response nobody wants to receive when they're coming to someone for help.

Massively up-voted answers like the ones discussed here are our moral equivalent of "did you plug it in?" Someone asks a seemingly trivial question, someone else leaves the obvious response, and it gets massively up-voted not because the answer actually provided value or great insights, but because it said what we're all thinking.

That is decidedly not what Stack Exchange is about. If you don't think a question is worth answering with a thoughtful, well-reasoned response, there are several options available:

  • Leave a comment explaining why the question is trivial and basic things the person can do
  • Down-vote the question as showing no research or effort
  • Vote to close the question as not constructive, too localized, or not a real question depending on the circumstances

But beyond that, the main premise behind Stack Exchange is that there are experts who are willing to help others who might not know as much as they do. It's important, crucial even, that self-evidence is just taken off the table entirely. The answer is not self-evident: if it was, it wouldn't have been asked.

Preferences vs. belief

Beyond the question of self-evidence is the issue of what constitutes subjectivity, and subjective answers can be roughly categorized into two types:

  • Personal preferences
  • Warranted beliefs

A personal preference is something you personally like or favor for whatever reason: I like mint chocolate chip ice cream. I enjoy sleeping in late. I want a slice of pizza for lunch today. These are unassailable: provided I'm not lying to you, you can't prove I don't like mint chocolate chip ice cream, no matter how hard you try. I said it, it's true.

A personal preference is the appropriate answer to a question asking for a poll: "What is your favorite ice cream flavor?", "What's your favorite book?", "Which programming language do you like the most?" Those questions, however, are straight up prohibited here and across the network. Personal preferences have no place in an answer, and are the smell of a bad question.

A warranted belief, on the other hand, is something you personally believe based on something you personally know to be true. That is, "I know X, and I know it because of Y."

This, and only this, is what subjectivity is all about on the Stack Exchange network, and it's the heart of the "back it up" mantra. We want—no, crave—answers that provide a belief of yours that you hold because of some special, personal insight.

But it's not enough to just provide the belief: how do I know what you're saying is true or credible (remember, self-evidence is off the table)? Tell me more about why you believe that and I'll decide for myself whether what you believe is truly warranted from what you know, or if there's some other insight to be gleaned.

Summary

In short, the person who has the burden of proof is always, always the answerer. If you can't support or are unwilling to support your claim with information about why you believe the claim to be true, it's not worth leaving the answer.

It wastes everyone's time: the question asker who doesn't get a useful answer, visitors who have to read through the answers, and yours when you have to justify it to people who question the claims made in your answer. To respect people's time, and make the internet a better place for everyone, tell us something we don't know and explain what it's all about.

Answers are like show-and-tell: we're all dying to hear the story behind why you believe PHP is the devil's work, or how you came up with the idea to use recursion to save little Timmy from the well. Don't hold out on us.

share
add comment

Let's go back to the original question and my answer that started this discussion. The question was:

Is there a point at which the process gets in the way and becomes an end unto itself?

To which I answered (emphasis on what I meant to say):

Heavy processes are common, unfortunately. Some people - especially management - religiously imagine that processes produce products. So they overdo the processes and forget that it's really a handful of hard-working, smart people who actually create the products. For upper management, it's frightening to even think that their business is in the hands of few geeks, and so the close their eyes from the reality and think of their dear "process" instead, which gives them the illusion of control.

In other words: Since it's really a handful of hard-working, smart people who actually create the products, obviously the process can get in way if it's too heavy. While I (and quite a few colleagues) like to emphasize the people, some (not-so-good) managers tend to think too much in terms of the process. That's the reason for overly heavy processes. The reason that (some not-so-good) managers prefer to overemphasize the process is that it is (or at least seems to be) controllable. Real people are way more difficult to control than some abstract process. Regardless, it's the real people that produce products, not the process.

All the above is said in my original answer, even if somewhat colorfully worded. I don't think there's any particularly significant or outrageous statement. It's mostly common sense written out.

Some folks seem to read it like this:

Heavy processes are common, unfortunately. Some people - especially management - religiously imagine that processes produce products. So they overdo the processes and forget that it's really a handful of hard-working, smart people who actually create the products. For upper management, it's frightening to even think that their business is in the hands of few geeks, and so the close their eyes from the reality and think of their dear "process" instead, which gives them the illusion of control.

If you take only the "anti-managerial tone" from the post, then of course it looks like a stupid rant. Maybe, just maybe, a more neutral tone should be preferred. But on the other hand, I prefer reading strongly written, concentrated statements instead of shy ones. People shouldn't be ashamed of their own biases.

My answer is not very good; it's certainly below my average answer quality (but some answers have to be below the average!), and its lack of quality is emphasized by the number of upvotes it got, which may be seem to be undeserved (but it's not certainly my fault). Still, calling it "mindless, populist drivel" is clearly a result of reading only the colorful part of it and missing the main point.

share
2  
People shouldn't be ashamed of their own biases but you basically admit that by writing in this way you've failed to communicate what you actually were trying to say. If you think that the entertainment contained in strong wording is more important than accurately making your point then you are being populist. But it seems that what you're actually doing is admitting that it's not a brilliantly written answer which begs the question why are you so vociferously defending it? –  Jon Hopkins Jul 21 '11 at 8:58
1  
@Jon Hopkins: I'm not defending that specific answer so much as I'm defending my (and others') right to write less-than-brilliant, and maybe even populist, answers. Probably nobody considers the existence of less-than-brilliant answers a serious problem (all answers can't be brilliant, right?), but the fact that sometimes they get lots of undeserved upvotes seems to be irritating - maybe rightly so. But the answer's writer can hardly be blamed for that? –  Joonas Pulakka Jul 21 '11 at 9:10
2  
@Jon Hopkins: About the failed communication. Communication always involves (at least) two parties. I obviously have failed to communicate my key point to some people who get easily irritated by entertaining text and see only that part of it. But I also believe that lots of readers can see the actual substance in it - therefore lots of upvotes. Yet another possibility is that most of the upvotes come from the entertaining part, but I do believe this community is mature enough so that it can't be so. –  Joonas Pulakka Jul 21 '11 at 9:16
2  
I do understand that but as you now admit yourself that it's not the best worded answer your strong defence of it seems odd. As it goes I don't think Aaronaught is really against someone like yourself who just happens to have got a bunch of votes for one of a weaker answers but generally posts at a higher level. I think this is more about those (whether voting or posting) who simply don't get what a good answer is. –  Jon Hopkins Jul 21 '11 at 9:45
1  
You've provided three different answers defending your answer, attempting to guess at why your answer is the target of discussion, but there have been a lot of comments and answers—not to mention a thorough explanation provided in Aaronaught's question—saying explicitly any significant claim needs evidence. It's not about the tone, or what words you claim people chose to focus on: you made a claim so provide some evidence of that claim. To that end, the revisions you've made to the answer in question have improved it. –  user8 Jul 21 '11 at 12:10
2  
@Mark Trapp: Thanks for encouragement, but I fail to see any improvement in the above two revisions in terms of evidence. The only difference is in tone. I'm surprised how much the tone affects the perception of evidence. It's probably similar to how wearing a suit improves your credibility, even though the suit itself of course has no actual effect on anything. –  Joonas Pulakka Jul 21 '11 at 12:16
1  
You added a whole section that provided some evidence of your original claims. That's not a change in tone, that's actually providing some backing to your claim and went a long way in improving it. That's what we expect from answers. To make this into an argument about your tone in the question detracts from the central issue and has caused this discussion to almost completely derail. It's not about tone. It's about backing statements up with some semblance of evidence. –  user8 Jul 21 '11 at 12:49
1  
@Mark Trapp: Sorry, I thought you meant the "In other words" part in this post. My bad. Still, does the added section about how some successful companies were once small startups add any evidence to the original answer? I'm confused about what counts as evidence and what doesn't. –  Joonas Pulakka Jul 21 '11 at 13:09
2  
Guidelines 1, 2, 4, and 5 in Good Subjective, Bad Subjective describe what we're looking for when we talk about evidence (although I strongly encourage you to read the whole thing). Saying something is X is not what we're looking for; saying something is X and explaining why you believe something is X is. To quote from "Good Subjective, Bad Subjective", "We like you. We want to believe you. But like wikipedia itself, {{citation needed}}." –  user8 Jul 21 '11 at 13:16
2  
"Common sense" - is invariably what people say when they're unable or unwilling to validate a claim. Common sense often, over a long enough period of time, turns out to be wrong; before Brooks, most managers would think nothing of staffing up in order to meet a short-term deadline. Now I won't nitpick the original answer itself further - as Mark says, it's certainly improved - but I'd really like for you to understand that we're not asking you to go out and do research, just explain why it is that you believe in some particular wisdom. –  Aaronaught Jul 21 '11 at 13:31
    
@Aarnoaught & Mark Trapp & everybody else: Thank you for your patience and sorry for wasting your time. I think our mutual understanding is now quite reasonable, if not perfect. I'll try to improve my original answer right when I have time for it. –  Joonas Pulakka Jul 22 '11 at 9:32
add comment

It's easy to agree with "yes, significant claims should be backed up", but it's a vain statement as long as there's no common agreement of what "significant" means.

The problem is: what is a significant claim? What's significant to you may not be significant to someone else, and vice versa. Is it reasonably possible to back up everything that someone out there may consider significant? If not, then who decides what's significant and what's not?

Are the following claims significant or not? Or is some of them significant, but so obvious that it doesn't need to be backed up because of that?

  • Developers are notoriously anti-management.
  • Managers love processes.
  • Projects tend to go overtime.
  • Startups can beat big, established corporations.
  • Big, established corporations can beat startups.
  • Stackexchange sites are popular.
  • Beginners tend to make mistakes.
  • Big corporations tend to have heavy processes.
  • Lighter processes have become more and more popular during the last 10 years.
  • Formal processes become a requirement when company grows to > 10 people.
  • Climate change is a major issue.
  • Java is one of the most used languages out there.
  • Smoking kills.
  • I feel that a couple of cups coffee does good to my programming efficiency.
  • Too much coffee is bad.
  • Lines Of Code Per Day (LOCPD) is a lousy measure of developer productivity.

Do all of these need to be backed up? If not, which ones, and why? If yes, how would you back up e.g. the first one ("Developers are notoriously anti-management.")?

Personally I think that since the definition of significance and obviousness depends totally on who you ask, these sites should be kept quite lax - much more lax than some extremists suggest. Backing one's claims should of course be encouraged, but it shouldn't be required. Attempting to suppress noise may improve signal-noise ratio, but it will also more or less distort the signal itself. The fallacy is that one starts to consider his own (or his community's) judgement as the only valid measure of significance or obviousness, and dismisses anything that deviates from it. How is that better than opinions without evidence? Questions with tens of upvotes getting closed is a sad example of this. Lots of people would be interested in the question (that's why they upvoted it), lots of interesting answers have already been written, and then... some police closes the question as "Not a real question" or something. It's really sad.

share
2  
Ignoring the claims that are completely off-topic for Programmers, I think if you base an answer in large part on any of those claims, you should either back it up or rephrase the claim. for example, "devs are notoriously anti-management, therefore X" can become "developers who are anti-management would do X". –  Anna Lear Jul 20 '11 at 15:49
1  
Additionally, backing up a claim can be done with references or experience. So "developers are notoriously anti-management" can become "Last place I worked at devs were strongly anti-management and it lead to X". Practical references to things that happened work well, imho, for backing up claims and providing context for the answer. –  Anna Lear Jul 20 '11 at 15:51
2  
I would have to say that most if not all of these are what I would call significant claims, especially the way in which they're worded here (most of these are broad and in some cases misleading or fallacious generalizations of a narrower set of well-known facts). –  Aaronaught Jul 20 '11 at 20:55
1  
@Aaronaught: All right, that's how you think about significance, and there's nothing wrong in it. The problem is in requiring similar mindset of everybody. I think that e.g. "Lines Of Code Per Day is a lousy measure of developer productivity" is not a significant claim; it's a well-known fact, like that "today's computers are faster than those 50 years ago". The line between a well-known fact and a misleading generalization is in the eye of the beholder. –  Joonas Pulakka Jul 21 '11 at 6:34
    
I don't think this is difficult, it's just about a basic understanding of English. The more extreme the claim, the more evidence would ideally be provided, so "is" and "always" will almost certainly require more evidence than "can", "usually" or "sometimes". –  Jon Hopkins Jul 21 '11 at 8:49
1  
What have popular questions got to do with providing evidence in answers? Something can be a great question with poor answers or vice versa. Up votes on a question say nothing about the quality of the answers, just that people are interested in getting answers. (That of course says nothing about whether it's a good fit for PSE). –  Jon Hopkins Jul 21 '11 at 8:51
1  
@Jon Hopkins: Popular questions have nothing to do with providing evidence in answers. I just used closed popular questions as an example of how some know-it-all people consider their own judgement as a valid measure of (in this case) the question's relevance / realness and close it, ignoring the fact that lots of people have upvoted it. I do think that lots of upvotes on a question should prevent its closing. Similarly, some folks consider their own judgement as the only valid measure of a claim's (in)significance, requiring evidence for stuff that's obvious. –  Joonas Pulakka Jul 21 '11 at 9:02
1  
@Joonas - Up voting is a measure of popularity, closing is a measure of relevance. They're different things. Every SE site is very clear that it has a specific remit and there is a very clear reason for that. Closing questions is not about being a know it all, it's about applying a pretty clearly laid out set of guidelines in which popularity is irrelevant. If you disagree with those guidelines that's a different matter but Jeff and Joel have made it very clear that it's not up for discussion so I suspect you're wasting your time. –  Jon Hopkins Jul 21 '11 at 9:38
1  
@Jon Hopkins: Yes, site owners have right to impose whatever rules they want, no matter how arbitrary, period. I don't quite agree with their rules, but then, nothing is perfect, not even SE :-) –  Joonas Pulakka Jul 21 '11 at 11:03
1  
Joonas - "well-known" actually means "widely believed", which has nothing to do with the significance of a claim. A claim's significance depends largely on its implied population ("all/most/some") frequency ("always/usually/sometimes") and modality ("must/should/can"). The severity of the claim ("smoking kills") and open-ended/ambiguous phrases ("major issue" / "beginners" / "heavy processes" / "too much coffee") may also make a claim sound more significant, to the extent that clarification may be required as opposed to actual evidence. –  Aaronaught Jul 21 '11 at 13:47
3  
"All/Most X are Y" is a significant claim and warrants a citation, no matter what X and Y are. "Some/Many X can be Y" is less significant and could be backed up with some examples or anecdotes. "The Xs I've known/experienced have generally been Y" is not a significant claim, although it would still be enriched by an actual example. –  Aaronaught Jul 21 '11 at 13:50
add comment

Thanks for bringing this discussion here. I honestly think it's important (and picking one person/post is only good; it provides something tangible to discuss). Let me present my opinions:

As much as I hate to pick on one person/post, this isn't information, it's entertainment. It's mindless, populist drivel clearly written with one purpose: to provoke an emotional response and farm upvotes.

But these sites aren't strictly about information. If you want strict information, you can go to Wikipedia, IEEE magazines, scientific papers. The world is full of official and dull information. This site is, like the guideline tells, largely about the stuff between objective and subjective. About stuff that one thinks is correct, but can't prove. Such stuff would go to /dev/null without sites like this.

My answer, surely, leans towards the subjective side. As for being "mindless, populist drivel", that's your personal opinion, not information. I certainly did not craft it to farm upvotes. I wrote my opinion, that's all.

But who is correct, him or me? Do we/should we have no standards whatsoever for evidence, or are answers here supposed to be more than just statements of opinion?

I think this largely depends on the question. Take these questions:

  1. Is there a point at which the process gets in the way and becomes an end unto itself? Is that even engineering?
  2. How to create an edit profile page like Google+?

The second one is about technology how-to. The first one is about opinion. Opinions grow from experiences and they vary with time. Answers to such questions are unavoidably quite subjective. How much evidence did you seen in the other answers to the same question?

My answer obviously does provoke emotional responses, and even if it were completely incorrect, wrong, false, at least it reveals something about the question itself: the right answer, whatever it is, is not clear. Therefore, there is no objective answer. Therefore, the answer must be subjective. And subjective answers are based on personal experiences - something that nobody can call incorrect.

share
4  
"Opinions grow from experiences" - this is true, but we don't care about the opinions here, we care about the experiences. This isn't an issue of subjectivity, it's an issue of what's behind it. The difference between subjective and opinion is precisely the basis for the guidelines and this site in general. –  Aaronaught Jul 19 '11 at 13:14
7  
Maybe a better way of putting it is this: Subjective does not simply mean what is your opinion? according to the guidelines. It means what is your opinion and how did you arrive at it? Without that critical second element, our site ceases to become a learning tool and simply becomes a polling tool. We can learn from facts, and we can learn from personal experiences, but we can't learn from vague claims of dubious origin. –  Aaronaught Jul 19 '11 at 13:24
6  
Subjective does not mean "anything goes" -- blog.stackoverflow.com/2010/09/good-subjective-bad-subjective –  Jeff Atwood Jul 19 '11 at 21:34
    
@Aaronaught: "Show your work"? –  Shog9 Jul 22 '11 at 21:09
add comment

Questions should support their premises where necessary. We've seen a lot of posts along the lines of "Why do experts say not to use OOP" (a made-up example for illustration purposes). Questions like that really should back up their assertions that 'experts' really do say such a thing. A false premise makes the question meaningless. I routinely vote down questions like that.

Answers, on the other hand, really dont have that responsibility. If someone makes a strong case, vote'm up. If they dont, vote'm down. Not every answer needs to be a treatise on a particular subject. Often experience or opinion is exactly whats called for in an answer. If you dont like that 'the unwashed public' votes up answers you personally find insufficient, well, too bad. Thats freedom for ya.

share
1  
Downvotes are not only available but now free on questions, so I'm not sure why this burden-of-proof double standard is supposed to make sense. –  Aaronaught Jul 19 '11 at 21:21
1  
Its not a double standard. If a question's premise is false, then that effects all attempts at answers. But an insufficient answer does not affect any other answers that others may post. –  GrandmasterB Jul 19 '11 at 21:39
1  
Existing answers definitely affect future answers. This is why sometimes all that's needed to stop a post from being not constructive is an early well-crafted answer. –  Anna Lear Jul 19 '11 at 22:20
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .