Mar 8 2016

TruthRank vs. PageRank

I was at dinner a few weeks ago with my long time friend and first business partner Dave Jilk. We ended up talking about how difficult it is to determine signal from noise, fact from fiction, truth from bullshit, and bullshit from complete-and-total-bullshit.

I recently hit the wall with all the political stuff that was popping up everywhere. I think the thing that flipped my switch from on to off was a satirical article about Hillary Clinton and all the horrible things she had done that was being passed around by people who I think considered it to be factual. As I read through it, I imagined all the derivative articles building on the sarcasm embedded in the article and then making arguments which would be cited by others as truth because they showed up credibly somewhere.

I probably would have recovered from this in a few days if I wasn’t then confronted yesterday by a Wall Street Journal article that was sent to me with a clear set of assertions built around a statement that I knew to be factually incorrect, but I’ve seen written exactly the same way in other articles to make a specious argument.

Software should be able to solve this for us. It appears that whenever Google talks about working on ranking based on trustworthiness anti-science advocates freak out about it. If you are interested in seeing the math (and some concepts) behind this, the paper by some Google folks titled Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources – while chewy – is very interesting (at least the parts I understood.)

Dave sent me a presentation he’d done on this topic for a Defrag Conference several years ago. I tossed it up on SlideShare and embedded it below.

We went back and forth on it a little more and Dave ended with a strong statement around skepticism.

“It seems like a consequence of a few drivers has caused there to be more awareness of the notion and techniques of skepticism. However, people are using it indiscriminately, i.e., just to attack the other side. It’s another form of bullshit, actually – they don’t care whether the skeptical criticism is valid, but it has some additional polemical value because it has an aura of aiming for truth. Some of the drivers of the new Skepticism are all the problems with media, climate change, and probably some other things I’m not thinking of.”

When I ponder the notion of peak oil, I pine away for the concept of peak bullshit. But, like peak oil, I suspect it is an elusive construct.