Buttbuttination – or – The Scunthorpe Problem

2. May 2010

I’m pretty sure everybody has come across this. You send a mail, type something in chat, or try to post something on a forum, and your statement is either rejected or comes out garbled. What’s at work there are filter systems, so you might find yourself talking about the “Buttbuttination of Abraham Lincoln” or the “Consbreastution of the United States”. Clbuttic. Okay, I’m embarbutted by the products of my fellow programmers…

Conclusion #1: Obscenity filters don’t work. At all.

Read the rest of this entry »


They’re doing it again! Internet censorship

24. April 2010

I wrote a rant about internet censorship infrastructure last year, and I probably should give you a follow-up on the subject. Mrs von der Leyen (Germany’s family minister) was unsuccessfull in creating an internet censorship infrastructure. Thank God. Well, actually, thank Democracy. There was an election later in the year, which lead to a new coalition of political parties in charge. Funnily enough, Mrs. von der Leyen still has the same job, but her plans to “fight internet child pornograpy” (read: implement a massive censorship infrastructure that’s not going to help fighting against child pornography at all) were put on a hold.

So… “Yay for democracy”, right?


Somebody figured: “if you cannot do that on a national level, lets do it on a european (EU) level instead”. And instead of having one country with censorship, lets make it mandatory for every country in the EU. The whole thing seems to be a great f**king hydra. Makes me wonder what would happen if somebody chopped off Mrs. von der Leyen’s head…

Anyways. This is where Cecilia Malmström, a swedish EU minister comes into the picture. She created the website www.cleanternet.org, and puts this funny video online:

Thanks Mrs. Malmström. It’s people like you that stop me from losing all hope in democracy.

I concluded my last post about the subject with “If you are searching for a country with intelligent politicians, look elsewhere. Ours certainly ain’t.” May be Sweden is.

Push Technology – Finally There?

22. August 2009

Real push technology is here. zOMG. And it only took 12 years.

What am I talking about? Lets go back in time a bit.

Sometime around 1997, Netscape came up with the CDF-Format, and said it would revolutionize the web, because it would “push the web to the users” instead of the users having to fetch it from the web. People were all excited, Microsoft said they would do something much better called the “Active Desktop”, and everyone was on the edge of their seats.

Then, what happened? Appearantly, not much. CDF never became popular, and more or less faded away. So did Netscape, eventually. Active Desktop was a continuously crashing thingy that nowbody could make real use of, which had very little in common with CDF, and shouldn’t have been programmed in the first place. The revolution did not happen. At least not yet.

Funnily enough, there was no “pushing” involved at all. The users would have an application on their systems (conveniently bundled with the Netscape browser), which would periodically fetch a list of things from the web (the CDF data), see if there was anything new. It would then present the new stuff to the user. Notice the “fetch” part. Its the user application checking periodically if something new has arrived, not the server saying “hey, I got something new”. So if you set that application to check every 60 mins, you might get an important information 59 minutes late – just as if you had set your email application to fetch new mails only once an hour. The only thing you could do to prevent this would be to check often, causing lots of unnecessary traffic and load on both your PC and the server. The same information- the “list-of-things”-CDF-file – would be sent over and over again.

From a technology point of view its pretty much like a browser getting a certain webpage over and over again, only to be able to tell you whether it had changed since the last time you saw it. Meh.

The idea behind CDF actually made some sense, and became popular in a different form: RSS feeds. But RSS feeds inherited the same problem: it still is your pc fetching (or: “polling”) the information from the web. The subscriber (you) will check for content again and again, the publisher (the website) will continually answer with “these are the latest 10 news items”, your computer will repeatedly check if there was anything new, and if so, tell you about it. What a waste of resources, all because the server couldn’t shout at your system when something new was there.

Up to now, it seemed that was the end of the story.

So, what happened? Google and a bunch of smart guys who obviously were better at coding stuff than at giving it a good name came up with pubsubhubbub.

It starts out with the same situation: A publisher and a subscriber (hence the pub-sub in the name). Whats new is the hub. With the new protocol (which is an extension to RSS), the publisher can actively inform the hub if there is anything new. It is not necessary that the hub checks for new stuff, and thus, the publisher does not have to wait for the hub to check. The hub, in turn, is able to actively inform the subscriber (you) about new information. Again, it is not necessary for the subscriber to check with either the hub or the publisher for news. The publisher is actually “pushing” the information to the subscriber, via the hub. The round trip time of this procedure should be in the region of seconds.

You turn on your PC. Your RSS feed reader checks for news, because it might have missed whatever happend while it was turned off. Up to this point its the same as it has been since 1997. But now your RSS feed reader tells the hub “I’m here, tell me when there’s anything new”, and goes to sleep. If something new is there, the hub will then wake up your RSS reader …. tadaaa! Thats all there is to it.

Effectively, this means your RSS feed reader will be able to get the news only seconds after the publisher publishes it, without the traffic that would be caused if you set your RSS reader to check every second. Your PC actually uses less computing power and bandwith than before.

It only took a mindboggingly long 12 years until technology did what marketing said. And no, thats not going to be a revolution, but it sure is a neat solution to a very common problem.

Oh, and before anybody asks: there is no “bub” in it. They just thought it was funny calling it that.


P.S. Wake my hub if you have anything new 😛

Photoshopping – when pictures lie.

25. July 2009

I guess almost everyone has a digital camera these days, and we have all tried to enhance an image in some sort of image editing software. Now those who tried probably found out that editing images is difficult, and that for the most part it cannot be done with a few mouse clicks. From Adobe’s famous image editing software PhotoShop, this art has gotten the name “photoshopping”. If you know what you’re doing, you can substantially alter an image to your tastes. Done right, it can be an art in itself.

So, what’s wrong with photoshopping?

Lets just take an example, okay? Take this image that came through Reuters in 2006. What do you think it shows?


It shows Beirut on fire, right? And heavy black smoke, right?

Readers who know what a “clone brush” is, please look closely at the smoke in the upper left corner. Does that look “cloned” to you? It sure did to somebody.

Now as it turned out, that image wa really a photoshopped version of this image:


That is Beirut with one building on fire and some smoke.

Now, I’m totally not getting into why there was an israelian air strike in the first place, and what would be right or wrong with that. What I’m aiming at, is that we just cannot trust images anymore the way we used to. There have been numerous occasions where photoshopped images were released as “news”, this just happens to be a very prominent example.

This raises quite a few questions:

Q1. Why was the image altered in the first place?

I don’t know much about Adnan Hajj, the photographer, so I can only do guesswork on question 1. A lot of photographers get paid only if their photo gets published. So if you are a photographer, you want to get images that are exciting and news-worthy. The manipulated image sure looks more alarming than the original. With the level of competition on the news market, news agencies have outsourced more and more staff. People who only get their money if they sell a photo will be more likely to forge something to make a living than somebody who gets a regular salery no matter what.

On the other hand, Hajj is lebanese, and might have done that to shift public oppinion. But using the same kind of reasoning, the whole thing could also have been a setup to discredit anybody reporting on that side of the war.

Sorry, I cannot give you the real answer on what happened here, but it turns out not to be as important as you might think. Sure, Reuters could rely more on their own staff, but they cannot do so entirely. And there will always be somebody tampering with images just to get them sold or to make a point.

Q2. Did Reuters know the image was modified?

I’m not sure which answer I should hope for. Either they didn’t, which would be an unbelievable level of neglegance, or they did, which would be a level of bias only few people would expect from them either.

With only very few exceptions, it is possible to detect image manipulation. Even on a large scale. If they had any provisions to check for altered pictures, that would have caught Hajj’s image. Any experienced digital artist can easily see that it was photoshopped – if the Reuters staff didn’t, they’re blind or plain incompetent. There are ways to automatically detect such conduct, or at least raise a flag for further investigation. They could probably collect the RAW camera image files, which are a bit harder to manipulate, and check if the image had been altered.

On the other hand, what if they did know it was manipulated? Is Reuters biased, as some people think? Honestly, I don’t know, but if that were the case, they would be pretty stupid in releasing this particular photo. As said before, its rather easy to notice that the image is photoshopped, even with the low quality copy shown above. If they had wanted to manipulate public opinion, I personally find it hard to believe that they wouldn’t do a better job at it.

In the end, they were either sloppy at manipulating it, or they were sloppy checking for manipulation. That probably says more about how a news agency works than anything else.

(side note: if you don’t believe that this could have been automatically detected, here are two white papers on how it very well can be done: Image Manipulation Detection with Binary Similarity Measures [PDF],Image Manipulation Detection [PDF] )

Q3.  What does all that mean to us?

To us, it does not really matter why the image had been altered, since there will a number of reasons to do so, and there would always be reasons to do that.

It does shed a bad light on Reuters, and we have no reason to believe that Reuters is any different from other news agencies.

In the end, it should teach us that we cannot believe something to be true, just because we see a photo of it. Not that that would be all that new – there have been manipulations before digital imaging, and you never know how and why a picture came about. It could have been a set-up, an optical/analog manipulation or a digital one.

A picture sure can say more than a thousand words, but it can also tell more than a thousand lies.

Q4. So why am I blogging about the whole subject, anyways?

Its not new… been there, done that, got the t-shirt, now what?

Any new technology brings new opportunities and new pitfalls. This time its digital image manipulation and a stubling news agency. Next time it will be something else. What we really need is discussion on the technology’s implications.

We need to set moral standards on how a technology may be used, and where it should not.

After all, the pitfall doesn’t suddenly turn the technology into one from hell. It is always people who decide what they make of it – just think “nuclear fission” here. It can be used for nuclear bombs and in power plants. It is up to us to decide which uses we allow, and which we prohibit. Despite how tricky it can be to monitor its use we do need an open and public discussion on the use of technology.

There are a number of pretty new technologies that really do need such a discussion. Photoshopping is just one of them. I’d love to read your comments on this.

What a great moment of geekdom.

16. May 2009

wolframalpha1Yes! I was there when it went live the first time!

Its gonna be off and on over the weekend according to their Twitter feed. So at least starting on Monday, March 18th you should be able to access the system as well. In the mean time, here are the answers to my previous questions

Afghanistan GDP per capita 217.79 USD (2005 estimate, about 1/200th of the US)

Afghanistan life expectancy 44.6 years.

AIDS infection rate in Africa 4.17%. more than a fourth of the population of Swaziland is infected. 2.261 million deaths per year, but sure, condoms are a bad idea.

Rwanda deaths unfornunately did not show a diagram, and did not distinguish how people died, but here’s the number anyways136,853 (2008 estimate)

wolframalpha2The actual launch did resemble WarGames a bit though. Lets hope its not SkyNet 🙂

In any case: More data available to the public, and making them accessible is a Good Thing(TM).

And just because people will insist asking that question over and over: No, it is not going to make people dumb!

How availability of information could possibly make people dumb is beyond me.

Wolfram Alpha going live

15. May 2009

A ranting blog, and I start with news… probably not a good idea, but this one is such a biggy that I had to do it anyway:

Tonight at 12am UTC, Wolfram Alpha will be connected to the internet.

Now whats the big deal? Thousands of websites are put online every day, so what? Now Wolfram Alpha might become the third source of reference to a lot of you. Google, Wikipedia, Wolfram Alpha. But then again, it might be one of the greatest internet failures of all times. Here is a screencast showing what it is said to be doing.

And since you’re probably reading this when it already has gone online, you can go ahead and test it yourself. Supposedly, it will answer questions like

Whats the GDP of Afghanistan, and what is the average life expectancy of a new born child there?

Now that is pretty cool, right? Yes, it is. The power of a huge database (and I do mean HUGE) is at your fingertips, and 10,000 processors to compute your answer. Deep Thought, anyone?

It will not, however make you ask that question in the first place. And this is where the ranting comes back in. I sure hope it’s as easy to use as that screencast suggests, but even if that is true, and if it had all the data you wanted, will people start putting the right questions in?

What is the AIDS infection rate in Africa? How many people are killed every day in Rwanda?

While politicians and the press totally freak out over swine flu it seems questions are not en vogue. Have questions ever been? At least recently, people have chosen quick answers over tough questions. And more people than you’d imagine will put “Am I sexy?” inYer. Sure. You’re so totally hot and sexy that you can now safely go away, stop using precious CPU cycles,  and lead your life of ignorance happily ever after, mkay?

For the rest of us, may be its time to ask questions again, and may be this time you have a better chance for answers. I for one thing would love to see our federal government spendings in that database, but hey, that might be a bit too dangerous.

The Binary Grunt.

P.S. And by the way, Wolfram Alpha is totally not a Google rival, just like Wikipedia isnt. It’s not even a search engine! Sorry, folks, you’re totally not getting it. Google is about searching the net, not about finding answers to questions. If you must compare WA to something, compare it to ask.com pwleeeze?

P.P.S. and H1N1 is not more dangerous to you than “regular” flu – but panicking is so much more fun, right?