BAIR "image filtering" has 0% accuracy rate

Bennett Haselton
6/6/2000

Introduction

Exotrope, Inc. sells a blocking program called BAIR ("Basic Artificial Intelligence Routine") which claims to recognize and block pornographic images through advanced artificial intelligence, using an "active information matrix that scans and evaluates incoming material". (The term "active information matrix" does not appear anywhere on the Web except Exotrope's pages, so they probably made it up.)

We tested the BAIR image filter against 50 random pornographic images (as well as a control group of 50 random non-pornographic images) and also left it running during some normal Web surfing. We found that:

  • None of the 50 pornographic images (or the 50 non-pornographic images) were blocked by the image filter. We double-checked that the image filter was in effect and that the browser cache had been emptied in advance, and confirmed that the filter was enabled by loading the HotMail banner and the CNet masthead that we knew to be blocked (see below).
  • During "random surfing", the only images that we found to be blocked were a HotMail MSN banner, the CNet masthead and an ad on CNet. But these same images would be unblocked if we added the string "?randomtext" to the end of the URL, even though this did not change the content of the images.

We also describe how third parties can test BAIR and confirm that the image filter doesn't work.



Contents:
Claims made by Exotrope, Inc. about the effectiveness of the BAIR filter
How we tested the BAIR filter
 - List of 50 random pornographic images and how we selected them
 - List of 50 random non-pornographic images for the control group
How you can test the BAIR filter
 - How to set your browser to use the BAIR proxy server
 - Suggestions for testing image blocking

Claims made by Exotrope, Inc.

Like almost every other blocking program, BAIR uses a pre-set "blacklist" of URL's that will be blocked no matter what content appears on the page, such as http://www.playboy.com/. But Exotrope claims that their software can also block images on sites that are not yet on their blacklist, if the images are detected to be pornographic. According to press materials from Exotrope about the image filter:

"For someone interested in Internet safety for the family, this means that brand-new websites that do not appear on obsolete "firewall" lists are filtered. "Safe" websites that have had pornography added to them by hackers are filtered. Pornographic websites disguised under unassuming names are filtered."

Exotrope's product information page for BAIR describes the image filter in impressive-sounding terms:

"The artificial intelligence creates an active information matrix that scans and evaluates incoming material. It has the ability to" teach" itself "on-the-fly" to consistently maintain accurate filtering of any Web site, and it does this whether the user is accessing the site for the first or the hundredth time."
The same page states that BAIR's image filtering is what gives it an edge over other products:
"Because of its proprietary AI-based filter, new pornographic pages, "clean" Web sites that have been hacked into, and pornographic Web sites disguised under unassuming names are safely blocked by The BAIR.

"Firewall approaches, 'Block' and 'Go' listings, time limitations, and other methods restricting access to objectionable sites are ineffective, easily outmoded, and allow the user to develop a false sense of security."

How we tested the BAIR filter

We configured Netscape 4.61 for Windows 98 to use the BAIR proxy server as described in the section below, How to set your browser to use the BAIR proxy server.

We then tested the BAIR filter using 50 randomly selected pornogaphic images from the Web. These images were downloaded and moved to a private Web site that could not possibly be on Exotrope's "blacklist", so that we could test whether the images would be blocked based on their content. We also used a control group of 50 randomly selected non-pornographic images, to see how many of those would be blocked by BAIR.

The list of pornographic images, as well as our method for collecting a "random" sample, is described here; the list of non-pornographic images that we used is here.

How you can test the BAIR filter

How to set your browser to use the BAIR proxy server

The BAIR program for Windows allows for the creation of "unfiltered users", "filtered users" and "ultra-filtered users", with dynamic image filtering only being applied at the "ultra-filtered" level. But the program is cumbersome to use, since it shuts down all other applications every time you switch between a "filtered" and "unfiltered" user, and while you are logged in as a "filtered" user, BAIR does not allow any instances of Notepad, Windows Explorer, or DOS prompts to run, warning that BAIR has detected "an application that can be used to bypass the BAIR filtering system". Also, the BAIR installation program that we tested overwrites system DLL's (such as asycfilt.dll) with older versions of the same DLL's -- generally regarded as a very amateurish Windows programming error, one that can cause all kinds of computer problems that are almost impossible to track down.

However, all that the BAIR program does is set your browser to use
  199.105.253.24 port 3137
as a proxy server, and then block users from disabling this proxy setting in Netscape. When you change users and log in as an "unfiltered user", BAIR connects to bair4.exotrope.com port 3000 and sends the string:

  209.180.197.220/0

followed by two newlines, where "209.180.197.220" is your IP address, and "/0" indicates that you are switching to an "unfiltered user". After that, the BAIR proxy server will not filter any pages requested from your IP address, until you switch to a "standard filtered" or "ultra filtered" user.

When you switch to a "standard filtered" user, BAIR connects to bair4.exotrope.com port 3000 and sends the string:

  209.180.197.220/3

followed by two newlines, where "209.180.197.220" is your IP address. When you switch to an "ultra filtered" user, BAIR connects to bair4.exotrope.com port 3000 and sends the string:

  209.180.197.220/1

followed by two newlines, where "209.180.197.220" is your IP address.

So, to test BAIR in "ultra filtered" mode:

Whenever you switch filtering settings, remember to empty your browser's cache -- hitting "Reload" is not generally sufficient to ensure that a page or image will be displayed using the latest filtering settings. This was a problem using Netscape 4.61 for Windows, although it's not really a bug in the browser since the browser was designed with the assumption that most proxy servers don't alter content. (What actually happens is, Netscape loads, say, www.yoursite.com/yourimage.jpg at 4:00 through an unfiltered connection. Then the user switches to a filtered proxy server and tries to load the page www.yoursite.com/yourimage.jpg at 4:01. Netscape sends a request to www.yoursite.com via the proxy server, saying, "I have a copy of yourimage.jpg that was current as of 4:00 -- has the page changed since then?" and www.yoursite.com sends back a reply, still going through the filtered proxy server, saying "No", so Netscape re-loads its cached copy of www.yoursite.com/yourimage.jpg, even though the image might look different if it were reloaded through the proxy server, which might alter or block the image based on its content.)

Suggestions for testing image blocking

If you leave your browser set to use the BAIR proxy server under the "ultra filtered" setting and surf normally, you will probably encounter some random blocked images. Even though none of the 100 images used in our experiment were blocked by BAIR, a half-hour of Web surfing can easily result in several times that many images being loaded, which is how we found the blocked HotMail banner and CNet masthead.

If you do find an image that is blocked, try loading the image URL with "?randomtext" appended at the end; this won't change the contents of the image, but the new image URL might not be blocked by the image filter.

If you download a pornographic image file from one Web site and upload it to your own Web site to see if BAIR will block it, rename the image if it has an obvious name like "fucking.jpg", otherwise BAIR will block it based on the URL, without letting the image filter examine the contents of the picture.