> Regarding Sitefinder, one of the ironies surrounding that issue was that what VeriSign was prohibited from doing, is now rather routine for ISPs and even browser manufacturers.

What VeriSign did with SiteFinder at that time and that generated a lot of heat
and ended in lawsuits and counter lawsuits was fairly simple and
slightly different
from what some ISPs are doing today.

Being the TLD operator for .COM and .NET the just added what is called
wildcard records on the zones they managed, sort of a *.COM and *.NET
pointing to a specific IP address that ended in the SiteFinder site.

A wildcard record is part of the DNS specification and commonly used for
example to direct all incoming email traffic to one or more mail servers for
a given domain. Say that you have MILTON.COM and many subdomains
and machines under that domain, a common Mail eXchanger record could
be something like:

*.MILTON.COM.   3600 IN MX 10 RELAY.MILTON.COM.

This record says any mail for all destinations ending with MILTON.COM
must be sent to the server RELAY.MILTON.COM.

Nothing wrong with it, commonly used and accepted.

What VeriSign did was to add a wilcard address record like:

*.COM.   3600  IN  A  123.123.123.123

being 123.123.123.123 the IP address of the SiteFinder server.

Then when any application sent a query for a given name under .COM, if
the name didn't exist instead of receiving the NXDOMAIN response the
application received the IP address assigned to the wildcard record,
if the application requesting the query was a web browser after
obtaining the IP address it initiated the HTTP connection via TCP and
sent the GET with the URL you were looking for, which SiteFinder
used to build a dynamic response page based on your request.

What many ISPs are doing looks similar, but they use a different trick
and it's worse because they are *tampering* with the DNS traffic.

Let me show how the queries/responses look like assuming first a
nonexistent TLD and a subdomain of it, lets say GREAT.MILTON.

I'll send the same query (ask for the IP address for GREAT.MILTON) to
two different name servers, one from NTT (129.250.35.250) and one
from Time Warner (24,93.41.125).


[root@tango /]# dig @129.250.35.250 -t A GREAT.MILTON

; <<>> DiG 9.2.1 <<>> @129.250.35.250 -t A GREAT.MILTON
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 58876
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;GREAT.MILTON.                  IN      A

;; AUTHORITY SECTION:
.                       9546    IN      SOA     A.ROOT-SERVERS.NET.
NSTLD.VERISIGN-GRS.COM. 2009100400 1800 900 604800 86400

;; Query time: 44 msec
;; SERVER: 129.250.35.250#53(129.250.35.250)
;; WHEN: Sun Oct  4 02:32:57 2009
;; MSG SIZE  rcvd: 105

In this case we received the correct response NXDOMAIN from the
NTT name server.

[root@tango /]# dig @24.93.41.125 -t A GREAT.MILTON

; <<>> DiG 9.2.1 <<>> @24.93.41.125 -t A GREAT.MILTON
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24888
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;GREAT.MILTON.                  IN      A

;; ANSWER SECTION:
GREAT.MILTON.           10      IN      A       24.28.193.9

;; Query time: 65 msec
;; SERVER: 24.93.41.125#53(24.93.41.125)
;; WHEN: Sun Oct  4 02:34:59 2009
;; MSG SIZE  rcvd: 58

In this case the Time Warner server is intercepting the response from the
root server that as we've seen before was that the domain does not exist
and replacing it with the IP address 24.28.193.9, which happens to be
the web server from TW/RoadRunner that will build the response page
based on the original URL you typed.

It does not matter if it is a non existent TLD, the interception works for any
TLD, gTLD, ccTLD.

[root@tango /]# dig @129.250.35.250 -t A IDONTTHINKTHISEXISTS.COM

; <<>> DiG 9.2.1 <<>> @129.250.35.250 -t A IDONTTHINKTHISEXISTS.COM
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 61322
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;IDONTTHINKTHISEXISTS.COM.      IN      A

;; AUTHORITY SECTION:
COM.                    900     IN      SOA     a.gtld-servers.net.
nstld.verisign-grs.COM. 1254642147 1800 900 604800 86400

;; Query time: 48 msec
;; SERVER: 129.250.35.250#53(129.250.35.250)
;; WHEN: Sun Oct  4 02:42:48 2009
;; MSG SIZE  rcvd: 115

Correct response (NXDOMAIN) from the NTT server.


[root@tango /]# dig @24.93.41.125 -t A IDONTTHINKTHISEXISTS.COM

; <<>> DiG 9.2.1 <<>> @24.93.41.125 -t A IDONTTHINKTHISEXISTS.COM
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20086
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;IDONTTHINKTHISEXISTS.COM.      IN      A

;; ANSWER SECTION:
IDONTTHINKTHISEXISTS.COM. 10    IN      A       24.28.193.9

;; Query time: 54 msec
;; SERVER: 24.93.41.125#53(24.93.41.125)
;; WHEN: Sun Oct  4 02:44:15 2009
;; MSG SIZE  rcvd: 82

Tampered response from Time Warner.

Web browsers have other "helpers" to suggest sites if the URL or site
does not exist.

What can we do about this, probably nothing, its a nuisance that does not
require you to change ISPs or web browsers, you have always the option
to manually configure the name servers your machine uses, you just need to
pick at least two that are not blocked and tampered with.

The real problem behind all this, and that was one of the fundamental issues
with SiteFinder, is that this type of manipulation of the responses from the
authoritative name servers change the normal behavior of the overall DNS
system, then this can brake applications and complicate any analysis or
modeling of the DNS (not just the root zone).

Why ISPs do this ? some say to "help" the clients, but actually when you
see the "sponsored links" it means somebody is paying to be on that list,
ie additional revenue for the ISP coffers.

My .02
Jorge