Fu-Hau Hsu
SUNY at Stony Brook
USA

Tzi-cker Chiueh
SUNY at Stony Brook
USA

As the operation of modern society gets more and more relied on computer networks, the damage and lose resulting from security
incidents are no longer local and trivial threats. From public
infrastructure to business running, from aviation system to national
forces, a compromised and crippled system imposes an unaffordable
price upon its users and brings impacts as serious as military actions
or organized crimes to them. As a result, developing effective
security solutions to protect networks becomes a serious and emergent
issue. Among diverse security problems, OS fingerprinting,
port-scanning, TCP/IP implementation bugs, buffer overflow attacks,
return-into-libc attacks, and TCP connection hijacking are 6 of the
most notorious security threats. Most of these threats are results of
TCP protocol abuses, therefore an appropriately designed host which
provides secure TCP/IP services could solve most of the security
problems. In this paper we propose a Centralized TCP/IP (CTCP)
architecture to provide secure TCP services. By redirecting traffic to
non-existent hosts or traffic to non-open-to-public ports to an
user-level CTCP process, CTCP creates an illusion that every IP
address belonging to the inner network is used by an inner host and
all of its TCP ports are open; thus, no matter what set of inner ports
is open to outside network, the scanners always get the same results,
making port-scanning a useless work. Allowing only TCP payload data to
be exchanged between the outside network and the inner protected
network enables CTCP to block the exchange of OS-sensitive data
between the two parties, such as packet headers; therefore, hampers
the reveal of inner hosts' critical information such as OS types. For
buffer overflow and return-into-libc payload generated by traditional
exploit-code-generating programs, by using addresses of an user
process stack as a hint, CTCP could detect them by only monitoring
the packet-level traffic. Moreover, through RST packets, CTCP could
automatically disconnect stealthy TCP session hijacking. Even though
providing so many protections, without sacrificing system performance
(in some tests CTCP even increases the throughput), CTCP is completely
transparent to all the hosts on both sides of it. As a result, even
though attackers plan to assault CTCP, they even don't know which
machine they should attack. Besides, without modifying any host in the
protected network, CTCP provides an architecture which reduces the
complexity to install or patch up-to-date security-related software.
Experiment results show that with a higher data throughout than
original routers', CTCP could effectively obstruct scout tools, such
as Nmap and p0h, to collect important information of hosts protected
by a CTCP host. For the six BOA and RL exploit strings we tested, CTCP
can detect all of them. At the same time for 404M bytes unharmful data
which consist of binary, image, graphic, document, html files, CTCP
creates 0 false positive. With traffic consisting of 10000 TCP connections and generated by a Pentium 4 machine, CTCP (1.1GHz)
equipped with Giga bit NICs has a 420.3 Mbits/sec throughput. Under
the same environment, a Linux router's throughput is 409.1 Mbits/sec.

Keywords: CTCP, TCP/IP server, buffer overflow attack, port-scanning, OS fingerprinting, return-into-libc, DoS/DDoS, TCP connection hijacking

Read Paper (in PDF)