Basically a layer 2 switch operates utilizing Mac addresses 
in it's caching table to quickly pass information from port 
to port. A layer 3 switch utilizes IP addresses to do the 
same. 

While the previous explanation is the "What", for folks in 
networking the following "How" is far more interesting. 

Essentially, A layer 2 switch is essentially a multiport 
bridge. A layer 2 switch will learn about MAC addresses 
connected to each port and passes frames marked for those 
ports. It also knows that if a frame is sent out a port but 
is looking for the MAC address of the port it is connected 
to and drop that frame. Whereas a single CPU Bridge runs in 
serial, todays hardware based switches run in parallel, 
translating to extremly fast switching. 

Layer 3 switching is a hybrid, as one can imagine, of a 
router and a switch. There are different types of layer 3 
switching, route caching andtopology-based. In route 
caching the switch required both a Route Processor (RP) and 
a Switch Engine (SE). The RP must listen to the first 
packet to determine the destination. At that point the 
Switch Engine makes a shortcut entry in the caching table 
for the rest of the packets to follow. Due to advancement 
in processing power and drastic reductions in the cost of 
memory, today's higher end layer 3 switches implement a 
topology-based switching which builds a lookup table and 
and poputlates it with the entire network's topology. The 
database is held in hardware and is referenced there to 
maintain high throughput. It utilizes the longest address 
match as the layer 3 destination. 

Now when and why would one use a l2 vs l3 vs a router? 
Simply put, a router will generally sit at the gateway 
between a private and a public network. A router can 
perform NAT whereas an l3 switch cannot (imagine a switch 
that had the topology entries for the ENTIRE Internet!!). 
In a small very flat network (meaning only one private 
network range for the whole site) a L2 switch to connect 
all the servers and clients to the internet is probably 
going to suffice. Larger networks, or those with the need 
to contain broadcast traffic or those utilizing VOIP, a 
multi network approach utilizing VLANs is appropriate, and 
when one is utilizing VLANs, L3 switches are a natural fit. 
While a router on a stick scenario can work, it can quickly 
overtax a router if there is any significant intervlan 
traffic since the router must make complicated routing 
decisions for every packet that it recieves.