Dynamic Arp Inspection , DAI

It strikes me that not many organizations enable Dynamic Arp inspection or DAI for short, on their switches users connect to. Why bother with DAI anyway you might ask. I would say this is something that it easy to activate to make switch block ARP messages that are not legit on your network. Those ARP messages can be utilized by an attacker to trick devices on a segment that the attacker computer is the router which will make traffic from client computers go to the attackers computers instead of the router.

First we need to think about how ARP work. When a host want to communicate with another host and it does not know the MAC address of the target host it send out an ARP messages asking who has this IP address, the target host should then do an ARP reply saying I have the IP address and MAC address you asked for. Next the communication can be established on Layer two between the learned mac addresses.

In comes the bad host answering the ARP reply (even though it should not) or sending a Gratuitous ARP (sort of an ARP reply without anyone asking for it). ARP reply can then contain the asked for IP address but with the MAC address of the bad host tricking the other host to send traffic to the bad host instead of the intended host. 

This can be stopped by enabling DAI on the switch which is quite easy.

ip arp inspection vlan x , where x is the vlan you enable DAI on

ip arp inspection vlan 101

To see info about DAI you use "show ip arp inspection"

Source Mac Validation      : Disabled

Destination Mac Validation : Disabled

IP Address Validation      : Disabled

 Vlan     Configuration    Operation   ACL Match          Static ACL

 ----     -------------    ---------   ---------          ----------

  101     Enabled          Active

 Vlan     ACL Logging      DHCP Logging      Probe Logging

 ----     -----------      ------------      -------------

  101     Deny             Deny              Off

But how does the switch know what mac address and IP addresses are legit? 

It uses the DHCP binding database. So to be able to use DAI you must first enable DHCP snooping to make switch to build the dhcp binding database (see my previous posts about DHCP ). This will introduce a problem for hosts with a static IP address not received from a DHCP server, the IP and MAC address binding is not in the snooping database on switch and will therefore be blocking the ARP replies making other hosts not be able to communicate with the host with static IP address since no one will learn its MAC address.

You will see this in the switch log.

*May  9 12:53:55.779: %SW_DAI-4-DHCP_SNOOPING_DENY: 1 Invalid ARPs (Res) on Et0/0, vlan 101.([0000.0c9f.f065/10.1.101.1/5000.0012.0000/10.1.101.11/14:53:54 EET Sun May 9 2021])

Log entry means that port E0/0 , vlan 101 has blocked an arp packet from 10.1.101.1 with MAC 0000.0c9f.f065 going to 10.1.101.11  with MAC 0000.0c9f.f065

There is two solutions to this. You can either set the interface as trusted.

SW3(config-if)#ip arp inspection trust

similar to the DHCP trusted ports function.

You probably do this with the uplink interface leading to other switches and routers. But remember that it is basically the same as disabling ARP inspection on that interface.

Or you can create a static binding with mac address, IP address, vlan and interface.

This is accomplished by creating an ARP access list and then apply it to the vlan.

arp access-list VLAN101

permit ip host 10.1.101.1 mac host 0000.0c9f.f065

Add as many IP address to MAC address mapping you need in this ACL and then apply the ARP ACL to the vlan

ip arp inspection filter VLAN101 vlan 101

You have now allowed a host to make ARP reply with IP address 10.1.101.1 to use MAC address of 0000.0c9f.f065 on vlan 101.

You can configure a lot more with DAI, such as rate limit  and logging and what parameters in the ARP package to verify.

With "show ip arp inspection" we saw that there was no inspection of mac-addresses and IP address. I suggest you enable validation of these fields in the ARP package.

ip arp inspection validate src-mac dst-mac ip

To rate limit ARP packages use the interface command "ip arp inspection limit rate x" where x is number of ARP packages per second. Default is 15 which might is too small on a trunk interfaces.

You see, DAI is not that complicated at all and is easy to enable.

DHCP take 2

Now we will look at a larger topology with multiple switches, routers, portchannel and SVI interfaces to see how DHCP work.

Clients are connected VLAN101 on switches with Layer 2 functionality only, they are then connected with multiple connections creating a logical Layer2 portchannel interface between itself and the Layer3 switches  (Port-Channel 1 for North-South connection, Port-Channel 2 diagonal connection). They are the first hop and run VRRP/HSRP for redundancy and they are further connected to routers which is the connection further where we have the actual DHCP servers.

Client is connected to Swicth3 and the switchport is configured as an access port in vlan 101. Vlan 101 is also enabled on the trunk to switch1 and 2.

On switch1 and 2 there is a SVI created for vlan 101 and there is also HSRP created on vlan 101 for clients to use as the Default Gateway.

When Client broadcast a DHCP Discover it will just pass through switch3 and land on SVI vlan 101 on switch1 and 2. on Interface vlan 101 we need to configure ip-helper (dhcp relay agent) to convert the broadcast to a direct unicast to the DHCP servers in the network. You can and should have multiple ip-helper addresses configured for redundancy and loadbalancing.

SW1(config)#interface Vlan101

SW1(config-if)#ip helper-address 10.10.10.1

One thing that might trip you off here is that when the DHCP relay agent on Switch1 or 2 convert the broadcast to unicast and send it to DHCP server it will use its own SVI interface address as the source and you must have correct routing between DHCP server and the SVI interface.

No surprise here even though it’s a larger environment than the small we started with in DHCP take 1

Now over to some security concerns regarding to DHCP.

What if someone introduce a new DHCP server (either by mistake or maliciously) on vlan where we have clients? This rouge DHCP server might be handing out IP addresses that don’t work on our network which will cause clients to not be able to reach its resources or even worse handing out correct addresses but with wrong options such as default gateway to another address where the "bad" people sit and listen for traffic from clients picking up all kinds of information that should be kept secret. This could be done without even clients noticing something bad is happening. Traffic will be instead of the wanted between client and directly to the HSRP address shared between Switch1 and 2 going from client to the bad gateway where traffic can be intercepted and send on to the correct gateway (the HSRP address).

There is switch function called "DHCP snooping" to our rescue here. DHCP snooping will actually block DHCP server traffic if the port is categorized as untrusted and only allow full DHCP traffic if the port is "DHCP snooping trusted". DHCP snooping sit and listen for DHCP traffic and over time build up a database of mac-addresses, IP-addresses, vlan, lease time and ports, this information will then be used by the switch to block DHCP traffic which don’t match what the dhcp snooping database say, such as sending a DHCP release from wrong port or mac-address. The Snooping information can also be used by other functions in the switch such as arp-inspection, ip-source guard.

Configure DHCP snooping:

On interfaces leading towards DHCP servers you set the port as trusted to allow DHCP server responses.

SW3(config)#interface port-channel 1

SW3(config-if)#ip dhcp snooping trust

Then you enable DHCP snooping globally on the switch

SW3(config)#ip dhcp snooping

You must then enable DHCP snooping on the vlan's

SW3(config)#ip dhcp snooping vlan 101

You would think this is enough, no no no. when we enable dhcp snooping on switch3 it will not only listen for the dhcp traffic and build the database but also inject DHCP option 82 (Agent Information Option). DHCP relay agent insert information in option 82 to signaling to DHCP server which scope to use, this is done with option 82 field "giaddr", (gateway interface/ip address). Since our switch3 is layer2 only and actually run the relay agent it set the giaddr to null. When traffic comes to DHCP relay agent on switch1 it will be dropped because of giaddr is blank. To see this you must first enable debug for dhcp server.

SW1#debug ip dhcp server packet

and then you will see

---

00:37:45: DHCPD: Reload workspace interface Vlan101 tableid 0.

00:37:45: DHCPD: tableid for 10.1.101.2 on Vlan101 is 0

00:37:45: DHCPD: client's VPN is .

00:37:45: DHCPD: inconsistent relay information.

00:37:45: DHCPD: relay information option exists, but giaddr is zero.

---

With this we need to configure the relay agent on switch1 to accept empty giaddr field, we do this by allow the interface to trust option 82 information.

SW1(config)#interface vlan 101

SW1(config-if)#ip dhcp relay information trusted

Wireshark capture also reveals that option 82 is in the DHCP Discover packet.

Information in option 82 can be used by the DHCP server to further make a choice which DHCP pool to use or parts of pools or options  to send back in the offer to clients. You can get some type of clients a special range of addresses or if clients are connected to a specific switch they will get some option in the offer. What information in option 82 and how its structured will differ between vendors and possibly software versions. I will not go into how to do this nor how to make the DHCP server make decisions on what to offer clients.

There are other option you can use on switch1 to allow the empty giaddr such as set all interface to trusted or trust all untrusted which will allow all DHCP discover packets with empty giaddr regardless of which interface it comes in on, that’s bad idea in my opinion, better to be specific. Don’t forget in case of a network failure where traffic is received on other interfaces to set the other interfaces to trust the empty giaddr.

DHCP take 1

You would think DHCP is easy, well it can be but it also can be quite tricky and sometimes it's a security risk you as the network administrator need to take care of.

I will start with a very small network to get the concepts straight and then I will do more complicated network and also add in the security layer.

Let's start with a small network, one switch with everything in same VLAN, a firewall, couple of servers and some clients (picture not showing all connected devices to the switch).

When client is connected to network (and is configured to get an DHCP assigned address) it will broadcast doing a DHCP Discover of a DHCP server. This discover will contain several parameters that client want to have offered to them, IP address is obvious but others such as default gateway, DNS servers etc..

DHCP server will answer with a broadcast, DHCP Offer containing IP address and possibly other parameters.

If there are multiple DHCP servers answering the Discover client will pick one of them but usually the first Offer that it receives.

Client will then answer back with a broadcast DHCP Request to notify the second DHCP server that it will not take his offer but rather the first DHCP servers Offer, info about which offer to use is in the DHCP Requst packet even though its a broadcast.

The second DHCP server will see this DHCP request and see its not for him so he simply set the offered IP address as free in its DHCP database.

The first DHCP server will see the DHCP Request and write the offer into its DHCP database as assigned to client and also respond back  to client with DHCP Ack.

A simple thing to memorize the process is to remember DORA, Discovery, Offer, Request, Ack.

Packets towards DHCP server is from UDP port 68 to port 67 and packets from server to client is the reverse, UDP from port 67 to port 68.

Here we see wireshark trace of the DHCP communication.

We see the Discover, Offer, Request and finally the ACK.

In the DHCP packets we can see all kinds of information, "Client Identifier (61)" is most of often the MAC address of clients interface configured with DHCP address, "Option (50)" is an option for clients to request a specific IP address, most likely what they had previously, "Option (55)" contains all kinds of additional parameters that client want the DHCP server to tell it to use.

In the Offer back to client we see the IP address, subnet mask, default gateway (Router), DNS server and some timer options.

Lease time is how long client can use this address, this example is one hour which in real life is very short, its often set to days.

Renewal time is set by this DHCP server (a windows server 2019) to half of the lease time. This tells the client when half of the lease time has passed they can try to renew the lease. When this happens client will send a unicast DHCP Request directly to DHCP server asking if they can still use the same IP address. DHCP server will answer back with a unicast DHCP Ack together with all the other parameters should use. By just changing DHCP options on the server , clients will eventually get them and change settings on client side. If DHCP Request fails clients will have a slightly different behavior depending on implementation but it's common for clients to still send the unicast DHCP Request at a more frequent time until the Rebinding time kicks in and then change to broadcast.

Rebinding time is default set to 7/8 of the lease time.

What about when there is no DHCP server on the same segment that client is connected to?

DHCP client is broadcasting and therefore the DHCP discover packets cannot reach the DHCP server, so we need help somehow. 

The help is in form of a DHCP relay function on the router or in this case a Layer3 switch with routing functionality. The DHCP relay function will see the DHCP Discover packet and it will translate the broadcast to a unicast and send it to the configured address (10.10.10.2) with source of the interface facing clients (10.10.101.1). This is often called IP-Helper and is configured on the interface towards clients.

DHCP server will see the Discover and if there is a DHCP scope for the 10.10.101.0/24 network it will reply back to 10.10.101.1 with an Offer.

Cisco IOS command is 

Switch(config-if)#ip helper-address 10.10.10.2

In this wireshark capture created on the DHCP server segment we can see that there is no broadcast but only unicast between DHCP server and the DHCP relay agent IP address. Other than this the traffic is the same as when DHCP server and client are located on the same segment.

When the Renewal time comes up client will send a unicast directly to the DHCP server in the say it did before asking for new options and IP address.

The DHCP RFC 2131 for all the details.

Next post I will do a larger topology and add in some other options into the mix.

New focus

It has been long time since I wrote anything on this blog. Short reason is that I have not had time (or taken the time) to do so.

For the people that know me I have switched focus to my second baby “networking” which now is my main focus.

The switchover has been a lot of hard work but in the process I also managed to get the CCIE certification in Route & Switch , now renamed to Enterprise Infrastructure Cisco Certified Internetwork Expert

I have some topics that I will write about in the future so stayed tuned for some articles on mostly Cisco networking stuff.

Exchange Web Access changes name again

Every since Exchange 5.0 introduced access to mailboxes via a webgui, its name has changed several times.
Latest namechange is from Outlook Web App to Outlook on the web.
New features in Outlook on the web
Why the name change ? Your guess is equally good as mine, but I like that I don’t need to educate users about a new URL. It’s still https://..../owa