Cable Feng Shui

Your network will always grow, and due to the laws of entropy, left to itself, it’ll grow messier. Here are some things to consider while it’s still small and manageable:

Naming and documenting things

Use a simple switch naming scheme that you can live with and that scales. It’s perfectly OK to call your switches sw001, sw002 etc, even if it feels a bit un-personal. I’ve tried adding descriptive bits to my switch names (like sw001-core, sw002-project), but in the end, simpler turned out to be preferable, especially when i had to get a bigger switch somewhere and relocate the smaller switch somewhere which didn’t correspond to its name.

Use one or even two leading zeroes so that the switches alphabetise nicely. If you need more than two leading zeroes, you’re either way ahead of me (and you don’t really need this text), or way ahead of yourself.

Label your switches. The only thing you really need on the switch is the switch’s name. Other useful things to label on the switch are its IP address, but we’ll get to that, and what the switch does. Remember that you’ll most need the label when you least want it (ie in an emergency), so make it large enough, in a standard place and with a big enough font. If you don’t have a label printer yet, get one. A cheap and portable one is good enough to go.

Document which switch is which. Document which cable goes where. Label each friggin' cable or cable bunch (we’ll get into trunks and aggregates in a while) with some kind of label or identifier on each end of the cable, and on each place it might be helpful, like half way through a cable draw. I’ve found white plastic tape and sharpie to be excellent tools for this (since my printed labels won’t nicely stick to the cables). My favourite thing to put on the label are the names of the two switches the cable bunches are going to.

Document your VLANs. Draw pretty pictures. Use them as a tool for communication. Document for the future you who might be on vacation when the network is down and you’d like your network-savvy colleague to be able to figure things out with a minimum of your consultance.

Get your gear

Don’t skimp on the switches. I’ve bought somewhat-commandline-configurable switches (HP OfficeConnect series), used them, cursed them, taken them out of production and will not have another of them on my network. While i was able to do some of the configuration over the command line, the syntax was quite different from the “proper” HP switches, and it wasn’t possible to take config backups over scp.

Having managed switches everywhere will pay off.

Also, don’t overdo it. Ten gig switches would be nice, but they currently cost nearly tenfold compared to one gig switches. Have data to show whether you truly need faster switches.

Don’t buy switches that don’t go to one gig.

Get a head light and store it in the server/network room (closet). Once your switches and servers are taken over by cables, it can be really difficult to see whether that cable is in port 11 or 13, and you’ll need both of your hands.

Buy switches that you can configure over the command line, using a console cable. You’ll need this functionality one day. Have several console cables. Buy cables that are long enough that you can have one end connected to the switch and the other side connected to your laptop, which you’ll eventually will need to leave on the floor for a while. You’ll curse your choice of cable length if it’s inadequate at this time.

Learn to love the command line.

Stick to the same kind of switches. I like the HP 2530 series, but that’s me. I’ve used Cisco switches too, but turned out to require a paid yearly license to get firmware updates (i don’t know if this is still the case) so i turned away from them. Also, Cisco switches use the exact opposite logic for configuring them, compared to HP switches, so having a Cisco-HP mixture would probably drive me nuts.

Always use the same kind of screws to install your gear into a rack. This means you’ll need just one tool to detach them. I prefer the ones with the hexagonal hole. Also, have one or two extra tools of the right make (Allen wrenches?) stored in strategic places.

Keep things neat

Have a management VLAN. Do all your management only through this VLAN and the console. If you have a switch that you’ll leave out in the wild, don’t leave the management VLAN on any open switch port.

Give IP addresses only to your management VLAN interface, the other interfaces can go without addresses. Use DHCP reservations. Have the last octet of the IP address to be the same as the switch’s asset number (so sw042 gets the IP address 10.0.0.42, for example … if you have over 240 switches, these tips probably don’t apply to you). Once in a while, you’ll need to give IP addresses to other VLANs as well for debugging purposes. You may need to give it a static address if you can’t get the switch to talk to the DHCP server. If so, give an address which is out of the DHCP range (i usually leave the last few addresses, say .241-254, out of the DHCP pool). Remember to clean up afterwards or you will get surprised by address overlaps one day.

Use different coloured cables for different purposes. Be consistent. If you use blue for management and orange for link aggregates (or whatever), then don’t use a green cable because that one’s the only thing you can find at an arm’s length when you need another one.

Buy enough of patching cables, of different colours and lengths. At the patching panel, the ones that actually bend are nicer than the ones that are shielded and won’t.

Use velcro tape to bundle cables that belong together, and to the sides of your rack. I like to build a little contraption of velcro tape and those metallic strings that cables come wired in to have my cables along the server rack wall; the metallic string is easy to bend through the holes of the rack bits and the velcro tape is held neatly by the metal wire. Buy a whole roll, the kind you can rip, if you can find it. Use long enough strips of velcro tape, since you’ll inevitably add more cables to that bundle.

Use your switch ports in a consistent way. Have the trunk/aggregate ports to one end of the switch. Assign an even number of ports for a purpose, so that your block of ports always start at the same side side of the switch. If you can, leave one or two ports

Vulcans invented velcro tape. They can’t be wrong. That would be illogical.

Try to use cables that are just the right length, or at least try to have cables that are the right length when they are by your switch or patch panel. Really try to keep your patch panel and switch face tidy. Yeah, i know it’s hard.

Have your switches configured in a similar fashion, so you don’t have to wonder why you can’t ssh into this one switch when it otherwise seems to be working alright.

Use unique passwords. It’s really painless if you use a password manager. gopass (which is the evolved version of pass) is really good, especially as you want to share the network passwords with your admin peers.

Use Link aggregates and VLAN trunking to connect your switches together. Having more than one cable connecting the switches together is a Good Thing, as is not having to wonder which cable connects to which VLAN.

Name your link aggregates according to which switch they connect to. If you have an aggregate between sw012 and sw027, call them Trk27 on switch sw012 and Trk12 on sw027 (substituting for local dialect).

Security and elegance

Switch off non-secure management protocols like telnet, tftp and http, as well as the ability to have the switch configured over DHCP (unless you really, really need it). In all likelyhood, this is overkill, but one fine day, it might not, and then you’re a happier person. ssh is your friend, and if your switch doesn’t know ssh, it might be time to find a better friend.

Keep your switches updated and at the same firmware version. Updating switches is not fun and will create network outages, but at least you’ll have the same capabilities on all switches. For example, NTP came to HP switches fairly recently, and before that, it’s been SNTP or TIMEP. I’d rather have just one time keeping protocol to worry about than three.

Take backups of your switch configurations before you make that next config change (and after you got it to run). scp and git are your friends (as is automation).

Have a baseline configuration you can just copy-paste into a new switch. I wrote about some nice HP switch settings before. Good Laziness is another good value of an admin (Larry Wall has written about the Three Great Virtues of a programmer; laziness, impatience and hubris).