 CCIE #14618 |
I finally PASSED the CCIE Routing & Switching Lab Exam today, May 5th 2005, out at RTP in North Carolina!
I have been studying for the CCIE exam for almost 3 years now. I am proud and ecstatic to announce that I finally passed today out at RTP in North Carolina. I just got back into Chicago and found out about an hour ago. So I am wired to say the least.
I came out of the exam feeling very confident that I had passed it. I knew I had absolutely nailed it. The worst part was having to endure the torture of waiting and not knowing the whole way home. I kept second guessing all kinds of issues that I ran into. The mind can be an evil thing when it is left to its own devices…
Anyway, I am so happy and thankful that this chapter of my life has finally come to an end (so are my wife and kids). I already packed my library of Cisco Press books back onto my bookshelf and powered my lab rack down. Yeehaw! I can actually have my life back and stop spending all my nights and weekends studying my butt off in the basement. What a huge burden off my shoulders. Now I might actually be able to interact with humans again instead of routers and switches. I never thought I could get sick of working on this stuff, but I proved myself wrong.
Time to book at least a 2 week vacation to St. Somewhere.
For those that are interested, here is my Lab in my basement. This is where I spent all of my spare time preparing to take the CCIE Routing & Switching Lab exam. Fortunately the whine of the fans on all this gear does not carry throughout the house. The nice thing is I
added some APC remote start power supplies which allow me to leave the rack powered down and I can fire it up at any time from anywhere remotely.
I collected a lot of this gear off of eBay and some of it was spare hardware from the office. This is about the bare minimum of equipment you need to run through many of the practice lab scenarios. The only equipment I am missing is an LS1010 ATM switch and a couple of ATM modules for my 2600’s. That gets quite a bit pricey, so I just rented lab time for that since it was an insignificant amount of material to cover with respect to all the other topics.
If you think I am an extreme geek, then you haven’t seen anything yet. Check out Scott Morris’ lab…the only non Cisco employee to have 4 CCIE certifications! The man is a machine when it comes to this stuff.
Lab Details
 |
I now have a BlueCoat proxy server for WCCP testing. I upgraded my backbone routers for route injection to 2610’s with an additional 3rd one. I added a second 2523 router to act as an extended frame-relay switch for more port density. I upgraded R6 to a 3725 with dual FastE’s and shifted some of the routers around. I also added 2 APC MasterSwitches for remote power up and shutdown capability, so I don’t have to run all my gear around the clock and waste power.BlueCoat SG400-1 Proxy server Cisco 2523 Frame-Relay Switch (2T, 8A/S, 1BRI-S/T, 1R) Cisco Catalyst 3550 EMI Switch #1 Cisco 3620 (NM-1E2W, WIC-1T, NM-1E1R) Cisco 3620 (NM-1E2W, WIC-1T, WIC-1B-U, NM-1E2W) Cisco 2610 (1E, WIC-2T, WIC-1B-U, NM-4A/S) Cisco 2611XM (1FE, WIC-1T, NM-1V, WIC-2FXS) Cisco 3725 (2FE, WIC-2T, WIC-1B-U) Cisco 2620 AIM VPN/EP (1FE, WIC-2T) Cisco 2610 Terminal Server (1E, NM-32A/S, WIC-1AM) Teltone ISDN Simulator (with 2 BRI U Interfaces) Cisco 2523 Frame-Relay Switch (2T, 8A/S, 1BRI-S/T, 1R) Cisco Catalyst 3550 EMI Switch #2 Cisco 2621XM (2FE, WIC-2T, WIC-1B-U) Cisco 2610 (1E, WIC-1T) Cisco 2610 (1E) Cisco 2610 (1E) 2 APC MasterSwitch AP9225 w/AP9606 SmartCard 8 port rack mount PDUNot pictured… Cisco 2620 (1E, WIC-2T, WIC-1B-U) |
| New picture coming soon… | Here you can see the lovely mess of cables stringing all of this together. It is kind of hard to do cable management on the pile of ethernet, serial, and console connections. I was previously using back to back 6 foot DB-60 to v.35 cables which was a nightmare. I have since replaced all of those cables with 3 foot back-to-back DB-60 DCE to DTE cables and DB-60 DCE to SmartSerial DTE cables which cleaned up quite a bit of mess. I cleaned up a lot of this cabling using velcro tie straps last summer before we moved. I never posted an updated picture and I still have yet to get everything fully cabled up again. I’ll post an update when I do. |
|
Below are a few of the books that I have spent a lot of time reading, memorizing and practicing lab scenarios from over the past 2 years. CCIE Routing & Switching Exam Certification Guide Beyond CCIE studies, I’ve added the following books to my collection: Internet Routing Architectures |
|
Rolande's Ramblings 
Hi Scott,
What a coincidence: I just found your site because was googling on ‘choke router’. I am doing a global IPv6 study for ABN AMRO at the moment and I read your suggestions for a choke router and I thought: why not click the ‘About me’ tab. And then I read you have been working in the time I was working on ABN AMRO as a technical consultant at Verizon (former MCI/Worldcom global network) from 2000 to 2007. You must remember that ABN AMRO had an ATM link in those days from Chicago to London (or was it to Amsterdam?).
I see that you have an interest for IPv6 (“IPv6 For Enterprise Networks”). Do you have experience with implementations in real life networks like the one from ABN AMRO. Up till now I find that no examples of global companies exist that have fully implemented dual stack or even have fixed planned to do so in the coming 2 years.
Regards,
Joost Tholhuijsen
LikeLike
By: Joost Tholhuisen on May 1, 2013
at 11:13 am
Hey Scott! Nice work on optimizing network setting. Hope you don’t mind. I have some questions though:
1. Is there a software that would easily do this?
2. 10.9 is coming. Will the setting stick?
3. If you have fiber connection do i have to change the setting?
4. Are there any more settings that could be tweaked?
Thank you and awesome work!
LikeLike
By: mark gonzales on October 14, 2013
at 11:31 pm
I have not found any software packages out there to appropriately manage and optimize these settings. The settings should stick if you created your own custom /etc/sysctl.conf file to be loaded at boot time. The question is if any of the settings change names or become irrelevant in the new IP stack on 10.9. Fiber is irrelevant. What matters is MTU, bandwidth, and latency. These settings are tuned for someone with an ADSL connection which has an MTU lower than 1500 and typically 25 to 30 millisecond round-trip latency to its closest destinations and usually less than 10 Meg of bandwidth. At this point, I am not really aware of anything else to tweak. I would have to review the currently available options to see if anything has changed and maybe address other connection types.
LikeLike
By: rolande on October 17, 2013
at 6:30 pm
Thanks Scott! By the way your settings are working smoothly in Mavericks. Scott, I need to be clarified on some of the settings:
1. You have mentioned that kern.ipc.maxsockbuf value should be the sum of the net.inet.tcp.sendspace and net.inet.tcp.recvspace variables. Considering your settings, how come when you add those 2 variables the sum is 524288 and your kern.ipc.maxsockbuf value is 4194304? Maybe I missed something here?
2. MSS of 1440 is for PPPoE connection as you have indicated. Is it safe to use the default 1460 for non PPPoE connection in conjunction with your shared settings? My Dsl connection uses MER where in your modem connects directly to the internet without the need of username & password
3. After using your shared settings I have noticed that UDP receive buffer size has changed from 9216 to 196724 as indicated in MacOSX Cocktail app. Is this the by product of your settings?
Thank you again for your awesome work. Keep it up!
LikeLike
By: mark gonzales on October 17, 2013
at 8:39 pm
The maxsockbuf value can’t be less than the sum of the sendspace and recvspace variables. The default that Apple is setting is high enough. If you have no overhead on your DSL line and it is using PTM for transport, then you should be able to set the MTU to 1500. I switched to AT&T Uverse service this past year which does not use PPPoATM like their legacy ADSL service. Uverse uses PTM over VDSL which incurs no encapsulation overhead within the IP packet. So I have reset my mssdflt to 1460. The change in UDP buffers is probably related to the maxsockbuf value. It is a factor of the max socket count and the amount of assigned memory per socket.
LikeLike
By: rolande on October 18, 2013
at 9:11 am
Scott, furthermore, do you happen to know the syntax for udp send & receive windows, so it could be hard coded? Is there a default & optimum setting? I do a lot of openvpn using udp ports. Tweaking it might improve the throughput
LikeLike
By: mark gonzales on October 18, 2013
at 6:23 am
There is no such thing as UDP send and receive windows. UDP is a connectionless protocol. The TCP window size has to do with how much data can be sent or received without having to send or receive an ACK packet. The more you can send or receive in a window as latency increases, the better the overall throughput you will experience. You will not spend time waiting for ACK packets before you can send more. With UDP, since it is connectionless, you never have to wait for ACK packets. So, there is no window. You can flood the network with UDP and never have to wait for a response.
LikeLike
By: rolande on October 18, 2013
at 9:15 am
Thank you very much scott for your awesome advice!
LikeLike
By: mark gonzales on October 18, 2013
at 3:35 pm
Scott thank you for updating the IP stack. I do have a follow up now. Hope is not too much ask:
1. When ca we expect settings optimized for Mavericks?
2. Can you help with the formula on net.inet.tcp.slowstart_flightsize? Details as follows:
mtu 1492
mss=1460
Speed= 8Mbps
Can you give some more details on how to estimate the no. of packets. I cant seem understand from the explanation. Maybe a step by step guide on how you got 20 packets. Something a non techy would be able to relate
3. Will other parameters need to be changed considering my 8Mbps speed? How?
Thanks again for the awesome works you are providing us!
LikeLike
By: mark on October 25, 2013
at 7:10 am
1. The existing settings I have posted should work just fine for new OSX releases, barring any drastic IP stack changes. I usually don’t upgrade until 2 or 3 patch releases and in the case of Lion, I skipped it completely. So, I won’t revisit it, until I install the OS myself.
2. If your MTU is 1492 then, to avoid packet fragmentation, your MSS should actually be 1452 and not 1460. IP packets have a 40 byte header you have to account for. The key bit of info you are missing is what your average round trip latency is to destinations on the Internet using your connection. My best case round trip to most sites is around 25msecs. But, many sites are more like 40-50msecs. I would rather configure my settings for the average worst case scenario. So, you multiply your bandwidth by your average-peak round trip latency to determine the amount of data you can actually have in-flight on the wire. For example, 8,000,000 bits per second x .05 seconds (50msecs) / 8 bits per byte = 50,000 Bytes. Then I take 10% of that number (my arbitrary choice) and divide it by your MSS value. 50,000 x 0.10 = 5,000 Bytes / 1452 Bytes per packet = 3.44 packets. You can round up to 4 if you want. So you probably don’t want to have more than 4 packets in-flight on a TCP slowstart.
So, in my case, here is how I calculated my slowstart flightsize:
Line Rate based on actual speed tests
47Mbps = 47,000,000 bits per second
MTU = 1500 Bytes
MSS = 1460 Bytes
Average-Peak Roundtrip latency based on my own testing
~50 milliseconds
47,000,000 x 0.05 seconds (50msecs) = 2,350,000 bits
2,350,000 bits / 8 bits per byte = 293,750 Bytes
293,750 Bytes x 10% = 29,375 Bytes
29,375 Bytes / 1460 Bytes per packet = 20.12 packets
LikeLike
By: rolande on October 25, 2013
at 9:37 am
Thanks Scott. I have a follow up though:
1. The IP header we are talking about is always 40 bytes regardless of the connection type?
2. How do we estimate the average roundtrip latency? I live in asia, so latency is longer here
3. I use a lot of openVPN connection vis UDP protocol. Does this need a special tweak?
LikeLike
By: mark gonzales on October 25, 2013
at 4:01 pm
Scott,
I experimented with the guidelines on setting the optimum TCT window found at speedguide.net. I’m now getting up to 90% + speed on my bittorrent via openvpn. It used to be up to 70%+. Throughput is a lot better
Their guidelines customizes the setting based on your line speed. I haven’t tried the speed on my LAN though. I also set my maxsockbuf to 2x my RWIN.
For the rest of the parameters, I have considered your settings
LikeLike
By: Mark on October 26, 2013
at 6:28 pm