1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Might be a Bone Head Simple Question But...

Discussion in 'General Cisco Certifications' started by NetEyeBall, May 31, 2007.

  1. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Here is the scenario...I have a server that loses connectivity for a couple seconds about 20 times a night between the hours of 2000 to 2100. The switch log shows where the link comes back up. Never shows it goes down though. I feel it is a utilization issue.

    How does a switch know that a device is connected? Does it send keep alives by broadcasting down that port? Its gotta be a broadcasted keep alive so that the switch knows there is something alive down that port. Gotta find an article on this...grrrrr
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  2. hbroomhall

    hbroomhall Petabyte Poster Gold Member

    6,623
    115
    224
    What does it actualy show when the link comes back up? This might give you a clue as to which machine is at fault here.

    What about the server logs?

    Harry.
     
    Certifications: ECDL A+ Network+ i-Net+
    WIP: Server+
  3. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Oh I got the server. Nothing in the server logs actually. Well there was a dns issue, but that was corrected and the issue still is occuring. Only impacting 1 machine.

    Regardless the dns issue was minor and didn't impact connectivity. The server logs don't show any connectivity issue that I can see.

    May 29 21:01:57: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:02:39: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:03:05: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:03:33: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:04:21: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:04:41: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 29 21:05:05: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:35:57: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:36:27: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:36:55: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:38:11: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:44:23: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:45:09: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:46:01: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:46:39: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:47:59: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:48:41: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:49:05: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:49:47: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:50:07: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:54:47: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:55:15: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:55:41: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:56:25: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:57:45: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:58:31: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:58:55: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 20:59:15: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 21:00:07: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up
    May 30 21:01:09: %LINK-3-UPDOWN: Interface FastEthernet0/11, changed state to up

    I found a suggestion from Cisco about making sure there is no power management on the PC/Server or NIC. Also you can set up a setting against jitter, but this is the only server in our network so far that I have ever found this issue with as of yet.

    Current configuration : 192 bytes
    !
    interface FastEthernet0/11
    description ENTRIISBMC01 (NADCWPWEBBMC02)
    switchport access vlan 293
    switchport mode access
    no ip address
    duplex full
    speed 100
    spanning-tree portfast
    end

    I already replace the cable and moved the port from f0/6 to f0/11. I should have moved it to a different switch fabric but I didn't think of it at the time and need to research more into that anyways...

    Thanks for the input!!!
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  4. hbroomhall

    hbroomhall Petabyte Poster Gold Member

    6,623
    115
    224
    First I'll say that I am *definitely* no Cisco expert!

    A possibility that comes to mind is that the NIC on the server is marginal with the switch, and at the times given something is running somewhere that just takes it further out.

    One thing to try which I'm always being told about by Cisco people is to not let the switch and server NIC be 'auto' but nail-up the correct info such as speed and duplex.

    Harry.
     
    Certifications: ECDL A+ Network+ i-Net+
    WIP: Server+
  5. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Auto-neg is definately not the best idea.

    duplex full
    speed 100

    Takes care of that issue though.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  6. hbroomhall

    hbroomhall Petabyte Poster Gold Member

    6,623
    115
    224
    Indeed - I saw that on the config you posted. But what about doing the same on the NIC on the server?

    Harry.
     
    Certifications: ECDL A+ Network+ i-Net+
    WIP: Server+
  7. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Good point. I don't have access to check the nic config on theserver. I can see the logs, but not the nic config. I bet I could do it from the command line though. ipconfig /all should tell me if it is auto or full...will have to check.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  8. Tinus1959

    Tinus1959 Gigabyte Poster

    1,539
    42
    106
    Then again, why does it only loose connection between 2000 and 2100 hour? That is if I read correct.
    The config for the NIC surely does not change for that specific hour.
     
    Certifications: See my signature
    WIP: MCSD, MCAD, CCNA, CCNP
  9. hbroomhall

    hbroomhall Petabyte Poster Gold Member

    6,623
    115
    224
    I was assuming some system load caused a flaky NIC to dip out.

    Perhaps the best thing to do is fit a known good card (like the Intel Pro series) and use that instead.

    Harry.
     
    Certifications: ECDL A+ Network+ i-Net+
    WIP: Server+
  10. zebulebu

    zebulebu Terabyte Poster

    3,748
    330
    187
    Since it happens every day at the same time, it has to be an issue with something running on the server causing over-utilisation.

    Its not something daft like a backup running across the network is it? Can you run a trace on the server and see whats happening at those times?
     
    Certifications: A few
    WIP: None - f*** 'em
  11. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    I do believe it is a Network Backup on our TSM system. The system runs a backup. However, I still wonder why the link drops. My conjecture is that the ethernet circuit uses some kind of broadcast keep alives to keep the circuit up. But I never read anything before on it. I know between Cisco Devices it sends a multicast keep alive (CDP). I would have to review the specifics so I might be a little off...please no flaming...I am on holiday tonight. :blink
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  12. Headache

    Headache Gigabyte Poster

    1,092
    9
    85
    I don't know whether the problem has anything to do with multicast or not. But one way to check is to examine output from the following:

    show ip mroute summary
    show ip mroute
    show ip mroute active
    show ip mroute count
     
    Certifications: CCNA
    WIP: CCNP
  13. r.h.lee

    r.h.lee Gigabyte Poster

    1,011
    52
    105
    NetEyeBall,

    Questions:
    1. How many physical ethernet cables are there between the switch and the server?
    2. What kind of server is connected to the switch?
     
    Certifications: MCSE, MCP+I, MCP, CCNA, A+
    WIP: CCDA
  14. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Should be a single span which like I said, we replaced and switched ports. Not that a physical issue still might be there, but it is odd that it is only happening between 2000 to 2100 hours approximately.

    It is a Windows 2k server of some variant.

    I opened a TAC case to get more info as to why we aren’t seeing a down alert, just an up alert in the log. Also server people are putting some perf logs in place.

    Will keep you updated. Thanks for the tips and suggestions thus far!
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  15. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Cisco TAC response thus far about the Up statement showing in the log and not the down statement...

    "Regarding the error message on the 3550 that says the link came up (and just up), it actually a bug in the older code. The link goes down for such a short period of time, that it only logs the 'link up' message. The fix / workaround for this bug is to use a different method of detecting link down when autonegotiation is enabled. The fix is to upgrade to 12.1(14)EA1 or above."

    So that answers one question. TAC is still researching the rest and my server guys are looking at the server.

    My gut is heavily leaning to a server side issue.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  16. r.h.lee

    r.h.lee Gigabyte Poster

    1,011
    52
    105
    NetEyeBall,

    Since I'm currently studying for the 640-861 DESGN exam and Cisco lists a prerequisite for the CCDA certificate of "CCNA level knowledge and BCMSN level knowledge is needed to prepare for the CCDA certification exam," I decided to get myself the Cisco Press Building Cisco Multilayer Switched Networks (BCMSN) (Authorized Self-Study Guide), 4th Edition book.

    I read that portfast is associated with STP, in particular PVST+. Under default conditions, STP assumes that all switch ports are to be part of the STP process unless configured otherwise. An example of "configured otherwise" is portfast. Basically, if you configure a switch port for portfast, instead of the four STP states of blocking, listening, learning, forwarding there are only two, blocking and forwarding. The purpose of portfast is that the network engineer/administrator knows that a particular switch port will terminate in a non-STP networking device such as a server. Since the portfast switch port leads to a network endpoint, the chances of switching loops is 0. Therefore, the portfast switch port is ignored from the STP process.

    As you should know as a CCNA certified individual, that STP runs the whole election for root bridge/switch, election for designated bridge/switches, then switch ports eventually settle down to either forwarding or blocking mode. So based on the "flip flop" state of the portfast switch port from down, to up, to down, to up, it seems to suggest that STP is being run that causes the porfast switch port to go from up to down, or in STP terminology forwarding to blocking. I'm guessing STP has converged permitting the portfast switch port to go from blocking directly to forwarding since it was configured for portfast.

    So, in order to cover yourself from a networking standpoint, verify that all switch ports in the same VLAN as the server are configured to be "portfast" also. That way, any unknown/authorized network change, such as plugging servers into different switch ports, will clearly be the fault of the server guys and not the network guys.

    Another thing to check is the STP and/or VTP structure. Maybe the STP flip-flop state may be due to an absence of a designed VTP Server, VTP Transparent, and VTP Client structure as well as configuration of the switch priority values to influence the STP election process so that there is a STP structure of one root bridge/switch, planned designated bridge/switches, and the rest of the switches are physically in an extended star configuration relative to the planned designated bridge/switches.

    Also, based on the TAC's response of "...The fix / workaround for this bug is to use a different method of detecting link down when autonegotiation is enabled..." suggests that you should specifically configure both the duplex and speed settings for the switch port leading to a server instead of relying on autonegotiation. It's kinda like the hazard of configuring trunk ports on opposite sides of a trunk to "auto" because side 1's configuration depends on side 2 and side 2 is waiting for side 1 to make up their mind. So by explicitly configuring the switch port speed and duplex, if the server's interface is configured for autonegotiation, then the server's interface should autonegotiate quickly to the switch's configured duplex and speed settings.

    I hope this helps.

    Source:
    1. CCDA-Career Certifications & Paths - Cisco Systems - http://www.cisco.com/web/learning/le3/le2/le0/le4/learning_certification_type_home.html
    2. Building Cisco Multilayer Switched Networks (BCMSN) (Authorized Self-Study Guide), 4th Edition - http://www.ciscopress.com/bookstore/product.asp?isbn=1587052733&rl=1
     
    Certifications: MCSE, MCP+I, MCP, CCNA, A+
    WIP: CCDA
  17. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    Thanks for the suggestions!

    The port is not set for auto-negotiation. Not sure why TAC thought that. The port is set for 100 full.

    I will check out the other point though.

    The server guys are swapping NICs. It is possible that the NIC is flaking when the traffic is reaching near maximum.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  18. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    I was thinking on what you suggested. Wouldn't a spanning tree recalc be like 20 seconds at least to move from blocking to forwarding? I will have to look at it a bit more in depth.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall
  19. r.h.lee

    r.h.lee Gigabyte Poster

    1,011
    52
    105
    NetEyeBall,

    The purpose of portfast is to skip the listening and learning steps of STP. Portfast is designed to go to an end point, like a server or workstation. Since the server or workstation isn't going to send STP BPDUs to the switch port it is connected to, the switch is not likely to "hear" anything to "listen" to, therefore it would have no new data to "learn" from. That's why portfast is designed to go from blocking, skip listening, skip learning, then to forwarding. The obvious benefit here is that with less steps, the STP convergence process is quicker than the time required for a complete STP cycle of blocking, listening, learning, and forwarding.
     
    Certifications: MCSE, MCP+I, MCP, CCNA, A+
    WIP: CCDA
  20. NetEyeBall

    NetEyeBall Kilobyte Poster

    279
    10
    45
    OK. We swapped the NIC. Still occuring at the same time. I have been feeling it has been due to network activity. So I started looking at the keep alive command. Tonight I upped the default keep alive to 30 seconds (from 10). The port sends keep alives to detect a down circuit. See the following command explination:

    http://www.cisco.com/univercd/cc/td...os122/122cgcr/finter_r/irfinter.htm#wp1017769

    We shall see if this helps or not.
     
    Certifications: CCNA, A+, N+, MCSE 4.0, CCA
    WIP: CCDA, CCNP, Cisco Firewall

Share This Page

Loading...