screwed up at work

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

friday - nightmare - i was changing a switchpot to a differant vlan on one of our core switches (as i'd forgot to do it in a change a few weeks ago) - and counted the wrong port number 45 instead of 46 - i ended up screwing the headoffice connection to the WAN - i couldnt believe it....felt like wanted the ground to swallow me up - luckily we got it back, big meeting tommorow to dicuss what went wrong - dont really know what i can say apart from i picked the wrong port - anyone else screwed up badly but genuinely at work and kept their job?

 

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

I think what you're asking is "What are my chances of getting to keep my job?". The answer really depends on how the management sees mistakes, even big ones. It sounds like, in the end, the problem was completely "reverseable", but that doesn't mean no harm was done if it interrupted the normal flow of business, whatever virtual meetings may have been going on at the time, critical information transfers and such.

One of the best pieces of management advice I've ever gotten was to "allow mistakes". That doesn't mean to let them keep going on, but to realize that mistakes happen and, as long as there's a plan for correction in place that is working, not to "fix" the mistake by getting rid of the person who made it. Often a person who makes a mistake and is sincere about not making it again learns and becomes better at their job...not just in the are of the mistake, but by focusing on how they can avoid other potential errors on the job (and after all, it's always possible to goof something up). This is also a good thing to eventually pass along to others as your career progresses. Chances are at some point, you'll be training someone else and you'll have the opportunity to use this situation as a teaching example.

The only advice I can give you is to tell the truth and don't try to palm off the problem. Tell them what you told us...you miscounted. It'll be an uncomfortable meeting for you at best so be prepared to want to squirm and hide under the table (don't actually do that, naturally).

Good luck at the meeting and I hope that it works out in your keeping the job and learning a valuable lesson.

 

Well i think you should use this opportunity to bring in some proper documentational policy, as if you had some then this wouldnt have happened.

We all make mistakes, and some downtime on a network is not the end of the world, but it probably has cost the company money.

Try to get some ideas of how things can be improved (such as labelling the cableing, documentation, backup configs and step by step guides of howtos for things such as this (and i mean idiot guides lol)

Dont beat yourself up too much, im sure youve done lots of good work, and the important things is you will learn from this

 

Trip's spot on... it all depends on the company.

That said... I have one of those stories.

Our company provided hourly support (every Wednesday morning) for a small bank - 3 branch offices. I was the only IT guy there, so any requests (workstation, server, anything) would stack up until I arrived on Wednesday. The bank president would meet with me every Wednesday morning and brief me on anything he wanted me to look at.

One Wednesday, I noticed in the logs that the servers were rebooting in the early morning. I asked the prez if he knew of anyone rebooting the servers, and he said that the bank often loses power for a few seconds each morning. The servers were on a shared APC UPS, so it shouldn't have been rebooting... so I decided to install the APC monitoring software (whoever did support for them prior to me didn't have it installed). I let it collect data for a while, and everything looked OK. I then decided to run a UPS test. After all, what's the worst that could happen, right?

You know that "spinning down" sound that happens when everything in a server room goes down? BWWOOOOOOOOOoooooooooooooooo.............

Uh-oh. What have I done?

The servers that handled ALL the bank's transactions had just gone down... at lunchtime, one of the busiest times of the day.

As the servers are rebooting, I tell the bank president what happened, and what I did to cause it to happen. He said, "No big deal, this has happened before." I'm thinking to myself... "Dude, I just downed your ENTIRE network so you can't handle ANY customers! And it's NO BIG DEAL?" Wow... just wow.

I proceeded to spec out a new APC UPS, which he purchased immediately - I installed it the following week. The batteries in the old UPS were bulging so badly, I couldn't remove them.

Anyway... back to your situation... if you are dealing with rational people, then you'll be OK. But by all means, tell the truth. Trying to sweep what you did under the rug might make things even worse... particularly if they figure out you're not being straight with them.

The previous posters are also correct in that it could have been worse... it might NOT have come back up, or you might NOT have realized what you had done. So don't beat yourself up - what's done is done. And you'll likely be a better tech in the long term for the experience. ;)

 

That's what i was going to say, if mission critical processes are taking place through/on a piece of hardware that needs "maintenance" surely some planing or precaution or backup should have been discussed?

I agree with what's been said don't lie or try to pass the buck if they see through it they come down on you harder. Everyone makes mistakes at some point no matter who you are and good management should realise that and work on how to reduce the chances of it happening again.

I hope it all went okay for you

 

Agree with AndiC.

Documentation, Documentation, Documentation, Documentation, and more Documentation.

 

So has Morph walked the plank then?

 

When I was in the Account Admin dept of our company, we were given rights to create/delete groups. During the course of a cleanup of groups, one of the guys in the servicedesk (he was managing the cleanup) asked me to delete the surplus groups. We accidentally deleted the only one of the groups required to remain. Cue a frantic attempt to retrieve the user list before the delete replicated (it moved fast, let me tell you).

All in all, we got it back, and we reported it straight away. Sure, our ability to delete groups was removed, but thats a consequence of the issue.

On another note, I heard tell of a couple of guys on one of our platforms that perforated the water pipe. Instead of reporting the incident (these things happen, after all), they kept schtum and duct taped the hole. A day or so later... No water for the platform - Cue a mass evacuation of staff until the issue could be resolved and water (drinking water at that) could be restored. Needless to say the culprits were swiftly identified, and heads rolled.

Base line, if you screw up, own up. Management seem to prefer people who can man up to their mistakes. And it puts you in a far better position to maintain your job. Mistakes happen. I sometimes publish some of our apps still pointing to the webservice on my local machine. When I go home at night, the connection suddenly dies.

 

Mistakes are always going to happen at some point, it's how you deal with it that counts (including communicating the problem, fixing it, changing procedures to help avoid a repeat etc)

As others have suggested may be a good time to implement procedures - change management?

Hope it went well!

 

well kept my job, now we dont do anything at all without proper changes, but got to the point that i really feel out of my depth (not related to the simple arse up last week) - i'm doing project work which i'm not 100% on, i got asked last week was i up to speed on the WAN migration...i havent even passed my ccna yet - the experience is invaluable, however coming into work each morning and generally feeling like crap at the end of each day...its not doing me any good - time for a step back to first line unfortunately i think....