Exchange 2007 PF oddness

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Hi All,

Some of you maybe aware that I am in a new position, and as such exchange is new to me coming from a groupwise background mostly.
I have been told that our exchange server slowly chews up the page file until after a week or two the server becomes unresponsive. I have read a few blogs and articles on how the pagefile should be set and ram +10 and have process monitored it which is suggesting that of the 23GB's of PF usage 20 of those is being taken up by the store process.

I could I guess schedule a regular overnight restart of the store process but this is mearly circumventing the issue rather than curing it. The server has 32GB of ram installed and the PF is currently system managed at 32+10.

Anyone got any suggestions?!

 

Oh man! Ad-blocking software has been detected! :'(

This website is run by the community, for the community... and it needs advertisements in order to keep running. Blocking our ads means your killing our stats!
Please disable your ad-block, or become a premium member to hide all advertisements and this notice.

Seeing as you have read up on page file sizing on exchange servers and recommendation have you implemented this yet? Obviously, this is taking into account that you've inherited this server and the changes were made then etc.

On the other hand just as a side note check your log files are not filling up and there's enough space for it to grow if need be.

I would also check your backup schedules to make sure it runs effectively as if it runs out of tapes this can impact on the log files growing on the mail server.

 

Growth of Exchange log files has zero effect on pagefile usage. If backups are not made regularly then the number of logfiles will increase - but how would that explain why the server is paging out to disk?

Exchange RAM can get clobbered by a number of things. You might wonder at first why store.exe is grabbing a huge amount of RAM, but it's actually just using whatever it can at the time it needs it and, unlike a number of other processes, is very good about releasing that back when it isn't needed. SQL Server does the same thing.

Things to check are:

An antivirus scanner running on the server (ScanMail in particular is an absolute hound for this)
What clients are connecting (if you've set up ActiveSync and have iPhone users for instance, they can pulverise your Exchange server - as can Mac users running Entourage) - use ExMon to see what kind of clients are connecting
Whether online maintenance is completing every night
Has the PAE setting been set (search 'Exchange'+'PAE' in Google to find the kb article explaining this)
Whitespace in your storage groups (look in the event viewer for eventid 1221 to see how much free space there is in the DB - if it's enormous - and it might be if a load of users have been deleted - you may want to consider an offline defrag)
Do you have a journalling mailbox set up? A lot of law firms do this - and these get absolutely bloody huge. Run Pfdavadmin and take a look at the item counts in your mailboxes - any system folder (inbox, sent items, deleted items etc) in a mailbox with over 5000 items can really hurt performance
Use ExMon to monitor the performance of your server. It's a cracking little tool, and there is lots of documentation around on the net about it

If you find that the rudimentary steps early on in the troubleshooting detailed above don't help, let me know and I'll try and dig out detailed guides for usage of exmon and pfdavadmin

 

Zeb, whilst I respect you as an individual I do however, find your attacks and comments on my replies some what malicious and childish.

I did state in my post reply the above comments as an elaboration as well as touching on areas that the OP would find helpful. This is a free forum and hence, keep your so called expert opinions to your self.

I can swallow my pride and put my hands up when constructively critic. Please, pretty please, act your age and not your shoe size.

 

They are not 'attacks'. Neither are they malicious or childish. If you post information that is pointless and/or inaccurate, you should expect to get pulled on it - the same way as anybody else on the forum would. The OP asked for advice on a specific issue - he didn't ask for an Exchange Best practices guide. I don't want to get into a public slanging match with you - so will not respond to the insult at the end of your post

 

Just out of curiosity, was your Exchange 2007 environment set up after consulting the storage calculator? If you were to run the calculator spreadsheet with todays numbers, would your server still be specced adequately? And have you run ExBPA (a very obvious first choice before consulting with, say Microsoft PSS)?

Just curious to hear what those two things would tell you now.

 

Glad to see that you've grown up, bravo

 

Stop trolling - it doesn't become you

 

Onoski, you have my respect... but I don't see anywhere in Zeb's post where he maliciously or childishly attacks you. He simply disagrees with your assessment, and he gives his reasoning why.

Just my opinion.

 

I too don't see any attack in Zeb's post. The problem might be in the fact that you can't express tone of your voice on the forum, hence why Zeb's post could sound like an attack. Onoski, cheer up mate. I too respect your presence on this forum.

Many times I've had the message written down and was about to click Submit and in the end decided not to, because my opinion could be inaccurate or I wasn't sure of my technical understanding of the subject.

But as has been mentioned:

... happened to me before

 

Thanks for the replies...., I've had a play with exmon which is producing some interesting results some rpc sucseeds are reported as 71 or higher. We have about 4 Ipads so far and there are about 70 mailboxes in total about 25 of those are over a gig in size some nearly 4 and way more than 5000 items. I am thinking about suggesting implimenting a mailbox size warning (no limit yet) just to see if that spurs any action from the users. The single storage group is hosted on a seperate drive so there should not be too much clashing of read write actions going on.

I'm quite surprised that they have a 32Gb exchange server for that many users, I would expect that amount of speccing to easily accomodate thousands of users seeing as my old groupwise server ran on 4gb's and we had 400 mailboxes without any issues at all. Still exchange is much easier to manage!

 

I would always start by looking at your AV/Anti Spam solution first, 9/10 it's those badboys that cause you dramas.

Another area, that I always consider if your RPC requests are sitting quite high, is taking a look at the number of items in peoples Calendars. Reoccouring events synching with iPhones and other mobile devices can cause havoc.

Run the Exchange BPA which is in the inbuilt 'Toolbox' in the EMC and then also run the Exchange Performance Analyser as the performance counters on this will normally point you in the right direction.

 

thanks for the hints everyone, I am getting somewhere slowly with it. the corporate virus product (avg pro, oh dear), wasn't excluding some of the recommended directories. There is also an addon called office adviser which probably isnt helping. Other things is that most users connect by rdp to ts servers for their desktop and judging by what I've found when I got there monday I would not be surprised if office wasn't installed in ts mode. There are 3 ts servers so I might recommend ripping one up and starting again and seeing what happens.

The page file thing is starting to make sense a little as its still at 23.7 gb but the actual usage is much smaller so I think its just naturally expanding to fit a similar size to the ram.

 

70 Users and 32 gigs of ram? Sounds like your server is scaled for way more than this. Off the top of my head, depending on the average mail size, you could run several thousand mailboxes on this server.

doesn't sound right... Really, do run ExBPA and do a new count with the exchange storage calculator. If you need some help, pm me and we can look at some of these numbers.

Depending on what features are enabled (owa , ActiveSync etc), we may see IOPS increase on your CAS servers, but this still sounds like you've got a memory leak somewhere.

In one company I worked for, an antivirus product in one branch caused loads of grief due to a version which had a known bug, so this too could be something to look into, but I'd need to know a little more about your environment to know for sure (if it has been specced accordingly).

This definitely shouldn't be happening Tho. By any chance, are you running multiple roles on this server, I.e. CAS and Hub as well?

 

This.

As per usual, I completely missed the important part - where you mentioned that you have 32Gb of RAM for 70 users. As an example, my Exchange server is a VM, houses 178 mailboxes, total DB size c.100Gb. It runs on 4Gb of RAM, like a dream.

Something is seriously wrong somewhere! What disk configuration is in this server? (speed, raid level etc). Like Shini says - check out the IOPS on that bad boy - there's some good stuff on sizing Exchange storage here - and a very good blog post somewhere on the You Had Me AT EHLO blog (you'll have to search for it on there as I don't have the link to hand)

 

Thanks zeb et all, I am now thinking that it may be a client issue due to being given crap information when I turned up on Monday. I think the server does need the occasional reboot but I dont think it is having any issues, I've been running exmon and exchange perfmon, which shows nothing more than unsually high RPC count. As I say I'm quite tempted to rebuild a TS server and see if that makes any difference. They appear to have a few problems with something called office adviser which plugs into outlook which again I guess could be an issue.

I'll try looking into client side issues and see what happens I think, Would be nice to get this sorted as it would be a welcome tick for me in the probationary period.