Question

I am facing a really bad SharePoint Performance since last week. It is noticeable if someone:

• Creates new Subsites (takes ~ 3min.)
• Edits document properties, press save (takes ~ 1min.)
• Edits lists in Quickedit mode and press save (takes ~ 2min.)
• Adding a Content Type to a library:

Things I did so far:

1. Checked the Browsers Developer Tools (F12) in Network tab.
2. Increased the Distributed Cache Size
3. IIS reset
4. ULS Viewer log live while creating new sub site
5. Developer Dashboard is empty (Usage and Health is running)
6. Deleted the recyle bin content (2. level)

Farm has 1 Application Server (32GB RAM, 4 Cores) and 1 Front-end (16GB RAM, 4 Cores) (here is Distributed Cache running).

SQL Server is not installed via Farm Configuration Wizard. Content Database isn't big. (~5GB)

The CPU and RAM usage is not high and also not increasing while creating or editing. I found those failed jobs:

 

• User Profile Service Application - Feed Cache Repopulation Job
• Delete Job History

   

But that can not be the problem, right?

 

 

Update:

Last week Kaspersky was installed on all servers. I know, not good, but that's the way it is. I stopped the program, but I has no effect. Is this maybe the problem?

Solution:

I uninstalled the Kaspersky Antivirus Software on all SharePoint Servers. Application, Frontend, Office Web Apps and SQL, rebooted all Servers and now I got my performance back.

Also the Developer Dashboard is now working (It happend to be empty (Usage and Health was running)). Thanks for all your help! Also thanks for the link Waqas Sarwar MCSE provided https://support.microsoft.com/en-us/kb/952167. I'll maybe try that if it has to be installed again.

Answer 1

Your update:

Last week Kaspersky was installed on all servers. I know, not good, but that's the way it is. I stopped the program, but I has no effect. Is this maybe the problem?

has rung a bell!

I remembered I had such performance problems with a customer of mine. We disabled the anti-virus, and the problem remained (the same for you). But the problem actually vanished definitely when we uninstalled completely the anti-virus (+reboot the servers).

Answer 2

Antivirus Always cause the issue. MSFT have the guideling regaring the AV. You ave to exclude the Certain folders from the AV scan.

• Hive Folder
• ULS Logs Folder
• IIS virtual Direcotries
• App / Local Fodler for the service account.
• SharePoint Installed Location and more Please check Below KB for complete list of the exclusion.

Certain folders may have to be excluded from antivirus scanning when you use file-level antivirus software in SharePoint

Answer 3

Updated

There could be numerous of things causing this issue. Custom Code, Configuration issues in IIS such as the Application Pool or Firewall settings between servers. Before we do anything else, take backup of the SharePoint databases you care about, such as Content databases, MySite databases, User Profile Databases (if customized), Managed Metadata Service database, Search Service Databases (if you have configured Managed Properties) and all the rest where you know some configuration is done. If you have solutions in your farm, download those as well. If custom branding is performed, save those too.

1. To start I would monitor Central Administration Website from the server it's running on. It should run on the App Server, but sometimes it can be on the Web Front End. It runs on a different web application but is still a SharePoint website. If it has equally poor loading times, you have serious Farm wide errors. But if the Central Admin responds quickly (less then a second) when you have filled your different caches, it's a good sign.

2. Assuming Central Admin is OK, I would create a new Web Application (with a new Application Pool with another App Pool Account than the failing one). Create a new Team Site Site Collection. This rules out any errors with the Web Front End itself. When you hit your brand new Team Site a couple of times, it should also respond less than a second.

3. Assuming your brand new Team Site is OK, we've narrowed it down to your Web Application, which is good. Now here things start to get interesting. Before doing this, make sure you have a backup of the content database. When you do we should copy the failing site collection to our brand new team site, to see if the new web app and site collection works better in its new location. Also make sure you don't crawl this new web app (it would confuse users have the same document in two different locations). Now run Backup-SPSite cmdlet with PowerShell Backup-SPSite http://theFailingSiteURL -Path C:\Backup\site_name.bak and then Restore-SPSite Restore-SPSite http://theNewTeamSiteURL -Path C:\Backup\site_name.bak. You may run into trouble restoring the site, and you may have to delete your new Team Site, but that's OK. Now we don't have any solutions implemented, and we shouldn't at this stage. Test your restored site and see what you got. Responding in less than a second after a few trials? If that's the case, you know you have errors in your old webapp or your custom solutions.

4. Deploy the solutions and activate features the same way as your old webapp. Monitor traffic and see if you have issues or not. If you're OK here in your new web app, you need to plan for a switch to the new webapp.

Let me know the result when you get this far, and make sure you keep a close eye on the ULS logs and Event Viewer.

Outdated Answer

In these cases I'd recommend looking at the SQL Server. What's the memory utilization showing when you create a sub site? Do you have long running transactions (see SQL Profiler). Is you LOG-disk full (or almost full)?

Answer 4

I read that you increased the Distributed Cache size. I had a performance drop and the solution was to STOP/START the Distributed Cache on the Service on Server page in CA. Maybe it will work for you.

Answer 5

What are the developer tools (F12) telling you? What is taking so much to load? If images as an example are loading super fast but htmls have a long TTFB it is your app server and not front server issue, please give out more information.

How do you configure the crawls? What are the servers telling you? (Performance wise, track this).

Looks like your structure and the amount of queries is what is diminishing your farm performance, probably structured navigation is also adding due to the complex subsite structure and constant crawling per site load.

Please complement your question, it is very vague and you say "done this done that" but you don't show any results.