Intrinium Information Technology Solutions Mon, 16 Mar 2015 22:43:25 +0000 en-US hourly 1 Windows Event Log Error: w3wp.exe shutting down unexpectedly Sat, 15 Nov 2014 03:06:16 +0000 As an IT consultant, it is always healthy to do a periodic review of client servers to prevent unexpected issues from arising. During a recent review of a client’s server, errors were found relating to w3wp.exe shutting down constantly. “w3wp.exe” is the process that runs an individual application pool in IIS. This particular client runs an internal website which is business critical. Seeing this event, raised my concern that the application pool was not functioning properly, and we were in for potential issues in the near future.

My first goal was to determine that this process did indeed belong to the business line application website. I opened task manager to determine if multiple w3wp processes were running on the server. Two different processes were found. The next step was to determine which process was shutting down all the time. Logging showed the process was shutting down around every 30 minutes to an hour. Careful monitoring determined that one of the processes stayed active as the memory usage hovered constantly around 100MB, and the other process would drop very low or disappear. I needed to confirm which application pool the active w3wp process belonged to. In order to do this, you will need the PID. If it is not shown by default in task manager, you can display it by going to View – Select Options – Check PID.

Having the PID will allow us to run a command line argument to determine which application pool the process belongs to. Depending on the operating system on the server and IIS version, you will need to enter different commands. First open a command prompt as administrator.

For Windows Server 2003/2003 R2:


For Windows Server 2008/2008 R2/2012 with IIS7:

%SystemRoot%\System32\inetsrv\appcmd list wp


You will receive a list of application pool names with their PIDs next to them. From here open IIS to view your websites. Under the properties of the website, the application pool associated with the site can be found. I was able to determine that the application pool that was staying active was for the business critical website, and the other application pool that was shutting down was the default application pool. Crisis averted.

With the worry dissipating, I wanted to determine why the default app pool was stopping and creating errors. If the application pool was not being used by the server, the simplest answer was to just disable the website and move on. This particular server was using the default website for other functions therefore troubleshooting began. First option was to look over the event logs for any other related errors at the time the application pool shutdown. At times of shutdown, no other correlated events were found. Taking a look at the times of alerts, showed that the events had a pattern. As stated above, it was consistently shutting down almost every 30 minutes. What if this wasn’t shutting down due to a crash? Some quick searched showed that there are options to have application pools shut themselves down due to being idle for certain periods of time. This is primarily to save resources on your server.

Idle time settings can be found by opening IIS, expand the server you are connected to and selecting Application Pools. You will need to find your application pool from the list and select properties or advance settings depending on version on IIS7. In the list of properties, you will find Idle Time-out field. Next to it will indicate the time. For our application pool, we found it was set to 20 minutes, the typical default value. We increased this threshold to two hours as we wanted to alleviate some of the error messages.

After update, the error logs have virtually no further messages for application pool shutdown. This helped clean up the event logs so we could see other potential issues if they arise. Without regular review of the client’s server, we would have never found the potential issue. Luckily the error turned out to be nothing major however finding it did help with troubleshooting in the future. Now event logs are cleaner without this message propagating every 30 minutes. Seeing this error could have lead us on a different path costing hours of troubleshooting.

]]> 0
Exchange Litigation Hold Fri, 25 Jul 2014 00:00:00 +0000 Once in a while you may be involved in a situation where you need to preserve E-mails and keep them in an unaltered state.  The reasons may vary but the nice thing is that you can easily configure a Litigation Hold within Exchange server to accomplish this.  Litigation holds can be set on a per mailbox basis for a given amount of time and can be configured a number of ways.  If you are in need of placing all mailboxes on litigation hold you can use the following Exchange PowerShell command:

Get-mailbox | Set-Mailbox -LitigationHoldEnabled $true

This will go through all mailboxes and enable the litigation hold.  You can also verify the command was successful by selecting Recipient Configuration, select a mailbox, right click and select properties, click Mailbox Settings, Click Messaging Records Management and selected Properties and you should see a check in the “Enable Litigation Hold” box.

This is a very easy way to place a hold on all mailboxes should you need to do so.

]]> 0
Adventures Managing User Permissions in Exchange 2013 and Outlook 2013 Wed, 02 Apr 2014 00:00:00 +0000 A few months back, I noticed that a user was unable to access a shared mailbox because Outlook needed a password for it. The computer had been setup with entirely separate Exchange accounts instead of adding permissions for the user to add the mailbox to their own account. I decided to change this and had a lot more trouble than I would have expected.

Before removing the extra account in Outlook, I went to add the required permissions. Exchange 2013 removed the GUI interface for managing Full and Send As permissions, so I ran the following command:

# Add-MailboxPermission

When I went back to Outlook, The accounts had been duplicated and despite removing the accounts that were previously setup, I would get errors when trying to open one of the two duplicates. Even creating a new Outlook profile didn’t make this go away. After some more research, I found that there is a parameter that can be added to the Add-MailboxPermission command to turn Auto Mapping off and Exchange will automatically add the account to your Outlook Account. While Auto Mapping is great, it was causing all kinds of problems with this account. I had to the Remove-MailboxPermission and then re-add the permissions with Auto Mapping set to false. After that everything worked again.

]]> 0
What is Change Management and why should I care? Tue, 25 Mar 2014 00:00:00 +0000 What is Change Management and why should I care?

The more or less textbook definition of Change Management is: the processes, tools, people and the techniques to effectively manage change to achieve a required business outcome. What does this mean exactly? Well to some extent that depends on who you talk to. It might mean one thing to an Executive, and something completely different to one of the grunts on the pointy end of the stick who has to do the actual work.

Let’s break this down into more manageable chunks and take a look at the respective pieces. In that context the first question is “What is a change?”.

A change is an event that is:

·         Approved by management

·         Implemented with  minimal and accepted risk to existing IT infrastructure

·         Provides increased value to the business from the use of the new or enhanced IT systemssoftwareinfrastructure

Change must be realistic, achievable and measurable. Everything can be measured with metrics of some sort, be it increased performance, reduced costs, reduction of FTE staff, etc… Of course negative impacts can also be measured, and should be, as part of the change management process.

Changes in the IT infrastructure may be due to reactive responses to problems or externally imposed requirements, or proactively from seeking  to improve efficiency and to enable business initiatives. Change is the one global constant within the IT world, and you can either manage it, or let it manage you.

Change management is responsible for any process involving:

·         Hardware

·         System software

·         All documentation and procedures associated with the running, support and maintenance of live systems.

As you can see this is a very broad-spread spectrum of items to take into consideration. Everything from IP phones to desktops/laptops to servers, switches, routers, etc, are covered. How on earth does someone handle this in any sort of systematic and logical fashion? The answer, of course, is to follow the change management process.

What is the process?

Any proposed change to the environment must be approved in the change management process. While change management makes the process happen, the decision authority is the Change Advisory Board (CAB), which generally is composed of people from multiple departments/functions within the organization.

The main activities of the CAB are:

·         Filtering changes

·         Managing changes and the change process

·         Chairing the CAB and the CAB/Emergency committee

·         Reviewing and closing of Requests for Change (RFCs)

·         Management reporting and providing management information

Choose the CAB membership wisely. The members of this committee can either streamline the change process, or hopelessly bog it down in minutia and trivial bureaucracy. Many organizations that implement change management for the first time make the mistake of locking things down so tightly that getting any RFC through committee becomes a nightmare. Kingdom building is always a danger with such an entity and is to be discouraged at all costs.

Yes, there needs to be controls put in place. And yes, loose cannons are not to be tolerated. However, there needs to be a balance that accommodates both good process and controls, and the ability to actually get work done. There will always be those who chafe at any imposition of a new procedure that “interferes” with implementing changes to systems. The challenge is to show them that having a structured change control process doesn’t mean having to wait three weeks to swap out the power supply on a server.

From a management perspective, good change control provides visibility and accountability into the day-to-day operations of the various departments in IT. It also should minimize, or mitigate, any downtime associated with changes to the corporate environment. Additionally, it provides a better method for Emergency RFC approvals than the traditional “call around till you find someone who will approve it” approach.

As you can see there is much to take into consideration when looking at change control, much more than I can cover in this short blog. Many good resources are available both on the Internet and in physical book format. I encourage you to take the time to familiarize yourself with at least some of this material before instituting a new Change Control into your environment, or deciding how to fine-tune an existing committee. It is certainly time well spent.

]]> 0
Listener.ora update needed after hostname change Fri, 07 Mar 2014 00:00:00 +0000 As more systems are integrated with Windows Operating Systems, users run into new scenarios to consider every time a change to the environment is made. One issue, I ran into the other day, was the need to update the listener.ora config. We recently demoted a DC and moved the device onto a new domain. A couple of days later, users started to notice that they could not connect to one of their production servers running an Oracle database. Troubleshooting pointed to the OracleXETNSListener not being started. This realization came from the errors received by the product stating the port was unavailable. Attempts to restart the OracleXETNSListener failed thus investigation into this service issue began.

Windows event logs did not provide any assistance as to why the service continued to stop. The log files for OracleXETNSListener, indicated that

]]> 0
FortiGate Interface Speed Thu, 06 Mar 2014 00:00:00 +0000 Usually when setting up a new firewall, the interface speeds, specifically the WAN interface speed is set to “auto” and will attempt to negotiate the speed of the connection.  Most of the time this is OK and you don’t have to adjust this setting and the connection will work fine.  Recently, I ran into an issue where the Internet speeds were very slow, and users could not get to a lot of sites necessary to complete their work.

After some troubleshooting, I found that the ISP needed the connection statically set at 10/Full to work with the service they were delivering.  After poking around on the FortiGate I saw that the WAN1 interface setting was set to “auto” by running this command:

#show system interface wan1

After confirming the speed that needed to be set on this interface to work with the ISP I used the commands:

#config system interface

#edit wan1

#set speed 10full


This immediately resolved the slow Internet speed issues.  Next time you have symptoms similar to what is described above, check the interface speed.

]]> 0
Interlink & Intrinium Merge Tue, 31 Dec 2013 23:02:26 +0000 We are pleased to announce that Interlink and Intrinium, another local technology company, have merged. Although we could have chosen either name, after much thought and deliberation, we decided to move forward under the Intrinium name. The Intrinium name is unique, and that matters when you have a strong national brand.

Our staff and locations are not changing. We will have new members of the team, but the people that you know will still be here to serve your needs. You’ll still be able to reach us in the manner which you are comfortable, and get the high-quality services you are accustomed to.

This merger brings new capabilities to Interlink including security management, more resources for your projects or potential IT emergencies, and more expertise in more products. We will also have a fully staffed Network Operations Center. With our combined staff, you are now working with one of the most accredited and certified providers in the Northwest, with more services offerings than ever before!

Kirt Runolfson, Founder and President of Interlink, and Nolan Garrett, Founder and Chief Architect of Intrinium, will be running Intrinium as Founders. We are both very excited to continue to grow Intrinium and expand and improve our services.


]]> 0
NPS Policy Failure Tue, 10 Dec 2013 00:00:00 +0000 Many companies use Windows Server Network Policy Server to control network access and authentication.  This works really well when you want to deploy secure wireless networks and use group policy to ensure that your wireless clients always connect securely to the networks you choose.  You can create various policies, for example, to use a RADUIS server to authenticate wireless clients so that users seamlessly connect.  Of course with these types of deployments you can almost always expect issues where the authentication does not work properly and will require troubleshooting.

I recently ran into an issue with a NPS policy that controlled wireless access for domain computers where the authentication was failing on the wireless NPS policy I had created.  After looking through the event logs, I found the following error:

“The message received was unexpected or badly formatted”

The specific event ID was 266.

This was really the only error I had to work with and after checking certificates etc. and making sure group policy was correct I was stumped.  After further research I found that the resolution was to add a specific registry key by doing the following:

Open regedit to the following key:


Create a new DWORD value SendTrustedIssuerList and set it to 0 (false)

Once I did this, the wireless client was able to connect successfully.  Basically, this registry key will prevent NPS from sending the trusted root certificates to clients.  Essentially there were too many root certificates being sent causing the error.  If you are running into a similar issues try this fix as there were no specific event errors pointing to this resolution.

]]> 0
Restarting FortiGate Services Mon, 02 Dec 2013 00:00:00 +0000 Recently we experienced an issue with a FortiGate firewall where you could not access the GUI using the management IP address although it had been working without issues previously.  SSH/Ping/Etc. were still functional but the GUI did not respond when trying to open the management page to log in.  Additionally the configuration had not changed nor had the device experienced any issues that required reloading the configuration.  After some research, the fix (if rebooting is not an option) is to access the device using SSH, login as admin, then execute the following commands:

# config global 

# get sys perf top – This will display all the running processes in the FortiGate (the second column is the process ID’s) note the ones you want to restart.  In our case it was the two “httpsd” processes.

# end

# diag sys kill 11 <process-id> – Using the process ID from above you can restart a process using this command.

After running these commands, the GUI was then accessible again.  Nice to know how to view the running processes for instances like this or resolving other issues.

]]> 0
FortiGate Debug Commands Fri, 22 Nov 2013 00:00:00 +0000 Quite often I have to use the CLI interface on FortiGate firewalls to troubleshoot traffic connections, VPNs, etc.  Using the built in CLI is very useful and powerful tool to isolate issues and resolve very quickly rather than pouring through traffic logs using the web interface.  The following are a list of common commands to be able to troubleshoot:

“diag debug enable” – This will enable debug logging

“diag debug disable” – This will turn off debug logging

“diag debug reset” – This will reset the debug logging

“diag debug flow show console enable” – This will output the debug logs to the CLI screen so they can been seen

“diag debug flow filter addr <IP address>” – This will show the flow of traffic from a particular IP address

“diag debug flow filter clear” – This will clear the logs for any flow filter debug command

Let’s say you wanted to see if a particular node was sending pings successfully on any interface:

“diag sniffer packet any ‘icmp and host x.x.x.x’ 4” – If pings are successfully hitting the appropriate interface, you will see the output on the CLI console

These are some basic commands that of course can be adjusted for certain troubleshooting needs but are essential to be able to turn on logging and see what is going on.  For more FortiGate specific CLI commands, go here:

]]> 0