Monday, November 14, 2011

Exchange 2010 DAG transaction logs are not truncated after VSS Full backup

A couple of months ago we implemented for a customer a hosting solution based on Exchange 2010. The solution has full redundancy for all components including a DAG with geographical redundancy.
Everything was configured accordingly to Microsoft recommendations and worked without any problems.

We configured Windows Server Backup to perform backups using a DAG member that was hosting only active databases. This means that we didn't have to do the Registry modifications as mentioned in TechNet (http://technet.microsoft.com/en-us/library/dd876851.aspx).
The backup was successful but not logs were deleted from the log's folder. In the Event Viewer the only relevant record was:

Information Store (2672) DB1: No log files can be truncated.

Exchange VSS Writer (instance acedea2e-c785-3e33-8257-ac38405e81af:5) has successfully completed the full or incremental backup of replicated database 'DB1'. The log files will be truncated after they have been replayed.

We tried to find a solution and nothing worked and we had to give up for a while in using DAG replication. Microsoft support has extensively investigated the issue for more than two months without success. In the end their suggestion was that between the geographically dispersed members of the DAG there was something that was blocking the traffic.

We discussed with the network infrastructure admin and all the ports seems to be open. The only thing that could block the communication was a Network Intrusion Detection System. This system was blocking SUNRPC and DCERPC protocols. After we disabled this protection and run the backup, all database logs were truncated.

If you encounter this issue, try to disable everything that could potentially block the communications between DAG members.

Sunday, November 13, 2011

Unable to authenticate users in Office 365 using ADFS services

Last week we migrated one of our clients to Office 365 including the configuration of ADFS services in order to authenticate the users using local Active Directory credentials. Everything worked as expected for a couple of days until one morning when nobody could authenticate to the service.
We investigated the ADFS server and we found out that the AD FS 2.0 Windows Service was stopped.
We tried to start it without success. At a deeper inspection we discovered that the account that should be used for running this account was deleted from AD.
We created a new account, assigned all the appropriate rights and we managed to start the service. At this point, the external users that were authenticated using the ADFS Proxy server were able to login to the Office 365 services but the internal users were unable to do so.
In the AD FS 2.0/Admin event viewer log there were a lot of events like:

Log Name: AD FS 2.0/AdminSource: AD FS 2.0
Date:          11/8/2011 10:15:12 AM
Event ID: 323Task Category: None
Level: ErrorKeywords: AD FS
User: DOMAIN\ServiceAccount
Computer:     COMPUTER.domain.com
Description:
The Federation Service could not authorize token issuance for the caller 'DOMAIN\ServiceAccount' on behalf of the subject 'user@domain.com
' to the relying party 'urn:federation:MicrosoftOnline'
. Please see event 501 with the same instance id for caller identity. Please see event 502 with the same instance id for OnBehalfOf identity, if any.
Additional Data
Instance id: 2ae34f5b-dc27-3ed1-af21-976e28a4e3f5
Exception details:
Microsoft.IdentityServer.Service.IssuancePipeline.OnBehalfOfAuthorizationException: MSIS5009: The impersonation authorization failed for caller identity and delegate for relying party trust https://claimapp1.treyresearch.net. at Microsoft.IdentityModel.Threading.AsyncResult.End(IAsyncResult result)
at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustServiceContract.ProcessCoreAsyncResult.End(IAsyncResult ar)
at Microsoft.IdentityModel.Protocols.WSTrust.WSTrustServiceContract.EndProcessCore(IAsyncResult ar, String requestAction, String responseAction, String trustNamespace)
User Action
Use Windows PowerShell comments for AD FS 2.0 to ensure that the caller is authorized on behalf of the subject to the relying party.

Log Name: AD FS 2.0/AdminSource: AD FS 2.0
Date:          11/8/2011 10:11:02 AM
Event ID: 364Task Category: None
Level: ErrorKeywords: AD FS
User: DOMAIN\ServiceAccount
Computer: COMPUTER.domain.com
Description:
Encountered error during federation passive request.
Additional Data
Exception details:
Microsoft.IdentityServer.Web.RequestFailedException: MSIS7012: An error occurred while processing the request. Contact your administrator for details. ---> System.ServiceModel.FaultException: MSIS3126: Access denied. at Microsoft.IdentityServer.Protocols.WSTrust.WSTrustClientManager.Issue(Message request, WCFResponseData responseData)
at Microsoft.IdentityServer.Protocols.WSTrust.WSTrustClient.Issue(RequestSecurityToken rst, WCFResponseData responseData)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.SubmitRequest(MSISRequestSecurityToken request)
--- End of inner exception stack trace ---
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.SubmitRequest(MSISRequestSecurityToken request)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.RequestBearerToken(MSISSignInRequestMessage signInRequest, SecurityTokenElement onBehalfOf, SecurityToken primaryAuthToken, String desiredTokenType, Uri& replyTo)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.RequestBearerToken(MSISSignInRequestMessage signInRequest, SecurityTokenElement onBehalfOf, SecurityToken primaryAuthToken, String desiredTokenType, MSISSession& session)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.BuildSignInResponseCoreWithSerializedToken(String signOnToken, WSFederationMessage incomingMessage)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.BuildSignInResponseForProtocolResponse(FederationPassiveContext federationPassiveContext)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.BuildSignInResponse(SecurityToken securityToken)
System.ServiceModel.FaultException: MSIS3126: Access denied. at Microsoft.IdentityServer.Protocols.WSTrust.WSTrustClientManager.Issue(Message request, WCFResponseData responseData)
at Microsoft.IdentityServer.Protocols.WSTrust.WSTrustClient.Issue(RequestSecurityToken rst, WCFResponseData responseData)
at Microsoft.IdentityServer.Web.FederationPassiveAuthentication.SubmitRequest(MSISRequestSecurityToken request)

There were also a lot of events 501 and 502.
We searched the internet for a solution and all we've found was http://social.technet.microsoft.com/wiki/contents/articles/2265.aspx which unfortunately did not work.
The only possible solution that came out was to disable the federation between Office 365 and internal AD infrastructure and re-enable it using a new service account for the AD FS service. To do that we had to follow the procedure:

1. Use PowerShell command Convert-MsolDomainToStandard to convert specified domain from single sign-on  to standard authentication.

If you didn't install the Office 365 cmdlets please follow the instructions from this blog post http://paulroman.pras.ro/2011/11/unable-to-sign-in-to-office-365-with-ad.html.

Run the following commands to disable single sign-on:

$cred = Get-Credential
Connect-MsolService -cred $cred         
convert-MsolDomainToStandard –DomainName domain.com –passwordfile password.txt –SkipUserConversion $false


This will generate temporary passwords for all the users and save them to the password.txt file.

2. Uninstall all ADFS and ADFS Proxy services from all the servers that were members in this process.

3. Following the Microsoft Procedure from http://onlinehelp.microsoft.com/en-us/office365-enterprises/hh125004.aspx reconfigure the ADFS integration with Office 365.

In no more than half an hour the ADFS service was functional again and all the users were able to access Office 365 from internal or external network.

Saturday, November 12, 2011

Unable to sign in to Office 365 with AD federated user

Last week, one of our customer's employee had trouble signing in to his Office 365 services using his federated account. All other accounts were working fine except this one. After investigating the issue we found out that the account UPN was changed because the user changed her name after she got married.
We created an incident in the Office 365 admin console and they suggested us a solution that did not work. After a couple of hours spent searching online for solutions we've created the following procedure.

This procedure should be used anytime you want to change the UPN for a federated user.

Connect to Office 365 using Powershell

Install the Office 365 cmdlets

To begin using the Office 365 cmdlets, you first need to install them. The requirements for installing the Office 365 cmdlets are as follows:
• You can install the cmdlets on a Windows 7 or Windows Server 2008 R2 computer.
• You must have Windows PowerShell and the .NET Framework 3.5.1 installed.
• You must have the Microsoft Online Services Sign-in Assistant installed. For more information, see Manually update and configure desktops for Office 365.

To install the cmdlets, perform the following steps.
1. Download one of the following:
Microsoft Online Services Module for Windows PowerShell (32-bit version)
Microsoft Online Services Module for Windows PowerShell (64-bit version)
2. To install the cmdlets, double-click the AdministrationConfig.msi file.
The installer will add a shortcut to your desktop and Start menu. Click the Microsoft Online Services Module shortcut to open a Windows PowerShell workspace with the cmdlets.
Alternatively, you can also load the Office 365 cmdlets manually by typing import-module MSOnline at the Windows PowerShell prompt.

Change the UPN suffix and disable single sign on for that user

Run from powershell the following command to disable temporary the single sign on and to change the UPN for that user:
set-msoluserprincipalname – UserPrincipalName username@domain.com -new UserPrincipalName username@domain.onmicorsoft.com

The command will allocate the UPN username@domain.onmicrosoft.com to the user and it will generate a temporary password. The password can be used to login to Office 365

Change the UPN for the user

Use Active Directory Users and Computers MMC to change the UPN for the user and force the synchronization between AD Office 365 by running DirSyncConfigShell.psc1 from C:\Program Files\Microsoft Online Directory Sync on the server that has the DirSync tool installed.

Run the following command:

Start-OnlineCoexistenceSync

This will change automatically the UPN for the user to the one that is defined in AD and we re-enable the single sign on process for that user.

Delete ADFS cache from the ADFS servers (not ADFS proxy)

To work around this issue, disable the local SID cache on the domain member computer. To do this, follow these steps:
1. Open Registry Editor.
2. Locate and then right-click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Point to New, and then click DWORD Value.
4. Type LsaLookupCacheMaxSize, and then press ENTER.
5. Right-click LsaLookupCacheMaxSize, and then click Modify.
6. In the Value data box, type 0, and then click OK.
7. Exit Registry Editor.
Note The LsaLookupCacheMaxSize registry entry sets the maximum number of cached mappings that can be saved in the local SID cache. The default maximum number is 128. When the LsaLookupCacheMaxSize registry entry is set to 0, the local SID cache is disabled.

Verify that the user can successfully login in into Office 365 using his federated account and delete the previous registry key.
  

Tuesday, August 23, 2011

Integrating Lync 2010 with Exchange 2010 – Part 2

This is the second part of the "Integrating Lync 2010 with Exchange 2010" article. It was published today (August 23rd, 2011) on the ExchangeInbox.com web site.

In Integrating Lync 2010 with Exchange 2010 – Part 1 we started our discussion focusing on integrating the Outlook Client interface. Today we turn our attention to the Outlook Web App and see how to provide users with Lync functionality through it.

You can read the whole article here.



Tuesday, August 2, 2011

Integrating Lync 2010 with Exchange 2010 – Part 1

I published a a new article on ExchangeInbox.com web site. If you need to integrate your Exchange infrastructure with Lync this is a good start.

Exchange 2010 and Lync 2010 are designed to improve communication and collaboration within the enterprise. The two are designed to work together but certain steps are required to integrate them together.
Exchange 2010 and Lync 2010 are at the moment two of the hottest products from Microsoft. Both are designed to improve communication and collaboration within the enterprise and together are covering all the requirements for the ultimate communication platform.
Exchange 2010 and Lync 2010 are designed to work together but not exactly out of the box. There are certain steps that need to be followed first. In all we have three areas that may be integrated:
Outlook local client - provides Outlook users with in-context access to the instant messaging, enhanced presence, telephony, and conferencing capabilities of Lync.
Outlook Web App - provides OWA users with web access to instant messaging and enhanced presence.
Voice integration - integrates voice capabilities of Lync with Exchange 2010 voice mail.
This article assumes you already have Exchange 2010 and Lync 2010 infrastructures in place. We also assume that the Lync Enterprise voice is enabled and working.

You can read the whole article here.

Monday, July 11, 2011

Information Store is not starting

This is an issue that I've met more than once on server running MS Exchange and Forefront Protection 2010 for Exchange.
After the Forefront definition updates were installed automatically the Information Store and the Transport service are refusing to start. When you try to start them you receive "The dependency service or group failed to start". If you investigate the dependencies you will see that both services are dependent of the FSCController service and that service cannot be started.
In the Application Event log you will find the error 2063 generated by the FSC controller - "An error occurred. Failed to initialize document."

To solve this you have to follow the steps:

1. Disconnect Forefront Protection services from MS Exchange.
From the folder "\Program Files (x86)\Microsoft Forefront Protection for Exchange Server\", using command prompt run: Fscutility /disable.
Run Fscutility /status to check is you disconnected the service.
This will disable the link between Forefront and MS Exchange allowing you to start the Exchange services.
You can read the whole Microsoft KB here.

2. Uninstall Forefront Protection 2010 for MS Exchange from Control Panel, Program and Features

3. Rename the folder "\Program Files (x86)\Microsoft Forefront Protection for Exchange Server\" to something else. If you get an error, you should restart the server before trying again to rename the folder.

3. Reinstall Forefront Protection 2010 for MS Exchange from the installation kit.

4. Reboot the server as requested and install all the updates related to Forefront Protection.

This should restore you the protection provided by Forefront for Exchange.

Friday, July 1, 2011

IT Camp 2011



Between 26th and 27th May I took part as a speaker at the IT Camp Premium Conference on Microsoft IT Pro and Dev technologies. It is great that I was chosen to be part of an excellent speaker list from Romania and abroad and I am looking forward to be there next year.
My presentation was called "High Availability for Exchange 2010" and as the title says I spoke about all the things that you need to do create an high availability infrastructure based on Exchange 2010 starting from Active Directory, Client Access Servers, DAGs and so on.
You can find below the presentation, or you can watch the video (sorry but it's in Romanian).
ITCamp 2011 - Paul Roman - High Availability for Exchange 2010

Thursday, April 14, 2011

Exchange 2010 Hosting Part 2 - The Easy Way

This is the second part of the "Exchange 2010 Hosting" article. It was published today (April 14th, 2011) on the ExchangeInbox.com web site.

In the previous article, Exchange 2010 Hosting Part 1, we discussed the implementation of an Exchange 2010 hosting infrastructure when you need to have all the interesting features included in the product. The installation and configuration method that I presented is somehow difficult and is not supported by Microsoft. If you were to have a problem with that configuration, you would have to find support from within the Exchange technical community and that is not all the time very handy.


If you want to create a hosting infrastructure and don't need features like:
• Exchange Management Console
• Public Folders
• Unified Messaging Server role
• Federation
• Business-to-Business features such as cross-premises message tracking and calendar sharing
• IRM
• Outlook 2003 support (EnableLegacyOutlook)
• Edge Transport Server role
...then, I strongly recommend you to follow this second article.

Even this article starts from the installation of a new Exchange organization. I will focus mainly on the details regarding the hosting configuration and will only briefly remind you the things that need to be done for any kind of Exchange 2010 installations.
Also this article does not cover advanced configuration for a hosting infrastructure, like coexistence with on-premises email infrastructure.

You can read the whole article here.

Saturday, April 2, 2011

MVP once more

Yesterday I received an email from Microsoft announcing me that I received the MVP (Most Valuable Professional) award for Microsoft Exchange for the second year in a row.
I had some doubts about this since I didn't know if what I did for the Exchange community last year was enough.
Even my "money making job" as at PRAS seems to take more and more from my free time, I will continue to work for the IT community by keeping this blog, writing technical articles for ExchangeInbox.com, helping others on forums or continuing to keep online or offline presentations for MS Exchange and more. And who knows, maybe next year I will be able to write a similar article :-).

Monday, March 14, 2011

Exchange technical articles on exchangeinbox.com

This month I started to write technical articles about Microsoft Exchange on exchangeinbox.com web site. The subject of this articles will be strictly technical and i will try to cover only interesting stuff . The first one is called “Exchange 2010 Hosting” and covers non standard configuration for hosting with Exchange 2010.


…Hosting multiple Organizations on a single Exchange Infrastructure can give significant hardware consolidation advantages. Exchange 2010 has greatly simplified the setup of such an environment, giving us two implementation options, the hard and the easy way. Today we go down the hard way...

Starting with Exchange 2000, Microsoft had the idea to implement hosting infrastructures based on MS Exchange. To provide support for such implementations MS released Hosted Solutions add-ons, starting with "The Solution for High Volume Exchange version 1.0" based on Exchange 2000.

Up till now, Microsoft released eight versions, with the release of Hosted Messaging and Collaboration HMC version 4.5 in June 2008. HMC 4.5 was working with Exchange 2007 SP1 together with Office Communication Server 2007, SharePoint Services 3.0 and Forefront Security for MS Exchange 2007.

In Exchange 2010 SP1 there is no need for an additional tool to configure a hosting environment. All you have to do is to run the setup for Exchange 2010 SP1 with the /hosting switch. It's an easy way to configure a multi-tenant infrastructure. However some features are missing, namely:
• Exchange Management Console
• Public Folders
• Unified Messaging Server Role
• Federation
• Business-to-Business features such as cross-premises message tracking and calendar sharing
• IRM
• Outlook 2003 support (EnableLegacyOutlook)
• Edge Transport Server role

You can read the whole article here.

Tuesday, February 8, 2011

How to configure Form Based Authentication in the same time for internal accessing the Exchange server directly and external users accessing the server via Forefront TMG

I answered a question today regarding the fact that when you publish an Exchange server via Forefront TMG and you want to have Form Based Authentication (FBA) for the external users, you have to disable it for the internal users.


Publishing OWA within TMG with FBA requires you to enable Basic Authentication for the owa (Default Web Site) virtual directory in Exchange and disable FBA .
If you don’t do that, when authenticating from outside your network you will be required to authenticate twice, once on TMG and another time on the Exchange Server.

In order to have Form Based Authentication in the same time for your internal and external users you have to create a new web site on the Exchange Server and create a new OWA virtual directory there.


The steps are:
1. Create a new Web site within your IIS management interface





 
 
 
 
 
 
 
 
 


2. Configure the new web site considering the followings:
- The IP address and port combination should be unique
- Create a folder for the Physical path and assign to the Local Service and the Network service read rights for that location
- Remember the site name because you will need it later




















3. Verify that the new web site was created and it’s running


 
 
 
 
 
 
 
 
 




4. In Exchange Powershell console run the command New-OwaVirtualDirectory -WebSiteName "Internal OWA"








5. Create the corresponding ECP virtual directory by running the command New-ECPVirtualDirectory -WebSiteName "Internal OWA"








6. Configure the newly created OWA and ECP virtual directory for FBA authentication using the following cmdlets:
- set-OwaVirtualDirectory -Identity "owa (Internal OWA)" -FormsAuthentication $true
- set-ECPVirtualDirectory -Identity "ecp (Internal OWA)" -FormsAuthentication $true

7. Restart the IIS by running in elevated command promt the command iisreset /noforce

8. Instruct the internal users to access the newly created virtual directory