Exchange AutoDiscover

Exchange 2007 and Exchange 2010 (in conjunction with Office 2007 onwards) use Autodiscover to automatically fill in the users account details the first time he or she logs in. It also takes care of "availability", out-of-office, offline address book and a few other things. In Exchange 2003 these things either were not done or they were achieved with public folders.

The client first queries active directory for the CAS server's autodiscover URL. It finds this by means of a "service connection point", which is located in the following location:

CN=[Server Name],CN=Autodiscover,CN=Protocols,CN=[Server Name],CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=[Exchange Organisation Name],CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=[Domain],DC=com

In order to edit this object you can use the set-ClientAccessServer -identity [CAS Server] -AutoDiscoverServiceInternalURI "[URL]" (it is stored under ServiceBindingInformation).


According to this guy (http://blogs.technet.com/b/rmilne/archive/2013/04/02/busting-the-set-autodiscovervirtualdirectory-myth.aspx) setting the "AutodiscoverVirtualDirectory" internal and external URLs does NOTHING.



Outlook Autodiscover Logging:


%temp%\Olkdisc.log

You can also hold control and right click the Outlook icon in the taskbar and it will give you Autodiscover tools.

Ontap Vif miscellany

A few things about networking on a FAS2050.

The FAS2050 has two ports per controller: e0a and e0b. There is a management port (with a wrench symbol) but I can find no trace of this from the software side of things.

From controller 1 I deleted all the vifs so that I would have to learn to recreate them via SSH. On the switch each port that the Netapp is plugged into is configured as follows:

interface FastEthernet0/2
 switchport trunk allowed vlan 100,302,303
 switchport mode trunk
 spanning-tree portfast

I decided that I ought to aggregate the two ports so I ran this command:

vif create multi dennis e0a e0b

The multi could have also been "single" or "lacp". Multi and LACP are forms of aggregation whereas single is an active / passive arrangement. I could have also added a "-b" switch to specify the type of balancing (IP, port, MAC or round robin - I think IP is default).

I then created the VLANs to correspond with the switch port settings:

vlan create dennis 100 302 303

I then decided I didn't need VLAN 302 and 303 for this test:

vlan delete dennis 302 303

Finally, I had to assign an IP address to the new interface. Hence:

ifconfig dennis-100 10.10.10.54 netmask 255.255.0.0

BSOD Special Edition

Client PCs at my company mysteriously reboot every now and again. I decided to try and decipher a dump file.

Before I begin, credit to this guy:

http://weblogs.asp.net/owscott/archive/2012/07/18/reading-a-memory-dmp-or-other-dmp-file.aspx

And in fact, I can't really improve in what he says so I will simply reproduce what he says:

1. Install the debugger
2. Browse to the installation directory in a cmd
3. Type kd –z [path to dump file]
4. Type .logopen [path to log file.txt]
5. Type .sympath srv*c:\symbols*http://msdl.microsoft.com/download/symbols
6. Type .reload;!analyze -v;r;kv;lmnt;.logclose;q

The contents of the dump file will be written to the log and the console window. Read it and solve the problem. Or don't, because in this case the dump files pointed to the graphics card but that turned out to be only a factor and not the entire problem..

Cisco Miscellany

interface FastEthernet2/0/1
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 100
 switchport mode trunk
 switchport voice vlan 2
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape 10 0 0 0
 mls qos trust cos
 auto qos voip trust
 spanning-tree portfast trunk

What does this mean?

Interface (port) running at 100MBs of identity Switch 2 / Stack 0 ?? / Port 1 has the following characteristics:

It is a trunk and its encapsulation is 802.1Q. Its native to VLAN 100 but will accept traffic from other VLANS too. It is configured to handle traffic from VLAN 2 (i.e. VOIP devices) differently that other traffic. This is shown with the bandwidth shaping and bandwidth sharing.

Each interface has 4 queues (this is known as egress queuing). You can specify what share of the bandwidth by issuing it a ratio of the total bandwidth. So srr-queue bandwidth share 10 10 60 20 would assign 60% of the bandwidth to queue 3.

The shaping command works differently as it denotes a percentage. Queue 1 in the output above will only be able to use 10% of the bandwidth.

The lines containing QoS relate to prioritising the VOIP traffic. The COS identifier (a value originating from the header of frames) is used to identify VOIP traffic which is then given priority over data frames.


It looks as though the VOIP traffic uses queue 1 because it is shaped and allows for "smoother" traffic flow.

Useful links:

https://supportforums.cisco.com/thread/165677

SQL House Keeping

I made the mistake of creating my SQL server virtual disks (database and log disks, both comprised of RAID 1 arrays) so that they encompassed the entire datastore. I didn't think this would be much of a problem but if you use snapshots in any way, shape or form then you will run into problems because the snapshots must (I think) go on the same datastore. We use vSphere Data Protection and so when it was doing a backup the SQL server floundered due to lack of disk space.

So apparently you can't shrink a virtual disk, or at least I don't know how to.

I had to move all the databases and logs onto a third disk and then remove and reconfigure the log and database disks and then move all the data back onto the newly resized disks. Here is how:

For each user database run this query for each database, index and log:

ALTER DATABASE SUSDB MODIFY FILE ( NAME = SUSDB , FILENAME = 'H:\Data\SUSDB.mdf' );
ALTER DATABASE SUSDB MODIFY FILE ( NAME = SUSDB_log , FILENAME = 'H:\Logs\SUSDB_log.ldf' );

The "name" field maps to the logical name of the file. The "Filename" should map to where you want to move the file.

After running this query, stop the SQL Server service and then manually move the files. Restart the service (and the agent if necessary) and check that the DB starts and then the files are where they should be. This command also helps to verify this:

SELECT name, physical_name AS CurrentLocation, state_desc
FROM sys.master_files
WHERE database_id = DB_ID(N'SUSDB');

IMPORTANT: It appears that the destination folder must have the following user account present in its ACL with full rights:

SQLServerMSSQLUser$fwsql$MSSQLSERVER

The Master DB is slightly different. You must edit the start up parameters in the Configuration Manager to the new locations. Config Manager > Services > MSSQLServer Properties > Advanced > Startup Parameters.

Within the parameters "-d" is the Master DB, "-e" is the error log and "-l" is the log.

Once you have changed these values, stop the service and move the files and then restart the service. Make sure the folder that you move it to has that unwieldy SQL account in its ACL.

The Model and the MSDB DBs are moved in the same way as the user databases.




Resources:

http://msdn.microsoft.com/en-us/library/ms345408.aspx

Talks of a "Startup Parameters" tab but there doesn't appear to be one.

Exchange 2003 ActiveSync

Creating a blog that acts as a reference to previous work is clearly not my forte.

We had an Exchange failure over the weekend. It wasn't actually an Exchange failure but a VMware ESXi 4.0 datastore that got out of control. The lesson learned was don't take snapshots unless you really have to and make sure you delete them as soon as possible. I already knew this but somebody, three years ago, took a snapshot called "test" and there it remained, undetected, consuming more and more of the disk until it was impossible to delete. It is impossible to delete because it needs to consolidate disks which are now too unwieldy to be merged on the remaining datastore. It is particularly difficult if you are using local storage and there's nowhere to go or grow.

So after a restore from tape everything was working again. Except ActiveSync.

So the environment consists of one front-end server and two back-end servers. In this scenario I am under the impression that as long as port 443 is open on your firewall the FE will catch all requests and pass them on to the BE servers. The default website config that the Exchange 2003 installation puts in place should just work. That is all the virtual directories (ExAdmin, Exchange, ExchWeb, Exchange-Server-ActiveSync, OMA and Public) will be set up with the correct security and functionality. The certificate should reside on the FE server which should also stipulate SSL.

In a scenario with no FE server AND SSL or forms-based authentication is enabled then you need to create a duplicate of the Exchange VD, assign it name (Microsoft seems to like "Exchange-OMA" but "Whatever-the-fuck-you-want" is ok too) and then point to it in a registry entry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MasSync\Parameters\ExchangeVDir

This should be case-sensitive and contain a string such as "/exchange-oma" or "/Whatever-the-fuck-you-want".

But you don't need this registry entry if you have a FE server. Of course in the upside-down environment I was working in all servers have this entry, even the FE server. So in order to get AS working I did the same on the restored server and it started working again. God knows why, but no doubt I will be returning to this at some point.

 A side note to this is about resetting the VDs should they stop working. Here are the steps:

1. In IIS delete all the VDs - the ones named above. You can take a backup of the website first but I'm not sure how useful this is.
2. Go to c:\inetpub\adminscripts in a cmd and type "adsutil delete ds2mb".
3. Restart the System Attendant and see the VDs recreated.

Add a route to a Cisco Switch

I tried this on two switches (one being a 3750 and the other being an old 10/100) and it only worked on the 3750.

So type en and enter and then the password.

Then

conf t

then

ip route 172.16.19.80 255.255.255.248 10.10.1.253

where the first address is the destination and the second the route.

You can see all the routes by typing show ip route.

LB 2003 to 2010 Exchange Migration p4

I went in to remove the Exchange 2003 server from the LB office and have the following observations.

The public folders went fairly smoothly, in that the "MoveAllReplicas" script seemed to do its job and clear the old server of all its replicas.

A new internet connector needed to be created for internet-bound email because ownership can't be given to the new server (due to it being in a different routing group).

The RUS service needs to be handed to the new server, which I didn't mention in past posts.

So after removing one system e-mail account (something to do with Free / Busy, but not sure what since this functionality should be derived from public folders...) the 2003 uninstall was ready to proceed. And so it did, but it also failed quite spectacularly and so, outlined below, is a guide on how to manually remove an orphaned Exchange 2003 server from AD:

1. Go into ADSIedit and delete the server object.

You should be able to run the BPA with no errors, and if you do have errors then they can probably be fixed by copying and pasting the DN from one field, in AD, to another. Not graceful, but there you go.

Cisco + NetApp + VMware p1

I have a free NetApp FAS2050, a very old Cisco 2950 100mb switch and a HP DL380 server. Can I make them work together?

My objective is to run ESXi 5.1 on the HP server with two guests (an Exchange 2010 server and a domain controller). The server only has 2 NICs unfortunately: one of them will be used for iSCSI and the other management. The NetApp must be configured in one big RAID 10 volume. They must communicate through the 2950.


The Cisco 2950

Log in as a super user:

en ↵
[password] ↵

The "show" command followed by "?" will display all the informational commands. To start off, good informational commands:

show running-config ↵
show interface status ↵ 

The switch that I picked up has most of its ports set to VLan 100, but I want VLan 302 because that is what the NetApp is set to. To create a VLan:

conf t ↵
vlan 302 ↵
name iSCSi ↵
Ctrl Z

Now to assign a port (or two) to the previously created VLan:

conf t ↵
int fa0/1 ↵
switchport mode access ↵
switchport access vlan 302 ↵
Ctrl Z

If you wanted to make that port a trunk you would have entered:

switchport mode trunk ↵

A trunk can accept multiple VLan traffic:

switchport trunk allowed vlan 100,302,303
 
I cheated a bit in the end. I had the config from the switch the NetApp used to be plugged into. From that I determined how the ports should be configured. This is the port configuration that I copied:

interface GigabitEthernet0/7
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 100,302,303
switchport mode trunk
spanning-tree portfast

After this the NetApp could be pinged and I could connect to it via putty.

WMIC

Invoke simply with "wmic" and see usage with "/?".

If you want to see the shares on another computer, you can type:

/node:[comp name] shares

or if you want to see the default gateway:

/node:[comp name] nicconfig get defaultIPgateway

The nicconfig command displays screeds of stuff so the "get defaultIPgateway" narrows it down.

LB 2003 to 2010 Exchange Migration p3

I was told today that they wouldn't mind a new name for their external OWA address and so the co-existence stuff isn't necessary anymore. Can't say I'm mortified by this news...

On Friday we managed to get through to the installation of SP2 and so I didn't have a great deal to do. The ESM presented me with a long terrifying error but was ok after a reboot. I put on RU5v2 and begun public folder replication:

.\AddReplicaToPFRecursive.ps1 -server "Exchange 2010 Server" -TopPublicFolder "\" -ServerToAdd "Exchange 2010 Server"

I will return to this company to remove the old public folders. I will probably run (to hurry it up):

Update-PublicFolderHierarchy -Server "Exchange 2010 Server"

To get rid of the replicas on the old server:

.\MoveAllReplicas.ps1 -Server "Exchange 2003 Server" -NewServer "Exchange 2010 Server"

To check on the progress:

Get-PublicFolder -recurse |fl name,replicas
Get-PublicFolder -recurse \non_ipm_subtree |fl name, replicas
Get-PublicFolderStatistics

The Exchange server's certificate was replaced by a wildcard certificates, which seem to simplify everything a lot over SAN certificates. There must be a catch.

The setting of the "legacy" URL was now uneccessary.

Set-OwaVirtualDirectory -Identity "esp-ho-ex2010a\owa (Default Web Site)" 

-Exchange2003Url https://legacy.crapulent.net/exchange

And there was no need to enable forms based authenication on the 2003 server, or for installing the Activesync hotfix to enable kerberos between the two servers.

The first Activesync test failed because of the domain admin protected group issue.

Other than that, job done for now. 

LB 2003 to 2010 Exchange Migration p2

Yesterday I copied and pasted a screed of unformated text which formed my attack plan for the migration. Today I have been at the company and gotten underway with the whole thing. So, inevitably with IT, my list wasn't comprehensive and certain problems have slowed the progress of the installation.

When Exchange 5.5 was around it used a thing called the "Active Directory Connector" which was for syncing with AD. 2003 did away with it and began using the recipient update service and the ADC was rendered redundant. Exchange 2007 / 2010 will not proceed with installation if there are any traces of the ADC left. Additionally, the Exchange orgnisation was in "mixed mode" and should have been in "native mode". So how do you remove these traces of ADC and change the mode of the organisation?

Firstly, uninstall the ADC service. In the case of this company it was installed on a DC. Uninstall failed citing "additional connectors". From here the ADC connectors need to be manually deleted from ADSIEdit. So go to Config > Services > Exchange > AD Connectors and delete the connectors not related to the ADC service. Once this is done the uninstall will proceed.

Some pesky ADC were still lingering, however. These we simply deleted. This can be achieved from ADSIEdit, as mentioned above, or AD Sites and Services (remember to show the hidden service options from the toolbar).

Now we thought we would be able to change to native mode, but still it was greyed out. The next thing to be done is to stop and disable the "Site replication service" on Exchange 2003. Still native mode was greyed out.

Finally we deleted the "Site replication service" in Exchange system manager under Tools. At last we could change to native mode and begin to prepare Legacy permissions (setup /pl), the schema (setup /ps) and the domain (setup /prepareAD).

LB 2003 to 2010 Exchange Migration p1

I have to go to some company tomorrow to help with their migration from Exchange 2003 to 2010. I've done this a few times but I think that I may have some blagging to do if they choose the co-existence route.

I wrote this in preparation for my visit.

Exchange 2003 - 2010 migration

Planning and Install

1.    Exchange Server 2010 Planning
a.    Is Exchange 2003 running SP2. If not it must be upgraded.
b.    Domain Functional level – must be 2003
c.    Hardware and storage considerations
i.    Run perfmon to find current IOPS requirement
d.    Does the environment require roles at different sites or can all roles be on one server?
e.    Exchange Server Deployment Assistant, or ExDeploy
f.    DR considerations. DAGs require Windows 2008 R2 Enterprise
g.    How many databases should be deployed for easy restoring?
h.    Exchange Best practice analyser – run to determine existing problems
i.    External access requirements:
i.    Additional certificate needed. SAN certificate recommended with “mail”, “autodiscover” and “legacy”.
ii.    How is OWA / ActiveSync published?
j.    What version of Office is being used? Is there a requirement for public folders?
k.    Archiving software – what needs to be done to move this to new server?
l.    What SMTP devices (e.g. MFDs, backup alerts, etc) need to be reconfigured?
2.    Windows 2008 R2 hotfixes
a.    http://technet.microsoft.com/en-us/library/bb691354%28v=exchg.141%29.aspx
3.    Exchange 2010 Installation
a.    Install the Windows RPC Over HTTP Proxy Component
b.    Prerequisites can be installed by the Exchange 2010 installer if SP1 is bundled.
c.    If external domain names are known they can be entered during installation, otherwise they should be entered in step 5
d.    SP1 and SP2 and rollup hotfixes
e.    Exchange best practice analyser – run again
f.    Configure the Exchange 2010 certificate
i.    Is there an internal CA available?
ii.    Provision for a legacy URL (if option 1, below, is to be followed)

Option 1 – Co-existence

Both servers will run in tandem for the duration of the migration. More complicated route but allows external OWA for users on both servers except during the period when their mailbox is moved (move is offline).

4.    Configure the client access server.
a.    Enable Outlook anywhere on client access server
b.    Set the Legacy URL on OWA
i.    Set-OwaVirtualDirectory <CAS2010>\OWA*  -Exchange2003Url https://legacy.contoso.com/exchange
1.    Enable forms-based authentication on Exchange 2003
c.    Change the OAB generation server to the new 2010 server and enable web based access
d.    Install hotfix for ActiveSync
i.    http://support.microsoft.com/?kbid=937031
5.    Publish the old Exchange server via the legacy URL
a.    Import the Exchange 2010 certificate into the Exchange 2003 server
i.    Use this recently imported certificate in IIS on the 2003 server by replacing the old one
b.    Arrange with DNS service provider to create an A record for the legacy URL, which should match the URL in 3.c.ii and 5.b. This can point to the same IP address if the publishing method allows this. E.g. ISA 2006.
c.    Use firewall / ISA to publish the old Exchange server from the new external IP / legacy URL
d.    Disable HTTP over RPC on any 2003 front end servers.
6.    Use ISA / firewall to publish the new Exchange server from the old DNS record.
a.    Users on both servers should now be able to reach their mailboxes via the old URL (and 2003 users should be able to use the legacy URL too).

Option 2 – Migration during maintenance window

The new Exchange server will be published for OWA and then all mailboxes can be moved to the new server. As soon as a user’s mailbox resides on the new server OWA will become available for him or her.

7.    Use the existing external URL and publishing rule on the firewall / ISA to publish the new Exchange server.
a.    The ECP sub site should also be published
b.    Users on the Exchange 2003 server will now be unable to access OWA.

Migration Process 

8.    Once option 1 or 2 above is complete then the mailboxes can be moved using the Exchange 2010 console or shell.
a.    Migration of mailboxes causes a large amount of logs and so disk space should be monitored.
9.    If public folders are in use then they should be replicated to the new server.
a.    .\AddReplicaToPFRecursive.ps1 -server "Exchange 2010 Server" -TopPublicFolder "\" -ServerToAdd "Exchange 2010 Server"

Post Migration processes

10.    Check public folders have finished replicating (get-publicfolderstatistics) move them to the new server:
a.    .\MoveAllReplicas.ps1 -Server "Exchange 2003 Server" -NewServer "Exchange 2010 Server"
b.    Use Get-PublicFolder -recurse |fl name,replicas and Get-PublicFolder -recurse \non_ipm_subtree |fl name, replicas to ascertain whether migration is complete.
11.    The public folder home server should be moved to the new server
12.    Shared mailboxes can be converted to resource mailboxes
13.    Uninstall Exchange 2003
a.    If it is impossible to uninstall Exchange due to remaining replicas in the public folder database then it can removed using ADSIedit before proceeding.
14.    Remove routing group connectors and the RUS using ADSIedit.

Log Parser

If you want to analyse IIS logs Log Parser is your man. Install it, cmd to its directory and run the executable followed by SQL query in quotes. Here is an example*:

LogParser "SELECT date, REVERSEDNS(c-ip) AS Client, COUNT(*) FROM ex130130.log WHERE sc-status<>200 GROUP BY date,client"

In this case I copied the log file from an Exchange server into the Log Parser install directory. The results:

date       Client                                           COUNT(ALL *)
---------- ------------------------------------------------ ------------
2013-01-30 *.com 29
2013-01-30 *.com 138
2013-01-30 *.com                                     8
2013-01-30 *.com                2
2013-01-30 *.net                 2
2013-01-30 *.com               276
2013-01-30 *.com   21
2013-01-30 *.com                2
2013-01-30 *.com                1
2013-01-30 *.net                 2
Press a key...
date       Client                           COUNT(ALL *)
---------- -------------------------------- ------------
2013-01-30 *.net 2
2013-01-30 *.net 1
2013-01-30 *.net 1
2013-01-30 *.net 1

Statistics:
-----------
Elements processed: 3034
Elements output:    14
Execution time:     4.34 seconds

* This blog digs deep to find the ins and outs, nooks and crannies, cracks and cheats that help the budding but hungover sysadmin. This is why you can find this example in the Log Parser help menu. It's the first entry.

DR Server Migration

The great virtualisation project mentioned in the last post hasn't really kicked off as I had envisioned and no hardware has made an appearance yet! From the outset this blog has been a roller coaster ride of exciting facts so apologies to my avid readers for the delay here.

Had a simple server move to do yesterday. We moved a switch, a storage server and its associated SAS-attached storage and an ESXi server from one data center to another. A NetApp FAS2050 was decommissioned and lies unused on our server room floor.

The decommisioned 2050 was the snapmirror target but was replaced by a hosted "vFiler". I intend on powering up the old one soon, but what do I have to do to it beforehand? Delete snapshots, that's what:

Show all snapshots on the current controller:
 
snap list

Show the status of the various snapmirrors:

snapmirror status

Delete a particular snapshot:

snap delete [volume_name] [snapshot name]

One of the snapmirrors to the decommisioned SAN was "busy" and so the snapshot could not be deleted. I used

snapmirror abort [source_filer]:[volume_name] [destination_filer:volume_name]

But it told me that it was already idle. So how could I delete a snapshot that was supposedly busy (but was not apparently involved in a snapmirror)?

Well it turns out that that volume was being snapmirrored to the new hosted vFiler and this was causing a lock. I'm not sure, since there are two snapshots for that volume, one for the old destination and one for the new destination, why this would happen but there you go.

Virtualisation Project - Part 1

After at least 6 months of procrastinating, one of the companies I work for has stumped up some money for some new hardware. Presently they have the most ramshackle IT setup that I have ever had the misfortune of witnessing and so, fortunately for me, it will be impossible not to improve upon it.

The title is somewhat misleading in that they already run virtual machines. However, their hosts are scattered across various versions of ESXi, ESX and even GSX (whatever that is) and cannot be centrally managed. Their backups criss-cross the network like the tangled web of a retarded spider. After being mismanaged for years, their Citrix and group policy is beyond comprehension.

So from tomorrow, I will be recording the interesting parts of this project.

ESX 3.5 CLI

Always seem to forget how this works so here is a tiny note on how the above works.

First of all, the syntax:

<command>.pl <conn_params> <params>

Second of all, an example:

vicfg-nics.pl --server 10.1.205.30 --list -l

The example above will prompt for username and password which can be included by using "--username" and "--password".

Thirdly, a link to the guide:

http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_rcli.pdf

And finally a link to a more colourful piece of work on the subject (see second comment):

http://communities.vmware.com/thread/176952