Connecting to TFS from any version of Visual Studio

2018-04-10

I've written a couple of blog posts on this subject before and thought it would  be nice to consolidate them into one big post now that Visual Studio 2013 is about to be released. There are 3 major things to consider when connecting to TFS from an older version of Visual Studio.
  • The version of TFS you want to connect to
  • The version of Visual Studio you're connecting from
  • The version of Windows you're running
In this blog post:
  1. Connecting to Team Foundation Server 2015, 2017, 2018 or Visual Studio Team Services.
  2. Connecting to Team Foundation Server 2013.
  3. Connecting to Team Foundation Server 2012.
  4. Connecting to Team Foundation Server 2010.
  5. Configuring the MSSCCI provider for Visual Studio
  6. Connecting from Visual Studio 2005, 2008 or the MSSCCI provider
Since writing this post, Microsoft has produced an updated piece of documentation (which is already behind in some aspects) that describes most of the client/server combinations. One thing it adds which I haven't described, is the list of features that are available depending on your version of Visual Studio/TFS.

If you're having issues connecting after updating, it might be required to clear your local client cache to clear up certain issues like this one. Either the official way or the hard way.

If you also want to install the Team Foundation Server Power tools to match your Visual Studio/TFS version, check out this separate post.

Update: Updated download links to the final version of Team Explorer 2013.

Update (13-11-2013): Updated download links to the final version of Visual Studio 2012 Update 4.
Update (25-11-2013): Added information on client cache and feature docs.
Update (6-1-2014): Added update for VS2010 for Windows 8 and Visual Studio 2012
Update (21-1-2014): Added update 1 for Visual Studio 2013
Update (25-8-2014): Added update 3 for Visual Studio 2013
Update (21-7-2015): Added update 5 for Visual Studio 2013 and Visual Studio and TFS 2015. Since Visual Studio 2013 Update 5 contains updates to handle TFS 2015 Team Project Renames, it's recommended for anyone connecting Visual Studio or MSSCCI 2013 to TFS 2015.
Update (22-7-2015): Added update 5RC for Visual studio 2012.Since Visual Studio 2012 Update 5RC contains updates to handle TFS 2015 Team Project Renames, it's recommended for anyone connecting Visual Studio or MSSCCI 2012 to TFS 2015.
Update (26-8-2015): Replaced Visual Studio 2012 Update 5 RC with the final version.
Update (30-8-2015): MSSCCI 2013 now also supports Visual Studio Team Foundation Server 2015. But it still depends on Team Explorer 2013.
Update (27-10-2015): IntelliJ IDE's support added
Update (8-1-2016): logging on using Azure Active Directory is no longer supported on Windows XP/2013 and systems with IE8 or older.
Update (4-7-2016):Added Visual Studio 2015 update 3
Update (16-11-2016): Added TFS 2015 (same as TFS2015)
Update (04-10-2018): Added TFS 2017 and TFS 2018 and Visual Studio 2017


Crafting complex Release Gate conditions in VSTS

2018-04-03

A recent addition to VSTS is the ability to run a quick check prior to triggering a release to an environment. These checks can be used to check that there are no new customer complaints, no mad users on twitter, no important jobs running in the environment etc.

I wanted to make use of this feature as part of the CI/CD Tasks for Extensions. With the recent introduction of validation of the extension, it's possible that your newly uploaded extension isn't immediately available. So before kicking off tests for your extension, you may want to wait for the validation to succeed.

This resulted in two tasks, the first is a Server Gate task that can wait for the extension validation to finish before triggering a release. The second does the same thing, but runs on the build agent. They both rely on the Marketplace REST API.

Disambiguate MSA and AAD accounts

2018-03-17

Microsoft is finally closing the loophole that allowed you to create an MSA account (LiveId) with the same unique name as your AAD (Azure Active Directory) account. While it has been very useful in many cases to use the same ID for both the MSA and the AAD account, most services that relied on only MSA are finally shipping updates to also support AAD.

I've always had my MSA and AAD account share the same identity ever since I created my Microsoft Account almost 15 years ago. And every since Microsoft introduced Azure Active Directory I've had to choose between a "work and school account" or a "personal account". It helps that I have a pretty good understanding of the difference, so for me it never really posed more than a minor inconvenience, but I see a lot of clients confused and frustrated by the, in their eyes, useless question:


Because of it's age a lot of profiles were associated to it, and changing the sign-in address of my MSA felt a bit scary. Just to give you an idea of the services linked to my MSA (jhouwing@xpirit.com):

  • Microsoft Certification Portal
  • Microsoft Most Valuable Professional Portal
  • Microsoft Partner Portal and Partner link to Xpirit
  • Microsoft Visual Studio Marketplace Publisher account
  • XBox Live account
  • Windows Phone Marketplace
  • Windows Developer Account
  • MSDN subscription (from MVP)
  • Azure Subscription (multiple)
  • Visual Studio Team Services (multiple)
  • Visual Studio Enterprise license in Visual Studio 2017 (through MSDN)
  • Azure AD Guest user in a number of partner directories
  • Windows Store
  • Groove Music
  • Family Office 365 subscription
  • OneDrive
  • Windows Insider
  • Skype
  • My personal laptop
  • My work laptop
  • My personal Xbox
At the same time a number of things were associated to my AAD account sharing the same identity (jhouwing@xpirit.com):
  • Microsoft Visual Studio Marketplace Publisher account
  • MSDN Subscription (from work)
  • Azure Subscription
  • Visual Studio Team Services (access to Microsoft owned accounts)
  • Azure AD Guest in the Microsoft directory
  • Work Office 365 subscription
  • Ondrive for Business
  • Skype for Business
  • Windows Insider
  • My work laptop
I'd switched identities on my Microsoft account before, when I left my previous employer and joined Xpirit, so I was accustomed to the process of re-associating in the Microsoft Partner Portal and switching the primary identity in my MSA account, but I'd always hit a few problems and over the years the number of additional devices and services has steadily grown.

To start the disambigiation process I first added a new secondary identity to my MSA account (jesse.houwing@gmail.com). This option is pretty hard to find if you don't' know what you're looking for. You can find it in the Microsoft Account portal:
Click the "Manage your sign-in email or phone number" link and there you can add additional sign-in addresses to your account. In my case I added a secondary sign-in address for my gmail account:
After confirming you own this address through your chosen method of security, you can now sign in to most services using either address. A few won't work through, as I found out:
  • Visual Studio Team Services won't allow to sign in with a secondary identity. It will however automatically swap you to your new identity once you make it primary. Account ownership will also be updated automatically nowadays. That was a great relief.

Form there I clicked the "Make Primary" link on my new primary identity and after that I checked whether I could still access all my accounts. Switching my primary identity had a few unexpected side-effects:
  • I had to update my MSA account information on my windows devices.
  • I had to sign into my Xbox again
  • I had to restore my Windows Insider details
  • I had to uninstall the Windows Feedback app and install it again (should be fixed in a later version)
  • I had to sign out fo Visual Studio completely and sign in again so refresh my license and to connect to Visual Studio Team Services.

After confirming I could still access all my services I crossed my fingers and went on to remove my old primary identity.

After clearing all cookies in my browsers I am now no longer greeted by disambigution prompts, which makes me very happy. I'd still love it if Microsoft would make this process simpler and if they'd be able to remove the issues I encountered, but the process was a lot easier than I had been dreading.

Ohh and while you're at it, you may as well update your security preferences, enable 2-factor authentication and set a stronger password ;).

If you're wondering whether a company could solve this problem for their users, the answer is no. There is no way for an organisation to query which users have the same ID for their AAD and their MSA account and there is no way for a company to change the primary identity on behalf of their employees. The MSA account is owned by the individual and privacy and legal reasons prevent Microsoft from solvign this on behalf of a company.

Fixing Edge, Start Menu and Cortana slowness

I've had issues with my machine for months now and it was very hard to pinpoint the culprit. It resulted in:

  • Start menu freezing
  • Cortana/explorer bar search staying black for up to 30 seconds
  • Edge not wanting to open or regularly freezing
I had tried all the tricks out there, none worked:
  • DISM /Online /Cleanup-Image /CheckHealth
  • DISM /Online /Cleanup-Image /ScanHealth
  • DISM /Online /Cleanup-Image /RestoreHealth
  • sfc /scannow
  • Windows 10 app troubleshooter
  • Windows 10 search & indexing troubleshooter
  • Windows 10 windows update troubleshooter
  • Reset Edge from the Apps & Features in Settings 
  • Delete edge data in the user profile folder (%userprofile%\AppData\Local\Packages\Microsoft.MicrosoftEdge_8wekyb3d8bbwe)
  • Reinstall Intel and Nvidia graphics drivers
  • Install Nvidia driver without the nView extensions
  • Uninstall 3rd party virus scanner.
  • Reinstall all store apps: Get-AppxPackage -AllUsers| Foreach {Add-AppxPackage -DisableDevelopmentMode -Register “$($_.InstallLocation)\AppXManifest.xml”}
Technically the issue appeared in a few ways:
  • After starting edge the following processes would cause high-CPU and edge would crash:
    • browser_broker
    • runtime_broker
  • After opening the start menu or Cortana the following process would cause high-CPU and would freeze the start menu or Cortana for up to 30 seconds:
    • dllhost.exe
    • com surrogate
  • The event viewer would list a number of DCOM errors:
    • The server {0002DF02-0000-0000-C000-000000000046} did not register with DCOM within the required timeout.
    • The server {2593F8B9-4EAF-457C-B68A-50F6B8EA6B54} did not register with DCOM within the required timeout.
    • The server {973D20D7-562D-44B9-B70B-5A0F49CCDF3F} did not register with DCOM within the required timeout.
One of the tips to remove this problem was to uninstall the Intel and Nvidia graphics drivers and have Windows reinstall them. This didn't help either but did expose me to the culprit. After reinstalling the video drivers my machine was unable to start causing a blue screen on the following driver: vdd2hookkmode.sys. To solve the blue screen issue I had to delete the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VDD2HookKmode

This driver ships with the Barco ClickShare Extension Pack, a set of drivers I use to share my screen to Barco's otherwise wonderful screen sharing devices we use in our training rooms. These drivers aren't updated through Windows Update or come with an update notification of any sort and removing them solves all my issues.

Their latest driver supposedly fixes some of these issues as well. But for me Edge becomes unusable the moment I install the latest version. For now I'll have to do without presenter view when using ClickShare together with PowerPoint.

New versions of TFS/VSTS build tasks

2018-03-16

I just pushed out new versions of the following tasks:


  • MsBuild Helper Task
    • Fixed a number of bugs when the Agent's work folder has a space in it somewhere.
  • Snyk
    • Fixed auto-update of built-in Snyk version
    • Added support for Path parameter to scan NuGet packages among other package types.
    • Added support for Severity Threshold parameter.
    • Updated built-in Snyk version to 1.70.2
  • Variable toolbox
    • Fixed PadLeft and PadRight
    • Fixed Regex search & replace
    • Upgraded to PowerShell3 handler to improve task performance
In the meantime I've been working on a number of other build-task related things, more to come soon:
  • TFVC Tasks
    • Upgrading to PowerShell3 hander
    • Using tf.exe instead of custom Client Object Model code
    • Consolidate separate tasks into a single task (like nuget task)
  • CI/CD tasks for Extensions
    • ServerGate support continue after validation has succeeded
    • Automatic versioning of extensions containing multiple build tasks
    • Moving tfx installation to a separate Tool Installer task
If you like my extensions, please leave a review or a donation. If you'd like to see a feature I haven't built, file an issue or, better yet, send me a pull request.

Clean up your Team Project Collection prior to migrating to VSTS

To prepare your TFS Project Collection for migration, you may want to remove (stale) old data to reduce the database size first/
Most actions are already documented here. Queries that can aid in detecting where your space is allocated are also found in this recent support ticket.

Delete old workspaces

Deleting workspaces and shelvesets can reduce your migration and upgrade times considerably. either use the tf commandline or leverage a tool like the TFS SideKicks to identify and delete these.

Build results

Not just build results, but often overlooked the actual build records can take up a considerable amount of data. Use tfsbuild destroy (XAML) to permanently delete the build records. In the past, I've encountered clients who had 1.8 million "hidden" builds in their database and removing them shaved off quite a considerable amount of data. These records were kept around for the warehouse.

Old team projects

Of course, destroying old team projects can give back a lot of data. Anything you don't need to send to azure helps. You could also consider splitting the collection and to leave behind the old projects. That will give you the option to detach that collection and store it somewhere, should you ever need that data again.

Redundant files

Deleted branches are a very common hidden size hog. When deleting things in TFVC, they are not actually deleted, they're just hidden. Finding deleted files and especially old development or feature branches can give you back a lot of data. Use tf destroy to get rid of them.
You may also want to look for checked in nuget package folders, those can quickly rack up a lot of space as well.

Test Attachments

Ohh yes, especially when you use test attachments, these can grow like crazy, depending on your TFS version either use the built-in test attachment cleanup features or use the Test Attachment Cleaner from the TFS power tools.

XAML Builds

The build definitions themselves won't take a lot of database space, but the build results may. But those have been covered in a previous section.
In the past, I've had to patch tfbuid.exe to handle (very) large amounts of build records, as it tends to try and fetch all build data locally before proceeding with the delete action. You may need to rely on the TFS Client Object Model to achieve a similar result.

Git Repositories

You may have data in your git repositories that are no longer accessible due to force pushes or deleted branches. It's also possible that certain data in Git could be packed more efficiently. To clean your repositories you have to clone them locally, clean them up, delete the remote repo from TFS and push the cleaned copy to a new repository (you can use the same name as the old one). Doing this will break references with existing build definitions and you will have to fix these up. While you're at it, you could also run the BFG repo Cleaner and convert the repositories to enable Git-LFS support to handle large binary files in your repositories more elegantly.
git clone --mirror <>
# optionally run BFG repo cleaner at thi s point
git reflog expire --expire=now --all 
git gc --prune=now --aggressive
git repack -adf
# Delete and recreate the remote repository with the same name
git push origin --all
git push origin --tags

Work item (attachments)

Work items can gather up a considerable amount of data, especially when people start attaching large attachments to them. You can use witadmin destroywi to delete work items with unreasonably large attachments. To retain the work item, but delete its attachments you can delete the attachments from the current work item and then clone it. After cloning, destroy the old work item to allow the attachments to be cleaned up.
Old work items that you no longer need (say the sprint items from 6 years ago) can also be deleted. My colleague RenĂ© has a nice tool that allows you to bulk-destroy by first creating the appropriate work item query.

Be sure to run the cleanup jobs

TFS often doesn't directly prune data from the database, in many cases, it just marks stuff as deleted for latest processing. To force the cleanup to happen immediately, run the following stored procedures on your Project Collection database:
EXEC prc_CleanupDeletedFileContent 1
# You may have to run the following command multiple times, the last
# parameter is the batch size, if there are more items to prune than the 
# passed in number, you will have to run it multiple times
EXEC prc_DeleteUnusedFiles 1, 0, 100000

Other useful queries

To identify how much data is stored in each section, there are a few useful queries you can run. The actual query depends on your TFS version, but since you're preparing for migration I suspect you're on TFS 2017 or 2018 at the moment.

Find the largest tables:
SELECT TOP 10 o.name, 
SUM(reserved_page_count) * 8.0 / 1024 SizeInMB,
SUM(CASE 
WHEN p.index_id <= 1 THEN p.row_count
ELSE 0
END) Row_Count
FROM sys.dm_db_partition_stats p
JOIN sys.objects o
ON p.object_id = o.object_id
GROUP BY o.name
ORDER BY SUM(reserved_page_count) DESC
Find the largest content contributors:
SELECT Owner = 
CASE
WHEN OwnerId = 0 THEN 'Generic' 
WHEN OwnerId = 1 THEN 'VersionControl'
WHEN OwnerId = 2 THEN 'WorkItemTracking'
WHEN OwnerId = 3 THEN 'TeamBuild'
WHEN OwnerId = 4 THEN 'TeamTest'
WHEN OwnerId = 5 THEN 'Servicing'
WHEN OwnerId = 6 THEN 'UnitTest'
WHEN OwnerId = 7 THEN 'WebAccess'
WHEN OwnerId = 8 THEN 'ProcessTemplate'
WHEN OwnerId = 9 THEN 'StrongBox'
WHEN OwnerId = 10 THEN 'FileContainer'
WHEN OwnerId = 11 THEN 'CodeSense'
WHEN OwnerId = 12 THEN 'Profile'
WHEN OwnerId = 13 THEN 'Aad'
WHEN OwnerId = 14 THEN 'Gallery'
WHEN OwnerId = 15 THEN 'BlobStore'
WHEN OwnerId = 255 THEN 'PendingDeletion'
END,
SUM(CompressedLength) / 1024.0 / 1024.0 AS BlobSizeInMB
FROM tbl_FileReference AS r
JOIN tbl_FileMetadata AS m
ON r.ResourceId = m.ResourceId
AND r.PartitionId = m.PartitionId
WHERE r.PartitionId = 1
GROUP BY OwnerId
ORDER BY 2 DESC
If file containers are the issue:
SELECT CASE WHEN Container = 'vstfs:///Buil' THEN 'Build'
WHEN Container = 'vstfs:///Git/' THEN 'Git'
WHEN Container = 'vstfs:///Dist' THEN 'DistributedTask'
ELSE Container 
END AS FileContainerOwner,
SUM(fm.CompressedLength) / 1024.0 / 1024.0 AS TotalSizeInMB
FROM (SELECT DISTINCT LEFT(c.ArtifactUri, 13) AS Container,
fr.ResourceId,
ci.PartitionId
FROM tbl_Container c
INNER JOIN tbl_ContainerItem ci
ON c.ContainerId = ci.ContainerId
AND c.PartitionId = ci.PartitionId
INNER JOIN tbl_FileReference fr
ON ci.fileId = fr.fileId
AND ci.DataspaceId = fr.DataspaceId
AND ci.PartitionId = fr.PartitionId) c
INNER JOIN tbl_FileMetadata fm
ON fm.ResourceId = c.ResourceId
AND fm.PartitionId = c.PartitionId
GROUP BY c.Container
ORDER BY TotalSizeInMB DESC

Original

TFS 2018 update 1 is available. So is the TFS Aggregator

2018-02-21

Biran Harry just announced the Release to Web of Team Foundation Server 2018 update 1. With mostly bug fixes and a few very small functional features. Luckily it shipped with no breaking changes in the server object model, so we were able to quickly ship a compatible version of the TFS Aggregator.

We had updated our solution structure and build process to make it easier to support multiple TFS versions. The TFS Aggregator solution now supports the following TFS versions:

  • Team Foundation Server 2013 update 5
  • Team Foundation Server 2015 
  • Team Foundation Server 2015 update 1
  • Team Foundation Server 2015 update 2
  • Team Foundation Server 2015 update 3
  • Team Foundation Server 2017 and update 1
  • Team Foundation Server 2017 update 2 and update 3
  • Team Foundation Server 2018 and update 1

All from a single solution. To support this we've upgraded to Visual Studio 2017 15.5. Its support for packageReferences in the project files allows us to dynamically switch between Nuget packages and to switch between different versions of the same NuGet packages. This greatly simplifies building a single codebase against a large number of dependent package versions.

You can find Beta 4 of the TFS Aggregator on the GitHub releases page.

Make a Group Team Administrator in VSTS/TFS

2018-02-19

In case your organisation is depending on an Identity and Access Management tool to assign rights to the tools in the toolchain, then it's likely that all of your permissions are handled through Group assignments and not to individuals.

Visual Studio Team Services and Team Foundation Server allow rights assignments to groups in almost every possible location and assigning those rights is doable through the TFSSecurity utility. Almost every location... It turns out that assigning a group to be a Team Administrator was not (is not in TFS) possible through the UI.

In the past, I've contributed to the TFS Team Tools from the ALM Rangers. That project had since been deprecated, though it still has a few unique features. I've decided to resurrect it and you can now find it on my GitHub account. While I was at it, I incorporated the functionality to list, add and remove Team Administrators. It supports adding individuals as well as groups.



You can now download a version of the TFS Team Tools from the GitHub.

C:\Tools\TfsTeamTools>tfsteams addteamadministrator /team:"Demo Team" 
    /collection:https://jessehouwing.visualstudio.com /teamproject:demo /user:"[Demo]\Portfolio Team"

C:\Tools\TfsTeamTools>tfsteams listteamadministrators /team:"Demo Team" 
    /collection:https://jessehouwing.visualstudio.com /teamproject:demo
[demo]\Portfolio team
Jesse Houwing

 

Most Reading

Archives