Binary promotion of Visual Studio Team Service Extension

2016-02-01

When creating a Visual Studio Team Services extension (such as a collection of build tasks), it makes sense to publish it as a private extension first. You can do this by overwriting the extensionid from the commandline, but there is no direct commandline switch to override the visibility of an extension. I used to patch the extension manifest, but today finally uncovered the correct way of doing it.

It turns out that the TFS cross platform commandline (tfx) has a --override option which you can pass a piece of json to which will override any json in the extension.

So if your standard publish command looks like this:

tfx extension publish --extension-id vsts-tfvc-tasks --publisher jessehouwing --vsix package.vsix

Then the command to publish the private test version would look like this:

tfx extension publish --extension-id vsts-tfvc-tasks-TEST --publisher jessehouwing --override "{""gallerySpecs"": [ ""Private"", ""Preview""], ""public"": false }" --share-with jessehouwing

Instead of passing the json on the commandline, you can also specify an --overridesFile.

Or, on the safer side, make the extension-manifest you commit to your source control repository always do a private preview release, and then use the --override option to merge the real publish settings.

Note: ensure you escape every double quote in the JSON by doubling it up.

Now I can setup a Visual Studio Release Management template to publish the tasks, test against my personal account and then promote that vsix unmodified to the marketplace. But that's for the another blog.

Publish Build Tasks to TFS 2015 without configuring Basic Auth on the server

2016-01-30

When you want to publish build tasks to a TFS 2015 server, you need a Basic Authentication enabled endpoint. Unless your server is configured with SSL, this is a major security concern. Until now the guidance I received was to setup a second Application Tier, configure it with Basic Auth and configure it to only accept traffic from localhost. The sole purpose of that machine is then to upload build tasks and later probably extensions to a TFS 2015 server.

Today I found a neat little trick that removes this need. Fiddler to the rescue!

Start Fiddler and enable it's option to auto-authenticate:

Mimic the XAML build's Output Location option in Build 2015

2016-01-26

If you've been using XAML builds in the past, you've probably discovered that, by default, it drops all the outputs of all your projects into one folder called the Binaries Folder. Unless you do anything using a post-build script, the contents of that folder will be copied verbatim to the drop location.

In TFS 2012 a new option appeared giving you a little more control over the output location; giving you 3 options:
  1. Single Folder (like it worked in 2010)
  2. Per Project (Creates a folder per solution)
  3. As Configured (don't redirect output, so the default would put the binaries under your project folders, as it would on your local machine by default).
If there's one feature of XAML builds which confuses the hell out of people, it's the Output Redirection. At first people don't expect this to happen (they expect As Configured to be the default) and then the naming of "Per Project" confuses people even more, as it turns out that it ends up creating a subfolder for each Solution or msbuild .proj file (based on the solution name).

Build 2015 takes away this issue, by having As Configured to be the default and by offering a simple Copy and Publish Artifacts task which collects the project output and copies it to the staging location. One of the nice advantages of this setup, is that it also makes it easier to do incremental builds, speeding up your CI builds.

But, if you're coming from a XAML build and you already have a lot of existing post-build steps that assume all your files to be in a specific folder or set of folders, it may be easier to replicate that behavior than to rewrite all these scripts.

Let us go through the three options in XAML build and let's see how you'd handle these in Build 2015.

As Configured

This is the default behavior in Build 2015. It does whatever your project files have configured. If you are sing the default project templates of Visual Studio, that means that files are dropped in the "Project\Bin\Configuration" folder.

You'd use a standard Visual Studio Build or MsBuild task followed by a Copy and Publish Artifacts task to recursively gather the output and copy them to the Staging Location and from there to the Drop Location.


The default configuration of the Copy and Publish Build Artifacts task are to scan the $(Build.SourcesDirectory) recursively and grab the contents of every "\bin\" folder.

When you use As Configured in XAML builds, nothing gets copied to your drop folder, unless you actually do that. The Copy and Publish Artifacts task will do this for you. Or you can run your own powershell, batch or other script to copy your files to the $(Build.ArtifactStagingDirectory). Either pass the target location using a argument or grab it from inside your powershell script using the environment variable $env:Build_ArtifactStagingDirectory:

Then call the Publish Build Artifacts task to publish them (note don't want to use the Copy and Publish task here, since your script has already copied the files).


Single Folder

When specifying Single Folder in XAML builds, all the binaries for all your projects will end up in the root of the drop location. This was the default in Team Foundation Server 2010 and caused a lot of confusion when people started adopting Team Build back in the days.

There are two ways to mimic this behavior. One is to build your solution with no additional options and then use the default Copy and Publish task. I assume that you may be building multiple configurations here:


If you want to grab all the folders under bin, you could simplify this further to: **\bin\.

This solution yields the same result, but may still cause issues with your current scripts, which may assume that msbuild redirects all outputs to a single folder. It's not hard to create a situation that's even closer to the old XAML build's functionality.

The first thing you need to do is instruct the Visual Studio Build task to redirect the build output to the $(Build.BinariesDirectory).


Then simply copy everything from under $(Build.BinariesDirectory) to the staging location and publish it:


The advantage of using the intermediate Binaries Directory, is that you can do incremental builds, if you were to build directly to the $(Build.ArtifactStagingDirectory) the output will be thrown away after each build, increasing your build time when you're not using Clean Repository and Clean Build.

Per Project

The third option in the XAML build allows you to create a subfolder for each... and here it gets confusing... Selected solution. It's called "Project", but I think this dates back from the 2008 era when you'd build an MsBuild Proj file. 

There is no easy way to do this in Build 2015. You could use Build Multiplexing or add multiple Visual Studio Build tasks followed by multiple Copy and Publish Build Artifacts tasks. Basically taking the technique from the SingleFolder option and adding in the name of the solution hardcoded in each task or through a custom variable.

Manipulating your TFVC repository from Build 2015

Though generally considered a bad practice I crafted two build tasks to manipulate your TFVC repository from the build. You can now edit, delete and add files and check those changes in when you're ready.

Before I explain how they work and how you can use them, I need to get the following out of the way:

I do not think it's a good idea to change your sources as part of a build.
Unless that's the only thing your build definition does.
In these cases it should not build your sources and produce your shipping artifacts.

The reason why I'm reluctant, is that changing your sources during the build has impact on other features of the Visual Studio Team Services platform. 
  1. As your code is being built, certain artifacts are generated, such as your debug symbols, test results, code coverage etc. And these artifacts are associated with the Changeset number of the build. This may break or cause problems with Remote debugging, Source Server support, Intellitrace and Test Impact Analysis among other features. Other tools that may be impacted are obfuscators.
  2. Since your scripts change files, features such as Incremental builds won't work. They'll always have to rebuild the projects, since there are always modified files. This is especially so when you use tasks to patch the AssemblyVersion attribute. This will slow down your builds slow down the feedback cycle as a result.

Building multiple configurations and/or platforms in Build 2015

2016-01-11

In XAML builds you could specify one or more configurations to build, if you left these values empty, the solution default would be built. Unless you were using a customized build template, each build|configuration pair would be built and tested in sequence.


The Build Agent will create a sub-folder for each Platform|Configuration if you specify multiple values.

In Build 2015, you can also build multiple configurations and/or platforms. And what's pretty cool, if you have multiple build agents available, you can build and test these in parallel. If you have a single agent, or are using the Hosted build with the standard build minutes, it will build all combinations in sequence.

Upgrading from XAML to Build 2015 with minimal changes

2016-01-10

This series is currently a work in progress

XAML builds on Visual Studio Team Services will be deprecated in September 2016. While support for XAML builds seems to stay around on-premise for a little while longer, it's time to upgrade to Build 2015.

You may have invested quite a bit in the XAML build system in the past, but as long as your changes have been mostly limited to custom MsBuild tasks and targets, Powershell or batch scripts, then upgrading to Build 2015 is (almost) a piece of cake.

In this series I'll go through the basic settings in the XAML Default Template and I'll explain how to transfer each setting over to the new system. There are still a few things which are not possible in the new system, such as configuring Gated Checkins for TFVC, but almost every other setting can be converted over.

Before going over each setting, I'll assume that you've installed at least one build agent, or are using Visual Studio Team Services with the Hosted Build option.
Other tips and tricks

Access multiple TFVC repositories in a Build 2015 definition

2016-01-08

When you have a TFS project collection with many Team Projects, it's a common practice to have a "shared" project containing a collection of commonly used libraries. This library may consist of Checkin policies, Build Activities, MsBuild tasks and Custom MsBuild targets. The alternative used to be to check in these files with each team project, running the risk of them getting fragmented quickly, leaving you in a situation that is very hard to maintain.



Nowadays I'd opt to put most of these dependencies in a vsix or in nuget package and distribute them using a custom gallery or the upcoming Package Management features in Visual Studio Team Services. 3rd party solutions like ProGet and MyGet can also fill in this demand.

Unfortunately, when migrating from one system to another, you don't get the option to change the whole solution structure or the way 3rd party dependencies are retrieved all at once, so you may need to mimic the old "shared repository" pattern when migrating build from XAML over to Build 2015.


 

Search This Blog

Loading...

Most Reading