build

Build Automation using Jenkins and TFS version control

March 26, 2014 TFS, Visual Studio , , , ,

I’ve used several version control systems for my source code and research projects.  Recently, I’ve consolidated them to TFS and Git. I have started using Visual Studio Online for my research work, CodePlex for my Open Source Frameworks and Tools and GitHub for my blog code.  Much of these decisions are driven based on the target audience and the nature of my work (code).  TFS, being an excellent ALM tool, offers more than just a version control system which can be particularly helpful when you are either having large teams to work with or have an open-source project whose build you would want to automate.  For my research development (that happens at home), I was keen on build automation for CInject project.

Visual Studio Online offers a complete package of Continuous Integration and Delivery to Azure Services but there is no such feature for TFS repositories of CodePlex.  For this case, I started evaluating other CI platforms like CruiseControl.NET, TeamCity and Jenkins.  I zeroed down on Jenkins for its ease of use, extremely light-weight deployment, my familiarity with it and zero-dollar cost in setting it up.

For this example, we would use CodePlex repository, but the method can be used with any on-premise TFS repository or even with Visual Studio Online

Downloading the latest version

A direct link to Windows Version of Jenkins is available here.  Installation of this will require administrative rights if you are doing it on Windows 7+, or Windows Server 2008+

Once the installation has been completed, you will have a service running in the Service Manager

image

 

If this service is running, you can access Jenkins console at:  http://localhost:8080/

Download TFS Plugin

Visit the Manage Plugin Page [Manage Jenkins > Manage Plugins > Available ] (shortcut) and search for Team Foundation Server plugin.  Click on the checkbox and press the button ‘Install without restart’

image

If your build machine/server has restricted access to the Internet, you will get an error downloading the file

java.io.IOException: Failed to download from http://updates.jenkins-ci.org/download/plugins/tfs/3.1.1/tfs.hpi
    at hudson.model.UpdateCenter$UpdateCenterConfiguration.download(UpdateCenter.java:781)
    at hudson.model.UpdateCenter$DownloadJob._run(UpdateCenter.java:1125)
    at hudson.model.UpdateCenter$InstallationJob._run(UpdateCenter.java:1285)
    at hudson.model.UpdateCenter$DownloadJob.run(UpdateCenter.java:1103)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104)
    at java.lang.Thread.run(Unknown Source)

In such cases, you need to download latest version of the plugin manually and copy it to

<<JenkinsInstallDirectory>>\plugins

image

You will need to restart Jenkins service if you had to install it manually.

Downloading Team Explorer Everywhere

If you are running this process on your desktop machine and you are a .NET developer, chances are that you would have Visual Studio 2012+ installed.  In such cases, you will have Team Foundation executables on your machine and also the path of TF executables will be mentioned in PATH system variable as well.

When TF executable path is not part of PATH system variable, you will have to configure Jenkins to look into the right directory (shortcut).  On my home laptop, I’ve configured it as,

image

If you are running this on your Windows Server which has only .NET framework installed, you will have to install TEE.  If you are running this on a Unix/Linux server, you will still need TEE.  This is a free executable from Microsoft and is very thin-client.  You can download and install it from here and setup the configuration as

X:\Jenkins\TEE-CLC-12.0.1\TEE-CLC-12.0.1\tf.cmd

You would need to run tf eula on Command Prompt to accept the Agreement of Use and Terms and Conditions and press ‘y’ when it prompts.  Similar configuration needs to be done for Unix / Linux.  The same CMD file will work for Unix and Linux platforms.

Setting up TFS repository in Jenkins

Click on the New Job link on the left menu to create a new Build Profile (shortcut)

image

On the next screen, you can customize the configuration to enable concurrent builds, retry count, etc. But for this article, the focus would be to configure TFS workspace.  So to get the TFS configuration, visit your CodePlex framework page and get the TFS URL.  If you are configuring for a repository other than CodePlex, you can check the URL with your administrator

image

Copy the link to Jenkins as shown below and click on Apply

image

Now you can trigger Build for environment and it should build successfully. 

You can use this method with any TFS repository – On-premise as well as Visual Studio Online.  One thing worth noting here is – every TFS repository (here https://tfs.codeplex.com:443/tfs) has multiple collections (TFS01…. TFSxx) and each collection has projects.  The Server URL should include the name of the collection, whereas the Project path should be a relative path ($/…) to the actual project.  

Using CodePlex SVN Path with Jenkins

You can use SVN link to access the Codeplex repository as well.  Please note that if you are using a non-Codeplex repository, you may not have a SVN URL.

image

So you can opt for any method if you are using Jenkins for CodePlex and can setup Build Automation and Continuous Integration as well.

Hope this helps.

[Solved] TF400324: Team Foundation Services are not available from server

March 13, 2014 ALM, TFS, Visual Studio , , , , ,

Possible Errors

TF400324: Team Foundation services are not available from server <Server/CollectionName>.
Technical Information (for administrator): Page not found

or

TF400324: Team Foundation services are not available from server <Server/CollectionName>.

Technical Information (for administrator): Unable to connect to the remote server

 

Root cause

This error occurs primarily due to corrupt Team Foundation Server cache.  There are several reasons why the cache can go corrupt

  • Abrupt closure of TFS services on your machine
  • Multiple versions of TFS (2010/2012/2013) configured on your machine
  • Upgrade in the version of TFS at server end, but pending upgrade on client machine
  • Corrupted installation of TFS client

Validate the issue

If you are working in a team, it is preferred to have it checked with another colleague of yours if he/she is facing the same issue.
If you are working alone on a hobby project (like I am doing), you can go ahead with solution

Solution – Clearing of cache

  • Close all instances of Visual Studio
  • Open Task Manager and check if any TFS Services are running.  Select each of them and click on End Process Tree
  • Browse to the folder below and delete all the contents and folders%LocalAppData%\Microsoft\Team Foundation\4.0\Cache
    %LocalAppData%\Microsoft\Team Foundation\5.0\Cache
  • Restart Visual Studio and try triggering build

If you are switching from one version of TFS to another, you may require repeating this solution again.  If you are using older version of TFS, you may have to navigate to

%LocalAppData%\Microsoft\Team Foundation\

and find out the version folder instead of navigating directly.

Post Solution Action

Post clearing of cache, you will be prompted to rebind your solution to TFS.  Don’t worry this would not let go of your local un-checked-in changes.

Team Foundation Server : Build Logging

January 9, 2014 ALM, TFS, Visual Studio , , , ,

When you use Team Foundation Server for Build Automation, you can configure build controller and agents.  In the Team Foundation Administration Console, you can view the Logs however these logs provide minimal information.  Even the Event Viewer provides minimal event logging rather than extensive logging.  Now even if these Logs or Event Viewer provide you information that you require, there may be cases that you do not have complete access to the build server and this build server is managed by your organization’s support groups.

In this case, you can enable detailed level logs of build service so that you can read the logs in Read Only mode without having to login to the server.  Note that this log file will have build information of all agents running on the controller and not specific to an agent.  So if you are using a Shared Build Controller, everyone with access to this log file will have information about your builds too.

To get started, you need to manually create a configuration file with following content,

  1. <configuration>
  2.     <system.diagnostics>
  3.         <switches>
  4.             <add name="BuildServiceTraceLevel" value="4"/>
  5.         </switches>
  6.         <trace autoflush="true" indentsize="4">
  7.             <listeners>
  8.                 <add name="myListener" type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,Microsoft.TeamFoundation.Common, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" initializeData="c:\Build\TFSBuildlogs\TFSBuildServiceHost.exe.log" />
  9.          <remove name="Default" />
  10.             </listeners>
  11.         </trace>
  12.     </system.diagnostics>
  13. </configuration>

 

Name this file as TFSBuildServiceHost.exe.config

You can customize this configuration file by adding more trace listeners or even changing the location of build log file from C:\Build\TFSBuildlogs (create if it does not exist).  If you are sharing this path with multiple teams, please make sure you only give Read Only rights to all parties.

As the name suggests, this is a configuration file for TFSBuildServiceHost.exe.  On your build server, you can locate this executable at –

%programfiles%\Microsoft Team Foundation Server 2010\Tools

You can copy this file in the above path.  If you are using Windows Server 2008/2012, it will validate your Administrative Privileges with UAC.

Ensure that no builds are in active state.  Start the Command Prompt with ‘Run As Administrator’ privileges and execute these commands

net stop tfsbuildservicehost
net start tfsbuildservicehost

This will restart the TFSBuildServiceHost service and voila! The log files will detailed logging will be created at C:\Build\TFSBuildlogs

Note: This has been tested on TFS 2012 / 2013.  I’ve not tested this on older versions of TFS.

Team Foundation Server : Build Automation

November 25, 2013 ALM, TFS, Visual Studio , , , ,

This article will focus on achieving build automation using TFS.  However, if you want to understand the architecture, configuration, project creation, product backlog, managing SPRINT and releases etc. you can go through these articles

All the examples, referred below, use Team Foundation Service (cloud-based TFS).  Some of the settings may be overridden by your TFS Administrator if you are using TFS on-premise (in your organization).

 

What is build automation and do I really need it?

Build automation as the word implies means automating the build process.  But the build automation does not end at compiling your code.  It encompasses several tasks such as

  • compiling source code (taken directly from a repository, here TFS)
  • packaging into binary code
  • running tests (optional)
  • deployment of binary code to production environment (optional)

When your team size is small (1-2 developers), you can compile the code on your machine to verify the accuracy of your and probably your team’s code.  But in large teams, it becomes essential that only code checked-in is the code that compiles and quality of code is not compromised.  This indirectly saves time and cost for the organizations.  Build automation allows you to configure set of schedules and processes to ensure that the above objectives are met.  You can have your code built on demand, or schedule it at a particular time (nightly-builds, incremental builds, etc.) or have gated-build to be triggered when any check-in is done.

Scheduled and gated-builds are examples of Continuous Integration (CI)

What build automation does not guarantee?

Bug-free code.  It ensures that your code compiles but does not ensure that your code is bug-free

How do we achieve Build Automation with TFS?

Build automation in TFS is achieved by running automated MSBuild scripts for your projects.  Generation of scripts for most of the common tasks is automated and you need to write script for only uncommon, complex tasks.  So let’s first see how to configure the common tasks

As a first step, you need to connect to a Team Foundation Server.  In this example, I am connected to my CInject repository.  Once you are connected to Team Foundation Server, you can go to your Team Explorer window in Visual Studio.

image

You can click on Builds and then on New Build Definition

image

If you do not have sufficient privileges to create build definition, you will get an error message (as below).  You can request your TFS Admin to grant you build rights (ManageBuildResources).

image

When the New Build Definition dialog opens, follow through these steps.  You need to create this definition once and all your team members will be able to view them.

image

On the General tab, choose a build definition name.  As a best practice, the build definition name should depict nature of build (nightly, gated), environment and application name

On Trigger tab, you will have following options

  • Manual – As the name suggests, you can trigger manual builds on the build server
  • Continuous Integration – Trigger a build when code is check-in
  • Rolling Builds – Trigger a build at every X minutes
  • Gated check-in – Trigger a build before check-in.  This ensures that only error-free code gets checked-in
  • Schedule – Schedule a build at a particular time of a day

On Source Settings tab, you can choose the working folders.  It is advisable to choose the closest branch in which the code resides as a working folder.

On Build Defaults tab, you can define your build controller and staging locations

  • When using Team Foundation as Service (cloud-based TFS), the build controller will have only one value in the drop down ‘Hosted Build Controller’. When using Team Foundation Server (on-premises), you will have to configure a build controller for yourself.  We will deal with building your own build controller in a separate article.
  • For staging locations, you can have the build output checked in to TFS directly or to a shared drive, or ignore the build output. 

    If you are staging it to a shared drive you need to make sure that the Service Account / Active Directory Account through which the build service is running on the build server has Full Control on the shared location

On Process tab, you can define your own build process.  We will deal with the customization of process template in a different article.  When using Team Foundation as Service (cloud-based TFS), you will have following templates

  • Default Template
  • Upgrade Template – When upgrading from older version of TFS
  • Azure Continuous Deployment – Build and deploy websites and services to Azure Cloud platform. Read more
  • Tfvc Template – Team Foundation Version Control template
  • Git Template – GIT version control template

Each template will give you different options to configure.  Some of the common options are

  • Version Control – Clean Workspace, Get Version, Label Sources
  • Build –
    • Projects – You can select solution file or individual projects that need to be built
    • Configurations – You can select a configuration and platform here.  The configuration selected here should be available under Configuration Manager of your solution otherwise the build will fail.  This can also be useful when you have configuration transforms for app.config / web.config files
    • Output Location – In TFS 2010, the build output of all the selected projects was always copied to a single folder.  With TFS 2012+, you can choose of the options – single folder, per project, or as configured in individual projects.
    • Advanced – This setting allows you to pass additional parameters to MSBuild, configure pre-build and post-build scripts and enable/disable code analysis.  For the uncommon complex tasks you can check-in your pre-build and post-build scripts in TFS and change the settings in Advanced section.  You can also choose to run FxCop to perform code analysis on your projects
  • Test – You can configure your test projects here.  By default, MSTest will be used as engine for unit-testing.  If you want to disable unit-test runs, you can set Disable Tests to true
  • Advanced -  Advanced allows you to configure your build agents and post-build behaviour. If you want to create a Build Bug for every failed build, or want to update Work Items with the build number, you can set it up in this setting

On Retention Policy tab, you can define how long you would want to retain build outcome. In your organization, this may be driven by some audit/compliance requirements or by best practices.

Once you have configured the build definition, you can view the build definitions in the same Team Explorer

image

You can Queue New Build, Edit Build Definition or even View Controller Queue from Team Explorer.  Any on-going build will be visible under ‘My Builds’.  You can double click on an on-going build to view its status.  With large teams, you can use Build Notifications tool that minimizes itself to system tray and pops-up with build notifications such as Build Triggered, Started, Completed, Failed.  This is very handy specially when you have large teams spread across geographies.

Hope this helps in understanding the process of creating a build with TFS.

Follow on Feedly