ALM

Continuous Delivery with Visual Studio Online and Azure

June 18, 2014 Sessions , , , ,

On June 26th 2014, I will be delivering a talk on Continuous Delivery with Visual Studio Online and Azure for a Brazil User Group. This will be a webcast for 1 hour and its agenda would be as outlined below. This webcast is a must for those interested in understanding Visual Studio Online and its ALM features that can be used to have a Continuous Delivery into Azure (cloud) platform. Agenda:

  • Introduction to Visual Studio Online and its features
  • ALM – Requirements Gathering, Construction, Unit Testing
  • Continuous Delivery – Build Automation and Deployment
  • Q&A

The webcast will be tailored considering how both Visual Studio Online (or its on-premise version) and Azure can be best used for enterprises to achieve efficiency and maturity in their product life cycle.

For any questions, you can contact me over twitter: @ganshani

or you can also contact Jucinei Santos (the organizer of this webcast)

To register for this webcast, please visit https://www.eventbrite.pt/e/continuous-delivery-with-visual-studio-online-and-azure-tickets-11762527035

[Solved] TF246017: Team Foundation Server could not connect to the database

April 25, 2014 ALM, TFS, Visual Studio , , ,

Error

TF246017: Team Foundation Server could not connect to the database.
Verify that the server that is hosting the database is operational, and that network problems are not blocking communication with the server.

Root Cause

This error can occur due to following reasons

  • Incorrect SQL Installation order especially if you are using Express edition
  • Database In Recovery
  • Database Out-of-Sync – during migration to newer version of TFS
  • Permission of Service Account is not right
  • SQL Database not running or not reachable
  • TFS Proxy not reachable

Solution

If your database server has restarted and you are trying during that time, you may face this issue. You may want to take a cup of coffee and retry after sometime.  If the error still persists, you can try debugging the below.

Check if you are able to connect to SQL database by yourself.  If you are not aware of the database connection, it is advisable to reach out to your support team in your organization to check the connectivity. 

If this issue arises only when you are checking in or doing an undo pending changes, chances are that SQL Server has logical consistency I/O error.  You can try repairing the database (or request your DBAs to do it for you) by running following command against all TFS databases

DBCC CHECKDB

If there are database errors, your DBA should be able to resolve it. 
If there no database errors, then you need to check if the service account through which the TFS services are running (on TFS server) has same credentials and if the permissions granted are correct.  You may want to sync-up the services

If you have migrated TFS / database, you may want to check if backup of all databases is restored.  Partial restoration of backups also can cause this issue.  You can refer to the MSDN link of list of TFS databases.  This list may differ based on version of TFS.

[Solved] TF400012: Full-text search failed

March 19, 2014 ALM, TFS, Visual Studio , , ,

Error

TF400012: Full-text search failed. Contact your Team Foundation Server administrator or try the operation again.
(Code 30046: SQL Server encountered error 0x8007042d while communicating with full-text filter daemon

Root Cause

SQL Full-text Filter Daemon Launcher may have shutdown

Solution

This error occurs only when the Daemon service is down.  TFS and its SharePoint site uses Microsoft SQL database for storing code.

image

Please note that the actions below have to be performed on the Data Tier of your TFS architecture.  Your data tier may reside on a data-farm if your TFS has multiple deployments.  Also if your TFS tiers are behind the organization firewall, you may be restricted in performing these actions.

To enable full text search, you can enable/configure SQL Full-text Filter Daemon Launcher (instance name) using following steps –

  • On the Start menu, point to All Programs, point to Microsoft SQL Server, point to Configuration Tools, and then click SQL Server Configuration Manager

  • In SQL Server Configuration Manager, click SQL Server Services, right-click SQL Full-text Filter Daemon Launcher (instance name), and then click Properties

  • Click the Log On tab of the dialog box, and then select or enter the account under which to run each process created by the SQL Full-text Filter Daemon Launcher service. If you choose not to use the Local Service account, then we recommend that you specify a low-privileged local user account with User rights and permissions on the local computer.

  • After you close the dialog box, click Restart to restart the SQL Full-text Filter Daemon Launcher service.

[Solved] TF400324: Team Foundation Services are not available from server

March 13, 2014 ALM, TFS, Visual Studio , , , , ,

Possible Errors

TF400324: Team Foundation services are not available from server <Server/CollectionName>.
Technical Information (for administrator): Page not found

or

TF400324: Team Foundation services are not available from server <Server/CollectionName>.

Technical Information (for administrator): Unable to connect to the remote server

 

Root cause

This error occurs primarily due to corrupt Team Foundation Server cache.  There are several reasons why the cache can go corrupt

  • Abrupt closure of TFS services on your machine
  • Multiple versions of TFS (2010/2012/2013) configured on your machine
  • Upgrade in the version of TFS at server end, but pending upgrade on client machine
  • Corrupted installation of TFS client

Validate the issue

If you are working in a team, it is preferred to have it checked with another colleague of yours if he/she is facing the same issue.
If you are working alone on a hobby project (like I am doing), you can go ahead with solution

Solution – Clearing of cache

  • Close all instances of Visual Studio
  • Open Task Manager and check if any TFS Services are running.  Select each of them and click on End Process Tree
  • Browse to the folder below and delete all the contents and folders%LocalAppData%\Microsoft\Team Foundation\4.0\Cache
    %LocalAppData%\Microsoft\Team Foundation\5.0\Cache
  • Restart Visual Studio and try triggering build

If you are switching from one version of TFS to another, you may require repeating this solution again.  If you are using older version of TFS, you may have to navigate to

%LocalAppData%\Microsoft\Team Foundation\

and find out the version folder instead of navigating directly.

Post Solution Action

Post clearing of cache, you will be prompted to rebind your solution to TFS.  Don’t worry this would not let go of your local un-checked-in changes.

Team Foundation Server : Removing a Collection (client side)

March 5, 2014 ALM, TFS, Visual Studio ,

There are 2 ways to delete and reconfigure a TFS collection on your desktop (or local machine).

Using Visual Studio. 

Using Team Explorer, you can first click on Connect.  If you are already connected to a collection, you will not be able to delete it.  You will have to Disconnect by clicking on the Context Menu.

Once disconnected, you can click on Remove.  This method works most of the time.  However if you have many such collections configured, then you can use the registry mechanism. 

Also, please note that Team Explorer does not a collection unless and until it has a project associated with it.  So to delete that you will have to use Registry Mechanism

image

Using Registry.

Team Explorer stores the configured collections in registry folder

HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\12.0\TeamFoundation\Instances

You can delete one of the sub-folder in Instances to delete the collection reference from Visual Studio

image

In the above snapshot, you can see 3 instances (collections) configured; however, in Team Explorer only 2 were visible as one of the collections does not have any project associated.

If you want to reconfigure your collection, you can again click on Connect in Team Explorer and follow through the wizard.

[Solved] TF271001: Connection timed out type SmtpException

February 21, 2014 ALM, TFS, Visual Studio , , ,

Error

TF271001: The server response was: 4.4.1 Connection timed out type SmtpException.
TFS Email Notifications Delayed or Not Sent

Root Cause

Following are the reasons why this error can occur

  • SMTP server is down or not responding
  • Email notification time/delay has been wrongly set
  • Team Foundation Background Agent has stopped
  • There is a huge backlog of emails pending to be sent out

Solution

The default delay in notifications is 2 minutes.  If you are experiencing more than 2 minutes of delay, you can check for following solutions –

First you should check for availability of SMTP server or if there is a backlog on SMTP server.  Clear the backlog, if exists, and then wait for the backlog in the database to clear. 

You may have to check for availability of a Team Foundation Background Agent service.  You can click on Start menu (or keyboard button) and type services.msc  If the Team Foundation Background Agent has stopped, you can restart it.  Alternatively, you can use following command on Command Prompt to check the status

sc query "TFSJobAgent" | find "RUNNING"

If the issue still does not resolve, you may want to change the email notification time using one of the two methods.  After implementing any of these 2 methods, please restart the TFS IIS AppPool and Background Agent service

You can manually clear the backlog in the TFS database by firing delete commands on tbl_EventNotification and tbl_ClientEvent tables.

Team Foundation Server : Disable Work Item to be marked as Resolved

February 15, 2014 ALM, TFS, Visual Studio , ,

When you check-in a code change using Team Explorer in Visual Studio, you can associate the check-in with one or more work items.

As shown in the snapshot below, the default option that appears against each work-item is ‘Resolve’.  What this means is that when you check-in the changes, the work item status will be changed from Active to Closed.  This also means that you can check-in code later to the same work item, but it’s status will not change.

image

Now, this can be really tricky if you are using TFS for ALM purposes.  You would want to give a right picture to your Team Leads, Scrum Masters and Managers.  So you would want to change the default to Associate.  This can be done by a simple registry tweak

image

I am using Visual Studio 2013, hence the folder is

HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\12.0\TeamFoundation\SourceControl\Behavior\

Value of Key ResolveAsDefaultCheckinAction needs to be set to False instead of True

For other versions of Visual Studio, you will have to replace 12.0 with appropriate version in the folder name

Team Foundation Server : Check-in Policies to ensure traceability

January 13, 2014 ALM, TFS, Visual Studio , ,

One of the things many teams thrive to achieve is 100% traceable code against Business Requirement Document (BRD) or Requirement Matrix.  With evolution of ALM tools, it has become very easy to create traceability matrices and to be able to view history of code and the rational behind design decisions that were taken at the time of its design and development.

So to ensure that you have 100% traceability (even with the most junior member working in your team), you need a set of tools that restrict checking in code without a a work item (or a requirement). But this also means that all your team members and stakeholders have to manage the project (SPRINT, iterations, burn-out etc.) and deliverables using the same tool. 

Assuming that everyone is using same tool and you have created the product backlog, you have to create work items (or requirements, for detailed information read my article on Product Backlog and WorkItems) and assign them to your team members with enough information such as

  • Expected Effort
  • Priority
  • SPRINT / Iteration
  • Description (and if possible, storyboard)
  • Test Cases (in many cases)
  • Sub-tasks (if the work can be divided into small tasks)

It is not necessary to define all the above attributes, but you can agree-upon, within your team, a set of standard attributes for each work-item (or requirement).

Now, the developers can check-in their code without actually associating this with any work-item.  This means even if you have created work-items (and possibly, sub-tasks or linked work-items) there is no link between a work-item and the checked-in code.  So we need some restriction on the action of the developer.  He/She should not be able to check-in without associate it with a work-item.  This is easily achievable in TFS

Whether you are using Visual Studio or Eclipse plugin, you can go to Team Explorer and click on Source Control. 

image

You are then prompted with a dialog where you can choose Check-in policies.  You can click on Add to view the possible Check-in policies.  Please note that this is feature is only available to users marked as Project Administrator and this has to be done once-per-project.

image

As a practice, I prefer to have a minimum two policies for all my collections

  • Changeset Comments Policy – Do not allow the developer to check-in without putting comments
  • Work Items – Do not allow the developer to check-in without associating a work-item

If you are keen on allowing check-in on gated builds (read: continuous integration), you can add “Builds” policy.  Click on OK.  Let’s say you want something more than a comment to be added as well for each check-in. For example, design document name.  You can go to Source Control dialog, click on ‘Check-in Notes’ and add fields in it and mark it as Required.

If any developer forgets to associate a work-item or to put a comment before checking in the code, he/she will be prompted with

image

You can now go to the work-item > History tab to view the code history as shown below

image

This ensures that you have 100% traceability of code with your requirements.  If your Business Analyst or Requirement Gathering personnel uses TFS Web Portal to key-in the requirements (read: create work-items) and the Team Lead is engaged in assigning tasks to the team, you can be rest assured that not a single requirement will be missed out and all stakeholders can get a graphical view of your iterations.

Team Foundation Server : Build Logging

January 9, 2014 ALM, TFS, Visual Studio , , , ,

When you use Team Foundation Server for Build Automation, you can configure build controller and agents.  In the Team Foundation Administration Console, you can view the Logs however these logs provide minimal information.  Even the Event Viewer provides minimal event logging rather than extensive logging.  Now even if these Logs or Event Viewer provide you information that you require, there may be cases that you do not have complete access to the build server and this build server is managed by your organization’s support groups.

In this case, you can enable detailed level logs of build service so that you can read the logs in Read Only mode without having to login to the server.  Note that this log file will have build information of all agents running on the controller and not specific to an agent.  So if you are using a Shared Build Controller, everyone with access to this log file will have information about your builds too.

To get started, you need to manually create a configuration file with following content,

  1. <configuration>
  2.     <system.diagnostics>
  3.         <switches>
  4.             <add name="BuildServiceTraceLevel" value="4"/>
  5.         </switches>
  6.         <trace autoflush="true" indentsize="4">
  7.             <listeners>
  8.                 <add name="myListener" type="Microsoft.TeamFoundation.TeamFoundationTextWriterTraceListener,Microsoft.TeamFoundation.Common, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" initializeData="c:\Build\TFSBuildlogs\TFSBuildServiceHost.exe.log" />
  9.          <remove name="Default" />
  10.             </listeners>
  11.         </trace>
  12.     </system.diagnostics>
  13. </configuration>

 

Name this file as TFSBuildServiceHost.exe.config

You can customize this configuration file by adding more trace listeners or even changing the location of build log file from C:\Build\TFSBuildlogs (create if it does not exist).  If you are sharing this path with multiple teams, please make sure you only give Read Only rights to all parties.

As the name suggests, this is a configuration file for TFSBuildServiceHost.exe.  On your build server, you can locate this executable at –

%programfiles%\Microsoft Team Foundation Server 2010\Tools

You can copy this file in the above path.  If you are using Windows Server 2008/2012, it will validate your Administrative Privileges with UAC.

Ensure that no builds are in active state.  Start the Command Prompt with ‘Run As Administrator’ privileges and execute these commands

net stop tfsbuildservicehost
net start tfsbuildservicehost

This will restart the TFSBuildServiceHost service and voila! The log files will detailed logging will be created at C:\Build\TFSBuildlogs

Note: This has been tested on TFS 2012 / 2013.  I’ve not tested this on older versions of TFS.

Team Foundation Server : Understanding the Architecture

November 23, 2013 ALM, TFS, Visual Studio , , , ,

This article will briefly describe the high-level architecture of Team Foundation Server.  It will help you understand the different tiers in TFS, their responsibilities and firewalls.

Heard of TFS as a Service, what is it?

TFS as a Service is a cloud-based application life management (ALM) system that integrates source control management, product backlog, build automation, quality and deployment and encourages team collaboration to achieve efficiency in following processes.  In its free service offering, you can register on http://tfs.visualstudio.com for an account and get started.  All the infrastructure (all tiers) is pre-configured on an elastic-cloud owned by Microsoft.  As a consumer, you would be provided with services without having to go through the configuration part of TFS.

But we are not allowed to upload our code to public clouds in my organization?

If you are working in an enterprise with restricted environment, chances are that you have your infrastructure on which TFS has been installed.  This is to have enhanced security and sometimes to comply to the laws of the nation where the data resides.

Now in this case, you will have to configure the Team Foundation Server on your own and hopefully this article will serve as a good start.

Architecture

TFS is primarily structured into 3 layers. 

  • Data Tier – TFS uses SQL Server as its data storage
  • Application Tier – The application tier consists of
    • TFS Server – Typically a Windows Server 2008/2012
    • Team Foundation Web Services  -  These web services (http/https) act as a facade for TFS clients.  They exchange data with their clients using Client Object Model.
    • Server Object Model – Data exchange between TFS Server and the SQL Server (may be on a different VM/physical box) happens using Server Object Model.
  • Build Tier – The build tier consists of
    • Build Service – The build service (ASP.NET Worker Process) contains of Build Controllers and Build Agents
    • Build Process Object Model – The process object model contains of the process workflow that has to be followed when defining the build definition
    • Client Object Model – Client Object Model is the data exchange between TFS Server (Application Tier) and Build Service

It is important to understand that the application tier referred above is the application tier of TFS and not your business application.  Based on several factors such as geographical spread, server-load, network topology, proxies, etc. you may have one or multiple servers in application tier.  You can configure the build tier on the server forming the application tier.  Some deployment scenarios prefer configuring one build server for multiple TFS servers.

Desktop

A TFS Server can hold more than one code-repository and have isolated & customized build process(es) set up on one or multiple build servers.

Let’s understand this with an example –

Your TFS server url is https://YourTFS.visualstudio.com. This TFS Server can host multiple TFS Projects say ProjectA, ProjectB and ProjectC.  Each project has a code repository say RepositoryA, RepositoryB, RepositoryC and each repository has 4 branches – Development, SIT, UAT, PROD.  The code in the repositories resides on the Data Tier and is accessible using client applications that interface through the Application Tier.

Now, for each branch you can configure one or multiple build processes.  For PROD, you may like to have Nightly Builds, for Development you want to have a build triggered at every check-in and so on…  These builds require a build server.  You can use 1 build server for all the builds or you can have a separate build server for each Repository, or you can share build server between 2 repositories.

The decision to use one-build server or more depends on the peak-load, expected build queue, h/w infrastructure and a lot of other parameters.  If you do not want build services, you do not need to configure a build server at all.

With this understanding, we will deep dive into the responsibilities of each tier

Data Tier

Data Tier is formed of a SQL Server running on a Windows Server VM/physical machine.

A SQL Server consists of following databases

  • TFS_Configuration  – Configuration Database that stores the TFS resources and settings
  • TFS_Warehouse  – Data warehouse database for report generation
  • TFS_Analysis – Aggregated data of all Projects on a TFS server
  • Project databases – Database for each Project in the collection

In TFS 2013, you may see some additional databases (with names starting with WSS) that are required to manage the TFS Administration Portal.  This administration portal runs on SharePoint Foundation.

Application Tier

The application tier consists of components such as

  • Version Control – Source code versioning & management
  • Build Service – Light-weight ASP.NET Worker Process that interacts with the Build Tier
  • Work Item Tracking – Product backlog, bug, work-item tracking & management
  • Team Foundation Framework Services – Web services, Registry, Security, Event, Identity Management, Administration
  • Administration Portal – Built on SharePoint Foundation, the portal serves as one-stop for defining projects, team, work-items, reports, permissions, etc.

All these components are hosted on IIS and Windows Services on a Windows Server VM/machine.  The Team Foundation Framework Services act as a facade for all client tools like TFS Explorer, Visual Studio, etc.

Build Tier

Build tier consists of build services that enable configuring a build controller and build agents.  A build controller controls the build queue and assigns the responsibility of build to a most available build agent.  If no build agent is free, it will queue the build request.

It is not necessary to have the controller and agent on the same machine.  This allows creating a build farm that can have multiple build machines.  One of the scenarios for an enterprise build environment would be

A build farm with S servers on which C (C<S) controllers are running and A (A>S>C) build agents are running. In such a scenario, the likely hood of a build getting queued for a long duration is rare.  However, actual values of A, S, C depend on number of applications, peak-load, build time, locale, infrastructure costs, cross-border restrictions, etc.

Firewalls

TFS architecture has been designed to allow extensibility and building a large farm at data-tier, application-tier or build-tier.  When configuring TFS on a farm, the inter-server communication happens through web services.  In an enterprise environment with high-security, you may have to check if any of the ports used in web service communication is blocked.

By default TFS uses following ports

  • Team Foundation Administration Portal and Reporting – port 80, 443
  • Team Foundation Client Web Services – port 8080
  • MS-SQL – port 1433 (TCP)
  • Team Foundation Application Tier Web Services – port 8080
  • Team Foundation Data Tier Web Services – port 8080
  • Build Services – net.pipe, 9191, 443

 

Hope this helps in understanding TFS architecture.

Follow on Feedly