Enabling TeamCity Push to GitHubMar26

Monday, 26 March 2012 by haemoglobin

The following documentation describes what is necessary to run commands such as git push to a GitHub repository from a build within TeamCity.
Without the correct SSH configuration, any call to remote git repositories in the build script will cause the agent to hang the build while it is attempting to ask the user to add the remote host to the ~/.ssh/known_hosts file.
This is the message that looks like the following (requiring user input), interaction which is not possible from the build:

    The authenticity of host 'github.com (207.97.227.239)' can't be established.
    RSA key fingerprint is d2:80:ef:7a:71:4b:92:89:c7:3d:fb:e6:f5:26:44:e1.
    Are you sure you want to continue connecting (yes/no)?

This text will also not be written to any build output or logs (due to the blocking call) making it difficult to diagnose.

Business Need

When we build our NuGet packages, we build them off a branch. This is because they are released libraries and we need a reference back to code that is in use in UAT/PROD etc. The source symbol paths that we write into the built PDB's (as per my last post on GitHub Source Symbol Indexing) are indexed with HTTP references back to GitHub on that branch, this is so source symbol stepping into our libraries will work for any developer using our libraries (and without them needing to have Git installed).

The source files that are downloaded into the developers Visual Studio debugging session (when stepping into our code) has an auto-generated file header describing when that file was built and what version of the library it is from.
The header looks similar to below, varying depending on the file:

    // YourLibrary SDK
    // YourCompany.YourLibrary\Config\ConfigureWindsorBuilder.cs
    // YourCompany.YourLibrary, Version 3.20.0.114, Published 07/03/2012 17:06
    // (c) Copyright YourCompany 
    // ----

If you are interested, the powershell script we are using to do this can be downloaded from here.

Since we do this as part of the build we need the build to push these changes back to the repository itself so the source stepping will download the file from GitHub with the added headers.
Build agents are able to interact with the repository if the TeamCity checkout mode is set to Automatically on agent.

The rest of this documentation will describe what steps are necessary to ensure the build agent is setup to support communicating with the remote repository and how TeamCity itself needs
to be configured.

The official TeamCity 7 documentation regarding git support can be found here: http://confluence.jetbrains.net/display/TCD7/Git+%28JetBrains%29
Mike Nichols has also written about TeamCity/GitHub interaction in his blog post here.

Configuring TeamCity for Agent Side Checkout

Steps

  • Attach a Git VCS root, configure the VCS root name, Fetch URL, Ref name (branch), User Name Style as usual.
  • Authentication Method needs to be set to "Default Private Key".
    • This is the only supported method for agent-side checkout with SSH.
  • Ensure "Ignore Known Hosts Database" is ticked.
    • This saves any potential hassle with the Java/Windows home drive mismatch below.
  • Everything else can be left as default, including "Path to git" which should be set to %env.TEAMCITY_GIT_PATH%.
  • Back in the main VCS Settings, ensure "Automatically on agent" is selected for the CVS checkout mode.
  • Currently, "Clean all files before build" needs to be checked to avoid a hanging build due to a TeamCity issue.
    • This should be solved in TeamCity 7.1 and there is already a patch available.

Configuring TeamCity Build Agent

Any agents that do not have git installed will appear *incompatible* with regards to this configuration, the incompatible agent requirement message env.TEAMCITY_GIT_PATH exists will be displayed until git is installed on the agent and the TeamCity agent service restarted.

Steps if Git is not already installed and configured on the Agent

  • Follow http://help.github.com/win-set-up-git/ to install msysgit on the build agent.
  • Choose the "Run Git from the Windows Command Prompt" option which enables us to call git commands easily from the build.
  • Using Git bash, generate ssh keys to the default "home directory" location (~/.ssh), with no passphrase (TeamCity does not support passphrases for Default Private Key authentication).
  • Add the public key to your GitHub account as per the instructions.
  • Run the following commands to ensure the agent appears correctly in the Git history when it makes commits (this is stored in the file ~/.gitconfig):
    • git config --global user.name "Build Agent"
    • git config --global user.email "buildagent@db.com"

NOTE: ~ above refers to the home directory, on Windows this is the concatenation of the HOMEDRIVE and HOMEPATH environment variables.

Bypass Known Hosts Check

It is the request to add the remote host to the list of known hosts that hangs the build during git calls involving remote repositories. If the remote host has not already been added by manual means on the agent (an entry will be in ~/.ssh/known_hosts if it has) then the following needs to be done so it is added automatically without requiring user interaction when the first request is made by the build.

Steps

  • Create a file named "config" (no extension) under ~/.ssh and add the following lines:
    • Host github.dbatlas.gto.intranet.db.com
    • StrictHostKeyChecking no

Check Java/Windows Home Directory Mismatch

Since TeamCity uses a Java implementation of SSH for the initial checkout, java considers the home directory (and hence the directory it looks for ssh keys) to be one level up of the the value of the registry key HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Shell Folders\\Desktop.

In a corporate environment this can sometimes be different to the HOMEDRIVE and HOMEPATH environment variables that git used above to write the ssh keys to.

This is described here. If there is a mismatch, you will likely see an Authentication failed error during the agent side git checkout since TeamCity will be looking for the SSH keys in the incorrect home directory location.

Steps

  • Within TeamCity, find the agent and view the "Agent Parameters". Under System Properties, check the variable *user.home* (this is the home directory that java considers it to be).
  • If this is different to where the ssh keys were created in the previous steps you have two options:
    • Copy the .ssh folder to this location as well, or
    • Create a blank .ssh folder here and add a file "config" (no extension) including the following line which maps to the private key created previously:
      • IdentityFile {path_to}\id\_rsa

NOTE: In the {path_to} replacement above, don't use any mapped networked drives, use the full network path instead. Also relative directory locations do not seem to work here.

Check Home Directory Variables Available to Agent

Since the git installation actually adds c:\Program Files (x86)\Git\cmd to the path, when you call git from a build script you are in
effect really calling c:\Program Files (x86)\Git\cmd\git.cmd which is another batch file.

Git.cmd combines the %HOMEDRIVE% and %HOMEPATH% environment variables into a %HOME% variable which git.exe uses for the location of the
.ssh keys, .gitconfig and config file as configured above.

If the TeamCity agent is running as a service (as opposed to started from the command prompt through c:\BuildAgent\bin\agent start), these variables may not be available and our calls to git in the build script will fail.

Steps

  • In c:\BuildAgent\conf\buildAgent.properties add the line:
    • env.HOME={path to user home directory containing the .ssh folder and .gitconfig}

IMPORTANT NOTE: Remember to use escape backslashes in this file for TeamCity to process these correctly, so c:\Users\MyHomeDirectory becomes c:\\Users\\MyHomeDirectory.
Also, do not use any network mapped drive names as this does not seem to resolve, i.e instead of X:\, use \\\\SERVERNAME\\NETWORKSHARE\\USERSHOMEDIR

Considerations when calling Git from a batch file in the build

Always ensure you begin any calls to git in a batch file with call.

In a Windows batch file, any call to another batch file will not return unless you use the call keyword beforehand. So prefer to use call git push over git push for example.

Debugging

The log files in c:\BuildAgent\logs can all have useful information when trying to resolve the issues discussed above, specifically the files teamcity-vcs.log, teamcity-build.log and teamcity-agent.log.

Conclusion

As you might imagine, this all took quite a bit of fiddling around to see how to get this all going - and I must say the documentation available does seem to be vague in a lot of areas. I think it is perfectly reasonable to argue that the extra build complication may outweigh the potentially “nice to have” use case there happens to be for pushing into the repository from the build, at least until this becomes less difficult.

But I hope that this is helpful for anyone doing the same thing!

Categories:   Development
Actions:   E-mail | Permalink | Comments
blog comments powered by Disqus

Powered by BlogEngine.NET 1.6.1.0 | Design by styleshout | Enhanced by GravityCube.net | 1.4.5 Changes by zembian.com | Adapted by HamishGraham.NET
(c) 2010 Hamish Graham. Banner Image (c) Chris Gin