GitHub Clone with Redmine ↯ xdissent.com

I’m in love with GitHub and I don’t mind paying a few bucks a monthto host private code repositories there. It’s not without its issues though,and I’ve often had trouble getting others to collaborate with me via GitHubfor one reason or another. Desiring more control, I was thrilled whenGitHub:FI was announced. Unfortunately, the licensing fees are staggeringand put the service far out of my reach. Recently, my buddy Marcus Whitneyhad been messing around with Redmine and my interest was piqued by his results.I decided to jump in head first and try to build a reliable GitHub:FIalternative using an Ubuntu VPS, Git, Gitosis, Gmail (or Google Appsfor Domains), and Bitnami‘s Redmine Stack.

Contents

Goals and Design Decisions

Redmine itself supports many of the features of GitHub, specifically:

  • Project membership
  • Issue/ticket tracker
  • Wikis
  • SCM with Git

The latest stable release even includes the much anticipated Git branchsupport. With a couple of plugins, Redmine can perform even more neattricks, like sending outgoing email through Google’s SMTP servers, andmanaging your Gitosis repositories and public keys automatically. I’veonly just begun to scratch the surface of functionality and I can alreadysense the power and flexibility of Redmine is going to dramatically increasemy productivity and calm sense of well-being.

Server Hygeine

There’s no reason even a fairly complex system like a Redmine installationcan’t play nice with other system services, although it takes someplanning. Much misinformation exists on the Web about running Redmine inparticular, so we must tread very lightly to sidestep the pitfalls of ashaky system architecture. I’m of the mind that you should fullyunderstand the intricacies of a piece of software before you attemptto run it – or at least understand clearly that you do not need toworry yourself with certain details – and hopefully when this setupis completely we’ll understand not only what we’ve done, but whywe’ve done it. That’s the key to building a solid system and is ofthe highest priority.

Different elements of the system should be reused where appropriate aswell. We want to only use the system-wide versions of things likeApache2, since they’ll probably be used for other services too and itmakes no sense to double your load when you don’t have to. By the sametoken, we don’t need to install services which are easily outsourcedto existing services or the cloud (like Gmail). In reality,our Redmine server might be moonlighting as a DNS server, firewall,development box – whatever – and we need to respect that. A generalpolicy of software and user isolation along with well controlled sharedresource management will ensure we aren’t wasting our time on aserver that will need to be rebuilt later to play some additional role.

Stacks

A “stack” is a complete software system packaged so that, once installed,it operates in a self contained environment that will not effect, or beeffected by, the rest of the host system. You generally don’t have to knowmuch more than how to run an installer script and you can be the proud adminof your very own LAMP/Rails/etc server!

Stacks as a concept are almost prehistoric yet they remain an effective meansof reliable package installation. The more complex the package, the moreappealing a stack distribution becomes. The author may finely tune the mostfragile system and distribute it as a stack without having to worry aboutit breaking in most cases.

A Redmine stack is a particularly appealing alternative to fighting Ubuntu’sawkward Rails packages that are notoriously difficult to use and maintain.Bitnami’s Redmine stack was chosen almost randomly, and I’m sure there areother options out there. Bitnami has a great reputation though, and theirconventions are very clean and logical in my opinion.

To be honest, I’m a total Ruby-phobe. Not bringing that up would be lyingthrough omission, and it’s a major reason I’ve decided to go with a stack- it’s a turn-key solution to a problem I (care to) know extremely littleabout otherwise. If you’re not sold on the idea by my humility, feelfree to substitute your own Redmine installation process. Most of thegeneral setup steps will be similar anyway.

Prerequisites

While this setup procedure should be fairly universal, there are a couple ofimportant things that must be in place before continuing. Firstly it’s assumedthat you have an Ubuntu server somewhere and a local development machine, whichis capable of reaching the Ubuntu server using a fully qualified domain name(FQDN). You don’t have to have working DNS entries necessarily – you can getaway with using your /etc/hosts files. On both of these machines you musthave access to a non-root user, and that user must have sudo access on theserver.

Fully Qualified Domain Name

Log into the non-root user account on the server via the console or an SSHsession. Check the server’s hostname to make sure a FQDN is provided:

[email protected]:~$ [email protected]:~$ hostname [email protected]:~$ hostname -fdev.local

If for some reason the output of hostname does not give a FQDN, you mighthave to adjust the contents of your /etc/hosts file and/or your/etc/hostname file. Your host must also be accessible to itself, so a quickping is in order.

[email protected]:~$ head -3 /etc/hosts127.0.0.1       localhost127.0.1.1       dev.hsd1.tn.comcast.net.        dev127.0.1.1       [email protected]:~$ cat /etc/[email protected]:~$ ping dev.localPING dev.local (127.0.1.1) 56(84) bytes of data.64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=1 ttl=64 time=0.021 ms64 bytes from dev.hsd1.tn.comcast.net. (127.0.1.1): icmp_seq=2 ttl=64 time=0.030 ms^C--- dev.local ping statistics ---2 packets transmitted, 2 received, 0% packet loss, time 1000msrtt min/avg/max/mdev = 0.021/0.025/0.030/0.006 ms

You may need to tell the server to take note of the changes in /etc/hostnameby running sudo hostname -F /etc/hostname. It’s pretty important to getyour FQDN set up correctly before you continue the installation, so double checkeverything now.

Development Machine Access

At this point you should also be able to ssh into the server from your localdevelopment machine. If you can’t ssh yet but are logged in via the console,make sure the openssh-server package is installed by running sudo apt-getinstall openssh-server. If you’re using a FQDN that doesn’t have real-worldDNS entries, add the FQDN to your development machine’s /etc/hosts file:

Stewart:~ xdissent$ echo <server-ip> <server-fqdn> | sudo tee -a /etc/hosts > /dev/null

Note

The tee command is used because root access is required to change the/etc/hosts file. STDOUT redirection is not granted the same access, soredirecting to a protected file will still result in a permissions error.The tee command simply redirects STDIN to a file, appending the contentsof STDIN if the -a option is passed. By default, the contents are alsodumped to STDOUT, so we redirect that to /dev/null to get rid of it.

Sudo Configuration

Finally, you must ensure that your non-root user on the server can gain accessto other non-root user accounts. Some Ubuntu virtual images come with morerestrictive sudoers configurations that only let your user sudo to root.As a simple test, try any random command as a random system user (who has ashell defined). For example, the www-data user is a decent test case:

[email protected]:~$ sudo -u www-data lsSorry, user xdissent is not allowed to execute '/bin/ls' as www-data on dev.local.

If access is denied, you must change the sudoers file using the commandsudo visudo. A common fix is to append %admin ALL=(ALL) ALL to the endof the file, and then add your non-root user to the admin group if notalready a member.

Required Packages

The whole idea behind using Ubuntu is to take advantage of its conveniences,like the package system. The Redmine stack comes with its own version ofmany standard system services which we’re going to disable in favor of thesystem versions. That way, we don’t have multiple instances of the sameservice consuming precious resources, and we move further towards a genericsystem that doesn’t rely on the stack for anything but the most specializedof services.

The standard Ubuntu services are easy to install using the various packagemanagement tools. Chances are, you’ve got many of these packages alreadyinstalled, but there are so many different configurations out there, it’seasiest to just try to install them all in one go. Don’t forget to updateyour Apt sources:

[email protected]:~$ sudo apt-get updateHit http://security.ubuntu.com karmic-security Release.gpgIgn http://security.ubuntu.com karmic-security/main Translation-en_US[...]Hit http://us.archive.ubuntu.com karmic-updates/mutiverse PackagesHit http://us.archive.ubuntu.com karmic-updates/mutiverse SourcesReading package lists... [email protected]:~$ sudo apt-get install apache2 mysql-server git-core python-setuptools git-core build-essential aclReading package lists...Reading state information...The following extra packages will be installed:[...]

ACL Mount Options

Access Control Lists (ACLs) are an often overlooked security feature thatallow additional, more specific access permissions to be applied to a fileor directory. They can control automatic file ownership settings inheritedfrom parent paths as well, which is crucial in situations where permissionsmust be maintained regardless of which user is operating on a shared directory.ACL support in Ubuntu is dependent upon the acl package which we installedearlier.

Simply installing the acl package does not enable ACLs for alldisks. Hard disk partitions must be configured in /etc/fstab to load theACL option and then must be remounted for the changes to take effect. Assuming/opt is on the same partition as the root filesystem, add the acloption to its /etc/fstab entry (using sudo):

[email protected]:~$ cat /etc/fstab# /etc/fstab: static file system information.## Use 'blkid -o value -s UUID' to print the universally unique identifier# for a device; this may be used with UUID= as a more robust way to name# devices that works even if disks are added and removed. See fstab(5).## <file system> <mount point>   <type>  <options>       <dump>  <pass>proc            /proc           proc    defaults        0       0# / was on /dev/sda1 during installationUUID=0c23e780-3453-4b2b-b42c-a1811722e941 /               ext4    acl,errors=remount-ro 0       1# swap was on /dev/sda5 during installationUUID=e3eb3872-b203-46e9-b020-b3caf72ccf99 none            swap    sw              0       0/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0/dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0

A reboot is required if ACLs were not already enabled and /opt is indeedon the root filesystem. If /opt is on a different (non-root) partition,you may get away with remounting it using mount -o remount,acl /opt orsimilar. Once you’ve got it working, the following command should give similaroutput:

[email protected]:~$ mount | grep acl/dev/sda1 on / type ext4 (rw,acl,errors=remount-ro)

ACLs aren’t absolutely required by this setup, but it’s good practice andwill probably save us a few headaches down the road, so just bite the bulletand load it up now. If you didn’t already see this coming, we’ll be using ACLfeatures to let Redmine talk to Git repositories without a file ownershipnightmare a little later.

Python Packages

We installed the python-setuptools package mere moments ago, whichinstalls the very popular easy_install Python package installer. Whileit’s a great little dude to have around, I much prefer a new-comer tothe Python package tool scene, Pip. It’s one Python package I’m comfortableinstalling globally, so we’ll use easy_install only once, to fetch its ownsuccessor and install it system-wide:

[email protected]:~$ sudo easy_install pipSearching for pipReading http://pypi.python.org/simple/pip/[...]Installed /usr/local/lib/python2.6/dist-packages/pip-0.7.1-py2.6.eggProcessing dependencies for pipFinished processing dependencies for pip

Now we can use the pip command to install any Python package we want,which is a much more well-advised notion than attempting to use the Ubuntupackage manager. Interestingly, the only thing the Apt system is really bad atis Python and Ruby package management, and this project deals with both.

On the subject of Python, it would be a great time to familiarize yourselfwith the Virtualenv Python package, and how it’s used to create self containedPython environments. Essentially, the virtualenv <path> command sets upa tiny Python distribution with its own package repository and <path>/bindirectory containing special binaries. The special binaries will automaticallybootstrap the intended Python environment upon execution, doing away with alot of path and environment variable nonsense that used to be required whenattempting advanced deployments. It really is a life saver if you run morethan a single Python app, and the idea of not impacting the system-wide Pythoninstallation is in line with the goals we’ve stated.

Install Virtualenv globally, which will be used when installing Gitosis:

[email protected]:~$ sudo pip install virtualenvDownloading/unpacking virtualenv  Downloading virtualenv-1.4.8.tar.gz (1.5Mb): 1.5Mb downloaded[...]Successfully installed virtualenvCleaning up...

Install Redmine

Bitnami really makes it simple to get up and running with Redmine. If we didn’tcare about running “two of everything”, it could be installed and configured inless than an hour with no sweat. Most of the extra steps we’re going to performwill actually disable many parts of the carefully architected Bitnami system,pointing the Redmine install to our standard system services instead.

Create a redmine user

A dedicated user is not technically required to run the Bitnami Redmine stack.In fact, I first ran it as www-data and it worked like a charm. Problemsarose when trying to integrate Gitosis, and it became obvious that it’s justeasier and cleaner to go ahead and set up a new system user for the redmineapplication. This configuration assumes the Redmine stack, and thus, thededicated Redmine user’s home directory, will reside at /opt/redmine. Feelfree to adjust this path at your whim. The obvious choice for the user namewas redmine, but you can change that if you must as well:

[email protected]:~$ sudo adduser --system --shell /bin/bash --gecos 'Redmine Administrator' --group --disabled-password --home /opt/redmine redmineAdding system user `redmine' (UID 106) ...Adding new group `redmine' (GID 115) ...Adding new user `redmine' (UID 106) with group `redmine' ...Creating home directory `/opt/redmine' ...

Install Bitnami Redmine Stack

The Bitnami Stack system is super easy to get running. All you have to do isdownload and run the installer and away you go. Before we get started,ensure that the system Apache2 and MySQL services have been stopped.That way the default values will be used by the Bitnami installer, whichare closer to the values we’ll ultimately want.

[email protected]:~$ sudo /etc/init.d/apache2 stop && sudo /etc/init.d/mysql stop * Stopping web server apache2   ...done. * Stopping MySQL database server mysqld   ...done.

Download and run the Bitnami Redmine Stack installer. Choose to install into/opt/redmine, or whichever folder you chose for the redmine user’s home.Do not choose to set up an SMTP server at this time, because we will handle thatwith Gmail later. Be sure to use sudo -u redmine to run the installer, sothe file ownership will be correct.

[email protected]:~$ wget http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.bin--2010-05-03 18:34:05--  http://bitnami.org/files/stacks/redmine/0.9.3-0/bitnami-redmine-0.9.3-0-linux-installer.binResolving bitnami.org... 216.235.167.25Connecting to bitnami.org|216.235.167.25|:80... connected.[...]2010-05-03 18:38:48 (550 KB/s) - `bitnami-redmine-0.9.3-0-linux-installer.bin' saved [159459281/159459281][email protected]:~$ chmod 755 [email protected]:~$ sudo -u redmine ./bitnami-redmine-0.9.3-0-linux-installer.bin----------------------------------------------------------------------------Welcome to the BitNami Redmine Stack Setup Wizard.Created with an evaluation version of BitRock InstallBuilder----------------------------------------------------------------------------Installation folderPlease, choose a folder to install BitNami Redmine StackSelect a folder [/home/xdissent/redmine-0.9.3-0]: /opt/redmine----------------------------------------------------------------------------Create Admin accountBitNami Redmine Stack admin user creationLogin [user]: adminPassword :Please confirm your password :Your real name [User Name]: Redmine Administrator[...]Please wait while Setup installs BitNami Redmine Stack on your computer. Installing 0% ______________ 50% ______________ 100% #########################################----------------------------------------------------------------------------Setup has finished installing BitNami Redmine Stack on your computer.Launch RedMine application. [Y/n]: YInfo: To access the BitNami Redmine Stack, go tohttp://localhost:8080 from your browser.Press [Enter] to continue :

That’s it! Redmine is running! You could open uphttp://<server-fqdn>:8080/redmine/ and bask in the glory – or you couldgrow a pair and just forge ahead with the setup. That’s what I thought, buddy.

A Brief Note on Redmine Management

Bitnami provides a completely self contained Redmine stack with tightly coupledcomponents that require a very specific (but minimal) shell environment to worktogether properly. For that reason, Bitnami includes a use_redmine script,which sets up the correct environment and runs a shell that is guaranteedto work with the Bitnami system. The use_redmine script should be run oncebefore any Redmine management task. If you invoke a management script froman environment other than the one set up by use_redmine, you run the riskof corrupting your system. All Redmine management should also be done as theredmine user for obvious reasons. The use_redmine script is convenient inthis regard, because you only have to run sudo once per administrationsession:

[email protected]:~$ cd [email protected]:/opt/redmine$ sudo -H -u redmine ./use_redminebash-4.0$ whoamiredmine

The Redmine stack services can be managed using the ~redmine/ctlscript.shscript. It follows the familiar ctlscript.sh <start|stop|restart|status><service> syntax and, of course, must be run after use_redmine.

Reconfigure the Redmine Stack to Use System Services

Firstly, stop all Bitnami stack services that are currently running. We’regoing to completely reconfigure Redmine anyway, and we won’t be needing anyof the other services from this point on.

Warning

These and all management commands must be run from within a Bitnamimanagement shell as the redmine user. Use the following command if youdon’t understand and want to be safe: sudo -H -u redmine/opt/redmine/use_redmine.

bash-4.0$ pwd/opt/redminebash-4.0$ ./ctlscript.sh stop/opt/redmine/subversion/scripts/ctl.sh : subversion stoppedSyntax OK/opt/redmine/apache2/scripts/ctl.sh : httpd stoppedstopping port 3001stopping port 3002/opt/redmine/mysql/scripts/ctl.sh : mysql stopped

Create a Redmine Database

Leave the Redmine management shell and start up the system MySQL service.Then we can create a database to host the Redmine application. Themysqladmin command can be used to create the DB.

bash-4.0$ [email protected]:/opt/redmine$ sudo /etc/init.d/mysql start * Starting MySQL database server mysqld   ...done. * Checking for corrupt, not cleanly closed and upgrade needing [email protected]:/opt/redmine$ mysqladmin --user=root --host=localhost create redmine

Alter Redmine’s Database Configuration

Redmine uses YAML configuration files, which is a fairly common Rails thingfrom what I understand. We need to point the database configuration towardsthe system MySQL service with the default settings. Edit the file~redmine/apps/redmine/config/database.yml and alter theproduction section to reflect the system settings:

production:  adapter: mysql  database: redmine  host: localhost  username: root  password:  socket: /var/run/mysqld/mysqld.sock

Now we can migrate the database, installing the Redmine application data on thesystem MySQL server. Rails apps use the rake command for these types oftasks, which I’ll admit is a pretty cool built-in tool. Rails also supportsthe concept of deployment environments, but we can ignore that for thissetup and only pay attention to the production environment.

Note

All rake commands must be run from within the application’s rootfolder, which is ~redmine/apps/redmine/ in this case.

Migrate the database and exit the Redmine management shell:

[email protected]:/opt/redmine$ sudo -H -u redmine ./use_redminebash-4.0$ cd apps/redminebash-4.0$ rake db:migrate RAILS_ENV=production(in /opt/redmine/apps/redmine)==  Setup: migrating ==========================================================-- create_table("attachments", {:force=>true})   -> 0.0151s[...]==  AddIndexOnChangesetsScmid: migrating ======================================-- add_index(:changesets, [:repository_id, :scmid], {:name=>:changesets_repos_scmid})   -> 0.0303s==  AddIndexOnChangesetsScmid: migrated (0.0305s) =============================bash-4.0$ exit

Change Permissions for the Mongrel Cluster Proxy

The Ubuntu Apache2 server has more restrictive proxy security settings thanis assumed by the Bitnami Redmine stack. We need to edit~redmine/apps/redmine/conf/redmine.conf and add the Order andAllow setting keys to permit Apache2 to access the Mongrel cluster:

ProxyPass /redmine balancer://redmineclusterProxyPassReverse /redmine balancer://redminecluster<Proxy balancer://redminecluster>  BalancerMember http://127.0.0.1:3001/redmine  BalancerMember http://127.0.0.1:3002/redmine  Order deny,allow  Allow from all</Proxy>

You don’t need to know much about Mongrel or clusters, which is another perkof an application stack. The default Bitnami setup runs two instances of theapplication and can load balance between them, which is totally fine for mostuse-cases. Just change the permissions and forget the word “mongrel” forever.

Configure System Apache2 Server

The Apache2 server only requires a couple of additional modules to be enabledin order to play nice with Redmine. They’re all of the mod_proxy familyand come in the default Apache2 package. The Redmine Apache2 configurationfile we just edited will need to be “included” in the main Apache2 config,which we’ll accomplish by creating a simple file in /etc/apache2/conf.d/:

[email protected]:/opt/redmine$ sudo a2enmod proxy*Enabling module proxy.Considering dependency proxy for proxy_ajp:Module proxy already enabledEnabling module proxy_ajp.Considering dependency proxy for proxy_balancer:Module proxy already enabledEnabling module proxy_balancer.Considering dependency proxy for proxy_connect:Module proxy already enabledEnabling module proxy_connect.Considering dependency proxy for proxy_ftp:Module proxy already enabledEnabling module proxy_ftp.Considering dependency proxy for proxy_http:Module proxy already enabledEnabling module proxy_http.Run '/etc/init.d/apache2 restart' to activate new [email protected]:/opt/redmine$ echo Include /opt/redmine/apps/redmine/conf/redmine.conf | sudo tee /etc/apache2/conf.d/redmine > /dev/null

Restart Apache2 and Redmine

All that’s left to do is to restart the Apache2 server and the Redmineapplication. The former must be done from outside of the Redminemanagement shell, while the latter must be done from within:

[email protected]:/opt/redmine$ sudo /etc/init.d/apache2 start * Starting web server apache2   [email protected]:/opt/redmine$ sudo -H -u redmine ./use_redminebash-4.0$ ./ctlscript.sh start redminestarting port 3001starting port 3002

Congratulations, Redmine is installed and running on the system Apache2 andMySQL servers!

https://i2.wp.com/xdissent.com/wp-content/uploads/2010/05/Redmine-Installed-1024x870.png?resize=500%2C424

Load Default Redmine Configuration

Log into Redmine at http://<server-fqdn>/redmine/ using the administrativeuser which the installer created, in my case, admin. You will be greeted witha screen prompting you to load the Redmine default configuration. This is highlyrecommended, so do it now. You should probably at least change the URL settingfor the site as well.

https://i1.wp.com/xdissent.com/wp-content/uploads/2010/05/Admin-Login-1024x870.png?resize=500%2C424

https://i2.wp.com/xdissent.com/wp-content/uploads/2010/05/Default-Configuration-1024x870.png?resize=500%2C424

Set Up Redmine and Gmail

Redmine occasionally needs to send users email to alert them to project changesand other events. We chose not to set up an SMTP server when installing so thatwe could hopefully interface with Gmail, avoiding yet another redundant servicerunning on our machine. It’s assumed that we have a Gmail or Google Apps forDomains account specifically reserved for Redmine – ideally[email protected]<server-fqdn>.

Google’s SMTP servers require a special type of authentication called TLS. TheRuby libraries on which Redmine is built do not support this authenticationscheme unfortunately, so we’ll need a Redmine plugin to get it working.Specifically, the ActionMailer class will be extended to support an optionalTLS authentication mode. The plugin that provides this extension is (creatively)called action_mailer_optional_tls.

Install the Plugin

Redmine (or is it Rails?) comes with a plugin installation script that makesit trivial to try out new extensions. Just like rake commands,the script/plugin script must be run from within the application’s rootfolder – ~redmine/apps/redmine/ in our case. It accepts an installsub-command and can install plugins directly from Git repositories:

bash-4.0$ cd ~/apps/redminebash-4.0$ script/plugin install git://github.com/collectiveidea/action_mailer_optional_tls.gitInitialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/action_mailer_optional_tls/.git/remote: Counting objects: 14, done.remote: Compressing objects: 100% (12/12), done.remote: Total 14 (delta 2), reused 2 (delta 0)Unpacking objects: 100% (14/14), done.From git://github.com/collectiveidea/action_mailer_optional_tls * branch            HEAD       -> FETCH_HEAD

Note

It’s standard practice to “migrate” your plugins after adding or removingany packages. The action_mailer_optional_tls plugin does not have its ownDB schema, so it’s safe to ignore this step here.

Create Redmine’s Email Configuration

Since we opted out of the SMTP configuration during installation, we probablywon’t have a ~redmine/apps/redmine/config/email.yml file, which we’ll needto create. It’s another YAML configuration file that should be fairly selfexplanatory. Create ~redmine/apps/redmine/config/email.yml (as theredmine user) and add in your own Gmail settings:

production:  delivery_method: :smtp  smtp_settings:    tls: true    address: "smtp.gmail.com"    port: 587    domain: "gmail.com"    authentication: :plain    user_name: "[email protected]"    password: "XXXXXXXX"

The tls: true setting is only allowed after the action_mailer_optional_tlsplugin has been installed. If you’re using Google Apps for Domains, you wouldsubstitute your domain for gmail.com in the domain and user_namesettings.

Restart Redmine

If the settings are correct we can restart Redmine and the changes will takeeffect:

bash-4.0$ cd ~bash-4.0$ ./ctlscript.sh restart redminestopping port 3001stopping port 3002starting port 3001starting port 3002

Try the “Send a test email” link from within the “Email Notifications” settingstab in Redmine and you should shortly receive an email, courtesy of Gmail’s servers!

https://i2.wp.com/xdissent.com/wp-content/uploads/2010/05/Send-Test-Email-700x594.png?resize=500%2C424

Install Gitosis

Gitosis is a great little piece of software that manages Git repositoryaccess, magically storing its own configuration in a Git repository as well.Technically, Gitosis relies on a single system user for access and managesmore fine grained access policies using public key management. It’s a verysolid system, on which GitHub itself is derived, and doesn’t clutter thesystem up with unnecessary users or daemons.

We’ll be using a Redmine plugin to manage our Gitosis access directly throughthe Redmine interface. Redmine users may manage their own public keys and theyare automatically added to Gitosis’s access control system for projectrepositories of which they are members. This is similar to the way publickeys work in GitHub so it should sound vaguely familiar.

Create the git User

Gitosis absolutely requires a system user – specifically one with a realhome directory somewhere. Disabling the password makes sure only those withthe Gitosis private key we’ll create will be able to gain access to the boxand change Gitosis settings. We’ll start by creating this user, just as wedid for the redmine user. Feel free to select a different home directoryor name for the user, but note these differences. Also note that Bash is thepreferred shell, since it will give us a little more control over the gituser environment when running the Gitosis commands, which would otherwise failto find the Gitosis commands on its path.

[email protected]:~$ sudo adduser --system --shell /bin/bash --gecos 'Git Administrator' --group --disabled-password --home /opt/gitosis gitAdding system user `git' (UID 105) ...Adding new group `git' (GID 114) ...Adding new user `git' (UID 105) with group `git' ...Creating home directory `/opt/gitosis' ...

The Master Key Pair

Under the hood, Gitosis access is controlled with the use of private/public keypairs. Each developer will create his own key pair, offering the public key tothe Gitosis system through SSH when a Git operation is requested. Gitosis willcheck the public key against its known key list and grant or deny accessappropriately.

Naturally, the key management system must be accessible in some regard in orderto actually add developer public keys. Gitosis uses a single master key pairin this case, which will always be allowed to manage keys and the Gitosisconfiguration. It’s a good idea to generate this key from within a secure shellon the server, and never transfer it over any network. Strict filepermissions are applied by default and should not be changed or the SSH systemmay reject a perfectly good key out of (justified) paranoia.

To generate a master key pair, use the ssh-keygen command. We will beusing DSA keys, but any common scheme will work. Be sure to generate the keyas the git user as to not totally blow up the file permissions:

[email protected]:~$ sudo -H -u git ssh-keygen -t dsaGenerating public/private dsa key pair.Enter file in which to save the key (/opt/gitosis/.ssh/id_dsa):Created directory '/opt/gitosis/.ssh'.Enter passphrase (empty for no passphrase):Enter same passphrase again:Your identification has been saved in /opt/gitosis/.ssh/id_dsa.Your public key has been saved in /opt/gitosis/.ssh/id_dsa.pub.The key fingerprint is:55:d7:4b:4c:c8:a1:f9:c3:4d:99:44:2a:af:03:71:fd [email protected]<server-fqdn>The key's randomart image is:+--[ DSA 1024]----+|            o.B= ||           .++oo+||         ..= o.+.||         .o = +. ||        S.   = E ||          . . .  ||           o     ||            .    ||                 |+-----------------+

Create a Virtualenv

Gitosis is “just Python”, so it can easily be installed using Pip. However,it’s a good idea to sequester the Gitosis installation to minimize the riskof impacting our host system’s Python setup. We’ll use Virtualenv to createa standalone Python environment solely used by Gitosis.

[email protected]:~$ sudo -u git virtualenv ~git/virtualenvNew python executable in /opt/gitosis/virtualenv/bin/pythonInstalling setuptools............done.

Now any time a Python script is run from within /opt/gitosis/virtualenv/bin,our Gitosis Python environment will be bootstrapped automatically. That meanscalling ~git/virtualenv/bin/pip install <package> will install a Pythonpackage to the Gitosis virtual environment only. With that in mind, we caninstall Gitosis straight from its Git repository:

[email protected]:~$ sudo -u git ~git/virtualenv/bin/pip install git+git://eagain.net/gitosis.gitDownloading/unpacking git+git://eagain.net/gitosis.git  Cloning Git repository git://eagain.net/gitosis.git to /tmp/pip-JyRa0m-build  Running setup.py egg_info for package from git+git://eagain.net/gitosis.gitRequirement already satisfied (use --upgrade to upgrade): setuptools>=0.6c5 in /opt/gitosis/virtualenv/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg (from gitosis)Installing collected packages: gitosis  Running setup.py install for gitosis    Installing gitosis-init script to /opt/gitosis/virtualenv/bin    Installing gitosis-run-hook script to /opt/gitosis/virtualenv/bin    Installing gitosis-serve script to /opt/gitosis/virtualenv/binSuccessfully installed gitosisCleaning up...

To actually force the virtual environment to take precedence over the systemenvironment, we can load the ~git/virtualenv/bin/activate script. It setsup the user’s PATH environment variable to prepend the special Pythonscripts, automatically overriding the system versions. Loading the script froma ~/.bashrc file will set up the virtual environment upon login, and in ourcase, immediately before looking for the Gitosis commands. We’ll need to add asimple ~/.bashrc for the git user to get it all working:

[email protected]:~$ echo "source $HOME/virtualenv/bin/activate" | sudo -u git tee -a ~git/.bashrc > /dev/null

Initialize the Gitosis Repository

Before it can manage any Git repositories, Gitosis needs to be initializedvia the gitosis-init script. The master public key should be providedon standard input and the -H flag for sudo must be used:

[email protected]:~$ sudo -u git cat ~git/.ssh/id_dsa.pub | sudo -H -u git ~git/virtualenv/bin/gitosis-initInitialized empty Git repository in /opt/gitosis/repositories/gitosis-admin.git/Reinitialized existing Git repository in /opt/gitosis/repositories/gitosis-admin.git/

Note

Many sources would suggest using a command similar to sudo -H -u git~git/virtualenv/bin/gitosis-init < ~git/.ssh/id_dsa.pub. If your filepermissions are correct, this will not work. This is the same IOredirection problem that required the use of tee earlier, and thecat command must be used with sudo to successfully read and outputthe git user’s public key.

Gitosis is now set up to manage keys and configuration through the specialgitosis-admin repository. Assuming they had loaded the master private key,any local or remote user should now be able to clone your Gitosis repositoryusing git clone [email protected]<server-fqdn>:gitosis-admin.git.

Install Redmine Gitosis Plugin

Public key management and automatic project-to-repository linking areprovided by the redmine-gitosis plugin. Just like before, plugin installationmust be done from within a Redmine management shell. The redmine-gitosisplugin uses the lockfile and inifile Gems, so we’ll need to installthose as well:

[email protected]:~$ sudo -H -u redmine ~redmine/use_redminebash-4.0$ gem install lockfile inifileSuccessfully installed lockfile-1.4.3Successfully installed inifile-0.3.02 gems installedInstalling ri documentation for lockfile-1.4.3...Installing ri documentation for inifile-0.3.0...Installing RDoc documentation for lockfile-1.4.3...Installing RDoc documentation for inifile-0.3.0...bash-4.0$ cd ~/apps/redminebash-4.0$ script/plugin install git://github.com/rocket-rentals/redmine-gitosis.gitInitialized empty Git repository in /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/.git/remote: Counting objects: 123, done.remote: Compressing objects: 100% (120/120), done.remote: Total 123 (delta 74), reused 0 (delta 0)Receiving objects: 100% (123/123), 13.19 KiB, done.Resolving deltas: 100% (74/74), done.From git://github.com/rocket-rentals/redmine-gitosis * branch            HEAD       -> FETCH_HEAD

Don’t try to use the plugin yet though – it requires modification beforeanything constructive will happen.

Patching the Plugin

Unfortunately, the redmine-gitosis plugin is a definite work in progress (read:hack) and needs to be slightly modified to work in any case. In the futureI’d love to have a fork available that will automatically work in this setup,but that presents a sort of chicken-vs-egg problem, doesn’t it?

Gitosis Configuration

Firstly, the plugin is actually configured by editing a Ruby source filerather than a YAML configuration file. We must edit~redmine/apps/redmine/vendor/plugins/redmine-gitosis/lib/gitosis.rb andchange the GITOSIS_URI and GITOSIS_BASE_PATH values to reflect oursetup. The GIT_SSH environment variable needs to be slightly tweaked aswell:

require 'lockfile'require 'inifile'require 'net/ssh'module Gitosis  # server config  GITOSIS_URI = '[email protected]<server-fqdn>:gitosis-admin.git'  GITOSIS_BASE_PATH = '/opt/gitosis/repositories/'  # commands  ENV['GIT_SSH'] = SSH_WITH_IDENTITY_FILE = File.join(RAILS_ROOT, 'vendor/plugins/redmine-gitosis/extra/ssh_with_identity_file.sh')

Warning

The default GIT_SSH environment variable is incorrect. The pathshould have a hypenated redmine-gitosis rather than the underscoreddefault of redmine_gitosis.

Linking Fix

Bitnami stacks are great! The only thing I can think of to make them evenbetter would be if they linked their software differently at compile time.It’s pretty technical (and very boring) but Bitnami could either staticallylink the more common libraries or at least specify an rpath for theresulting dynamically linked binaries. That way the infamousLD_LIBRARY_PATH environment variable could be avoided entirely, promotingresponsible Linux software packaging. In the meantime, we’ll have to edit~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh_with_identity_file.shto change the LD_LIBRARY_PATH temporarily before calling ssh to add theprivate key. This prevents a linking error that would otherwise cause allpublic keys to be rejected. The modified ssh_with_identity_file.sh shouldlook like the following:

#!/bin/bashLD_LIBRARY_PATH=""exec ssh -i `dirname $0`/ssh/private_key "[email protected]"

Knowing Your Host

SSH maintains a list of “known hosts” for each user, which that user hasdetermined should be trusted. We should create a quick SSH session to theFQDN of the server as the redmine user, which will prompt SSH to addthe hostname to the list of known hosts. Redmine would not be ableto connect regardless of private key if the host is not previously knownto SSH, specifically for the redmine user. After the password prompt isdisplayed, simply ctrl-c to cancel the session:

bash-4.0$ ssh <server-fqdn>The authenticity of host '<server-fqdn> (127.0.1.1)' can't be established.RSA key fingerprint is d1:fb:2a:0e:53:4f:27:64:63:66:a5:09:28:bd:20:86.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '<server-fqdn>' (RSA) to the list of known [email protected]<server-fqdn>'s password:bash-4.0$

Migrate the Plugin

Unlike the action_mailer_optional_tls plugin, redmine-gitosis does use thedatabase internally. For this reason we need to “migrate” the plugin beforeattempting to use it. The rake command is used again, just like whenmigrating the initial Redmine application database. After the migration iscomplete, it’s safe to go ahead and restart Redmine since we’re done alteringits configuration:

bash-4.0$ rake db:migrate_plugins RAILS_ENV=production(in /opt/redmine/apps/redmine)Migrating engines...[...]Migrating redmine-gitosis...==  CreateGitosisPublicKeys: migrating ========================================-- create_table(:gitosis_public_keys)   -> 0.0136s==  CreateGitosisPublicKeys: migrated (0.0139s) ===============================Migrating rfpdf...Migrating ruby-net-ldap-0.0.4...bash-4.0$ cdbash-4.0$ ./ctlscript.sh restart redminestopping port 3001stopping port 3002starting port 3001starting port 3002bash-4.0$ exit

Redmine, the Key Master

It was mentioned that the master Gitosis key pair is used to manage the Gitosisconfiguration and public keys. Redmine will need access to the private keyin order to manage the public keys for us, but we have been very careful toprevent access to the master key from anyone except the git user! This isa perfect use-case for an ACL.

For some reason, the redmine-gitosis plugin comes with a default private key,which should be removed:

[email protected]:~$ sudo -u redmine rm ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key

Now we need to create a symlink from the path we just remove to the realGitosis master key at ~git/.ssh/id_dsa. Note that you won’t currentlybe able to access the file:

[email protected]:~$ sudo -u redmine ln -s ~git/.ssh/id_dsa ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/[email protected]:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_keycat: /opt/redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key: Permission denied

We need to install an ACL that will allow the redmine user to read theprivate key. ACLs are complicated but essentially we will add a rule thatonly grants the redmine user read-only access to the actual key fileand the parent directory, which is also required. The mask parametermust also be modified so the effective ACL permissions will be what weare expecting:

[email protected]:~$ sudo setfacl -m user:redmine:r-x,mask:r-x ~git/[email protected]:~$ sudo setfacl -m user:redmine:r--,mask:r-- ~git/.ssh/[email protected]:~$ sudo -u redmine cat ~redmine/apps/redmine/vendor/plugins/redmine-gitosis/extra/ssh/private_key-----BEGIN DSA PRIVATE KEY-----XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX-----END DSA PRIVATE KEY-----

The redmine user now has access to the Gitosis private key, but there’sone last ACL we need to implement before we’re done. The redmine-gitosisplugin occasionally manipulates the Gitosis repositories directly, usingthe path defined as GITOSIS_BASE_PATH above. To ensure the plugin cansuccessfully access the repositories, we’ve got to add a “default” ACL tothe repositories directory. In the future, any repositories that are createdwill inherit the “default” ACL, granting the plugin the required access:

[email protected]:~$ sudo setfacl -m default:user::rwx,default:group::r-x,default:other:---,default:user:redmine:rwx,default:mask:rwx ~git/repositories

The redmine-gitosis plugin should now have all the permissions it needsto access the Gitosis repositories, without unnecessarily exposing them toother users. Log into Redmine using the administrator user and verify thatthe redmine-gitosis plugin is active athttp://<server-fqdn>/redmine/admin/plugins.

https://i0.wp.com/xdissent.com/wp-content/uploads/2010/05/Plugins-1024x870.png?resize=500%2C424

Working With Redmine and Gitosis

Believe it or not, our GitHub clone is finished! Interacting with Redmineshould be very familiar if you’ve used GitHub. To get started, we’ll needa Redmine project and at least one project member. Create a project athttp://<server-fqdn>/redmine/projects/new/, noting the unique identifier.

https://i2.wp.com/xdissent.com/wp-content/uploads/2010/05/Create-Example-Project-1024x870.png?resize=500%2C424

The redmine-gitosis plugin will use the project identifier to determine thepath each project’s repository. A project whose ID is example will beavailable at [email protected]<server-fqdn>:example.git. To actually create theproject’s repository, a Redmine user (with a valid key pair) must push abranch to the Gitosis server. Create a developer user athttp://<server-fqdn>/redmine/users/new who will later create our repo.

https://i2.wp.com/xdissent.com/wp-content/uploads/2010/05/Create-Developer-1024x870.png?resize=500%2C424

Add your new developer user as a member of the project athttp://<server-fqdn>/redmine/projects/<project-id>/settings/members/,granting Git access to the project’s repository.

https://i1.wp.com/xdissent.com/wp-content/uploads/2010/05/Add-Project-Member-1024x870.png?resize=500%2C424

Now we can log out of the administrator account, since it’s really onlyuseful for project and user creation from now on. To enable your developer userto push to the Gitosis repositories, a public key must be assigned. Log into Redmine as your new developer user and navigate tohttp://<server-fqdn>/redmine/my/public_keys to add a public key.

https://i1.wp.com/xdissent.com/wp-content/uploads/2010/05/Add-Public-Key-1024x870.png?resize=500%2C424

To generate a key for use from a development machine, run ssh-keygen foryour local user exactly as we did when setting up Gitosis, except without thesudo. We need to paste the public key into the Redmine interface, namingthe key appropriately. The general naming convention is<local-user>@<local-fqdn>. When successfully added, the new public key showsup in the key list.

https://i1.wp.com/xdissent.com/wp-content/uploads/2010/05/Added-Public-Key-1024x870.png?resize=500%2C424

Now our local development user should be able to create the project’s Gitrepository from a development machine. It’s exactly the same process usedby GitHub, which is super convenient since I’m not sure I have the brainspace left to remember another command sequence. Once we push a branch tothe Gitosis server, the “Repository” tab in Redmine will be populated withour brand new Git repo.

Stewart:Code xdissent$ mkdir exampleStewart:Code xdissent$ cd example/Stewart:example xdissent$ git initInitialized empty Git repository in /Users/xdissent/Code/example/.git/Stewart:example xdissent$ touch README.rstStewart:example xdissent$ git add README.rstStewart:example xdissent$ git commit -m "Initial commit."[master (root-commit) 016b344] Initial commit. 0 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 README.rstStewart:example xdissent$ git remote add origin [email protected]<server-fqdn>:example.gitStewart:example xdissent$ git push origin masterInitialized empty Git repository in /opt/gitosis/repositories/example.git/Counting objects: 3, done.Writing objects: 100% (3/3), 219 bytes, done.Total 3 (delta 0), reused 0 (delta 0)To [email protected]<server-fqdn>:example.git * [new branch]      master -> master

https://i0.wp.com/xdissent.com/wp-content/uploads/2010/05/Initial-Commit-1024x870.png?resize=500%2C424

Redmine already includes great SCM tools like diff views and trackerintegration, just like GitHub. Git commit users are mapped to Redmine usersautomatically as well so the interface is often even more intuitive thanGitHub’s when you’ve accidentally borked a commit or merge.

Future Enhancements

It’s pretty damn close, but – even with Gitosis integration – Redmine isnot GitHub. The feature that’s most likely to be missed is the per-userrepository configuration. It might be a sizable undertaking, but this couldpresumably be implemented in another Redmine plugin. Pull requests, hookmanagement, and pretty graphs are probably next on the list, but should beless complicated to build.

Regardless, Redmine is obviously exciting and it’s my hope that more peopleget involved in the development. I’ve already brushed up on my Ruby basicsto tweak a few things so – who knows – I might even try my hand at puttingtogether a plugin or two of my own. In the meantime, I’ve got more thanenough functionality (and the promise of flexibility) to start using Redminefull-time in my development work.

Leave a Reply

Your email address will not be published. Required fields are marked *