Wednesday, December 23, 2015

HOWTO STRUCTURE MANY PROJECTS IN A SINGLE REPOSITORY - PART 3

Overview

At this point in time you have a active GIT Server that is working well for your team. You and your team have been creating projects that can be centrally shared and viewed by other team members. You can even browse the source using a web browser.

New projects can be initially setup and then pushed onto the server pretty easily as well.

But now you are running into a problem. As the number of projects increases, so does the list of projects on the central GIT Server. This is pretty manageable at a client level since they clone only what is needed but on the server, the flat directory structure is confusing and hard to manage.

The goal of this HOWTO is to show the process for cleaning up your directory structure on the GIT server to allow some form of logical grouping. 

In my example, I just want to group together by purpose allowing a new tools area. The way things are grouped does not matter and can be done any way you choose. The technique remains the same.

Fine Print

I am not a security expert nor am I a complete GIT expert. This HOWTO does its best to get you up and running with GIT so your team can get to work.

Assumptions

I am doing this using 

  • Ubuntu Linux 15.10
  • GIT based on the Ubuntu distribution version
  • GITWEB based on the Ubuntu distribution version
  • PERL based on the Ubuntu distribution version
  • GITOLITE based on a git clone from github.com
  • You already have the Ubuntu distribution version of Apache2 installed and running.

For this document a project called mr_line_number will be moved using uuklanger as an administrator.

My Ideal Setup

Part Description
Server Name atomserver
Access Method SSH
GITWEB http://atomserver/gitweb/
User Management GITOLITE
REPO Base /opt/data/git/repositories/
GIT User git
GIT User Home /home/git/

The Scenario

Some time ago, mr_line_number was added to the central GIT server to allow other developers to enjoy the benefits of this tool. However, tools like this are now mixed in with a long list of other projects where some are production code.

To help avoid confusion, it has been decided to split the projects up into logical grouping

  • Products
  • Tools
  • Support

The venerable mr_line_number will be the first to be moved to the new tools area. All users of this will need re-pointed so their remote note uses the new location.

High Level Procedure

To make this work, the following will need done

  1. Create a directory for the new logical grouping and move mr_line_number into it
  2. Set the gitolite permission/config to add tools to the affected project
  3. Update the projects.list to add the new relative path
  4. Provide the command to update the remote of each user affected.

Once done, pushing or pulling changes from a client to the new remote location will be seamless. There is no need to “re-clone” the project.

Server Side Move of Project

As the git user, the mr_line_number project will need to be moved into a new folder called tools

$ sudo su - git
$ cd /opt/data/git/repositories/
$ mkdir tools
$ mv mr_line_number tools

Once completed, you will have one less project in your repositories base directory. The project will now be in tools.

Setup Gitolite Re-Point to the new project location

As the user who administers gitolite, a small change will need to be made to re-point gitolite and those who have permissions to see it, to a the new tools directory.

This assumes that the gitolite setup followed PART 1 of this series.

$ cd $HOME/Admin/git/gitolite-admin/conf
$ vi gitolite.conf
$ cat gitolite.conf 
repo gitolite-admin
    RW+     =   uuklanger

repo testing
    RW+     =   @all

repo tools/mr_line_number 
    RW+     =   uuklanger
    R       =   gitweb
$ git commit -m “Moving mr_line_number into the tools directory” gitolite.conf
$ git push origin master

In front the line mr_line_number was changed to be tools/mr_line_number. The changes are then committed and pushed.

IF YOU FORGET THIS STEP or do it wrong, you will see a new repository created in the repository base directory that is empty. During push, you should not see any message about something getting initialized or created.

Once completed, you should only see mr_line_number in <GIT_REPO_BASE>/tools/.

projects.list Update

Back as the git user, the projects.list will need updated to point to the new tools directory.

$ sudo su - git
$ vi projects.list
$ cat projects.list
tools/mr_line_number.git
testing.git

Now GITWEB will know where to look for mr_line_number among other projects.

Checks so far

If you visit the server’s URL for gitweb, you will see tools/mr_line_number as a link you can click on vs just mr_line_number

Click on the new link and confirm you can navigate the tree and see everything expected.

There is a possibility that Apache may need restarted if you do not see tools in front of mr_line_number.

Update the end users to re-point their remote to the new location

Each and every remote user who currently has this project cloned will still be pointed to the old location. If they try to push changes, it will not go very well for them. 

The good part is that re-pointing each remote is very easy and does not require re-clone or other scary stuff.

This is done on the client’s system that points the remote updated above.

$ cd work/mr_line_number
$ git remote -v
origin git@atomserver:mr_line_number.git (fetch)
origin git@atomserver:mr_line_number.git (push)
$ git remote set-url origin git@atomserver:tools/mr_line_number.git
$ git remote -v 
origin git@atomserver:tools/mr_line_number.git (fetch)
origin git@atomserver:tools/mr_line_number.git (push)

Once completed, the next git push origin master will successfully flow to the new location on atomserver

The changes pushed should be visible in GITWEB as well.

New Clone

If a user new to this tool decides to clone it after this operation is completed they will need to ensure the path includes tools. Assuming that this user keeps their clones in a work directory…

$ cd work
$ git clone git@atomserver:tools/mr_line_number.git
$ git remote -v
origin git@atomserver:tools/mr_line_number.git (fetch)
origin git@atomserver:tools/mr_line_number.git (push)

Once completed, the project should be cloned normally and the remote should be set as expected.

CONCLUSION

Following the above general technique, it should be possible for a GIT administrator to be able to re-structure project repos logically for easier navigation and maintenance. 

Created 23-December–2015 by uuklanger

No comments: