If you have been working with version control system for a while, you would know or speculated about the possibility of including references from other projects into your project. From a programmer’s point of view, this feature is useful especially when you need to include code libraries owned by someone else or common header/footer files to be used across different repositories.
Like Subversion externals, git submodule allows one to work with multiple repositories within one main repository. Unfortunately, git submodule resources were scarce on the web. It could also be confusing for people coming from subversion background because it operates differently from it. Despite sharing the same concept, the general feedback was that many people find it hard to digest at a first glance.
In this tutorial, we will be going through an example of how to use git submodule in a typical software development process. We will also be setting up bare repositories from the start, so feel free to skip the first 3 steps if you already know the basics. A bit of basic Git and command line knowledge is assumed.
Say we have 2 developers working on an app that uses a common library (submodule) and this is the workflow:
- Scenario 1: dev1 is the lead developer and sets up bare origin repository structure under the /mnt/demo directory.
- Scenario 2: submodule-origin needs something before the team continue. Dev1 creates a master branch and commits a file to the empty submodule-origin.
- Scenario 3: app-origin cannot be empty as well. Again, dev1 creates a master branch on app-origin and adds a file to it.
- Scenario 4: The team now has the essential environment to start playing with git submodule. dev1 adds the submodule to its local repo, commits the file and pushes it to submodule-origin.
- Scenario 5: dev1 is pretty happy with everything and decides to tag his changes.
- Scenario 6: dev2 comes along and wants to make some code changes to both the app and the submodule. He does a clone, modify some files and commits.
- Scenario 7: Dev2 is happy with his changes and decides to tag it with v1.0.1
- Scenario 8: Dev1 now wants to get dev2 changes and does a pull.
- Scenario 9: Dev1 is not very happy with dev2 changes and decides to revert everything back to v1.0.0
Hopefully, the snapshots of the command line and comments are self-explainatory…
Scenario 1: dev1 is the lead developer and sets up bare origin repository structure under the /mnt/demo directory.
# init user globals first
dev1@localhost:~$ git config --global user.name "dev1"
dev1@localhost:~$ git config --global user.email dev1.localhost
# initialise bare app and submodule repo
dev1@localhost:~$ cd /mnt
dev1@localhost:/mnt$ sudo mkdir {demo,demo/app-origin,demo/submodule-origin}
dev1@localhost:/mnt$ sudo chmod 777 -R demo
dev1@localhost:~/mnt$ cd demo
dev1@localhost:/mnt/demo$ git init --bare app-origin/
dev1@localhost:/mnt/demo$ git init --bare submodule-origin/
Scenario 2: submodule-origin needs something before the team continue. Dev1 creates a master branch and commits a file to the empty submodule-origin.
dev1@localhost:~$ mkdir submodule
dev1@localhost:~$ cd submodule
dev1@localhost:~/submodule$ git init
Initialized empty Git repository in /home/dev1/submodule/.git/
dev1@localhost:~/submodule$ echo submodule > submodule
dev1@localhost:~/submodule$ git add submodule
dev1@localhost:~/submodule$ git commit -m"init"
[master (root-commit) c61fab7] init
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 submodule
dev1@localhost:~/submodule$ git remote add origin /mnt/demo/submodule-origin/
dev1@localhost:~/submodule$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 207 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/submodule-origin/
* [new branch] master -> master
Scenario 3: app-origin cannot be empty as well. Again, dev1 creates a master branch on app-origin and adds a file to it.
# initialse the dev1 directory
dev1@localhost:~$ mkdir app
dev1@localhost:~$ cd app
dev1@localhost:~/app$ git init
Initialized empty Git repository in /home/dev1/app/.git/
dev1@localhost:~/app$ echo dev1 > dev1
dev1@localhost:~/app$ git add app
dev1@localhost:~/app$ git commit -m "init"
[master (root-commit) 28eda11] init
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 dev1
dev1@localhost:~/app$ git remote add origin /mnt/demo/app-origin/
dev1@localhost:~/app$ git push origin master
Counting objects: 3, done.
Writing objects: 100% (3/3), 199 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
* [new branch] master -> master
Scenario 4: The team now has the essential environment to start playing with git submodule. dev1 adds the submodule to its local repo, commits the file and pushes it to submodule-origin.
# add a submodule
dev1@localhost:~/app$ git submodule add /mnt/demo/submodule-origin/ submodule
if you are doing it correctly, you should see a .gitmodules file being created.
dev1@localhost:~/dev1$ cat .gitmodules
[submodule "submodule"]
path = submodule
url = /mnt/demo/submodule-origin
dev1@localhost:~/app$ git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD ..." to unstage)
#
# new file: .gitmodules
# new file: submodule
Let’s add the new files and push
dev1@localhost:~/app$ git add *
dev1@localhost:~/app$ git commit -m "add submodule"
[master c78abf9] add submodule
2 files changed, 4 insertions(+), 0 deletions(-)
create mode 100644 .gitmodules
create mode 160000 submodule
dev1@localhost:~/app$ git push origin master
Counting objects: 4, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 350 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
a94d038..c78abf9 master -> master
Scenario 5: dev1 is pretty happy with everything and decides to tag his changes.
dev1@localhost:~/app$ git tag v1.0.0
dev1@localhost:~/app$ git push --tags
Total 0 (delta 0), reused 0 (delta 0)
To /mnt/demo/app-origin/
* [new tag] v1.0.0 -> v1.0.0
Scenario 6: dev2 comes along and wants to make some code changes to both the app and the submodule. He does a clone, modify some files and commits.
# init user
dev2@localhost:~$ git config --global user.name dev2
dev2@localhost:~$ git config --global user.email dev2@localhost.com
# clone the app-origin first.
dev2@localhost:~$ git clone /mnt/demo/app-origin/ app
Initialized empty Git repository in /home/dev2/app/.git/
dev2@localhost:~/app$ ls
dev1 submodule
dev2 modifies the dev1 file and add it to be committed later.
dev2@localhost:~/app$ echo "dev1 dev2" > dev1
dev2@localhost:~/app$ git add dev1
let’s see what’s in the submodule
dev2@localhost:~/app$ ls submodule
Nothing?? Remember that git submodules are to be treated differently? Cloning the app doesn’t clone the submodule. Dev2 will have to do that manually.
dev2@localhost:~/app$ git submodule init
Submodule 'submodule' (/mnt/demo/submodule-origin) registered for path 'submodule'
dev2@localhost:~/app$ git submodule update
Initialized empty Git repository in /home/dev2/app/submodule/.git/
Submodule path 'submodule': checked out 'c61fab7265554f3e99632376102831e6712622b5'
now, we should see something in the submodule dir.
dev2@localhost:~/app$ ls submodule/
submodule
the “git submodule update” also created a dummy branch for you
dev2@localhost:~/app$ cd submodule/
dev2@localhost:~/app/submodule$ git branch
* (no branch)
master
Now say dev2 wants to fix a bug in the submodule and he wants everyone using the submodule to get it. To push changes in the submodule, he needs to re-checkout the master branch and push. Note that he needs to push again in the submodule dir because the submodule is now a repo within another repo. Despite the hierarchy, both repo need to be managed as if they are separate.
dev2 now checkout the master branch, modify the submodule file and commits.
dev2@localhost:~/app/submodule$ git checkout master
dev2@localhost:~/app/submodule$ echo "fixed bug" >> submodule
dev2@localhost:~/app/submodule$ git add submodule
dev2@localhost:~/app/submodule$ git commit -m "fixed bug"
[master 9ddfa96] fixed bug
1 files changed, 1 insertions(+), 0 deletions(-)
dev2@localhost:~/app/submodule$ git push origin master
Counting objects: 5, done.
Writing objects: 100% (3/3), 260 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/submodule-origin
c61fab7..9ddfa96 master -> master
back to the root, we should see the files/dir modified.
dev2@localhost:~/app/submodule$ cd ..
dev2@localhost:~/app$ git status
# On branch master
# Changed but not updated:
# (use "git add ..." to update what will be committed)
# (use "git checkout -- ..." to discard changes in working directory)
#
# modified: dev1
# modified: submodule
#
dev1 need to do another commit and push in the top level to include the submodule changes.
dev2@localhost:~/app$ git add dev1 submodule
dev2@localhost:~/app$ git commit -m "dev2 made both app and submodule changes"
[master a542a08] dev2 made both app and submodule changes
2 files changed, 2 insertions(+), 2 deletions(-)
dev2@localhost:~/app$ git push origin master
Counting objects: 5, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 327 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /mnt/demo/app-origin/
c78abf9..a542a08 master -> master
Scenario 7: ev2 is happy with his changes and decides to tag it with v1.0.1dev2 comes along and wants to make some code changes to both the app and the submodule, so he does a clone, modify the files and commit.
dev2@localhost:~/app$ git fetch --tags
From /mnt/demo/app-origin
* [new tag] v1.0.0 -> v1.0.0
dev2@localhost:~/app$ git tag v1.0.1
dev2@localhost:~/app$ git push --tags
Total 0 (delta 0), reused 0 (delta 0)
To /mnt/demo/app-origin/
* [new tag] v1.0.1 -> v1.0.1
Scenario 8: Dev1 now wants to get dev2 changes and does a pull.
dev1@localhost:~/app$ git pull origin master
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /mnt/demo/app-origin
* branch master -> FETCH_HEAD
Updating c78abf9..a542a08
Fast-forward
dev1 | 2 +-
submodule | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
Let’s check on the submodule file…
dev1@localhost:~/app$ cat submodule/submodule
submodule
Opps, where was the change that dev2 made on the submodule/submodule file? Dev1 tries doing a “git submodule update” and got the changes.
dev1@localhost:~/app$ git submodule update
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From /mnt/demo/submodule-origin
c61fab7..9ddfa96 master -> origin/master
Submodule path 'submodule': checked out '9ddfa961f29c3866cdcbef697811a185af3fc407'
Scenario 9: Dev1 is not very happy with dev2 changes and decides to revert everything back to v1.0.0
dev1@localhost:~/app$ git checkout v1.0.0
M submodule
Note: checking out 'v1.0.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at c78abf9... add submodule
dev1@localhost:~/app$ cat dev1
dev1
fixed bug
dev1 file is now reverted. How about the submodule?
dev1@localhost:~/app$ cat submodule/submodule
submodule
Opps, submodule file has not been reverted. dev1 does another submodule update to grab the old changes for the submodule.
dev1@localhost:~/app$ git submodule update
Submodule path 'submodule': checked out 'c61fab7265554f3e99632376102831e6712622b5'
dev1@localhost:~/app$ cat submodule/submodule
submodule
Hopefully, this example covers the basic use of git submodule. The trick is to treat the submodule as a separate repo within the top working repo. Remember to do “git submodule update” after you do a pull (if you wish to update the submodule).
