Ramblings of Daniel Graziotin

How to synchronize an Overleaf LaTeX paper with a Github repository

H

As a happy PeerJ user, I found out that I was eligible for a complimentary account at Overleaf (a.k.a. writeLaTeX). Overleaf is a collaborative writing and publishing system built around LaTeX. Strictly speaking, Overleaf is a Google Docs for LaTeX writers. It is neat and simple to use, to the point that it awakened my desire to use LaTeX again.

https://twitter.com/dgraziotin/status/555000333395824640

I encourage you to try it out. One of the features that I love about Overleaf is the integrated Git server for documents. This allows working off-line and 2-way synchronizing the changes with Overleaf server. Neat. Here is a nicely written review of the git feature and some example usage.

Scientists (not only computer scientists) are increasingly adopting Github for working on data and writing papers. Several papers, written publicly or through private repositories, are on Github’s git repositories. Overleaf does not yet offer a direct way to synchronize to Github git repositories1.

Luckily, there is a way to synchronize an article between Overleaf, a local computer, and a Github repository. This is because it is in git’s nature to be distributed (and decentralized, but this does not matter for us now). While this is quite simple for those accomplished with git, I understand that many users not from computer science might struggle in how to achieve this 3-way synchronization. That is why I explain how to achieve this below, using simple steps.

Suppose you have an Overleaf article at

https://www.overleaf.com/2029559gkypzx

.
Overleaf provides a related git repository at

https://git.overleaf.com/2029559gkypzx

.

Create a Github repository (public or private, it will work both ways). Let’s say that the repository is called

paper

, and it is reachable at

https://github.com/dgraziotin/paper.git

.

First, clone your Overleaf paper’s git repository

$ git clone https://git.overleaf.com/2029559gkypzx paper
Cloning into 'paper'...
remote: Counting objects: 13, done
remote: Finding sources: 100% (13/13)
remote: Getting sizes: 100% (8/8)
remote: Compressing objects: 100% (103172/103172)
remote: Total 13 (delta 1), reused 4 (delta 1)
Unpacking objects: 100% (13/13), done.
Checking connectivity... done.

Overleaf git repository becomes the so called

origin

endpoint. You can optionally rename it to overleaf by giving the following from inside the paper folder:

$ git remote rename origin overleaf

Anytime you want to synchronize locally the changes done via Overleaf, you will pull from the overleaf repository.

$ git pull overleaf master
remote: Counting objects: 5, done
remote: Finding sources: 100% (3/3)
remote: Getting sizes: 100% (4/4)
remote: Compressing objects: 100% (3018/3018)
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From https://git.overleaf.com/2029559gkypzx
 * branch            master     -> FETCH_HEAD
   25386da..2e1ed55  master     -> overleaf/master
Updating 25386da..2e1ed55
Fast-forward
 main.tex | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Anytime you want to synchronize the local changes to Overleaf, you will commit a change (line 1) and push to the overleaf repository (line 5).

$ git commit -a -m "Added ref to Smith et al. 2015"
[master 09d880e] Added ref to Smith et al. 2015
 1 file changed, 2 insertions(+), 2 deletions(-)

$ git push overleaf master
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 326 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1)
remote: Updating references: 100% (1/1)
To https://git.overleaf.com/2029559gkypzx
   2e1ed55..09d880e  master -> master

So far, so good. The same commands would have been used when employing a Github git repository 2. How about having both of them? We can add our Github repository with the following command:

$ git remote add github https://github.com/dgraziotin/paper.git

Alright. Now we can pull from Overleaf and push to Github. There is the need to do a first code push.

$ git push github
Counting objects: 19, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (19/19), done.
Writing objects: 100% (19/19), 97.82 KiB | 0 bytes/s, done.
Total 19 (delta 4), reused 0 (delta 0)
To https://github.com/dgraziotin/paper.git
 * [new branch]      master -> master

From now own, a flow might be

git pull overleaf master

and

git push overleaf master

.

Of course, you could pull from overleaf, do some changes, commit, and push again to overleaf, then to Github. Or, you could pull from Github, do some changes, commit, and push again to Github and Overleaf. Still, there is some redundancy we can get rid of. How about pushing to both repositories? Let’s create an endpoint called

both

.

$ git remote add both https://git.overleaf.com/2029559gkypzx
$ git remote set-url --add --push both https://git.overleaf.com/2029559gkypzx
$ git remote set-url --add --push both https://github.com/dgraziotin/paper.git

The first line creates a new remote repository URL called

both

. It points to Overleaf git server (again). However, the second command adds a new URL for pushing changes to the both endpoint, which points to the Github server. From now own, we can

push both

repositories at the same time (line 4).

$ git commit -a -m "Added a test word in the first line of introduction"
[master af9546d] Added a test word in the first line of introduction
 1 file changed, 1 insertion(+), 1 deletion(-)
git push both
Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 941 bytes | 0 bytes/s, done.
Total 5 (delta 4), reused 0 (delta 0)
To https://github.com/dgraziotin/paper.git
   67c8294..022a8f5  master -> master
Counting objects: 9, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 941 bytes | 0 bytes/s, done.
Total 5 (delta 4), reused 0 (delta 0)
remote: Resolving deltas: 100% (4/4)
remote: Updating references: 100% (1/1)
To https://git.overleaf.com/2029559gkypzx
   67c8294..022a8f5  master -> master

The example above is a local change propagated to both Overleaf and Github. Finally, in order to propagate a change from Overleaf to local and Github, the following commands will be required.

$ git pull overleaf master
remote: Counting objects: 1, done
remote: Finding sources: 100% (1/1)
remote: Total 1 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (1/1), done.
From https://git.overleaf.com/2029559gkypzx
 * branch            master     -> FETCH_HEAD
   6cc6b05..409e750  master     -> overleaf/master
Updating 6cc6b05..409e750
Fast-forward
 main.tex | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
$ git push github
Counting objects: 8, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 650 bytes | 0 bytes/s, done.
Total 6 (delta 2), reused 0 (delta 0)
To https://github.com/dgraziotin/paper.git
   af9546d..409e750  master -> master

That is all. Happy LaTeX writing!

  1. This feature has been requested.
  2. Please note that using Git with several people working on the same files is far more complicated than how it is shown here. There are many useful posts on how to use git with teams. Please take the time to read them.

About the author

dgraziotin

Dr. Daniel Graziotin received his PhD in computer science, software engineering at the Free University of Bozen-Bolzano, Italy. His research interests include human aspects in empirical software engineering with psychological measurements, Web engineering, and open science. He researches, publishes, and reviews for venues in software engineering, human-computer interaction, and psychology. Daniel is the founder of the psychoempirical software engineering discipline and guidelines. He is associate editor at the Journal of Open Research Software, academic editor at the Research Ideas and Outcomes (RIO) journal, and academic editor at the Open Communications in Computer Science journal. He is the local coordinator of the Italian Open science local group for the Open Knowledge Foundation. He is a member of ACM, SIGSOFT, and IEEE.

10 comments

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • The registered changes in the “Versions” tab of the online editor, are registered as commits messages in the repository log.

  • Am I missing something- I can push local changes to overleaf just fine, but if changes are made in overleaf, i cannot pull them down locally- I continue to get an “Already-up-to-date” message in my local repo. Is there some way I need to stage changes in Overleaf for git to recognize it? (that seems silly)

  • Thanks for this; I got the basic setup working except for one big problem: Overleaf is rejecting a load of support files I have (e.g. a makefile I have for building locally).

    Have you encountered this and know a good way to work around it? I thought about selectively pushing only accepted files to Overleaf and pushing everything to Github, but it sounds messy no matter how I try to think about it.

  • I feel like you are missing a huge point here. The fact that any change made to the document directly via the Overleaf page results in non manageable git commits that all use the same commit message ‘Update on Overleaf’ sort of messes up the git-“integration”. How am I supposed to make sense of the git history of my contributors when all I get is an amazingly large commit with no other info but the obvious ‘Update on Overleaf’?

    • This is how Overleaf works internally. Nothing to do with any synchronization mechanism described here, unfortunately. You would see the same mess when working directly with the Overleaf git repository. We could see an improvement after the merge with ShareLaTeX, especially if they are going to adopt ShareLaTeX integration with GitHub.

Ramblings of Daniel Graziotin

About Author

dgraziotin

Dr. Daniel Graziotin received his PhD in computer science, software engineering at the Free University of Bozen-Bolzano, Italy. His research interests include human aspects in empirical software engineering with psychological measurements, Web engineering, and open science. He researches, publishes, and reviews for venues in software engineering, human-computer interaction, and psychology. Daniel is the founder of the psychoempirical software engineering discipline and guidelines. He is associate editor at the Journal of Open Research Software, academic editor at the Research Ideas and Outcomes (RIO) journal, and academic editor at the Open Communications in Computer Science journal. He is the local coordinator of the Italian Open science local group for the Open Knowledge Foundation. He is a member of ACM, SIGSOFT, and IEEE.