How to use Git LFS to store big files on GitHub

Michelle Duke - Mar 18 '21 - - Dev Community

I've been recently working on building my own game in Unity. Yes, it's partly a prebuilt game you can configure, but it's a game nonetheless. Like a lot of my live coding projects, I wanted to store all the code from this game on GitHub.

One of the things I quickly became aware of are the issues surrounding using git for game dev. Here are the three main problems I encountered:

  • long file names
  • a lot of files (like 10,000s)
  • some large files (over 100MB)

We quickly solved the long file naming problem. Check out my article below to see how to use the command line to change and commit all these long files.

Unfortunately, there's not too much we can do about the swear volume of files itself. But there's something we can do about the large files.

Uploading files to GitHub

Some of you have probably tried to manually upload files to GitHub. If you go to any repo and click "Add file", you have the option to create a new file or upload files. If you select "Upload files", you'll be able to choose files to upload or drag and drop. The functionality for this feature however is limited.

You can only add files not folders, and you can't upload anything more than 25MB. If you want to do either of these things, it's best to upload the files/folders using either the command line or GitHub Desktop.

github-desktop

But even that has its limits. While there's no limit (or I'm yet to discover one) on the number of files you can upload, there's a limit to the size of files. No single file can be more than 100MB. So what do you do if your file is over the limit? Sometimes there are files you definitely want to add to your repo.

Storing large files on GitHub

If you want to store a large file on GitHub you can. You'll need to use something called Git Large File Storage (LFS). Install Git LFS on your computer and then you can begin.

Once you've installed Git LFS, check to see if it's working:

git lfs

This will also provide you will a list of useful commands. Next, know where the file you want to store is located on your local machine. Navigate to that folder:

cd # type the location of the file here

Next, use the track command, to add the large file. For example, if your file is called "Devtopost.mp4" then you can track it by:

git lfs track "Devtopost.mp4"

If for example you had a whole lot of large *.mp4 files, you can track all the files at once:

git lfs track "*.mp4"

In this way you don't need to have all the individual files.

Next, we want to make sure all these files are being stored in git:

git add .gitattributes

This file above stores all the information about each of the large files. Finally, you can add, commit, and push as usual:

git add . This adds all the committed files. or use git add Devtopost.mp4 to add just the one changed file.

git commit -m "add those chunky files"

git push

Now you should have your large files stored on GitHub. Remember, there is a limit to the amount of files you can store on GitHub. It will depend on your plan.

lfs-website

Hopefully that gives you a bit of information on how to go about using version control on your large projects. You can check out my repo where we were using LFS for storage, and if you want to watch back all the goodness talked about here, you can rewatch the recording of our live stream.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player