git-annex/ tips/ migrating two seperate disconnected directories to git annex

Note: this is the reverse of splitting a repository.

## Scenario

You are a new git-annex user. You have already files spread around many computers and wish to migrate those into git-annex, without having to recopy all files all over the place.

Let's say, for example, you have a server, named `marcos` and a workstation named `angela`. You have your audio collection stored in `/srv/mp3` in `marcos` and `~/mp3` on `angela`, but only `marcos` has all the files, and `angela` only has a subset.

We also assume that `marcos` has an SSH server.

How do you add all this stuff to git-annex?

## Create the biggest git-annex repository

Start with `marcos`, with the complete directory:

    cd /srv/mp3
    git init
    git annex init
    git annex add .
    git commit -m "git annex yay"

This will checksum all files and add them to the `git-annex` branch of the git repository. Wait for this process to complete.

## Create the smaller repo and synchronise

On `angela`, we want to synchronise the git annex metadata with `marcos`. We need to initialize a git repo with `marcos` as a remote:

    cd ~/mp3
    git init
    git remote add marcos marcos.example.com:/srv/mp3
    git fetch marcos
    git annex info # this should display the two repos
    git annex add .

This will, again, checksum all files and add them to git annex. Once that is done, you can verify that the files are really the same as marcos with `whereis`:

    git annex whereis

This should display something like:

    whereis Orange Seeds/I remember.wav (2 copies)
            b7802161-c984-4c9f-8d05-787a29c41cfe -- marcos (anarcat@marcos:/srv/mp3)
            c2ca4a13-9a5f-461b-a44b-53255ed3e2f9 -- here (anarcat@angela)
    ok

Once you are sure things went on okay, you can synchronise this with `marcos`:

    git annex sync --allow-unrelated-histories

This will push the metadata information to marcos, so it knows which files are available on `angela`. From there on, you can freely get and move files between the two repos!

## Importing files from a third directory

Say that some files on `angela` are actually spread out outside of the `~/mp3` directory. You can use the `git annex import` command to add those extra directories:

    cd ~/mp3
    git annex import ~/music/

(!) Be careful that `~/music` is not a git-annex repository.

## Deleting deleted files

It is quite possible some files were removed (or renamed!) on `marcos` but not on `angela`, since it was synchronised only some time ago. A good way to find out about those files is to use the `--not --in` argument, for example, on `angela`:

    git annex whereis --in here --not --in marcos

This will show files that are on `angela` and not on `marcos`. They could be new files that were only added on `angela`, so be careful! A manual analysis is necessary, but let's say you are certain those files are not relevant anymore, you can delete them from `angela`:

    git annex drop <file>

comment 2

Indeed, you will need to use `git-annex sync --allow-unrelated-histories` now in that situation. I have updated the tip.

Comment by joey — Tue Jul 22 18:11:47 2025
This guide fails with "fatal: refusing to merge unrelated histories"

This no longer works, here is a MWE to copy-paste (uses /tmp/{A,B}):

    mkdir /tmp/A && touch /tmp/A/bigfile
    mkdir /tmp/B && touch /tmp/B/bigfile
    cd /tmp/A
    git init
    git annex init
    git annex add .
    git commit -m "git annex yay"
    cd /tmp/B
    git init
    git remote add A /tmp/A
    git fetch A
    git annex info # this should display the two repos
    git annex add .
    git annex whereis
    git annex sync

This fails with

    commit
    [main (root-commit) e9435bf] git-annex in stefan@notebook:/tmp/B
     1 file changed, 1 insertion(+)
     create mode 120000 bigfile
    ok
    pull A

    fatal: refusing to merge unrelated histories
    failed
    push A
    Enumerating objects: 19, done.
    Counting objects: 100% (19/19), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (11/11), done.
    Writing objects: 100% (14/14), 1.37 KiB | 1.37 MiB/s, done.
    Total 14 (delta 2), reused 0 (delta 0), pack-reused 0
    To /tmp/A
     * [new branch]      main -> synced/main
     * [new branch]      git-annex -> synced/git-annex
    To /tmp/A
     ! [rejected]        main -> main (non-fast-forward)
    error: failed to push some refs to '/tmp/A'
    hint: Updates were rejected because the tip of your current branch is behind
    hint: its remote counterpart. Integrate the remote changes (e.g.
    hint: 'git pull ...') before pushing again.
    hint: See the 'Note about fast-forwards' in 'git push --help' for details.
    ok
    sync: 1 failed

Comment by Stefan — Tue Jul 22 18:11:47 2025
Comments on this page are closed.