git-annex/ design/ assistant

These are the design pages for the git-annex assistant.

Parts of the design is still being fleshed out, still many ideas and use cases to add. Feel free to chip in with comments! --Joey

See roadmap for current plans, as this list was mostly completed.

## initial development kickstarter year overview (2012-2013)

- Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
- Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
- Month 3 "easy setup": [[!traillink configurators]] [[!traillink pairing]]
- Month 4 "cloud": [[!traillink cloud]] [[!traillink transfer_control]]
- Month 5 "cloud continued": [[!traillink xmpp]] [[!traillink more_cloud_providers]]
- Month 6 "9k bonus round": [[!traillink desymlink]]
- Month 7: user-driven features and polishing; presentation at LCA2013
- Month 8: [[!traillink Android]]
- Month 9: screencasts and polishing
- Month 10: bugfixing, Android webapp
- Month 11: [[!traillink Windows]] porting, finishing touches
- Month 12: presentation at SELF2013, finishing touches

## porting

- OSX port is in fairly good shape, but still has some room for improvement
- android port is zooming along
- Windows port is barely getting started

## not yet on the map:

- rate limiting
- partial content
- deltas
- leftovers
- other todo items

## polls

I post polls occasionally to make decisions. You can vote!

## blog

I'm blogging about my progress in the devblog on a semi-daily basis. Follow along!

macports

The average OSX user has a) no idea what macports is, and b) will not be able to install it. Anything that requires a user to do anything with a commandline (or really anything other than using a GUI installer) is effectively a dealbreaker. For our use cases OSX is definitely a requirement, but it must only use standard OSX installation methods in order to be usable. Being in the appstore would be ideal, but standard dmg/pkg installers are still common enough that they are also acceptable.

FWIW this is the same reason many git GUIs were not usable for our OSX users: they required separate installation of the git commandline tools.

Comment by http://wiggy.net/ — Wed Oct 29 14:05:27 2025
Homebrew instead of MacPorts

Homebrew is a much better package manager than MacPorts IMO.

Comment by https://www.google.com/accounts/o8/id?id=AItOawldKnauegZulM7X6JoHJs7Gd5PnDjcgx-E — Wed Oct 29 14:05:27 2025
comment 7
Thanks, that's already been useful to me. You might as well skip the debian-specific "bpo" tags too.
Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
comment 6

It's not much for now... but see http://www.sgenomics.org/~jtang/gitbuilder-git-annex-x00-x86_64-apple-darwin10.8.0/ I'm ignoring the debian-stable and pristine-tar branches for now, as I am just building and testing on osx 10.7.

Hope the autobuilder will help you develop the OSX side of things without having direct access to an osx machine! I will try and get gitbuilder to spit out appropriately named tarballs of the compiled binaries in a few days when I have more time.

Comment by https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus — Wed Oct 29 14:05:27 2025
comment 5

In relation to macports, I often found that haskell in macports are often behind other distros, and I'm not willing to put much effort into maintaining or updating those ports. I found that to build git-annex, installing macports manually and then installing haskell-platform from the upstream to be the best way to get the most up to date dependancies for git-annex.

fyi in macports ghc is at version 6.10.4 and haskell platform is at version 2009.2, so there are a significant number of ports to update.

I was thinking about this a bit more and I reckon it might be easier to try and build a self contained .pkg package and have all the needed binaries in a .app styled package, that would work well when the webapp comes along. I will take a look at it in a week or two (currently moving house so I dont have much time)

Comment by https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus — Wed Oct 29 14:05:27 2025
comment 4

I always appreciate your OSX work Jimmy...

Could it be put into macports?

Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
comment 3
I'd agree getting it into the main distros is the way to go, if you need OSX binaries, I could volunteer to setup an autobuilder to generate binaries for OSX users, however it would rely on users to have macports with the correct ports installed to use it (things like coreutils etc...)
Comment by https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus — Wed Oct 29 14:05:27 2025
comment 20
@Georg, DMG have been built for OSX for a long time. See OSX
Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
comment 2
Jimmy, I hope to make it as easy as possible to install. I've been focusing on getting it directly into popular Linux distributions, rather than shipping my own binary. The OSX binary is static, and while I lack a OSX machine, I would like to get it easier to distribute to OSX users.
Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
windows port
I'd love to use the Windows port of the assistant by Christmas. Keep up the good work! :)
Comment by https://www.google.com/accounts/o8/id?id=AItOawlnoH5btjn_BLib3_IhES5uMhrzuOiwCYo — Wed Oct 29 14:05:27 2025
for OSX, package managers (homebrew and macports) are really second-class
at least with regards to wide spread use. I agree with Wichert, the average OSX user won't use them. I myself would go with homebrew (because it is far less of a pain than macports), but actually static built binaries that are just dragged from a DMG to your programs folder is the way to go if you can't go osx app store. It's just how most Apple users are wired. PKG would be fine, too, since most will know them, but those are really only needed if you need to place stuff in /Library/Frameworks or things like that. If it can be done as a all-inclusive OSX .app, do it that way and just pack it up in a .DMG.
Comment by https://www.google.com/accounts/o8/id?id=AItOawm7AuSfii_tCkLyspL6Mr0ATlO6OxLNYOo — Wed Oct 29 14:05:27 2025
comment 17
@gdr A package with the assistant is available in Debian unstable.
Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
Maybe a DEB?2
Month 3 was all about easy setup, so I kind of expected to download a deb package and just install it, not to download a whole bunch of haskell libraries. Is there a chance that you will release some packages?
Comment by https://launchpad.net/~gdr-go2 — Wed Oct 29 14:05:27 2025
comment 15

We're on month 4 of work, and most of months 1-3 is done well enough for a first pass. Very little of what's listed in month 4 has happened yet, due to my being maybe 2 weeks behind schedule, but bits of cloud are being planned.

I've made a small adjustment, I think it'll make sense to spend a month on user-driven features before getting into Android.

Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
you rock! & roadmap update?

joey, you rock. I just want to push on that point - it doesn't seem like there's that many tools that do what git-annex is trying to do out there, and you seem to be doing an incredible job at doing it, so this is great, keep going!

i was wondering - i am glad to see the progress, but it is unclear to me where you actually are in the roadmap. are things going according to plan? are we at month 3? 4? or 1-4? :) just little updates on that roadmap section above would be quite useful!

thanks again!

Comment by https://id.koumbit.net/anarcat — Wed Oct 29 14:05:27 2025
comment 13
@Jörn, two days ago I added support in the webapp for multiple independent repositories. So you can create independent repos using it, and switch between them from a menu. Under the hood there are multiple git-annex assistant daemons running, but this is an implementation detail; it's not like these use any more memory or other resources than would a single daemon that managed multiple repositories.
Comment by http://joeyh.name/ — Wed Oct 29 14:05:27 2025
Multiple annexes?

Thank you for this great piece of software which is now becoming even better with the assistant.

Just one question: will one instance of the assistant be able to track multiple git annex repositories each building up their own network of annexes OR would I need to run multiple instances of the assistant?

Thanks, Jörn

Comment by https://www.google.com/accounts/o8/id?id=AItOawlu-fdXIt_RF9ggvg4zP0yBbtjWQwHAMS4 — Wed Oct 29 14:05:27 2025
comment 11

Yeah definately go with homebrew rather than macports if possible. macports and fink, whilst great systems, have a tendency to sort of create their own alternative-dimension of files within the system that just dont always feel particularly well integrated. As a result "brew" has become increasingly more popular to the point its almost ubuquitous now.

Plus its brew-doctor thing is awesome.

The best approach though thats agnostic to distro systems is to simply go for a generic installer.

Comment by https://www.google.com/accounts/o8/id?id=AItOawneJXwhacIb0YvvdYFxhlNVpz6Wpg6V7AA — Wed Oct 29 14:05:27 2025
Watch also possible with git?

Hi,

it seems that you put a lot of efforts in handling race conditions. Thats great. I wonder if the watch can also be used with git (i.e. changes are commited into git and not as annex)? I know that other projects follow this idea but why using different tools if the git-annex assistant could handle both...

Comment by http://www.klomp.eu/ — Wed Oct 29 14:05:27 2025
comment 1
Will statically linked binaries be provided for say Linux, OSX and *BSD? I think having some statically linked binaries will certainly help and appeal to a lot of users.
Comment by https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus — Wed Oct 29 14:05:27 2025
Comments on this page are closed.