Unite the Files API 🗂

[daviddias]

Important note: This is not an entirely new proposal, it has been discussed several times over the course of the last 9 months in different situations with different views and opinions across the IPFS team. This comes to formalize the process so that we can commit and move it forward :)

Several of our users have been mislead to think of IPFS of File System instead of a Graph Database (fair point that is the name of the project after all:D). This mostly comes from the fact that the cat, get and addcommands come as first order commands and are the ones mostly used by demos.

Now, with the addition of the 'Files API', another layer of complexity and indirection was added. The common reaction to it is "Wait, what is Files API, weren't we adding Files all this time?".

With this, we miss the chance to lead our users to understand what a great Graph Database primitives IPFS offers and we also make it really hard for users to understand what Files API is, specially when it has such a generic name.

So, were is the proposal (this is not just me, although I'm the one writing it) that we've discussed through several points in time.

Rename the Files API to mfs, this will enable us to have one non generic keyword to call it that we can use with our users and also something that the community will be able to search for or aspect specially, since it has very technical details.

Move the cat, get and add commands to under the files umbrela.

In practical terms, this means:

# currently, the file manipulation commands are:
ipfs cat
ipfs get
ipfs add
ipfs files mkdir
ipfs files flush
ipfs files read
ipfs files cp
ipfs files ls
ipfs files stat
ipfs files rm
ipfs files write

Next:

# Manipulation of files
ipfs files cat
ipfs files get
ipfs files add

# mfs - Mutable File System
ipfs mfs mkdir
ipfs mfs flush
ipfs mfs read
ipfs mfs cp
ipfs mfs ls
ipfs mfs stat
ipfs mfs rm
ipfs mfs write

[dignifiedquire]

There is also currently ipfs files ls which needs to be reviewed in this context

[Kubuxu]

Also the names are long established so even if ipfs get was to be moved somewhere else backward support should be provided.

This separation is meaningful from interworks perspective but from user perspective it makes almost no sense. I can say that most of my ipfs command usage is ipfs get, ipfs cat, ipfs add and ipfs ls.

[hackergrrl]

This may also be a signal that the ipfs command is turning into a grab bag for too many abstractions one could build over basic IPFS blocks (object, files, and mfs).

A slightly more radical idea is to break these out into separate commands altogether, and have ipfs be a lightweight core that only manipulates objects (or maybe even just blocks) and we offer other programs to interpret those objects (or blocks) as these various things.

[whyrusleeping]

Action items:

• reference issues from go-ipfs where the decisions for the ipfs files commands were made
• discuss how ipfs add can be unified into the files api by making an entry below the files root -> Unite the Files API 🗂 #98 (comment)
• request feedback from community

[jbenet]

links to take into account:

• Change all mfs references to unixfs kubo#2233 (comment)
• Implement mfs API kubo#1550 (comments, mostly hidden)
• Implement mfs API kubo#1550 (comment)
• implement ipfs files command kubo#1769
• Update help text to avoid confusion of where files can be added from kubo#1826 (comment)
• Fast add stuff kubo#2039

[daviddias]

@whyrusleeping could you elaborate on the point number two: discuss how ipfs add can be unified into the files api by making an entry below the files root, so that everyone can participate in that discussion. Thank you!

[whyrusleeping]

What i was thinking was this:

Every call to ipfs add should make an entry inside the files api namespace, perhaps at /added/<DATE>/filename or /added/<INDEX>/filename. Doing this would accomplish two things:

1. Solve one of our oldest issues: "Where do i see the files i've added?"
2. Make the ipfs add more at home as ipfs files add

[Kubuxu]

Unifying ipfs add and ipfs files add right now will change hashes returned by the ipfs add as ipfs files add uses trickledag.

[daviddias]

@Kubuxu could you confirm, AFAIK ipfs files add doesn't exist

[Kubuxu]

Oh, there isn't ipfs files add but ipfs files write does that.

[whyrusleeping]

@Kubuxu there isnt an issue, ipfs files add can continue using the standard chunking scheme and also create entries into the files api namespace. The only caveat is that you wouldnt get the same hash for a file if you used ipfs files write to recreate it

[Kubuxu]

Or we can wait for 0.5 where we introduce CIDv1 so there the pain of changing hash function (which is the case) is much smaller and can be easily checked for (version byte).

[whyrusleeping]

The hashes will not "change totally" in 0.5. The default object format will change so adding the same content on 0.5 will produce a different hash than adding that content with 0.4. You will still be able to select the old hashing and chunking/encoding format as an option to add and get the same output as before. Also, old hashes will still be accessible and available, all thats happening is a change in defaults.

[daviddias]

Can we have this as one of the discussions for Monday Sprint hangouts? //cc @RichardLitt @em-ly @flyingzumwalt

It seems to me that we have all the information we need to move forward :)

[flyingzumwalt]

I added this to the agenda for the all-hands call. That's a chance to briefly tell everyone about the issue, make sure everyone knows that the breakout discussion will be happening, and find out who wants to be included in that discussion.

[jbenet]

• ipfs mfs write <thing> seems way over the top to me, it doesn't pass the red face test trying to tell people like Brewster to use this command.

---

I don't full understand why this is bad:

ipfs files cat
ipfs files get
ipfs files add
ipfs files ls
ipfs files mkdir
ipfs files flush
ipfs files read
ipfs files cp
ipfs files stat
ipfs files rm
ipfs files write

• Particularly if we teach ipfs files cat to cat from the $repohash/files space if the identifier has no hash prefix. is there a problem with this @whyrusleeping ?

[jbenet]

• i know there's "sort of two plumbing things" here, but we're talking about a porcelain command.
• i'm more OK with ipfs mfs being a plumbing command, but the command users are told to use and encouraged to use (porcelain) should be way more obvious and understandable to them. ipfs mfs is like "wat". ipfs files says "hey, use this to store and get files".

[daviddias]

I don't necessarily see it as being bad, specially with the addition of adding to mfs the hashes that get added by add, so that it is easy to see what is being stored.

[jbenet]

Yeah i think that would solve all our problems, no?
On Sun, Aug 21, 2016 at 10:34 David Dias notifications@github.com wrote:

[image: image]
https://cloud.githubusercontent.com/assets/1211152/17837793/a5862e56-67b4-11e6-9f87-917d2aea74a0.png

I don't necessarily see it as being bad, specially with the addition of
adding to mfs the hashes that get added by add, so that it is easy to see
what is being stored.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#98 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIcodg_ixkGfs-NNdHJF62ppo5T4cDsks5qiGH5gaJpZM4IaiDn
.

[kevina]

Slightly related ipfs/kubo#2811 as how ever this is solved might solve that issue.

[daviddias]

Just to keep record. Me and @whyrusleeping had a quick chat whole we were together in Lisbon and we agreed that #98 (comment) is a good way to go next.

@whyrusleeping please Ack just to make sure we continue in sync :)

[lidel]

I believe this discussion continues in https://github.com/ipfs/interface-js-ipfs-core/issues/284
@daviddias do you feel we should close/archive this one?