Support for Bintray

[ghost]

Hi,

This solution is pretty cool. Setting up Amazon S3 looks scary though. Something like bintray might be cool?

https://bintray.com/

[tmspzz]

I'll look into it!

In the mean time you can also use https://www.minio.io/

Take a look at #118

[tmspzz]

@rob-nash I took a look at Bintray and I have few questions I could not find an answer to.

• Bintray's Authentication requires Username/APIKey. Is there a standard location to store these? like ~/.bintray/credentials
• Bintray's repos are namespaced with :subject.

GET https://dl.bintray.com/:subject/:repo/:file_path
GET https://:subject.bintray.com/:repo/:file_path // Enterprise only

I guess this value should be configurable rather than just take the username of the current user?

I don't have access to the enterprise version so I can't implement anything that requires that version.

[ghost]

Hi there.

I am new to Bintray myself. I setup an open source account yesterday and uploaded some binaries via their API. Seems pretty good.

Open source accounts are free. I think I need to accept a EULA before I attempt my first download of one of these binaries. I'll get back to you on that.

Facebook v0.3.0 is 18Mb so my Carthage bootstrap is pretty slow! This is a snapshot of an API call I built with postman.

The body contains a zip of all the files built by Carthage.

baseURL = https://api.bintray.com

Authentication is basic. So base64 encode the following.

username:APIKey

example

rob-nash:57fexamplekey

A checksum of the file you're uploaded is probably worth doing. Can achieve like so

openssl dgst -sha256 /full/path/to/file

Maybe just support open source accounts, for now?

[ghost]

If I attempt to download one of these binaries myself with

https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip

I don't need to accept a EULA.

The repo is public so I think someone attempting to download this without a Binary account, might get an error code and must accept a EULA, beforehand.

It might be easier to mention in README

1. create a free account
2. accept EULA
3. use Rome

[tmspzz]

Hi,

The cache structure that Rome uses is not quite what you have there, so I'll explain to understand if this is fine also for Bintray.

.dSYM, .framework, .version, .bsymbolmap are handled separately. They are zipped separately and stored at separate locations. Locations where artifacts are stored follow some conventions derived from the Cartfile.resolved.

See https://github.com/blender/Rome#cache-structure

For example, dequable would be stored at:

https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.1.3.0.zip
https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.dSYM.1.3.0.zip
https://dl.bintray.com/rob-nash/Dequable/Dequable.version.1.3.0.zip

If I break this down in Bintray terms this would be

https://dl.bintray.com/:subject/:repo    /:file_path
https://dl.bintray.com/rob-nash/Dequable/iOS/Dequable.framework.1.3.0.zip

I'm trying to understand if this is legal with the API or not.

[ghost]

The 'file_path' parameter can specify any path. So for instance, I currently have one file located under /iOS

But I guess there is no harm is building any file structure you want, inside the bintray repo.

Could we not just create one file named Dequable.zip which contains all 3?

[ghost]

The following would work.

https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.framework.1.3.0.zip
https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.framework.dSym.1.3.0.zip
https://api.bintray.com/content/rob-nash/BinaryDump/iOS/Dequable.version.1.3.0.zip

[tmspzz]

Cool! I'll play some more around and see that I can do.

The reasons for not having one zip are:

1. Have the version file accessible ahead of downloading the binary (the version file is like a manifest)
2. Allow upload/download of resources separately
3. Don't have to replace the whole zip if one file is missing

[tmspzz]

@rob-nash When you uploaded your binaries, what did you use as version in the api?

In my idea this would be the version of the library you're trying to upload as stated in the Cartfile.resolved .

What is the point of version ? I don't see how it's used in the download API.

[ghost]

For Uploading

Set a header in your request

X-Bintray-Version

For Downloading

In the above example Dequable is a package.

Get package will give you the available versions.

Get Version Files returns the files found for a given version.

You will be able to extract a path from these responses.

[tmspzz]

So what version is at https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip ? Where is the package information here?

I find this service very confusing in general, or at least confusing to use given how Rome works.

Rome has to go from a Cartfile.resolved like this

github "Alamofire/Alamofire" "4.7.2"
git "ssh://git@stash.someserver.com:7999/iossdk/ios-rest.git" "v7.1.0"
binary "https://github.com/Building42/Specs/master/Carthage/Answers.json" "1.3.6" 

to upload/download locations in Bintray

The upload API is:

PUT /content/:subject/:repo/:package/:version/:file_path

:subject will be something configurable but :repo and :package have to be derivable from the Cartfile.resolved

How would Rome derive :repo and :package from the above Cartfile.resolved for all 3 cases?

Note that ios-rest is not the final name of the framework binary, this is why there is a https://github.com/blender/Rome#repositorymap section

[ghost]

You have a lot of work and Bintray looks like quite a bit of effort. Let me see if I can build a small wrapper for Bintray, deployable as a cross-platform executable.

It might take me a while, so you might want to close this ticket for now.

If I am successful, I will re-open this ticket, so you can take a look at my work and assess if it is suitable for Rome.

What you think?

[ghost]

Just had a look at the API.

So what version is at https://dl.bintray.com/rob-nash/BinaryDump/iOS/dequable.zip ? Where is the package information here?

My binary profile currently has Dequable 1.3.0

When I attempt to upload 1.2.0 with the same :file_path and the following headers, I receive a 409 conflict error code and a message.

{
"message": "Unable to upload files: An artifact with the path 'iOS/dequable.zip' already exists under another version"
}

Navigating the website, I can see that the file directory responds to version filters. So it's important to realise two things here.

1. The metadata must include the version number
2. The file path must be unique.

So going forward, a naming convention is required. Such as

/content/rob-nash/BinaryDump/iOS/dequable_1.2.0.zip
/content/rob-nash/BinaryDump/iOS/dequable_1.3.0.zip

:subject will be something configurable but :repo and :package have to be derivable from the Cartfile.resolved. How would Rome derive :repo and :package from the above Cartfile.resolved for all 3 cases?

:subject = rob-dash
Agreed, this must be configurable.

:repo = BinaryDump
One caveat to using this service, is that you need to create and name a repo. There can be many repos. The onus should be on the user, in my opinion, to create one and provide the name to Rome. So, this also has to be configurable.

:package = Dequable
This can be extracted from Cartfile.resolved. In your above example, it would be Alamofire.

Note that ios-rest is not the final name of the framework binary, this is why there is a https://github.com/blender/Rome#repositorymap section

So we can supply the following paths

iOS/Dequable.framework.1.3.0.zip
iOS/Dequable.framework.dSym.1.3.0.zip
Dequable.version.1.3.0.zip

Get Version Files will return information about all of these files.

[ghost]

Looks like there is already a wrapper 👍

Phew!

https://www.npmjs.com/package/bintray

https://github.com/h2non/node-bintray

[tmspzz]

If I understand correctly your idea is to have 1 :repo where each dependency in the Cartfile.resolved is a :package.

So, in practice, given the following Cartfile.resolved

github "Alamofire/Alamofire" "4.7.2"
git "ssh://git@stash.someserver.com:7999/iossdk/ios-rest.git" "v7.1.0"
binary "https://github.com/Building42/Specs/master/Carthage/Answers.json" "1.3.6"

where ios-rest.git resolves to 2 targets Rest.framework and Helper.framework

It would translate to (omitting bcsymbolmaps):

PUT /content/:subject   /:repo    /:package  /:version/:file_path
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/iOS/Alamofire.framework.1.3.0.zip
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/iOS/Alamofire.framework.1.3.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/Alamofire/   4.7.2   /Alamofire/Alamofire.version.1.3.0.zip

PUT /content/:subject   /:repo  /:package  /:version /:file_path
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Rest.framework.7.1.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Rest.framework.7.1.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/ios-rest.version.1.3.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Helper.framework.7.1.0.zip
PUT /content/rob-nash/BinaryDump/ios-rest/   7.1.0   /ios-rest/iOS/Helper.framework.7.1.0.dSYM.zip

PUT /content/:subject   /:repo  /:package  /:version   /:file_path
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/iOS/Alamofire.framework.1.3.0.zip
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/iOS/Alamofire.framework.1.3.0.dSYM.zip
PUT /content/rob-nash/BinaryDump/Answers/   1.3.6      /Answers/Answers.version.1.3.0.zip

So, puts are solved.

How about Gets?

Should the flow be:

• get package
• get version
• get final URL ?

[ghost]

Hi. Sorry for delay.

Yes, I believe it should be as you say. Perform get 1, perform get 2, then build URL.

[ghost]

It looks like the API is still released as version 1, even after several years of it being in the public domain.

You may want to use that specific version, to be safe.

https://bintray.com/api/v1

[tmspzz]

@rob-nash Thanks to @BalestraPatrick you can now use whatever backend you want

[tmspzz]

You can try the pre-release at: https://github.com/blender/Rome/releases/tag/v0.22.0.59