Skip to content

Commit

Permalink
feat(improve-indy-testnet-docker): auto-start server on container sta…
Browse files Browse the repository at this point in the history
…rtup #1308

Signed-off-by: Takuma TAKEUCHI <takeuchi.takuma@fujitsu.com>
  • Loading branch information
izuru0 authored and takeutak committed Sep 17, 2021
1 parent 3051ddf commit 35b20ac
Show file tree
Hide file tree
Showing 13 changed files with 438 additions and 79 deletions.
5 changes: 2 additions & 3 deletions tools/docker/indy-testnet/.env
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
INDY_SDK_ROOT=../../../../indy-sdk/ci
HTTP_PROXY=$HTTP_PROXY
NO_PROXY=$NO_PROXY
HTTP_PROXY=
NO_PROXY=

83 changes: 67 additions & 16 deletions tools/docker/indy-testnet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,13 @@ One bridge network and three containers will be started by `docker-compse up` co
- TODO: Configure to proxy requests. Enable uWSGI (towards the python container).
- IP address: 172.16.0.3 on indy_net
- port 10080 is open to the world
- python container
- clientbase
- this is a temporary container based on template image for indy client (used by validator/Dockerfile)
- immediately exits when you docker-compose up
- validator container
- validator code in the container receives requests from validators, which requests are proxied by the nginx container, and interacts with indy nodes in the indy_pool container
- NOTE: At the moment, this container does nothing.
- TODO: write validator code.
- IP address: 172.16.0.4
- port 80 is open to containers on indy_net
- port 8000 is open to containers on indy_net
- indy_pool container
- indy nodes run in this container and serves requests from validator in the python container
- it mounts sandbox directory of host environment to save the status of Indy nodes
Expand All @@ -34,29 +35,19 @@ One bridge network and three containers will be started by `docker-compse up` co

## How to build

- get a copy of indy-sdk
- edit `.env` to set environment variables
- run `run-before-build.sh`
- run `docker-compose build`

### How to get a copy of indy-sdk

You can use the following command to get a copy of indy-sdk source tree.

```
git clone https://github.com/hyperledger/indy-sdk.git -b v1.16.0
```
### How to edit `.env` file

Edit `.env` file in this directory to set the environment variables.

```
INDY_SDK_ROOT=../../git/indy-sdk/ci
HTTP_PROXY=http://<proxy_host>:<proxy_port>
NO_PROXY=
NO_PROXY=<local addresses>
```

`INDY_SDK_ROOT` is the location of a Dockerfile (`indy-pool.dockerfile`) in the indy-sdk source tree, which will be used by the docker-compose command in this build process.

Set `HTTP_PROXY` and `NO_PROXY` if your network requires HTTP proxy access to reach the internet.


Expand All @@ -77,3 +68,63 @@ docker-compse up

Press CTRL-C to stop the containers.

## How to verify that validator and ledger is up

Copy python script from example directory

1. Copy files from cactus and indy-sdk and edit address of the application (BLP)

Create a directory for a container.

```bash
cd cactus/tools/docker/indy-testnet
mkdir indycli
```

Copy python files from indy-sdk and cactus example directory and edit variable `http_req_params.url` in file `req_discounted_cartrade.py` to point to `http://host.docker.internal:5034/api/v1/bl/trades/` (localhost --> host.docker.internal).

```bash
cp -frp ../../../../indy-sdk/samples/python indycli
cp -frp cactus/examples/register-indy-data/req_discounted_cartrade.py indycli/python/src
cd indycli/python/src
vi req_discounted_cartrade.py
```

Copy javascript files and edit variable `const url = "http://localhost:8000"` in file `testsock.js` to point to `http://host.docker.internal:10080`. This port is served by nginx container (and forwarded to the Indy validator).

```bash
cd ../../..
cp -frp validator/cactus_validator_socketio/testcli/ indycli
cd indycli/testcli
vi testsock.js
```

1. Start a container to run the initialization script

```bash
cd ../../..
docker run -it -v $(pwd)/indycli:/tmp/indycli --user indy --name indycli --network indy_net --add-host host.docker.internal:host-gateway clientbase /bin/bash
```

1. Run the initialization script

In the container, run the python script

```bash
. ~/.venv/bin/activate
cd /tmp/indycli/python
TEST_POOL_IP=172.16.0.2 python -m src.req_discounted_cartrade
```

This script might fail at end because it tries to `POST` a JSON to an application (BLP), which is not ready if you have not started it yet. You can assume the test was successful if the script created `myproof.json` in current directory.

1. Copy myproof.json and run validator test client

In the container, copy myproof.json to testcli directory and run testsock.js.

```bash
cp myproof.json ../testcli
cd ../testcli
node testsock.js
```

36 changes: 36 additions & 0 deletions tools/docker/indy-testnet/clientbase/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
FROM ubuntu:xenial

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get upgrade -y && apt-get install -y software-properties-common apt-transport-https supervisor

# indy libraries and tool
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CE7709D068DB5E88
RUN add-apt-repository "deb https://repo.sovrin.org/sdk/deb xenial stable"
RUN apt-get update
RUN apt-get install -y libindy libnullpay libvcx indy-cli

# Python 3.9 (indy-sdk requires 3.6 and later)
RUN add-apt-repository -y ppa:deadsnakes/ppa
RUN apt-get update && apt-get install -y python3.9 python3.9-dev python3.9-venv python3.9-distutils python3.9-venv python3-venv

# Node.js v12 (Cactus requres v12)
RUN apt-get install -y nodejs npm curl
RUN npm install n -g
RUN n 12
RUN apt purge -y nodejs npm
RUN npm install indy-sdk

# user should run their scripts as indy (home: /home/indy)
RUN groupadd indy
RUN useradd -m indy -g indy

COPY --chown=indy:indy from-indy-sdk /home/indy/from-indy-sdk

WORKDIR /home/indy
USER indy
RUN python3.9 -m venv .venv
RUN . .venv/bin/activate && pip install python3-indy requests


# immediately exits
55 changes: 55 additions & 0 deletions tools/docker/indy-testnet/clientbase/from-indy-sdk/util.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"use strict";
const mkdirp = require('mkdirp');
const fs = require('fs');
const os = require('os');

async function getPoolGenesisTxnPath(poolName) {
let path = `${os.tmpdir()}/indy/${poolName}.txn`;
await savePoolGenesisTxnFile(path);
return path
};

async function poolGenesisTxnData() {
let poolIp = process.env.TEST_POOL_IP || "127.0.0.1";
return `{"reqSignature": {}, "txn": {"data": {"data": {"alias": "Node1", "blskey": "4N8aUNHSgjQVgkpm8nhNEfDf6txHznoYREg9kirmJrkivgL4oSEimFF6nsQ6M41QvhM2Z33nves5vfSn9n1UwNFJBYtWVnHYMATn76vLuL3zU88KyeAYcHfsih3He6UHcXDxcaecHVz6jhCYz1P2UZn2bDVruL5wXpehgBfBaLKm3Ba", "blskey_pop": "RahHYiCvoNCtPTrVtP7nMC5eTYrsUA8WjXbdhNc8debh1agE9bGiJxWBXYNFbnJXoXhWFMvyqhqhRoq737YQemH5ik9oL7R4NTTCz2LEZhkgLJzB3QRQqJyBNyv7acbdHrAT8nQ9UkLbaVL9NBpnWXBTw4LEMePaSHEw66RzPNdAX1", "client_ip": "${poolIp}", "client_port": 9702, "node_ip": "${poolIp}", "node_port": 9701, "services": ["VALIDATOR"]}, "dest": "Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"}, "metadata": {"from": "Th7MpTaRZVRYnPiabds81Y"}, "type": "0"}, "txnMetadata": {"seqNo": 1, "txnId": "fea82e10e894419fe2bea7d96296a6d46f50f93f9eeda954ec461b2ed2950b62"}, "ver": "1"}
{"reqSignature": {}, "txn": {"data": {"data": {"alias": "Node2", "blskey": "37rAPpXVoxzKhz7d9gkUe52XuXryuLXoM6P6LbWDB7LSbG62Lsb33sfG7zqS8TK1MXwuCHj1FKNzVpsnafmqLG1vXN88rt38mNFs9TENzm4QHdBzsvCuoBnPH7rpYYDo9DZNJePaDvRvqJKByCabubJz3XXKbEeshzpz4Ma5QYpJqjk", "blskey_pop": "Qr658mWZ2YC8JXGXwMDQTzuZCWF7NK9EwxphGmcBvCh6ybUuLxbG65nsX4JvD4SPNtkJ2w9ug1yLTj6fgmuDg41TgECXjLCij3RMsV8CwewBVgVN67wsA45DFWvqvLtu4rjNnE9JbdFTc1Z4WCPA3Xan44K1HoHAq9EVeaRYs8zoF5", "client_ip": "${poolIp}", "client_port": 9704, "node_ip": "${poolIp}", "node_port": 9703, "services": ["VALIDATOR"]}, "dest": "8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"}, "metadata": {"from": "EbP4aYNeTHL6q385GuVpRV"}, "type": "0"}, "txnMetadata": {"seqNo": 2, "txnId": "1ac8aece2a18ced660fef8694b61aac3af08ba875ce3026a160acbc3a3af35fc"}, "ver": "1"}
{"reqSignature": {}, "txn": {"data": {"data": {"alias": "Node3", "blskey": "3WFpdbg7C5cnLYZwFZevJqhubkFALBfCBBok15GdrKMUhUjGsk3jV6QKj6MZgEubF7oqCafxNdkm7eswgA4sdKTRc82tLGzZBd6vNqU8dupzup6uYUf32KTHTPQbuUM8Yk4QFXjEf2Usu2TJcNkdgpyeUSX42u5LqdDDpNSWUK5deC5", "blskey_pop": "QwDeb2CkNSx6r8QC8vGQK3GRv7Yndn84TGNijX8YXHPiagXajyfTjoR87rXUu4G4QLk2cF8NNyqWiYMus1623dELWwx57rLCFqGh7N4ZRbGDRP4fnVcaKg1BcUxQ866Ven4gw8y4N56S5HzxXNBZtLYmhGHvDtk6PFkFwCvxYrNYjh", "client_ip": "${poolIp}", "client_port": 9706, "node_ip": "${poolIp}", "node_port": 9705, "services": ["VALIDATOR"]}, "dest": "DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya"}, "metadata": {"from": "4cU41vWW82ArfxJxHkzXPG"}, "type": "0"}, "txnMetadata": {"seqNo": 3, "txnId": "7e9f355dffa78ed24668f0e0e369fd8c224076571c51e2ea8be5f26479edebe4"}, "ver": "1"}
{"reqSignature": {}, "txn": {"data": {"data": {"alias": "Node4", "blskey": "2zN3bHM1m4rLz54MJHYSwvqzPchYp8jkHswveCLAEJVcX6Mm1wHQD1SkPYMzUDTZvWvhuE6VNAkK3KxVeEmsanSmvjVkReDeBEMxeDaayjcZjFGPydyey1qxBHmTvAnBKoPydvuTAqx5f7YNNRAdeLmUi99gERUU7TD8KfAa6MpQ9bw", "blskey_pop": "RPLagxaR5xdimFzwmzYnz4ZhWtYQEj8iR5ZU53T2gitPCyCHQneUn2Huc4oeLd2B2HzkGnjAff4hWTJT6C7qHYB1Mv2wU5iHHGFWkhnTX9WsEAbunJCV2qcaXScKj4tTfvdDKfLiVuU2av6hbsMztirRze7LvYBkRHV3tGwyCptsrP", "client_ip": "${poolIp}", "client_port": 9708, "node_ip": "${poolIp}", "node_port": 9707, "services": ["VALIDATOR"]}, "dest": "4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA"}, "metadata": {"from": "TWwCRQRZ2ZHMJFn9TzLp7W"}, "type": "0"}, "txnMetadata": {"seqNo": 4, "txnId": "aa5e817d7cc626170eca175822029339a444eb0ee8f0bd20d3b0b76e566fb008"}, "ver": "1"}`;
}

async function savePoolGenesisTxnFile(filePath) {
let data = await poolGenesisTxnData();
await mkdir(filePath);
return fs.writeFileSync(filePath, data, 'utf8');
}

async function mkdir(filePath) {
return new Promise((resolve, reject) => {
let folderPath = filePath.split('/').slice(0, filePath.split('/').length - 1).join('/');
mkdirp(folderPath, function(err, res) {
if(err) reject(err);
else resolve(res);
})
})
}

function pathToIndyClientHome() {
return require('os').homedir() + "/.indy_client"
}

function sleep(duration){
return new Promise(resolve => {
setTimeout(resolve,duration)
})
}

function getCurrentTimeInSeconds() {
return Math.floor(Date.now() / 1000)
}

module.exports = {
getPoolGenesisTxnPath,
getPathToIndyClientHome: pathToIndyClientHome,
sleep,
getCurrentTimeInSeconds
}
63 changes: 63 additions & 0 deletions tools/docker/indy-testnet/clientbase/from-indy-sdk/utils.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import asyncio
import json
from os import environ
from pathlib import Path
from tempfile import gettempdir

import time
from indy import ledger


PROTOCOL_VERSION = 2


def path_home() -> Path:
return Path.home().joinpath(".indy_client")


def get_pool_genesis_txn_path(pool_name):
path_temp = Path(gettempdir()).joinpath("indy")
path = path_temp.joinpath("{}.txn".format(pool_name))
save_pool_genesis_txn_file(path)
return path


def pool_genesis_txn_data():
pool_ip = environ.get("TEST_POOL_IP", "127.0.0.1")

return "\n".join([
'{{"reqSignature":{{}},"txn":{{"data":{{"data":{{"alias":"Node1","blskey":"4N8aUNHSgjQVgkpm8nhNEfDf6txHznoYREg9kirmJrkivgL4oSEimFF6nsQ6M41QvhM2Z33nves5vfSn9n1UwNFJBYtWVnHYMATn76vLuL3zU88KyeAYcHfsih3He6UHcXDxcaecHVz6jhCYz1P2UZn2bDVruL5wXpehgBfBaLKm3Ba","blskey_pop":"RahHYiCvoNCtPTrVtP7nMC5eTYrsUA8WjXbdhNc8debh1agE9bGiJxWBXYNFbnJXoXhWFMvyqhqhRoq737YQemH5ik9oL7R4NTTCz2LEZhkgLJzB3QRQqJyBNyv7acbdHrAT8nQ9UkLbaVL9NBpnWXBTw4LEMePaSHEw66RzPNdAX1","client_ip":"{}","client_port":9702,"node_ip":"{}","node_port":9701,"services":["VALIDATOR"]}},"dest":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"}},"metadata":{{"from":"Th7MpTaRZVRYnPiabds81Y"}},"type":"0"}},"txnMetadata":{{"seqNo":1,"txnId":"fea82e10e894419fe2bea7d96296a6d46f50f93f9eeda954ec461b2ed2950b62"}},"ver":"1"}}'.format(
pool_ip, pool_ip),
'{{"reqSignature":{{}},"txn":{{"data":{{"data":{{"alias":"Node2","blskey":"37rAPpXVoxzKhz7d9gkUe52XuXryuLXoM6P6LbWDB7LSbG62Lsb33sfG7zqS8TK1MXwuCHj1FKNzVpsnafmqLG1vXN88rt38mNFs9TENzm4QHdBzsvCuoBnPH7rpYYDo9DZNJePaDvRvqJKByCabubJz3XXKbEeshzpz4Ma5QYpJqjk","blskey_pop":"Qr658mWZ2YC8JXGXwMDQTzuZCWF7NK9EwxphGmcBvCh6ybUuLxbG65nsX4JvD4SPNtkJ2w9ug1yLTj6fgmuDg41TgECXjLCij3RMsV8CwewBVgVN67wsA45DFWvqvLtu4rjNnE9JbdFTc1Z4WCPA3Xan44K1HoHAq9EVeaRYs8zoF5","client_ip":"{}","client_port":9704,"node_ip":"{}","node_port":9703,"services":["VALIDATOR"]}},"dest":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"}},"metadata":{{"from":"EbP4aYNeTHL6q385GuVpRV"}},"type":"0"}},"txnMetadata":{{"seqNo":2,"txnId":"1ac8aece2a18ced660fef8694b61aac3af08ba875ce3026a160acbc3a3af35fc"}},"ver":"1"}}'.format(
pool_ip, pool_ip),
'{{"reqSignature":{{}},"txn":{{"data":{{"data":{{"alias":"Node3","blskey":"3WFpdbg7C5cnLYZwFZevJqhubkFALBfCBBok15GdrKMUhUjGsk3jV6QKj6MZgEubF7oqCafxNdkm7eswgA4sdKTRc82tLGzZBd6vNqU8dupzup6uYUf32KTHTPQbuUM8Yk4QFXjEf2Usu2TJcNkdgpyeUSX42u5LqdDDpNSWUK5deC5","blskey_pop":"QwDeb2CkNSx6r8QC8vGQK3GRv7Yndn84TGNijX8YXHPiagXajyfTjoR87rXUu4G4QLk2cF8NNyqWiYMus1623dELWwx57rLCFqGh7N4ZRbGDRP4fnVcaKg1BcUxQ866Ven4gw8y4N56S5HzxXNBZtLYmhGHvDtk6PFkFwCvxYrNYjh","client_ip":"{}","client_port":9706,"node_ip":"{}","node_port":9705,"services":["VALIDATOR"]}},"dest":"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya"}},"metadata":{{"from":"4cU41vWW82ArfxJxHkzXPG"}},"type":"0"}},"txnMetadata":{{"seqNo":3,"txnId":"7e9f355dffa78ed24668f0e0e369fd8c224076571c51e2ea8be5f26479edebe4"}},"ver":"1"}}'.format(
pool_ip, pool_ip),
'{{"reqSignature":{{}},"txn":{{"data":{{"data":{{"alias":"Node4","blskey":"2zN3bHM1m4rLz54MJHYSwvqzPchYp8jkHswveCLAEJVcX6Mm1wHQD1SkPYMzUDTZvWvhuE6VNAkK3KxVeEmsanSmvjVkReDeBEMxeDaayjcZjFGPydyey1qxBHmTvAnBKoPydvuTAqx5f7YNNRAdeLmUi99gERUU7TD8KfAa6MpQ9bw","blskey_pop":"RPLagxaR5xdimFzwmzYnz4ZhWtYQEj8iR5ZU53T2gitPCyCHQneUn2Huc4oeLd2B2HzkGnjAff4hWTJT6C7qHYB1Mv2wU5iHHGFWkhnTX9WsEAbunJCV2qcaXScKj4tTfvdDKfLiVuU2av6hbsMztirRze7LvYBkRHV3tGwyCptsrP","client_ip":"{}","client_port":9708,"node_ip":"{}","node_port":9707,"services":["VALIDATOR"]}},"dest":"4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA"}},"metadata":{{"from":"TWwCRQRZ2ZHMJFn9TzLp7W"}},"type":"0"}},"txnMetadata":{{"seqNo":4,"txnId":"aa5e817d7cc626170eca175822029339a444eb0ee8f0bd20d3b0b76e566fb008"}},"ver":"1"}}'.format(
pool_ip, pool_ip)
])


def save_pool_genesis_txn_file(path):
data = pool_genesis_txn_data()

path.parent.mkdir(parents=True, exist_ok=True)

with open(str(path), "w+") as f:
f.writelines(data)


def run_coroutine(coroutine, loop=None):
if loop is None:
loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine())


async def ensure_previous_request_applied(pool_handle, checker_request, checker):
for _ in range(3):
response = json.loads(await ledger.submit_request(pool_handle, checker_request))
try:
if checker(response):
return json.dumps(response)
except TypeError:
pass
time.sleep(5)
74 changes: 18 additions & 56 deletions tools/docker/indy-testnet/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,28 +5,8 @@ services:
container_name: indy_pool
image: indy_pool
build:
# context: ./indy_pool
# dockerfile: indy_pool.dockerfile
context: ${INDY_SDK_ROOT}
context: ./indy_pool
dockerfile: indy-pool.dockerfile
args:
- HTTP_PROXY=$HTTP_PROXY
- http_proxy=$HTTP_PROXY
- HTTPS_PROXY=$HTTP_PROXY
- https_proxy=$HTTP_PROXY
- FTP_PROXY=$HTTP_PROXY
- ftp_proxy=$HTTP_PROXY
- NO_PROXY=$NO_PROXY
- no_proxy=$NO_PROXY
environment:
- HTTP_PROXY=$HTTP_PROXY
- http_proxy=$HTTP_PROXY
- HTTPS_PROXY=$HTTP_PROXY
- https_proxy=$HTTP_PROXY
- FTP_PROXY=$HTTP_PROXY
- ftp_proxy=$HTTP_PROXY
- NO_PROXY=$NO_PROXY
- no_proxy=$NO_PROXY
ports:
- "9701:9701"
- "9702:9702"
Expand All @@ -40,7 +20,7 @@ services:
indy_net:
ipv4_address: 172.16.0.2
volumes:
- ./sandbox:/var/lib/indy/sandbox/
- ./indy_pool/sandbox:/var/lib/indy/sandbox/

nginx:
container_name: nginx
Expand All @@ -58,46 +38,29 @@ services:
indy_net:
ipv4_address: 172.16.0.3

validator:
container_name: validator
clientbase:
container_name: immediately_exits
image: clientbase
build:
context: ../../../packages-python/cactus_validator_socketio/
dockerfile: build/valipy.dockerfile
args:
- pool_ip=172.16.0.2
- HTTP_PROXY=$HTTP_PROXY
- http_proxy=$HTTP_PROXY
- HTTPS_PROXY=$HTTP_PROXY
- https_proxy=$HTTP_PROXY
- FTP_PROXY=$HTTP_PROXY
- ftp_proxy=$HTTP_proxy
- NO_PROXY=$NO_PROXY
- no_proxy=$no_proxy
context: ./clientbase/
dockerfile: Dockerfile

validator:
container_name: validator
image: validator
environment:
- HTTP_PROXY=$HTTP_PROXY
- http_proxy=$HTTP_PROXY
- HTTPS_PROXY=$HTTP_PROXY
- https_proxy=$HTTP_PROXY
- FTP_PROXY=$HTTP_PROXY
- ftp_proxy=$HTTP_proxy
- NO_PROXY=$NO_PROXY
- no_proxy=$no_proxy
image: valipy
- TEST_POOL_IP=172.16.0.2
depends_on:
- clientbase
build:
context: ./validator
dockerfile: Dockerfile
ports:
- "8000:8000"
networks:
indy_net:
ipv4_address: 172.16.0.4
volumes:
- type: bind
source: "../../../packages-python/cactus_validator_socketio/validator-python/"
target: "/root/validator"
- type: bind
source: "../../../packages-python/cactus_validator_socketio/etc/cactus/"
target: "/etc/cactus"
- type: bind
source: "../../../../indy-sdk/"
target: "/root/indy-sdk"

networks:
indy_net:
name: indy_net
Expand All @@ -106,4 +69,3 @@ networks:
driver: default
config:
- subnet: 172.16.0.0/24

2 changes: 1 addition & 1 deletion tools/docker/indy-testnet/fig1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 35b20ac

Please sign in to comment.