Skip to content

Commit

Permalink
feat(cosmic-swingset): use a fake chain for scenario3 (#322)
Browse files Browse the repository at this point in the history
* feat(cosmic-swingset): use a fake chain for scenario3

This introduces block latency so that the scenario3 chain
behaves much more like a real blockchain, but within the
same process (for debuggability) and without the
actual Cosmos SDK usage.

* fix(Makefile): refine rules

Make a `deprecated-scenario3-setup` and `deprecated-scenario3-run-client`
to illustrate the old scenario3.

Create `scenario3-run` to make tab-completion better.

* doc(README): remove Golang prerequisite: no longer needed
  • Loading branch information
michaelfig committed Jan 17, 2020
1 parent 8dc31a0 commit f833610
Show file tree
Hide file tree
Showing 7 changed files with 213 additions and 25 deletions.
28 changes: 9 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,19 @@ repository: instead you should [follow our instructions for getting started](htt

But if you are improving the platform itself, this is the repository to use.

## Pre-requisites
## Prerequisites

* Git
* Node.js (version 11 or higher)
* Golang (1.13 or higher) (TODO: only require this for cosmic-swingset)
* Yarn (`npm install -g yarn`)

You don't need Golang if you just want to test contracts and run the
"scenario3" simulator. Golang (1.13 or higher) is needed only if you
want to build/debug Cosmos SDK support. (The `1.12` release will work, but
it will modify `packages/cosmic-swingset/go.mod` upon each build (by adding
a dependency upon `appengine`). The `1.13` release will leave the `go.mod`
file correctly unmodified.

## Build

From a new checkout of this repository, run:
Expand All @@ -36,8 +42,7 @@ section tells us when symlinks could not be used (generally because e.g.
`ERTP` wants `marshal@0.1.0`, but `packages/marshal/package.json` says it's
actually `0.2.0`). We want to get rid of all mismatched dependencies.

The `yarn build` step generates kernel bundles, and compiles the Go code in
cosmic-swingset.
The `yarn build` step generates kernel bundles.

## Test

Expand Down Expand Up @@ -110,18 +115,3 @@ To create a new (empty) package (e.g. spinning Zoe out from ERTP):
* commit everything to a new branch, push, check the GitHub `Actions` tab to
make sure CI tested everything properly
* merge with a PR

## Running without Go

A golang installation is necessary for building `cosmic-swingset`. At
present, this build happens during `yarn install`, which is also necessary to
set up the monorepo's cross-package symlinks.

Until we change this, to build everything else without a Go install, just
edit the top-level `package.json` and remove `packages/cosmic-swingset` from
the `workspaces` clause.

We recommend Go `1.13`. The `1.12` release will work, but it will modify
`packages/cosmic-swingset/go.mod` upon each build (by adding a dependency
upon `appengine`). The `1.13` release will leave the `go.mod` file correctly
unmodified.
34 changes: 29 additions & 5 deletions packages/cosmic-swingset/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ REPOSITORY = agoric/cosmic-swingset
CHAIN_ID = agoric
INITIAL_TOKENS = 1000agmedallion

# By default, make the fake chain in scenario3 produce
# "blocks" at 5-second intervals.
FAKE_CHAIN_DELAY = 5

NUM_SOLOS?=1
BASE_PORT?=8000

Expand Down Expand Up @@ -62,7 +66,7 @@ scenario2-setup: build-cosmos
$(AGC) validate-genesis
../deployment/set-json.js ~/.ag-chain-cosmos/config/genesis.json --agoric-genesis-overrides
$(MAKE) set-local-gci-ingress
@echo "ROLE=two_chain BOOT_ADDRESS=\`cat t1/$(BASE_PORT)/ag-cosmos-helper-address\` agc start"
@echo "ROLE=two_chain BOOT_ADDRESS=\`cat t1/$(BASE_PORT)/ag-cosmos-helper-address\` $(AGC) start"
@echo "(cd t1/$(BASE_PORT) && ../bin/ag-solo start --role=two_client)"

scenario2-run-chain:
Expand All @@ -74,14 +78,34 @@ scenario2-run-chain:
scenario2-run-client:
cd t1/$(BASE_PORT) && ../../bin/ag-solo start --role=two_client

# scenario3 is a single JS process without any Golang. However,
# the client and the chain within the process run two separate
# kernels. There is an artificial delay when handling messages
# destined for the chain kernel, to prevent you from accidentally
# creating programs that won't work on the real blockchain.
#
# If you still want the client/chain separation without delay,
# then run: make scenario3-setup FAKE_CHAIN_DELAY=0
scenario3-setup:
rm -rf t3
bin/ag-solo init t3 --egresses=fake
(cd t3 && \
../bin/ag-solo set-fake-chain --role=two_chain --delay=$(FAKE_CHAIN_DELAY) myFakeGCI)
@echo 'Execute `make scenario3-run` to run the client and simulated chain'

# This runs both the client and the fake chain.
scenario3-run-client: scenario3-run
scenario3-run:
cd t3 && ../bin/ag-solo start --role=two_client

# These rules are the old scenario3. No fake delay at all.
# It's generally better to use the new scenario3.
deprecated-scenario3-setup:
rm -rf t3
bin/ag-solo init t3 --egresses=none
@echo 'Ignore advice above, instead run `make scenario3-run-client`'
scenario3-run-client:

deprecated-scenario3-run-client:
cd t3 && ../bin/ag-solo start --role=three_client
scenario3-run-chain:
@echo 'No local chain needs to run in scenario3'

docker-pull:
for f in '' -pserver -setup -setup-solo -solo; do \
Expand Down
6 changes: 6 additions & 0 deletions packages/cosmic-swingset/changelogs/321.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Introduce "fake chain" to scenario3 configuration.

Notably, have a simulated 5-second block time. To
reset this to the old behaviour, use:

make scenario3-setup FAKE_CHAIN_DELAY=0
98 changes: 98 additions & 0 deletions packages/cosmic-swingset/lib/ag-solo/fake-chain.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
/* eslint-disable no-await-in-loop */
import path from 'path';
import fs from 'fs';
import stringify from '@agoric/swingset-vat/src/kernel/json-stable-stringify';
import { launch } from '../launch-chain';

const PRETEND_BLOCK_DELAY = 5;

async function readMap(file) {
let content;
const map = new Map();
try {
content = await fs.promises.readFile(file);
} catch (e) {
return map;
}
const obj = JSON.parse(content);
Object.entries(obj).forEach(([k, v]) => map.set(k, v));
return map;
}

async function writeMap(file, map) {
const obj = {};
[...map.entries()].forEach(([k, v]) => (obj[k] = v));
const json = stringify(obj);
await fs.promises.writeFile(file, json);
}

export async function connectToFakeChain(basedir, GCI, role, delay, inbound) {
const stateFile = path.join(basedir, `fake-chain-${GCI}-state.json`);
const mailboxFile = path.join(basedir, `fake-chain-${GCI}-mailbox.json`);
const bootAddress = `${GCI}-client`;

const mailboxStorage = await readMap(mailboxFile);

const vatsdir = path.join(basedir, 'vats');
const argv = [`--role=${role}`, bootAddress];
const s = await launch(mailboxStorage, stateFile, vatsdir, argv);
const { deliverInbound, deliverStartBlock } = s;

let pretendLast = Date.now();
let blockHeight = 0;
let intoChain = [];
let thisBlock = [];
async function simulateBlock() {
const actualStart = Date.now();
// Gather up the new messages into the latest block.
thisBlock.push(...intoChain);
intoChain = [];

try {
const commitStamp = pretendLast + PRETEND_BLOCK_DELAY * 1000;
const blockTime = Math.floor(commitStamp / 1000);
await deliverStartBlock(blockHeight, blockTime);
for (let i = 0; i < thisBlock.length; i += 1) {
const [newMessages, acknum] = thisBlock[i];
await deliverInbound(
bootAddress,
newMessages,
acknum,
blockHeight,
blockTime,
);
}

// Done processing, "commit the block".
await writeMap(mailboxFile, mailboxStorage);
thisBlock = [];
pretendLast = commitStamp + Date.now() - actualStart;
blockHeight += 1;
} catch (e) {
console.log(`error fake processing`, e);
}

if (delay) {
setTimeout(simulateBlock, delay * 1000);
}

// TODO: maybe add latency to the inbound messages.
const mailbox = JSON.parse(mailboxStorage.get(`mailbox.${bootAddress}`));
const { outbox, ack } = mailbox || {
outbox: [],
ack: 0,
};
inbound(GCI, outbox, ack);
}

async function deliver(newMessages, acknum) {
intoChain.push([newMessages, acknum]);
if (!delay) {
await simulateBlock();
}
}
if (delay) {
setTimeout(simulateBlock, delay * 1000);
}
return deliver;
}
8 changes: 7 additions & 1 deletion packages/cosmic-swingset/lib/ag-solo/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { insist } from './insist';
import bundle from './bundle';
import initBasedir from './init-basedir';
import setGCIIngress from './set-gci-ingress';
import setFakeChain from './set-fake-chain';
import start from './start';

// As we add more egress types, put the default types in a comma-separated
Expand Down Expand Up @@ -63,14 +64,19 @@ start
const subdir = subArgs[1];
insist(basedir !== undefined, 'you must provide a BASEDIR');
initBasedir(basedir, webport, webhost, subdir, egresses.split(','));
console.error(`Run '(cd ${basedir} && ${progname} start)' to start the vat machine`);
// console.error(`Run '(cd ${basedir} && ${progname} start)' to start the vat machine`);
} else if (argv[0] === 'set-gci-ingress') {
const basedir = insistIsBasedir();
const { _: subArgs, ...subOpts } = parseArgs(argv.slice(1), {});
const GCI = subArgs[0];
const chainID = subOpts.chainID || 'agoric';
const rpcAddresses = subArgs.slice(1);
setGCIIngress(basedir, GCI, rpcAddresses, chainID);
} else if (argv[0] === 'set-fake-chain') {
const basedir = insistIsBasedir();
const { _: subArgs, role, delay } = parseArgs(argv.slice(1), {});
const GCI = subArgs[0];
setFakeChain(basedir, GCI, role, delay);
} else if (argv[0] === 'start') {
const basedir = insistIsBasedir();
const withSES = true;
Expand Down
51 changes: 51 additions & 0 deletions packages/cosmic-swingset/lib/ag-solo/set-fake-chain.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import fs from 'fs';
import path from 'path';

export default function setFakeChain(basedir, GCI, role, fakeDelay) {
const fn = path.join(basedir, 'connections.json');
const connsByType = {};
const add = c => {
const { type } = c;
const conns = connsByType[type];
if (!conns) {
connsByType[type] = [c];
return;
}

switch (type) {
case 'fake-chain': {
// Replace duplicate GCIs.
const { GCI: newGCI } = c;
const index = conns.findIndex(({ GCI: oldGCI }) => oldGCI === newGCI);
if (index < 0) {
conns.push(c);
} else {
conns[index] = c;
}
break;
}
default:
conns.push(c);
}
};

JSON.parse(fs.readFileSync(fn)).forEach(add);
const newconn = {
type: 'fake-chain',
GCI,
fakeDelay,
role,
};
add(newconn);
const connections = [];
Object.entries(connsByType).forEach(([_type, conns]) =>
connections.push(...conns),
);
fs.writeFileSync(fn, `${JSON.stringify(connections, undefined, 2)}\n`);

const gciFileContents = `\
export const GCI = ${JSON.stringify(GCI)};
`;
const bfn = path.join(basedir, 'vats', 'gci.js');
fs.writeFileSync(bfn, gciFileContents);
}
13 changes: 13 additions & 0 deletions packages/cosmic-swingset/lib/ag-solo/start.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { deliver, addDeliveryTarget } from './outbound';
import { makeHTTPListener } from './web';

import { connectToChain } from './chain-cosmos-sdk';
import { connectToFakeChain } from './fake-chain';
import bundle from './bundle';

// import { makeChainFollower } from './follower';
Expand Down Expand Up @@ -230,6 +231,18 @@ export default async function start(basedir, withSES, argv) {
addDeliveryTarget(c.GCI, deliverator);
}
break;
case 'fake-chain': {
console.log(`adding follower/sender for fake chain ${c.role} ${c.GCI}`);
const deliverator = await connectToFakeChain(
basedir,
c.GCI,
c.role,
c.fakeDelay,
inbound,
);
addDeliveryTarget(c.GCI, deliverator);
break;
}
case 'http':
console.log(`adding HTTP/WS listener on ${c.host}:${c.port}`);
if (broadcastJSON) {
Expand Down

0 comments on commit f833610

Please sign in to comment.