Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Buffer Usage #801

Merged
merged 40 commits into from
Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
b60a57f
Remove buffer usage in favor or Uint8Array and Dataview
algochoi Jul 6, 2023
1cce5ff
Add byteoffset when creating DataView
algochoi Jul 6, 2023
31104e7
Increase Indexer sleep time
algochoi Jul 7, 2023
f3d871a
Add polyfill for base64 decoding and update examples
algochoi Jul 10, 2023
0289253
Update examples to use utility function for Buffer functionality
algochoi Jul 10, 2023
c161ec2
Revert smoke test
algochoi Jul 10, 2023
5c188c5
Add a roundtrip test for sanity check
algochoi Jul 10, 2023
3a6ab1d
Fix base64 encoding test
algochoi Jul 11, 2023
f6634fc
Remove buffer in transaction
algochoi Jul 11, 2023
a722245
Remove unnecessary casting
algochoi Jul 11, 2023
736ca01
Merge branch 'remove-buffer-abi' into replace-buffer
algochoi Jul 11, 2023
c8495f6
Unify return type to Uint8Array
algochoi Jul 11, 2023
0c9b49a
Remove buffer usage in source code except client
algochoi Jul 12, 2023
20861bd
Standardize tests and add more utf-8 strings
algochoi Jul 12, 2023
5d86934
Add hex encoding and string encoding tests
algochoi Jul 12, 2023
fd9bdf4
Merge branch 'remove-buffer-abi' into replace-buffer
algochoi Jul 12, 2023
4fe705a
Fix some code broken by existing tests
algochoi Jul 13, 2023
209cb4e
Merge branch '3.0.0' into remove-buffer-abi
algochoi Jul 13, 2023
1437020
Update tests for edge bytes
algochoi Jul 13, 2023
42f9a12
Export byte conversion utilities
algochoi Jul 13, 2023
0c5e393
Fix imports to prefix algosdk
algochoi Jul 17, 2023
982e5be
Merge branch 'remove-buffer-abi' into replace-buffer
algochoi Jul 17, 2023
56b518d
Merge branch '3.0.0' into replace-buffer
algochoi Jul 18, 2023
f8230e5
Resolving merge conflicts
algochoi Jul 18, 2023
f12c1f6
Remove some generated types from stable release
algochoi Jul 18, 2023
f765cd9
Fix node tests
algochoi Jul 19, 2023
3ddfaec
Try loadResource in browser
algochoi Jul 19, 2023
84f73c3
Update buffer usage in client files
algochoi Jul 19, 2023
3fc5200
Revert cucumber browser loadResource
algochoi Jul 19, 2023
a84101e
Fix browser tests
algochoi Jul 19, 2023
792cfe0
Fix node tests
algochoi Jul 19, 2023
3d8bff6
Add alternative path for decoding composer responses
algochoi Jul 20, 2023
729e65b
Merge branch '3.0.0' into replace-buffer
algochoi Jul 20, 2023
e7aa3d3
Remove browser buffer dependency, fix cucumber tests
algochoi Jul 27, 2023
50e03cf
Revert cucumber node index.js file to use buffer again
algochoi Jul 27, 2023
d7a4dfe
Fix some conversion methods in err messages
algochoi Jul 27, 2023
950d3ff
Update src/encoding/binarydata.ts
algochoi Jul 27, 2023
7463f4e
Run prettier on commit suggestion
algochoi Jul 27, 2023
ee3e2e1
Add toString conversion for buffers
algochoi Jul 27, 2023
624bf92
Update tests/cucumber/browser/test.js
algochoi Jul 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/asa.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ async function main() {
console.log(`Asset Params: ${JSON.stringify(assetInfo.params)}`);
// example: ASSET_INFO

await new Promise((f) => setTimeout(f, 2000)); // sleep to ensure indexer is caught up
await new Promise((f) => setTimeout(f, 5000)); // sleep to ensure indexer is caught up
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks frequently flaky on CI, but I haven't seen it fail locally 👀


// example: INDEXER_LOOKUP_ASSET
const indexer = getLocalIndexerClient();
Expand Down
2 changes: 1 addition & 1 deletion examples/codec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
/* eslint-disable no-promise-executor-return */
/* eslint-disable no-console */
import algosdk from '../src';
import { getLocalAlgodClient, getLocalAccounts } from './utils';
import { getLocalAccounts, getLocalAlgodClient } from './utils';

async function main() {
const client = getLocalAlgodClient();
Expand Down
2 changes: 1 addition & 1 deletion examples/lsig.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
/* eslint-disable no-promise-executor-return */
/* eslint-disable no-console */
import algosdk from '../src';
import { getLocalAlgodClient, getLocalAccounts } from './utils';
import { getLocalAccounts, getLocalAlgodClient } from './utils';

async function main() {
const client = getLocalAlgodClient();
Expand Down
11 changes: 5 additions & 6 deletions src/bid.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { Buffer } from 'buffer';
import * as address from './encoding/address';
import * as encoding from './encoding/encoding';
import * as nacl from './nacl/naclWrappers';
Expand Down Expand Up @@ -27,7 +26,7 @@ export type BidOptions = Omit<
* */
export default class Bid implements BidStorageStructure {
name = 'Bid';
tag = Buffer.from([97, 66]); // "aB"
tag = Uint8Array.from([97, 66]); // "aB"

bidderKey: Address;
bidAmount: number;
Expand Down Expand Up @@ -67,23 +66,23 @@ export default class Bid implements BidStorageStructure {
// eslint-disable-next-line camelcase
get_obj_for_encoding() {
return {
bidder: Buffer.from(this.bidderKey.publicKey),
bidder: this.bidderKey.publicKey,
cur: this.bidAmount,
price: this.maxPrice,
id: this.bidID,
auc: Buffer.from(this.auctionKey.publicKey),
auc: this.auctionKey.publicKey,
aid: this.auctionID,
};
}

signBid(sk: Uint8Array) {
const encodedMsg = encoding.encode(this.get_obj_for_encoding());
const toBeSigned = Buffer.from(utils.concatArrays(this.tag, encodedMsg));
const toBeSigned = utils.concatArrays(this.tag, encodedMsg);
const sig = nacl.sign(toBeSigned, sk);

// construct signed message
const sBid = {
sig: Buffer.from(sig),
sig,
bid: this.get_obj_for_encoding(),
};

Expand Down
7 changes: 3 additions & 4 deletions src/client/client.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { Buffer } from 'buffer';
import * as utils from '../utils/utils';
import {
BaseHTTPClient,
Expand Down Expand Up @@ -151,10 +150,10 @@ export default class HTTPClient {
return new Uint8Array(0); // empty Uint8Array
}
if (requestHeaders['content-type'] === 'application/json') {
return new Uint8Array(Buffer.from(JSON.stringify(data)));
return new TextEncoder().encode(JSON.stringify(data));
}
if (typeof data === 'string') {
return new Uint8Array(Buffer.from(data));
return new TextEncoder().encode(data);
}
if (data instanceof Uint8Array) {
return data;
Expand All @@ -179,7 +178,7 @@ export default class HTTPClient {
let text;

if (format !== 'application/msgpack') {
text = (body && Buffer.from(body).toString()) || '';
text = (body && new TextDecoder().decode(body)) || '';
}

if (parseBody && format === 'application/json') {
Expand Down
35 changes: 19 additions & 16 deletions src/client/kmd.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import { Buffer } from 'buffer';
import ServiceClient from './v2/serviceClient';
import {
base64ToBytes,
bytesToBase64,
coerceToBytes,
} from '../encoding/binarydata';
import * as txn from '../transaction';
import { CustomTokenHeader, KMDTokenHeader } from './urlTokenBaseHTTPClient';
import ServiceClient from './v2/serviceClient';

export default class Kmd extends ServiceClient {
constructor(
Expand Down Expand Up @@ -50,7 +54,7 @@ export default class Kmd extends ServiceClient {
wallet_name: walletName,
wallet_driver_name: walletDriverName,
wallet_password: walletPassword,
master_derivation_key: Buffer.from(walletMDK).toString('base64'),
master_derivation_key: bytesToBase64(walletMDK),
};
const res = await this.c.post('/v1/wallet', req);
return res.body;
Expand Down Expand Up @@ -157,10 +161,7 @@ export default class Kmd extends ServiceClient {
};
const res = await this.c.post('/v1/master-key/export', req);
return {
master_derivation_key: Buffer.from(
res.body.master_derivation_key,
'base64'
),
master_derivation_key: base64ToBytes(res.body.master_derivation_key),
};
}

Expand All @@ -174,7 +175,7 @@ export default class Kmd extends ServiceClient {
async importKey(walletHandle: string, secretKey: Uint8Array) {
const req = {
wallet_handle_token: walletHandle,
private_key: Buffer.from(secretKey).toString('base64'),
private_key: bytesToBase64(secretKey),
};
const res = await this.c.post('/v1/key/import', req);
return res.body;
Expand All @@ -195,7 +196,7 @@ export default class Kmd extends ServiceClient {
wallet_password: walletPassword,
};
const res = await this.c.post('/v1/key/export', req);
return { private_key: Buffer.from(res.body.private_key, 'base64') };
return { private_key: base64ToBytes(res.body.private_key) };
}

/**
Expand Down Expand Up @@ -266,12 +267,12 @@ export default class Kmd extends ServiceClient {
const req = {
wallet_handle_token: walletHandle,
wallet_password: walletPassword,
transaction: Buffer.from(tx.toByte()).toString('base64'),
transaction: bytesToBase64(tx.toByte()),
};
const res = await this.c.post('/v1/transaction/sign', req);

if (res.status === 200) {
return Buffer.from(res.body.signed_transaction, 'base64');
return base64ToBytes(res.body.signed_transaction);
}
return res.body;
}
Expand All @@ -293,17 +294,18 @@ export default class Kmd extends ServiceClient {
publicKey: Uint8Array | string
) {
const tx = txn.instantiateTxnIfNeeded(transaction);
const pk = coerceToBytes(publicKey);

const req = {
wallet_handle_token: walletHandle,
wallet_password: walletPassword,
transaction: Buffer.from(tx.toByte()).toString('base64'),
public_key: Buffer.from(publicKey).toString('base64'),
transaction: bytesToBase64(tx.toByte()),
public_key: bytesToBase64(pk),
};
const res = await this.c.post('/v1/transaction/sign', req);

if (res.status === 200) {
return Buffer.from(res.body.signed_transaction, 'base64');
return base64ToBytes(res.body.signed_transaction);
}
return res.body;
}
Expand Down Expand Up @@ -389,10 +391,11 @@ export default class Kmd extends ServiceClient {
partial: string
) {
const tx = txn.instantiateTxnIfNeeded(transaction);
const pubkey = coerceToBytes(pk);
const req = {
wallet_handle_token: walletHandle,
transaction: Buffer.from(tx.toByte()).toString('base64'),
public_key: Buffer.from(pk).toString('base64'),
transaction: bytesToBase64(tx.toByte()),
public_key: bytesToBase64(pubkey),
partial_multisig: partial,
wallet_password: pw,
};
Expand Down
3 changes: 1 addition & 2 deletions src/client/urlTokenBaseHTTPClient.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { Buffer } from 'buffer';
import { fetch, Response, Headers } from 'cross-fetch';
import {
BaseHTTPClient,
Expand Down Expand Up @@ -118,7 +117,7 @@ export class URLTokenBaseHTTPClient implements BaseHTTPClient {
try {
body = new Uint8Array(await res.arrayBuffer());
const decoded: Record<string, any> = JSON.parse(
Buffer.from(body).toString()
new TextDecoder().decode(body)
);
if (decoded.message) {
bodyErrorMessage = decoded.message;
Expand Down
6 changes: 3 additions & 3 deletions src/client/v2/algod/compile.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Buffer } from 'buffer';
import JSONRequest from '../jsonrequest';
import { coerceToBytes } from '../../../encoding/binarydata';
import HTTPClient from '../../client';
import JSONRequest from '../jsonrequest';

/**
* Sets the default header (if not previously set)
Expand Down Expand Up @@ -42,7 +42,7 @@ export default class Compile extends JSONRequest {
const txHeaders = setHeaders(headers);
const res = await this.c.post(
this.path(),
Buffer.from(this.source),
coerceToBytes(this.source),
txHeaders,
this.query
);
Expand Down
6 changes: 3 additions & 3 deletions src/client/v2/algod/disassemble.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Buffer } from 'buffer';
import JSONRequest from '../jsonrequest';
import { coerceToBytes } from '../../../encoding/binarydata';
import HTTPClient from '../../client';
import JSONRequest from '../jsonrequest';

/**
* Sets the default header (if not previously set)
Expand Down Expand Up @@ -37,7 +37,7 @@ export default class Disassemble extends JSONRequest {
const txHeaders = setHeaders(headers);
const res = await this.c.post(
this.path(),
Buffer.from(this.source),
coerceToBytes(this.source),
txHeaders,
this.query
);
Expand Down
13 changes: 4 additions & 9 deletions src/client/v2/algod/dryrun.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
import { Buffer } from 'buffer';
import JSONRequest from '../jsonrequest';
import HTTPClient from '../../client';
import * as modelsv2 from './models/types';
import * as encoding from '../../../encoding/encoding';
import HTTPClient from '../../client';
import JSONRequest from '../jsonrequest';
import { setHeaders } from './compile';
import * as modelsv2 from './models/types';

export default class Dryrun extends JSONRequest {
private blob: Uint8Array;
Expand All @@ -24,11 +23,7 @@ export default class Dryrun extends JSONRequest {
*/
async do(headers = {}) {
const txHeaders = setHeaders(headers);
const res = await this.c.post(
this.path(),
Buffer.from(this.blob),
txHeaders
);
const res = await this.c.post(this.path(), this.blob, txHeaders);
return res.body;
}
}
8 changes: 4 additions & 4 deletions src/client/v2/algod/getApplicationBoxByName.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Buffer } from 'buffer';
import JSONRequest from '../jsonrequest';
import HTTPClient from '../../client';
import { bytesToBase64 } from '../../../encoding/binarydata';
import IntDecoding from '../../../types/intDecoding';
import HTTPClient from '../../client';
import JSONRequest from '../jsonrequest';
import { Box } from './models/types';

/**
Expand Down Expand Up @@ -32,7 +32,7 @@ export default class GetApplicationBoxByName extends JSONRequest<
super(c, intDecoding);
this.index = index;
// Encode name in base64 format and append the encoding prefix.
const encodedName = Buffer.from(name).toString('base64');
const encodedName = bytesToBase64(name);
this.query.name = encodeURI(`b64:${encodedName}`);
}

Expand Down
Loading