Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SHA256 #1285

Merged
merged 58 commits into from
Jan 17, 2024
Merged

SHA256 #1285

Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
cc46f5b
it works
Trivo25 Dec 1, 2023
35a15f5
add dummy api
Trivo25 Dec 1, 2023
a1e8f17
maybe some improvements
Trivo25 Dec 1, 2023
aa85082
slight refac
Trivo25 Dec 1, 2023
880a412
more efficient hashing
Trivo25 Dec 1, 2023
79068e3
faster rotation of 3 values at once
mitschabaude Dec 5, 2023
e2bcd0d
save some range checks
mitschabaude Dec 6, 2023
a82c63b
clean up a bit
mitschabaude Dec 6, 2023
17c7c88
support Field as input to UInt32.xor
mitschabaude Dec 6, 2023
62db5b2
cover delta0/1 with fast 3-rotation
mitschabaude Dec 6, 2023
c783258
remove unused function
mitschabaude Dec 6, 2023
ef874db
express maj with fewer xors
mitschabaude Dec 6, 2023
0ba733a
speed up Ch by replacing 1 XOR with addition
mitschabaude Dec 6, 2023
cf48659
simplify logic
mitschabaude Dec 6, 2023
3efe56f
let's not introduce an unused function
mitschabaude Dec 6, 2023
1b4279b
do smaller range checks on the quotient for add mod 32
mitschabaude Dec 6, 2023
5223042
Merge pull request #1296 from o1-labs/perf/sha2-multi-rot
Trivo25 Dec 6, 2023
7d0bbea
Merge pull request #1299 from o1-labs/perf/sha2-maj-trick
Trivo25 Dec 15, 2023
d8d3ec8
Merge branch 'feature/gadgets-uint' into dog-food-sha256
Trivo25 Dec 19, 2023
99094d6
start touching up SHA256 API
Trivo25 Dec 19, 2023
691ac67
finish API, start tests
Trivo25 Dec 19, 2023
dc4ddb9
fix block size, add tests
Trivo25 Dec 19, 2023
c49e07b
add tests
Trivo25 Dec 19, 2023
ea83f74
add trivial example
Trivo25 Dec 19, 2023
9b61ead
move example, dump VKs
Trivo25 Dec 19, 2023
9e9a3cb
add SHA256 to Hash namespace
Trivo25 Dec 19, 2023
a6252bd
make witness constant
Trivo25 Dec 20, 2023
3ba3824
simplify
Trivo25 Dec 20, 2023
7197424
add doc comments
Trivo25 Dec 20, 2023
c779b24
Merge branch 'main' into dog-food-sha256
Trivo25 Dec 20, 2023
498ab34
Update sha256.ts
Trivo25 Dec 20, 2023
ebe2028
undo accidental commit
Trivo25 Dec 20, 2023
68b6d1a
add commit back :X
Trivo25 Dec 20, 2023
175228b
add flow comments
Trivo25 Dec 20, 2023
3e77309
Merge branch 'dog-food-sha256' of https://github.com/o1-labs/o1js int…
Trivo25 Dec 20, 2023
06afa39
dump vks
Trivo25 Dec 20, 2023
0a60312
move example
Trivo25 Dec 21, 2023
f789d5a
dump vks
Trivo25 Dec 21, 2023
ca4c7c9
dump vks
Trivo25 Dec 21, 2023
a816814
improve doc comment
Trivo25 Dec 21, 2023
7747e75
address part of feedback
Trivo25 Dec 21, 2023
198ecb8
move constants
Trivo25 Dec 21, 2023
98bed63
improve tests
Trivo25 Dec 21, 2023
e98fae9
move keccak helper functions
Trivo25 Dec 21, 2023
a1aa7c8
move helpers
Trivo25 Dec 21, 2023
015db6b
simply decomposition of variables
Trivo25 Dec 21, 2023
a477a0c
fix dependency hell
Trivo25 Jan 2, 2024
e318e3e
add assert to export
Trivo25 Jan 2, 2024
9307007
fix import cycles by moving high-level dependencies out of gadgets/co…
mitschabaude Jan 8, 2024
dd7b88a
move other bit slicing gadget to bit slicing file
mitschabaude Jan 8, 2024
c202d9d
remove cyclic top-level gadgets dep from sha256.ts
mitschabaude Jan 8, 2024
1f7ebb2
remove unused function
mitschabaude Jan 8, 2024
f077283
dump vks
mitschabaude Jan 8, 2024
6b8d48a
save a few constraints with seal()
mitschabaude Jan 8, 2024
d71872d
add comments
mitschabaude Jan 8, 2024
fd57e7f
Merge branch 'main' into dog-food-sha256
Trivo25 Jan 16, 2024
b16d146
update changelog
Trivo25 Jan 16, 2024
23d096c
address feedback
Trivo25 Jan 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

## [Unreleased](https://github.com/o1-labs/o1js/compare/08ba27329...HEAD)

### Added

- **SHA256 hash function** exposed via `Hash.SHA2_256` or `Gadgets.SHA256`. https://github.com/o1-labs/o1js/pull/1285

### Fixed

- Fix approving of complex account update layouts https://github.com/o1-labs/o1js/pull/1364
Expand Down Expand Up @@ -57,6 +61,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- bitwise AND via `{UInt32, UInt64}.and()`
- Example for using actions to store a map data structure https://github.com/o1-labs/o1js/pull/1300
- `Provable.constraintSystem()` and `{ZkProgram,SmartContract}.analyzeMethods()` return a `summary()` method to return a summary of the constraints used by a method https://github.com/o1-labs/o1js/pull/1007
- `assert()` asserts that a given statement is true https://github.com/o1-labs/o1js/pull/1285

### Fixed

Expand Down
23 changes: 23 additions & 0 deletions src/examples/crypto/sha256/run.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import { Bytes12, SHA256Program } from './sha256.js';

console.time('compile');
await SHA256Program.compile();
console.timeEnd('compile');

let preimage = Bytes12.fromString('hello world!');

console.log('sha256 rows:', SHA256Program.analyzeMethods().sha256.rows);

console.time('prove');
let proof = await SHA256Program.sha256(preimage);
console.timeEnd('prove');
let isValid = await SHA256Program.verify(proof);

console.log('digest:', proof.publicOutput.toHex());

if (
proof.publicOutput.toHex() !==
'7509e5bda0c762d2bac7f90d758b5b2263fa01ccbc542ab5e3df163be08e6ca9'
)
throw new Error('Invalid sha256 digest!');
if (!isValid) throw new Error('Invalid proof');
18 changes: 18 additions & 0 deletions src/examples/crypto/sha256/sha256.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { Bytes, Gadgets, ZkProgram } from 'o1js';

export { SHA256Program, Bytes12 };

class Bytes12 extends Bytes(12) {}

let SHA256Program = ZkProgram({
name: 'sha256',
publicOutput: Bytes(32).provable,
methods: {
sha256: {
privateInputs: [Bytes12.provable],
method(xs: Bytes12) {
return Gadgets.SHA256.hash(xs);
},
},
},
});
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ export { Poseidon, TokenSymbol } from './lib/hash.js';
export { Keccak } from './lib/keccak.js';
export { Hash } from './lib/hashes-combined.js';

export { assert } from './lib/gadgets/common.js';

export * from './lib/signature.js';
export type {
ProvableExtended,
Expand Down
13 changes: 9 additions & 4 deletions src/lib/gadgets/arithmetic.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import { Bool } from '../bool.js';
import { provableTuple } from '../circuit_value.js';
import { Field } from '../core.js';
import { assert } from '../errors.js';
import { Provable } from '../provable.js';
import { rangeCheck32 } from './range-check.js';
import { rangeCheck32, rangeCheckN } from './range-check.js';

export { divMod32, addMod32 };

function divMod32(n: Field) {
function divMod32(n: Field, quotientBits = 32) {
if (n.isConstant()) {
assert(
n.toBigInt() < 1n << 64n,
Expand All @@ -32,7 +33,11 @@ function divMod32(n: Field) {
}
);

rangeCheck32(quotient);
if (quotientBits === 1) {
Bool.check(Bool.Unsafe.ofField(quotient));
} else {
rangeCheckN(quotientBits, quotient);
}
rangeCheck32(remainder);

n.assertEquals(quotient.mul(1n << 32n).add(remainder));
Expand All @@ -44,5 +49,5 @@ function divMod32(n: Field) {
}

function addMod32(x: Field, y: Field) {
return divMod32(x.add(y)).remainder;
return divMod32(x.add(y), 1).remainder;
}
156 changes: 156 additions & 0 deletions src/lib/gadgets/bit-slices.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
/**
* Gadgets for converting between field elements and bit slices of various lengths
*/
import { bigIntToBits } from '../../bindings/crypto/bigint-helpers.js';
import { Bool } from '../bool.js';
import { Field } from '../field.js';
import { UInt8 } from '../int.js';
import { Provable } from '../provable.js';
import { chunk } from '../util/arrays.js';
import { assert, exists } from './common.js';
import type { Field3 } from './foreign-field.js';
import { l } from './range-check.js';

export { bytesToWord, wordToBytes, wordsToBytes, bytesToWords, sliceField3 };

// conversion between bytes and multi-byte words

/**
* Convert an array of UInt8 to a Field element. Expects little endian representation.
*/
function bytesToWord(wordBytes: UInt8[]): Field {
Trivo25 marked this conversation as resolved.
Show resolved Hide resolved
return wordBytes.reduce((acc, byte, idx) => {
const shift = 1n << BigInt(8 * idx);
return acc.add(byte.value.mul(shift));
}, Field.from(0));
}

/**
* Convert a Field element to an array of UInt8. Expects little endian representation.
* @param bytesPerWord number of bytes per word
*/
function wordToBytes(word: Field, bytesPerWord = 8): UInt8[] {
Trivo25 marked this conversation as resolved.
Show resolved Hide resolved
let bytes = Provable.witness(Provable.Array(UInt8, bytesPerWord), () => {
let w = word.toBigInt();
return Array.from({ length: bytesPerWord }, (_, k) =>
UInt8.from((w >> BigInt(8 * k)) & 0xffn)
);
});

// check decomposition
bytesToWord(bytes).assertEquals(word);

return bytes;
}

/**
* Convert an array of Field elements to an array of UInt8. Expects little endian representation.
* @param bytesPerWord number of bytes per word
*/
function wordsToBytes(words: Field[], bytesPerWord = 8): UInt8[] {
Trivo25 marked this conversation as resolved.
Show resolved Hide resolved
return words.flatMap((w) => wordToBytes(w, bytesPerWord));
}
/**
* Convert an array of UInt8 to an array of Field elements. Expects little endian representation.
* @param bytesPerWord number of bytes per word
*/
function bytesToWords(bytes: UInt8[], bytesPerWord = 8): Field[] {
Trivo25 marked this conversation as resolved.
Show resolved Hide resolved
return chunk(bytes, bytesPerWord).map(bytesToWord);
}

// conversion between 3-limb foreign fields and arbitrary bit slices

/**
* Provable method for slicing a 3x88-bit bigint into smaller bit chunks of length `chunkSize`
*
* This serves as a range check that the input is in [0, 2^maxBits)
*/
function sliceField3(
[x0, x1, x2]: Field3,
{ maxBits, chunkSize }: { maxBits: number; chunkSize: number }
) {
let l_ = Number(l);
assert(maxBits <= 3 * l_, `expected max bits <= 3*${l_}, got ${maxBits}`);

// first limb
let result0 = sliceField(x0, Math.min(l_, maxBits), chunkSize);
if (maxBits <= l_) return result0.chunks;
maxBits -= l_;

// second limb
let result1 = sliceField(x1, Math.min(l_, maxBits), chunkSize, result0);
if (maxBits <= l_) return result0.chunks.concat(result1.chunks);
maxBits -= l_;

// third limb
let result2 = sliceField(x2, maxBits, chunkSize, result1);
return result0.chunks.concat(result1.chunks, result2.chunks);
}

/**
* Provable method for slicing a field element into smaller bit chunks of length `chunkSize`.
*
* This serves as a range check that the input is in [0, 2^maxBits)
*
* If `chunkSize` does not divide `maxBits`, the last chunk will be smaller.
* We return the number of free bits in the last chunk, and optionally accept such a result from a previous call,
* so that this function can be used to slice up a bigint of multiple limbs into homogeneous chunks.
*
* TODO: atm this uses expensive boolean checks for each bit.
* For larger chunks, we should use more efficient range checks.
*/
function sliceField(
x: Field,
maxBits: number,
chunkSize: number,
leftover?: { chunks: Field[]; leftoverSize: number }
) {
let bits = exists(maxBits, () => {
let bits = bigIntToBits(x.toBigInt());
// normalize length
if (bits.length > maxBits) bits = bits.slice(0, maxBits);
if (bits.length < maxBits)
bits = bits.concat(Array(maxBits - bits.length).fill(false));
return bits.map(BigInt);
});

let chunks = [];
let sum = Field.from(0n);

// if there's a leftover chunk from a previous sliceField() call, we complete it
if (leftover !== undefined) {
let { chunks: previous, leftoverSize: size } = leftover;
let remainingChunk = Field.from(0n);
for (let i = 0; i < size; i++) {
let bit = bits[i];
Bool.check(Bool.Unsafe.ofField(bit));
remainingChunk = remainingChunk.add(bit.mul(1n << BigInt(i)));
}
sum = remainingChunk = remainingChunk.seal();
let chunk = previous[previous.length - 1];
previous[previous.length - 1] = chunk.add(
remainingChunk.mul(1n << BigInt(chunkSize - size))
);
}

let i = leftover?.leftoverSize ?? 0;
for (; i < maxBits; i += chunkSize) {
// prove that chunk has `chunkSize` bits
// TODO: this inner sum should be replaced with a more efficient range check when possible
let chunk = Field.from(0n);
let size = Math.min(maxBits - i, chunkSize); // last chunk might be smaller
for (let j = 0; j < size; j++) {
let bit = bits[i + j];
Bool.check(Bool.Unsafe.ofField(bit));
chunk = chunk.add(bit.mul(1n << BigInt(j)));
}
chunk = chunk.seal();
// prove that chunks add up to x
sum = sum.add(chunk.mul(1n << BigInt(i)));
chunks.push(chunk);
}
sum.assertEquals(x);

let leftoverSize = i - maxBits;
return { chunks, leftoverSize } as const;
}
11 changes: 9 additions & 2 deletions src/lib/gadgets/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Field, FieldConst, FieldVar, VarField } from '../field.js';
import { Tuple, TupleN } from '../util/types.js';
import { Snarky } from '../../snarky.js';
import { MlArray } from '../ml/base.js';
import { Bool } from '../bool.js';

const MAX_BITS = 64 as const;

Expand Down Expand Up @@ -62,8 +63,14 @@ function toVars<T extends Tuple<Field | bigint>>(
return Tuple.map(fields, toVar);
}

function assert(stmt: boolean, message?: string): asserts stmt {
if (!stmt) {
/**
* Assert that a statement is true. If the statement is false, throws an error with the given message.
* Can be used in provable code.
*/
function assert(stmt: boolean | Bool, message?: string): asserts stmt {
if (stmt instanceof Bool) {
stmt.assertTrue(message ?? 'Assertion failed');
} else if (!stmt) {
throw Error(message ?? 'Assertion failed');
}
}
Expand Down
Loading