be400aefbc4c83e9aab51e82b8d4b12760653b47
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4b2356b1ed |
poly1305: modify s390x assembly to implement MAC interface
The vector (vx) implementation has been updated to read in the state and update it - as opposed to being a single shot function. This has allowed the new MAC interface can be implemented. For performance reasons s390x uses a larger buffer than the generic implementation. There is a relatively high fixed cost to read the state, calculate the key coefficients and serialize the state, so it makes sense to buffer more blocks before calling it. For now I've had to remove the faster VMSL implementation. It is too complex for me to update in time for Go 1.15. At some point I'd like to revisit it but for now it looks like using the MAC interface is more of a win than using VMSL. The benchmarks show considerable improvements when using the MAC interface. The Sum benchmarks show slowdown due to a combination of the removal of the VMSL implementation and also the added overhead from splitting the summation function into multiple parts. poly1305: name old speed new speed delta 64 1.33GB/s ± 0% 0.80GB/s ± 1% -39.51% (p=0.000 n=16+20) 1K 4.04GB/s ± 0% 2.97GB/s ± 0% -26.46% (p=0.000 n=19+19) 2M 5.32GB/s ± 1% 3.63GB/s ± 0% -31.76% (p=0.000 n=20+19) 64Unaligned 1.33GB/s ± 0% 0.80GB/s ± 0% -39.80% (p=0.000 n=19+18) 1KUnaligned 4.09GB/s ± 1% 2.94GB/s ± 0% -28.23% (p=0.000 n=19+18) 2MUnaligned 5.33GB/s ± 1% 3.52GB/s ± 0% -34.04% (p=0.000 n=20+19) Write64 1.03GB/s ± 1% 1.49GB/s ± 1% +44.34% (p=0.000 n=20+20) Write1K 1.21GB/s ± 0% 3.24GB/s ± 0% +169.02% (p=0.000 n=20+17) Write2M 1.24GB/s ± 1% 3.63GB/s ± 0% +192.36% (p=0.000 n=20+19) Write64Unaligned 1.04GB/s ± 1% 1.50GB/s ± 0% +44.16% (p=0.000 n=19+14) Write1KUnaligned 1.21GB/s ± 0% 3.20GB/s ± 0% +164.55% (p=0.000 n=20+16) Write2MUnaligned 1.24GB/s ± 1% 3.51GB/s ± 0% +183.96% (p=0.000 n=20+19) chacha20poly1305 (this vs. using generic MAC interface - post CL 206977): name old speed new speed delta Open-64 147MB/s ± 2% 156MB/s ± 1% +6.15% (p=0.000 n=20+19) Seal-64 151MB/s ± 0% 164MB/s ± 1% +8.86% (p=0.000 n=19+16) Open-64-X 104MB/s ± 2% 111MB/s ± 1% +6.24% (p=0.000 n=20+20) Seal-64-X 109MB/s ± 2% 111MB/s ± 1% +2.11% (p=0.000 n=20+19) Open-1350 555MB/s ± 0% 751MB/s ± 1% +35.19% (p=0.000 n=20+20) Seal-1350 557MB/s ± 0% 759MB/s ± 0% +36.23% (p=0.000 n=20+20) Open-1350-X 517MB/s ± 1% 683MB/s ± 1% +31.97% (p=0.000 n=20+20) Seal-1350-X 511MB/s ± 0% 683MB/s ± 0% +33.77% (p=0.000 n=18+19) Open-8192 672MB/s ± 0% 1013MB/s ± 0% +50.65% (p=0.000 n=19+19) Seal-8192 674MB/s ± 0% 1018MB/s ± 0% +50.98% (p=0.000 n=18+20) Open-8192-X 663MB/s ± 0% 979MB/s ± 0% +47.57% (p=0.000 n=20+20) Seal-8192-X 658MB/s ± 0% 985MB/s ± 0% +49.62% (p=0.000 n=18+20) name old allocs/op new allocs/op delta Open-64 0.00 0.00 ~ (all equal) Seal-64 0.00 0.00 ~ (all equal) Open-64-X 0.00 0.00 ~ (all equal) Seal-64-X 0.00 0.00 ~ (all equal) Open-1350 0.00 0.00 ~ (all equal) Seal-1350 0.00 0.00 ~ (all equal) Open-1350-X 0.00 0.00 ~ (all equal) Seal-1350-X 0.00 0.00 ~ (all equal) Open-8192 0.00 0.00 ~ (all equal) Seal-8192 0.00 0.00 ~ (all equal) Open-8192-X 0.00 0.00 ~ (all equal) Seal-8192-X 0.00 0.00 ~ (all equal) chacha20poly1305 (this vs. using asm Sum interface - pre CL 206977): name old speed new speed delta Open-64 144MB/s ± 0% 156MB/s ± 1% +8.16% (p=0.000 n=20+19) Seal-64 150MB/s ± 0% 164MB/s ± 1% +9.35% (p=0.000 n=20+16) Open-64-X 104MB/s ± 1% 111MB/s ± 1% +6.15% (p=0.000 n=19+20) Seal-64-X 109MB/s ± 1% 111MB/s ± 1% +1.43% (p=0.000 n=19+19) Open-1350 702MB/s ± 1% 751MB/s ± 1% +6.98% (p=0.000 n=20+20) Seal-1350 715MB/s ± 0% 759MB/s ± 0% +6.09% (p=0.000 n=19+20) Open-1350-X 642MB/s ± 0% 683MB/s ± 1% +6.37% (p=0.000 n=19+20) Seal-1350-X 639MB/s ± 0% 683MB/s ± 0% +6.98% (p=0.000 n=20+19) Open-8192 994MB/s ± 0% 1013MB/s ± 0% +1.85% (p=0.000 n=20+19) Seal-8192 1.00GB/s ± 0% 1.02GB/s ± 0% +1.90% (p=0.000 n=20+20) Open-8192-X 965MB/s ± 0% 979MB/s ± 0% +1.43% (p=0.000 n=19+20) Seal-8192-X 962MB/s ± 0% 985MB/s ± 0% +2.39% (p=0.000 n=20+20) name old allocs/op new allocs/op delta Open-64 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-64 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Open-64-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-64-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Open-1350 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-1350 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Open-1350-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-1350-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Open-8192 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-8192 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Open-8192-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Seal-8192-X 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20) Updates golang/go#25219. Change-Id: Ib491e3a47b6b3ec8bbbe1f41f7bf42ad82f5c249 Reviewed-on: https://go-review.googlesource.com/c/crypto/+/219057 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> |
||
|
|
44a6062868 |
poly1305: add (*MAC).Verify API and use it in chacha20poly1305
Also, make sure New inlines so it does not cause an allocation. With this, we have a zero allocation poly1305 flow and a zero allocation generic chacha20poly1305 composition! \o/ While at it, remove some redundant code, and prepare to drop some complexity once the last assembly implementation of sum is dropped. Benchstat with "-tags purego" on amd64 name old time/op new time/op delta Open-64-8 461ns ± 2% 415ns ± 1% -9.93% (p=0.000 n=10+8) Seal-64-8 450ns ± 3% 412ns ± 3% -8.41% (p=0.000 n=10+10) Open-64-X-8 603ns ± 2% 544ns ± 2% -9.84% (p=0.000 n=10+10) Seal-64-X-8 580ns ± 3% 553ns ± 1% -4.56% (p=0.000 n=9+8) Open-1350-8 3.98µs ± 2% 3.65µs ± 2% -8.28% (p=0.000 n=9+10) Seal-1350-8 3.95µs ± 2% 3.64µs ± 1% -7.93% (p=0.000 n=9+10) Open-1350-X-8 4.06µs ± 1% 3.68µs ± 3% -9.31% (p=0.000 n=9+10) Seal-1350-X-8 4.08µs ± 4% 3.64µs ± 1% -10.71% (p=0.000 n=9+10) Open-8192-8 21.7µs ± 3% 18.9µs ± 2% -13.10% (p=0.000 n=9+10) Seal-8192-8 21.5µs ± 3% 18.8µs ± 1% -12.51% (p=0.000 n=9+9) Open-8192-X-8 21.4µs ± 1% 19.1µs ± 2% -10.88% (p=0.000 n=10+10) Seal-8192-X-8 21.3µs ± 2% 19.0µs ± 3% -10.92% (p=0.000 n=10+10) name old speed new speed delta Open-64-8 139MB/s ± 2% 154MB/s ± 2% +11.05% (p=0.000 n=10+8) Seal-64-8 142MB/s ± 3% 155MB/s ± 3% +9.11% (p=0.000 n=10+10) Open-64-X-8 106MB/s ± 2% 118MB/s ± 2% +10.93% (p=0.000 n=10+10) Seal-64-X-8 110MB/s ± 3% 116MB/s ± 1% +4.75% (p=0.000 n=9+8) Open-1350-8 339MB/s ± 2% 370MB/s ± 2% +9.04% (p=0.000 n=9+10) Seal-1350-8 342MB/s ± 2% 371MB/s ± 1% +8.60% (p=0.000 n=9+10) Open-1350-X-8 333MB/s ± 1% 367MB/s ± 3% +10.30% (p=0.000 n=9+10) Seal-1350-X-8 331MB/s ± 4% 371MB/s ± 2% +11.96% (p=0.000 n=9+10) Open-8192-8 377MB/s ± 3% 434MB/s ± 2% +15.07% (p=0.000 n=9+10) Seal-8192-8 381MB/s ± 3% 436MB/s ± 1% +14.29% (p=0.000 n=9+9) Open-8192-X-8 383MB/s ± 1% 429MB/s ± 2% +12.21% (p=0.000 n=10+10) Seal-8192-X-8 385MB/s ± 2% 432MB/s ± 3% +12.26% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Open-64-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) Seal-64-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) Open-64-X-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) Seal-64-X-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10) Open-1350-8 1.41kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Seal-1350-8 1.41kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Open-1350-X-8 1.41kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Seal-1350-X-8 1.41kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Open-8192-8 9.47kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Seal-8192-8 9.47kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Open-8192-X-8 9.47kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) Seal-8192-X-8 9.47kB ± 0% 0.00kB -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Open-64-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-64-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Open-64-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-64-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Open-1350-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-1350-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Open-1350-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-1350-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Open-8192-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-8192-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Open-8192-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Seal-8192-X-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10) Change-Id: I2c30ddc960a889b49c8ee8ff8073ffc4e75f43af Reviewed-on: https://go-review.googlesource.com/c/crypto/+/206977 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> |
||
|
|
2dbfe9001f |
poly1305: rewrite the Go implementation with 64-bit limbs
The new code is meant to be readable without external references for Poly1305, and explains the field logic. The generic code is now 30-50% faster on a Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz, and even better on a 3.1 GHz i7 MacBook. name old time/op new time/op delta 64-48 126ns ± 0% 80ns ± 1% -36.24% (p=0.000 n=16+20) 1K-48 1.07µs ± 0% 0.81µs ± 2% -23.63% (p=0.000 n=19+20) 2M-48 2.07ms ± 0% 1.61ms ± 1% -22.31% (p=0.000 n=20+20) Write64-48 79.3ns ± 0% 58.0ns ± 1% -26.89% (p=0.000 n=20+19) Write1K-48 1.02µs ± 0% 0.79µs ± 1% -22.91% (p=0.000 n=19+19) Write2M-48 2.07ms ± 0% 1.61ms ± 2% -22.33% (p=0.000 n=17+20) name old speed new speed delta 64-48 508MB/s ± 0% 797MB/s ± 1% +56.95% (p=0.000 n=16+20) 1K-48 960MB/s ± 0% 1257MB/s ± 2% +30.94% (p=0.000 n=18+20) 2M-48 1.01GB/s ± 0% 1.30GB/s ± 1% +28.73% (p=0.000 n=20+20) Write64-48 807MB/s ± 0% 1104MB/s ± 1% +36.78% (p=0.000 n=18+19) Write1K-48 1.00GB/s ± 0% 1.30GB/s ± 1% +29.71% (p=0.000 n=18+19) Write2M-48 1.01GB/s ± 0% 1.31GB/s ± 2% +28.77% (p=0.000 n=17+20) The assembly is still 50-90% faster on the Xeon, 30-60% on the MacBook. The Go code does not use all the arithmetic tricks the assembly does, and it does not have access to the three operand wide shift instruction. name old time/op new time/op delta 64-48 80.3ns ± 1% 54.2ns ± 0% -32.50% (p=0.000 n=20+17) 1K-48 815ns ± 2% 446ns ± 1% -45.27% (p=0.000 n=20+20) 2M-48 1.61ms ± 1% 0.86ms ± 0% -46.54% (p=0.000 n=20+17) Write64-48 58.0ns ± 1% 34.0ns ± 0% -41.34% (p=0.000 n=19+20) Write1K-48 790ns ± 1% 427ns ± 0% -45.92% (p=0.000 n=19+17) Write2M-48 1.61ms ± 2% 0.86ms ± 0% -46.51% (p=0.000 n=20+20) name old speed new speed delta 64-48 797MB/s ± 1% 1180MB/s ± 0% +48.09% (p=0.000 n=20+19) 1K-48 1.26GB/s ± 2% 2.30GB/s ± 1% +82.71% (p=0.000 n=20+20) 2M-48 1.30GB/s ± 1% 2.44GB/s ± 0% +87.04% (p=0.000 n=20+17) Write64-48 1.10GB/s ± 1% 1.88GB/s ± 0% +70.52% (p=0.000 n=19+18) Write1K-48 1.30GB/s ± 1% 2.40GB/s ± 0% +84.84% (p=0.000 n=19+18) Write2M-48 1.31GB/s ± 2% 2.44GB/s ± 0% +86.93% (p=0.000 n=20+20) Hopefully this will also avoid the need for an arm64 implementation. Since now the Go and the amd64/ppc64le assembly use the same limb schedule, drop the assembly initialize and finalize implementations, and make the wrapper code match. It comes with a minor slowdown. name old time/op new time/op delta 64-48 50.3ns ± 0% 54.2ns ± 0% +7.73% (p=0.000 n=20+17) 1K-48 441ns ± 0% 446ns ± 1% +1.10% (p=0.000 n=19+20) 2M-48 860µs ± 0% 859µs ± 0% ~ (p=0.178 n=19+17) Write64-48 34.0ns ± 0% 34.0ns ± 0% ~ (all equal) Write1K-48 424ns ± 0% 427ns ± 0% +0.71% (p=0.000 n=17+17) Write2M-48 860µs ± 0% 859µs ± 0% -0.04% (p=0.000 n=19+20) name old speed new speed delta 64-48 1.27GB/s ± 0% 1.18GB/s ± 0% -7.20% (p=0.000 n=20+19) 1K-48 2.32GB/s ± 0% 2.30GB/s ± 1% -1.07% (p=0.000 n=18+20) 2M-48 2.44GB/s ± 0% 2.44GB/s ± 0% ~ (p=0.173 n=19+17) Write64-48 1.88GB/s ± 0% 1.88GB/s ± 0% +0.04% (p=0.000 n=19+18) Write1K-48 2.41GB/s ± 0% 2.40GB/s ± 0% -0.67% (p=0.000 n=19+18) Write2M-48 2.44GB/s ± 0% 2.44GB/s ± 0% +0.04% (p=0.000 n=19+20) Since poly1305/sum_generic.go was almost entirely rewritten, it's probably best reviewed on gitiles. This is the implementation published at https://blog.filippo.io/a-literate-go-implementation-of-poly1305/ Updates #31470 Change-Id: I74f9011d3ee317a43b05ae7f05d96081d08bffd3 Reviewed-on: https://go-review.googlesource.com/c/crypto/+/169037 Reviewed-by: Katie Hockman <katie@golang.org> |
||
|
|
c2843e01d9 |
poly1305: implement a subset of the hash.Hash interface
This CL adds the poly1305.MAC type which implements a subset of the hash.Hash interface. With MAC it is possible to compute an authentication tag of data without copying it into a single byte slice. This commit modifies the reference/generic and the AMD64 assembler but not the ARM/s390x implementation to support an io.Writer interface. Updates golang/go#25219 Change-Id: I7ee5a9eadd43387cf3cd887d734c625575eee47d Reviewed-on: https://go-review.googlesource.com/c/crypto/+/111335 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> |
||
|
|
425cc7d9a7 |
poly1305: add additional test cases
Increase the number of test vectors in this package to provide better validation of new SIMD implementations. Change-Id: Ia89883609e78cef53ba40a9cae41f4e0a3bccc80 Reviewed-on: https://go-review.googlesource.com/112855 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> |
||
|
|
4eb8c2c8d8 |
poly1305: add optimized s390x SIMD implementation with VMSL
SIMD implementation based the on the algorithm outlined in: NEON crypto, Daniel J. Bernstein and Peter Schwabe https://cryptojedi.org/papers/neoncrypto-20120320.pdf and as modified for VMSL as described in Accelerating Poly1305 Cryptographic Message Authentication on the z14 O'Farrell, Gadriwala, et al, CASCON 2017, p48-55 https://ibm.ent.box.com/s/jf9gedj0e9d2vjctfyh186shaztavnht name old new delta 64 485MB/s 1315 MB/s +171.58% 1K 607MB/s 4352 MB/s +616.97% 64Unaligned 485MB/s 1373 MB/s +183.09% 1KUnaligned 606MB/s 4286 MB/s +607.26% 2M 607MB/s 5529 MB/s +810.87% Change-Id: I31ccc25ced09180d99ea5c9233f0dcdc8666fc98 Reviewed-on: https://go-review.googlesource.com/110297 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Munday <mike.munday@ibm.com> |
||
|
|
453249f01c |
poly1305: add burn-in test.
This is the test that I use to sanity-check significant changes to the package, thus it's probably worth checking it in. Since it's very slow, it's disabled by default. (Note that while it stands a good chance of catching errors in 32-bit implementations, no amount of random testing is going to get useful coverage for 64-bit implementations. Thus it really is just a sanity check, despite the long run-time.) Change-Id: I95b321eec6f3026dafbbc157a7ef35a27e88d247 Reviewed-on: https://go-review.googlesource.com/36566 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> |
||
|
|
dc137beb6c |
poly1305: add test vectors for edge cases
Often intermediate results of poly1305 calculations are only reduced to the range [0, 2^130). These new test vectors exercise the code that reduces the final output to the range [0, 2^130-5). This improves the test coverage of CL 35294 and CL 32812. Change-Id: Ifd2f64d4668c08a396ed81db3e88969a49baf777 Reviewed-on: https://go-review.googlesource.com/35918 Run-TryBot: Michael Munday <munday@ca.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Adam Langley <agl@golang.org> |
||
|
|
7682e7e394 |
poly1305: add test for carry edge-case.
This change adds a test that catches the bug which existed in [https://go-review.googlesource.com/#/c/29993/, /30101/). Change-Id: I71177fec0fcbca0dcec7fb11d3ad0a48df1e0b82 Reviewed-on: https://go-review.googlesource.com/30214 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> |
||
|
|
81bf7719a6 |
x/crypto/poly1305: fix memory alignment fault in ARM
The current ARM implementation assumes that the input message is memory aligned and so it can cause alignment fault when it is not enabled. Also it may generate incorrect outputs in ARMv5. This change fixes this issue by temporarily copying the input to a local aligned space. Although there may be a better way to handle unaligned access, this would be a safe way in all ARM versions. This change also added a test and benchmarks with unaligned data. The benchmark result on RasberryPI 2 is Benchmark64 2000000 812 ns/op 78.81 MB/s Benchmark1K 200000 7809 ns/op 131.12 MB/s Benchmark64Unaligned 2000000 967 ns/op 66.13 MB/s Benchmark1KUnaligned 200000 10316 ns/op 99.26 MB/s Change-Id: I189cc1b7bb6c67a04c9877271fb27326f2896e82 Reviewed-on: https://go-review.googlesource.com/12797 Reviewed-by: Adam Langley <agl@golang.org> |
||
|
|
6814ed3bb5 |
go.crypto/poly1305: add package.
(Reference implementation by dchest. amd64 disabled pending 6l fix.) R=golang-dev, dchest CC=golang-dev https://golang.org/cl/6494105 |