-
-
-# Usage
-
-This section assumes you know the basics of Reed-Solomon encoding.
-A good start is this [Backblaze blog post](https://www.backblaze.com/blog/reed-solomon/).
-
-This package performs the calculation of the parity sets. The usage is therefore relatively simple.
-
-First of all, you need to choose your distribution of data and parity shards.
-A 'good' distribution is very subjective, and will depend a lot on your usage scenario.
-
-To create an encoder with 10 data shards (where your data goes) and 3 parity shards (calculated):
-```Go
- enc, err := reedsolomon.New(10, 3)
-```
-This encoder will work for all parity sets with this distribution of data and parity shards.
-
-If you will primarily be using it with one shard size it is recommended to use
-[`WithAutoGoroutines(shardSize)`](https://pkg.go.dev/github.com/klauspost/reedsolomon?tab=doc#WithAutoGoroutines)
-as an additional parameter. This will attempt to calculate the optimal number of goroutines to use for the best speed.
-It is not required that all shards are this size.
-
-Then you send and receive data that is a simple slice of byte slices; `[][]byte`.
-In the example above, the top slice must have a length of 13.
-
-```Go
- data := make([][]byte, 13)
-```
-You should then fill the 10 first slices with *equally sized* data,
-and create parity shards that will be populated with parity data. In this case we create the data in memory,
-but you could for instance also use [mmap](https://github.com/edsrzf/mmap-go) to map files.
-
-```Go
- // Create all shards, size them at 50000 each
- for i := range input {
- data[i] := make([]byte, 50000)
- }
-
-
- // Fill some data into the data shards
- for i, in := range data[:10] {
- for j:= range in {
- in[j] = byte((i+j)&0xff)
- }
- }
-```
-
-To populate the parity shards, you simply call `Encode()` with your data.
-```Go
- err = enc.Encode(data)
-```
-The only cases where you should get an error is, if the data shards aren't of equal size.
-The last 3 shards now contain parity data. You can verify this by calling `Verify()`:
-
-```Go
- ok, err = enc.Verify(data)
-```
-
-The final (and important) part is to be able to reconstruct missing shards.
-For this to work, you need to know which parts of your data is missing.
-The encoder *does not know which parts are invalid*, so if data corruption is a likely scenario,
-you need to implement a hash check for each shard.
-
-If a byte has changed in your set, and you don't know which it is, there is no way to reconstruct the data set.
-
-To indicate missing data, you set the shard to nil before calling `Reconstruct()`:
-
-```Go
- // Delete two data shards
- data[3] = nil
- data[7] = nil
-
- // Reconstruct the missing shards
- err := enc.Reconstruct(data)
-```
-The missing data and parity shards will be recreated. If more than 3 shards are missing, the reconstruction will fail.
-
-If you are only interested in the data shards (for reading purposes) you can call `ReconstructData()`:
-
-```Go
- // Delete two data shards
- data[3] = nil
- data[7] = nil
-
- // Reconstruct just the missing data shards
- err := enc.ReconstructData(data)
-```
-
-If you don't need all data shards you can use `ReconstructSome()`:
-
-```Go
- // Delete two data shards
- data[3] = nil
- data[7] = nil
-
- // Reconstruct just the shard 3
- err := enc.ReconstructSome(data, []bool{false, false, false, true, false, false, false, false})
-```
-
-So to sum up reconstruction:
-* The number of data/parity shards must match the numbers used for encoding.
-* The order of shards must be the same as used when encoding.
-* You may only supply data you know is valid.
-* Invalid shards should be set to nil.
-
-For complete examples of an encoder and decoder see the
-[examples folder](https://github.com/klauspost/reedsolomon/tree/master/examples).
-
-# Splitting/Joining Data
-
-You might have a large slice of data.
-To help you split this, there are some helper functions that can split and join a single byte slice.
-
-```Go
- bigfile, _ := ioutil.Readfile("myfile.data")
-
- // Split the file
- split, err := enc.Split(bigfile)
-```
-This will split the file into the number of data shards set when creating the encoder and create empty parity shards.
-
-An important thing to note is that you have to *keep track of the exact input size*.
-If the size of the input isn't divisible by the number of data shards, extra zeros will be inserted in the last shard.
-
-To join a data set, use the `Join()` function, which will join the shards and write it to the `io.Writer` you supply:
-```Go
- // Join a data set and write it to io.Discard.
- err = enc.Join(io.Discard, data, len(bigfile))
-```
-
-# Progressive encoding
-
-It is possible to encode individual shards using EncodeIdx:
-
-```Go
- // EncodeIdx will add parity for a single data shard.
- // Parity shards should start out as 0. The caller must zero them.
- // Data shards must be delivered exactly once. There is no check for this.
- // The parity shards will always be updated and the data shards will remain the same.
- EncodeIdx(dataShard []byte, idx int, parity [][]byte) error
-```
-
-This allows progressively encoding the parity by sending individual data shards.
-There is no requirement on shards being delivered in order,
-but when sent in order it allows encoding shards one at the time,
-effectively allowing the operation to be streaming.
-
-The result will be the same as encoding all shards at once.
-There is a minor speed penalty using this method, so send
-shards at once if they are available.
-
-## Example
-
-```Go
-func test() {
- // Create an encoder with 7 data and 3 parity slices.
- enc, _ := reedsolomon.New(7, 3)
-
- // This will be our output parity.
- parity := make([][]byte, 3)
- for i := range parity {
- parity[i] = make([]byte, 10000)
- }
-
- for i := 0; i < 7; i++ {
- // Send data shards one at the time.
- _ = enc.EncodeIdx(make([]byte, 10000), i, parity)
- }
-
- // parity now contains parity, as if all data was sent in one call.
-}
-```
-
-# Streaming/Merging
-
-It might seem like a limitation that all data should be in memory,
-but an important property is that *as long as the number of data/parity shards are the same,
-you can merge/split data sets*, and they will remain valid as a separate set.
-
-```Go
- // Split the data set of 50000 elements into two of 25000
- splitA := make([][]byte, 13)
- splitB := make([][]byte, 13)
-
- // Merge into a 100000 element set
- merged := make([][]byte, 13)
-
- for i := range data {
- splitA[i] = data[i][:25000]
- splitB[i] = data[i][25000:]
-
- // Concatenate it to itself
- merged[i] = append(make([]byte, 0, len(data[i])*2), data[i]...)
- merged[i] = append(merged[i], data[i]...)
- }
-
- // Each part should still verify as ok.
- ok, err := enc.Verify(splitA)
- if ok && err == nil {
- log.Println("splitA ok")
- }
-
- ok, err = enc.Verify(splitB)
- if ok && err == nil {
- log.Println("splitB ok")
- }
-
- ok, err = enc.Verify(merge)
- if ok && err == nil {
- log.Println("merge ok")
- }
-```
-
-This means that if you have a data set that may not fit into memory, you can split processing into smaller blocks.
-For the best throughput, don't use too small blocks.
-
-This also means that you can divide big input up into smaller blocks, and do reconstruction on parts of your data.
-This doesn't give the same flexibility of a higher number of data shards, but it will be much more performant.
-
-# Streaming API
-
-There has been added support for a streaming API, to help perform fully streaming operations,
-which enables you to do the same operations, but on streams.
-To use the stream API, use [`NewStream`](https://godoc.org/github.com/klauspost/reedsolomon#NewStream) function
-to create the encoding/decoding interfaces.
-
-You can use [`WithConcurrentStreams`](https://godoc.org/github.com/klauspost/reedsolomon#WithConcurrentStreams)
-to ready an interface that reads/writes concurrently from the streams.
-
-You can specify the size of each operation using
-[`WithStreamBlockSize`](https://godoc.org/github.com/klauspost/reedsolomon#WithStreamBlockSize).
-This will set the size of each read/write operation.
-
-Input is delivered as `[]io.Reader`, output as `[]io.Writer`, and functionality corresponds to the in-memory API.
-Each stream must supply the same amount of data, similar to how each slice must be similar size with the in-memory API.
-If an error occurs in relation to a stream,
-a [`StreamReadError`](https://godoc.org/github.com/klauspost/reedsolomon#StreamReadError)
-or [`StreamWriteError`](https://godoc.org/github.com/klauspost/reedsolomon#StreamWriteError)
-will help you determine which stream was the offender.
-
-There is no buffering or timeouts/retry specified. If you want to add that, you need to add it to the Reader/Writer.
-
-For complete examples of a streaming encoder and decoder see the
-[examples folder](https://github.com/klauspost/reedsolomon/tree/master/examples).
-
-GF16 (more than 256 shards) is not supported by the streaming interface.
-
-# Advanced Options
-
-You can modify internal options which affects how jobs are split between and processed by goroutines.
-
-To create options, use the WithXXX functions. You can supply options to `New`, `NewStream`.
-If no Options are supplied, default options are used.
-
-Example of how to supply options:
-
- ```Go
- enc, err := reedsolomon.New(10, 3, WithMaxGoroutines(25))
- ```
-
-# Leopard Compatible GF16
-
-When you encode more than 256 shards the library will switch to a [Leopard-RS](https://github.com/catid/leopard) implementation.
-
-This allows encoding up to 65536 shards (data+parity) with the following limitations, similar to leopard:
-
-* The original and recovery data must not exceed 65536 pieces.
-* The shard size *must* each be a multiple of 64 bytes.
-* Each buffer should have the same number of bytes.
-* Even the last shard must be rounded up to the block size.
-
-| | Regular | Leopard |
-|-----------------|---------|---------|
-| Encode | ✓ | ✓ |
-| EncodeIdx | ✓ | - |
-| Verify | ✓ | ✓ |
-| Reconstruct | ✓ | ✓ |
-| ReconstructData | ✓ | ✓ |
-| ReconstructSome | ✓ | ✓ (+) |
-| Update | ✓ | - |
-| Split | ✓ | ✓ |
-| Join | ✓ | ✓ |
-
-* (+) Same as calling `ReconstructData`.
-
-The Split/Join functions will help to split an input to the proper sizes.
-
-Speed can be expected to be `O(N*log(N))`, compared to the `O(N*N)`.
-Reconstruction matrix calculation is more time-consuming,
-so be sure to include that as part of any benchmark you run.
-
-For now SSSE3, AVX2 and AVX512 assembly are available on AMD64 platforms.
-
-Leopard mode currently always runs as a single goroutine, since multiple gorouties doesn't provide any worthwhile speedup.
-
-## Forcing Leopard GF16
-
-The `WithLeopardGF16(true)` can be used to use Leopard GF16 for all operations.
-This is *not* compatible with the Leopard library that has a separate GF8 implementation.
-
-Benchmark Encoding and Reconstructing *1KB* shards with variable number of shards.
-For Cauchy matrix the inversion cache is disabled for a more "fair" test.
-Speed is total shard size for each operation. Data shard throughput is speed/2.
-AVX2 is used.
-
-| Encoder | Shards | Encode | Recover All | Recover One |
-|--------------|-------------|---------------|--------------|--------------|
-| Cauchy | 4+4 | 23076.83 MB/s | 3048.86 MB/s | 5620.84 MB/s |
-| Cauchy | 8+8 | 15206.87 MB/s | 3041.99 MB/s | 7173.71 MB/s |
-| Cauchy | 16+16 | 7427.47 MB/s | 1384.58 MB/s | 6343.85 MB/s |
-| Cauchy | 32+32 | 3785.64 MB/s | 557.60 MB/s | 4660.27 MB/s |
-| Cauchy | 64+64 | 1911.93 MB/s | 160.54 MB/s | 2864.63 MB/s |
-| Cauchy | 128+128 | 963.83 MB/s | 42.81 MB/s | 1597.93 MB/s |
-| Leopard GF16 | 4+4 | 18468.32 MB/s | 10.45 MB/s | 10.30 MB/s |
-| Leopard GF16 | 8+8 | 10293.79 MB/s | 20.83 MB/s | 20.51 MB/s |
-| Leopard GF16 | 16+16 | 12386.04 MB/s | 40.80 MB/s | 40.47 MB/s |
-| Leopard GF16 | 32+32 | 7347.35 MB/s | 81.15 MB/s | 79.80 MB/s |
-| Leopard GF16 | 64+64 | 8299.63 MB/s | 150.47 MB/s | 154.15 MB/s |
-| Leopard GF16 | 128+128 | 5629.04 MB/s | 278.84 MB/s | 289.15 MB/s |
-| Leopard GF16 | 256+256 | 6158.66 MB/s | 454.14 MB/s | 506.70 MB/s |
-| Leopard GF16 | 512+512 | 4418.58 MB/s | 685.75 MB/s | 801.63 MB/s |
-| Leopard GF16 | 1024+1024 | 4778.05 MB/s | 814.51 MB/s | 1080.19 MB/s |
-| Leopard GF16 | 2048+2048 | 3417.05 MB/s | 911.64 MB/s | 1179.48 MB/s |
-| Leopard GF16 | 4096+4096 | 3209.41 MB/s | 729.13 MB/s | 1135.06 MB/s |
-| Leopard GF16 | 8192+8192 | 2034.11 MB/s | 604.52 MB/s | 842.13 MB/s |
-| Leopard GF16 | 16384+16384 | 1525.88 MB/s | 486.74 MB/s | 750.01 MB/s |
-| Leopard GF16 | 32768+32768 | 1138.67 MB/s | 482.81 MB/s | 712.73 MB/s |
-
-"Traditional" encoding is faster until somewhere between 16 and 32 shards.
-Leopard provides fast encoding in all cases, but shows a significant overhead for reconstruction.
-
-Calculating the reconstruction matrix takes a significant amount of computation.
-With bigger shards that will be smaller. Arguably, fewer shards typically also means bigger shards.
-Due to the high shard count caching reconstruction matrices generally isn't feasible for Leopard.
-
-# Performance
-
-Performance depends mainly on the number of parity shards.
-In rough terms, doubling the number of parity shards will double the encoding time.
-
-Here are the throughput numbers with some different selections of data and parity shards.
-For reference each shard is 1MB random data, and 16 CPU cores are used for encoding.
-
-| Data | Parity | Go MB/s | SSSE3 MB/s | AVX2 MB/s |
-|------|--------|---------|------------|-----------|
-| 5 | 2 | 20,772 | 66,355 | 108,755 |
-| 8 | 8 | 6,815 | 38,338 | 70,516 |
-| 10 | 4 | 9,245 | 48,237 | 93,875 |
-| 50 | 20 | 2,063 | 12,130 | 22,828 |
-
-The throughput numbers here is the size of the encoded data and parity shards.
-
-If `runtime.GOMAXPROCS()` is set to a value higher than 1,
-the encoder will use multiple goroutines to perform the calculations in `Verify`, `Encode` and `Reconstruct`.
-
-
-Benchmarking `Reconstruct()` followed by a `Verify()` (=`all`) versus just calling `ReconstructData()` (=`data`) gives the following result:
-```
-benchmark all MB/s data MB/s speedup
-BenchmarkReconstruct10x2x10000-8 2011.67 10530.10 5.23x
-BenchmarkReconstruct50x5x50000-8 4585.41 14301.60 3.12x
-BenchmarkReconstruct10x2x1M-8 8081.15 28216.41 3.49x
-BenchmarkReconstruct5x2x1M-8 5780.07 28015.37 4.85x
-BenchmarkReconstruct10x4x1M-8 4352.56 14367.61 3.30x
-BenchmarkReconstruct50x20x1M-8 1364.35 4189.79 3.07x
-BenchmarkReconstruct10x4x16M-8 1484.35 5779.53 3.89x
-```
-
-The performance on AVX512 has been accelerated for CPUs when available.
-
-## ARM64 NEON
-
-By exploiting NEON instructions the performance for ARM has been accelerated.
-Below are the performance numbers for a single core on an EC2 m6g.16xlarge (Graviton2) instance (Amazon Linux 2):
-
-```
-BenchmarkGalois128K-64 119562 10028 ns/op 13070.78 MB/s
-BenchmarkGalois1M-64 14380 83424 ns/op 12569.22 MB/s
-BenchmarkGaloisXor128K-64 96508 12432 ns/op 10543.29 MB/s
-BenchmarkGaloisXor1M-64 10000 100322 ns/op 10452.13 MB/s
-```
-
-# Performance on ppc64le
-
-The performance for ppc64le has been accelerated.
-This gives roughly a 10x performance improvement on this architecture as can be seen below:
-
-```
-benchmark old MB/s new MB/s speedup
-BenchmarkGalois128K-160 948.87 8878.85 9.36x
-BenchmarkGalois1M-160 968.85 9041.92 9.33x
-BenchmarkGaloisXor128K-160 862.02 7905.00 9.17x
-BenchmarkGaloisXor1M-160 784.60 6296.65 8.03x
-```
-
-
-# Links
-* [Backblaze Open Sources Reed-Solomon Erasure Coding Source Code](https://www.backblaze.com/blog/reed-solomon/).
-* [JavaReedSolomon](https://github.com/Backblaze/JavaReedSolomon). Compatible java library by Backblaze.
-* [ocaml-reed-solomon-erasure](https://gitlab.com/darrenldl/ocaml-reed-solomon-erasure). Compatible OCaml implementation.
-* [reedsolomon-c](https://github.com/jannson/reedsolomon-c). C version, compatible with output from this package.
-* [Reed-Solomon Erasure Coding in Haskell](https://github.com/NicolasT/reedsolomon). Haskell port of the package with similar performance.
-* [reed-solomon-erasure](https://github.com/darrenldl/reed-solomon-erasure). Compatible Rust implementation.
-* [go-erasure](https://github.com/somethingnew2-0/go-erasure). A similar library using cgo, slower in my tests.
-* [Screaming Fast Galois Field Arithmetic](http://www.snia.org/sites/default/files2/SDC2013/presentations/NewThinking/EthanMiller_Screaming_Fast_Galois_Field%20Arithmetic_SIMD%20Instructions.pdf). Basis for SSE3 optimizations.
-* [Leopard-RS](https://github.com/catid/leopard) C library used as basis for GF16 implementation.
-
-# License
-
-This code, as the original [JavaReedSolomon](https://github.com/Backblaze/JavaReedSolomon) is published under an MIT license. See LICENSE file for more information.
diff --git a/vendor/github.com/klauspost/reedsolomon/galois.go b/vendor/github.com/klauspost/reedsolomon/galois.go
deleted file mode 100644
index 703f209..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois.go
+++ /dev/null
@@ -1,954 +0,0 @@
-/**
- * 8-bit Galois Field
- * Copyright 2015, Klaus Post
- * Copyright 2015, Backblaze, Inc. All rights reserved.
- */
-
-package reedsolomon
-
-import "encoding/binary"
-
-const (
- // The number of elements in the field.
- fieldSize = 256
-
- // The polynomial used to generate the logarithm table.
- //
- // There are a number of polynomials that work to generate
- // a Galois field of 256 elements. The choice is arbitrary,
- // and we just use the first one.
- //
- // The possibilities are: 29, 43, 45, 77, 95, 99, 101, 105,
- //* 113, 135, 141, 169, 195, 207, 231, and 245.
- generatingPolynomial = 29
-)
-
-var logTable = [fieldSize]byte{
- 0, 0, 1, 25, 2, 50, 26, 198,
- 3, 223, 51, 238, 27, 104, 199, 75,
- 4, 100, 224, 14, 52, 141, 239, 129,
- 28, 193, 105, 248, 200, 8, 76, 113,
- 5, 138, 101, 47, 225, 36, 15, 33,
- 53, 147, 142, 218, 240, 18, 130, 69,
- 29, 181, 194, 125, 106, 39, 249, 185,
- 201, 154, 9, 120, 77, 228, 114, 166,
- 6, 191, 139, 98, 102, 221, 48, 253,
- 226, 152, 37, 179, 16, 145, 34, 136,
- 54, 208, 148, 206, 143, 150, 219, 189,
- 241, 210, 19, 92, 131, 56, 70, 64,
- 30, 66, 182, 163, 195, 72, 126, 110,
- 107, 58, 40, 84, 250, 133, 186, 61,
- 202, 94, 155, 159, 10, 21, 121, 43,
- 78, 212, 229, 172, 115, 243, 167, 87,
- 7, 112, 192, 247, 140, 128, 99, 13,
- 103, 74, 222, 237, 49, 197, 254, 24,
- 227, 165, 153, 119, 38, 184, 180, 124,
- 17, 68, 146, 217, 35, 32, 137, 46,
- 55, 63, 209, 91, 149, 188, 207, 205,
- 144, 135, 151, 178, 220, 252, 190, 97,
- 242, 86, 211, 171, 20, 42, 93, 158,
- 132, 60, 57, 83, 71, 109, 65, 162,
- 31, 45, 67, 216, 183, 123, 164, 118,
- 196, 23, 73, 236, 127, 12, 111, 246,
- 108, 161, 59, 82, 41, 157, 85, 170,
- 251, 96, 134, 177, 187, 204, 62, 90,
- 203, 89, 95, 176, 156, 169, 160, 81,
- 11, 245, 22, 235, 122, 117, 44, 215,
- 79, 174, 213, 233, 230, 231, 173, 232,
- 116, 214, 244, 234, 168, 80, 88, 175,
-}
-
-/**
- * Inverse of the logarithm table. Maps integer logarithms
- * to members of the field. There is no entry for 255
- * because the highest log is 254.
- *
- * This table was generated by `go run gentables.go`
- */
-var expTable = []byte{0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80, 0x1d, 0x3a, 0x74, 0xe8, 0xcd, 0x87, 0x13, 0x26, 0x4c, 0x98, 0x2d, 0x5a, 0xb4, 0x75, 0xea, 0xc9, 0x8f, 0x3, 0x6, 0xc, 0x18, 0x30, 0x60, 0xc0, 0x9d, 0x27, 0x4e, 0x9c, 0x25, 0x4a, 0x94, 0x35, 0x6a, 0xd4, 0xb5, 0x77, 0xee, 0xc1, 0x9f, 0x23, 0x46, 0x8c, 0x5, 0xa, 0x14, 0x28, 0x50, 0xa0, 0x5d, 0xba, 0x69, 0xd2, 0xb9, 0x6f, 0xde, 0xa1, 0x5f, 0xbe, 0x61, 0xc2, 0x99, 0x2f, 0x5e, 0xbc, 0x65, 0xca, 0x89, 0xf, 0x1e, 0x3c, 0x78, 0xf0, 0xfd, 0xe7, 0xd3, 0xbb, 0x6b, 0xd6, 0xb1, 0x7f, 0xfe, 0xe1, 0xdf, 0xa3, 0x5b, 0xb6, 0x71, 0xe2, 0xd9, 0xaf, 0x43, 0x86, 0x11, 0x22, 0x44, 0x88, 0xd, 0x1a, 0x34, 0x68, 0xd0, 0xbd, 0x67, 0xce, 0x81, 0x1f, 0x3e, 0x7c, 0xf8, 0xed, 0xc7, 0x93, 0x3b, 0x76, 0xec, 0xc5, 0x97, 0x33, 0x66, 0xcc, 0x85, 0x17, 0x2e, 0x5c, 0xb8, 0x6d, 0xda, 0xa9, 0x4f, 0x9e, 0x21, 0x42, 0x84, 0x15, 0x2a, 0x54, 0xa8, 0x4d, 0x9a, 0x29, 0x52, 0xa4, 0x55, 0xaa, 0x49, 0x92, 0x39, 0x72, 0xe4, 0xd5, 0xb7, 0x73, 0xe6, 0xd1, 0xbf, 0x63, 0xc6, 0x91, 0x3f, 0x7e, 0xfc, 0xe5, 0xd7, 0xb3, 0x7b, 0xf6, 0xf1, 0xff, 0xe3, 0xdb, 0xab, 0x4b, 0x96, 0x31, 0x62, 0xc4, 0x95, 0x37, 0x6e, 0xdc, 0xa5, 0x57, 0xae, 0x41, 0x82, 0x19, 0x32, 0x64, 0xc8, 0x8d, 0x7, 0xe, 0x1c, 0x38, 0x70, 0xe0, 0xdd, 0xa7, 0x53, 0xa6, 0x51, 0xa2, 0x59, 0xb2, 0x79, 0xf2, 0xf9, 0xef, 0xc3, 0x9b, 0x2b, 0x56, 0xac, 0x45, 0x8a, 0x9, 0x12, 0x24, 0x48, 0x90, 0x3d, 0x7a, 0xf4, 0xf5, 0xf7, 0xf3, 0xfb, 0xeb, 0xcb, 0x8b, 0xb, 0x16, 0x2c, 0x58, 0xb0, 0x7d, 0xfa, 0xe9, 0xcf, 0x83, 0x1b, 0x36, 0x6c, 0xd8, 0xad, 0x47, 0x8e, 0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80, 0x1d, 0x3a, 0x74, 0xe8, 0xcd, 0x87, 0x13, 0x26, 0x4c, 0x98, 0x2d, 0x5a, 0xb4, 0x75, 0xea, 0xc9, 0x8f, 0x3, 0x6, 0xc, 0x18, 0x30, 0x60, 0xc0, 0x9d, 0x27, 0x4e, 0x9c, 0x25, 0x4a, 0x94, 0x35, 0x6a, 0xd4, 0xb5, 0x77, 0xee, 0xc1, 0x9f, 0x23, 0x46, 0x8c, 0x5, 0xa, 0x14, 0x28, 0x50, 0xa0, 0x5d, 0xba, 0x69, 0xd2, 0xb9, 0x6f, 0xde, 0xa1, 0x5f, 0xbe, 0x61, 0xc2, 0x99, 0x2f, 0x5e, 0xbc, 0x65, 0xca, 0x89, 0xf, 0x1e, 0x3c, 0x78, 0xf0, 0xfd, 0xe7, 0xd3, 0xbb, 0x6b, 0xd6, 0xb1, 0x7f, 0xfe, 0xe1, 0xdf, 0xa3, 0x5b, 0xb6, 0x71, 0xe2, 0xd9, 0xaf, 0x43, 0x86, 0x11, 0x22, 0x44, 0x88, 0xd, 0x1a, 0x34, 0x68, 0xd0, 0xbd, 0x67, 0xce, 0x81, 0x1f, 0x3e, 0x7c, 0xf8, 0xed, 0xc7, 0x93, 0x3b, 0x76, 0xec, 0xc5, 0x97, 0x33, 0x66, 0xcc, 0x85, 0x17, 0x2e, 0x5c, 0xb8, 0x6d, 0xda, 0xa9, 0x4f, 0x9e, 0x21, 0x42, 0x84, 0x15, 0x2a, 0x54, 0xa8, 0x4d, 0x9a, 0x29, 0x52, 0xa4, 0x55, 0xaa, 0x49, 0x92, 0x39, 0x72, 0xe4, 0xd5, 0xb7, 0x73, 0xe6, 0xd1, 0xbf, 0x63, 0xc6, 0x91, 0x3f, 0x7e, 0xfc, 0xe5, 0xd7, 0xb3, 0x7b, 0xf6, 0xf1, 0xff, 0xe3, 0xdb, 0xab, 0x4b, 0x96, 0x31, 0x62, 0xc4, 0x95, 0x37, 0x6e, 0xdc, 0xa5, 0x57, 0xae, 0x41, 0x82, 0x19, 0x32, 0x64, 0xc8, 0x8d, 0x7, 0xe, 0x1c, 0x38, 0x70, 0xe0, 0xdd, 0xa7, 0x53, 0xa6, 0x51, 0xa2, 0x59, 0xb2, 0x79, 0xf2, 0xf9, 0xef, 0xc3, 0x9b, 0x2b, 0x56, 0xac, 0x45, 0x8a, 0x9, 0x12, 0x24, 0x48, 0x90, 0x3d, 0x7a, 0xf4, 0xf5, 0xf7, 0xf3, 0xfb, 0xeb, 0xcb, 0x8b, 0xb, 0x16, 0x2c, 0x58, 0xb0, 0x7d, 0xfa, 0xe9, 0xcf, 0x83, 0x1b, 0x36, 0x6c, 0xd8, 0xad, 0x47, 0x8e}
-
-func galAdd(a, b byte) byte {
- return a ^ b
-}
-
-func galSub(a, b byte) byte {
- return a ^ b
-}
-
-// Table from https://github.com/templexxx/reedsolomon
-var invTable = [256]byte{0x0, 0x1, 0x8e, 0xf4, 0x47, 0xa7, 0x7a, 0xba, 0xad, 0x9d, 0xdd, 0x98, 0x3d, 0xaa, 0x5d, 0x96, 0xd8, 0x72, 0xc0, 0x58, 0xe0, 0x3e, 0x4c, 0x66, 0x90, 0xde, 0x55, 0x80, 0xa0, 0x83, 0x4b, 0x2a, 0x6c, 0xed, 0x39, 0x51, 0x60, 0x56, 0x2c, 0x8a, 0x70, 0xd0, 0x1f, 0x4a, 0x26, 0x8b, 0x33, 0x6e, 0x48, 0x89, 0x6f, 0x2e, 0xa4, 0xc3, 0x40, 0x5e, 0x50, 0x22, 0xcf, 0xa9, 0xab, 0xc, 0x15, 0xe1, 0x36, 0x5f, 0xf8, 0xd5, 0x92, 0x4e, 0xa6, 0x4, 0x30, 0x88, 0x2b, 0x1e, 0x16, 0x67, 0x45, 0x93, 0x38, 0x23, 0x68, 0x8c, 0x81, 0x1a, 0x25, 0x61, 0x13, 0xc1, 0xcb, 0x63, 0x97, 0xe, 0x37, 0x41, 0x24, 0x57, 0xca, 0x5b, 0xb9, 0xc4, 0x17, 0x4d, 0x52, 0x8d, 0xef, 0xb3, 0x20, 0xec, 0x2f, 0x32, 0x28, 0xd1, 0x11, 0xd9, 0xe9, 0xfb, 0xda, 0x79, 0xdb, 0x77, 0x6, 0xbb, 0x84, 0xcd, 0xfe, 0xfc, 0x1b, 0x54, 0xa1, 0x1d, 0x7c, 0xcc, 0xe4, 0xb0, 0x49, 0x31, 0x27, 0x2d, 0x53, 0x69, 0x2, 0xf5, 0x18, 0xdf, 0x44, 0x4f, 0x9b, 0xbc, 0xf, 0x5c, 0xb, 0xdc, 0xbd, 0x94, 0xac, 0x9, 0xc7, 0xa2, 0x1c, 0x82, 0x9f, 0xc6, 0x34, 0xc2, 0x46, 0x5, 0xce, 0x3b, 0xd, 0x3c, 0x9c, 0x8, 0xbe, 0xb7, 0x87, 0xe5, 0xee, 0x6b, 0xeb, 0xf2, 0xbf, 0xaf, 0xc5, 0x64, 0x7, 0x7b, 0x95, 0x9a, 0xae, 0xb6, 0x12, 0x59, 0xa5, 0x35, 0x65, 0xb8, 0xa3, 0x9e, 0xd2, 0xf7, 0x62, 0x5a, 0x85, 0x7d, 0xa8, 0x3a, 0x29, 0x71, 0xc8, 0xf6, 0xf9, 0x43, 0xd7, 0xd6, 0x10, 0x73, 0x76, 0x78, 0x99, 0xa, 0x19, 0x91, 0x14, 0x3f, 0xe6, 0xf0, 0x86, 0xb1, 0xe2, 0xf1, 0xfa, 0x74, 0xf3, 0xb4, 0x6d, 0x21, 0xb2, 0x6a, 0xe3, 0xe7, 0xb5, 0xea, 0x3, 0x8f, 0xd3, 0xc9, 0x42, 0xd4, 0xe8, 0x75, 0x7f, 0xff, 0x7e, 0xfd}
-
-var mulTable = [256][256]uint8{{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
- {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f, 0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f, 0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f, 0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f, 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f, 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f, 0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7, 0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf, 0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf, 0xc0, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xd0, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf, 0xe0, 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef, 0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff},
- {0x0, 0x2, 0x4, 0x6, 0x8, 0xa, 0xc, 0xe, 0x10, 0x12, 0x14, 0x16, 0x18, 0x1a, 0x1c, 0x1e, 0x20, 0x22, 0x24, 0x26, 0x28, 0x2a, 0x2c, 0x2e, 0x30, 0x32, 0x34, 0x36, 0x38, 0x3a, 0x3c, 0x3e, 0x40, 0x42, 0x44, 0x46, 0x48, 0x4a, 0x4c, 0x4e, 0x50, 0x52, 0x54, 0x56, 0x58, 0x5a, 0x5c, 0x5e, 0x60, 0x62, 0x64, 0x66, 0x68, 0x6a, 0x6c, 0x6e, 0x70, 0x72, 0x74, 0x76, 0x78, 0x7a, 0x7c, 0x7e, 0x80, 0x82, 0x84, 0x86, 0x88, 0x8a, 0x8c, 0x8e, 0x90, 0x92, 0x94, 0x96, 0x98, 0x9a, 0x9c, 0x9e, 0xa0, 0xa2, 0xa4, 0xa6, 0xa8, 0xaa, 0xac, 0xae, 0xb0, 0xb2, 0xb4, 0xb6, 0xb8, 0xba, 0xbc, 0xbe, 0xc0, 0xc2, 0xc4, 0xc6, 0xc8, 0xca, 0xcc, 0xce, 0xd0, 0xd2, 0xd4, 0xd6, 0xd8, 0xda, 0xdc, 0xde, 0xe0, 0xe2, 0xe4, 0xe6, 0xe8, 0xea, 0xec, 0xee, 0xf0, 0xf2, 0xf4, 0xf6, 0xf8, 0xfa, 0xfc, 0xfe, 0x1d, 0x1f, 0x19, 0x1b, 0x15, 0x17, 0x11, 0x13, 0xd, 0xf, 0x9, 0xb, 0x5, 0x7, 0x1, 0x3, 0x3d, 0x3f, 0x39, 0x3b, 0x35, 0x37, 0x31, 0x33, 0x2d, 0x2f, 0x29, 0x2b, 0x25, 0x27, 0x21, 0x23, 0x5d, 0x5f, 0x59, 0x5b, 0x55, 0x57, 0x51, 0x53, 0x4d, 0x4f, 0x49, 0x4b, 0x45, 0x47, 0x41, 0x43, 0x7d, 0x7f, 0x79, 0x7b, 0x75, 0x77, 0x71, 0x73, 0x6d, 0x6f, 0x69, 0x6b, 0x65, 0x67, 0x61, 0x63, 0x9d, 0x9f, 0x99, 0x9b, 0x95, 0x97, 0x91, 0x93, 0x8d, 0x8f, 0x89, 0x8b, 0x85, 0x87, 0x81, 0x83, 0xbd, 0xbf, 0xb9, 0xbb, 0xb5, 0xb7, 0xb1, 0xb3, 0xad, 0xaf, 0xa9, 0xab, 0xa5, 0xa7, 0xa1, 0xa3, 0xdd, 0xdf, 0xd9, 0xdb, 0xd5, 0xd7, 0xd1, 0xd3, 0xcd, 0xcf, 0xc9, 0xcb, 0xc5, 0xc7, 0xc1, 0xc3, 0xfd, 0xff, 0xf9, 0xfb, 0xf5, 0xf7, 0xf1, 0xf3, 0xed, 0xef, 0xe9, 0xeb, 0xe5, 0xe7, 0xe1, 0xe3},
- {0x0, 0x3, 0x6, 0x5, 0xc, 0xf, 0xa, 0x9, 0x18, 0x1b, 0x1e, 0x1d, 0x14, 0x17, 0x12, 0x11, 0x30, 0x33, 0x36, 0x35, 0x3c, 0x3f, 0x3a, 0x39, 0x28, 0x2b, 0x2e, 0x2d, 0x24, 0x27, 0x22, 0x21, 0x60, 0x63, 0x66, 0x65, 0x6c, 0x6f, 0x6a, 0x69, 0x78, 0x7b, 0x7e, 0x7d, 0x74, 0x77, 0x72, 0x71, 0x50, 0x53, 0x56, 0x55, 0x5c, 0x5f, 0x5a, 0x59, 0x48, 0x4b, 0x4e, 0x4d, 0x44, 0x47, 0x42, 0x41, 0xc0, 0xc3, 0xc6, 0xc5, 0xcc, 0xcf, 0xca, 0xc9, 0xd8, 0xdb, 0xde, 0xdd, 0xd4, 0xd7, 0xd2, 0xd1, 0xf0, 0xf3, 0xf6, 0xf5, 0xfc, 0xff, 0xfa, 0xf9, 0xe8, 0xeb, 0xee, 0xed, 0xe4, 0xe7, 0xe2, 0xe1, 0xa0, 0xa3, 0xa6, 0xa5, 0xac, 0xaf, 0xaa, 0xa9, 0xb8, 0xbb, 0xbe, 0xbd, 0xb4, 0xb7, 0xb2, 0xb1, 0x90, 0x93, 0x96, 0x95, 0x9c, 0x9f, 0x9a, 0x99, 0x88, 0x8b, 0x8e, 0x8d, 0x84, 0x87, 0x82, 0x81, 0x9d, 0x9e, 0x9b, 0x98, 0x91, 0x92, 0x97, 0x94, 0x85, 0x86, 0x83, 0x80, 0x89, 0x8a, 0x8f, 0x8c, 0xad, 0xae, 0xab, 0xa8, 0xa1, 0xa2, 0xa7, 0xa4, 0xb5, 0xb6, 0xb3, 0xb0, 0xb9, 0xba, 0xbf, 0xbc, 0xfd, 0xfe, 0xfb, 0xf8, 0xf1, 0xf2, 0xf7, 0xf4, 0xe5, 0xe6, 0xe3, 0xe0, 0xe9, 0xea, 0xef, 0xec, 0xcd, 0xce, 0xcb, 0xc8, 0xc1, 0xc2, 0xc7, 0xc4, 0xd5, 0xd6, 0xd3, 0xd0, 0xd9, 0xda, 0xdf, 0xdc, 0x5d, 0x5e, 0x5b, 0x58, 0x51, 0x52, 0x57, 0x54, 0x45, 0x46, 0x43, 0x40, 0x49, 0x4a, 0x4f, 0x4c, 0x6d, 0x6e, 0x6b, 0x68, 0x61, 0x62, 0x67, 0x64, 0x75, 0x76, 0x73, 0x70, 0x79, 0x7a, 0x7f, 0x7c, 0x3d, 0x3e, 0x3b, 0x38, 0x31, 0x32, 0x37, 0x34, 0x25, 0x26, 0x23, 0x20, 0x29, 0x2a, 0x2f, 0x2c, 0xd, 0xe, 0xb, 0x8, 0x1, 0x2, 0x7, 0x4, 0x15, 0x16, 0x13, 0x10, 0x19, 0x1a, 0x1f, 0x1c},
- {0x0, 0x4, 0x8, 0xc, 0x10, 0x14, 0x18, 0x1c, 0x20, 0x24, 0x28, 0x2c, 0x30, 0x34, 0x38, 0x3c, 0x40, 0x44, 0x48, 0x4c, 0x50, 0x54, 0x58, 0x5c, 0x60, 0x64, 0x68, 0x6c, 0x70, 0x74, 0x78, 0x7c, 0x80, 0x84, 0x88, 0x8c, 0x90, 0x94, 0x98, 0x9c, 0xa0, 0xa4, 0xa8, 0xac, 0xb0, 0xb4, 0xb8, 0xbc, 0xc0, 0xc4, 0xc8, 0xcc, 0xd0, 0xd4, 0xd8, 0xdc, 0xe0, 0xe4, 0xe8, 0xec, 0xf0, 0xf4, 0xf8, 0xfc, 0x1d, 0x19, 0x15, 0x11, 0xd, 0x9, 0x5, 0x1, 0x3d, 0x39, 0x35, 0x31, 0x2d, 0x29, 0x25, 0x21, 0x5d, 0x59, 0x55, 0x51, 0x4d, 0x49, 0x45, 0x41, 0x7d, 0x79, 0x75, 0x71, 0x6d, 0x69, 0x65, 0x61, 0x9d, 0x99, 0x95, 0x91, 0x8d, 0x89, 0x85, 0x81, 0xbd, 0xb9, 0xb5, 0xb1, 0xad, 0xa9, 0xa5, 0xa1, 0xdd, 0xd9, 0xd5, 0xd1, 0xcd, 0xc9, 0xc5, 0xc1, 0xfd, 0xf9, 0xf5, 0xf1, 0xed, 0xe9, 0xe5, 0xe1, 0x3a, 0x3e, 0x32, 0x36, 0x2a, 0x2e, 0x22, 0x26, 0x1a, 0x1e, 0x12, 0x16, 0xa, 0xe, 0x2, 0x6, 0x7a, 0x7e, 0x72, 0x76, 0x6a, 0x6e, 0x62, 0x66, 0x5a, 0x5e, 0x52, 0x56, 0x4a, 0x4e, 0x42, 0x46, 0xba, 0xbe, 0xb2, 0xb6, 0xaa, 0xae, 0xa2, 0xa6, 0x9a, 0x9e, 0x92, 0x96, 0x8a, 0x8e, 0x82, 0x86, 0xfa, 0xfe, 0xf2, 0xf6, 0xea, 0xee, 0xe2, 0xe6, 0xda, 0xde, 0xd2, 0xd6, 0xca, 0xce, 0xc2, 0xc6, 0x27, 0x23, 0x2f, 0x2b, 0x37, 0x33, 0x3f, 0x3b, 0x7, 0x3, 0xf, 0xb, 0x17, 0x13, 0x1f, 0x1b, 0x67, 0x63, 0x6f, 0x6b, 0x77, 0x73, 0x7f, 0x7b, 0x47, 0x43, 0x4f, 0x4b, 0x57, 0x53, 0x5f, 0x5b, 0xa7, 0xa3, 0xaf, 0xab, 0xb7, 0xb3, 0xbf, 0xbb, 0x87, 0x83, 0x8f, 0x8b, 0x97, 0x93, 0x9f, 0x9b, 0xe7, 0xe3, 0xef, 0xeb, 0xf7, 0xf3, 0xff, 0xfb, 0xc7, 0xc3, 0xcf, 0xcb, 0xd7, 0xd3, 0xdf, 0xdb},
- {0x0, 0x5, 0xa, 0xf, 0x14, 0x11, 0x1e, 0x1b, 0x28, 0x2d, 0x22, 0x27, 0x3c, 0x39, 0x36, 0x33, 0x50, 0x55, 0x5a, 0x5f, 0x44, 0x41, 0x4e, 0x4b, 0x78, 0x7d, 0x72, 0x77, 0x6c, 0x69, 0x66, 0x63, 0xa0, 0xa5, 0xaa, 0xaf, 0xb4, 0xb1, 0xbe, 0xbb, 0x88, 0x8d, 0x82, 0x87, 0x9c, 0x99, 0x96, 0x93, 0xf0, 0xf5, 0xfa, 0xff, 0xe4, 0xe1, 0xee, 0xeb, 0xd8, 0xdd, 0xd2, 0xd7, 0xcc, 0xc9, 0xc6, 0xc3, 0x5d, 0x58, 0x57, 0x52, 0x49, 0x4c, 0x43, 0x46, 0x75, 0x70, 0x7f, 0x7a, 0x61, 0x64, 0x6b, 0x6e, 0xd, 0x8, 0x7, 0x2, 0x19, 0x1c, 0x13, 0x16, 0x25, 0x20, 0x2f, 0x2a, 0x31, 0x34, 0x3b, 0x3e, 0xfd, 0xf8, 0xf7, 0xf2, 0xe9, 0xec, 0xe3, 0xe6, 0xd5, 0xd0, 0xdf, 0xda, 0xc1, 0xc4, 0xcb, 0xce, 0xad, 0xa8, 0xa7, 0xa2, 0xb9, 0xbc, 0xb3, 0xb6, 0x85, 0x80, 0x8f, 0x8a, 0x91, 0x94, 0x9b, 0x9e, 0xba, 0xbf, 0xb0, 0xb5, 0xae, 0xab, 0xa4, 0xa1, 0x92, 0x97, 0x98, 0x9d, 0x86, 0x83, 0x8c, 0x89, 0xea, 0xef, 0xe0, 0xe5, 0xfe, 0xfb, 0xf4, 0xf1, 0xc2, 0xc7, 0xc8, 0xcd, 0xd6, 0xd3, 0xdc, 0xd9, 0x1a, 0x1f, 0x10, 0x15, 0xe, 0xb, 0x4, 0x1, 0x32, 0x37, 0x38, 0x3d, 0x26, 0x23, 0x2c, 0x29, 0x4a, 0x4f, 0x40, 0x45, 0x5e, 0x5b, 0x54, 0x51, 0x62, 0x67, 0x68, 0x6d, 0x76, 0x73, 0x7c, 0x79, 0xe7, 0xe2, 0xed, 0xe8, 0xf3, 0xf6, 0xf9, 0xfc, 0xcf, 0xca, 0xc5, 0xc0, 0xdb, 0xde, 0xd1, 0xd4, 0xb7, 0xb2, 0xbd, 0xb8, 0xa3, 0xa6, 0xa9, 0xac, 0x9f, 0x9a, 0x95, 0x90, 0x8b, 0x8e, 0x81, 0x84, 0x47, 0x42, 0x4d, 0x48, 0x53, 0x56, 0x59, 0x5c, 0x6f, 0x6a, 0x65, 0x60, 0x7b, 0x7e, 0x71, 0x74, 0x17, 0x12, 0x1d, 0x18, 0x3, 0x6, 0x9, 0xc, 0x3f, 0x3a, 0x35, 0x30, 0x2b, 0x2e, 0x21, 0x24},
- {0x0, 0x6, 0xc, 0xa, 0x18, 0x1e, 0x14, 0x12, 0x30, 0x36, 0x3c, 0x3a, 0x28, 0x2e, 0x24, 0x22, 0x60, 0x66, 0x6c, 0x6a, 0x78, 0x7e, 0x74, 0x72, 0x50, 0x56, 0x5c, 0x5a, 0x48, 0x4e, 0x44, 0x42, 0xc0, 0xc6, 0xcc, 0xca, 0xd8, 0xde, 0xd4, 0xd2, 0xf0, 0xf6, 0xfc, 0xfa, 0xe8, 0xee, 0xe4, 0xe2, 0xa0, 0xa6, 0xac, 0xaa, 0xb8, 0xbe, 0xb4, 0xb2, 0x90, 0x96, 0x9c, 0x9a, 0x88, 0x8e, 0x84, 0x82, 0x9d, 0x9b, 0x91, 0x97, 0x85, 0x83, 0x89, 0x8f, 0xad, 0xab, 0xa1, 0xa7, 0xb5, 0xb3, 0xb9, 0xbf, 0xfd, 0xfb, 0xf1, 0xf7, 0xe5, 0xe3, 0xe9, 0xef, 0xcd, 0xcb, 0xc1, 0xc7, 0xd5, 0xd3, 0xd9, 0xdf, 0x5d, 0x5b, 0x51, 0x57, 0x45, 0x43, 0x49, 0x4f, 0x6d, 0x6b, 0x61, 0x67, 0x75, 0x73, 0x79, 0x7f, 0x3d, 0x3b, 0x31, 0x37, 0x25, 0x23, 0x29, 0x2f, 0xd, 0xb, 0x1, 0x7, 0x15, 0x13, 0x19, 0x1f, 0x27, 0x21, 0x2b, 0x2d, 0x3f, 0x39, 0x33, 0x35, 0x17, 0x11, 0x1b, 0x1d, 0xf, 0x9, 0x3, 0x5, 0x47, 0x41, 0x4b, 0x4d, 0x5f, 0x59, 0x53, 0x55, 0x77, 0x71, 0x7b, 0x7d, 0x6f, 0x69, 0x63, 0x65, 0xe7, 0xe1, 0xeb, 0xed, 0xff, 0xf9, 0xf3, 0xf5, 0xd7, 0xd1, 0xdb, 0xdd, 0xcf, 0xc9, 0xc3, 0xc5, 0x87, 0x81, 0x8b, 0x8d, 0x9f, 0x99, 0x93, 0x95, 0xb7, 0xb1, 0xbb, 0xbd, 0xaf, 0xa9, 0xa3, 0xa5, 0xba, 0xbc, 0xb6, 0xb0, 0xa2, 0xa4, 0xae, 0xa8, 0x8a, 0x8c, 0x86, 0x80, 0x92, 0x94, 0x9e, 0x98, 0xda, 0xdc, 0xd6, 0xd0, 0xc2, 0xc4, 0xce, 0xc8, 0xea, 0xec, 0xe6, 0xe0, 0xf2, 0xf4, 0xfe, 0xf8, 0x7a, 0x7c, 0x76, 0x70, 0x62, 0x64, 0x6e, 0x68, 0x4a, 0x4c, 0x46, 0x40, 0x52, 0x54, 0x5e, 0x58, 0x1a, 0x1c, 0x16, 0x10, 0x2, 0x4, 0xe, 0x8, 0x2a, 0x2c, 0x26, 0x20, 0x32, 0x34, 0x3e, 0x38},
- {0x0, 0x7, 0xe, 0x9, 0x1c, 0x1b, 0x12, 0x15, 0x38, 0x3f, 0x36, 0x31, 0x24, 0x23, 0x2a, 0x2d, 0x70, 0x77, 0x7e, 0x79, 0x6c, 0x6b, 0x62, 0x65, 0x48, 0x4f, 0x46, 0x41, 0x54, 0x53, 0x5a, 0x5d, 0xe0, 0xe7, 0xee, 0xe9, 0xfc, 0xfb, 0xf2, 0xf5, 0xd8, 0xdf, 0xd6, 0xd1, 0xc4, 0xc3, 0xca, 0xcd, 0x90, 0x97, 0x9e, 0x99, 0x8c, 0x8b, 0x82, 0x85, 0xa8, 0xaf, 0xa6, 0xa1, 0xb4, 0xb3, 0xba, 0xbd, 0xdd, 0xda, 0xd3, 0xd4, 0xc1, 0xc6, 0xcf, 0xc8, 0xe5, 0xe2, 0xeb, 0xec, 0xf9, 0xfe, 0xf7, 0xf0, 0xad, 0xaa, 0xa3, 0xa4, 0xb1, 0xb6, 0xbf, 0xb8, 0x95, 0x92, 0x9b, 0x9c, 0x89, 0x8e, 0x87, 0x80, 0x3d, 0x3a, 0x33, 0x34, 0x21, 0x26, 0x2f, 0x28, 0x5, 0x2, 0xb, 0xc, 0x19, 0x1e, 0x17, 0x10, 0x4d, 0x4a, 0x43, 0x44, 0x51, 0x56, 0x5f, 0x58, 0x75, 0x72, 0x7b, 0x7c, 0x69, 0x6e, 0x67, 0x60, 0xa7, 0xa0, 0xa9, 0xae, 0xbb, 0xbc, 0xb5, 0xb2, 0x9f, 0x98, 0x91, 0x96, 0x83, 0x84, 0x8d, 0x8a, 0xd7, 0xd0, 0xd9, 0xde, 0xcb, 0xcc, 0xc5, 0xc2, 0xef, 0xe8, 0xe1, 0xe6, 0xf3, 0xf4, 0xfd, 0xfa, 0x47, 0x40, 0x49, 0x4e, 0x5b, 0x5c, 0x55, 0x52, 0x7f, 0x78, 0x71, 0x76, 0x63, 0x64, 0x6d, 0x6a, 0x37, 0x30, 0x39, 0x3e, 0x2b, 0x2c, 0x25, 0x22, 0xf, 0x8, 0x1, 0x6, 0x13, 0x14, 0x1d, 0x1a, 0x7a, 0x7d, 0x74, 0x73, 0x66, 0x61, 0x68, 0x6f, 0x42, 0x45, 0x4c, 0x4b, 0x5e, 0x59, 0x50, 0x57, 0xa, 0xd, 0x4, 0x3, 0x16, 0x11, 0x18, 0x1f, 0x32, 0x35, 0x3c, 0x3b, 0x2e, 0x29, 0x20, 0x27, 0x9a, 0x9d, 0x94, 0x93, 0x86, 0x81, 0x88, 0x8f, 0xa2, 0xa5, 0xac, 0xab, 0xbe, 0xb9, 0xb0, 0xb7, 0xea, 0xed, 0xe4, 0xe3, 0xf6, 0xf1, 0xf8, 0xff, 0xd2, 0xd5, 0xdc, 0xdb, 0xce, 0xc9, 0xc0, 0xc7},
- {0x0, 0x8, 0x10, 0x18, 0x20, 0x28, 0x30, 0x38, 0x40, 0x48, 0x50, 0x58, 0x60, 0x68, 0x70, 0x78, 0x80, 0x88, 0x90, 0x98, 0xa0, 0xa8, 0xb0, 0xb8, 0xc0, 0xc8, 0xd0, 0xd8, 0xe0, 0xe8, 0xf0, 0xf8, 0x1d, 0x15, 0xd, 0x5, 0x3d, 0x35, 0x2d, 0x25, 0x5d, 0x55, 0x4d, 0x45, 0x7d, 0x75, 0x6d, 0x65, 0x9d, 0x95, 0x8d, 0x85, 0xbd, 0xb5, 0xad, 0xa5, 0xdd, 0xd5, 0xcd, 0xc5, 0xfd, 0xf5, 0xed, 0xe5, 0x3a, 0x32, 0x2a, 0x22, 0x1a, 0x12, 0xa, 0x2, 0x7a, 0x72, 0x6a, 0x62, 0x5a, 0x52, 0x4a, 0x42, 0xba, 0xb2, 0xaa, 0xa2, 0x9a, 0x92, 0x8a, 0x82, 0xfa, 0xf2, 0xea, 0xe2, 0xda, 0xd2, 0xca, 0xc2, 0x27, 0x2f, 0x37, 0x3f, 0x7, 0xf, 0x17, 0x1f, 0x67, 0x6f, 0x77, 0x7f, 0x47, 0x4f, 0x57, 0x5f, 0xa7, 0xaf, 0xb7, 0xbf, 0x87, 0x8f, 0x97, 0x9f, 0xe7, 0xef, 0xf7, 0xff, 0xc7, 0xcf, 0xd7, 0xdf, 0x74, 0x7c, 0x64, 0x6c, 0x54, 0x5c, 0x44, 0x4c, 0x34, 0x3c, 0x24, 0x2c, 0x14, 0x1c, 0x4, 0xc, 0xf4, 0xfc, 0xe4, 0xec, 0xd4, 0xdc, 0xc4, 0xcc, 0xb4, 0xbc, 0xa4, 0xac, 0x94, 0x9c, 0x84, 0x8c, 0x69, 0x61, 0x79, 0x71, 0x49, 0x41, 0x59, 0x51, 0x29, 0x21, 0x39, 0x31, 0x9, 0x1, 0x19, 0x11, 0xe9, 0xe1, 0xf9, 0xf1, 0xc9, 0xc1, 0xd9, 0xd1, 0xa9, 0xa1, 0xb9, 0xb1, 0x89, 0x81, 0x99, 0x91, 0x4e, 0x46, 0x5e, 0x56, 0x6e, 0x66, 0x7e, 0x76, 0xe, 0x6, 0x1e, 0x16, 0x2e, 0x26, 0x3e, 0x36, 0xce, 0xc6, 0xde, 0xd6, 0xee, 0xe6, 0xfe, 0xf6, 0x8e, 0x86, 0x9e, 0x96, 0xae, 0xa6, 0xbe, 0xb6, 0x53, 0x5b, 0x43, 0x4b, 0x73, 0x7b, 0x63, 0x6b, 0x13, 0x1b, 0x3, 0xb, 0x33, 0x3b, 0x23, 0x2b, 0xd3, 0xdb, 0xc3, 0xcb, 0xf3, 0xfb, 0xe3, 0xeb, 0x93, 0x9b, 0x83, 0x8b, 0xb3, 0xbb, 0xa3, 0xab},
- {0x0, 0x9, 0x12, 0x1b, 0x24, 0x2d, 0x36, 0x3f, 0x48, 0x41, 0x5a, 0x53, 0x6c, 0x65, 0x7e, 0x77, 0x90, 0x99, 0x82, 0x8b, 0xb4, 0xbd, 0xa6, 0xaf, 0xd8, 0xd1, 0xca, 0xc3, 0xfc, 0xf5, 0xee, 0xe7, 0x3d, 0x34, 0x2f, 0x26, 0x19, 0x10, 0xb, 0x2, 0x75, 0x7c, 0x67, 0x6e, 0x51, 0x58, 0x43, 0x4a, 0xad, 0xa4, 0xbf, 0xb6, 0x89, 0x80, 0x9b, 0x92, 0xe5, 0xec, 0xf7, 0xfe, 0xc1, 0xc8, 0xd3, 0xda, 0x7a, 0x73, 0x68, 0x61, 0x5e, 0x57, 0x4c, 0x45, 0x32, 0x3b, 0x20, 0x29, 0x16, 0x1f, 0x4, 0xd, 0xea, 0xe3, 0xf8, 0xf1, 0xce, 0xc7, 0xdc, 0xd5, 0xa2, 0xab, 0xb0, 0xb9, 0x86, 0x8f, 0x94, 0x9d, 0x47, 0x4e, 0x55, 0x5c, 0x63, 0x6a, 0x71, 0x78, 0xf, 0x6, 0x1d, 0x14, 0x2b, 0x22, 0x39, 0x30, 0xd7, 0xde, 0xc5, 0xcc, 0xf3, 0xfa, 0xe1, 0xe8, 0x9f, 0x96, 0x8d, 0x84, 0xbb, 0xb2, 0xa9, 0xa0, 0xf4, 0xfd, 0xe6, 0xef, 0xd0, 0xd9, 0xc2, 0xcb, 0xbc, 0xb5, 0xae, 0xa7, 0x98, 0x91, 0x8a, 0x83, 0x64, 0x6d, 0x76, 0x7f, 0x40, 0x49, 0x52, 0x5b, 0x2c, 0x25, 0x3e, 0x37, 0x8, 0x1, 0x1a, 0x13, 0xc9, 0xc0, 0xdb, 0xd2, 0xed, 0xe4, 0xff, 0xf6, 0x81, 0x88, 0x93, 0x9a, 0xa5, 0xac, 0xb7, 0xbe, 0x59, 0x50, 0x4b, 0x42, 0x7d, 0x74, 0x6f, 0x66, 0x11, 0x18, 0x3, 0xa, 0x35, 0x3c, 0x27, 0x2e, 0x8e, 0x87, 0x9c, 0x95, 0xaa, 0xa3, 0xb8, 0xb1, 0xc6, 0xcf, 0xd4, 0xdd, 0xe2, 0xeb, 0xf0, 0xf9, 0x1e, 0x17, 0xc, 0x5, 0x3a, 0x33, 0x28, 0x21, 0x56, 0x5f, 0x44, 0x4d, 0x72, 0x7b, 0x60, 0x69, 0xb3, 0xba, 0xa1, 0xa8, 0x97, 0x9e, 0x85, 0x8c, 0xfb, 0xf2, 0xe9, 0xe0, 0xdf, 0xd6, 0xcd, 0xc4, 0x23, 0x2a, 0x31, 0x38, 0x7, 0xe, 0x15, 0x1c, 0x6b, 0x62, 0x79, 0x70, 0x4f, 0x46, 0x5d, 0x54},
- {0x0, 0xa, 0x14, 0x1e, 0x28, 0x22, 0x3c, 0x36, 0x50, 0x5a, 0x44, 0x4e, 0x78, 0x72, 0x6c, 0x66, 0xa0, 0xaa, 0xb4, 0xbe, 0x88, 0x82, 0x9c, 0x96, 0xf0, 0xfa, 0xe4, 0xee, 0xd8, 0xd2, 0xcc, 0xc6, 0x5d, 0x57, 0x49, 0x43, 0x75, 0x7f, 0x61, 0x6b, 0xd, 0x7, 0x19, 0x13, 0x25, 0x2f, 0x31, 0x3b, 0xfd, 0xf7, 0xe9, 0xe3, 0xd5, 0xdf, 0xc1, 0xcb, 0xad, 0xa7, 0xb9, 0xb3, 0x85, 0x8f, 0x91, 0x9b, 0xba, 0xb0, 0xae, 0xa4, 0x92, 0x98, 0x86, 0x8c, 0xea, 0xe0, 0xfe, 0xf4, 0xc2, 0xc8, 0xd6, 0xdc, 0x1a, 0x10, 0xe, 0x4, 0x32, 0x38, 0x26, 0x2c, 0x4a, 0x40, 0x5e, 0x54, 0x62, 0x68, 0x76, 0x7c, 0xe7, 0xed, 0xf3, 0xf9, 0xcf, 0xc5, 0xdb, 0xd1, 0xb7, 0xbd, 0xa3, 0xa9, 0x9f, 0x95, 0x8b, 0x81, 0x47, 0x4d, 0x53, 0x59, 0x6f, 0x65, 0x7b, 0x71, 0x17, 0x1d, 0x3, 0x9, 0x3f, 0x35, 0x2b, 0x21, 0x69, 0x63, 0x7d, 0x77, 0x41, 0x4b, 0x55, 0x5f, 0x39, 0x33, 0x2d, 0x27, 0x11, 0x1b, 0x5, 0xf, 0xc9, 0xc3, 0xdd, 0xd7, 0xe1, 0xeb, 0xf5, 0xff, 0x99, 0x93, 0x8d, 0x87, 0xb1, 0xbb, 0xa5, 0xaf, 0x34, 0x3e, 0x20, 0x2a, 0x1c, 0x16, 0x8, 0x2, 0x64, 0x6e, 0x70, 0x7a, 0x4c, 0x46, 0x58, 0x52, 0x94, 0x9e, 0x80, 0x8a, 0xbc, 0xb6, 0xa8, 0xa2, 0xc4, 0xce, 0xd0, 0xda, 0xec, 0xe6, 0xf8, 0xf2, 0xd3, 0xd9, 0xc7, 0xcd, 0xfb, 0xf1, 0xef, 0xe5, 0x83, 0x89, 0x97, 0x9d, 0xab, 0xa1, 0xbf, 0xb5, 0x73, 0x79, 0x67, 0x6d, 0x5b, 0x51, 0x4f, 0x45, 0x23, 0x29, 0x37, 0x3d, 0xb, 0x1, 0x1f, 0x15, 0x8e, 0x84, 0x9a, 0x90, 0xa6, 0xac, 0xb2, 0xb8, 0xde, 0xd4, 0xca, 0xc0, 0xf6, 0xfc, 0xe2, 0xe8, 0x2e, 0x24, 0x3a, 0x30, 0x6, 0xc, 0x12, 0x18, 0x7e, 0x74, 0x6a, 0x60, 0x56, 0x5c, 0x42, 0x48},
- {0x0, 0xb, 0x16, 0x1d, 0x2c, 0x27, 0x3a, 0x31, 0x58, 0x53, 0x4e, 0x45, 0x74, 0x7f, 0x62, 0x69, 0xb0, 0xbb, 0xa6, 0xad, 0x9c, 0x97, 0x8a, 0x81, 0xe8, 0xe3, 0xfe, 0xf5, 0xc4, 0xcf, 0xd2, 0xd9, 0x7d, 0x76, 0x6b, 0x60, 0x51, 0x5a, 0x47, 0x4c, 0x25, 0x2e, 0x33, 0x38, 0x9, 0x2, 0x1f, 0x14, 0xcd, 0xc6, 0xdb, 0xd0, 0xe1, 0xea, 0xf7, 0xfc, 0x95, 0x9e, 0x83, 0x88, 0xb9, 0xb2, 0xaf, 0xa4, 0xfa, 0xf1, 0xec, 0xe7, 0xd6, 0xdd, 0xc0, 0xcb, 0xa2, 0xa9, 0xb4, 0xbf, 0x8e, 0x85, 0x98, 0x93, 0x4a, 0x41, 0x5c, 0x57, 0x66, 0x6d, 0x70, 0x7b, 0x12, 0x19, 0x4, 0xf, 0x3e, 0x35, 0x28, 0x23, 0x87, 0x8c, 0x91, 0x9a, 0xab, 0xa0, 0xbd, 0xb6, 0xdf, 0xd4, 0xc9, 0xc2, 0xf3, 0xf8, 0xe5, 0xee, 0x37, 0x3c, 0x21, 0x2a, 0x1b, 0x10, 0xd, 0x6, 0x6f, 0x64, 0x79, 0x72, 0x43, 0x48, 0x55, 0x5e, 0xe9, 0xe2, 0xff, 0xf4, 0xc5, 0xce, 0xd3, 0xd8, 0xb1, 0xba, 0xa7, 0xac, 0x9d, 0x96, 0x8b, 0x80, 0x59, 0x52, 0x4f, 0x44, 0x75, 0x7e, 0x63, 0x68, 0x1, 0xa, 0x17, 0x1c, 0x2d, 0x26, 0x3b, 0x30, 0x94, 0x9f, 0x82, 0x89, 0xb8, 0xb3, 0xae, 0xa5, 0xcc, 0xc7, 0xda, 0xd1, 0xe0, 0xeb, 0xf6, 0xfd, 0x24, 0x2f, 0x32, 0x39, 0x8, 0x3, 0x1e, 0x15, 0x7c, 0x77, 0x6a, 0x61, 0x50, 0x5b, 0x46, 0x4d, 0x13, 0x18, 0x5, 0xe, 0x3f, 0x34, 0x29, 0x22, 0x4b, 0x40, 0x5d, 0x56, 0x67, 0x6c, 0x71, 0x7a, 0xa3, 0xa8, 0xb5, 0xbe, 0x8f, 0x84, 0x99, 0x92, 0xfb, 0xf0, 0xed, 0xe6, 0xd7, 0xdc, 0xc1, 0xca, 0x6e, 0x65, 0x78, 0x73, 0x42, 0x49, 0x54, 0x5f, 0x36, 0x3d, 0x20, 0x2b, 0x1a, 0x11, 0xc, 0x7, 0xde, 0xd5, 0xc8, 0xc3, 0xf2, 0xf9, 0xe4, 0xef, 0x86, 0x8d, 0x90, 0x9b, 0xaa, 0xa1, 0xbc, 0xb7},
- {0x0, 0xc, 0x18, 0x14, 0x30, 0x3c, 0x28, 0x24, 0x60, 0x6c, 0x78, 0x74, 0x50, 0x5c, 0x48, 0x44, 0xc0, 0xcc, 0xd8, 0xd4, 0xf0, 0xfc, 0xe8, 0xe4, 0xa0, 0xac, 0xb8, 0xb4, 0x90, 0x9c, 0x88, 0x84, 0x9d, 0x91, 0x85, 0x89, 0xad, 0xa1, 0xb5, 0xb9, 0xfd, 0xf1, 0xe5, 0xe9, 0xcd, 0xc1, 0xd5, 0xd9, 0x5d, 0x51, 0x45, 0x49, 0x6d, 0x61, 0x75, 0x79, 0x3d, 0x31, 0x25, 0x29, 0xd, 0x1, 0x15, 0x19, 0x27, 0x2b, 0x3f, 0x33, 0x17, 0x1b, 0xf, 0x3, 0x47, 0x4b, 0x5f, 0x53, 0x77, 0x7b, 0x6f, 0x63, 0xe7, 0xeb, 0xff, 0xf3, 0xd7, 0xdb, 0xcf, 0xc3, 0x87, 0x8b, 0x9f, 0x93, 0xb7, 0xbb, 0xaf, 0xa3, 0xba, 0xb6, 0xa2, 0xae, 0x8a, 0x86, 0x92, 0x9e, 0xda, 0xd6, 0xc2, 0xce, 0xea, 0xe6, 0xf2, 0xfe, 0x7a, 0x76, 0x62, 0x6e, 0x4a, 0x46, 0x52, 0x5e, 0x1a, 0x16, 0x2, 0xe, 0x2a, 0x26, 0x32, 0x3e, 0x4e, 0x42, 0x56, 0x5a, 0x7e, 0x72, 0x66, 0x6a, 0x2e, 0x22, 0x36, 0x3a, 0x1e, 0x12, 0x6, 0xa, 0x8e, 0x82, 0x96, 0x9a, 0xbe, 0xb2, 0xa6, 0xaa, 0xee, 0xe2, 0xf6, 0xfa, 0xde, 0xd2, 0xc6, 0xca, 0xd3, 0xdf, 0xcb, 0xc7, 0xe3, 0xef, 0xfb, 0xf7, 0xb3, 0xbf, 0xab, 0xa7, 0x83, 0x8f, 0x9b, 0x97, 0x13, 0x1f, 0xb, 0x7, 0x23, 0x2f, 0x3b, 0x37, 0x73, 0x7f, 0x6b, 0x67, 0x43, 0x4f, 0x5b, 0x57, 0x69, 0x65, 0x71, 0x7d, 0x59, 0x55, 0x41, 0x4d, 0x9, 0x5, 0x11, 0x1d, 0x39, 0x35, 0x21, 0x2d, 0xa9, 0xa5, 0xb1, 0xbd, 0x99, 0x95, 0x81, 0x8d, 0xc9, 0xc5, 0xd1, 0xdd, 0xf9, 0xf5, 0xe1, 0xed, 0xf4, 0xf8, 0xec, 0xe0, 0xc4, 0xc8, 0xdc, 0xd0, 0x94, 0x98, 0x8c, 0x80, 0xa4, 0xa8, 0xbc, 0xb0, 0x34, 0x38, 0x2c, 0x20, 0x4, 0x8, 0x1c, 0x10, 0x54, 0x58, 0x4c, 0x40, 0x64, 0x68, 0x7c, 0x70},
- {0x0, 0xd, 0x1a, 0x17, 0x34, 0x39, 0x2e, 0x23, 0x68, 0x65, 0x72, 0x7f, 0x5c, 0x51, 0x46, 0x4b, 0xd0, 0xdd, 0xca, 0xc7, 0xe4, 0xe9, 0xfe, 0xf3, 0xb8, 0xb5, 0xa2, 0xaf, 0x8c, 0x81, 0x96, 0x9b, 0xbd, 0xb0, 0xa7, 0xaa, 0x89, 0x84, 0x93, 0x9e, 0xd5, 0xd8, 0xcf, 0xc2, 0xe1, 0xec, 0xfb, 0xf6, 0x6d, 0x60, 0x77, 0x7a, 0x59, 0x54, 0x43, 0x4e, 0x5, 0x8, 0x1f, 0x12, 0x31, 0x3c, 0x2b, 0x26, 0x67, 0x6a, 0x7d, 0x70, 0x53, 0x5e, 0x49, 0x44, 0xf, 0x2, 0x15, 0x18, 0x3b, 0x36, 0x21, 0x2c, 0xb7, 0xba, 0xad, 0xa0, 0x83, 0x8e, 0x99, 0x94, 0xdf, 0xd2, 0xc5, 0xc8, 0xeb, 0xe6, 0xf1, 0xfc, 0xda, 0xd7, 0xc0, 0xcd, 0xee, 0xe3, 0xf4, 0xf9, 0xb2, 0xbf, 0xa8, 0xa5, 0x86, 0x8b, 0x9c, 0x91, 0xa, 0x7, 0x10, 0x1d, 0x3e, 0x33, 0x24, 0x29, 0x62, 0x6f, 0x78, 0x75, 0x56, 0x5b, 0x4c, 0x41, 0xce, 0xc3, 0xd4, 0xd9, 0xfa, 0xf7, 0xe0, 0xed, 0xa6, 0xab, 0xbc, 0xb1, 0x92, 0x9f, 0x88, 0x85, 0x1e, 0x13, 0x4, 0x9, 0x2a, 0x27, 0x30, 0x3d, 0x76, 0x7b, 0x6c, 0x61, 0x42, 0x4f, 0x58, 0x55, 0x73, 0x7e, 0x69, 0x64, 0x47, 0x4a, 0x5d, 0x50, 0x1b, 0x16, 0x1, 0xc, 0x2f, 0x22, 0x35, 0x38, 0xa3, 0xae, 0xb9, 0xb4, 0x97, 0x9a, 0x8d, 0x80, 0xcb, 0xc6, 0xd1, 0xdc, 0xff, 0xf2, 0xe5, 0xe8, 0xa9, 0xa4, 0xb3, 0xbe, 0x9d, 0x90, 0x87, 0x8a, 0xc1, 0xcc, 0xdb, 0xd6, 0xf5, 0xf8, 0xef, 0xe2, 0x79, 0x74, 0x63, 0x6e, 0x4d, 0x40, 0x57, 0x5a, 0x11, 0x1c, 0xb, 0x6, 0x25, 0x28, 0x3f, 0x32, 0x14, 0x19, 0xe, 0x3, 0x20, 0x2d, 0x3a, 0x37, 0x7c, 0x71, 0x66, 0x6b, 0x48, 0x45, 0x52, 0x5f, 0xc4, 0xc9, 0xde, 0xd3, 0xf0, 0xfd, 0xea, 0xe7, 0xac, 0xa1, 0xb6, 0xbb, 0x98, 0x95, 0x82, 0x8f},
- {0x0, 0xe, 0x1c, 0x12, 0x38, 0x36, 0x24, 0x2a, 0x70, 0x7e, 0x6c, 0x62, 0x48, 0x46, 0x54, 0x5a, 0xe0, 0xee, 0xfc, 0xf2, 0xd8, 0xd6, 0xc4, 0xca, 0x90, 0x9e, 0x8c, 0x82, 0xa8, 0xa6, 0xb4, 0xba, 0xdd, 0xd3, 0xc1, 0xcf, 0xe5, 0xeb, 0xf9, 0xf7, 0xad, 0xa3, 0xb1, 0xbf, 0x95, 0x9b, 0x89, 0x87, 0x3d, 0x33, 0x21, 0x2f, 0x5, 0xb, 0x19, 0x17, 0x4d, 0x43, 0x51, 0x5f, 0x75, 0x7b, 0x69, 0x67, 0xa7, 0xa9, 0xbb, 0xb5, 0x9f, 0x91, 0x83, 0x8d, 0xd7, 0xd9, 0xcb, 0xc5, 0xef, 0xe1, 0xf3, 0xfd, 0x47, 0x49, 0x5b, 0x55, 0x7f, 0x71, 0x63, 0x6d, 0x37, 0x39, 0x2b, 0x25, 0xf, 0x1, 0x13, 0x1d, 0x7a, 0x74, 0x66, 0x68, 0x42, 0x4c, 0x5e, 0x50, 0xa, 0x4, 0x16, 0x18, 0x32, 0x3c, 0x2e, 0x20, 0x9a, 0x94, 0x86, 0x88, 0xa2, 0xac, 0xbe, 0xb0, 0xea, 0xe4, 0xf6, 0xf8, 0xd2, 0xdc, 0xce, 0xc0, 0x53, 0x5d, 0x4f, 0x41, 0x6b, 0x65, 0x77, 0x79, 0x23, 0x2d, 0x3f, 0x31, 0x1b, 0x15, 0x7, 0x9, 0xb3, 0xbd, 0xaf, 0xa1, 0x8b, 0x85, 0x97, 0x99, 0xc3, 0xcd, 0xdf, 0xd1, 0xfb, 0xf5, 0xe7, 0xe9, 0x8e, 0x80, 0x92, 0x9c, 0xb6, 0xb8, 0xaa, 0xa4, 0xfe, 0xf0, 0xe2, 0xec, 0xc6, 0xc8, 0xda, 0xd4, 0x6e, 0x60, 0x72, 0x7c, 0x56, 0x58, 0x4a, 0x44, 0x1e, 0x10, 0x2, 0xc, 0x26, 0x28, 0x3a, 0x34, 0xf4, 0xfa, 0xe8, 0xe6, 0xcc, 0xc2, 0xd0, 0xde, 0x84, 0x8a, 0x98, 0x96, 0xbc, 0xb2, 0xa0, 0xae, 0x14, 0x1a, 0x8, 0x6, 0x2c, 0x22, 0x30, 0x3e, 0x64, 0x6a, 0x78, 0x76, 0x5c, 0x52, 0x40, 0x4e, 0x29, 0x27, 0x35, 0x3b, 0x11, 0x1f, 0xd, 0x3, 0x59, 0x57, 0x45, 0x4b, 0x61, 0x6f, 0x7d, 0x73, 0xc9, 0xc7, 0xd5, 0xdb, 0xf1, 0xff, 0xed, 0xe3, 0xb9, 0xb7, 0xa5, 0xab, 0x81, 0x8f, 0x9d, 0x93},
- {0x0, 0xf, 0x1e, 0x11, 0x3c, 0x33, 0x22, 0x2d, 0x78, 0x77, 0x66, 0x69, 0x44, 0x4b, 0x5a, 0x55, 0xf0, 0xff, 0xee, 0xe1, 0xcc, 0xc3, 0xd2, 0xdd, 0x88, 0x87, 0x96, 0x99, 0xb4, 0xbb, 0xaa, 0xa5, 0xfd, 0xf2, 0xe3, 0xec, 0xc1, 0xce, 0xdf, 0xd0, 0x85, 0x8a, 0x9b, 0x94, 0xb9, 0xb6, 0xa7, 0xa8, 0xd, 0x2, 0x13, 0x1c, 0x31, 0x3e, 0x2f, 0x20, 0x75, 0x7a, 0x6b, 0x64, 0x49, 0x46, 0x57, 0x58, 0xe7, 0xe8, 0xf9, 0xf6, 0xdb, 0xd4, 0xc5, 0xca, 0x9f, 0x90, 0x81, 0x8e, 0xa3, 0xac, 0xbd, 0xb2, 0x17, 0x18, 0x9, 0x6, 0x2b, 0x24, 0x35, 0x3a, 0x6f, 0x60, 0x71, 0x7e, 0x53, 0x5c, 0x4d, 0x42, 0x1a, 0x15, 0x4, 0xb, 0x26, 0x29, 0x38, 0x37, 0x62, 0x6d, 0x7c, 0x73, 0x5e, 0x51, 0x40, 0x4f, 0xea, 0xe5, 0xf4, 0xfb, 0xd6, 0xd9, 0xc8, 0xc7, 0x92, 0x9d, 0x8c, 0x83, 0xae, 0xa1, 0xb0, 0xbf, 0xd3, 0xdc, 0xcd, 0xc2, 0xef, 0xe0, 0xf1, 0xfe, 0xab, 0xa4, 0xb5, 0xba, 0x97, 0x98, 0x89, 0x86, 0x23, 0x2c, 0x3d, 0x32, 0x1f, 0x10, 0x1, 0xe, 0x5b, 0x54, 0x45, 0x4a, 0x67, 0x68, 0x79, 0x76, 0x2e, 0x21, 0x30, 0x3f, 0x12, 0x1d, 0xc, 0x3, 0x56, 0x59, 0x48, 0x47, 0x6a, 0x65, 0x74, 0x7b, 0xde, 0xd1, 0xc0, 0xcf, 0xe2, 0xed, 0xfc, 0xf3, 0xa6, 0xa9, 0xb8, 0xb7, 0x9a, 0x95, 0x84, 0x8b, 0x34, 0x3b, 0x2a, 0x25, 0x8, 0x7, 0x16, 0x19, 0x4c, 0x43, 0x52, 0x5d, 0x70, 0x7f, 0x6e, 0x61, 0xc4, 0xcb, 0xda, 0xd5, 0xf8, 0xf7, 0xe6, 0xe9, 0xbc, 0xb3, 0xa2, 0xad, 0x80, 0x8f, 0x9e, 0x91, 0xc9, 0xc6, 0xd7, 0xd8, 0xf5, 0xfa, 0xeb, 0xe4, 0xb1, 0xbe, 0xaf, 0xa0, 0x8d, 0x82, 0x93, 0x9c, 0x39, 0x36, 0x27, 0x28, 0x5, 0xa, 0x1b, 0x14, 0x41, 0x4e, 0x5f, 0x50, 0x7d, 0x72, 0x63, 0x6c},
- {0x0, 0x10, 0x20, 0x30, 0x40, 0x50, 0x60, 0x70, 0x80, 0x90, 0xa0, 0xb0, 0xc0, 0xd0, 0xe0, 0xf0, 0x1d, 0xd, 0x3d, 0x2d, 0x5d, 0x4d, 0x7d, 0x6d, 0x9d, 0x8d, 0xbd, 0xad, 0xdd, 0xcd, 0xfd, 0xed, 0x3a, 0x2a, 0x1a, 0xa, 0x7a, 0x6a, 0x5a, 0x4a, 0xba, 0xaa, 0x9a, 0x8a, 0xfa, 0xea, 0xda, 0xca, 0x27, 0x37, 0x7, 0x17, 0x67, 0x77, 0x47, 0x57, 0xa7, 0xb7, 0x87, 0x97, 0xe7, 0xf7, 0xc7, 0xd7, 0x74, 0x64, 0x54, 0x44, 0x34, 0x24, 0x14, 0x4, 0xf4, 0xe4, 0xd4, 0xc4, 0xb4, 0xa4, 0x94, 0x84, 0x69, 0x79, 0x49, 0x59, 0x29, 0x39, 0x9, 0x19, 0xe9, 0xf9, 0xc9, 0xd9, 0xa9, 0xb9, 0x89, 0x99, 0x4e, 0x5e, 0x6e, 0x7e, 0xe, 0x1e, 0x2e, 0x3e, 0xce, 0xde, 0xee, 0xfe, 0x8e, 0x9e, 0xae, 0xbe, 0x53, 0x43, 0x73, 0x63, 0x13, 0x3, 0x33, 0x23, 0xd3, 0xc3, 0xf3, 0xe3, 0x93, 0x83, 0xb3, 0xa3, 0xe8, 0xf8, 0xc8, 0xd8, 0xa8, 0xb8, 0x88, 0x98, 0x68, 0x78, 0x48, 0x58, 0x28, 0x38, 0x8, 0x18, 0xf5, 0xe5, 0xd5, 0xc5, 0xb5, 0xa5, 0x95, 0x85, 0x75, 0x65, 0x55, 0x45, 0x35, 0x25, 0x15, 0x5, 0xd2, 0xc2, 0xf2, 0xe2, 0x92, 0x82, 0xb2, 0xa2, 0x52, 0x42, 0x72, 0x62, 0x12, 0x2, 0x32, 0x22, 0xcf, 0xdf, 0xef, 0xff, 0x8f, 0x9f, 0xaf, 0xbf, 0x4f, 0x5f, 0x6f, 0x7f, 0xf, 0x1f, 0x2f, 0x3f, 0x9c, 0x8c, 0xbc, 0xac, 0xdc, 0xcc, 0xfc, 0xec, 0x1c, 0xc, 0x3c, 0x2c, 0x5c, 0x4c, 0x7c, 0x6c, 0x81, 0x91, 0xa1, 0xb1, 0xc1, 0xd1, 0xe1, 0xf1, 0x1, 0x11, 0x21, 0x31, 0x41, 0x51, 0x61, 0x71, 0xa6, 0xb6, 0x86, 0x96, 0xe6, 0xf6, 0xc6, 0xd6, 0x26, 0x36, 0x6, 0x16, 0x66, 0x76, 0x46, 0x56, 0xbb, 0xab, 0x9b, 0x8b, 0xfb, 0xeb, 0xdb, 0xcb, 0x3b, 0x2b, 0x1b, 0xb, 0x7b, 0x6b, 0x5b, 0x4b},
- {0x0, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff, 0xd, 0x1c, 0x2f, 0x3e, 0x49, 0x58, 0x6b, 0x7a, 0x85, 0x94, 0xa7, 0xb6, 0xc1, 0xd0, 0xe3, 0xf2, 0x1a, 0xb, 0x38, 0x29, 0x5e, 0x4f, 0x7c, 0x6d, 0x92, 0x83, 0xb0, 0xa1, 0xd6, 0xc7, 0xf4, 0xe5, 0x17, 0x6, 0x35, 0x24, 0x53, 0x42, 0x71, 0x60, 0x9f, 0x8e, 0xbd, 0xac, 0xdb, 0xca, 0xf9, 0xe8, 0x34, 0x25, 0x16, 0x7, 0x70, 0x61, 0x52, 0x43, 0xbc, 0xad, 0x9e, 0x8f, 0xf8, 0xe9, 0xda, 0xcb, 0x39, 0x28, 0x1b, 0xa, 0x7d, 0x6c, 0x5f, 0x4e, 0xb1, 0xa0, 0x93, 0x82, 0xf5, 0xe4, 0xd7, 0xc6, 0x2e, 0x3f, 0xc, 0x1d, 0x6a, 0x7b, 0x48, 0x59, 0xa6, 0xb7, 0x84, 0x95, 0xe2, 0xf3, 0xc0, 0xd1, 0x23, 0x32, 0x1, 0x10, 0x67, 0x76, 0x45, 0x54, 0xab, 0xba, 0x89, 0x98, 0xef, 0xfe, 0xcd, 0xdc, 0x68, 0x79, 0x4a, 0x5b, 0x2c, 0x3d, 0xe, 0x1f, 0xe0, 0xf1, 0xc2, 0xd3, 0xa4, 0xb5, 0x86, 0x97, 0x65, 0x74, 0x47, 0x56, 0x21, 0x30, 0x3, 0x12, 0xed, 0xfc, 0xcf, 0xde, 0xa9, 0xb8, 0x8b, 0x9a, 0x72, 0x63, 0x50, 0x41, 0x36, 0x27, 0x14, 0x5, 0xfa, 0xeb, 0xd8, 0xc9, 0xbe, 0xaf, 0x9c, 0x8d, 0x7f, 0x6e, 0x5d, 0x4c, 0x3b, 0x2a, 0x19, 0x8, 0xf7, 0xe6, 0xd5, 0xc4, 0xb3, 0xa2, 0x91, 0x80, 0x5c, 0x4d, 0x7e, 0x6f, 0x18, 0x9, 0x3a, 0x2b, 0xd4, 0xc5, 0xf6, 0xe7, 0x90, 0x81, 0xb2, 0xa3, 0x51, 0x40, 0x73, 0x62, 0x15, 0x4, 0x37, 0x26, 0xd9, 0xc8, 0xfb, 0xea, 0x9d, 0x8c, 0xbf, 0xae, 0x46, 0x57, 0x64, 0x75, 0x2, 0x13, 0x20, 0x31, 0xce, 0xdf, 0xec, 0xfd, 0x8a, 0x9b, 0xa8, 0xb9, 0x4b, 0x5a, 0x69, 0x78, 0xf, 0x1e, 0x2d, 0x3c, 0xc3, 0xd2, 0xe1, 0xf0, 0x87, 0x96, 0xa5, 0xb4},
- {0x0, 0x12, 0x24, 0x36, 0x48, 0x5a, 0x6c, 0x7e, 0x90, 0x82, 0xb4, 0xa6, 0xd8, 0xca, 0xfc, 0xee, 0x3d, 0x2f, 0x19, 0xb, 0x75, 0x67, 0x51, 0x43, 0xad, 0xbf, 0x89, 0x9b, 0xe5, 0xf7, 0xc1, 0xd3, 0x7a, 0x68, 0x5e, 0x4c, 0x32, 0x20, 0x16, 0x4, 0xea, 0xf8, 0xce, 0xdc, 0xa2, 0xb0, 0x86, 0x94, 0x47, 0x55, 0x63, 0x71, 0xf, 0x1d, 0x2b, 0x39, 0xd7, 0xc5, 0xf3, 0xe1, 0x9f, 0x8d, 0xbb, 0xa9, 0xf4, 0xe6, 0xd0, 0xc2, 0xbc, 0xae, 0x98, 0x8a, 0x64, 0x76, 0x40, 0x52, 0x2c, 0x3e, 0x8, 0x1a, 0xc9, 0xdb, 0xed, 0xff, 0x81, 0x93, 0xa5, 0xb7, 0x59, 0x4b, 0x7d, 0x6f, 0x11, 0x3, 0x35, 0x27, 0x8e, 0x9c, 0xaa, 0xb8, 0xc6, 0xd4, 0xe2, 0xf0, 0x1e, 0xc, 0x3a, 0x28, 0x56, 0x44, 0x72, 0x60, 0xb3, 0xa1, 0x97, 0x85, 0xfb, 0xe9, 0xdf, 0xcd, 0x23, 0x31, 0x7, 0x15, 0x6b, 0x79, 0x4f, 0x5d, 0xf5, 0xe7, 0xd1, 0xc3, 0xbd, 0xaf, 0x99, 0x8b, 0x65, 0x77, 0x41, 0x53, 0x2d, 0x3f, 0x9, 0x1b, 0xc8, 0xda, 0xec, 0xfe, 0x80, 0x92, 0xa4, 0xb6, 0x58, 0x4a, 0x7c, 0x6e, 0x10, 0x2, 0x34, 0x26, 0x8f, 0x9d, 0xab, 0xb9, 0xc7, 0xd5, 0xe3, 0xf1, 0x1f, 0xd, 0x3b, 0x29, 0x57, 0x45, 0x73, 0x61, 0xb2, 0xa0, 0x96, 0x84, 0xfa, 0xe8, 0xde, 0xcc, 0x22, 0x30, 0x6, 0x14, 0x6a, 0x78, 0x4e, 0x5c, 0x1, 0x13, 0x25, 0x37, 0x49, 0x5b, 0x6d, 0x7f, 0x91, 0x83, 0xb5, 0xa7, 0xd9, 0xcb, 0xfd, 0xef, 0x3c, 0x2e, 0x18, 0xa, 0x74, 0x66, 0x50, 0x42, 0xac, 0xbe, 0x88, 0x9a, 0xe4, 0xf6, 0xc0, 0xd2, 0x7b, 0x69, 0x5f, 0x4d, 0x33, 0x21, 0x17, 0x5, 0xeb, 0xf9, 0xcf, 0xdd, 0xa3, 0xb1, 0x87, 0x95, 0x46, 0x54, 0x62, 0x70, 0xe, 0x1c, 0x2a, 0x38, 0xd6, 0xc4, 0xf2, 0xe0, 0x9e, 0x8c, 0xba, 0xa8},
- {0x0, 0x13, 0x26, 0x35, 0x4c, 0x5f, 0x6a, 0x79, 0x98, 0x8b, 0xbe, 0xad, 0xd4, 0xc7, 0xf2, 0xe1, 0x2d, 0x3e, 0xb, 0x18, 0x61, 0x72, 0x47, 0x54, 0xb5, 0xa6, 0x93, 0x80, 0xf9, 0xea, 0xdf, 0xcc, 0x5a, 0x49, 0x7c, 0x6f, 0x16, 0x5, 0x30, 0x23, 0xc2, 0xd1, 0xe4, 0xf7, 0x8e, 0x9d, 0xa8, 0xbb, 0x77, 0x64, 0x51, 0x42, 0x3b, 0x28, 0x1d, 0xe, 0xef, 0xfc, 0xc9, 0xda, 0xa3, 0xb0, 0x85, 0x96, 0xb4, 0xa7, 0x92, 0x81, 0xf8, 0xeb, 0xde, 0xcd, 0x2c, 0x3f, 0xa, 0x19, 0x60, 0x73, 0x46, 0x55, 0x99, 0x8a, 0xbf, 0xac, 0xd5, 0xc6, 0xf3, 0xe0, 0x1, 0x12, 0x27, 0x34, 0x4d, 0x5e, 0x6b, 0x78, 0xee, 0xfd, 0xc8, 0xdb, 0xa2, 0xb1, 0x84, 0x97, 0x76, 0x65, 0x50, 0x43, 0x3a, 0x29, 0x1c, 0xf, 0xc3, 0xd0, 0xe5, 0xf6, 0x8f, 0x9c, 0xa9, 0xba, 0x5b, 0x48, 0x7d, 0x6e, 0x17, 0x4, 0x31, 0x22, 0x75, 0x66, 0x53, 0x40, 0x39, 0x2a, 0x1f, 0xc, 0xed, 0xfe, 0xcb, 0xd8, 0xa1, 0xb2, 0x87, 0x94, 0x58, 0x4b, 0x7e, 0x6d, 0x14, 0x7, 0x32, 0x21, 0xc0, 0xd3, 0xe6, 0xf5, 0x8c, 0x9f, 0xaa, 0xb9, 0x2f, 0x3c, 0x9, 0x1a, 0x63, 0x70, 0x45, 0x56, 0xb7, 0xa4, 0x91, 0x82, 0xfb, 0xe8, 0xdd, 0xce, 0x2, 0x11, 0x24, 0x37, 0x4e, 0x5d, 0x68, 0x7b, 0x9a, 0x89, 0xbc, 0xaf, 0xd6, 0xc5, 0xf0, 0xe3, 0xc1, 0xd2, 0xe7, 0xf4, 0x8d, 0x9e, 0xab, 0xb8, 0x59, 0x4a, 0x7f, 0x6c, 0x15, 0x6, 0x33, 0x20, 0xec, 0xff, 0xca, 0xd9, 0xa0, 0xb3, 0x86, 0x95, 0x74, 0x67, 0x52, 0x41, 0x38, 0x2b, 0x1e, 0xd, 0x9b, 0x88, 0xbd, 0xae, 0xd7, 0xc4, 0xf1, 0xe2, 0x3, 0x10, 0x25, 0x36, 0x4f, 0x5c, 0x69, 0x7a, 0xb6, 0xa5, 0x90, 0x83, 0xfa, 0xe9, 0xdc, 0xcf, 0x2e, 0x3d, 0x8, 0x1b, 0x62, 0x71, 0x44, 0x57},
- {0x0, 0x14, 0x28, 0x3c, 0x50, 0x44, 0x78, 0x6c, 0xa0, 0xb4, 0x88, 0x9c, 0xf0, 0xe4, 0xd8, 0xcc, 0x5d, 0x49, 0x75, 0x61, 0xd, 0x19, 0x25, 0x31, 0xfd, 0xe9, 0xd5, 0xc1, 0xad, 0xb9, 0x85, 0x91, 0xba, 0xae, 0x92, 0x86, 0xea, 0xfe, 0xc2, 0xd6, 0x1a, 0xe, 0x32, 0x26, 0x4a, 0x5e, 0x62, 0x76, 0xe7, 0xf3, 0xcf, 0xdb, 0xb7, 0xa3, 0x9f, 0x8b, 0x47, 0x53, 0x6f, 0x7b, 0x17, 0x3, 0x3f, 0x2b, 0x69, 0x7d, 0x41, 0x55, 0x39, 0x2d, 0x11, 0x5, 0xc9, 0xdd, 0xe1, 0xf5, 0x99, 0x8d, 0xb1, 0xa5, 0x34, 0x20, 0x1c, 0x8, 0x64, 0x70, 0x4c, 0x58, 0x94, 0x80, 0xbc, 0xa8, 0xc4, 0xd0, 0xec, 0xf8, 0xd3, 0xc7, 0xfb, 0xef, 0x83, 0x97, 0xab, 0xbf, 0x73, 0x67, 0x5b, 0x4f, 0x23, 0x37, 0xb, 0x1f, 0x8e, 0x9a, 0xa6, 0xb2, 0xde, 0xca, 0xf6, 0xe2, 0x2e, 0x3a, 0x6, 0x12, 0x7e, 0x6a, 0x56, 0x42, 0xd2, 0xc6, 0xfa, 0xee, 0x82, 0x96, 0xaa, 0xbe, 0x72, 0x66, 0x5a, 0x4e, 0x22, 0x36, 0xa, 0x1e, 0x8f, 0x9b, 0xa7, 0xb3, 0xdf, 0xcb, 0xf7, 0xe3, 0x2f, 0x3b, 0x7, 0x13, 0x7f, 0x6b, 0x57, 0x43, 0x68, 0x7c, 0x40, 0x54, 0x38, 0x2c, 0x10, 0x4, 0xc8, 0xdc, 0xe0, 0xf4, 0x98, 0x8c, 0xb0, 0xa4, 0x35, 0x21, 0x1d, 0x9, 0x65, 0x71, 0x4d, 0x59, 0x95, 0x81, 0xbd, 0xa9, 0xc5, 0xd1, 0xed, 0xf9, 0xbb, 0xaf, 0x93, 0x87, 0xeb, 0xff, 0xc3, 0xd7, 0x1b, 0xf, 0x33, 0x27, 0x4b, 0x5f, 0x63, 0x77, 0xe6, 0xf2, 0xce, 0xda, 0xb6, 0xa2, 0x9e, 0x8a, 0x46, 0x52, 0x6e, 0x7a, 0x16, 0x2, 0x3e, 0x2a, 0x1, 0x15, 0x29, 0x3d, 0x51, 0x45, 0x79, 0x6d, 0xa1, 0xb5, 0x89, 0x9d, 0xf1, 0xe5, 0xd9, 0xcd, 0x5c, 0x48, 0x74, 0x60, 0xc, 0x18, 0x24, 0x30, 0xfc, 0xe8, 0xd4, 0xc0, 0xac, 0xb8, 0x84, 0x90},
- {0x0, 0x15, 0x2a, 0x3f, 0x54, 0x41, 0x7e, 0x6b, 0xa8, 0xbd, 0x82, 0x97, 0xfc, 0xe9, 0xd6, 0xc3, 0x4d, 0x58, 0x67, 0x72, 0x19, 0xc, 0x33, 0x26, 0xe5, 0xf0, 0xcf, 0xda, 0xb1, 0xa4, 0x9b, 0x8e, 0x9a, 0x8f, 0xb0, 0xa5, 0xce, 0xdb, 0xe4, 0xf1, 0x32, 0x27, 0x18, 0xd, 0x66, 0x73, 0x4c, 0x59, 0xd7, 0xc2, 0xfd, 0xe8, 0x83, 0x96, 0xa9, 0xbc, 0x7f, 0x6a, 0x55, 0x40, 0x2b, 0x3e, 0x1, 0x14, 0x29, 0x3c, 0x3, 0x16, 0x7d, 0x68, 0x57, 0x42, 0x81, 0x94, 0xab, 0xbe, 0xd5, 0xc0, 0xff, 0xea, 0x64, 0x71, 0x4e, 0x5b, 0x30, 0x25, 0x1a, 0xf, 0xcc, 0xd9, 0xe6, 0xf3, 0x98, 0x8d, 0xb2, 0xa7, 0xb3, 0xa6, 0x99, 0x8c, 0xe7, 0xf2, 0xcd, 0xd8, 0x1b, 0xe, 0x31, 0x24, 0x4f, 0x5a, 0x65, 0x70, 0xfe, 0xeb, 0xd4, 0xc1, 0xaa, 0xbf, 0x80, 0x95, 0x56, 0x43, 0x7c, 0x69, 0x2, 0x17, 0x28, 0x3d, 0x52, 0x47, 0x78, 0x6d, 0x6, 0x13, 0x2c, 0x39, 0xfa, 0xef, 0xd0, 0xc5, 0xae, 0xbb, 0x84, 0x91, 0x1f, 0xa, 0x35, 0x20, 0x4b, 0x5e, 0x61, 0x74, 0xb7, 0xa2, 0x9d, 0x88, 0xe3, 0xf6, 0xc9, 0xdc, 0xc8, 0xdd, 0xe2, 0xf7, 0x9c, 0x89, 0xb6, 0xa3, 0x60, 0x75, 0x4a, 0x5f, 0x34, 0x21, 0x1e, 0xb, 0x85, 0x90, 0xaf, 0xba, 0xd1, 0xc4, 0xfb, 0xee, 0x2d, 0x38, 0x7, 0x12, 0x79, 0x6c, 0x53, 0x46, 0x7b, 0x6e, 0x51, 0x44, 0x2f, 0x3a, 0x5, 0x10, 0xd3, 0xc6, 0xf9, 0xec, 0x87, 0x92, 0xad, 0xb8, 0x36, 0x23, 0x1c, 0x9, 0x62, 0x77, 0x48, 0x5d, 0x9e, 0x8b, 0xb4, 0xa1, 0xca, 0xdf, 0xe0, 0xf5, 0xe1, 0xf4, 0xcb, 0xde, 0xb5, 0xa0, 0x9f, 0x8a, 0x49, 0x5c, 0x63, 0x76, 0x1d, 0x8, 0x37, 0x22, 0xac, 0xb9, 0x86, 0x93, 0xf8, 0xed, 0xd2, 0xc7, 0x4, 0x11, 0x2e, 0x3b, 0x50, 0x45, 0x7a, 0x6f},
- {0x0, 0x16, 0x2c, 0x3a, 0x58, 0x4e, 0x74, 0x62, 0xb0, 0xa6, 0x9c, 0x8a, 0xe8, 0xfe, 0xc4, 0xd2, 0x7d, 0x6b, 0x51, 0x47, 0x25, 0x33, 0x9, 0x1f, 0xcd, 0xdb, 0xe1, 0xf7, 0x95, 0x83, 0xb9, 0xaf, 0xfa, 0xec, 0xd6, 0xc0, 0xa2, 0xb4, 0x8e, 0x98, 0x4a, 0x5c, 0x66, 0x70, 0x12, 0x4, 0x3e, 0x28, 0x87, 0x91, 0xab, 0xbd, 0xdf, 0xc9, 0xf3, 0xe5, 0x37, 0x21, 0x1b, 0xd, 0x6f, 0x79, 0x43, 0x55, 0xe9, 0xff, 0xc5, 0xd3, 0xb1, 0xa7, 0x9d, 0x8b, 0x59, 0x4f, 0x75, 0x63, 0x1, 0x17, 0x2d, 0x3b, 0x94, 0x82, 0xb8, 0xae, 0xcc, 0xda, 0xe0, 0xf6, 0x24, 0x32, 0x8, 0x1e, 0x7c, 0x6a, 0x50, 0x46, 0x13, 0x5, 0x3f, 0x29, 0x4b, 0x5d, 0x67, 0x71, 0xa3, 0xb5, 0x8f, 0x99, 0xfb, 0xed, 0xd7, 0xc1, 0x6e, 0x78, 0x42, 0x54, 0x36, 0x20, 0x1a, 0xc, 0xde, 0xc8, 0xf2, 0xe4, 0x86, 0x90, 0xaa, 0xbc, 0xcf, 0xd9, 0xe3, 0xf5, 0x97, 0x81, 0xbb, 0xad, 0x7f, 0x69, 0x53, 0x45, 0x27, 0x31, 0xb, 0x1d, 0xb2, 0xa4, 0x9e, 0x88, 0xea, 0xfc, 0xc6, 0xd0, 0x2, 0x14, 0x2e, 0x38, 0x5a, 0x4c, 0x76, 0x60, 0x35, 0x23, 0x19, 0xf, 0x6d, 0x7b, 0x41, 0x57, 0x85, 0x93, 0xa9, 0xbf, 0xdd, 0xcb, 0xf1, 0xe7, 0x48, 0x5e, 0x64, 0x72, 0x10, 0x6, 0x3c, 0x2a, 0xf8, 0xee, 0xd4, 0xc2, 0xa0, 0xb6, 0x8c, 0x9a, 0x26, 0x30, 0xa, 0x1c, 0x7e, 0x68, 0x52, 0x44, 0x96, 0x80, 0xba, 0xac, 0xce, 0xd8, 0xe2, 0xf4, 0x5b, 0x4d, 0x77, 0x61, 0x3, 0x15, 0x2f, 0x39, 0xeb, 0xfd, 0xc7, 0xd1, 0xb3, 0xa5, 0x9f, 0x89, 0xdc, 0xca, 0xf0, 0xe6, 0x84, 0x92, 0xa8, 0xbe, 0x6c, 0x7a, 0x40, 0x56, 0x34, 0x22, 0x18, 0xe, 0xa1, 0xb7, 0x8d, 0x9b, 0xf9, 0xef, 0xd5, 0xc3, 0x11, 0x7, 0x3d, 0x2b, 0x49, 0x5f, 0x65, 0x73},
- {0x0, 0x17, 0x2e, 0x39, 0x5c, 0x4b, 0x72, 0x65, 0xb8, 0xaf, 0x96, 0x81, 0xe4, 0xf3, 0xca, 0xdd, 0x6d, 0x7a, 0x43, 0x54, 0x31, 0x26, 0x1f, 0x8, 0xd5, 0xc2, 0xfb, 0xec, 0x89, 0x9e, 0xa7, 0xb0, 0xda, 0xcd, 0xf4, 0xe3, 0x86, 0x91, 0xa8, 0xbf, 0x62, 0x75, 0x4c, 0x5b, 0x3e, 0x29, 0x10, 0x7, 0xb7, 0xa0, 0x99, 0x8e, 0xeb, 0xfc, 0xc5, 0xd2, 0xf, 0x18, 0x21, 0x36, 0x53, 0x44, 0x7d, 0x6a, 0xa9, 0xbe, 0x87, 0x90, 0xf5, 0xe2, 0xdb, 0xcc, 0x11, 0x6, 0x3f, 0x28, 0x4d, 0x5a, 0x63, 0x74, 0xc4, 0xd3, 0xea, 0xfd, 0x98, 0x8f, 0xb6, 0xa1, 0x7c, 0x6b, 0x52, 0x45, 0x20, 0x37, 0xe, 0x19, 0x73, 0x64, 0x5d, 0x4a, 0x2f, 0x38, 0x1, 0x16, 0xcb, 0xdc, 0xe5, 0xf2, 0x97, 0x80, 0xb9, 0xae, 0x1e, 0x9, 0x30, 0x27, 0x42, 0x55, 0x6c, 0x7b, 0xa6, 0xb1, 0x88, 0x9f, 0xfa, 0xed, 0xd4, 0xc3, 0x4f, 0x58, 0x61, 0x76, 0x13, 0x4, 0x3d, 0x2a, 0xf7, 0xe0, 0xd9, 0xce, 0xab, 0xbc, 0x85, 0x92, 0x22, 0x35, 0xc, 0x1b, 0x7e, 0x69, 0x50, 0x47, 0x9a, 0x8d, 0xb4, 0xa3, 0xc6, 0xd1, 0xe8, 0xff, 0x95, 0x82, 0xbb, 0xac, 0xc9, 0xde, 0xe7, 0xf0, 0x2d, 0x3a, 0x3, 0x14, 0x71, 0x66, 0x5f, 0x48, 0xf8, 0xef, 0xd6, 0xc1, 0xa4, 0xb3, 0x8a, 0x9d, 0x40, 0x57, 0x6e, 0x79, 0x1c, 0xb, 0x32, 0x25, 0xe6, 0xf1, 0xc8, 0xdf, 0xba, 0xad, 0x94, 0x83, 0x5e, 0x49, 0x70, 0x67, 0x2, 0x15, 0x2c, 0x3b, 0x8b, 0x9c, 0xa5, 0xb2, 0xd7, 0xc0, 0xf9, 0xee, 0x33, 0x24, 0x1d, 0xa, 0x6f, 0x78, 0x41, 0x56, 0x3c, 0x2b, 0x12, 0x5, 0x60, 0x77, 0x4e, 0x59, 0x84, 0x93, 0xaa, 0xbd, 0xd8, 0xcf, 0xf6, 0xe1, 0x51, 0x46, 0x7f, 0x68, 0xd, 0x1a, 0x23, 0x34, 0xe9, 0xfe, 0xc7, 0xd0, 0xb5, 0xa2, 0x9b, 0x8c},
- {0x0, 0x18, 0x30, 0x28, 0x60, 0x78, 0x50, 0x48, 0xc0, 0xd8, 0xf0, 0xe8, 0xa0, 0xb8, 0x90, 0x88, 0x9d, 0x85, 0xad, 0xb5, 0xfd, 0xe5, 0xcd, 0xd5, 0x5d, 0x45, 0x6d, 0x75, 0x3d, 0x25, 0xd, 0x15, 0x27, 0x3f, 0x17, 0xf, 0x47, 0x5f, 0x77, 0x6f, 0xe7, 0xff, 0xd7, 0xcf, 0x87, 0x9f, 0xb7, 0xaf, 0xba, 0xa2, 0x8a, 0x92, 0xda, 0xc2, 0xea, 0xf2, 0x7a, 0x62, 0x4a, 0x52, 0x1a, 0x2, 0x2a, 0x32, 0x4e, 0x56, 0x7e, 0x66, 0x2e, 0x36, 0x1e, 0x6, 0x8e, 0x96, 0xbe, 0xa6, 0xee, 0xf6, 0xde, 0xc6, 0xd3, 0xcb, 0xe3, 0xfb, 0xb3, 0xab, 0x83, 0x9b, 0x13, 0xb, 0x23, 0x3b, 0x73, 0x6b, 0x43, 0x5b, 0x69, 0x71, 0x59, 0x41, 0x9, 0x11, 0x39, 0x21, 0xa9, 0xb1, 0x99, 0x81, 0xc9, 0xd1, 0xf9, 0xe1, 0xf4, 0xec, 0xc4, 0xdc, 0x94, 0x8c, 0xa4, 0xbc, 0x34, 0x2c, 0x4, 0x1c, 0x54, 0x4c, 0x64, 0x7c, 0x9c, 0x84, 0xac, 0xb4, 0xfc, 0xe4, 0xcc, 0xd4, 0x5c, 0x44, 0x6c, 0x74, 0x3c, 0x24, 0xc, 0x14, 0x1, 0x19, 0x31, 0x29, 0x61, 0x79, 0x51, 0x49, 0xc1, 0xd9, 0xf1, 0xe9, 0xa1, 0xb9, 0x91, 0x89, 0xbb, 0xa3, 0x8b, 0x93, 0xdb, 0xc3, 0xeb, 0xf3, 0x7b, 0x63, 0x4b, 0x53, 0x1b, 0x3, 0x2b, 0x33, 0x26, 0x3e, 0x16, 0xe, 0x46, 0x5e, 0x76, 0x6e, 0xe6, 0xfe, 0xd6, 0xce, 0x86, 0x9e, 0xb6, 0xae, 0xd2, 0xca, 0xe2, 0xfa, 0xb2, 0xaa, 0x82, 0x9a, 0x12, 0xa, 0x22, 0x3a, 0x72, 0x6a, 0x42, 0x5a, 0x4f, 0x57, 0x7f, 0x67, 0x2f, 0x37, 0x1f, 0x7, 0x8f, 0x97, 0xbf, 0xa7, 0xef, 0xf7, 0xdf, 0xc7, 0xf5, 0xed, 0xc5, 0xdd, 0x95, 0x8d, 0xa5, 0xbd, 0x35, 0x2d, 0x5, 0x1d, 0x55, 0x4d, 0x65, 0x7d, 0x68, 0x70, 0x58, 0x40, 0x8, 0x10, 0x38, 0x20, 0xa8, 0xb0, 0x98, 0x80, 0xc8, 0xd0, 0xf8, 0xe0},
- {0x0, 0x19, 0x32, 0x2b, 0x64, 0x7d, 0x56, 0x4f, 0xc8, 0xd1, 0xfa, 0xe3, 0xac, 0xb5, 0x9e, 0x87, 0x8d, 0x94, 0xbf, 0xa6, 0xe9, 0xf0, 0xdb, 0xc2, 0x45, 0x5c, 0x77, 0x6e, 0x21, 0x38, 0x13, 0xa, 0x7, 0x1e, 0x35, 0x2c, 0x63, 0x7a, 0x51, 0x48, 0xcf, 0xd6, 0xfd, 0xe4, 0xab, 0xb2, 0x99, 0x80, 0x8a, 0x93, 0xb8, 0xa1, 0xee, 0xf7, 0xdc, 0xc5, 0x42, 0x5b, 0x70, 0x69, 0x26, 0x3f, 0x14, 0xd, 0xe, 0x17, 0x3c, 0x25, 0x6a, 0x73, 0x58, 0x41, 0xc6, 0xdf, 0xf4, 0xed, 0xa2, 0xbb, 0x90, 0x89, 0x83, 0x9a, 0xb1, 0xa8, 0xe7, 0xfe, 0xd5, 0xcc, 0x4b, 0x52, 0x79, 0x60, 0x2f, 0x36, 0x1d, 0x4, 0x9, 0x10, 0x3b, 0x22, 0x6d, 0x74, 0x5f, 0x46, 0xc1, 0xd8, 0xf3, 0xea, 0xa5, 0xbc, 0x97, 0x8e, 0x84, 0x9d, 0xb6, 0xaf, 0xe0, 0xf9, 0xd2, 0xcb, 0x4c, 0x55, 0x7e, 0x67, 0x28, 0x31, 0x1a, 0x3, 0x1c, 0x5, 0x2e, 0x37, 0x78, 0x61, 0x4a, 0x53, 0xd4, 0xcd, 0xe6, 0xff, 0xb0, 0xa9, 0x82, 0x9b, 0x91, 0x88, 0xa3, 0xba, 0xf5, 0xec, 0xc7, 0xde, 0x59, 0x40, 0x6b, 0x72, 0x3d, 0x24, 0xf, 0x16, 0x1b, 0x2, 0x29, 0x30, 0x7f, 0x66, 0x4d, 0x54, 0xd3, 0xca, 0xe1, 0xf8, 0xb7, 0xae, 0x85, 0x9c, 0x96, 0x8f, 0xa4, 0xbd, 0xf2, 0xeb, 0xc0, 0xd9, 0x5e, 0x47, 0x6c, 0x75, 0x3a, 0x23, 0x8, 0x11, 0x12, 0xb, 0x20, 0x39, 0x76, 0x6f, 0x44, 0x5d, 0xda, 0xc3, 0xe8, 0xf1, 0xbe, 0xa7, 0x8c, 0x95, 0x9f, 0x86, 0xad, 0xb4, 0xfb, 0xe2, 0xc9, 0xd0, 0x57, 0x4e, 0x65, 0x7c, 0x33, 0x2a, 0x1, 0x18, 0x15, 0xc, 0x27, 0x3e, 0x71, 0x68, 0x43, 0x5a, 0xdd, 0xc4, 0xef, 0xf6, 0xb9, 0xa0, 0x8b, 0x92, 0x98, 0x81, 0xaa, 0xb3, 0xfc, 0xe5, 0xce, 0xd7, 0x50, 0x49, 0x62, 0x7b, 0x34, 0x2d, 0x6, 0x1f},
- {0x0, 0x1a, 0x34, 0x2e, 0x68, 0x72, 0x5c, 0x46, 0xd0, 0xca, 0xe4, 0xfe, 0xb8, 0xa2, 0x8c, 0x96, 0xbd, 0xa7, 0x89, 0x93, 0xd5, 0xcf, 0xe1, 0xfb, 0x6d, 0x77, 0x59, 0x43, 0x5, 0x1f, 0x31, 0x2b, 0x67, 0x7d, 0x53, 0x49, 0xf, 0x15, 0x3b, 0x21, 0xb7, 0xad, 0x83, 0x99, 0xdf, 0xc5, 0xeb, 0xf1, 0xda, 0xc0, 0xee, 0xf4, 0xb2, 0xa8, 0x86, 0x9c, 0xa, 0x10, 0x3e, 0x24, 0x62, 0x78, 0x56, 0x4c, 0xce, 0xd4, 0xfa, 0xe0, 0xa6, 0xbc, 0x92, 0x88, 0x1e, 0x4, 0x2a, 0x30, 0x76, 0x6c, 0x42, 0x58, 0x73, 0x69, 0x47, 0x5d, 0x1b, 0x1, 0x2f, 0x35, 0xa3, 0xb9, 0x97, 0x8d, 0xcb, 0xd1, 0xff, 0xe5, 0xa9, 0xb3, 0x9d, 0x87, 0xc1, 0xdb, 0xf5, 0xef, 0x79, 0x63, 0x4d, 0x57, 0x11, 0xb, 0x25, 0x3f, 0x14, 0xe, 0x20, 0x3a, 0x7c, 0x66, 0x48, 0x52, 0xc4, 0xde, 0xf0, 0xea, 0xac, 0xb6, 0x98, 0x82, 0x81, 0x9b, 0xb5, 0xaf, 0xe9, 0xf3, 0xdd, 0xc7, 0x51, 0x4b, 0x65, 0x7f, 0x39, 0x23, 0xd, 0x17, 0x3c, 0x26, 0x8, 0x12, 0x54, 0x4e, 0x60, 0x7a, 0xec, 0xf6, 0xd8, 0xc2, 0x84, 0x9e, 0xb0, 0xaa, 0xe6, 0xfc, 0xd2, 0xc8, 0x8e, 0x94, 0xba, 0xa0, 0x36, 0x2c, 0x2, 0x18, 0x5e, 0x44, 0x6a, 0x70, 0x5b, 0x41, 0x6f, 0x75, 0x33, 0x29, 0x7, 0x1d, 0x8b, 0x91, 0xbf, 0xa5, 0xe3, 0xf9, 0xd7, 0xcd, 0x4f, 0x55, 0x7b, 0x61, 0x27, 0x3d, 0x13, 0x9, 0x9f, 0x85, 0xab, 0xb1, 0xf7, 0xed, 0xc3, 0xd9, 0xf2, 0xe8, 0xc6, 0xdc, 0x9a, 0x80, 0xae, 0xb4, 0x22, 0x38, 0x16, 0xc, 0x4a, 0x50, 0x7e, 0x64, 0x28, 0x32, 0x1c, 0x6, 0x40, 0x5a, 0x74, 0x6e, 0xf8, 0xe2, 0xcc, 0xd6, 0x90, 0x8a, 0xa4, 0xbe, 0x95, 0x8f, 0xa1, 0xbb, 0xfd, 0xe7, 0xc9, 0xd3, 0x45, 0x5f, 0x71, 0x6b, 0x2d, 0x37, 0x19, 0x3},
- {0x0, 0x1b, 0x36, 0x2d, 0x6c, 0x77, 0x5a, 0x41, 0xd8, 0xc3, 0xee, 0xf5, 0xb4, 0xaf, 0x82, 0x99, 0xad, 0xb6, 0x9b, 0x80, 0xc1, 0xda, 0xf7, 0xec, 0x75, 0x6e, 0x43, 0x58, 0x19, 0x2, 0x2f, 0x34, 0x47, 0x5c, 0x71, 0x6a, 0x2b, 0x30, 0x1d, 0x6, 0x9f, 0x84, 0xa9, 0xb2, 0xf3, 0xe8, 0xc5, 0xde, 0xea, 0xf1, 0xdc, 0xc7, 0x86, 0x9d, 0xb0, 0xab, 0x32, 0x29, 0x4, 0x1f, 0x5e, 0x45, 0x68, 0x73, 0x8e, 0x95, 0xb8, 0xa3, 0xe2, 0xf9, 0xd4, 0xcf, 0x56, 0x4d, 0x60, 0x7b, 0x3a, 0x21, 0xc, 0x17, 0x23, 0x38, 0x15, 0xe, 0x4f, 0x54, 0x79, 0x62, 0xfb, 0xe0, 0xcd, 0xd6, 0x97, 0x8c, 0xa1, 0xba, 0xc9, 0xd2, 0xff, 0xe4, 0xa5, 0xbe, 0x93, 0x88, 0x11, 0xa, 0x27, 0x3c, 0x7d, 0x66, 0x4b, 0x50, 0x64, 0x7f, 0x52, 0x49, 0x8, 0x13, 0x3e, 0x25, 0xbc, 0xa7, 0x8a, 0x91, 0xd0, 0xcb, 0xe6, 0xfd, 0x1, 0x1a, 0x37, 0x2c, 0x6d, 0x76, 0x5b, 0x40, 0xd9, 0xc2, 0xef, 0xf4, 0xb5, 0xae, 0x83, 0x98, 0xac, 0xb7, 0x9a, 0x81, 0xc0, 0xdb, 0xf6, 0xed, 0x74, 0x6f, 0x42, 0x59, 0x18, 0x3, 0x2e, 0x35, 0x46, 0x5d, 0x70, 0x6b, 0x2a, 0x31, 0x1c, 0x7, 0x9e, 0x85, 0xa8, 0xb3, 0xf2, 0xe9, 0xc4, 0xdf, 0xeb, 0xf0, 0xdd, 0xc6, 0x87, 0x9c, 0xb1, 0xaa, 0x33, 0x28, 0x5, 0x1e, 0x5f, 0x44, 0x69, 0x72, 0x8f, 0x94, 0xb9, 0xa2, 0xe3, 0xf8, 0xd5, 0xce, 0x57, 0x4c, 0x61, 0x7a, 0x3b, 0x20, 0xd, 0x16, 0x22, 0x39, 0x14, 0xf, 0x4e, 0x55, 0x78, 0x63, 0xfa, 0xe1, 0xcc, 0xd7, 0x96, 0x8d, 0xa0, 0xbb, 0xc8, 0xd3, 0xfe, 0xe5, 0xa4, 0xbf, 0x92, 0x89, 0x10, 0xb, 0x26, 0x3d, 0x7c, 0x67, 0x4a, 0x51, 0x65, 0x7e, 0x53, 0x48, 0x9, 0x12, 0x3f, 0x24, 0xbd, 0xa6, 0x8b, 0x90, 0xd1, 0xca, 0xe7, 0xfc},
- {0x0, 0x1c, 0x38, 0x24, 0x70, 0x6c, 0x48, 0x54, 0xe0, 0xfc, 0xd8, 0xc4, 0x90, 0x8c, 0xa8, 0xb4, 0xdd, 0xc1, 0xe5, 0xf9, 0xad, 0xb1, 0x95, 0x89, 0x3d, 0x21, 0x5, 0x19, 0x4d, 0x51, 0x75, 0x69, 0xa7, 0xbb, 0x9f, 0x83, 0xd7, 0xcb, 0xef, 0xf3, 0x47, 0x5b, 0x7f, 0x63, 0x37, 0x2b, 0xf, 0x13, 0x7a, 0x66, 0x42, 0x5e, 0xa, 0x16, 0x32, 0x2e, 0x9a, 0x86, 0xa2, 0xbe, 0xea, 0xf6, 0xd2, 0xce, 0x53, 0x4f, 0x6b, 0x77, 0x23, 0x3f, 0x1b, 0x7, 0xb3, 0xaf, 0x8b, 0x97, 0xc3, 0xdf, 0xfb, 0xe7, 0x8e, 0x92, 0xb6, 0xaa, 0xfe, 0xe2, 0xc6, 0xda, 0x6e, 0x72, 0x56, 0x4a, 0x1e, 0x2, 0x26, 0x3a, 0xf4, 0xe8, 0xcc, 0xd0, 0x84, 0x98, 0xbc, 0xa0, 0x14, 0x8, 0x2c, 0x30, 0x64, 0x78, 0x5c, 0x40, 0x29, 0x35, 0x11, 0xd, 0x59, 0x45, 0x61, 0x7d, 0xc9, 0xd5, 0xf1, 0xed, 0xb9, 0xa5, 0x81, 0x9d, 0xa6, 0xba, 0x9e, 0x82, 0xd6, 0xca, 0xee, 0xf2, 0x46, 0x5a, 0x7e, 0x62, 0x36, 0x2a, 0xe, 0x12, 0x7b, 0x67, 0x43, 0x5f, 0xb, 0x17, 0x33, 0x2f, 0x9b, 0x87, 0xa3, 0xbf, 0xeb, 0xf7, 0xd3, 0xcf, 0x1, 0x1d, 0x39, 0x25, 0x71, 0x6d, 0x49, 0x55, 0xe1, 0xfd, 0xd9, 0xc5, 0x91, 0x8d, 0xa9, 0xb5, 0xdc, 0xc0, 0xe4, 0xf8, 0xac, 0xb0, 0x94, 0x88, 0x3c, 0x20, 0x4, 0x18, 0x4c, 0x50, 0x74, 0x68, 0xf5, 0xe9, 0xcd, 0xd1, 0x85, 0x99, 0xbd, 0xa1, 0x15, 0x9, 0x2d, 0x31, 0x65, 0x79, 0x5d, 0x41, 0x28, 0x34, 0x10, 0xc, 0x58, 0x44, 0x60, 0x7c, 0xc8, 0xd4, 0xf0, 0xec, 0xb8, 0xa4, 0x80, 0x9c, 0x52, 0x4e, 0x6a, 0x76, 0x22, 0x3e, 0x1a, 0x6, 0xb2, 0xae, 0x8a, 0x96, 0xc2, 0xde, 0xfa, 0xe6, 0x8f, 0x93, 0xb7, 0xab, 0xff, 0xe3, 0xc7, 0xdb, 0x6f, 0x73, 0x57, 0x4b, 0x1f, 0x3, 0x27, 0x3b},
- {0x0, 0x1d, 0x3a, 0x27, 0x74, 0x69, 0x4e, 0x53, 0xe8, 0xf5, 0xd2, 0xcf, 0x9c, 0x81, 0xa6, 0xbb, 0xcd, 0xd0, 0xf7, 0xea, 0xb9, 0xa4, 0x83, 0x9e, 0x25, 0x38, 0x1f, 0x2, 0x51, 0x4c, 0x6b, 0x76, 0x87, 0x9a, 0xbd, 0xa0, 0xf3, 0xee, 0xc9, 0xd4, 0x6f, 0x72, 0x55, 0x48, 0x1b, 0x6, 0x21, 0x3c, 0x4a, 0x57, 0x70, 0x6d, 0x3e, 0x23, 0x4, 0x19, 0xa2, 0xbf, 0x98, 0x85, 0xd6, 0xcb, 0xec, 0xf1, 0x13, 0xe, 0x29, 0x34, 0x67, 0x7a, 0x5d, 0x40, 0xfb, 0xe6, 0xc1, 0xdc, 0x8f, 0x92, 0xb5, 0xa8, 0xde, 0xc3, 0xe4, 0xf9, 0xaa, 0xb7, 0x90, 0x8d, 0x36, 0x2b, 0xc, 0x11, 0x42, 0x5f, 0x78, 0x65, 0x94, 0x89, 0xae, 0xb3, 0xe0, 0xfd, 0xda, 0xc7, 0x7c, 0x61, 0x46, 0x5b, 0x8, 0x15, 0x32, 0x2f, 0x59, 0x44, 0x63, 0x7e, 0x2d, 0x30, 0x17, 0xa, 0xb1, 0xac, 0x8b, 0x96, 0xc5, 0xd8, 0xff, 0xe2, 0x26, 0x3b, 0x1c, 0x1, 0x52, 0x4f, 0x68, 0x75, 0xce, 0xd3, 0xf4, 0xe9, 0xba, 0xa7, 0x80, 0x9d, 0xeb, 0xf6, 0xd1, 0xcc, 0x9f, 0x82, 0xa5, 0xb8, 0x3, 0x1e, 0x39, 0x24, 0x77, 0x6a, 0x4d, 0x50, 0xa1, 0xbc, 0x9b, 0x86, 0xd5, 0xc8, 0xef, 0xf2, 0x49, 0x54, 0x73, 0x6e, 0x3d, 0x20, 0x7, 0x1a, 0x6c, 0x71, 0x56, 0x4b, 0x18, 0x5, 0x22, 0x3f, 0x84, 0x99, 0xbe, 0xa3, 0xf0, 0xed, 0xca, 0xd7, 0x35, 0x28, 0xf, 0x12, 0x41, 0x5c, 0x7b, 0x66, 0xdd, 0xc0, 0xe7, 0xfa, 0xa9, 0xb4, 0x93, 0x8e, 0xf8, 0xe5, 0xc2, 0xdf, 0x8c, 0x91, 0xb6, 0xab, 0x10, 0xd, 0x2a, 0x37, 0x64, 0x79, 0x5e, 0x43, 0xb2, 0xaf, 0x88, 0x95, 0xc6, 0xdb, 0xfc, 0xe1, 0x5a, 0x47, 0x60, 0x7d, 0x2e, 0x33, 0x14, 0x9, 0x7f, 0x62, 0x45, 0x58, 0xb, 0x16, 0x31, 0x2c, 0x97, 0x8a, 0xad, 0xb0, 0xe3, 0xfe, 0xd9, 0xc4},
- {0x0, 0x1e, 0x3c, 0x22, 0x78, 0x66, 0x44, 0x5a, 0xf0, 0xee, 0xcc, 0xd2, 0x88, 0x96, 0xb4, 0xaa, 0xfd, 0xe3, 0xc1, 0xdf, 0x85, 0x9b, 0xb9, 0xa7, 0xd, 0x13, 0x31, 0x2f, 0x75, 0x6b, 0x49, 0x57, 0xe7, 0xf9, 0xdb, 0xc5, 0x9f, 0x81, 0xa3, 0xbd, 0x17, 0x9, 0x2b, 0x35, 0x6f, 0x71, 0x53, 0x4d, 0x1a, 0x4, 0x26, 0x38, 0x62, 0x7c, 0x5e, 0x40, 0xea, 0xf4, 0xd6, 0xc8, 0x92, 0x8c, 0xae, 0xb0, 0xd3, 0xcd, 0xef, 0xf1, 0xab, 0xb5, 0x97, 0x89, 0x23, 0x3d, 0x1f, 0x1, 0x5b, 0x45, 0x67, 0x79, 0x2e, 0x30, 0x12, 0xc, 0x56, 0x48, 0x6a, 0x74, 0xde, 0xc0, 0xe2, 0xfc, 0xa6, 0xb8, 0x9a, 0x84, 0x34, 0x2a, 0x8, 0x16, 0x4c, 0x52, 0x70, 0x6e, 0xc4, 0xda, 0xf8, 0xe6, 0xbc, 0xa2, 0x80, 0x9e, 0xc9, 0xd7, 0xf5, 0xeb, 0xb1, 0xaf, 0x8d, 0x93, 0x39, 0x27, 0x5, 0x1b, 0x41, 0x5f, 0x7d, 0x63, 0xbb, 0xa5, 0x87, 0x99, 0xc3, 0xdd, 0xff, 0xe1, 0x4b, 0x55, 0x77, 0x69, 0x33, 0x2d, 0xf, 0x11, 0x46, 0x58, 0x7a, 0x64, 0x3e, 0x20, 0x2, 0x1c, 0xb6, 0xa8, 0x8a, 0x94, 0xce, 0xd0, 0xf2, 0xec, 0x5c, 0x42, 0x60, 0x7e, 0x24, 0x3a, 0x18, 0x6, 0xac, 0xb2, 0x90, 0x8e, 0xd4, 0xca, 0xe8, 0xf6, 0xa1, 0xbf, 0x9d, 0x83, 0xd9, 0xc7, 0xe5, 0xfb, 0x51, 0x4f, 0x6d, 0x73, 0x29, 0x37, 0x15, 0xb, 0x68, 0x76, 0x54, 0x4a, 0x10, 0xe, 0x2c, 0x32, 0x98, 0x86, 0xa4, 0xba, 0xe0, 0xfe, 0xdc, 0xc2, 0x95, 0x8b, 0xa9, 0xb7, 0xed, 0xf3, 0xd1, 0xcf, 0x65, 0x7b, 0x59, 0x47, 0x1d, 0x3, 0x21, 0x3f, 0x8f, 0x91, 0xb3, 0xad, 0xf7, 0xe9, 0xcb, 0xd5, 0x7f, 0x61, 0x43, 0x5d, 0x7, 0x19, 0x3b, 0x25, 0x72, 0x6c, 0x4e, 0x50, 0xa, 0x14, 0x36, 0x28, 0x82, 0x9c, 0xbe, 0xa0, 0xfa, 0xe4, 0xc6, 0xd8},
- {0x0, 0x1f, 0x3e, 0x21, 0x7c, 0x63, 0x42, 0x5d, 0xf8, 0xe7, 0xc6, 0xd9, 0x84, 0x9b, 0xba, 0xa5, 0xed, 0xf2, 0xd3, 0xcc, 0x91, 0x8e, 0xaf, 0xb0, 0x15, 0xa, 0x2b, 0x34, 0x69, 0x76, 0x57, 0x48, 0xc7, 0xd8, 0xf9, 0xe6, 0xbb, 0xa4, 0x85, 0x9a, 0x3f, 0x20, 0x1, 0x1e, 0x43, 0x5c, 0x7d, 0x62, 0x2a, 0x35, 0x14, 0xb, 0x56, 0x49, 0x68, 0x77, 0xd2, 0xcd, 0xec, 0xf3, 0xae, 0xb1, 0x90, 0x8f, 0x93, 0x8c, 0xad, 0xb2, 0xef, 0xf0, 0xd1, 0xce, 0x6b, 0x74, 0x55, 0x4a, 0x17, 0x8, 0x29, 0x36, 0x7e, 0x61, 0x40, 0x5f, 0x2, 0x1d, 0x3c, 0x23, 0x86, 0x99, 0xb8, 0xa7, 0xfa, 0xe5, 0xc4, 0xdb, 0x54, 0x4b, 0x6a, 0x75, 0x28, 0x37, 0x16, 0x9, 0xac, 0xb3, 0x92, 0x8d, 0xd0, 0xcf, 0xee, 0xf1, 0xb9, 0xa6, 0x87, 0x98, 0xc5, 0xda, 0xfb, 0xe4, 0x41, 0x5e, 0x7f, 0x60, 0x3d, 0x22, 0x3, 0x1c, 0x3b, 0x24, 0x5, 0x1a, 0x47, 0x58, 0x79, 0x66, 0xc3, 0xdc, 0xfd, 0xe2, 0xbf, 0xa0, 0x81, 0x9e, 0xd6, 0xc9, 0xe8, 0xf7, 0xaa, 0xb5, 0x94, 0x8b, 0x2e, 0x31, 0x10, 0xf, 0x52, 0x4d, 0x6c, 0x73, 0xfc, 0xe3, 0xc2, 0xdd, 0x80, 0x9f, 0xbe, 0xa1, 0x4, 0x1b, 0x3a, 0x25, 0x78, 0x67, 0x46, 0x59, 0x11, 0xe, 0x2f, 0x30, 0x6d, 0x72, 0x53, 0x4c, 0xe9, 0xf6, 0xd7, 0xc8, 0x95, 0x8a, 0xab, 0xb4, 0xa8, 0xb7, 0x96, 0x89, 0xd4, 0xcb, 0xea, 0xf5, 0x50, 0x4f, 0x6e, 0x71, 0x2c, 0x33, 0x12, 0xd, 0x45, 0x5a, 0x7b, 0x64, 0x39, 0x26, 0x7, 0x18, 0xbd, 0xa2, 0x83, 0x9c, 0xc1, 0xde, 0xff, 0xe0, 0x6f, 0x70, 0x51, 0x4e, 0x13, 0xc, 0x2d, 0x32, 0x97, 0x88, 0xa9, 0xb6, 0xeb, 0xf4, 0xd5, 0xca, 0x82, 0x9d, 0xbc, 0xa3, 0xfe, 0xe1, 0xc0, 0xdf, 0x7a, 0x65, 0x44, 0x5b, 0x6, 0x19, 0x38, 0x27},
- {0x0, 0x20, 0x40, 0x60, 0x80, 0xa0, 0xc0, 0xe0, 0x1d, 0x3d, 0x5d, 0x7d, 0x9d, 0xbd, 0xdd, 0xfd, 0x3a, 0x1a, 0x7a, 0x5a, 0xba, 0x9a, 0xfa, 0xda, 0x27, 0x7, 0x67, 0x47, 0xa7, 0x87, 0xe7, 0xc7, 0x74, 0x54, 0x34, 0x14, 0xf4, 0xd4, 0xb4, 0x94, 0x69, 0x49, 0x29, 0x9, 0xe9, 0xc9, 0xa9, 0x89, 0x4e, 0x6e, 0xe, 0x2e, 0xce, 0xee, 0x8e, 0xae, 0x53, 0x73, 0x13, 0x33, 0xd3, 0xf3, 0x93, 0xb3, 0xe8, 0xc8, 0xa8, 0x88, 0x68, 0x48, 0x28, 0x8, 0xf5, 0xd5, 0xb5, 0x95, 0x75, 0x55, 0x35, 0x15, 0xd2, 0xf2, 0x92, 0xb2, 0x52, 0x72, 0x12, 0x32, 0xcf, 0xef, 0x8f, 0xaf, 0x4f, 0x6f, 0xf, 0x2f, 0x9c, 0xbc, 0xdc, 0xfc, 0x1c, 0x3c, 0x5c, 0x7c, 0x81, 0xa1, 0xc1, 0xe1, 0x1, 0x21, 0x41, 0x61, 0xa6, 0x86, 0xe6, 0xc6, 0x26, 0x6, 0x66, 0x46, 0xbb, 0x9b, 0xfb, 0xdb, 0x3b, 0x1b, 0x7b, 0x5b, 0xcd, 0xed, 0x8d, 0xad, 0x4d, 0x6d, 0xd, 0x2d, 0xd0, 0xf0, 0x90, 0xb0, 0x50, 0x70, 0x10, 0x30, 0xf7, 0xd7, 0xb7, 0x97, 0x77, 0x57, 0x37, 0x17, 0xea, 0xca, 0xaa, 0x8a, 0x6a, 0x4a, 0x2a, 0xa, 0xb9, 0x99, 0xf9, 0xd9, 0x39, 0x19, 0x79, 0x59, 0xa4, 0x84, 0xe4, 0xc4, 0x24, 0x4, 0x64, 0x44, 0x83, 0xa3, 0xc3, 0xe3, 0x3, 0x23, 0x43, 0x63, 0x9e, 0xbe, 0xde, 0xfe, 0x1e, 0x3e, 0x5e, 0x7e, 0x25, 0x5, 0x65, 0x45, 0xa5, 0x85, 0xe5, 0xc5, 0x38, 0x18, 0x78, 0x58, 0xb8, 0x98, 0xf8, 0xd8, 0x1f, 0x3f, 0x5f, 0x7f, 0x9f, 0xbf, 0xdf, 0xff, 0x2, 0x22, 0x42, 0x62, 0x82, 0xa2, 0xc2, 0xe2, 0x51, 0x71, 0x11, 0x31, 0xd1, 0xf1, 0x91, 0xb1, 0x4c, 0x6c, 0xc, 0x2c, 0xcc, 0xec, 0x8c, 0xac, 0x6b, 0x4b, 0x2b, 0xb, 0xeb, 0xcb, 0xab, 0x8b, 0x76, 0x56, 0x36, 0x16, 0xf6, 0xd6, 0xb6, 0x96},
- {0x0, 0x21, 0x42, 0x63, 0x84, 0xa5, 0xc6, 0xe7, 0x15, 0x34, 0x57, 0x76, 0x91, 0xb0, 0xd3, 0xf2, 0x2a, 0xb, 0x68, 0x49, 0xae, 0x8f, 0xec, 0xcd, 0x3f, 0x1e, 0x7d, 0x5c, 0xbb, 0x9a, 0xf9, 0xd8, 0x54, 0x75, 0x16, 0x37, 0xd0, 0xf1, 0x92, 0xb3, 0x41, 0x60, 0x3, 0x22, 0xc5, 0xe4, 0x87, 0xa6, 0x7e, 0x5f, 0x3c, 0x1d, 0xfa, 0xdb, 0xb8, 0x99, 0x6b, 0x4a, 0x29, 0x8, 0xef, 0xce, 0xad, 0x8c, 0xa8, 0x89, 0xea, 0xcb, 0x2c, 0xd, 0x6e, 0x4f, 0xbd, 0x9c, 0xff, 0xde, 0x39, 0x18, 0x7b, 0x5a, 0x82, 0xa3, 0xc0, 0xe1, 0x6, 0x27, 0x44, 0x65, 0x97, 0xb6, 0xd5, 0xf4, 0x13, 0x32, 0x51, 0x70, 0xfc, 0xdd, 0xbe, 0x9f, 0x78, 0x59, 0x3a, 0x1b, 0xe9, 0xc8, 0xab, 0x8a, 0x6d, 0x4c, 0x2f, 0xe, 0xd6, 0xf7, 0x94, 0xb5, 0x52, 0x73, 0x10, 0x31, 0xc3, 0xe2, 0x81, 0xa0, 0x47, 0x66, 0x5, 0x24, 0x4d, 0x6c, 0xf, 0x2e, 0xc9, 0xe8, 0x8b, 0xaa, 0x58, 0x79, 0x1a, 0x3b, 0xdc, 0xfd, 0x9e, 0xbf, 0x67, 0x46, 0x25, 0x4, 0xe3, 0xc2, 0xa1, 0x80, 0x72, 0x53, 0x30, 0x11, 0xf6, 0xd7, 0xb4, 0x95, 0x19, 0x38, 0x5b, 0x7a, 0x9d, 0xbc, 0xdf, 0xfe, 0xc, 0x2d, 0x4e, 0x6f, 0x88, 0xa9, 0xca, 0xeb, 0x33, 0x12, 0x71, 0x50, 0xb7, 0x96, 0xf5, 0xd4, 0x26, 0x7, 0x64, 0x45, 0xa2, 0x83, 0xe0, 0xc1, 0xe5, 0xc4, 0xa7, 0x86, 0x61, 0x40, 0x23, 0x2, 0xf0, 0xd1, 0xb2, 0x93, 0x74, 0x55, 0x36, 0x17, 0xcf, 0xee, 0x8d, 0xac, 0x4b, 0x6a, 0x9, 0x28, 0xda, 0xfb, 0x98, 0xb9, 0x5e, 0x7f, 0x1c, 0x3d, 0xb1, 0x90, 0xf3, 0xd2, 0x35, 0x14, 0x77, 0x56, 0xa4, 0x85, 0xe6, 0xc7, 0x20, 0x1, 0x62, 0x43, 0x9b, 0xba, 0xd9, 0xf8, 0x1f, 0x3e, 0x5d, 0x7c, 0x8e, 0xaf, 0xcc, 0xed, 0xa, 0x2b, 0x48, 0x69},
- {0x0, 0x22, 0x44, 0x66, 0x88, 0xaa, 0xcc, 0xee, 0xd, 0x2f, 0x49, 0x6b, 0x85, 0xa7, 0xc1, 0xe3, 0x1a, 0x38, 0x5e, 0x7c, 0x92, 0xb0, 0xd6, 0xf4, 0x17, 0x35, 0x53, 0x71, 0x9f, 0xbd, 0xdb, 0xf9, 0x34, 0x16, 0x70, 0x52, 0xbc, 0x9e, 0xf8, 0xda, 0x39, 0x1b, 0x7d, 0x5f, 0xb1, 0x93, 0xf5, 0xd7, 0x2e, 0xc, 0x6a, 0x48, 0xa6, 0x84, 0xe2, 0xc0, 0x23, 0x1, 0x67, 0x45, 0xab, 0x89, 0xef, 0xcd, 0x68, 0x4a, 0x2c, 0xe, 0xe0, 0xc2, 0xa4, 0x86, 0x65, 0x47, 0x21, 0x3, 0xed, 0xcf, 0xa9, 0x8b, 0x72, 0x50, 0x36, 0x14, 0xfa, 0xd8, 0xbe, 0x9c, 0x7f, 0x5d, 0x3b, 0x19, 0xf7, 0xd5, 0xb3, 0x91, 0x5c, 0x7e, 0x18, 0x3a, 0xd4, 0xf6, 0x90, 0xb2, 0x51, 0x73, 0x15, 0x37, 0xd9, 0xfb, 0x9d, 0xbf, 0x46, 0x64, 0x2, 0x20, 0xce, 0xec, 0x8a, 0xa8, 0x4b, 0x69, 0xf, 0x2d, 0xc3, 0xe1, 0x87, 0xa5, 0xd0, 0xf2, 0x94, 0xb6, 0x58, 0x7a, 0x1c, 0x3e, 0xdd, 0xff, 0x99, 0xbb, 0x55, 0x77, 0x11, 0x33, 0xca, 0xe8, 0x8e, 0xac, 0x42, 0x60, 0x6, 0x24, 0xc7, 0xe5, 0x83, 0xa1, 0x4f, 0x6d, 0xb, 0x29, 0xe4, 0xc6, 0xa0, 0x82, 0x6c, 0x4e, 0x28, 0xa, 0xe9, 0xcb, 0xad, 0x8f, 0x61, 0x43, 0x25, 0x7, 0xfe, 0xdc, 0xba, 0x98, 0x76, 0x54, 0x32, 0x10, 0xf3, 0xd1, 0xb7, 0x95, 0x7b, 0x59, 0x3f, 0x1d, 0xb8, 0x9a, 0xfc, 0xde, 0x30, 0x12, 0x74, 0x56, 0xb5, 0x97, 0xf1, 0xd3, 0x3d, 0x1f, 0x79, 0x5b, 0xa2, 0x80, 0xe6, 0xc4, 0x2a, 0x8, 0x6e, 0x4c, 0xaf, 0x8d, 0xeb, 0xc9, 0x27, 0x5, 0x63, 0x41, 0x8c, 0xae, 0xc8, 0xea, 0x4, 0x26, 0x40, 0x62, 0x81, 0xa3, 0xc5, 0xe7, 0x9, 0x2b, 0x4d, 0x6f, 0x96, 0xb4, 0xd2, 0xf0, 0x1e, 0x3c, 0x5a, 0x78, 0x9b, 0xb9, 0xdf, 0xfd, 0x13, 0x31, 0x57, 0x75},
- {0x0, 0x23, 0x46, 0x65, 0x8c, 0xaf, 0xca, 0xe9, 0x5, 0x26, 0x43, 0x60, 0x89, 0xaa, 0xcf, 0xec, 0xa, 0x29, 0x4c, 0x6f, 0x86, 0xa5, 0xc0, 0xe3, 0xf, 0x2c, 0x49, 0x6a, 0x83, 0xa0, 0xc5, 0xe6, 0x14, 0x37, 0x52, 0x71, 0x98, 0xbb, 0xde, 0xfd, 0x11, 0x32, 0x57, 0x74, 0x9d, 0xbe, 0xdb, 0xf8, 0x1e, 0x3d, 0x58, 0x7b, 0x92, 0xb1, 0xd4, 0xf7, 0x1b, 0x38, 0x5d, 0x7e, 0x97, 0xb4, 0xd1, 0xf2, 0x28, 0xb, 0x6e, 0x4d, 0xa4, 0x87, 0xe2, 0xc1, 0x2d, 0xe, 0x6b, 0x48, 0xa1, 0x82, 0xe7, 0xc4, 0x22, 0x1, 0x64, 0x47, 0xae, 0x8d, 0xe8, 0xcb, 0x27, 0x4, 0x61, 0x42, 0xab, 0x88, 0xed, 0xce, 0x3c, 0x1f, 0x7a, 0x59, 0xb0, 0x93, 0xf6, 0xd5, 0x39, 0x1a, 0x7f, 0x5c, 0xb5, 0x96, 0xf3, 0xd0, 0x36, 0x15, 0x70, 0x53, 0xba, 0x99, 0xfc, 0xdf, 0x33, 0x10, 0x75, 0x56, 0xbf, 0x9c, 0xf9, 0xda, 0x50, 0x73, 0x16, 0x35, 0xdc, 0xff, 0x9a, 0xb9, 0x55, 0x76, 0x13, 0x30, 0xd9, 0xfa, 0x9f, 0xbc, 0x5a, 0x79, 0x1c, 0x3f, 0xd6, 0xf5, 0x90, 0xb3, 0x5f, 0x7c, 0x19, 0x3a, 0xd3, 0xf0, 0x95, 0xb6, 0x44, 0x67, 0x2, 0x21, 0xc8, 0xeb, 0x8e, 0xad, 0x41, 0x62, 0x7, 0x24, 0xcd, 0xee, 0x8b, 0xa8, 0x4e, 0x6d, 0x8, 0x2b, 0xc2, 0xe1, 0x84, 0xa7, 0x4b, 0x68, 0xd, 0x2e, 0xc7, 0xe4, 0x81, 0xa2, 0x78, 0x5b, 0x3e, 0x1d, 0xf4, 0xd7, 0xb2, 0x91, 0x7d, 0x5e, 0x3b, 0x18, 0xf1, 0xd2, 0xb7, 0x94, 0x72, 0x51, 0x34, 0x17, 0xfe, 0xdd, 0xb8, 0x9b, 0x77, 0x54, 0x31, 0x12, 0xfb, 0xd8, 0xbd, 0x9e, 0x6c, 0x4f, 0x2a, 0x9, 0xe0, 0xc3, 0xa6, 0x85, 0x69, 0x4a, 0x2f, 0xc, 0xe5, 0xc6, 0xa3, 0x80, 0x66, 0x45, 0x20, 0x3, 0xea, 0xc9, 0xac, 0x8f, 0x63, 0x40, 0x25, 0x6, 0xef, 0xcc, 0xa9, 0x8a},
- {0x0, 0x24, 0x48, 0x6c, 0x90, 0xb4, 0xd8, 0xfc, 0x3d, 0x19, 0x75, 0x51, 0xad, 0x89, 0xe5, 0xc1, 0x7a, 0x5e, 0x32, 0x16, 0xea, 0xce, 0xa2, 0x86, 0x47, 0x63, 0xf, 0x2b, 0xd7, 0xf3, 0x9f, 0xbb, 0xf4, 0xd0, 0xbc, 0x98, 0x64, 0x40, 0x2c, 0x8, 0xc9, 0xed, 0x81, 0xa5, 0x59, 0x7d, 0x11, 0x35, 0x8e, 0xaa, 0xc6, 0xe2, 0x1e, 0x3a, 0x56, 0x72, 0xb3, 0x97, 0xfb, 0xdf, 0x23, 0x7, 0x6b, 0x4f, 0xf5, 0xd1, 0xbd, 0x99, 0x65, 0x41, 0x2d, 0x9, 0xc8, 0xec, 0x80, 0xa4, 0x58, 0x7c, 0x10, 0x34, 0x8f, 0xab, 0xc7, 0xe3, 0x1f, 0x3b, 0x57, 0x73, 0xb2, 0x96, 0xfa, 0xde, 0x22, 0x6, 0x6a, 0x4e, 0x1, 0x25, 0x49, 0x6d, 0x91, 0xb5, 0xd9, 0xfd, 0x3c, 0x18, 0x74, 0x50, 0xac, 0x88, 0xe4, 0xc0, 0x7b, 0x5f, 0x33, 0x17, 0xeb, 0xcf, 0xa3, 0x87, 0x46, 0x62, 0xe, 0x2a, 0xd6, 0xf2, 0x9e, 0xba, 0xf7, 0xd3, 0xbf, 0x9b, 0x67, 0x43, 0x2f, 0xb, 0xca, 0xee, 0x82, 0xa6, 0x5a, 0x7e, 0x12, 0x36, 0x8d, 0xa9, 0xc5, 0xe1, 0x1d, 0x39, 0x55, 0x71, 0xb0, 0x94, 0xf8, 0xdc, 0x20, 0x4, 0x68, 0x4c, 0x3, 0x27, 0x4b, 0x6f, 0x93, 0xb7, 0xdb, 0xff, 0x3e, 0x1a, 0x76, 0x52, 0xae, 0x8a, 0xe6, 0xc2, 0x79, 0x5d, 0x31, 0x15, 0xe9, 0xcd, 0xa1, 0x85, 0x44, 0x60, 0xc, 0x28, 0xd4, 0xf0, 0x9c, 0xb8, 0x2, 0x26, 0x4a, 0x6e, 0x92, 0xb6, 0xda, 0xfe, 0x3f, 0x1b, 0x77, 0x53, 0xaf, 0x8b, 0xe7, 0xc3, 0x78, 0x5c, 0x30, 0x14, 0xe8, 0xcc, 0xa0, 0x84, 0x45, 0x61, 0xd, 0x29, 0xd5, 0xf1, 0x9d, 0xb9, 0xf6, 0xd2, 0xbe, 0x9a, 0x66, 0x42, 0x2e, 0xa, 0xcb, 0xef, 0x83, 0xa7, 0x5b, 0x7f, 0x13, 0x37, 0x8c, 0xa8, 0xc4, 0xe0, 0x1c, 0x38, 0x54, 0x70, 0xb1, 0x95, 0xf9, 0xdd, 0x21, 0x5, 0x69, 0x4d},
- {0x0, 0x25, 0x4a, 0x6f, 0x94, 0xb1, 0xde, 0xfb, 0x35, 0x10, 0x7f, 0x5a, 0xa1, 0x84, 0xeb, 0xce, 0x6a, 0x4f, 0x20, 0x5, 0xfe, 0xdb, 0xb4, 0x91, 0x5f, 0x7a, 0x15, 0x30, 0xcb, 0xee, 0x81, 0xa4, 0xd4, 0xf1, 0x9e, 0xbb, 0x40, 0x65, 0xa, 0x2f, 0xe1, 0xc4, 0xab, 0x8e, 0x75, 0x50, 0x3f, 0x1a, 0xbe, 0x9b, 0xf4, 0xd1, 0x2a, 0xf, 0x60, 0x45, 0x8b, 0xae, 0xc1, 0xe4, 0x1f, 0x3a, 0x55, 0x70, 0xb5, 0x90, 0xff, 0xda, 0x21, 0x4, 0x6b, 0x4e, 0x80, 0xa5, 0xca, 0xef, 0x14, 0x31, 0x5e, 0x7b, 0xdf, 0xfa, 0x95, 0xb0, 0x4b, 0x6e, 0x1, 0x24, 0xea, 0xcf, 0xa0, 0x85, 0x7e, 0x5b, 0x34, 0x11, 0x61, 0x44, 0x2b, 0xe, 0xf5, 0xd0, 0xbf, 0x9a, 0x54, 0x71, 0x1e, 0x3b, 0xc0, 0xe5, 0x8a, 0xaf, 0xb, 0x2e, 0x41, 0x64, 0x9f, 0xba, 0xd5, 0xf0, 0x3e, 0x1b, 0x74, 0x51, 0xaa, 0x8f, 0xe0, 0xc5, 0x77, 0x52, 0x3d, 0x18, 0xe3, 0xc6, 0xa9, 0x8c, 0x42, 0x67, 0x8, 0x2d, 0xd6, 0xf3, 0x9c, 0xb9, 0x1d, 0x38, 0x57, 0x72, 0x89, 0xac, 0xc3, 0xe6, 0x28, 0xd, 0x62, 0x47, 0xbc, 0x99, 0xf6, 0xd3, 0xa3, 0x86, 0xe9, 0xcc, 0x37, 0x12, 0x7d, 0x58, 0x96, 0xb3, 0xdc, 0xf9, 0x2, 0x27, 0x48, 0x6d, 0xc9, 0xec, 0x83, 0xa6, 0x5d, 0x78, 0x17, 0x32, 0xfc, 0xd9, 0xb6, 0x93, 0x68, 0x4d, 0x22, 0x7, 0xc2, 0xe7, 0x88, 0xad, 0x56, 0x73, 0x1c, 0x39, 0xf7, 0xd2, 0xbd, 0x98, 0x63, 0x46, 0x29, 0xc, 0xa8, 0x8d, 0xe2, 0xc7, 0x3c, 0x19, 0x76, 0x53, 0x9d, 0xb8, 0xd7, 0xf2, 0x9, 0x2c, 0x43, 0x66, 0x16, 0x33, 0x5c, 0x79, 0x82, 0xa7, 0xc8, 0xed, 0x23, 0x6, 0x69, 0x4c, 0xb7, 0x92, 0xfd, 0xd8, 0x7c, 0x59, 0x36, 0x13, 0xe8, 0xcd, 0xa2, 0x87, 0x49, 0x6c, 0x3, 0x26, 0xdd, 0xf8, 0x97, 0xb2},
- {0x0, 0x26, 0x4c, 0x6a, 0x98, 0xbe, 0xd4, 0xf2, 0x2d, 0xb, 0x61, 0x47, 0xb5, 0x93, 0xf9, 0xdf, 0x5a, 0x7c, 0x16, 0x30, 0xc2, 0xe4, 0x8e, 0xa8, 0x77, 0x51, 0x3b, 0x1d, 0xef, 0xc9, 0xa3, 0x85, 0xb4, 0x92, 0xf8, 0xde, 0x2c, 0xa, 0x60, 0x46, 0x99, 0xbf, 0xd5, 0xf3, 0x1, 0x27, 0x4d, 0x6b, 0xee, 0xc8, 0xa2, 0x84, 0x76, 0x50, 0x3a, 0x1c, 0xc3, 0xe5, 0x8f, 0xa9, 0x5b, 0x7d, 0x17, 0x31, 0x75, 0x53, 0x39, 0x1f, 0xed, 0xcb, 0xa1, 0x87, 0x58, 0x7e, 0x14, 0x32, 0xc0, 0xe6, 0x8c, 0xaa, 0x2f, 0x9, 0x63, 0x45, 0xb7, 0x91, 0xfb, 0xdd, 0x2, 0x24, 0x4e, 0x68, 0x9a, 0xbc, 0xd6, 0xf0, 0xc1, 0xe7, 0x8d, 0xab, 0x59, 0x7f, 0x15, 0x33, 0xec, 0xca, 0xa0, 0x86, 0x74, 0x52, 0x38, 0x1e, 0x9b, 0xbd, 0xd7, 0xf1, 0x3, 0x25, 0x4f, 0x69, 0xb6, 0x90, 0xfa, 0xdc, 0x2e, 0x8, 0x62, 0x44, 0xea, 0xcc, 0xa6, 0x80, 0x72, 0x54, 0x3e, 0x18, 0xc7, 0xe1, 0x8b, 0xad, 0x5f, 0x79, 0x13, 0x35, 0xb0, 0x96, 0xfc, 0xda, 0x28, 0xe, 0x64, 0x42, 0x9d, 0xbb, 0xd1, 0xf7, 0x5, 0x23, 0x49, 0x6f, 0x5e, 0x78, 0x12, 0x34, 0xc6, 0xe0, 0x8a, 0xac, 0x73, 0x55, 0x3f, 0x19, 0xeb, 0xcd, 0xa7, 0x81, 0x4, 0x22, 0x48, 0x6e, 0x9c, 0xba, 0xd0, 0xf6, 0x29, 0xf, 0x65, 0x43, 0xb1, 0x97, 0xfd, 0xdb, 0x9f, 0xb9, 0xd3, 0xf5, 0x7, 0x21, 0x4b, 0x6d, 0xb2, 0x94, 0xfe, 0xd8, 0x2a, 0xc, 0x66, 0x40, 0xc5, 0xe3, 0x89, 0xaf, 0x5d, 0x7b, 0x11, 0x37, 0xe8, 0xce, 0xa4, 0x82, 0x70, 0x56, 0x3c, 0x1a, 0x2b, 0xd, 0x67, 0x41, 0xb3, 0x95, 0xff, 0xd9, 0x6, 0x20, 0x4a, 0x6c, 0x9e, 0xb8, 0xd2, 0xf4, 0x71, 0x57, 0x3d, 0x1b, 0xe9, 0xcf, 0xa5, 0x83, 0x5c, 0x7a, 0x10, 0x36, 0xc4, 0xe2, 0x88, 0xae},
- {0x0, 0x27, 0x4e, 0x69, 0x9c, 0xbb, 0xd2, 0xf5, 0x25, 0x2, 0x6b, 0x4c, 0xb9, 0x9e, 0xf7, 0xd0, 0x4a, 0x6d, 0x4, 0x23, 0xd6, 0xf1, 0x98, 0xbf, 0x6f, 0x48, 0x21, 0x6, 0xf3, 0xd4, 0xbd, 0x9a, 0x94, 0xb3, 0xda, 0xfd, 0x8, 0x2f, 0x46, 0x61, 0xb1, 0x96, 0xff, 0xd8, 0x2d, 0xa, 0x63, 0x44, 0xde, 0xf9, 0x90, 0xb7, 0x42, 0x65, 0xc, 0x2b, 0xfb, 0xdc, 0xb5, 0x92, 0x67, 0x40, 0x29, 0xe, 0x35, 0x12, 0x7b, 0x5c, 0xa9, 0x8e, 0xe7, 0xc0, 0x10, 0x37, 0x5e, 0x79, 0x8c, 0xab, 0xc2, 0xe5, 0x7f, 0x58, 0x31, 0x16, 0xe3, 0xc4, 0xad, 0x8a, 0x5a, 0x7d, 0x14, 0x33, 0xc6, 0xe1, 0x88, 0xaf, 0xa1, 0x86, 0xef, 0xc8, 0x3d, 0x1a, 0x73, 0x54, 0x84, 0xa3, 0xca, 0xed, 0x18, 0x3f, 0x56, 0x71, 0xeb, 0xcc, 0xa5, 0x82, 0x77, 0x50, 0x39, 0x1e, 0xce, 0xe9, 0x80, 0xa7, 0x52, 0x75, 0x1c, 0x3b, 0x6a, 0x4d, 0x24, 0x3, 0xf6, 0xd1, 0xb8, 0x9f, 0x4f, 0x68, 0x1, 0x26, 0xd3, 0xf4, 0x9d, 0xba, 0x20, 0x7, 0x6e, 0x49, 0xbc, 0x9b, 0xf2, 0xd5, 0x5, 0x22, 0x4b, 0x6c, 0x99, 0xbe, 0xd7, 0xf0, 0xfe, 0xd9, 0xb0, 0x97, 0x62, 0x45, 0x2c, 0xb, 0xdb, 0xfc, 0x95, 0xb2, 0x47, 0x60, 0x9, 0x2e, 0xb4, 0x93, 0xfa, 0xdd, 0x28, 0xf, 0x66, 0x41, 0x91, 0xb6, 0xdf, 0xf8, 0xd, 0x2a, 0x43, 0x64, 0x5f, 0x78, 0x11, 0x36, 0xc3, 0xe4, 0x8d, 0xaa, 0x7a, 0x5d, 0x34, 0x13, 0xe6, 0xc1, 0xa8, 0x8f, 0x15, 0x32, 0x5b, 0x7c, 0x89, 0xae, 0xc7, 0xe0, 0x30, 0x17, 0x7e, 0x59, 0xac, 0x8b, 0xe2, 0xc5, 0xcb, 0xec, 0x85, 0xa2, 0x57, 0x70, 0x19, 0x3e, 0xee, 0xc9, 0xa0, 0x87, 0x72, 0x55, 0x3c, 0x1b, 0x81, 0xa6, 0xcf, 0xe8, 0x1d, 0x3a, 0x53, 0x74, 0xa4, 0x83, 0xea, 0xcd, 0x38, 0x1f, 0x76, 0x51},
- {0x0, 0x28, 0x50, 0x78, 0xa0, 0x88, 0xf0, 0xd8, 0x5d, 0x75, 0xd, 0x25, 0xfd, 0xd5, 0xad, 0x85, 0xba, 0x92, 0xea, 0xc2, 0x1a, 0x32, 0x4a, 0x62, 0xe7, 0xcf, 0xb7, 0x9f, 0x47, 0x6f, 0x17, 0x3f, 0x69, 0x41, 0x39, 0x11, 0xc9, 0xe1, 0x99, 0xb1, 0x34, 0x1c, 0x64, 0x4c, 0x94, 0xbc, 0xc4, 0xec, 0xd3, 0xfb, 0x83, 0xab, 0x73, 0x5b, 0x23, 0xb, 0x8e, 0xa6, 0xde, 0xf6, 0x2e, 0x6, 0x7e, 0x56, 0xd2, 0xfa, 0x82, 0xaa, 0x72, 0x5a, 0x22, 0xa, 0x8f, 0xa7, 0xdf, 0xf7, 0x2f, 0x7, 0x7f, 0x57, 0x68, 0x40, 0x38, 0x10, 0xc8, 0xe0, 0x98, 0xb0, 0x35, 0x1d, 0x65, 0x4d, 0x95, 0xbd, 0xc5, 0xed, 0xbb, 0x93, 0xeb, 0xc3, 0x1b, 0x33, 0x4b, 0x63, 0xe6, 0xce, 0xb6, 0x9e, 0x46, 0x6e, 0x16, 0x3e, 0x1, 0x29, 0x51, 0x79, 0xa1, 0x89, 0xf1, 0xd9, 0x5c, 0x74, 0xc, 0x24, 0xfc, 0xd4, 0xac, 0x84, 0xb9, 0x91, 0xe9, 0xc1, 0x19, 0x31, 0x49, 0x61, 0xe4, 0xcc, 0xb4, 0x9c, 0x44, 0x6c, 0x14, 0x3c, 0x3, 0x2b, 0x53, 0x7b, 0xa3, 0x8b, 0xf3, 0xdb, 0x5e, 0x76, 0xe, 0x26, 0xfe, 0xd6, 0xae, 0x86, 0xd0, 0xf8, 0x80, 0xa8, 0x70, 0x58, 0x20, 0x8, 0x8d, 0xa5, 0xdd, 0xf5, 0x2d, 0x5, 0x7d, 0x55, 0x6a, 0x42, 0x3a, 0x12, 0xca, 0xe2, 0x9a, 0xb2, 0x37, 0x1f, 0x67, 0x4f, 0x97, 0xbf, 0xc7, 0xef, 0x6b, 0x43, 0x3b, 0x13, 0xcb, 0xe3, 0x9b, 0xb3, 0x36, 0x1e, 0x66, 0x4e, 0x96, 0xbe, 0xc6, 0xee, 0xd1, 0xf9, 0x81, 0xa9, 0x71, 0x59, 0x21, 0x9, 0x8c, 0xa4, 0xdc, 0xf4, 0x2c, 0x4, 0x7c, 0x54, 0x2, 0x2a, 0x52, 0x7a, 0xa2, 0x8a, 0xf2, 0xda, 0x5f, 0x77, 0xf, 0x27, 0xff, 0xd7, 0xaf, 0x87, 0xb8, 0x90, 0xe8, 0xc0, 0x18, 0x30, 0x48, 0x60, 0xe5, 0xcd, 0xb5, 0x9d, 0x45, 0x6d, 0x15, 0x3d},
- {0x0, 0x29, 0x52, 0x7b, 0xa4, 0x8d, 0xf6, 0xdf, 0x55, 0x7c, 0x7, 0x2e, 0xf1, 0xd8, 0xa3, 0x8a, 0xaa, 0x83, 0xf8, 0xd1, 0xe, 0x27, 0x5c, 0x75, 0xff, 0xd6, 0xad, 0x84, 0x5b, 0x72, 0x9, 0x20, 0x49, 0x60, 0x1b, 0x32, 0xed, 0xc4, 0xbf, 0x96, 0x1c, 0x35, 0x4e, 0x67, 0xb8, 0x91, 0xea, 0xc3, 0xe3, 0xca, 0xb1, 0x98, 0x47, 0x6e, 0x15, 0x3c, 0xb6, 0x9f, 0xe4, 0xcd, 0x12, 0x3b, 0x40, 0x69, 0x92, 0xbb, 0xc0, 0xe9, 0x36, 0x1f, 0x64, 0x4d, 0xc7, 0xee, 0x95, 0xbc, 0x63, 0x4a, 0x31, 0x18, 0x38, 0x11, 0x6a, 0x43, 0x9c, 0xb5, 0xce, 0xe7, 0x6d, 0x44, 0x3f, 0x16, 0xc9, 0xe0, 0x9b, 0xb2, 0xdb, 0xf2, 0x89, 0xa0, 0x7f, 0x56, 0x2d, 0x4, 0x8e, 0xa7, 0xdc, 0xf5, 0x2a, 0x3, 0x78, 0x51, 0x71, 0x58, 0x23, 0xa, 0xd5, 0xfc, 0x87, 0xae, 0x24, 0xd, 0x76, 0x5f, 0x80, 0xa9, 0xd2, 0xfb, 0x39, 0x10, 0x6b, 0x42, 0x9d, 0xb4, 0xcf, 0xe6, 0x6c, 0x45, 0x3e, 0x17, 0xc8, 0xe1, 0x9a, 0xb3, 0x93, 0xba, 0xc1, 0xe8, 0x37, 0x1e, 0x65, 0x4c, 0xc6, 0xef, 0x94, 0xbd, 0x62, 0x4b, 0x30, 0x19, 0x70, 0x59, 0x22, 0xb, 0xd4, 0xfd, 0x86, 0xaf, 0x25, 0xc, 0x77, 0x5e, 0x81, 0xa8, 0xd3, 0xfa, 0xda, 0xf3, 0x88, 0xa1, 0x7e, 0x57, 0x2c, 0x5, 0x8f, 0xa6, 0xdd, 0xf4, 0x2b, 0x2, 0x79, 0x50, 0xab, 0x82, 0xf9, 0xd0, 0xf, 0x26, 0x5d, 0x74, 0xfe, 0xd7, 0xac, 0x85, 0x5a, 0x73, 0x8, 0x21, 0x1, 0x28, 0x53, 0x7a, 0xa5, 0x8c, 0xf7, 0xde, 0x54, 0x7d, 0x6, 0x2f, 0xf0, 0xd9, 0xa2, 0x8b, 0xe2, 0xcb, 0xb0, 0x99, 0x46, 0x6f, 0x14, 0x3d, 0xb7, 0x9e, 0xe5, 0xcc, 0x13, 0x3a, 0x41, 0x68, 0x48, 0x61, 0x1a, 0x33, 0xec, 0xc5, 0xbe, 0x97, 0x1d, 0x34, 0x4f, 0x66, 0xb9, 0x90, 0xeb, 0xc2},
- {0x0, 0x2a, 0x54, 0x7e, 0xa8, 0x82, 0xfc, 0xd6, 0x4d, 0x67, 0x19, 0x33, 0xe5, 0xcf, 0xb1, 0x9b, 0x9a, 0xb0, 0xce, 0xe4, 0x32, 0x18, 0x66, 0x4c, 0xd7, 0xfd, 0x83, 0xa9, 0x7f, 0x55, 0x2b, 0x1, 0x29, 0x3, 0x7d, 0x57, 0x81, 0xab, 0xd5, 0xff, 0x64, 0x4e, 0x30, 0x1a, 0xcc, 0xe6, 0x98, 0xb2, 0xb3, 0x99, 0xe7, 0xcd, 0x1b, 0x31, 0x4f, 0x65, 0xfe, 0xd4, 0xaa, 0x80, 0x56, 0x7c, 0x2, 0x28, 0x52, 0x78, 0x6, 0x2c, 0xfa, 0xd0, 0xae, 0x84, 0x1f, 0x35, 0x4b, 0x61, 0xb7, 0x9d, 0xe3, 0xc9, 0xc8, 0xe2, 0x9c, 0xb6, 0x60, 0x4a, 0x34, 0x1e, 0x85, 0xaf, 0xd1, 0xfb, 0x2d, 0x7, 0x79, 0x53, 0x7b, 0x51, 0x2f, 0x5, 0xd3, 0xf9, 0x87, 0xad, 0x36, 0x1c, 0x62, 0x48, 0x9e, 0xb4, 0xca, 0xe0, 0xe1, 0xcb, 0xb5, 0x9f, 0x49, 0x63, 0x1d, 0x37, 0xac, 0x86, 0xf8, 0xd2, 0x4, 0x2e, 0x50, 0x7a, 0xa4, 0x8e, 0xf0, 0xda, 0xc, 0x26, 0x58, 0x72, 0xe9, 0xc3, 0xbd, 0x97, 0x41, 0x6b, 0x15, 0x3f, 0x3e, 0x14, 0x6a, 0x40, 0x96, 0xbc, 0xc2, 0xe8, 0x73, 0x59, 0x27, 0xd, 0xdb, 0xf1, 0x8f, 0xa5, 0x8d, 0xa7, 0xd9, 0xf3, 0x25, 0xf, 0x71, 0x5b, 0xc0, 0xea, 0x94, 0xbe, 0x68, 0x42, 0x3c, 0x16, 0x17, 0x3d, 0x43, 0x69, 0xbf, 0x95, 0xeb, 0xc1, 0x5a, 0x70, 0xe, 0x24, 0xf2, 0xd8, 0xa6, 0x8c, 0xf6, 0xdc, 0xa2, 0x88, 0x5e, 0x74, 0xa, 0x20, 0xbb, 0x91, 0xef, 0xc5, 0x13, 0x39, 0x47, 0x6d, 0x6c, 0x46, 0x38, 0x12, 0xc4, 0xee, 0x90, 0xba, 0x21, 0xb, 0x75, 0x5f, 0x89, 0xa3, 0xdd, 0xf7, 0xdf, 0xf5, 0x8b, 0xa1, 0x77, 0x5d, 0x23, 0x9, 0x92, 0xb8, 0xc6, 0xec, 0x3a, 0x10, 0x6e, 0x44, 0x45, 0x6f, 0x11, 0x3b, 0xed, 0xc7, 0xb9, 0x93, 0x8, 0x22, 0x5c, 0x76, 0xa0, 0x8a, 0xf4, 0xde},
- {0x0, 0x2b, 0x56, 0x7d, 0xac, 0x87, 0xfa, 0xd1, 0x45, 0x6e, 0x13, 0x38, 0xe9, 0xc2, 0xbf, 0x94, 0x8a, 0xa1, 0xdc, 0xf7, 0x26, 0xd, 0x70, 0x5b, 0xcf, 0xe4, 0x99, 0xb2, 0x63, 0x48, 0x35, 0x1e, 0x9, 0x22, 0x5f, 0x74, 0xa5, 0x8e, 0xf3, 0xd8, 0x4c, 0x67, 0x1a, 0x31, 0xe0, 0xcb, 0xb6, 0x9d, 0x83, 0xa8, 0xd5, 0xfe, 0x2f, 0x4, 0x79, 0x52, 0xc6, 0xed, 0x90, 0xbb, 0x6a, 0x41, 0x3c, 0x17, 0x12, 0x39, 0x44, 0x6f, 0xbe, 0x95, 0xe8, 0xc3, 0x57, 0x7c, 0x1, 0x2a, 0xfb, 0xd0, 0xad, 0x86, 0x98, 0xb3, 0xce, 0xe5, 0x34, 0x1f, 0x62, 0x49, 0xdd, 0xf6, 0x8b, 0xa0, 0x71, 0x5a, 0x27, 0xc, 0x1b, 0x30, 0x4d, 0x66, 0xb7, 0x9c, 0xe1, 0xca, 0x5e, 0x75, 0x8, 0x23, 0xf2, 0xd9, 0xa4, 0x8f, 0x91, 0xba, 0xc7, 0xec, 0x3d, 0x16, 0x6b, 0x40, 0xd4, 0xff, 0x82, 0xa9, 0x78, 0x53, 0x2e, 0x5, 0x24, 0xf, 0x72, 0x59, 0x88, 0xa3, 0xde, 0xf5, 0x61, 0x4a, 0x37, 0x1c, 0xcd, 0xe6, 0x9b, 0xb0, 0xae, 0x85, 0xf8, 0xd3, 0x2, 0x29, 0x54, 0x7f, 0xeb, 0xc0, 0xbd, 0x96, 0x47, 0x6c, 0x11, 0x3a, 0x2d, 0x6, 0x7b, 0x50, 0x81, 0xaa, 0xd7, 0xfc, 0x68, 0x43, 0x3e, 0x15, 0xc4, 0xef, 0x92, 0xb9, 0xa7, 0x8c, 0xf1, 0xda, 0xb, 0x20, 0x5d, 0x76, 0xe2, 0xc9, 0xb4, 0x9f, 0x4e, 0x65, 0x18, 0x33, 0x36, 0x1d, 0x60, 0x4b, 0x9a, 0xb1, 0xcc, 0xe7, 0x73, 0x58, 0x25, 0xe, 0xdf, 0xf4, 0x89, 0xa2, 0xbc, 0x97, 0xea, 0xc1, 0x10, 0x3b, 0x46, 0x6d, 0xf9, 0xd2, 0xaf, 0x84, 0x55, 0x7e, 0x3, 0x28, 0x3f, 0x14, 0x69, 0x42, 0x93, 0xb8, 0xc5, 0xee, 0x7a, 0x51, 0x2c, 0x7, 0xd6, 0xfd, 0x80, 0xab, 0xb5, 0x9e, 0xe3, 0xc8, 0x19, 0x32, 0x4f, 0x64, 0xf0, 0xdb, 0xa6, 0x8d, 0x5c, 0x77, 0xa, 0x21},
- {0x0, 0x2c, 0x58, 0x74, 0xb0, 0x9c, 0xe8, 0xc4, 0x7d, 0x51, 0x25, 0x9, 0xcd, 0xe1, 0x95, 0xb9, 0xfa, 0xd6, 0xa2, 0x8e, 0x4a, 0x66, 0x12, 0x3e, 0x87, 0xab, 0xdf, 0xf3, 0x37, 0x1b, 0x6f, 0x43, 0xe9, 0xc5, 0xb1, 0x9d, 0x59, 0x75, 0x1, 0x2d, 0x94, 0xb8, 0xcc, 0xe0, 0x24, 0x8, 0x7c, 0x50, 0x13, 0x3f, 0x4b, 0x67, 0xa3, 0x8f, 0xfb, 0xd7, 0x6e, 0x42, 0x36, 0x1a, 0xde, 0xf2, 0x86, 0xaa, 0xcf, 0xe3, 0x97, 0xbb, 0x7f, 0x53, 0x27, 0xb, 0xb2, 0x9e, 0xea, 0xc6, 0x2, 0x2e, 0x5a, 0x76, 0x35, 0x19, 0x6d, 0x41, 0x85, 0xa9, 0xdd, 0xf1, 0x48, 0x64, 0x10, 0x3c, 0xf8, 0xd4, 0xa0, 0x8c, 0x26, 0xa, 0x7e, 0x52, 0x96, 0xba, 0xce, 0xe2, 0x5b, 0x77, 0x3, 0x2f, 0xeb, 0xc7, 0xb3, 0x9f, 0xdc, 0xf0, 0x84, 0xa8, 0x6c, 0x40, 0x34, 0x18, 0xa1, 0x8d, 0xf9, 0xd5, 0x11, 0x3d, 0x49, 0x65, 0x83, 0xaf, 0xdb, 0xf7, 0x33, 0x1f, 0x6b, 0x47, 0xfe, 0xd2, 0xa6, 0x8a, 0x4e, 0x62, 0x16, 0x3a, 0x79, 0x55, 0x21, 0xd, 0xc9, 0xe5, 0x91, 0xbd, 0x4, 0x28, 0x5c, 0x70, 0xb4, 0x98, 0xec, 0xc0, 0x6a, 0x46, 0x32, 0x1e, 0xda, 0xf6, 0x82, 0xae, 0x17, 0x3b, 0x4f, 0x63, 0xa7, 0x8b, 0xff, 0xd3, 0x90, 0xbc, 0xc8, 0xe4, 0x20, 0xc, 0x78, 0x54, 0xed, 0xc1, 0xb5, 0x99, 0x5d, 0x71, 0x5, 0x29, 0x4c, 0x60, 0x14, 0x38, 0xfc, 0xd0, 0xa4, 0x88, 0x31, 0x1d, 0x69, 0x45, 0x81, 0xad, 0xd9, 0xf5, 0xb6, 0x9a, 0xee, 0xc2, 0x6, 0x2a, 0x5e, 0x72, 0xcb, 0xe7, 0x93, 0xbf, 0x7b, 0x57, 0x23, 0xf, 0xa5, 0x89, 0xfd, 0xd1, 0x15, 0x39, 0x4d, 0x61, 0xd8, 0xf4, 0x80, 0xac, 0x68, 0x44, 0x30, 0x1c, 0x5f, 0x73, 0x7, 0x2b, 0xef, 0xc3, 0xb7, 0x9b, 0x22, 0xe, 0x7a, 0x56, 0x92, 0xbe, 0xca, 0xe6},
- {0x0, 0x2d, 0x5a, 0x77, 0xb4, 0x99, 0xee, 0xc3, 0x75, 0x58, 0x2f, 0x2, 0xc1, 0xec, 0x9b, 0xb6, 0xea, 0xc7, 0xb0, 0x9d, 0x5e, 0x73, 0x4, 0x29, 0x9f, 0xb2, 0xc5, 0xe8, 0x2b, 0x6, 0x71, 0x5c, 0xc9, 0xe4, 0x93, 0xbe, 0x7d, 0x50, 0x27, 0xa, 0xbc, 0x91, 0xe6, 0xcb, 0x8, 0x25, 0x52, 0x7f, 0x23, 0xe, 0x79, 0x54, 0x97, 0xba, 0xcd, 0xe0, 0x56, 0x7b, 0xc, 0x21, 0xe2, 0xcf, 0xb8, 0x95, 0x8f, 0xa2, 0xd5, 0xf8, 0x3b, 0x16, 0x61, 0x4c, 0xfa, 0xd7, 0xa0, 0x8d, 0x4e, 0x63, 0x14, 0x39, 0x65, 0x48, 0x3f, 0x12, 0xd1, 0xfc, 0x8b, 0xa6, 0x10, 0x3d, 0x4a, 0x67, 0xa4, 0x89, 0xfe, 0xd3, 0x46, 0x6b, 0x1c, 0x31, 0xf2, 0xdf, 0xa8, 0x85, 0x33, 0x1e, 0x69, 0x44, 0x87, 0xaa, 0xdd, 0xf0, 0xac, 0x81, 0xf6, 0xdb, 0x18, 0x35, 0x42, 0x6f, 0xd9, 0xf4, 0x83, 0xae, 0x6d, 0x40, 0x37, 0x1a, 0x3, 0x2e, 0x59, 0x74, 0xb7, 0x9a, 0xed, 0xc0, 0x76, 0x5b, 0x2c, 0x1, 0xc2, 0xef, 0x98, 0xb5, 0xe9, 0xc4, 0xb3, 0x9e, 0x5d, 0x70, 0x7, 0x2a, 0x9c, 0xb1, 0xc6, 0xeb, 0x28, 0x5, 0x72, 0x5f, 0xca, 0xe7, 0x90, 0xbd, 0x7e, 0x53, 0x24, 0x9, 0xbf, 0x92, 0xe5, 0xc8, 0xb, 0x26, 0x51, 0x7c, 0x20, 0xd, 0x7a, 0x57, 0x94, 0xb9, 0xce, 0xe3, 0x55, 0x78, 0xf, 0x22, 0xe1, 0xcc, 0xbb, 0x96, 0x8c, 0xa1, 0xd6, 0xfb, 0x38, 0x15, 0x62, 0x4f, 0xf9, 0xd4, 0xa3, 0x8e, 0x4d, 0x60, 0x17, 0x3a, 0x66, 0x4b, 0x3c, 0x11, 0xd2, 0xff, 0x88, 0xa5, 0x13, 0x3e, 0x49, 0x64, 0xa7, 0x8a, 0xfd, 0xd0, 0x45, 0x68, 0x1f, 0x32, 0xf1, 0xdc, 0xab, 0x86, 0x30, 0x1d, 0x6a, 0x47, 0x84, 0xa9, 0xde, 0xf3, 0xaf, 0x82, 0xf5, 0xd8, 0x1b, 0x36, 0x41, 0x6c, 0xda, 0xf7, 0x80, 0xad, 0x6e, 0x43, 0x34, 0x19},
- {0x0, 0x2e, 0x5c, 0x72, 0xb8, 0x96, 0xe4, 0xca, 0x6d, 0x43, 0x31, 0x1f, 0xd5, 0xfb, 0x89, 0xa7, 0xda, 0xf4, 0x86, 0xa8, 0x62, 0x4c, 0x3e, 0x10, 0xb7, 0x99, 0xeb, 0xc5, 0xf, 0x21, 0x53, 0x7d, 0xa9, 0x87, 0xf5, 0xdb, 0x11, 0x3f, 0x4d, 0x63, 0xc4, 0xea, 0x98, 0xb6, 0x7c, 0x52, 0x20, 0xe, 0x73, 0x5d, 0x2f, 0x1, 0xcb, 0xe5, 0x97, 0xb9, 0x1e, 0x30, 0x42, 0x6c, 0xa6, 0x88, 0xfa, 0xd4, 0x4f, 0x61, 0x13, 0x3d, 0xf7, 0xd9, 0xab, 0x85, 0x22, 0xc, 0x7e, 0x50, 0x9a, 0xb4, 0xc6, 0xe8, 0x95, 0xbb, 0xc9, 0xe7, 0x2d, 0x3, 0x71, 0x5f, 0xf8, 0xd6, 0xa4, 0x8a, 0x40, 0x6e, 0x1c, 0x32, 0xe6, 0xc8, 0xba, 0x94, 0x5e, 0x70, 0x2, 0x2c, 0x8b, 0xa5, 0xd7, 0xf9, 0x33, 0x1d, 0x6f, 0x41, 0x3c, 0x12, 0x60, 0x4e, 0x84, 0xaa, 0xd8, 0xf6, 0x51, 0x7f, 0xd, 0x23, 0xe9, 0xc7, 0xb5, 0x9b, 0x9e, 0xb0, 0xc2, 0xec, 0x26, 0x8, 0x7a, 0x54, 0xf3, 0xdd, 0xaf, 0x81, 0x4b, 0x65, 0x17, 0x39, 0x44, 0x6a, 0x18, 0x36, 0xfc, 0xd2, 0xa0, 0x8e, 0x29, 0x7, 0x75, 0x5b, 0x91, 0xbf, 0xcd, 0xe3, 0x37, 0x19, 0x6b, 0x45, 0x8f, 0xa1, 0xd3, 0xfd, 0x5a, 0x74, 0x6, 0x28, 0xe2, 0xcc, 0xbe, 0x90, 0xed, 0xc3, 0xb1, 0x9f, 0x55, 0x7b, 0x9, 0x27, 0x80, 0xae, 0xdc, 0xf2, 0x38, 0x16, 0x64, 0x4a, 0xd1, 0xff, 0x8d, 0xa3, 0x69, 0x47, 0x35, 0x1b, 0xbc, 0x92, 0xe0, 0xce, 0x4, 0x2a, 0x58, 0x76, 0xb, 0x25, 0x57, 0x79, 0xb3, 0x9d, 0xef, 0xc1, 0x66, 0x48, 0x3a, 0x14, 0xde, 0xf0, 0x82, 0xac, 0x78, 0x56, 0x24, 0xa, 0xc0, 0xee, 0x9c, 0xb2, 0x15, 0x3b, 0x49, 0x67, 0xad, 0x83, 0xf1, 0xdf, 0xa2, 0x8c, 0xfe, 0xd0, 0x1a, 0x34, 0x46, 0x68, 0xcf, 0xe1, 0x93, 0xbd, 0x77, 0x59, 0x2b, 0x5},
- {0x0, 0x2f, 0x5e, 0x71, 0xbc, 0x93, 0xe2, 0xcd, 0x65, 0x4a, 0x3b, 0x14, 0xd9, 0xf6, 0x87, 0xa8, 0xca, 0xe5, 0x94, 0xbb, 0x76, 0x59, 0x28, 0x7, 0xaf, 0x80, 0xf1, 0xde, 0x13, 0x3c, 0x4d, 0x62, 0x89, 0xa6, 0xd7, 0xf8, 0x35, 0x1a, 0x6b, 0x44, 0xec, 0xc3, 0xb2, 0x9d, 0x50, 0x7f, 0xe, 0x21, 0x43, 0x6c, 0x1d, 0x32, 0xff, 0xd0, 0xa1, 0x8e, 0x26, 0x9, 0x78, 0x57, 0x9a, 0xb5, 0xc4, 0xeb, 0xf, 0x20, 0x51, 0x7e, 0xb3, 0x9c, 0xed, 0xc2, 0x6a, 0x45, 0x34, 0x1b, 0xd6, 0xf9, 0x88, 0xa7, 0xc5, 0xea, 0x9b, 0xb4, 0x79, 0x56, 0x27, 0x8, 0xa0, 0x8f, 0xfe, 0xd1, 0x1c, 0x33, 0x42, 0x6d, 0x86, 0xa9, 0xd8, 0xf7, 0x3a, 0x15, 0x64, 0x4b, 0xe3, 0xcc, 0xbd, 0x92, 0x5f, 0x70, 0x1, 0x2e, 0x4c, 0x63, 0x12, 0x3d, 0xf0, 0xdf, 0xae, 0x81, 0x29, 0x6, 0x77, 0x58, 0x95, 0xba, 0xcb, 0xe4, 0x1e, 0x31, 0x40, 0x6f, 0xa2, 0x8d, 0xfc, 0xd3, 0x7b, 0x54, 0x25, 0xa, 0xc7, 0xe8, 0x99, 0xb6, 0xd4, 0xfb, 0x8a, 0xa5, 0x68, 0x47, 0x36, 0x19, 0xb1, 0x9e, 0xef, 0xc0, 0xd, 0x22, 0x53, 0x7c, 0x97, 0xb8, 0xc9, 0xe6, 0x2b, 0x4, 0x75, 0x5a, 0xf2, 0xdd, 0xac, 0x83, 0x4e, 0x61, 0x10, 0x3f, 0x5d, 0x72, 0x3, 0x2c, 0xe1, 0xce, 0xbf, 0x90, 0x38, 0x17, 0x66, 0x49, 0x84, 0xab, 0xda, 0xf5, 0x11, 0x3e, 0x4f, 0x60, 0xad, 0x82, 0xf3, 0xdc, 0x74, 0x5b, 0x2a, 0x5, 0xc8, 0xe7, 0x96, 0xb9, 0xdb, 0xf4, 0x85, 0xaa, 0x67, 0x48, 0x39, 0x16, 0xbe, 0x91, 0xe0, 0xcf, 0x2, 0x2d, 0x5c, 0x73, 0x98, 0xb7, 0xc6, 0xe9, 0x24, 0xb, 0x7a, 0x55, 0xfd, 0xd2, 0xa3, 0x8c, 0x41, 0x6e, 0x1f, 0x30, 0x52, 0x7d, 0xc, 0x23, 0xee, 0xc1, 0xb0, 0x9f, 0x37, 0x18, 0x69, 0x46, 0x8b, 0xa4, 0xd5, 0xfa},
- {0x0, 0x30, 0x60, 0x50, 0xc0, 0xf0, 0xa0, 0x90, 0x9d, 0xad, 0xfd, 0xcd, 0x5d, 0x6d, 0x3d, 0xd, 0x27, 0x17, 0x47, 0x77, 0xe7, 0xd7, 0x87, 0xb7, 0xba, 0x8a, 0xda, 0xea, 0x7a, 0x4a, 0x1a, 0x2a, 0x4e, 0x7e, 0x2e, 0x1e, 0x8e, 0xbe, 0xee, 0xde, 0xd3, 0xe3, 0xb3, 0x83, 0x13, 0x23, 0x73, 0x43, 0x69, 0x59, 0x9, 0x39, 0xa9, 0x99, 0xc9, 0xf9, 0xf4, 0xc4, 0x94, 0xa4, 0x34, 0x4, 0x54, 0x64, 0x9c, 0xac, 0xfc, 0xcc, 0x5c, 0x6c, 0x3c, 0xc, 0x1, 0x31, 0x61, 0x51, 0xc1, 0xf1, 0xa1, 0x91, 0xbb, 0x8b, 0xdb, 0xeb, 0x7b, 0x4b, 0x1b, 0x2b, 0x26, 0x16, 0x46, 0x76, 0xe6, 0xd6, 0x86, 0xb6, 0xd2, 0xe2, 0xb2, 0x82, 0x12, 0x22, 0x72, 0x42, 0x4f, 0x7f, 0x2f, 0x1f, 0x8f, 0xbf, 0xef, 0xdf, 0xf5, 0xc5, 0x95, 0xa5, 0x35, 0x5, 0x55, 0x65, 0x68, 0x58, 0x8, 0x38, 0xa8, 0x98, 0xc8, 0xf8, 0x25, 0x15, 0x45, 0x75, 0xe5, 0xd5, 0x85, 0xb5, 0xb8, 0x88, 0xd8, 0xe8, 0x78, 0x48, 0x18, 0x28, 0x2, 0x32, 0x62, 0x52, 0xc2, 0xf2, 0xa2, 0x92, 0x9f, 0xaf, 0xff, 0xcf, 0x5f, 0x6f, 0x3f, 0xf, 0x6b, 0x5b, 0xb, 0x3b, 0xab, 0x9b, 0xcb, 0xfb, 0xf6, 0xc6, 0x96, 0xa6, 0x36, 0x6, 0x56, 0x66, 0x4c, 0x7c, 0x2c, 0x1c, 0x8c, 0xbc, 0xec, 0xdc, 0xd1, 0xe1, 0xb1, 0x81, 0x11, 0x21, 0x71, 0x41, 0xb9, 0x89, 0xd9, 0xe9, 0x79, 0x49, 0x19, 0x29, 0x24, 0x14, 0x44, 0x74, 0xe4, 0xd4, 0x84, 0xb4, 0x9e, 0xae, 0xfe, 0xce, 0x5e, 0x6e, 0x3e, 0xe, 0x3, 0x33, 0x63, 0x53, 0xc3, 0xf3, 0xa3, 0x93, 0xf7, 0xc7, 0x97, 0xa7, 0x37, 0x7, 0x57, 0x67, 0x6a, 0x5a, 0xa, 0x3a, 0xaa, 0x9a, 0xca, 0xfa, 0xd0, 0xe0, 0xb0, 0x80, 0x10, 0x20, 0x70, 0x40, 0x4d, 0x7d, 0x2d, 0x1d, 0x8d, 0xbd, 0xed, 0xdd},
- {0x0, 0x31, 0x62, 0x53, 0xc4, 0xf5, 0xa6, 0x97, 0x95, 0xa4, 0xf7, 0xc6, 0x51, 0x60, 0x33, 0x2, 0x37, 0x6, 0x55, 0x64, 0xf3, 0xc2, 0x91, 0xa0, 0xa2, 0x93, 0xc0, 0xf1, 0x66, 0x57, 0x4, 0x35, 0x6e, 0x5f, 0xc, 0x3d, 0xaa, 0x9b, 0xc8, 0xf9, 0xfb, 0xca, 0x99, 0xa8, 0x3f, 0xe, 0x5d, 0x6c, 0x59, 0x68, 0x3b, 0xa, 0x9d, 0xac, 0xff, 0xce, 0xcc, 0xfd, 0xae, 0x9f, 0x8, 0x39, 0x6a, 0x5b, 0xdc, 0xed, 0xbe, 0x8f, 0x18, 0x29, 0x7a, 0x4b, 0x49, 0x78, 0x2b, 0x1a, 0x8d, 0xbc, 0xef, 0xde, 0xeb, 0xda, 0x89, 0xb8, 0x2f, 0x1e, 0x4d, 0x7c, 0x7e, 0x4f, 0x1c, 0x2d, 0xba, 0x8b, 0xd8, 0xe9, 0xb2, 0x83, 0xd0, 0xe1, 0x76, 0x47, 0x14, 0x25, 0x27, 0x16, 0x45, 0x74, 0xe3, 0xd2, 0x81, 0xb0, 0x85, 0xb4, 0xe7, 0xd6, 0x41, 0x70, 0x23, 0x12, 0x10, 0x21, 0x72, 0x43, 0xd4, 0xe5, 0xb6, 0x87, 0xa5, 0x94, 0xc7, 0xf6, 0x61, 0x50, 0x3, 0x32, 0x30, 0x1, 0x52, 0x63, 0xf4, 0xc5, 0x96, 0xa7, 0x92, 0xa3, 0xf0, 0xc1, 0x56, 0x67, 0x34, 0x5, 0x7, 0x36, 0x65, 0x54, 0xc3, 0xf2, 0xa1, 0x90, 0xcb, 0xfa, 0xa9, 0x98, 0xf, 0x3e, 0x6d, 0x5c, 0x5e, 0x6f, 0x3c, 0xd, 0x9a, 0xab, 0xf8, 0xc9, 0xfc, 0xcd, 0x9e, 0xaf, 0x38, 0x9, 0x5a, 0x6b, 0x69, 0x58, 0xb, 0x3a, 0xad, 0x9c, 0xcf, 0xfe, 0x79, 0x48, 0x1b, 0x2a, 0xbd, 0x8c, 0xdf, 0xee, 0xec, 0xdd, 0x8e, 0xbf, 0x28, 0x19, 0x4a, 0x7b, 0x4e, 0x7f, 0x2c, 0x1d, 0x8a, 0xbb, 0xe8, 0xd9, 0xdb, 0xea, 0xb9, 0x88, 0x1f, 0x2e, 0x7d, 0x4c, 0x17, 0x26, 0x75, 0x44, 0xd3, 0xe2, 0xb1, 0x80, 0x82, 0xb3, 0xe0, 0xd1, 0x46, 0x77, 0x24, 0x15, 0x20, 0x11, 0x42, 0x73, 0xe4, 0xd5, 0x86, 0xb7, 0xb5, 0x84, 0xd7, 0xe6, 0x71, 0x40, 0x13, 0x22},
- {0x0, 0x32, 0x64, 0x56, 0xc8, 0xfa, 0xac, 0x9e, 0x8d, 0xbf, 0xe9, 0xdb, 0x45, 0x77, 0x21, 0x13, 0x7, 0x35, 0x63, 0x51, 0xcf, 0xfd, 0xab, 0x99, 0x8a, 0xb8, 0xee, 0xdc, 0x42, 0x70, 0x26, 0x14, 0xe, 0x3c, 0x6a, 0x58, 0xc6, 0xf4, 0xa2, 0x90, 0x83, 0xb1, 0xe7, 0xd5, 0x4b, 0x79, 0x2f, 0x1d, 0x9, 0x3b, 0x6d, 0x5f, 0xc1, 0xf3, 0xa5, 0x97, 0x84, 0xb6, 0xe0, 0xd2, 0x4c, 0x7e, 0x28, 0x1a, 0x1c, 0x2e, 0x78, 0x4a, 0xd4, 0xe6, 0xb0, 0x82, 0x91, 0xa3, 0xf5, 0xc7, 0x59, 0x6b, 0x3d, 0xf, 0x1b, 0x29, 0x7f, 0x4d, 0xd3, 0xe1, 0xb7, 0x85, 0x96, 0xa4, 0xf2, 0xc0, 0x5e, 0x6c, 0x3a, 0x8, 0x12, 0x20, 0x76, 0x44, 0xda, 0xe8, 0xbe, 0x8c, 0x9f, 0xad, 0xfb, 0xc9, 0x57, 0x65, 0x33, 0x1, 0x15, 0x27, 0x71, 0x43, 0xdd, 0xef, 0xb9, 0x8b, 0x98, 0xaa, 0xfc, 0xce, 0x50, 0x62, 0x34, 0x6, 0x38, 0xa, 0x5c, 0x6e, 0xf0, 0xc2, 0x94, 0xa6, 0xb5, 0x87, 0xd1, 0xe3, 0x7d, 0x4f, 0x19, 0x2b, 0x3f, 0xd, 0x5b, 0x69, 0xf7, 0xc5, 0x93, 0xa1, 0xb2, 0x80, 0xd6, 0xe4, 0x7a, 0x48, 0x1e, 0x2c, 0x36, 0x4, 0x52, 0x60, 0xfe, 0xcc, 0x9a, 0xa8, 0xbb, 0x89, 0xdf, 0xed, 0x73, 0x41, 0x17, 0x25, 0x31, 0x3, 0x55, 0x67, 0xf9, 0xcb, 0x9d, 0xaf, 0xbc, 0x8e, 0xd8, 0xea, 0x74, 0x46, 0x10, 0x22, 0x24, 0x16, 0x40, 0x72, 0xec, 0xde, 0x88, 0xba, 0xa9, 0x9b, 0xcd, 0xff, 0x61, 0x53, 0x5, 0x37, 0x23, 0x11, 0x47, 0x75, 0xeb, 0xd9, 0x8f, 0xbd, 0xae, 0x9c, 0xca, 0xf8, 0x66, 0x54, 0x2, 0x30, 0x2a, 0x18, 0x4e, 0x7c, 0xe2, 0xd0, 0x86, 0xb4, 0xa7, 0x95, 0xc3, 0xf1, 0x6f, 0x5d, 0xb, 0x39, 0x2d, 0x1f, 0x49, 0x7b, 0xe5, 0xd7, 0x81, 0xb3, 0xa0, 0x92, 0xc4, 0xf6, 0x68, 0x5a, 0xc, 0x3e},
- {0x0, 0x33, 0x66, 0x55, 0xcc, 0xff, 0xaa, 0x99, 0x85, 0xb6, 0xe3, 0xd0, 0x49, 0x7a, 0x2f, 0x1c, 0x17, 0x24, 0x71, 0x42, 0xdb, 0xe8, 0xbd, 0x8e, 0x92, 0xa1, 0xf4, 0xc7, 0x5e, 0x6d, 0x38, 0xb, 0x2e, 0x1d, 0x48, 0x7b, 0xe2, 0xd1, 0x84, 0xb7, 0xab, 0x98, 0xcd, 0xfe, 0x67, 0x54, 0x1, 0x32, 0x39, 0xa, 0x5f, 0x6c, 0xf5, 0xc6, 0x93, 0xa0, 0xbc, 0x8f, 0xda, 0xe9, 0x70, 0x43, 0x16, 0x25, 0x5c, 0x6f, 0x3a, 0x9, 0x90, 0xa3, 0xf6, 0xc5, 0xd9, 0xea, 0xbf, 0x8c, 0x15, 0x26, 0x73, 0x40, 0x4b, 0x78, 0x2d, 0x1e, 0x87, 0xb4, 0xe1, 0xd2, 0xce, 0xfd, 0xa8, 0x9b, 0x2, 0x31, 0x64, 0x57, 0x72, 0x41, 0x14, 0x27, 0xbe, 0x8d, 0xd8, 0xeb, 0xf7, 0xc4, 0x91, 0xa2, 0x3b, 0x8, 0x5d, 0x6e, 0x65, 0x56, 0x3, 0x30, 0xa9, 0x9a, 0xcf, 0xfc, 0xe0, 0xd3, 0x86, 0xb5, 0x2c, 0x1f, 0x4a, 0x79, 0xb8, 0x8b, 0xde, 0xed, 0x74, 0x47, 0x12, 0x21, 0x3d, 0xe, 0x5b, 0x68, 0xf1, 0xc2, 0x97, 0xa4, 0xaf, 0x9c, 0xc9, 0xfa, 0x63, 0x50, 0x5, 0x36, 0x2a, 0x19, 0x4c, 0x7f, 0xe6, 0xd5, 0x80, 0xb3, 0x96, 0xa5, 0xf0, 0xc3, 0x5a, 0x69, 0x3c, 0xf, 0x13, 0x20, 0x75, 0x46, 0xdf, 0xec, 0xb9, 0x8a, 0x81, 0xb2, 0xe7, 0xd4, 0x4d, 0x7e, 0x2b, 0x18, 0x4, 0x37, 0x62, 0x51, 0xc8, 0xfb, 0xae, 0x9d, 0xe4, 0xd7, 0x82, 0xb1, 0x28, 0x1b, 0x4e, 0x7d, 0x61, 0x52, 0x7, 0x34, 0xad, 0x9e, 0xcb, 0xf8, 0xf3, 0xc0, 0x95, 0xa6, 0x3f, 0xc, 0x59, 0x6a, 0x76, 0x45, 0x10, 0x23, 0xba, 0x89, 0xdc, 0xef, 0xca, 0xf9, 0xac, 0x9f, 0x6, 0x35, 0x60, 0x53, 0x4f, 0x7c, 0x29, 0x1a, 0x83, 0xb0, 0xe5, 0xd6, 0xdd, 0xee, 0xbb, 0x88, 0x11, 0x22, 0x77, 0x44, 0x58, 0x6b, 0x3e, 0xd, 0x94, 0xa7, 0xf2, 0xc1},
- {0x0, 0x34, 0x68, 0x5c, 0xd0, 0xe4, 0xb8, 0x8c, 0xbd, 0x89, 0xd5, 0xe1, 0x6d, 0x59, 0x5, 0x31, 0x67, 0x53, 0xf, 0x3b, 0xb7, 0x83, 0xdf, 0xeb, 0xda, 0xee, 0xb2, 0x86, 0xa, 0x3e, 0x62, 0x56, 0xce, 0xfa, 0xa6, 0x92, 0x1e, 0x2a, 0x76, 0x42, 0x73, 0x47, 0x1b, 0x2f, 0xa3, 0x97, 0xcb, 0xff, 0xa9, 0x9d, 0xc1, 0xf5, 0x79, 0x4d, 0x11, 0x25, 0x14, 0x20, 0x7c, 0x48, 0xc4, 0xf0, 0xac, 0x98, 0x81, 0xb5, 0xe9, 0xdd, 0x51, 0x65, 0x39, 0xd, 0x3c, 0x8, 0x54, 0x60, 0xec, 0xd8, 0x84, 0xb0, 0xe6, 0xd2, 0x8e, 0xba, 0x36, 0x2, 0x5e, 0x6a, 0x5b, 0x6f, 0x33, 0x7, 0x8b, 0xbf, 0xe3, 0xd7, 0x4f, 0x7b, 0x27, 0x13, 0x9f, 0xab, 0xf7, 0xc3, 0xf2, 0xc6, 0x9a, 0xae, 0x22, 0x16, 0x4a, 0x7e, 0x28, 0x1c, 0x40, 0x74, 0xf8, 0xcc, 0x90, 0xa4, 0x95, 0xa1, 0xfd, 0xc9, 0x45, 0x71, 0x2d, 0x19, 0x1f, 0x2b, 0x77, 0x43, 0xcf, 0xfb, 0xa7, 0x93, 0xa2, 0x96, 0xca, 0xfe, 0x72, 0x46, 0x1a, 0x2e, 0x78, 0x4c, 0x10, 0x24, 0xa8, 0x9c, 0xc0, 0xf4, 0xc5, 0xf1, 0xad, 0x99, 0x15, 0x21, 0x7d, 0x49, 0xd1, 0xe5, 0xb9, 0x8d, 0x1, 0x35, 0x69, 0x5d, 0x6c, 0x58, 0x4, 0x30, 0xbc, 0x88, 0xd4, 0xe0, 0xb6, 0x82, 0xde, 0xea, 0x66, 0x52, 0xe, 0x3a, 0xb, 0x3f, 0x63, 0x57, 0xdb, 0xef, 0xb3, 0x87, 0x9e, 0xaa, 0xf6, 0xc2, 0x4e, 0x7a, 0x26, 0x12, 0x23, 0x17, 0x4b, 0x7f, 0xf3, 0xc7, 0x9b, 0xaf, 0xf9, 0xcd, 0x91, 0xa5, 0x29, 0x1d, 0x41, 0x75, 0x44, 0x70, 0x2c, 0x18, 0x94, 0xa0, 0xfc, 0xc8, 0x50, 0x64, 0x38, 0xc, 0x80, 0xb4, 0xe8, 0xdc, 0xed, 0xd9, 0x85, 0xb1, 0x3d, 0x9, 0x55, 0x61, 0x37, 0x3, 0x5f, 0x6b, 0xe7, 0xd3, 0x8f, 0xbb, 0x8a, 0xbe, 0xe2, 0xd6, 0x5a, 0x6e, 0x32, 0x6},
- {0x0, 0x35, 0x6a, 0x5f, 0xd4, 0xe1, 0xbe, 0x8b, 0xb5, 0x80, 0xdf, 0xea, 0x61, 0x54, 0xb, 0x3e, 0x77, 0x42, 0x1d, 0x28, 0xa3, 0x96, 0xc9, 0xfc, 0xc2, 0xf7, 0xa8, 0x9d, 0x16, 0x23, 0x7c, 0x49, 0xee, 0xdb, 0x84, 0xb1, 0x3a, 0xf, 0x50, 0x65, 0x5b, 0x6e, 0x31, 0x4, 0x8f, 0xba, 0xe5, 0xd0, 0x99, 0xac, 0xf3, 0xc6, 0x4d, 0x78, 0x27, 0x12, 0x2c, 0x19, 0x46, 0x73, 0xf8, 0xcd, 0x92, 0xa7, 0xc1, 0xf4, 0xab, 0x9e, 0x15, 0x20, 0x7f, 0x4a, 0x74, 0x41, 0x1e, 0x2b, 0xa0, 0x95, 0xca, 0xff, 0xb6, 0x83, 0xdc, 0xe9, 0x62, 0x57, 0x8, 0x3d, 0x3, 0x36, 0x69, 0x5c, 0xd7, 0xe2, 0xbd, 0x88, 0x2f, 0x1a, 0x45, 0x70, 0xfb, 0xce, 0x91, 0xa4, 0x9a, 0xaf, 0xf0, 0xc5, 0x4e, 0x7b, 0x24, 0x11, 0x58, 0x6d, 0x32, 0x7, 0x8c, 0xb9, 0xe6, 0xd3, 0xed, 0xd8, 0x87, 0xb2, 0x39, 0xc, 0x53, 0x66, 0x9f, 0xaa, 0xf5, 0xc0, 0x4b, 0x7e, 0x21, 0x14, 0x2a, 0x1f, 0x40, 0x75, 0xfe, 0xcb, 0x94, 0xa1, 0xe8, 0xdd, 0x82, 0xb7, 0x3c, 0x9, 0x56, 0x63, 0x5d, 0x68, 0x37, 0x2, 0x89, 0xbc, 0xe3, 0xd6, 0x71, 0x44, 0x1b, 0x2e, 0xa5, 0x90, 0xcf, 0xfa, 0xc4, 0xf1, 0xae, 0x9b, 0x10, 0x25, 0x7a, 0x4f, 0x6, 0x33, 0x6c, 0x59, 0xd2, 0xe7, 0xb8, 0x8d, 0xb3, 0x86, 0xd9, 0xec, 0x67, 0x52, 0xd, 0x38, 0x5e, 0x6b, 0x34, 0x1, 0x8a, 0xbf, 0xe0, 0xd5, 0xeb, 0xde, 0x81, 0xb4, 0x3f, 0xa, 0x55, 0x60, 0x29, 0x1c, 0x43, 0x76, 0xfd, 0xc8, 0x97, 0xa2, 0x9c, 0xa9, 0xf6, 0xc3, 0x48, 0x7d, 0x22, 0x17, 0xb0, 0x85, 0xda, 0xef, 0x64, 0x51, 0xe, 0x3b, 0x5, 0x30, 0x6f, 0x5a, 0xd1, 0xe4, 0xbb, 0x8e, 0xc7, 0xf2, 0xad, 0x98, 0x13, 0x26, 0x79, 0x4c, 0x72, 0x47, 0x18, 0x2d, 0xa6, 0x93, 0xcc, 0xf9},
- {0x0, 0x36, 0x6c, 0x5a, 0xd8, 0xee, 0xb4, 0x82, 0xad, 0x9b, 0xc1, 0xf7, 0x75, 0x43, 0x19, 0x2f, 0x47, 0x71, 0x2b, 0x1d, 0x9f, 0xa9, 0xf3, 0xc5, 0xea, 0xdc, 0x86, 0xb0, 0x32, 0x4, 0x5e, 0x68, 0x8e, 0xb8, 0xe2, 0xd4, 0x56, 0x60, 0x3a, 0xc, 0x23, 0x15, 0x4f, 0x79, 0xfb, 0xcd, 0x97, 0xa1, 0xc9, 0xff, 0xa5, 0x93, 0x11, 0x27, 0x7d, 0x4b, 0x64, 0x52, 0x8, 0x3e, 0xbc, 0x8a, 0xd0, 0xe6, 0x1, 0x37, 0x6d, 0x5b, 0xd9, 0xef, 0xb5, 0x83, 0xac, 0x9a, 0xc0, 0xf6, 0x74, 0x42, 0x18, 0x2e, 0x46, 0x70, 0x2a, 0x1c, 0x9e, 0xa8, 0xf2, 0xc4, 0xeb, 0xdd, 0x87, 0xb1, 0x33, 0x5, 0x5f, 0x69, 0x8f, 0xb9, 0xe3, 0xd5, 0x57, 0x61, 0x3b, 0xd, 0x22, 0x14, 0x4e, 0x78, 0xfa, 0xcc, 0x96, 0xa0, 0xc8, 0xfe, 0xa4, 0x92, 0x10, 0x26, 0x7c, 0x4a, 0x65, 0x53, 0x9, 0x3f, 0xbd, 0x8b, 0xd1, 0xe7, 0x2, 0x34, 0x6e, 0x58, 0xda, 0xec, 0xb6, 0x80, 0xaf, 0x99, 0xc3, 0xf5, 0x77, 0x41, 0x1b, 0x2d, 0x45, 0x73, 0x29, 0x1f, 0x9d, 0xab, 0xf1, 0xc7, 0xe8, 0xde, 0x84, 0xb2, 0x30, 0x6, 0x5c, 0x6a, 0x8c, 0xba, 0xe0, 0xd6, 0x54, 0x62, 0x38, 0xe, 0x21, 0x17, 0x4d, 0x7b, 0xf9, 0xcf, 0x95, 0xa3, 0xcb, 0xfd, 0xa7, 0x91, 0x13, 0x25, 0x7f, 0x49, 0x66, 0x50, 0xa, 0x3c, 0xbe, 0x88, 0xd2, 0xe4, 0x3, 0x35, 0x6f, 0x59, 0xdb, 0xed, 0xb7, 0x81, 0xae, 0x98, 0xc2, 0xf4, 0x76, 0x40, 0x1a, 0x2c, 0x44, 0x72, 0x28, 0x1e, 0x9c, 0xaa, 0xf0, 0xc6, 0xe9, 0xdf, 0x85, 0xb3, 0x31, 0x7, 0x5d, 0x6b, 0x8d, 0xbb, 0xe1, 0xd7, 0x55, 0x63, 0x39, 0xf, 0x20, 0x16, 0x4c, 0x7a, 0xf8, 0xce, 0x94, 0xa2, 0xca, 0xfc, 0xa6, 0x90, 0x12, 0x24, 0x7e, 0x48, 0x67, 0x51, 0xb, 0x3d, 0xbf, 0x89, 0xd3, 0xe5},
- {0x0, 0x37, 0x6e, 0x59, 0xdc, 0xeb, 0xb2, 0x85, 0xa5, 0x92, 0xcb, 0xfc, 0x79, 0x4e, 0x17, 0x20, 0x57, 0x60, 0x39, 0xe, 0x8b, 0xbc, 0xe5, 0xd2, 0xf2, 0xc5, 0x9c, 0xab, 0x2e, 0x19, 0x40, 0x77, 0xae, 0x99, 0xc0, 0xf7, 0x72, 0x45, 0x1c, 0x2b, 0xb, 0x3c, 0x65, 0x52, 0xd7, 0xe0, 0xb9, 0x8e, 0xf9, 0xce, 0x97, 0xa0, 0x25, 0x12, 0x4b, 0x7c, 0x5c, 0x6b, 0x32, 0x5, 0x80, 0xb7, 0xee, 0xd9, 0x41, 0x76, 0x2f, 0x18, 0x9d, 0xaa, 0xf3, 0xc4, 0xe4, 0xd3, 0x8a, 0xbd, 0x38, 0xf, 0x56, 0x61, 0x16, 0x21, 0x78, 0x4f, 0xca, 0xfd, 0xa4, 0x93, 0xb3, 0x84, 0xdd, 0xea, 0x6f, 0x58, 0x1, 0x36, 0xef, 0xd8, 0x81, 0xb6, 0x33, 0x4, 0x5d, 0x6a, 0x4a, 0x7d, 0x24, 0x13, 0x96, 0xa1, 0xf8, 0xcf, 0xb8, 0x8f, 0xd6, 0xe1, 0x64, 0x53, 0xa, 0x3d, 0x1d, 0x2a, 0x73, 0x44, 0xc1, 0xf6, 0xaf, 0x98, 0x82, 0xb5, 0xec, 0xdb, 0x5e, 0x69, 0x30, 0x7, 0x27, 0x10, 0x49, 0x7e, 0xfb, 0xcc, 0x95, 0xa2, 0xd5, 0xe2, 0xbb, 0x8c, 0x9, 0x3e, 0x67, 0x50, 0x70, 0x47, 0x1e, 0x29, 0xac, 0x9b, 0xc2, 0xf5, 0x2c, 0x1b, 0x42, 0x75, 0xf0, 0xc7, 0x9e, 0xa9, 0x89, 0xbe, 0xe7, 0xd0, 0x55, 0x62, 0x3b, 0xc, 0x7b, 0x4c, 0x15, 0x22, 0xa7, 0x90, 0xc9, 0xfe, 0xde, 0xe9, 0xb0, 0x87, 0x2, 0x35, 0x6c, 0x5b, 0xc3, 0xf4, 0xad, 0x9a, 0x1f, 0x28, 0x71, 0x46, 0x66, 0x51, 0x8, 0x3f, 0xba, 0x8d, 0xd4, 0xe3, 0x94, 0xa3, 0xfa, 0xcd, 0x48, 0x7f, 0x26, 0x11, 0x31, 0x6, 0x5f, 0x68, 0xed, 0xda, 0x83, 0xb4, 0x6d, 0x5a, 0x3, 0x34, 0xb1, 0x86, 0xdf, 0xe8, 0xc8, 0xff, 0xa6, 0x91, 0x14, 0x23, 0x7a, 0x4d, 0x3a, 0xd, 0x54, 0x63, 0xe6, 0xd1, 0x88, 0xbf, 0x9f, 0xa8, 0xf1, 0xc6, 0x43, 0x74, 0x2d, 0x1a},
- {0x0, 0x38, 0x70, 0x48, 0xe0, 0xd8, 0x90, 0xa8, 0xdd, 0xe5, 0xad, 0x95, 0x3d, 0x5, 0x4d, 0x75, 0xa7, 0x9f, 0xd7, 0xef, 0x47, 0x7f, 0x37, 0xf, 0x7a, 0x42, 0xa, 0x32, 0x9a, 0xa2, 0xea, 0xd2, 0x53, 0x6b, 0x23, 0x1b, 0xb3, 0x8b, 0xc3, 0xfb, 0x8e, 0xb6, 0xfe, 0xc6, 0x6e, 0x56, 0x1e, 0x26, 0xf4, 0xcc, 0x84, 0xbc, 0x14, 0x2c, 0x64, 0x5c, 0x29, 0x11, 0x59, 0x61, 0xc9, 0xf1, 0xb9, 0x81, 0xa6, 0x9e, 0xd6, 0xee, 0x46, 0x7e, 0x36, 0xe, 0x7b, 0x43, 0xb, 0x33, 0x9b, 0xa3, 0xeb, 0xd3, 0x1, 0x39, 0x71, 0x49, 0xe1, 0xd9, 0x91, 0xa9, 0xdc, 0xe4, 0xac, 0x94, 0x3c, 0x4, 0x4c, 0x74, 0xf5, 0xcd, 0x85, 0xbd, 0x15, 0x2d, 0x65, 0x5d, 0x28, 0x10, 0x58, 0x60, 0xc8, 0xf0, 0xb8, 0x80, 0x52, 0x6a, 0x22, 0x1a, 0xb2, 0x8a, 0xc2, 0xfa, 0x8f, 0xb7, 0xff, 0xc7, 0x6f, 0x57, 0x1f, 0x27, 0x51, 0x69, 0x21, 0x19, 0xb1, 0x89, 0xc1, 0xf9, 0x8c, 0xb4, 0xfc, 0xc4, 0x6c, 0x54, 0x1c, 0x24, 0xf6, 0xce, 0x86, 0xbe, 0x16, 0x2e, 0x66, 0x5e, 0x2b, 0x13, 0x5b, 0x63, 0xcb, 0xf3, 0xbb, 0x83, 0x2, 0x3a, 0x72, 0x4a, 0xe2, 0xda, 0x92, 0xaa, 0xdf, 0xe7, 0xaf, 0x97, 0x3f, 0x7, 0x4f, 0x77, 0xa5, 0x9d, 0xd5, 0xed, 0x45, 0x7d, 0x35, 0xd, 0x78, 0x40, 0x8, 0x30, 0x98, 0xa0, 0xe8, 0xd0, 0xf7, 0xcf, 0x87, 0xbf, 0x17, 0x2f, 0x67, 0x5f, 0x2a, 0x12, 0x5a, 0x62, 0xca, 0xf2, 0xba, 0x82, 0x50, 0x68, 0x20, 0x18, 0xb0, 0x88, 0xc0, 0xf8, 0x8d, 0xb5, 0xfd, 0xc5, 0x6d, 0x55, 0x1d, 0x25, 0xa4, 0x9c, 0xd4, 0xec, 0x44, 0x7c, 0x34, 0xc, 0x79, 0x41, 0x9, 0x31, 0x99, 0xa1, 0xe9, 0xd1, 0x3, 0x3b, 0x73, 0x4b, 0xe3, 0xdb, 0x93, 0xab, 0xde, 0xe6, 0xae, 0x96, 0x3e, 0x6, 0x4e, 0x76},
- {0x0, 0x39, 0x72, 0x4b, 0xe4, 0xdd, 0x96, 0xaf, 0xd5, 0xec, 0xa7, 0x9e, 0x31, 0x8, 0x43, 0x7a, 0xb7, 0x8e, 0xc5, 0xfc, 0x53, 0x6a, 0x21, 0x18, 0x62, 0x5b, 0x10, 0x29, 0x86, 0xbf, 0xf4, 0xcd, 0x73, 0x4a, 0x1, 0x38, 0x97, 0xae, 0xe5, 0xdc, 0xa6, 0x9f, 0xd4, 0xed, 0x42, 0x7b, 0x30, 0x9, 0xc4, 0xfd, 0xb6, 0x8f, 0x20, 0x19, 0x52, 0x6b, 0x11, 0x28, 0x63, 0x5a, 0xf5, 0xcc, 0x87, 0xbe, 0xe6, 0xdf, 0x94, 0xad, 0x2, 0x3b, 0x70, 0x49, 0x33, 0xa, 0x41, 0x78, 0xd7, 0xee, 0xa5, 0x9c, 0x51, 0x68, 0x23, 0x1a, 0xb5, 0x8c, 0xc7, 0xfe, 0x84, 0xbd, 0xf6, 0xcf, 0x60, 0x59, 0x12, 0x2b, 0x95, 0xac, 0xe7, 0xde, 0x71, 0x48, 0x3, 0x3a, 0x40, 0x79, 0x32, 0xb, 0xa4, 0x9d, 0xd6, 0xef, 0x22, 0x1b, 0x50, 0x69, 0xc6, 0xff, 0xb4, 0x8d, 0xf7, 0xce, 0x85, 0xbc, 0x13, 0x2a, 0x61, 0x58, 0xd1, 0xe8, 0xa3, 0x9a, 0x35, 0xc, 0x47, 0x7e, 0x4, 0x3d, 0x76, 0x4f, 0xe0, 0xd9, 0x92, 0xab, 0x66, 0x5f, 0x14, 0x2d, 0x82, 0xbb, 0xf0, 0xc9, 0xb3, 0x8a, 0xc1, 0xf8, 0x57, 0x6e, 0x25, 0x1c, 0xa2, 0x9b, 0xd0, 0xe9, 0x46, 0x7f, 0x34, 0xd, 0x77, 0x4e, 0x5, 0x3c, 0x93, 0xaa, 0xe1, 0xd8, 0x15, 0x2c, 0x67, 0x5e, 0xf1, 0xc8, 0x83, 0xba, 0xc0, 0xf9, 0xb2, 0x8b, 0x24, 0x1d, 0x56, 0x6f, 0x37, 0xe, 0x45, 0x7c, 0xd3, 0xea, 0xa1, 0x98, 0xe2, 0xdb, 0x90, 0xa9, 0x6, 0x3f, 0x74, 0x4d, 0x80, 0xb9, 0xf2, 0xcb, 0x64, 0x5d, 0x16, 0x2f, 0x55, 0x6c, 0x27, 0x1e, 0xb1, 0x88, 0xc3, 0xfa, 0x44, 0x7d, 0x36, 0xf, 0xa0, 0x99, 0xd2, 0xeb, 0x91, 0xa8, 0xe3, 0xda, 0x75, 0x4c, 0x7, 0x3e, 0xf3, 0xca, 0x81, 0xb8, 0x17, 0x2e, 0x65, 0x5c, 0x26, 0x1f, 0x54, 0x6d, 0xc2, 0xfb, 0xb0, 0x89},
- {0x0, 0x3a, 0x74, 0x4e, 0xe8, 0xd2, 0x9c, 0xa6, 0xcd, 0xf7, 0xb9, 0x83, 0x25, 0x1f, 0x51, 0x6b, 0x87, 0xbd, 0xf3, 0xc9, 0x6f, 0x55, 0x1b, 0x21, 0x4a, 0x70, 0x3e, 0x4, 0xa2, 0x98, 0xd6, 0xec, 0x13, 0x29, 0x67, 0x5d, 0xfb, 0xc1, 0x8f, 0xb5, 0xde, 0xe4, 0xaa, 0x90, 0x36, 0xc, 0x42, 0x78, 0x94, 0xae, 0xe0, 0xda, 0x7c, 0x46, 0x8, 0x32, 0x59, 0x63, 0x2d, 0x17, 0xb1, 0x8b, 0xc5, 0xff, 0x26, 0x1c, 0x52, 0x68, 0xce, 0xf4, 0xba, 0x80, 0xeb, 0xd1, 0x9f, 0xa5, 0x3, 0x39, 0x77, 0x4d, 0xa1, 0x9b, 0xd5, 0xef, 0x49, 0x73, 0x3d, 0x7, 0x6c, 0x56, 0x18, 0x22, 0x84, 0xbe, 0xf0, 0xca, 0x35, 0xf, 0x41, 0x7b, 0xdd, 0xe7, 0xa9, 0x93, 0xf8, 0xc2, 0x8c, 0xb6, 0x10, 0x2a, 0x64, 0x5e, 0xb2, 0x88, 0xc6, 0xfc, 0x5a, 0x60, 0x2e, 0x14, 0x7f, 0x45, 0xb, 0x31, 0x97, 0xad, 0xe3, 0xd9, 0x4c, 0x76, 0x38, 0x2, 0xa4, 0x9e, 0xd0, 0xea, 0x81, 0xbb, 0xf5, 0xcf, 0x69, 0x53, 0x1d, 0x27, 0xcb, 0xf1, 0xbf, 0x85, 0x23, 0x19, 0x57, 0x6d, 0x6, 0x3c, 0x72, 0x48, 0xee, 0xd4, 0x9a, 0xa0, 0x5f, 0x65, 0x2b, 0x11, 0xb7, 0x8d, 0xc3, 0xf9, 0x92, 0xa8, 0xe6, 0xdc, 0x7a, 0x40, 0xe, 0x34, 0xd8, 0xe2, 0xac, 0x96, 0x30, 0xa, 0x44, 0x7e, 0x15, 0x2f, 0x61, 0x5b, 0xfd, 0xc7, 0x89, 0xb3, 0x6a, 0x50, 0x1e, 0x24, 0x82, 0xb8, 0xf6, 0xcc, 0xa7, 0x9d, 0xd3, 0xe9, 0x4f, 0x75, 0x3b, 0x1, 0xed, 0xd7, 0x99, 0xa3, 0x5, 0x3f, 0x71, 0x4b, 0x20, 0x1a, 0x54, 0x6e, 0xc8, 0xf2, 0xbc, 0x86, 0x79, 0x43, 0xd, 0x37, 0x91, 0xab, 0xe5, 0xdf, 0xb4, 0x8e, 0xc0, 0xfa, 0x5c, 0x66, 0x28, 0x12, 0xfe, 0xc4, 0x8a, 0xb0, 0x16, 0x2c, 0x62, 0x58, 0x33, 0x9, 0x47, 0x7d, 0xdb, 0xe1, 0xaf, 0x95},
- {0x0, 0x3b, 0x76, 0x4d, 0xec, 0xd7, 0x9a, 0xa1, 0xc5, 0xfe, 0xb3, 0x88, 0x29, 0x12, 0x5f, 0x64, 0x97, 0xac, 0xe1, 0xda, 0x7b, 0x40, 0xd, 0x36, 0x52, 0x69, 0x24, 0x1f, 0xbe, 0x85, 0xc8, 0xf3, 0x33, 0x8, 0x45, 0x7e, 0xdf, 0xe4, 0xa9, 0x92, 0xf6, 0xcd, 0x80, 0xbb, 0x1a, 0x21, 0x6c, 0x57, 0xa4, 0x9f, 0xd2, 0xe9, 0x48, 0x73, 0x3e, 0x5, 0x61, 0x5a, 0x17, 0x2c, 0x8d, 0xb6, 0xfb, 0xc0, 0x66, 0x5d, 0x10, 0x2b, 0x8a, 0xb1, 0xfc, 0xc7, 0xa3, 0x98, 0xd5, 0xee, 0x4f, 0x74, 0x39, 0x2, 0xf1, 0xca, 0x87, 0xbc, 0x1d, 0x26, 0x6b, 0x50, 0x34, 0xf, 0x42, 0x79, 0xd8, 0xe3, 0xae, 0x95, 0x55, 0x6e, 0x23, 0x18, 0xb9, 0x82, 0xcf, 0xf4, 0x90, 0xab, 0xe6, 0xdd, 0x7c, 0x47, 0xa, 0x31, 0xc2, 0xf9, 0xb4, 0x8f, 0x2e, 0x15, 0x58, 0x63, 0x7, 0x3c, 0x71, 0x4a, 0xeb, 0xd0, 0x9d, 0xa6, 0xcc, 0xf7, 0xba, 0x81, 0x20, 0x1b, 0x56, 0x6d, 0x9, 0x32, 0x7f, 0x44, 0xe5, 0xde, 0x93, 0xa8, 0x5b, 0x60, 0x2d, 0x16, 0xb7, 0x8c, 0xc1, 0xfa, 0x9e, 0xa5, 0xe8, 0xd3, 0x72, 0x49, 0x4, 0x3f, 0xff, 0xc4, 0x89, 0xb2, 0x13, 0x28, 0x65, 0x5e, 0x3a, 0x1, 0x4c, 0x77, 0xd6, 0xed, 0xa0, 0x9b, 0x68, 0x53, 0x1e, 0x25, 0x84, 0xbf, 0xf2, 0xc9, 0xad, 0x96, 0xdb, 0xe0, 0x41, 0x7a, 0x37, 0xc, 0xaa, 0x91, 0xdc, 0xe7, 0x46, 0x7d, 0x30, 0xb, 0x6f, 0x54, 0x19, 0x22, 0x83, 0xb8, 0xf5, 0xce, 0x3d, 0x6, 0x4b, 0x70, 0xd1, 0xea, 0xa7, 0x9c, 0xf8, 0xc3, 0x8e, 0xb5, 0x14, 0x2f, 0x62, 0x59, 0x99, 0xa2, 0xef, 0xd4, 0x75, 0x4e, 0x3, 0x38, 0x5c, 0x67, 0x2a, 0x11, 0xb0, 0x8b, 0xc6, 0xfd, 0xe, 0x35, 0x78, 0x43, 0xe2, 0xd9, 0x94, 0xaf, 0xcb, 0xf0, 0xbd, 0x86, 0x27, 0x1c, 0x51, 0x6a},
- {0x0, 0x3c, 0x78, 0x44, 0xf0, 0xcc, 0x88, 0xb4, 0xfd, 0xc1, 0x85, 0xb9, 0xd, 0x31, 0x75, 0x49, 0xe7, 0xdb, 0x9f, 0xa3, 0x17, 0x2b, 0x6f, 0x53, 0x1a, 0x26, 0x62, 0x5e, 0xea, 0xd6, 0x92, 0xae, 0xd3, 0xef, 0xab, 0x97, 0x23, 0x1f, 0x5b, 0x67, 0x2e, 0x12, 0x56, 0x6a, 0xde, 0xe2, 0xa6, 0x9a, 0x34, 0x8, 0x4c, 0x70, 0xc4, 0xf8, 0xbc, 0x80, 0xc9, 0xf5, 0xb1, 0x8d, 0x39, 0x5, 0x41, 0x7d, 0xbb, 0x87, 0xc3, 0xff, 0x4b, 0x77, 0x33, 0xf, 0x46, 0x7a, 0x3e, 0x2, 0xb6, 0x8a, 0xce, 0xf2, 0x5c, 0x60, 0x24, 0x18, 0xac, 0x90, 0xd4, 0xe8, 0xa1, 0x9d, 0xd9, 0xe5, 0x51, 0x6d, 0x29, 0x15, 0x68, 0x54, 0x10, 0x2c, 0x98, 0xa4, 0xe0, 0xdc, 0x95, 0xa9, 0xed, 0xd1, 0x65, 0x59, 0x1d, 0x21, 0x8f, 0xb3, 0xf7, 0xcb, 0x7f, 0x43, 0x7, 0x3b, 0x72, 0x4e, 0xa, 0x36, 0x82, 0xbe, 0xfa, 0xc6, 0x6b, 0x57, 0x13, 0x2f, 0x9b, 0xa7, 0xe3, 0xdf, 0x96, 0xaa, 0xee, 0xd2, 0x66, 0x5a, 0x1e, 0x22, 0x8c, 0xb0, 0xf4, 0xc8, 0x7c, 0x40, 0x4, 0x38, 0x71, 0x4d, 0x9, 0x35, 0x81, 0xbd, 0xf9, 0xc5, 0xb8, 0x84, 0xc0, 0xfc, 0x48, 0x74, 0x30, 0xc, 0x45, 0x79, 0x3d, 0x1, 0xb5, 0x89, 0xcd, 0xf1, 0x5f, 0x63, 0x27, 0x1b, 0xaf, 0x93, 0xd7, 0xeb, 0xa2, 0x9e, 0xda, 0xe6, 0x52, 0x6e, 0x2a, 0x16, 0xd0, 0xec, 0xa8, 0x94, 0x20, 0x1c, 0x58, 0x64, 0x2d, 0x11, 0x55, 0x69, 0xdd, 0xe1, 0xa5, 0x99, 0x37, 0xb, 0x4f, 0x73, 0xc7, 0xfb, 0xbf, 0x83, 0xca, 0xf6, 0xb2, 0x8e, 0x3a, 0x6, 0x42, 0x7e, 0x3, 0x3f, 0x7b, 0x47, 0xf3, 0xcf, 0x8b, 0xb7, 0xfe, 0xc2, 0x86, 0xba, 0xe, 0x32, 0x76, 0x4a, 0xe4, 0xd8, 0x9c, 0xa0, 0x14, 0x28, 0x6c, 0x50, 0x19, 0x25, 0x61, 0x5d, 0xe9, 0xd5, 0x91, 0xad},
- {0x0, 0x3d, 0x7a, 0x47, 0xf4, 0xc9, 0x8e, 0xb3, 0xf5, 0xc8, 0x8f, 0xb2, 0x1, 0x3c, 0x7b, 0x46, 0xf7, 0xca, 0x8d, 0xb0, 0x3, 0x3e, 0x79, 0x44, 0x2, 0x3f, 0x78, 0x45, 0xf6, 0xcb, 0x8c, 0xb1, 0xf3, 0xce, 0x89, 0xb4, 0x7, 0x3a, 0x7d, 0x40, 0x6, 0x3b, 0x7c, 0x41, 0xf2, 0xcf, 0x88, 0xb5, 0x4, 0x39, 0x7e, 0x43, 0xf0, 0xcd, 0x8a, 0xb7, 0xf1, 0xcc, 0x8b, 0xb6, 0x5, 0x38, 0x7f, 0x42, 0xfb, 0xc6, 0x81, 0xbc, 0xf, 0x32, 0x75, 0x48, 0xe, 0x33, 0x74, 0x49, 0xfa, 0xc7, 0x80, 0xbd, 0xc, 0x31, 0x76, 0x4b, 0xf8, 0xc5, 0x82, 0xbf, 0xf9, 0xc4, 0x83, 0xbe, 0xd, 0x30, 0x77, 0x4a, 0x8, 0x35, 0x72, 0x4f, 0xfc, 0xc1, 0x86, 0xbb, 0xfd, 0xc0, 0x87, 0xba, 0x9, 0x34, 0x73, 0x4e, 0xff, 0xc2, 0x85, 0xb8, 0xb, 0x36, 0x71, 0x4c, 0xa, 0x37, 0x70, 0x4d, 0xfe, 0xc3, 0x84, 0xb9, 0xeb, 0xd6, 0x91, 0xac, 0x1f, 0x22, 0x65, 0x58, 0x1e, 0x23, 0x64, 0x59, 0xea, 0xd7, 0x90, 0xad, 0x1c, 0x21, 0x66, 0x5b, 0xe8, 0xd5, 0x92, 0xaf, 0xe9, 0xd4, 0x93, 0xae, 0x1d, 0x20, 0x67, 0x5a, 0x18, 0x25, 0x62, 0x5f, 0xec, 0xd1, 0x96, 0xab, 0xed, 0xd0, 0x97, 0xaa, 0x19, 0x24, 0x63, 0x5e, 0xef, 0xd2, 0x95, 0xa8, 0x1b, 0x26, 0x61, 0x5c, 0x1a, 0x27, 0x60, 0x5d, 0xee, 0xd3, 0x94, 0xa9, 0x10, 0x2d, 0x6a, 0x57, 0xe4, 0xd9, 0x9e, 0xa3, 0xe5, 0xd8, 0x9f, 0xa2, 0x11, 0x2c, 0x6b, 0x56, 0xe7, 0xda, 0x9d, 0xa0, 0x13, 0x2e, 0x69, 0x54, 0x12, 0x2f, 0x68, 0x55, 0xe6, 0xdb, 0x9c, 0xa1, 0xe3, 0xde, 0x99, 0xa4, 0x17, 0x2a, 0x6d, 0x50, 0x16, 0x2b, 0x6c, 0x51, 0xe2, 0xdf, 0x98, 0xa5, 0x14, 0x29, 0x6e, 0x53, 0xe0, 0xdd, 0x9a, 0xa7, 0xe1, 0xdc, 0x9b, 0xa6, 0x15, 0x28, 0x6f, 0x52},
- {0x0, 0x3e, 0x7c, 0x42, 0xf8, 0xc6, 0x84, 0xba, 0xed, 0xd3, 0x91, 0xaf, 0x15, 0x2b, 0x69, 0x57, 0xc7, 0xf9, 0xbb, 0x85, 0x3f, 0x1, 0x43, 0x7d, 0x2a, 0x14, 0x56, 0x68, 0xd2, 0xec, 0xae, 0x90, 0x93, 0xad, 0xef, 0xd1, 0x6b, 0x55, 0x17, 0x29, 0x7e, 0x40, 0x2, 0x3c, 0x86, 0xb8, 0xfa, 0xc4, 0x54, 0x6a, 0x28, 0x16, 0xac, 0x92, 0xd0, 0xee, 0xb9, 0x87, 0xc5, 0xfb, 0x41, 0x7f, 0x3d, 0x3, 0x3b, 0x5, 0x47, 0x79, 0xc3, 0xfd, 0xbf, 0x81, 0xd6, 0xe8, 0xaa, 0x94, 0x2e, 0x10, 0x52, 0x6c, 0xfc, 0xc2, 0x80, 0xbe, 0x4, 0x3a, 0x78, 0x46, 0x11, 0x2f, 0x6d, 0x53, 0xe9, 0xd7, 0x95, 0xab, 0xa8, 0x96, 0xd4, 0xea, 0x50, 0x6e, 0x2c, 0x12, 0x45, 0x7b, 0x39, 0x7, 0xbd, 0x83, 0xc1, 0xff, 0x6f, 0x51, 0x13, 0x2d, 0x97, 0xa9, 0xeb, 0xd5, 0x82, 0xbc, 0xfe, 0xc0, 0x7a, 0x44, 0x6, 0x38, 0x76, 0x48, 0xa, 0x34, 0x8e, 0xb0, 0xf2, 0xcc, 0x9b, 0xa5, 0xe7, 0xd9, 0x63, 0x5d, 0x1f, 0x21, 0xb1, 0x8f, 0xcd, 0xf3, 0x49, 0x77, 0x35, 0xb, 0x5c, 0x62, 0x20, 0x1e, 0xa4, 0x9a, 0xd8, 0xe6, 0xe5, 0xdb, 0x99, 0xa7, 0x1d, 0x23, 0x61, 0x5f, 0x8, 0x36, 0x74, 0x4a, 0xf0, 0xce, 0x8c, 0xb2, 0x22, 0x1c, 0x5e, 0x60, 0xda, 0xe4, 0xa6, 0x98, 0xcf, 0xf1, 0xb3, 0x8d, 0x37, 0x9, 0x4b, 0x75, 0x4d, 0x73, 0x31, 0xf, 0xb5, 0x8b, 0xc9, 0xf7, 0xa0, 0x9e, 0xdc, 0xe2, 0x58, 0x66, 0x24, 0x1a, 0x8a, 0xb4, 0xf6, 0xc8, 0x72, 0x4c, 0xe, 0x30, 0x67, 0x59, 0x1b, 0x25, 0x9f, 0xa1, 0xe3, 0xdd, 0xde, 0xe0, 0xa2, 0x9c, 0x26, 0x18, 0x5a, 0x64, 0x33, 0xd, 0x4f, 0x71, 0xcb, 0xf5, 0xb7, 0x89, 0x19, 0x27, 0x65, 0x5b, 0xe1, 0xdf, 0x9d, 0xa3, 0xf4, 0xca, 0x88, 0xb6, 0xc, 0x32, 0x70, 0x4e},
- {0x0, 0x3f, 0x7e, 0x41, 0xfc, 0xc3, 0x82, 0xbd, 0xe5, 0xda, 0x9b, 0xa4, 0x19, 0x26, 0x67, 0x58, 0xd7, 0xe8, 0xa9, 0x96, 0x2b, 0x14, 0x55, 0x6a, 0x32, 0xd, 0x4c, 0x73, 0xce, 0xf1, 0xb0, 0x8f, 0xb3, 0x8c, 0xcd, 0xf2, 0x4f, 0x70, 0x31, 0xe, 0x56, 0x69, 0x28, 0x17, 0xaa, 0x95, 0xd4, 0xeb, 0x64, 0x5b, 0x1a, 0x25, 0x98, 0xa7, 0xe6, 0xd9, 0x81, 0xbe, 0xff, 0xc0, 0x7d, 0x42, 0x3, 0x3c, 0x7b, 0x44, 0x5, 0x3a, 0x87, 0xb8, 0xf9, 0xc6, 0x9e, 0xa1, 0xe0, 0xdf, 0x62, 0x5d, 0x1c, 0x23, 0xac, 0x93, 0xd2, 0xed, 0x50, 0x6f, 0x2e, 0x11, 0x49, 0x76, 0x37, 0x8, 0xb5, 0x8a, 0xcb, 0xf4, 0xc8, 0xf7, 0xb6, 0x89, 0x34, 0xb, 0x4a, 0x75, 0x2d, 0x12, 0x53, 0x6c, 0xd1, 0xee, 0xaf, 0x90, 0x1f, 0x20, 0x61, 0x5e, 0xe3, 0xdc, 0x9d, 0xa2, 0xfa, 0xc5, 0x84, 0xbb, 0x6, 0x39, 0x78, 0x47, 0xf6, 0xc9, 0x88, 0xb7, 0xa, 0x35, 0x74, 0x4b, 0x13, 0x2c, 0x6d, 0x52, 0xef, 0xd0, 0x91, 0xae, 0x21, 0x1e, 0x5f, 0x60, 0xdd, 0xe2, 0xa3, 0x9c, 0xc4, 0xfb, 0xba, 0x85, 0x38, 0x7, 0x46, 0x79, 0x45, 0x7a, 0x3b, 0x4, 0xb9, 0x86, 0xc7, 0xf8, 0xa0, 0x9f, 0xde, 0xe1, 0x5c, 0x63, 0x22, 0x1d, 0x92, 0xad, 0xec, 0xd3, 0x6e, 0x51, 0x10, 0x2f, 0x77, 0x48, 0x9, 0x36, 0x8b, 0xb4, 0xf5, 0xca, 0x8d, 0xb2, 0xf3, 0xcc, 0x71, 0x4e, 0xf, 0x30, 0x68, 0x57, 0x16, 0x29, 0x94, 0xab, 0xea, 0xd5, 0x5a, 0x65, 0x24, 0x1b, 0xa6, 0x99, 0xd8, 0xe7, 0xbf, 0x80, 0xc1, 0xfe, 0x43, 0x7c, 0x3d, 0x2, 0x3e, 0x1, 0x40, 0x7f, 0xc2, 0xfd, 0xbc, 0x83, 0xdb, 0xe4, 0xa5, 0x9a, 0x27, 0x18, 0x59, 0x66, 0xe9, 0xd6, 0x97, 0xa8, 0x15, 0x2a, 0x6b, 0x54, 0xc, 0x33, 0x72, 0x4d, 0xf0, 0xcf, 0x8e, 0xb1},
- {0x0, 0x40, 0x80, 0xc0, 0x1d, 0x5d, 0x9d, 0xdd, 0x3a, 0x7a, 0xba, 0xfa, 0x27, 0x67, 0xa7, 0xe7, 0x74, 0x34, 0xf4, 0xb4, 0x69, 0x29, 0xe9, 0xa9, 0x4e, 0xe, 0xce, 0x8e, 0x53, 0x13, 0xd3, 0x93, 0xe8, 0xa8, 0x68, 0x28, 0xf5, 0xb5, 0x75, 0x35, 0xd2, 0x92, 0x52, 0x12, 0xcf, 0x8f, 0x4f, 0xf, 0x9c, 0xdc, 0x1c, 0x5c, 0x81, 0xc1, 0x1, 0x41, 0xa6, 0xe6, 0x26, 0x66, 0xbb, 0xfb, 0x3b, 0x7b, 0xcd, 0x8d, 0x4d, 0xd, 0xd0, 0x90, 0x50, 0x10, 0xf7, 0xb7, 0x77, 0x37, 0xea, 0xaa, 0x6a, 0x2a, 0xb9, 0xf9, 0x39, 0x79, 0xa4, 0xe4, 0x24, 0x64, 0x83, 0xc3, 0x3, 0x43, 0x9e, 0xde, 0x1e, 0x5e, 0x25, 0x65, 0xa5, 0xe5, 0x38, 0x78, 0xb8, 0xf8, 0x1f, 0x5f, 0x9f, 0xdf, 0x2, 0x42, 0x82, 0xc2, 0x51, 0x11, 0xd1, 0x91, 0x4c, 0xc, 0xcc, 0x8c, 0x6b, 0x2b, 0xeb, 0xab, 0x76, 0x36, 0xf6, 0xb6, 0x87, 0xc7, 0x7, 0x47, 0x9a, 0xda, 0x1a, 0x5a, 0xbd, 0xfd, 0x3d, 0x7d, 0xa0, 0xe0, 0x20, 0x60, 0xf3, 0xb3, 0x73, 0x33, 0xee, 0xae, 0x6e, 0x2e, 0xc9, 0x89, 0x49, 0x9, 0xd4, 0x94, 0x54, 0x14, 0x6f, 0x2f, 0xef, 0xaf, 0x72, 0x32, 0xf2, 0xb2, 0x55, 0x15, 0xd5, 0x95, 0x48, 0x8, 0xc8, 0x88, 0x1b, 0x5b, 0x9b, 0xdb, 0x6, 0x46, 0x86, 0xc6, 0x21, 0x61, 0xa1, 0xe1, 0x3c, 0x7c, 0xbc, 0xfc, 0x4a, 0xa, 0xca, 0x8a, 0x57, 0x17, 0xd7, 0x97, 0x70, 0x30, 0xf0, 0xb0, 0x6d, 0x2d, 0xed, 0xad, 0x3e, 0x7e, 0xbe, 0xfe, 0x23, 0x63, 0xa3, 0xe3, 0x4, 0x44, 0x84, 0xc4, 0x19, 0x59, 0x99, 0xd9, 0xa2, 0xe2, 0x22, 0x62, 0xbf, 0xff, 0x3f, 0x7f, 0x98, 0xd8, 0x18, 0x58, 0x85, 0xc5, 0x5, 0x45, 0xd6, 0x96, 0x56, 0x16, 0xcb, 0x8b, 0x4b, 0xb, 0xec, 0xac, 0x6c, 0x2c, 0xf1, 0xb1, 0x71, 0x31},
- {0x0, 0x41, 0x82, 0xc3, 0x19, 0x58, 0x9b, 0xda, 0x32, 0x73, 0xb0, 0xf1, 0x2b, 0x6a, 0xa9, 0xe8, 0x64, 0x25, 0xe6, 0xa7, 0x7d, 0x3c, 0xff, 0xbe, 0x56, 0x17, 0xd4, 0x95, 0x4f, 0xe, 0xcd, 0x8c, 0xc8, 0x89, 0x4a, 0xb, 0xd1, 0x90, 0x53, 0x12, 0xfa, 0xbb, 0x78, 0x39, 0xe3, 0xa2, 0x61, 0x20, 0xac, 0xed, 0x2e, 0x6f, 0xb5, 0xf4, 0x37, 0x76, 0x9e, 0xdf, 0x1c, 0x5d, 0x87, 0xc6, 0x5, 0x44, 0x8d, 0xcc, 0xf, 0x4e, 0x94, 0xd5, 0x16, 0x57, 0xbf, 0xfe, 0x3d, 0x7c, 0xa6, 0xe7, 0x24, 0x65, 0xe9, 0xa8, 0x6b, 0x2a, 0xf0, 0xb1, 0x72, 0x33, 0xdb, 0x9a, 0x59, 0x18, 0xc2, 0x83, 0x40, 0x1, 0x45, 0x4, 0xc7, 0x86, 0x5c, 0x1d, 0xde, 0x9f, 0x77, 0x36, 0xf5, 0xb4, 0x6e, 0x2f, 0xec, 0xad, 0x21, 0x60, 0xa3, 0xe2, 0x38, 0x79, 0xba, 0xfb, 0x13, 0x52, 0x91, 0xd0, 0xa, 0x4b, 0x88, 0xc9, 0x7, 0x46, 0x85, 0xc4, 0x1e, 0x5f, 0x9c, 0xdd, 0x35, 0x74, 0xb7, 0xf6, 0x2c, 0x6d, 0xae, 0xef, 0x63, 0x22, 0xe1, 0xa0, 0x7a, 0x3b, 0xf8, 0xb9, 0x51, 0x10, 0xd3, 0x92, 0x48, 0x9, 0xca, 0x8b, 0xcf, 0x8e, 0x4d, 0xc, 0xd6, 0x97, 0x54, 0x15, 0xfd, 0xbc, 0x7f, 0x3e, 0xe4, 0xa5, 0x66, 0x27, 0xab, 0xea, 0x29, 0x68, 0xb2, 0xf3, 0x30, 0x71, 0x99, 0xd8, 0x1b, 0x5a, 0x80, 0xc1, 0x2, 0x43, 0x8a, 0xcb, 0x8, 0x49, 0x93, 0xd2, 0x11, 0x50, 0xb8, 0xf9, 0x3a, 0x7b, 0xa1, 0xe0, 0x23, 0x62, 0xee, 0xaf, 0x6c, 0x2d, 0xf7, 0xb6, 0x75, 0x34, 0xdc, 0x9d, 0x5e, 0x1f, 0xc5, 0x84, 0x47, 0x6, 0x42, 0x3, 0xc0, 0x81, 0x5b, 0x1a, 0xd9, 0x98, 0x70, 0x31, 0xf2, 0xb3, 0x69, 0x28, 0xeb, 0xaa, 0x26, 0x67, 0xa4, 0xe5, 0x3f, 0x7e, 0xbd, 0xfc, 0x14, 0x55, 0x96, 0xd7, 0xd, 0x4c, 0x8f, 0xce},
- {0x0, 0x42, 0x84, 0xc6, 0x15, 0x57, 0x91, 0xd3, 0x2a, 0x68, 0xae, 0xec, 0x3f, 0x7d, 0xbb, 0xf9, 0x54, 0x16, 0xd0, 0x92, 0x41, 0x3, 0xc5, 0x87, 0x7e, 0x3c, 0xfa, 0xb8, 0x6b, 0x29, 0xef, 0xad, 0xa8, 0xea, 0x2c, 0x6e, 0xbd, 0xff, 0x39, 0x7b, 0x82, 0xc0, 0x6, 0x44, 0x97, 0xd5, 0x13, 0x51, 0xfc, 0xbe, 0x78, 0x3a, 0xe9, 0xab, 0x6d, 0x2f, 0xd6, 0x94, 0x52, 0x10, 0xc3, 0x81, 0x47, 0x5, 0x4d, 0xf, 0xc9, 0x8b, 0x58, 0x1a, 0xdc, 0x9e, 0x67, 0x25, 0xe3, 0xa1, 0x72, 0x30, 0xf6, 0xb4, 0x19, 0x5b, 0x9d, 0xdf, 0xc, 0x4e, 0x88, 0xca, 0x33, 0x71, 0xb7, 0xf5, 0x26, 0x64, 0xa2, 0xe0, 0xe5, 0xa7, 0x61, 0x23, 0xf0, 0xb2, 0x74, 0x36, 0xcf, 0x8d, 0x4b, 0x9, 0xda, 0x98, 0x5e, 0x1c, 0xb1, 0xf3, 0x35, 0x77, 0xa4, 0xe6, 0x20, 0x62, 0x9b, 0xd9, 0x1f, 0x5d, 0x8e, 0xcc, 0xa, 0x48, 0x9a, 0xd8, 0x1e, 0x5c, 0x8f, 0xcd, 0xb, 0x49, 0xb0, 0xf2, 0x34, 0x76, 0xa5, 0xe7, 0x21, 0x63, 0xce, 0x8c, 0x4a, 0x8, 0xdb, 0x99, 0x5f, 0x1d, 0xe4, 0xa6, 0x60, 0x22, 0xf1, 0xb3, 0x75, 0x37, 0x32, 0x70, 0xb6, 0xf4, 0x27, 0x65, 0xa3, 0xe1, 0x18, 0x5a, 0x9c, 0xde, 0xd, 0x4f, 0x89, 0xcb, 0x66, 0x24, 0xe2, 0xa0, 0x73, 0x31, 0xf7, 0xb5, 0x4c, 0xe, 0xc8, 0x8a, 0x59, 0x1b, 0xdd, 0x9f, 0xd7, 0x95, 0x53, 0x11, 0xc2, 0x80, 0x46, 0x4, 0xfd, 0xbf, 0x79, 0x3b, 0xe8, 0xaa, 0x6c, 0x2e, 0x83, 0xc1, 0x7, 0x45, 0x96, 0xd4, 0x12, 0x50, 0xa9, 0xeb, 0x2d, 0x6f, 0xbc, 0xfe, 0x38, 0x7a, 0x7f, 0x3d, 0xfb, 0xb9, 0x6a, 0x28, 0xee, 0xac, 0x55, 0x17, 0xd1, 0x93, 0x40, 0x2, 0xc4, 0x86, 0x2b, 0x69, 0xaf, 0xed, 0x3e, 0x7c, 0xba, 0xf8, 0x1, 0x43, 0x85, 0xc7, 0x14, 0x56, 0x90, 0xd2},
- {0x0, 0x43, 0x86, 0xc5, 0x11, 0x52, 0x97, 0xd4, 0x22, 0x61, 0xa4, 0xe7, 0x33, 0x70, 0xb5, 0xf6, 0x44, 0x7, 0xc2, 0x81, 0x55, 0x16, 0xd3, 0x90, 0x66, 0x25, 0xe0, 0xa3, 0x77, 0x34, 0xf1, 0xb2, 0x88, 0xcb, 0xe, 0x4d, 0x99, 0xda, 0x1f, 0x5c, 0xaa, 0xe9, 0x2c, 0x6f, 0xbb, 0xf8, 0x3d, 0x7e, 0xcc, 0x8f, 0x4a, 0x9, 0xdd, 0x9e, 0x5b, 0x18, 0xee, 0xad, 0x68, 0x2b, 0xff, 0xbc, 0x79, 0x3a, 0xd, 0x4e, 0x8b, 0xc8, 0x1c, 0x5f, 0x9a, 0xd9, 0x2f, 0x6c, 0xa9, 0xea, 0x3e, 0x7d, 0xb8, 0xfb, 0x49, 0xa, 0xcf, 0x8c, 0x58, 0x1b, 0xde, 0x9d, 0x6b, 0x28, 0xed, 0xae, 0x7a, 0x39, 0xfc, 0xbf, 0x85, 0xc6, 0x3, 0x40, 0x94, 0xd7, 0x12, 0x51, 0xa7, 0xe4, 0x21, 0x62, 0xb6, 0xf5, 0x30, 0x73, 0xc1, 0x82, 0x47, 0x4, 0xd0, 0x93, 0x56, 0x15, 0xe3, 0xa0, 0x65, 0x26, 0xf2, 0xb1, 0x74, 0x37, 0x1a, 0x59, 0x9c, 0xdf, 0xb, 0x48, 0x8d, 0xce, 0x38, 0x7b, 0xbe, 0xfd, 0x29, 0x6a, 0xaf, 0xec, 0x5e, 0x1d, 0xd8, 0x9b, 0x4f, 0xc, 0xc9, 0x8a, 0x7c, 0x3f, 0xfa, 0xb9, 0x6d, 0x2e, 0xeb, 0xa8, 0x92, 0xd1, 0x14, 0x57, 0x83, 0xc0, 0x5, 0x46, 0xb0, 0xf3, 0x36, 0x75, 0xa1, 0xe2, 0x27, 0x64, 0xd6, 0x95, 0x50, 0x13, 0xc7, 0x84, 0x41, 0x2, 0xf4, 0xb7, 0x72, 0x31, 0xe5, 0xa6, 0x63, 0x20, 0x17, 0x54, 0x91, 0xd2, 0x6, 0x45, 0x80, 0xc3, 0x35, 0x76, 0xb3, 0xf0, 0x24, 0x67, 0xa2, 0xe1, 0x53, 0x10, 0xd5, 0x96, 0x42, 0x1, 0xc4, 0x87, 0x71, 0x32, 0xf7, 0xb4, 0x60, 0x23, 0xe6, 0xa5, 0x9f, 0xdc, 0x19, 0x5a, 0x8e, 0xcd, 0x8, 0x4b, 0xbd, 0xfe, 0x3b, 0x78, 0xac, 0xef, 0x2a, 0x69, 0xdb, 0x98, 0x5d, 0x1e, 0xca, 0x89, 0x4c, 0xf, 0xf9, 0xba, 0x7f, 0x3c, 0xe8, 0xab, 0x6e, 0x2d},
- {0x0, 0x44, 0x88, 0xcc, 0xd, 0x49, 0x85, 0xc1, 0x1a, 0x5e, 0x92, 0xd6, 0x17, 0x53, 0x9f, 0xdb, 0x34, 0x70, 0xbc, 0xf8, 0x39, 0x7d, 0xb1, 0xf5, 0x2e, 0x6a, 0xa6, 0xe2, 0x23, 0x67, 0xab, 0xef, 0x68, 0x2c, 0xe0, 0xa4, 0x65, 0x21, 0xed, 0xa9, 0x72, 0x36, 0xfa, 0xbe, 0x7f, 0x3b, 0xf7, 0xb3, 0x5c, 0x18, 0xd4, 0x90, 0x51, 0x15, 0xd9, 0x9d, 0x46, 0x2, 0xce, 0x8a, 0x4b, 0xf, 0xc3, 0x87, 0xd0, 0x94, 0x58, 0x1c, 0xdd, 0x99, 0x55, 0x11, 0xca, 0x8e, 0x42, 0x6, 0xc7, 0x83, 0x4f, 0xb, 0xe4, 0xa0, 0x6c, 0x28, 0xe9, 0xad, 0x61, 0x25, 0xfe, 0xba, 0x76, 0x32, 0xf3, 0xb7, 0x7b, 0x3f, 0xb8, 0xfc, 0x30, 0x74, 0xb5, 0xf1, 0x3d, 0x79, 0xa2, 0xe6, 0x2a, 0x6e, 0xaf, 0xeb, 0x27, 0x63, 0x8c, 0xc8, 0x4, 0x40, 0x81, 0xc5, 0x9, 0x4d, 0x96, 0xd2, 0x1e, 0x5a, 0x9b, 0xdf, 0x13, 0x57, 0xbd, 0xf9, 0x35, 0x71, 0xb0, 0xf4, 0x38, 0x7c, 0xa7, 0xe3, 0x2f, 0x6b, 0xaa, 0xee, 0x22, 0x66, 0x89, 0xcd, 0x1, 0x45, 0x84, 0xc0, 0xc, 0x48, 0x93, 0xd7, 0x1b, 0x5f, 0x9e, 0xda, 0x16, 0x52, 0xd5, 0x91, 0x5d, 0x19, 0xd8, 0x9c, 0x50, 0x14, 0xcf, 0x8b, 0x47, 0x3, 0xc2, 0x86, 0x4a, 0xe, 0xe1, 0xa5, 0x69, 0x2d, 0xec, 0xa8, 0x64, 0x20, 0xfb, 0xbf, 0x73, 0x37, 0xf6, 0xb2, 0x7e, 0x3a, 0x6d, 0x29, 0xe5, 0xa1, 0x60, 0x24, 0xe8, 0xac, 0x77, 0x33, 0xff, 0xbb, 0x7a, 0x3e, 0xf2, 0xb6, 0x59, 0x1d, 0xd1, 0x95, 0x54, 0x10, 0xdc, 0x98, 0x43, 0x7, 0xcb, 0x8f, 0x4e, 0xa, 0xc6, 0x82, 0x5, 0x41, 0x8d, 0xc9, 0x8, 0x4c, 0x80, 0xc4, 0x1f, 0x5b, 0x97, 0xd3, 0x12, 0x56, 0x9a, 0xde, 0x31, 0x75, 0xb9, 0xfd, 0x3c, 0x78, 0xb4, 0xf0, 0x2b, 0x6f, 0xa3, 0xe7, 0x26, 0x62, 0xae, 0xea},
- {0x0, 0x45, 0x8a, 0xcf, 0x9, 0x4c, 0x83, 0xc6, 0x12, 0x57, 0x98, 0xdd, 0x1b, 0x5e, 0x91, 0xd4, 0x24, 0x61, 0xae, 0xeb, 0x2d, 0x68, 0xa7, 0xe2, 0x36, 0x73, 0xbc, 0xf9, 0x3f, 0x7a, 0xb5, 0xf0, 0x48, 0xd, 0xc2, 0x87, 0x41, 0x4, 0xcb, 0x8e, 0x5a, 0x1f, 0xd0, 0x95, 0x53, 0x16, 0xd9, 0x9c, 0x6c, 0x29, 0xe6, 0xa3, 0x65, 0x20, 0xef, 0xaa, 0x7e, 0x3b, 0xf4, 0xb1, 0x77, 0x32, 0xfd, 0xb8, 0x90, 0xd5, 0x1a, 0x5f, 0x99, 0xdc, 0x13, 0x56, 0x82, 0xc7, 0x8, 0x4d, 0x8b, 0xce, 0x1, 0x44, 0xb4, 0xf1, 0x3e, 0x7b, 0xbd, 0xf8, 0x37, 0x72, 0xa6, 0xe3, 0x2c, 0x69, 0xaf, 0xea, 0x25, 0x60, 0xd8, 0x9d, 0x52, 0x17, 0xd1, 0x94, 0x5b, 0x1e, 0xca, 0x8f, 0x40, 0x5, 0xc3, 0x86, 0x49, 0xc, 0xfc, 0xb9, 0x76, 0x33, 0xf5, 0xb0, 0x7f, 0x3a, 0xee, 0xab, 0x64, 0x21, 0xe7, 0xa2, 0x6d, 0x28, 0x3d, 0x78, 0xb7, 0xf2, 0x34, 0x71, 0xbe, 0xfb, 0x2f, 0x6a, 0xa5, 0xe0, 0x26, 0x63, 0xac, 0xe9, 0x19, 0x5c, 0x93, 0xd6, 0x10, 0x55, 0x9a, 0xdf, 0xb, 0x4e, 0x81, 0xc4, 0x2, 0x47, 0x88, 0xcd, 0x75, 0x30, 0xff, 0xba, 0x7c, 0x39, 0xf6, 0xb3, 0x67, 0x22, 0xed, 0xa8, 0x6e, 0x2b, 0xe4, 0xa1, 0x51, 0x14, 0xdb, 0x9e, 0x58, 0x1d, 0xd2, 0x97, 0x43, 0x6, 0xc9, 0x8c, 0x4a, 0xf, 0xc0, 0x85, 0xad, 0xe8, 0x27, 0x62, 0xa4, 0xe1, 0x2e, 0x6b, 0xbf, 0xfa, 0x35, 0x70, 0xb6, 0xf3, 0x3c, 0x79, 0x89, 0xcc, 0x3, 0x46, 0x80, 0xc5, 0xa, 0x4f, 0x9b, 0xde, 0x11, 0x54, 0x92, 0xd7, 0x18, 0x5d, 0xe5, 0xa0, 0x6f, 0x2a, 0xec, 0xa9, 0x66, 0x23, 0xf7, 0xb2, 0x7d, 0x38, 0xfe, 0xbb, 0x74, 0x31, 0xc1, 0x84, 0x4b, 0xe, 0xc8, 0x8d, 0x42, 0x7, 0xd3, 0x96, 0x59, 0x1c, 0xda, 0x9f, 0x50, 0x15},
- {0x0, 0x46, 0x8c, 0xca, 0x5, 0x43, 0x89, 0xcf, 0xa, 0x4c, 0x86, 0xc0, 0xf, 0x49, 0x83, 0xc5, 0x14, 0x52, 0x98, 0xde, 0x11, 0x57, 0x9d, 0xdb, 0x1e, 0x58, 0x92, 0xd4, 0x1b, 0x5d, 0x97, 0xd1, 0x28, 0x6e, 0xa4, 0xe2, 0x2d, 0x6b, 0xa1, 0xe7, 0x22, 0x64, 0xae, 0xe8, 0x27, 0x61, 0xab, 0xed, 0x3c, 0x7a, 0xb0, 0xf6, 0x39, 0x7f, 0xb5, 0xf3, 0x36, 0x70, 0xba, 0xfc, 0x33, 0x75, 0xbf, 0xf9, 0x50, 0x16, 0xdc, 0x9a, 0x55, 0x13, 0xd9, 0x9f, 0x5a, 0x1c, 0xd6, 0x90, 0x5f, 0x19, 0xd3, 0x95, 0x44, 0x2, 0xc8, 0x8e, 0x41, 0x7, 0xcd, 0x8b, 0x4e, 0x8, 0xc2, 0x84, 0x4b, 0xd, 0xc7, 0x81, 0x78, 0x3e, 0xf4, 0xb2, 0x7d, 0x3b, 0xf1, 0xb7, 0x72, 0x34, 0xfe, 0xb8, 0x77, 0x31, 0xfb, 0xbd, 0x6c, 0x2a, 0xe0, 0xa6, 0x69, 0x2f, 0xe5, 0xa3, 0x66, 0x20, 0xea, 0xac, 0x63, 0x25, 0xef, 0xa9, 0xa0, 0xe6, 0x2c, 0x6a, 0xa5, 0xe3, 0x29, 0x6f, 0xaa, 0xec, 0x26, 0x60, 0xaf, 0xe9, 0x23, 0x65, 0xb4, 0xf2, 0x38, 0x7e, 0xb1, 0xf7, 0x3d, 0x7b, 0xbe, 0xf8, 0x32, 0x74, 0xbb, 0xfd, 0x37, 0x71, 0x88, 0xce, 0x4, 0x42, 0x8d, 0xcb, 0x1, 0x47, 0x82, 0xc4, 0xe, 0x48, 0x87, 0xc1, 0xb, 0x4d, 0x9c, 0xda, 0x10, 0x56, 0x99, 0xdf, 0x15, 0x53, 0x96, 0xd0, 0x1a, 0x5c, 0x93, 0xd5, 0x1f, 0x59, 0xf0, 0xb6, 0x7c, 0x3a, 0xf5, 0xb3, 0x79, 0x3f, 0xfa, 0xbc, 0x76, 0x30, 0xff, 0xb9, 0x73, 0x35, 0xe4, 0xa2, 0x68, 0x2e, 0xe1, 0xa7, 0x6d, 0x2b, 0xee, 0xa8, 0x62, 0x24, 0xeb, 0xad, 0x67, 0x21, 0xd8, 0x9e, 0x54, 0x12, 0xdd, 0x9b, 0x51, 0x17, 0xd2, 0x94, 0x5e, 0x18, 0xd7, 0x91, 0x5b, 0x1d, 0xcc, 0x8a, 0x40, 0x6, 0xc9, 0x8f, 0x45, 0x3, 0xc6, 0x80, 0x4a, 0xc, 0xc3, 0x85, 0x4f, 0x9},
- {0x0, 0x47, 0x8e, 0xc9, 0x1, 0x46, 0x8f, 0xc8, 0x2, 0x45, 0x8c, 0xcb, 0x3, 0x44, 0x8d, 0xca, 0x4, 0x43, 0x8a, 0xcd, 0x5, 0x42, 0x8b, 0xcc, 0x6, 0x41, 0x88, 0xcf, 0x7, 0x40, 0x89, 0xce, 0x8, 0x4f, 0x86, 0xc1, 0x9, 0x4e, 0x87, 0xc0, 0xa, 0x4d, 0x84, 0xc3, 0xb, 0x4c, 0x85, 0xc2, 0xc, 0x4b, 0x82, 0xc5, 0xd, 0x4a, 0x83, 0xc4, 0xe, 0x49, 0x80, 0xc7, 0xf, 0x48, 0x81, 0xc6, 0x10, 0x57, 0x9e, 0xd9, 0x11, 0x56, 0x9f, 0xd8, 0x12, 0x55, 0x9c, 0xdb, 0x13, 0x54, 0x9d, 0xda, 0x14, 0x53, 0x9a, 0xdd, 0x15, 0x52, 0x9b, 0xdc, 0x16, 0x51, 0x98, 0xdf, 0x17, 0x50, 0x99, 0xde, 0x18, 0x5f, 0x96, 0xd1, 0x19, 0x5e, 0x97, 0xd0, 0x1a, 0x5d, 0x94, 0xd3, 0x1b, 0x5c, 0x95, 0xd2, 0x1c, 0x5b, 0x92, 0xd5, 0x1d, 0x5a, 0x93, 0xd4, 0x1e, 0x59, 0x90, 0xd7, 0x1f, 0x58, 0x91, 0xd6, 0x20, 0x67, 0xae, 0xe9, 0x21, 0x66, 0xaf, 0xe8, 0x22, 0x65, 0xac, 0xeb, 0x23, 0x64, 0xad, 0xea, 0x24, 0x63, 0xaa, 0xed, 0x25, 0x62, 0xab, 0xec, 0x26, 0x61, 0xa8, 0xef, 0x27, 0x60, 0xa9, 0xee, 0x28, 0x6f, 0xa6, 0xe1, 0x29, 0x6e, 0xa7, 0xe0, 0x2a, 0x6d, 0xa4, 0xe3, 0x2b, 0x6c, 0xa5, 0xe2, 0x2c, 0x6b, 0xa2, 0xe5, 0x2d, 0x6a, 0xa3, 0xe4, 0x2e, 0x69, 0xa0, 0xe7, 0x2f, 0x68, 0xa1, 0xe6, 0x30, 0x77, 0xbe, 0xf9, 0x31, 0x76, 0xbf, 0xf8, 0x32, 0x75, 0xbc, 0xfb, 0x33, 0x74, 0xbd, 0xfa, 0x34, 0x73, 0xba, 0xfd, 0x35, 0x72, 0xbb, 0xfc, 0x36, 0x71, 0xb8, 0xff, 0x37, 0x70, 0xb9, 0xfe, 0x38, 0x7f, 0xb6, 0xf1, 0x39, 0x7e, 0xb7, 0xf0, 0x3a, 0x7d, 0xb4, 0xf3, 0x3b, 0x7c, 0xb5, 0xf2, 0x3c, 0x7b, 0xb2, 0xf5, 0x3d, 0x7a, 0xb3, 0xf4, 0x3e, 0x79, 0xb0, 0xf7, 0x3f, 0x78, 0xb1, 0xf6},
- {0x0, 0x48, 0x90, 0xd8, 0x3d, 0x75, 0xad, 0xe5, 0x7a, 0x32, 0xea, 0xa2, 0x47, 0xf, 0xd7, 0x9f, 0xf4, 0xbc, 0x64, 0x2c, 0xc9, 0x81, 0x59, 0x11, 0x8e, 0xc6, 0x1e, 0x56, 0xb3, 0xfb, 0x23, 0x6b, 0xf5, 0xbd, 0x65, 0x2d, 0xc8, 0x80, 0x58, 0x10, 0x8f, 0xc7, 0x1f, 0x57, 0xb2, 0xfa, 0x22, 0x6a, 0x1, 0x49, 0x91, 0xd9, 0x3c, 0x74, 0xac, 0xe4, 0x7b, 0x33, 0xeb, 0xa3, 0x46, 0xe, 0xd6, 0x9e, 0xf7, 0xbf, 0x67, 0x2f, 0xca, 0x82, 0x5a, 0x12, 0x8d, 0xc5, 0x1d, 0x55, 0xb0, 0xf8, 0x20, 0x68, 0x3, 0x4b, 0x93, 0xdb, 0x3e, 0x76, 0xae, 0xe6, 0x79, 0x31, 0xe9, 0xa1, 0x44, 0xc, 0xd4, 0x9c, 0x2, 0x4a, 0x92, 0xda, 0x3f, 0x77, 0xaf, 0xe7, 0x78, 0x30, 0xe8, 0xa0, 0x45, 0xd, 0xd5, 0x9d, 0xf6, 0xbe, 0x66, 0x2e, 0xcb, 0x83, 0x5b, 0x13, 0x8c, 0xc4, 0x1c, 0x54, 0xb1, 0xf9, 0x21, 0x69, 0xf3, 0xbb, 0x63, 0x2b, 0xce, 0x86, 0x5e, 0x16, 0x89, 0xc1, 0x19, 0x51, 0xb4, 0xfc, 0x24, 0x6c, 0x7, 0x4f, 0x97, 0xdf, 0x3a, 0x72, 0xaa, 0xe2, 0x7d, 0x35, 0xed, 0xa5, 0x40, 0x8, 0xd0, 0x98, 0x6, 0x4e, 0x96, 0xde, 0x3b, 0x73, 0xab, 0xe3, 0x7c, 0x34, 0xec, 0xa4, 0x41, 0x9, 0xd1, 0x99, 0xf2, 0xba, 0x62, 0x2a, 0xcf, 0x87, 0x5f, 0x17, 0x88, 0xc0, 0x18, 0x50, 0xb5, 0xfd, 0x25, 0x6d, 0x4, 0x4c, 0x94, 0xdc, 0x39, 0x71, 0xa9, 0xe1, 0x7e, 0x36, 0xee, 0xa6, 0x43, 0xb, 0xd3, 0x9b, 0xf0, 0xb8, 0x60, 0x28, 0xcd, 0x85, 0x5d, 0x15, 0x8a, 0xc2, 0x1a, 0x52, 0xb7, 0xff, 0x27, 0x6f, 0xf1, 0xb9, 0x61, 0x29, 0xcc, 0x84, 0x5c, 0x14, 0x8b, 0xc3, 0x1b, 0x53, 0xb6, 0xfe, 0x26, 0x6e, 0x5, 0x4d, 0x95, 0xdd, 0x38, 0x70, 0xa8, 0xe0, 0x7f, 0x37, 0xef, 0xa7, 0x42, 0xa, 0xd2, 0x9a},
- {0x0, 0x49, 0x92, 0xdb, 0x39, 0x70, 0xab, 0xe2, 0x72, 0x3b, 0xe0, 0xa9, 0x4b, 0x2, 0xd9, 0x90, 0xe4, 0xad, 0x76, 0x3f, 0xdd, 0x94, 0x4f, 0x6, 0x96, 0xdf, 0x4, 0x4d, 0xaf, 0xe6, 0x3d, 0x74, 0xd5, 0x9c, 0x47, 0xe, 0xec, 0xa5, 0x7e, 0x37, 0xa7, 0xee, 0x35, 0x7c, 0x9e, 0xd7, 0xc, 0x45, 0x31, 0x78, 0xa3, 0xea, 0x8, 0x41, 0x9a, 0xd3, 0x43, 0xa, 0xd1, 0x98, 0x7a, 0x33, 0xe8, 0xa1, 0xb7, 0xfe, 0x25, 0x6c, 0x8e, 0xc7, 0x1c, 0x55, 0xc5, 0x8c, 0x57, 0x1e, 0xfc, 0xb5, 0x6e, 0x27, 0x53, 0x1a, 0xc1, 0x88, 0x6a, 0x23, 0xf8, 0xb1, 0x21, 0x68, 0xb3, 0xfa, 0x18, 0x51, 0x8a, 0xc3, 0x62, 0x2b, 0xf0, 0xb9, 0x5b, 0x12, 0xc9, 0x80, 0x10, 0x59, 0x82, 0xcb, 0x29, 0x60, 0xbb, 0xf2, 0x86, 0xcf, 0x14, 0x5d, 0xbf, 0xf6, 0x2d, 0x64, 0xf4, 0xbd, 0x66, 0x2f, 0xcd, 0x84, 0x5f, 0x16, 0x73, 0x3a, 0xe1, 0xa8, 0x4a, 0x3, 0xd8, 0x91, 0x1, 0x48, 0x93, 0xda, 0x38, 0x71, 0xaa, 0xe3, 0x97, 0xde, 0x5, 0x4c, 0xae, 0xe7, 0x3c, 0x75, 0xe5, 0xac, 0x77, 0x3e, 0xdc, 0x95, 0x4e, 0x7, 0xa6, 0xef, 0x34, 0x7d, 0x9f, 0xd6, 0xd, 0x44, 0xd4, 0x9d, 0x46, 0xf, 0xed, 0xa4, 0x7f, 0x36, 0x42, 0xb, 0xd0, 0x99, 0x7b, 0x32, 0xe9, 0xa0, 0x30, 0x79, 0xa2, 0xeb, 0x9, 0x40, 0x9b, 0xd2, 0xc4, 0x8d, 0x56, 0x1f, 0xfd, 0xb4, 0x6f, 0x26, 0xb6, 0xff, 0x24, 0x6d, 0x8f, 0xc6, 0x1d, 0x54, 0x20, 0x69, 0xb2, 0xfb, 0x19, 0x50, 0x8b, 0xc2, 0x52, 0x1b, 0xc0, 0x89, 0x6b, 0x22, 0xf9, 0xb0, 0x11, 0x58, 0x83, 0xca, 0x28, 0x61, 0xba, 0xf3, 0x63, 0x2a, 0xf1, 0xb8, 0x5a, 0x13, 0xc8, 0x81, 0xf5, 0xbc, 0x67, 0x2e, 0xcc, 0x85, 0x5e, 0x17, 0x87, 0xce, 0x15, 0x5c, 0xbe, 0xf7, 0x2c, 0x65},
- {0x0, 0x4a, 0x94, 0xde, 0x35, 0x7f, 0xa1, 0xeb, 0x6a, 0x20, 0xfe, 0xb4, 0x5f, 0x15, 0xcb, 0x81, 0xd4, 0x9e, 0x40, 0xa, 0xe1, 0xab, 0x75, 0x3f, 0xbe, 0xf4, 0x2a, 0x60, 0x8b, 0xc1, 0x1f, 0x55, 0xb5, 0xff, 0x21, 0x6b, 0x80, 0xca, 0x14, 0x5e, 0xdf, 0x95, 0x4b, 0x1, 0xea, 0xa0, 0x7e, 0x34, 0x61, 0x2b, 0xf5, 0xbf, 0x54, 0x1e, 0xc0, 0x8a, 0xb, 0x41, 0x9f, 0xd5, 0x3e, 0x74, 0xaa, 0xe0, 0x77, 0x3d, 0xe3, 0xa9, 0x42, 0x8, 0xd6, 0x9c, 0x1d, 0x57, 0x89, 0xc3, 0x28, 0x62, 0xbc, 0xf6, 0xa3, 0xe9, 0x37, 0x7d, 0x96, 0xdc, 0x2, 0x48, 0xc9, 0x83, 0x5d, 0x17, 0xfc, 0xb6, 0x68, 0x22, 0xc2, 0x88, 0x56, 0x1c, 0xf7, 0xbd, 0x63, 0x29, 0xa8, 0xe2, 0x3c, 0x76, 0x9d, 0xd7, 0x9, 0x43, 0x16, 0x5c, 0x82, 0xc8, 0x23, 0x69, 0xb7, 0xfd, 0x7c, 0x36, 0xe8, 0xa2, 0x49, 0x3, 0xdd, 0x97, 0xee, 0xa4, 0x7a, 0x30, 0xdb, 0x91, 0x4f, 0x5, 0x84, 0xce, 0x10, 0x5a, 0xb1, 0xfb, 0x25, 0x6f, 0x3a, 0x70, 0xae, 0xe4, 0xf, 0x45, 0x9b, 0xd1, 0x50, 0x1a, 0xc4, 0x8e, 0x65, 0x2f, 0xf1, 0xbb, 0x5b, 0x11, 0xcf, 0x85, 0x6e, 0x24, 0xfa, 0xb0, 0x31, 0x7b, 0xa5, 0xef, 0x4, 0x4e, 0x90, 0xda, 0x8f, 0xc5, 0x1b, 0x51, 0xba, 0xf0, 0x2e, 0x64, 0xe5, 0xaf, 0x71, 0x3b, 0xd0, 0x9a, 0x44, 0xe, 0x99, 0xd3, 0xd, 0x47, 0xac, 0xe6, 0x38, 0x72, 0xf3, 0xb9, 0x67, 0x2d, 0xc6, 0x8c, 0x52, 0x18, 0x4d, 0x7, 0xd9, 0x93, 0x78, 0x32, 0xec, 0xa6, 0x27, 0x6d, 0xb3, 0xf9, 0x12, 0x58, 0x86, 0xcc, 0x2c, 0x66, 0xb8, 0xf2, 0x19, 0x53, 0x8d, 0xc7, 0x46, 0xc, 0xd2, 0x98, 0x73, 0x39, 0xe7, 0xad, 0xf8, 0xb2, 0x6c, 0x26, 0xcd, 0x87, 0x59, 0x13, 0x92, 0xd8, 0x6, 0x4c, 0xa7, 0xed, 0x33, 0x79},
- {0x0, 0x4b, 0x96, 0xdd, 0x31, 0x7a, 0xa7, 0xec, 0x62, 0x29, 0xf4, 0xbf, 0x53, 0x18, 0xc5, 0x8e, 0xc4, 0x8f, 0x52, 0x19, 0xf5, 0xbe, 0x63, 0x28, 0xa6, 0xed, 0x30, 0x7b, 0x97, 0xdc, 0x1, 0x4a, 0x95, 0xde, 0x3, 0x48, 0xa4, 0xef, 0x32, 0x79, 0xf7, 0xbc, 0x61, 0x2a, 0xc6, 0x8d, 0x50, 0x1b, 0x51, 0x1a, 0xc7, 0x8c, 0x60, 0x2b, 0xf6, 0xbd, 0x33, 0x78, 0xa5, 0xee, 0x2, 0x49, 0x94, 0xdf, 0x37, 0x7c, 0xa1, 0xea, 0x6, 0x4d, 0x90, 0xdb, 0x55, 0x1e, 0xc3, 0x88, 0x64, 0x2f, 0xf2, 0xb9, 0xf3, 0xb8, 0x65, 0x2e, 0xc2, 0x89, 0x54, 0x1f, 0x91, 0xda, 0x7, 0x4c, 0xa0, 0xeb, 0x36, 0x7d, 0xa2, 0xe9, 0x34, 0x7f, 0x93, 0xd8, 0x5, 0x4e, 0xc0, 0x8b, 0x56, 0x1d, 0xf1, 0xba, 0x67, 0x2c, 0x66, 0x2d, 0xf0, 0xbb, 0x57, 0x1c, 0xc1, 0x8a, 0x4, 0x4f, 0x92, 0xd9, 0x35, 0x7e, 0xa3, 0xe8, 0x6e, 0x25, 0xf8, 0xb3, 0x5f, 0x14, 0xc9, 0x82, 0xc, 0x47, 0x9a, 0xd1, 0x3d, 0x76, 0xab, 0xe0, 0xaa, 0xe1, 0x3c, 0x77, 0x9b, 0xd0, 0xd, 0x46, 0xc8, 0x83, 0x5e, 0x15, 0xf9, 0xb2, 0x6f, 0x24, 0xfb, 0xb0, 0x6d, 0x26, 0xca, 0x81, 0x5c, 0x17, 0x99, 0xd2, 0xf, 0x44, 0xa8, 0xe3, 0x3e, 0x75, 0x3f, 0x74, 0xa9, 0xe2, 0xe, 0x45, 0x98, 0xd3, 0x5d, 0x16, 0xcb, 0x80, 0x6c, 0x27, 0xfa, 0xb1, 0x59, 0x12, 0xcf, 0x84, 0x68, 0x23, 0xfe, 0xb5, 0x3b, 0x70, 0xad, 0xe6, 0xa, 0x41, 0x9c, 0xd7, 0x9d, 0xd6, 0xb, 0x40, 0xac, 0xe7, 0x3a, 0x71, 0xff, 0xb4, 0x69, 0x22, 0xce, 0x85, 0x58, 0x13, 0xcc, 0x87, 0x5a, 0x11, 0xfd, 0xb6, 0x6b, 0x20, 0xae, 0xe5, 0x38, 0x73, 0x9f, 0xd4, 0x9, 0x42, 0x8, 0x43, 0x9e, 0xd5, 0x39, 0x72, 0xaf, 0xe4, 0x6a, 0x21, 0xfc, 0xb7, 0x5b, 0x10, 0xcd, 0x86},
- {0x0, 0x4c, 0x98, 0xd4, 0x2d, 0x61, 0xb5, 0xf9, 0x5a, 0x16, 0xc2, 0x8e, 0x77, 0x3b, 0xef, 0xa3, 0xb4, 0xf8, 0x2c, 0x60, 0x99, 0xd5, 0x1, 0x4d, 0xee, 0xa2, 0x76, 0x3a, 0xc3, 0x8f, 0x5b, 0x17, 0x75, 0x39, 0xed, 0xa1, 0x58, 0x14, 0xc0, 0x8c, 0x2f, 0x63, 0xb7, 0xfb, 0x2, 0x4e, 0x9a, 0xd6, 0xc1, 0x8d, 0x59, 0x15, 0xec, 0xa0, 0x74, 0x38, 0x9b, 0xd7, 0x3, 0x4f, 0xb6, 0xfa, 0x2e, 0x62, 0xea, 0xa6, 0x72, 0x3e, 0xc7, 0x8b, 0x5f, 0x13, 0xb0, 0xfc, 0x28, 0x64, 0x9d, 0xd1, 0x5, 0x49, 0x5e, 0x12, 0xc6, 0x8a, 0x73, 0x3f, 0xeb, 0xa7, 0x4, 0x48, 0x9c, 0xd0, 0x29, 0x65, 0xb1, 0xfd, 0x9f, 0xd3, 0x7, 0x4b, 0xb2, 0xfe, 0x2a, 0x66, 0xc5, 0x89, 0x5d, 0x11, 0xe8, 0xa4, 0x70, 0x3c, 0x2b, 0x67, 0xb3, 0xff, 0x6, 0x4a, 0x9e, 0xd2, 0x71, 0x3d, 0xe9, 0xa5, 0x5c, 0x10, 0xc4, 0x88, 0xc9, 0x85, 0x51, 0x1d, 0xe4, 0xa8, 0x7c, 0x30, 0x93, 0xdf, 0xb, 0x47, 0xbe, 0xf2, 0x26, 0x6a, 0x7d, 0x31, 0xe5, 0xa9, 0x50, 0x1c, 0xc8, 0x84, 0x27, 0x6b, 0xbf, 0xf3, 0xa, 0x46, 0x92, 0xde, 0xbc, 0xf0, 0x24, 0x68, 0x91, 0xdd, 0x9, 0x45, 0xe6, 0xaa, 0x7e, 0x32, 0xcb, 0x87, 0x53, 0x1f, 0x8, 0x44, 0x90, 0xdc, 0x25, 0x69, 0xbd, 0xf1, 0x52, 0x1e, 0xca, 0x86, 0x7f, 0x33, 0xe7, 0xab, 0x23, 0x6f, 0xbb, 0xf7, 0xe, 0x42, 0x96, 0xda, 0x79, 0x35, 0xe1, 0xad, 0x54, 0x18, 0xcc, 0x80, 0x97, 0xdb, 0xf, 0x43, 0xba, 0xf6, 0x22, 0x6e, 0xcd, 0x81, 0x55, 0x19, 0xe0, 0xac, 0x78, 0x34, 0x56, 0x1a, 0xce, 0x82, 0x7b, 0x37, 0xe3, 0xaf, 0xc, 0x40, 0x94, 0xd8, 0x21, 0x6d, 0xb9, 0xf5, 0xe2, 0xae, 0x7a, 0x36, 0xcf, 0x83, 0x57, 0x1b, 0xb8, 0xf4, 0x20, 0x6c, 0x95, 0xd9, 0xd, 0x41},
- {0x0, 0x4d, 0x9a, 0xd7, 0x29, 0x64, 0xb3, 0xfe, 0x52, 0x1f, 0xc8, 0x85, 0x7b, 0x36, 0xe1, 0xac, 0xa4, 0xe9, 0x3e, 0x73, 0x8d, 0xc0, 0x17, 0x5a, 0xf6, 0xbb, 0x6c, 0x21, 0xdf, 0x92, 0x45, 0x8, 0x55, 0x18, 0xcf, 0x82, 0x7c, 0x31, 0xe6, 0xab, 0x7, 0x4a, 0x9d, 0xd0, 0x2e, 0x63, 0xb4, 0xf9, 0xf1, 0xbc, 0x6b, 0x26, 0xd8, 0x95, 0x42, 0xf, 0xa3, 0xee, 0x39, 0x74, 0x8a, 0xc7, 0x10, 0x5d, 0xaa, 0xe7, 0x30, 0x7d, 0x83, 0xce, 0x19, 0x54, 0xf8, 0xb5, 0x62, 0x2f, 0xd1, 0x9c, 0x4b, 0x6, 0xe, 0x43, 0x94, 0xd9, 0x27, 0x6a, 0xbd, 0xf0, 0x5c, 0x11, 0xc6, 0x8b, 0x75, 0x38, 0xef, 0xa2, 0xff, 0xb2, 0x65, 0x28, 0xd6, 0x9b, 0x4c, 0x1, 0xad, 0xe0, 0x37, 0x7a, 0x84, 0xc9, 0x1e, 0x53, 0x5b, 0x16, 0xc1, 0x8c, 0x72, 0x3f, 0xe8, 0xa5, 0x9, 0x44, 0x93, 0xde, 0x20, 0x6d, 0xba, 0xf7, 0x49, 0x4, 0xd3, 0x9e, 0x60, 0x2d, 0xfa, 0xb7, 0x1b, 0x56, 0x81, 0xcc, 0x32, 0x7f, 0xa8, 0xe5, 0xed, 0xa0, 0x77, 0x3a, 0xc4, 0x89, 0x5e, 0x13, 0xbf, 0xf2, 0x25, 0x68, 0x96, 0xdb, 0xc, 0x41, 0x1c, 0x51, 0x86, 0xcb, 0x35, 0x78, 0xaf, 0xe2, 0x4e, 0x3, 0xd4, 0x99, 0x67, 0x2a, 0xfd, 0xb0, 0xb8, 0xf5, 0x22, 0x6f, 0x91, 0xdc, 0xb, 0x46, 0xea, 0xa7, 0x70, 0x3d, 0xc3, 0x8e, 0x59, 0x14, 0xe3, 0xae, 0x79, 0x34, 0xca, 0x87, 0x50, 0x1d, 0xb1, 0xfc, 0x2b, 0x66, 0x98, 0xd5, 0x2, 0x4f, 0x47, 0xa, 0xdd, 0x90, 0x6e, 0x23, 0xf4, 0xb9, 0x15, 0x58, 0x8f, 0xc2, 0x3c, 0x71, 0xa6, 0xeb, 0xb6, 0xfb, 0x2c, 0x61, 0x9f, 0xd2, 0x5, 0x48, 0xe4, 0xa9, 0x7e, 0x33, 0xcd, 0x80, 0x57, 0x1a, 0x12, 0x5f, 0x88, 0xc5, 0x3b, 0x76, 0xa1, 0xec, 0x40, 0xd, 0xda, 0x97, 0x69, 0x24, 0xf3, 0xbe},
- {0x0, 0x4e, 0x9c, 0xd2, 0x25, 0x6b, 0xb9, 0xf7, 0x4a, 0x4, 0xd6, 0x98, 0x6f, 0x21, 0xf3, 0xbd, 0x94, 0xda, 0x8, 0x46, 0xb1, 0xff, 0x2d, 0x63, 0xde, 0x90, 0x42, 0xc, 0xfb, 0xb5, 0x67, 0x29, 0x35, 0x7b, 0xa9, 0xe7, 0x10, 0x5e, 0x8c, 0xc2, 0x7f, 0x31, 0xe3, 0xad, 0x5a, 0x14, 0xc6, 0x88, 0xa1, 0xef, 0x3d, 0x73, 0x84, 0xca, 0x18, 0x56, 0xeb, 0xa5, 0x77, 0x39, 0xce, 0x80, 0x52, 0x1c, 0x6a, 0x24, 0xf6, 0xb8, 0x4f, 0x1, 0xd3, 0x9d, 0x20, 0x6e, 0xbc, 0xf2, 0x5, 0x4b, 0x99, 0xd7, 0xfe, 0xb0, 0x62, 0x2c, 0xdb, 0x95, 0x47, 0x9, 0xb4, 0xfa, 0x28, 0x66, 0x91, 0xdf, 0xd, 0x43, 0x5f, 0x11, 0xc3, 0x8d, 0x7a, 0x34, 0xe6, 0xa8, 0x15, 0x5b, 0x89, 0xc7, 0x30, 0x7e, 0xac, 0xe2, 0xcb, 0x85, 0x57, 0x19, 0xee, 0xa0, 0x72, 0x3c, 0x81, 0xcf, 0x1d, 0x53, 0xa4, 0xea, 0x38, 0x76, 0xd4, 0x9a, 0x48, 0x6, 0xf1, 0xbf, 0x6d, 0x23, 0x9e, 0xd0, 0x2, 0x4c, 0xbb, 0xf5, 0x27, 0x69, 0x40, 0xe, 0xdc, 0x92, 0x65, 0x2b, 0xf9, 0xb7, 0xa, 0x44, 0x96, 0xd8, 0x2f, 0x61, 0xb3, 0xfd, 0xe1, 0xaf, 0x7d, 0x33, 0xc4, 0x8a, 0x58, 0x16, 0xab, 0xe5, 0x37, 0x79, 0x8e, 0xc0, 0x12, 0x5c, 0x75, 0x3b, 0xe9, 0xa7, 0x50, 0x1e, 0xcc, 0x82, 0x3f, 0x71, 0xa3, 0xed, 0x1a, 0x54, 0x86, 0xc8, 0xbe, 0xf0, 0x22, 0x6c, 0x9b, 0xd5, 0x7, 0x49, 0xf4, 0xba, 0x68, 0x26, 0xd1, 0x9f, 0x4d, 0x3, 0x2a, 0x64, 0xb6, 0xf8, 0xf, 0x41, 0x93, 0xdd, 0x60, 0x2e, 0xfc, 0xb2, 0x45, 0xb, 0xd9, 0x97, 0x8b, 0xc5, 0x17, 0x59, 0xae, 0xe0, 0x32, 0x7c, 0xc1, 0x8f, 0x5d, 0x13, 0xe4, 0xaa, 0x78, 0x36, 0x1f, 0x51, 0x83, 0xcd, 0x3a, 0x74, 0xa6, 0xe8, 0x55, 0x1b, 0xc9, 0x87, 0x70, 0x3e, 0xec, 0xa2},
- {0x0, 0x4f, 0x9e, 0xd1, 0x21, 0x6e, 0xbf, 0xf0, 0x42, 0xd, 0xdc, 0x93, 0x63, 0x2c, 0xfd, 0xb2, 0x84, 0xcb, 0x1a, 0x55, 0xa5, 0xea, 0x3b, 0x74, 0xc6, 0x89, 0x58, 0x17, 0xe7, 0xa8, 0x79, 0x36, 0x15, 0x5a, 0x8b, 0xc4, 0x34, 0x7b, 0xaa, 0xe5, 0x57, 0x18, 0xc9, 0x86, 0x76, 0x39, 0xe8, 0xa7, 0x91, 0xde, 0xf, 0x40, 0xb0, 0xff, 0x2e, 0x61, 0xd3, 0x9c, 0x4d, 0x2, 0xf2, 0xbd, 0x6c, 0x23, 0x2a, 0x65, 0xb4, 0xfb, 0xb, 0x44, 0x95, 0xda, 0x68, 0x27, 0xf6, 0xb9, 0x49, 0x6, 0xd7, 0x98, 0xae, 0xe1, 0x30, 0x7f, 0x8f, 0xc0, 0x11, 0x5e, 0xec, 0xa3, 0x72, 0x3d, 0xcd, 0x82, 0x53, 0x1c, 0x3f, 0x70, 0xa1, 0xee, 0x1e, 0x51, 0x80, 0xcf, 0x7d, 0x32, 0xe3, 0xac, 0x5c, 0x13, 0xc2, 0x8d, 0xbb, 0xf4, 0x25, 0x6a, 0x9a, 0xd5, 0x4, 0x4b, 0xf9, 0xb6, 0x67, 0x28, 0xd8, 0x97, 0x46, 0x9, 0x54, 0x1b, 0xca, 0x85, 0x75, 0x3a, 0xeb, 0xa4, 0x16, 0x59, 0x88, 0xc7, 0x37, 0x78, 0xa9, 0xe6, 0xd0, 0x9f, 0x4e, 0x1, 0xf1, 0xbe, 0x6f, 0x20, 0x92, 0xdd, 0xc, 0x43, 0xb3, 0xfc, 0x2d, 0x62, 0x41, 0xe, 0xdf, 0x90, 0x60, 0x2f, 0xfe, 0xb1, 0x3, 0x4c, 0x9d, 0xd2, 0x22, 0x6d, 0xbc, 0xf3, 0xc5, 0x8a, 0x5b, 0x14, 0xe4, 0xab, 0x7a, 0x35, 0x87, 0xc8, 0x19, 0x56, 0xa6, 0xe9, 0x38, 0x77, 0x7e, 0x31, 0xe0, 0xaf, 0x5f, 0x10, 0xc1, 0x8e, 0x3c, 0x73, 0xa2, 0xed, 0x1d, 0x52, 0x83, 0xcc, 0xfa, 0xb5, 0x64, 0x2b, 0xdb, 0x94, 0x45, 0xa, 0xb8, 0xf7, 0x26, 0x69, 0x99, 0xd6, 0x7, 0x48, 0x6b, 0x24, 0xf5, 0xba, 0x4a, 0x5, 0xd4, 0x9b, 0x29, 0x66, 0xb7, 0xf8, 0x8, 0x47, 0x96, 0xd9, 0xef, 0xa0, 0x71, 0x3e, 0xce, 0x81, 0x50, 0x1f, 0xad, 0xe2, 0x33, 0x7c, 0x8c, 0xc3, 0x12, 0x5d},
- {0x0, 0x50, 0xa0, 0xf0, 0x5d, 0xd, 0xfd, 0xad, 0xba, 0xea, 0x1a, 0x4a, 0xe7, 0xb7, 0x47, 0x17, 0x69, 0x39, 0xc9, 0x99, 0x34, 0x64, 0x94, 0xc4, 0xd3, 0x83, 0x73, 0x23, 0x8e, 0xde, 0x2e, 0x7e, 0xd2, 0x82, 0x72, 0x22, 0x8f, 0xdf, 0x2f, 0x7f, 0x68, 0x38, 0xc8, 0x98, 0x35, 0x65, 0x95, 0xc5, 0xbb, 0xeb, 0x1b, 0x4b, 0xe6, 0xb6, 0x46, 0x16, 0x1, 0x51, 0xa1, 0xf1, 0x5c, 0xc, 0xfc, 0xac, 0xb9, 0xe9, 0x19, 0x49, 0xe4, 0xb4, 0x44, 0x14, 0x3, 0x53, 0xa3, 0xf3, 0x5e, 0xe, 0xfe, 0xae, 0xd0, 0x80, 0x70, 0x20, 0x8d, 0xdd, 0x2d, 0x7d, 0x6a, 0x3a, 0xca, 0x9a, 0x37, 0x67, 0x97, 0xc7, 0x6b, 0x3b, 0xcb, 0x9b, 0x36, 0x66, 0x96, 0xc6, 0xd1, 0x81, 0x71, 0x21, 0x8c, 0xdc, 0x2c, 0x7c, 0x2, 0x52, 0xa2, 0xf2, 0x5f, 0xf, 0xff, 0xaf, 0xb8, 0xe8, 0x18, 0x48, 0xe5, 0xb5, 0x45, 0x15, 0x6f, 0x3f, 0xcf, 0x9f, 0x32, 0x62, 0x92, 0xc2, 0xd5, 0x85, 0x75, 0x25, 0x88, 0xd8, 0x28, 0x78, 0x6, 0x56, 0xa6, 0xf6, 0x5b, 0xb, 0xfb, 0xab, 0xbc, 0xec, 0x1c, 0x4c, 0xe1, 0xb1, 0x41, 0x11, 0xbd, 0xed, 0x1d, 0x4d, 0xe0, 0xb0, 0x40, 0x10, 0x7, 0x57, 0xa7, 0xf7, 0x5a, 0xa, 0xfa, 0xaa, 0xd4, 0x84, 0x74, 0x24, 0x89, 0xd9, 0x29, 0x79, 0x6e, 0x3e, 0xce, 0x9e, 0x33, 0x63, 0x93, 0xc3, 0xd6, 0x86, 0x76, 0x26, 0x8b, 0xdb, 0x2b, 0x7b, 0x6c, 0x3c, 0xcc, 0x9c, 0x31, 0x61, 0x91, 0xc1, 0xbf, 0xef, 0x1f, 0x4f, 0xe2, 0xb2, 0x42, 0x12, 0x5, 0x55, 0xa5, 0xf5, 0x58, 0x8, 0xf8, 0xa8, 0x4, 0x54, 0xa4, 0xf4, 0x59, 0x9, 0xf9, 0xa9, 0xbe, 0xee, 0x1e, 0x4e, 0xe3, 0xb3, 0x43, 0x13, 0x6d, 0x3d, 0xcd, 0x9d, 0x30, 0x60, 0x90, 0xc0, 0xd7, 0x87, 0x77, 0x27, 0x8a, 0xda, 0x2a, 0x7a},
- {0x0, 0x51, 0xa2, 0xf3, 0x59, 0x8, 0xfb, 0xaa, 0xb2, 0xe3, 0x10, 0x41, 0xeb, 0xba, 0x49, 0x18, 0x79, 0x28, 0xdb, 0x8a, 0x20, 0x71, 0x82, 0xd3, 0xcb, 0x9a, 0x69, 0x38, 0x92, 0xc3, 0x30, 0x61, 0xf2, 0xa3, 0x50, 0x1, 0xab, 0xfa, 0x9, 0x58, 0x40, 0x11, 0xe2, 0xb3, 0x19, 0x48, 0xbb, 0xea, 0x8b, 0xda, 0x29, 0x78, 0xd2, 0x83, 0x70, 0x21, 0x39, 0x68, 0x9b, 0xca, 0x60, 0x31, 0xc2, 0x93, 0xf9, 0xa8, 0x5b, 0xa, 0xa0, 0xf1, 0x2, 0x53, 0x4b, 0x1a, 0xe9, 0xb8, 0x12, 0x43, 0xb0, 0xe1, 0x80, 0xd1, 0x22, 0x73, 0xd9, 0x88, 0x7b, 0x2a, 0x32, 0x63, 0x90, 0xc1, 0x6b, 0x3a, 0xc9, 0x98, 0xb, 0x5a, 0xa9, 0xf8, 0x52, 0x3, 0xf0, 0xa1, 0xb9, 0xe8, 0x1b, 0x4a, 0xe0, 0xb1, 0x42, 0x13, 0x72, 0x23, 0xd0, 0x81, 0x2b, 0x7a, 0x89, 0xd8, 0xc0, 0x91, 0x62, 0x33, 0x99, 0xc8, 0x3b, 0x6a, 0xef, 0xbe, 0x4d, 0x1c, 0xb6, 0xe7, 0x14, 0x45, 0x5d, 0xc, 0xff, 0xae, 0x4, 0x55, 0xa6, 0xf7, 0x96, 0xc7, 0x34, 0x65, 0xcf, 0x9e, 0x6d, 0x3c, 0x24, 0x75, 0x86, 0xd7, 0x7d, 0x2c, 0xdf, 0x8e, 0x1d, 0x4c, 0xbf, 0xee, 0x44, 0x15, 0xe6, 0xb7, 0xaf, 0xfe, 0xd, 0x5c, 0xf6, 0xa7, 0x54, 0x5, 0x64, 0x35, 0xc6, 0x97, 0x3d, 0x6c, 0x9f, 0xce, 0xd6, 0x87, 0x74, 0x25, 0x8f, 0xde, 0x2d, 0x7c, 0x16, 0x47, 0xb4, 0xe5, 0x4f, 0x1e, 0xed, 0xbc, 0xa4, 0xf5, 0x6, 0x57, 0xfd, 0xac, 0x5f, 0xe, 0x6f, 0x3e, 0xcd, 0x9c, 0x36, 0x67, 0x94, 0xc5, 0xdd, 0x8c, 0x7f, 0x2e, 0x84, 0xd5, 0x26, 0x77, 0xe4, 0xb5, 0x46, 0x17, 0xbd, 0xec, 0x1f, 0x4e, 0x56, 0x7, 0xf4, 0xa5, 0xf, 0x5e, 0xad, 0xfc, 0x9d, 0xcc, 0x3f, 0x6e, 0xc4, 0x95, 0x66, 0x37, 0x2f, 0x7e, 0x8d, 0xdc, 0x76, 0x27, 0xd4, 0x85},
- {0x0, 0x52, 0xa4, 0xf6, 0x55, 0x7, 0xf1, 0xa3, 0xaa, 0xf8, 0xe, 0x5c, 0xff, 0xad, 0x5b, 0x9, 0x49, 0x1b, 0xed, 0xbf, 0x1c, 0x4e, 0xb8, 0xea, 0xe3, 0xb1, 0x47, 0x15, 0xb6, 0xe4, 0x12, 0x40, 0x92, 0xc0, 0x36, 0x64, 0xc7, 0x95, 0x63, 0x31, 0x38, 0x6a, 0x9c, 0xce, 0x6d, 0x3f, 0xc9, 0x9b, 0xdb, 0x89, 0x7f, 0x2d, 0x8e, 0xdc, 0x2a, 0x78, 0x71, 0x23, 0xd5, 0x87, 0x24, 0x76, 0x80, 0xd2, 0x39, 0x6b, 0x9d, 0xcf, 0x6c, 0x3e, 0xc8, 0x9a, 0x93, 0xc1, 0x37, 0x65, 0xc6, 0x94, 0x62, 0x30, 0x70, 0x22, 0xd4, 0x86, 0x25, 0x77, 0x81, 0xd3, 0xda, 0x88, 0x7e, 0x2c, 0x8f, 0xdd, 0x2b, 0x79, 0xab, 0xf9, 0xf, 0x5d, 0xfe, 0xac, 0x5a, 0x8, 0x1, 0x53, 0xa5, 0xf7, 0x54, 0x6, 0xf0, 0xa2, 0xe2, 0xb0, 0x46, 0x14, 0xb7, 0xe5, 0x13, 0x41, 0x48, 0x1a, 0xec, 0xbe, 0x1d, 0x4f, 0xb9, 0xeb, 0x72, 0x20, 0xd6, 0x84, 0x27, 0x75, 0x83, 0xd1, 0xd8, 0x8a, 0x7c, 0x2e, 0x8d, 0xdf, 0x29, 0x7b, 0x3b, 0x69, 0x9f, 0xcd, 0x6e, 0x3c, 0xca, 0x98, 0x91, 0xc3, 0x35, 0x67, 0xc4, 0x96, 0x60, 0x32, 0xe0, 0xb2, 0x44, 0x16, 0xb5, 0xe7, 0x11, 0x43, 0x4a, 0x18, 0xee, 0xbc, 0x1f, 0x4d, 0xbb, 0xe9, 0xa9, 0xfb, 0xd, 0x5f, 0xfc, 0xae, 0x58, 0xa, 0x3, 0x51, 0xa7, 0xf5, 0x56, 0x4, 0xf2, 0xa0, 0x4b, 0x19, 0xef, 0xbd, 0x1e, 0x4c, 0xba, 0xe8, 0xe1, 0xb3, 0x45, 0x17, 0xb4, 0xe6, 0x10, 0x42, 0x2, 0x50, 0xa6, 0xf4, 0x57, 0x5, 0xf3, 0xa1, 0xa8, 0xfa, 0xc, 0x5e, 0xfd, 0xaf, 0x59, 0xb, 0xd9, 0x8b, 0x7d, 0x2f, 0x8c, 0xde, 0x28, 0x7a, 0x73, 0x21, 0xd7, 0x85, 0x26, 0x74, 0x82, 0xd0, 0x90, 0xc2, 0x34, 0x66, 0xc5, 0x97, 0x61, 0x33, 0x3a, 0x68, 0x9e, 0xcc, 0x6f, 0x3d, 0xcb, 0x99},
- {0x0, 0x53, 0xa6, 0xf5, 0x51, 0x2, 0xf7, 0xa4, 0xa2, 0xf1, 0x4, 0x57, 0xf3, 0xa0, 0x55, 0x6, 0x59, 0xa, 0xff, 0xac, 0x8, 0x5b, 0xae, 0xfd, 0xfb, 0xa8, 0x5d, 0xe, 0xaa, 0xf9, 0xc, 0x5f, 0xb2, 0xe1, 0x14, 0x47, 0xe3, 0xb0, 0x45, 0x16, 0x10, 0x43, 0xb6, 0xe5, 0x41, 0x12, 0xe7, 0xb4, 0xeb, 0xb8, 0x4d, 0x1e, 0xba, 0xe9, 0x1c, 0x4f, 0x49, 0x1a, 0xef, 0xbc, 0x18, 0x4b, 0xbe, 0xed, 0x79, 0x2a, 0xdf, 0x8c, 0x28, 0x7b, 0x8e, 0xdd, 0xdb, 0x88, 0x7d, 0x2e, 0x8a, 0xd9, 0x2c, 0x7f, 0x20, 0x73, 0x86, 0xd5, 0x71, 0x22, 0xd7, 0x84, 0x82, 0xd1, 0x24, 0x77, 0xd3, 0x80, 0x75, 0x26, 0xcb, 0x98, 0x6d, 0x3e, 0x9a, 0xc9, 0x3c, 0x6f, 0x69, 0x3a, 0xcf, 0x9c, 0x38, 0x6b, 0x9e, 0xcd, 0x92, 0xc1, 0x34, 0x67, 0xc3, 0x90, 0x65, 0x36, 0x30, 0x63, 0x96, 0xc5, 0x61, 0x32, 0xc7, 0x94, 0xf2, 0xa1, 0x54, 0x7, 0xa3, 0xf0, 0x5, 0x56, 0x50, 0x3, 0xf6, 0xa5, 0x1, 0x52, 0xa7, 0xf4, 0xab, 0xf8, 0xd, 0x5e, 0xfa, 0xa9, 0x5c, 0xf, 0x9, 0x5a, 0xaf, 0xfc, 0x58, 0xb, 0xfe, 0xad, 0x40, 0x13, 0xe6, 0xb5, 0x11, 0x42, 0xb7, 0xe4, 0xe2, 0xb1, 0x44, 0x17, 0xb3, 0xe0, 0x15, 0x46, 0x19, 0x4a, 0xbf, 0xec, 0x48, 0x1b, 0xee, 0xbd, 0xbb, 0xe8, 0x1d, 0x4e, 0xea, 0xb9, 0x4c, 0x1f, 0x8b, 0xd8, 0x2d, 0x7e, 0xda, 0x89, 0x7c, 0x2f, 0x29, 0x7a, 0x8f, 0xdc, 0x78, 0x2b, 0xde, 0x8d, 0xd2, 0x81, 0x74, 0x27, 0x83, 0xd0, 0x25, 0x76, 0x70, 0x23, 0xd6, 0x85, 0x21, 0x72, 0x87, 0xd4, 0x39, 0x6a, 0x9f, 0xcc, 0x68, 0x3b, 0xce, 0x9d, 0x9b, 0xc8, 0x3d, 0x6e, 0xca, 0x99, 0x6c, 0x3f, 0x60, 0x33, 0xc6, 0x95, 0x31, 0x62, 0x97, 0xc4, 0xc2, 0x91, 0x64, 0x37, 0x93, 0xc0, 0x35, 0x66},
- {0x0, 0x54, 0xa8, 0xfc, 0x4d, 0x19, 0xe5, 0xb1, 0x9a, 0xce, 0x32, 0x66, 0xd7, 0x83, 0x7f, 0x2b, 0x29, 0x7d, 0x81, 0xd5, 0x64, 0x30, 0xcc, 0x98, 0xb3, 0xe7, 0x1b, 0x4f, 0xfe, 0xaa, 0x56, 0x2, 0x52, 0x6, 0xfa, 0xae, 0x1f, 0x4b, 0xb7, 0xe3, 0xc8, 0x9c, 0x60, 0x34, 0x85, 0xd1, 0x2d, 0x79, 0x7b, 0x2f, 0xd3, 0x87, 0x36, 0x62, 0x9e, 0xca, 0xe1, 0xb5, 0x49, 0x1d, 0xac, 0xf8, 0x4, 0x50, 0xa4, 0xf0, 0xc, 0x58, 0xe9, 0xbd, 0x41, 0x15, 0x3e, 0x6a, 0x96, 0xc2, 0x73, 0x27, 0xdb, 0x8f, 0x8d, 0xd9, 0x25, 0x71, 0xc0, 0x94, 0x68, 0x3c, 0x17, 0x43, 0xbf, 0xeb, 0x5a, 0xe, 0xf2, 0xa6, 0xf6, 0xa2, 0x5e, 0xa, 0xbb, 0xef, 0x13, 0x47, 0x6c, 0x38, 0xc4, 0x90, 0x21, 0x75, 0x89, 0xdd, 0xdf, 0x8b, 0x77, 0x23, 0x92, 0xc6, 0x3a, 0x6e, 0x45, 0x11, 0xed, 0xb9, 0x8, 0x5c, 0xa0, 0xf4, 0x55, 0x1, 0xfd, 0xa9, 0x18, 0x4c, 0xb0, 0xe4, 0xcf, 0x9b, 0x67, 0x33, 0x82, 0xd6, 0x2a, 0x7e, 0x7c, 0x28, 0xd4, 0x80, 0x31, 0x65, 0x99, 0xcd, 0xe6, 0xb2, 0x4e, 0x1a, 0xab, 0xff, 0x3, 0x57, 0x7, 0x53, 0xaf, 0xfb, 0x4a, 0x1e, 0xe2, 0xb6, 0x9d, 0xc9, 0x35, 0x61, 0xd0, 0x84, 0x78, 0x2c, 0x2e, 0x7a, 0x86, 0xd2, 0x63, 0x37, 0xcb, 0x9f, 0xb4, 0xe0, 0x1c, 0x48, 0xf9, 0xad, 0x51, 0x5, 0xf1, 0xa5, 0x59, 0xd, 0xbc, 0xe8, 0x14, 0x40, 0x6b, 0x3f, 0xc3, 0x97, 0x26, 0x72, 0x8e, 0xda, 0xd8, 0x8c, 0x70, 0x24, 0x95, 0xc1, 0x3d, 0x69, 0x42, 0x16, 0xea, 0xbe, 0xf, 0x5b, 0xa7, 0xf3, 0xa3, 0xf7, 0xb, 0x5f, 0xee, 0xba, 0x46, 0x12, 0x39, 0x6d, 0x91, 0xc5, 0x74, 0x20, 0xdc, 0x88, 0x8a, 0xde, 0x22, 0x76, 0xc7, 0x93, 0x6f, 0x3b, 0x10, 0x44, 0xb8, 0xec, 0x5d, 0x9, 0xf5, 0xa1},
- {0x0, 0x55, 0xaa, 0xff, 0x49, 0x1c, 0xe3, 0xb6, 0x92, 0xc7, 0x38, 0x6d, 0xdb, 0x8e, 0x71, 0x24, 0x39, 0x6c, 0x93, 0xc6, 0x70, 0x25, 0xda, 0x8f, 0xab, 0xfe, 0x1, 0x54, 0xe2, 0xb7, 0x48, 0x1d, 0x72, 0x27, 0xd8, 0x8d, 0x3b, 0x6e, 0x91, 0xc4, 0xe0, 0xb5, 0x4a, 0x1f, 0xa9, 0xfc, 0x3, 0x56, 0x4b, 0x1e, 0xe1, 0xb4, 0x2, 0x57, 0xa8, 0xfd, 0xd9, 0x8c, 0x73, 0x26, 0x90, 0xc5, 0x3a, 0x6f, 0xe4, 0xb1, 0x4e, 0x1b, 0xad, 0xf8, 0x7, 0x52, 0x76, 0x23, 0xdc, 0x89, 0x3f, 0x6a, 0x95, 0xc0, 0xdd, 0x88, 0x77, 0x22, 0x94, 0xc1, 0x3e, 0x6b, 0x4f, 0x1a, 0xe5, 0xb0, 0x6, 0x53, 0xac, 0xf9, 0x96, 0xc3, 0x3c, 0x69, 0xdf, 0x8a, 0x75, 0x20, 0x4, 0x51, 0xae, 0xfb, 0x4d, 0x18, 0xe7, 0xb2, 0xaf, 0xfa, 0x5, 0x50, 0xe6, 0xb3, 0x4c, 0x19, 0x3d, 0x68, 0x97, 0xc2, 0x74, 0x21, 0xde, 0x8b, 0xd5, 0x80, 0x7f, 0x2a, 0x9c, 0xc9, 0x36, 0x63, 0x47, 0x12, 0xed, 0xb8, 0xe, 0x5b, 0xa4, 0xf1, 0xec, 0xb9, 0x46, 0x13, 0xa5, 0xf0, 0xf, 0x5a, 0x7e, 0x2b, 0xd4, 0x81, 0x37, 0x62, 0x9d, 0xc8, 0xa7, 0xf2, 0xd, 0x58, 0xee, 0xbb, 0x44, 0x11, 0x35, 0x60, 0x9f, 0xca, 0x7c, 0x29, 0xd6, 0x83, 0x9e, 0xcb, 0x34, 0x61, 0xd7, 0x82, 0x7d, 0x28, 0xc, 0x59, 0xa6, 0xf3, 0x45, 0x10, 0xef, 0xba, 0x31, 0x64, 0x9b, 0xce, 0x78, 0x2d, 0xd2, 0x87, 0xa3, 0xf6, 0x9, 0x5c, 0xea, 0xbf, 0x40, 0x15, 0x8, 0x5d, 0xa2, 0xf7, 0x41, 0x14, 0xeb, 0xbe, 0x9a, 0xcf, 0x30, 0x65, 0xd3, 0x86, 0x79, 0x2c, 0x43, 0x16, 0xe9, 0xbc, 0xa, 0x5f, 0xa0, 0xf5, 0xd1, 0x84, 0x7b, 0x2e, 0x98, 0xcd, 0x32, 0x67, 0x7a, 0x2f, 0xd0, 0x85, 0x33, 0x66, 0x99, 0xcc, 0xe8, 0xbd, 0x42, 0x17, 0xa1, 0xf4, 0xb, 0x5e},
- {0x0, 0x56, 0xac, 0xfa, 0x45, 0x13, 0xe9, 0xbf, 0x8a, 0xdc, 0x26, 0x70, 0xcf, 0x99, 0x63, 0x35, 0x9, 0x5f, 0xa5, 0xf3, 0x4c, 0x1a, 0xe0, 0xb6, 0x83, 0xd5, 0x2f, 0x79, 0xc6, 0x90, 0x6a, 0x3c, 0x12, 0x44, 0xbe, 0xe8, 0x57, 0x1, 0xfb, 0xad, 0x98, 0xce, 0x34, 0x62, 0xdd, 0x8b, 0x71, 0x27, 0x1b, 0x4d, 0xb7, 0xe1, 0x5e, 0x8, 0xf2, 0xa4, 0x91, 0xc7, 0x3d, 0x6b, 0xd4, 0x82, 0x78, 0x2e, 0x24, 0x72, 0x88, 0xde, 0x61, 0x37, 0xcd, 0x9b, 0xae, 0xf8, 0x2, 0x54, 0xeb, 0xbd, 0x47, 0x11, 0x2d, 0x7b, 0x81, 0xd7, 0x68, 0x3e, 0xc4, 0x92, 0xa7, 0xf1, 0xb, 0x5d, 0xe2, 0xb4, 0x4e, 0x18, 0x36, 0x60, 0x9a, 0xcc, 0x73, 0x25, 0xdf, 0x89, 0xbc, 0xea, 0x10, 0x46, 0xf9, 0xaf, 0x55, 0x3, 0x3f, 0x69, 0x93, 0xc5, 0x7a, 0x2c, 0xd6, 0x80, 0xb5, 0xe3, 0x19, 0x4f, 0xf0, 0xa6, 0x5c, 0xa, 0x48, 0x1e, 0xe4, 0xb2, 0xd, 0x5b, 0xa1, 0xf7, 0xc2, 0x94, 0x6e, 0x38, 0x87, 0xd1, 0x2b, 0x7d, 0x41, 0x17, 0xed, 0xbb, 0x4, 0x52, 0xa8, 0xfe, 0xcb, 0x9d, 0x67, 0x31, 0x8e, 0xd8, 0x22, 0x74, 0x5a, 0xc, 0xf6, 0xa0, 0x1f, 0x49, 0xb3, 0xe5, 0xd0, 0x86, 0x7c, 0x2a, 0x95, 0xc3, 0x39, 0x6f, 0x53, 0x5, 0xff, 0xa9, 0x16, 0x40, 0xba, 0xec, 0xd9, 0x8f, 0x75, 0x23, 0x9c, 0xca, 0x30, 0x66, 0x6c, 0x3a, 0xc0, 0x96, 0x29, 0x7f, 0x85, 0xd3, 0xe6, 0xb0, 0x4a, 0x1c, 0xa3, 0xf5, 0xf, 0x59, 0x65, 0x33, 0xc9, 0x9f, 0x20, 0x76, 0x8c, 0xda, 0xef, 0xb9, 0x43, 0x15, 0xaa, 0xfc, 0x6, 0x50, 0x7e, 0x28, 0xd2, 0x84, 0x3b, 0x6d, 0x97, 0xc1, 0xf4, 0xa2, 0x58, 0xe, 0xb1, 0xe7, 0x1d, 0x4b, 0x77, 0x21, 0xdb, 0x8d, 0x32, 0x64, 0x9e, 0xc8, 0xfd, 0xab, 0x51, 0x7, 0xb8, 0xee, 0x14, 0x42},
- {0x0, 0x57, 0xae, 0xf9, 0x41, 0x16, 0xef, 0xb8, 0x82, 0xd5, 0x2c, 0x7b, 0xc3, 0x94, 0x6d, 0x3a, 0x19, 0x4e, 0xb7, 0xe0, 0x58, 0xf, 0xf6, 0xa1, 0x9b, 0xcc, 0x35, 0x62, 0xda, 0x8d, 0x74, 0x23, 0x32, 0x65, 0x9c, 0xcb, 0x73, 0x24, 0xdd, 0x8a, 0xb0, 0xe7, 0x1e, 0x49, 0xf1, 0xa6, 0x5f, 0x8, 0x2b, 0x7c, 0x85, 0xd2, 0x6a, 0x3d, 0xc4, 0x93, 0xa9, 0xfe, 0x7, 0x50, 0xe8, 0xbf, 0x46, 0x11, 0x64, 0x33, 0xca, 0x9d, 0x25, 0x72, 0x8b, 0xdc, 0xe6, 0xb1, 0x48, 0x1f, 0xa7, 0xf0, 0x9, 0x5e, 0x7d, 0x2a, 0xd3, 0x84, 0x3c, 0x6b, 0x92, 0xc5, 0xff, 0xa8, 0x51, 0x6, 0xbe, 0xe9, 0x10, 0x47, 0x56, 0x1, 0xf8, 0xaf, 0x17, 0x40, 0xb9, 0xee, 0xd4, 0x83, 0x7a, 0x2d, 0x95, 0xc2, 0x3b, 0x6c, 0x4f, 0x18, 0xe1, 0xb6, 0xe, 0x59, 0xa0, 0xf7, 0xcd, 0x9a, 0x63, 0x34, 0x8c, 0xdb, 0x22, 0x75, 0xc8, 0x9f, 0x66, 0x31, 0x89, 0xde, 0x27, 0x70, 0x4a, 0x1d, 0xe4, 0xb3, 0xb, 0x5c, 0xa5, 0xf2, 0xd1, 0x86, 0x7f, 0x28, 0x90, 0xc7, 0x3e, 0x69, 0x53, 0x4, 0xfd, 0xaa, 0x12, 0x45, 0xbc, 0xeb, 0xfa, 0xad, 0x54, 0x3, 0xbb, 0xec, 0x15, 0x42, 0x78, 0x2f, 0xd6, 0x81, 0x39, 0x6e, 0x97, 0xc0, 0xe3, 0xb4, 0x4d, 0x1a, 0xa2, 0xf5, 0xc, 0x5b, 0x61, 0x36, 0xcf, 0x98, 0x20, 0x77, 0x8e, 0xd9, 0xac, 0xfb, 0x2, 0x55, 0xed, 0xba, 0x43, 0x14, 0x2e, 0x79, 0x80, 0xd7, 0x6f, 0x38, 0xc1, 0x96, 0xb5, 0xe2, 0x1b, 0x4c, 0xf4, 0xa3, 0x5a, 0xd, 0x37, 0x60, 0x99, 0xce, 0x76, 0x21, 0xd8, 0x8f, 0x9e, 0xc9, 0x30, 0x67, 0xdf, 0x88, 0x71, 0x26, 0x1c, 0x4b, 0xb2, 0xe5, 0x5d, 0xa, 0xf3, 0xa4, 0x87, 0xd0, 0x29, 0x7e, 0xc6, 0x91, 0x68, 0x3f, 0x5, 0x52, 0xab, 0xfc, 0x44, 0x13, 0xea, 0xbd},
- {0x0, 0x58, 0xb0, 0xe8, 0x7d, 0x25, 0xcd, 0x95, 0xfa, 0xa2, 0x4a, 0x12, 0x87, 0xdf, 0x37, 0x6f, 0xe9, 0xb1, 0x59, 0x1, 0x94, 0xcc, 0x24, 0x7c, 0x13, 0x4b, 0xa3, 0xfb, 0x6e, 0x36, 0xde, 0x86, 0xcf, 0x97, 0x7f, 0x27, 0xb2, 0xea, 0x2, 0x5a, 0x35, 0x6d, 0x85, 0xdd, 0x48, 0x10, 0xf8, 0xa0, 0x26, 0x7e, 0x96, 0xce, 0x5b, 0x3, 0xeb, 0xb3, 0xdc, 0x84, 0x6c, 0x34, 0xa1, 0xf9, 0x11, 0x49, 0x83, 0xdb, 0x33, 0x6b, 0xfe, 0xa6, 0x4e, 0x16, 0x79, 0x21, 0xc9, 0x91, 0x4, 0x5c, 0xb4, 0xec, 0x6a, 0x32, 0xda, 0x82, 0x17, 0x4f, 0xa7, 0xff, 0x90, 0xc8, 0x20, 0x78, 0xed, 0xb5, 0x5d, 0x5, 0x4c, 0x14, 0xfc, 0xa4, 0x31, 0x69, 0x81, 0xd9, 0xb6, 0xee, 0x6, 0x5e, 0xcb, 0x93, 0x7b, 0x23, 0xa5, 0xfd, 0x15, 0x4d, 0xd8, 0x80, 0x68, 0x30, 0x5f, 0x7, 0xef, 0xb7, 0x22, 0x7a, 0x92, 0xca, 0x1b, 0x43, 0xab, 0xf3, 0x66, 0x3e, 0xd6, 0x8e, 0xe1, 0xb9, 0x51, 0x9, 0x9c, 0xc4, 0x2c, 0x74, 0xf2, 0xaa, 0x42, 0x1a, 0x8f, 0xd7, 0x3f, 0x67, 0x8, 0x50, 0xb8, 0xe0, 0x75, 0x2d, 0xc5, 0x9d, 0xd4, 0x8c, 0x64, 0x3c, 0xa9, 0xf1, 0x19, 0x41, 0x2e, 0x76, 0x9e, 0xc6, 0x53, 0xb, 0xe3, 0xbb, 0x3d, 0x65, 0x8d, 0xd5, 0x40, 0x18, 0xf0, 0xa8, 0xc7, 0x9f, 0x77, 0x2f, 0xba, 0xe2, 0xa, 0x52, 0x98, 0xc0, 0x28, 0x70, 0xe5, 0xbd, 0x55, 0xd, 0x62, 0x3a, 0xd2, 0x8a, 0x1f, 0x47, 0xaf, 0xf7, 0x71, 0x29, 0xc1, 0x99, 0xc, 0x54, 0xbc, 0xe4, 0x8b, 0xd3, 0x3b, 0x63, 0xf6, 0xae, 0x46, 0x1e, 0x57, 0xf, 0xe7, 0xbf, 0x2a, 0x72, 0x9a, 0xc2, 0xad, 0xf5, 0x1d, 0x45, 0xd0, 0x88, 0x60, 0x38, 0xbe, 0xe6, 0xe, 0x56, 0xc3, 0x9b, 0x73, 0x2b, 0x44, 0x1c, 0xf4, 0xac, 0x39, 0x61, 0x89, 0xd1},
- {0x0, 0x59, 0xb2, 0xeb, 0x79, 0x20, 0xcb, 0x92, 0xf2, 0xab, 0x40, 0x19, 0x8b, 0xd2, 0x39, 0x60, 0xf9, 0xa0, 0x4b, 0x12, 0x80, 0xd9, 0x32, 0x6b, 0xb, 0x52, 0xb9, 0xe0, 0x72, 0x2b, 0xc0, 0x99, 0xef, 0xb6, 0x5d, 0x4, 0x96, 0xcf, 0x24, 0x7d, 0x1d, 0x44, 0xaf, 0xf6, 0x64, 0x3d, 0xd6, 0x8f, 0x16, 0x4f, 0xa4, 0xfd, 0x6f, 0x36, 0xdd, 0x84, 0xe4, 0xbd, 0x56, 0xf, 0x9d, 0xc4, 0x2f, 0x76, 0xc3, 0x9a, 0x71, 0x28, 0xba, 0xe3, 0x8, 0x51, 0x31, 0x68, 0x83, 0xda, 0x48, 0x11, 0xfa, 0xa3, 0x3a, 0x63, 0x88, 0xd1, 0x43, 0x1a, 0xf1, 0xa8, 0xc8, 0x91, 0x7a, 0x23, 0xb1, 0xe8, 0x3, 0x5a, 0x2c, 0x75, 0x9e, 0xc7, 0x55, 0xc, 0xe7, 0xbe, 0xde, 0x87, 0x6c, 0x35, 0xa7, 0xfe, 0x15, 0x4c, 0xd5, 0x8c, 0x67, 0x3e, 0xac, 0xf5, 0x1e, 0x47, 0x27, 0x7e, 0x95, 0xcc, 0x5e, 0x7, 0xec, 0xb5, 0x9b, 0xc2, 0x29, 0x70, 0xe2, 0xbb, 0x50, 0x9, 0x69, 0x30, 0xdb, 0x82, 0x10, 0x49, 0xa2, 0xfb, 0x62, 0x3b, 0xd0, 0x89, 0x1b, 0x42, 0xa9, 0xf0, 0x90, 0xc9, 0x22, 0x7b, 0xe9, 0xb0, 0x5b, 0x2, 0x74, 0x2d, 0xc6, 0x9f, 0xd, 0x54, 0xbf, 0xe6, 0x86, 0xdf, 0x34, 0x6d, 0xff, 0xa6, 0x4d, 0x14, 0x8d, 0xd4, 0x3f, 0x66, 0xf4, 0xad, 0x46, 0x1f, 0x7f, 0x26, 0xcd, 0x94, 0x6, 0x5f, 0xb4, 0xed, 0x58, 0x1, 0xea, 0xb3, 0x21, 0x78, 0x93, 0xca, 0xaa, 0xf3, 0x18, 0x41, 0xd3, 0x8a, 0x61, 0x38, 0xa1, 0xf8, 0x13, 0x4a, 0xd8, 0x81, 0x6a, 0x33, 0x53, 0xa, 0xe1, 0xb8, 0x2a, 0x73, 0x98, 0xc1, 0xb7, 0xee, 0x5, 0x5c, 0xce, 0x97, 0x7c, 0x25, 0x45, 0x1c, 0xf7, 0xae, 0x3c, 0x65, 0x8e, 0xd7, 0x4e, 0x17, 0xfc, 0xa5, 0x37, 0x6e, 0x85, 0xdc, 0xbc, 0xe5, 0xe, 0x57, 0xc5, 0x9c, 0x77, 0x2e},
- {0x0, 0x5a, 0xb4, 0xee, 0x75, 0x2f, 0xc1, 0x9b, 0xea, 0xb0, 0x5e, 0x4, 0x9f, 0xc5, 0x2b, 0x71, 0xc9, 0x93, 0x7d, 0x27, 0xbc, 0xe6, 0x8, 0x52, 0x23, 0x79, 0x97, 0xcd, 0x56, 0xc, 0xe2, 0xb8, 0x8f, 0xd5, 0x3b, 0x61, 0xfa, 0xa0, 0x4e, 0x14, 0x65, 0x3f, 0xd1, 0x8b, 0x10, 0x4a, 0xa4, 0xfe, 0x46, 0x1c, 0xf2, 0xa8, 0x33, 0x69, 0x87, 0xdd, 0xac, 0xf6, 0x18, 0x42, 0xd9, 0x83, 0x6d, 0x37, 0x3, 0x59, 0xb7, 0xed, 0x76, 0x2c, 0xc2, 0x98, 0xe9, 0xb3, 0x5d, 0x7, 0x9c, 0xc6, 0x28, 0x72, 0xca, 0x90, 0x7e, 0x24, 0xbf, 0xe5, 0xb, 0x51, 0x20, 0x7a, 0x94, 0xce, 0x55, 0xf, 0xe1, 0xbb, 0x8c, 0xd6, 0x38, 0x62, 0xf9, 0xa3, 0x4d, 0x17, 0x66, 0x3c, 0xd2, 0x88, 0x13, 0x49, 0xa7, 0xfd, 0x45, 0x1f, 0xf1, 0xab, 0x30, 0x6a, 0x84, 0xde, 0xaf, 0xf5, 0x1b, 0x41, 0xda, 0x80, 0x6e, 0x34, 0x6, 0x5c, 0xb2, 0xe8, 0x73, 0x29, 0xc7, 0x9d, 0xec, 0xb6, 0x58, 0x2, 0x99, 0xc3, 0x2d, 0x77, 0xcf, 0x95, 0x7b, 0x21, 0xba, 0xe0, 0xe, 0x54, 0x25, 0x7f, 0x91, 0xcb, 0x50, 0xa, 0xe4, 0xbe, 0x89, 0xd3, 0x3d, 0x67, 0xfc, 0xa6, 0x48, 0x12, 0x63, 0x39, 0xd7, 0x8d, 0x16, 0x4c, 0xa2, 0xf8, 0x40, 0x1a, 0xf4, 0xae, 0x35, 0x6f, 0x81, 0xdb, 0xaa, 0xf0, 0x1e, 0x44, 0xdf, 0x85, 0x6b, 0x31, 0x5, 0x5f, 0xb1, 0xeb, 0x70, 0x2a, 0xc4, 0x9e, 0xef, 0xb5, 0x5b, 0x1, 0x9a, 0xc0, 0x2e, 0x74, 0xcc, 0x96, 0x78, 0x22, 0xb9, 0xe3, 0xd, 0x57, 0x26, 0x7c, 0x92, 0xc8, 0x53, 0x9, 0xe7, 0xbd, 0x8a, 0xd0, 0x3e, 0x64, 0xff, 0xa5, 0x4b, 0x11, 0x60, 0x3a, 0xd4, 0x8e, 0x15, 0x4f, 0xa1, 0xfb, 0x43, 0x19, 0xf7, 0xad, 0x36, 0x6c, 0x82, 0xd8, 0xa9, 0xf3, 0x1d, 0x47, 0xdc, 0x86, 0x68, 0x32},
- {0x0, 0x5b, 0xb6, 0xed, 0x71, 0x2a, 0xc7, 0x9c, 0xe2, 0xb9, 0x54, 0xf, 0x93, 0xc8, 0x25, 0x7e, 0xd9, 0x82, 0x6f, 0x34, 0xa8, 0xf3, 0x1e, 0x45, 0x3b, 0x60, 0x8d, 0xd6, 0x4a, 0x11, 0xfc, 0xa7, 0xaf, 0xf4, 0x19, 0x42, 0xde, 0x85, 0x68, 0x33, 0x4d, 0x16, 0xfb, 0xa0, 0x3c, 0x67, 0x8a, 0xd1, 0x76, 0x2d, 0xc0, 0x9b, 0x7, 0x5c, 0xb1, 0xea, 0x94, 0xcf, 0x22, 0x79, 0xe5, 0xbe, 0x53, 0x8, 0x43, 0x18, 0xf5, 0xae, 0x32, 0x69, 0x84, 0xdf, 0xa1, 0xfa, 0x17, 0x4c, 0xd0, 0x8b, 0x66, 0x3d, 0x9a, 0xc1, 0x2c, 0x77, 0xeb, 0xb0, 0x5d, 0x6, 0x78, 0x23, 0xce, 0x95, 0x9, 0x52, 0xbf, 0xe4, 0xec, 0xb7, 0x5a, 0x1, 0x9d, 0xc6, 0x2b, 0x70, 0xe, 0x55, 0xb8, 0xe3, 0x7f, 0x24, 0xc9, 0x92, 0x35, 0x6e, 0x83, 0xd8, 0x44, 0x1f, 0xf2, 0xa9, 0xd7, 0x8c, 0x61, 0x3a, 0xa6, 0xfd, 0x10, 0x4b, 0x86, 0xdd, 0x30, 0x6b, 0xf7, 0xac, 0x41, 0x1a, 0x64, 0x3f, 0xd2, 0x89, 0x15, 0x4e, 0xa3, 0xf8, 0x5f, 0x4, 0xe9, 0xb2, 0x2e, 0x75, 0x98, 0xc3, 0xbd, 0xe6, 0xb, 0x50, 0xcc, 0x97, 0x7a, 0x21, 0x29, 0x72, 0x9f, 0xc4, 0x58, 0x3, 0xee, 0xb5, 0xcb, 0x90, 0x7d, 0x26, 0xba, 0xe1, 0xc, 0x57, 0xf0, 0xab, 0x46, 0x1d, 0x81, 0xda, 0x37, 0x6c, 0x12, 0x49, 0xa4, 0xff, 0x63, 0x38, 0xd5, 0x8e, 0xc5, 0x9e, 0x73, 0x28, 0xb4, 0xef, 0x2, 0x59, 0x27, 0x7c, 0x91, 0xca, 0x56, 0xd, 0xe0, 0xbb, 0x1c, 0x47, 0xaa, 0xf1, 0x6d, 0x36, 0xdb, 0x80, 0xfe, 0xa5, 0x48, 0x13, 0x8f, 0xd4, 0x39, 0x62, 0x6a, 0x31, 0xdc, 0x87, 0x1b, 0x40, 0xad, 0xf6, 0x88, 0xd3, 0x3e, 0x65, 0xf9, 0xa2, 0x4f, 0x14, 0xb3, 0xe8, 0x5, 0x5e, 0xc2, 0x99, 0x74, 0x2f, 0x51, 0xa, 0xe7, 0xbc, 0x20, 0x7b, 0x96, 0xcd},
- {0x0, 0x5c, 0xb8, 0xe4, 0x6d, 0x31, 0xd5, 0x89, 0xda, 0x86, 0x62, 0x3e, 0xb7, 0xeb, 0xf, 0x53, 0xa9, 0xf5, 0x11, 0x4d, 0xc4, 0x98, 0x7c, 0x20, 0x73, 0x2f, 0xcb, 0x97, 0x1e, 0x42, 0xa6, 0xfa, 0x4f, 0x13, 0xf7, 0xab, 0x22, 0x7e, 0x9a, 0xc6, 0x95, 0xc9, 0x2d, 0x71, 0xf8, 0xa4, 0x40, 0x1c, 0xe6, 0xba, 0x5e, 0x2, 0x8b, 0xd7, 0x33, 0x6f, 0x3c, 0x60, 0x84, 0xd8, 0x51, 0xd, 0xe9, 0xb5, 0x9e, 0xc2, 0x26, 0x7a, 0xf3, 0xaf, 0x4b, 0x17, 0x44, 0x18, 0xfc, 0xa0, 0x29, 0x75, 0x91, 0xcd, 0x37, 0x6b, 0x8f, 0xd3, 0x5a, 0x6, 0xe2, 0xbe, 0xed, 0xb1, 0x55, 0x9, 0x80, 0xdc, 0x38, 0x64, 0xd1, 0x8d, 0x69, 0x35, 0xbc, 0xe0, 0x4, 0x58, 0xb, 0x57, 0xb3, 0xef, 0x66, 0x3a, 0xde, 0x82, 0x78, 0x24, 0xc0, 0x9c, 0x15, 0x49, 0xad, 0xf1, 0xa2, 0xfe, 0x1a, 0x46, 0xcf, 0x93, 0x77, 0x2b, 0x21, 0x7d, 0x99, 0xc5, 0x4c, 0x10, 0xf4, 0xa8, 0xfb, 0xa7, 0x43, 0x1f, 0x96, 0xca, 0x2e, 0x72, 0x88, 0xd4, 0x30, 0x6c, 0xe5, 0xb9, 0x5d, 0x1, 0x52, 0xe, 0xea, 0xb6, 0x3f, 0x63, 0x87, 0xdb, 0x6e, 0x32, 0xd6, 0x8a, 0x3, 0x5f, 0xbb, 0xe7, 0xb4, 0xe8, 0xc, 0x50, 0xd9, 0x85, 0x61, 0x3d, 0xc7, 0x9b, 0x7f, 0x23, 0xaa, 0xf6, 0x12, 0x4e, 0x1d, 0x41, 0xa5, 0xf9, 0x70, 0x2c, 0xc8, 0x94, 0xbf, 0xe3, 0x7, 0x5b, 0xd2, 0x8e, 0x6a, 0x36, 0x65, 0x39, 0xdd, 0x81, 0x8, 0x54, 0xb0, 0xec, 0x16, 0x4a, 0xae, 0xf2, 0x7b, 0x27, 0xc3, 0x9f, 0xcc, 0x90, 0x74, 0x28, 0xa1, 0xfd, 0x19, 0x45, 0xf0, 0xac, 0x48, 0x14, 0x9d, 0xc1, 0x25, 0x79, 0x2a, 0x76, 0x92, 0xce, 0x47, 0x1b, 0xff, 0xa3, 0x59, 0x5, 0xe1, 0xbd, 0x34, 0x68, 0x8c, 0xd0, 0x83, 0xdf, 0x3b, 0x67, 0xee, 0xb2, 0x56, 0xa},
- {0x0, 0x5d, 0xba, 0xe7, 0x69, 0x34, 0xd3, 0x8e, 0xd2, 0x8f, 0x68, 0x35, 0xbb, 0xe6, 0x1, 0x5c, 0xb9, 0xe4, 0x3, 0x5e, 0xd0, 0x8d, 0x6a, 0x37, 0x6b, 0x36, 0xd1, 0x8c, 0x2, 0x5f, 0xb8, 0xe5, 0x6f, 0x32, 0xd5, 0x88, 0x6, 0x5b, 0xbc, 0xe1, 0xbd, 0xe0, 0x7, 0x5a, 0xd4, 0x89, 0x6e, 0x33, 0xd6, 0x8b, 0x6c, 0x31, 0xbf, 0xe2, 0x5, 0x58, 0x4, 0x59, 0xbe, 0xe3, 0x6d, 0x30, 0xd7, 0x8a, 0xde, 0x83, 0x64, 0x39, 0xb7, 0xea, 0xd, 0x50, 0xc, 0x51, 0xb6, 0xeb, 0x65, 0x38, 0xdf, 0x82, 0x67, 0x3a, 0xdd, 0x80, 0xe, 0x53, 0xb4, 0xe9, 0xb5, 0xe8, 0xf, 0x52, 0xdc, 0x81, 0x66, 0x3b, 0xb1, 0xec, 0xb, 0x56, 0xd8, 0x85, 0x62, 0x3f, 0x63, 0x3e, 0xd9, 0x84, 0xa, 0x57, 0xb0, 0xed, 0x8, 0x55, 0xb2, 0xef, 0x61, 0x3c, 0xdb, 0x86, 0xda, 0x87, 0x60, 0x3d, 0xb3, 0xee, 0x9, 0x54, 0xa1, 0xfc, 0x1b, 0x46, 0xc8, 0x95, 0x72, 0x2f, 0x73, 0x2e, 0xc9, 0x94, 0x1a, 0x47, 0xa0, 0xfd, 0x18, 0x45, 0xa2, 0xff, 0x71, 0x2c, 0xcb, 0x96, 0xca, 0x97, 0x70, 0x2d, 0xa3, 0xfe, 0x19, 0x44, 0xce, 0x93, 0x74, 0x29, 0xa7, 0xfa, 0x1d, 0x40, 0x1c, 0x41, 0xa6, 0xfb, 0x75, 0x28, 0xcf, 0x92, 0x77, 0x2a, 0xcd, 0x90, 0x1e, 0x43, 0xa4, 0xf9, 0xa5, 0xf8, 0x1f, 0x42, 0xcc, 0x91, 0x76, 0x2b, 0x7f, 0x22, 0xc5, 0x98, 0x16, 0x4b, 0xac, 0xf1, 0xad, 0xf0, 0x17, 0x4a, 0xc4, 0x99, 0x7e, 0x23, 0xc6, 0x9b, 0x7c, 0x21, 0xaf, 0xf2, 0x15, 0x48, 0x14, 0x49, 0xae, 0xf3, 0x7d, 0x20, 0xc7, 0x9a, 0x10, 0x4d, 0xaa, 0xf7, 0x79, 0x24, 0xc3, 0x9e, 0xc2, 0x9f, 0x78, 0x25, 0xab, 0xf6, 0x11, 0x4c, 0xa9, 0xf4, 0x13, 0x4e, 0xc0, 0x9d, 0x7a, 0x27, 0x7b, 0x26, 0xc1, 0x9c, 0x12, 0x4f, 0xa8, 0xf5},
- {0x0, 0x5e, 0xbc, 0xe2, 0x65, 0x3b, 0xd9, 0x87, 0xca, 0x94, 0x76, 0x28, 0xaf, 0xf1, 0x13, 0x4d, 0x89, 0xd7, 0x35, 0x6b, 0xec, 0xb2, 0x50, 0xe, 0x43, 0x1d, 0xff, 0xa1, 0x26, 0x78, 0x9a, 0xc4, 0xf, 0x51, 0xb3, 0xed, 0x6a, 0x34, 0xd6, 0x88, 0xc5, 0x9b, 0x79, 0x27, 0xa0, 0xfe, 0x1c, 0x42, 0x86, 0xd8, 0x3a, 0x64, 0xe3, 0xbd, 0x5f, 0x1, 0x4c, 0x12, 0xf0, 0xae, 0x29, 0x77, 0x95, 0xcb, 0x1e, 0x40, 0xa2, 0xfc, 0x7b, 0x25, 0xc7, 0x99, 0xd4, 0x8a, 0x68, 0x36, 0xb1, 0xef, 0xd, 0x53, 0x97, 0xc9, 0x2b, 0x75, 0xf2, 0xac, 0x4e, 0x10, 0x5d, 0x3, 0xe1, 0xbf, 0x38, 0x66, 0x84, 0xda, 0x11, 0x4f, 0xad, 0xf3, 0x74, 0x2a, 0xc8, 0x96, 0xdb, 0x85, 0x67, 0x39, 0xbe, 0xe0, 0x2, 0x5c, 0x98, 0xc6, 0x24, 0x7a, 0xfd, 0xa3, 0x41, 0x1f, 0x52, 0xc, 0xee, 0xb0, 0x37, 0x69, 0x8b, 0xd5, 0x3c, 0x62, 0x80, 0xde, 0x59, 0x7, 0xe5, 0xbb, 0xf6, 0xa8, 0x4a, 0x14, 0x93, 0xcd, 0x2f, 0x71, 0xb5, 0xeb, 0x9, 0x57, 0xd0, 0x8e, 0x6c, 0x32, 0x7f, 0x21, 0xc3, 0x9d, 0x1a, 0x44, 0xa6, 0xf8, 0x33, 0x6d, 0x8f, 0xd1, 0x56, 0x8, 0xea, 0xb4, 0xf9, 0xa7, 0x45, 0x1b, 0x9c, 0xc2, 0x20, 0x7e, 0xba, 0xe4, 0x6, 0x58, 0xdf, 0x81, 0x63, 0x3d, 0x70, 0x2e, 0xcc, 0x92, 0x15, 0x4b, 0xa9, 0xf7, 0x22, 0x7c, 0x9e, 0xc0, 0x47, 0x19, 0xfb, 0xa5, 0xe8, 0xb6, 0x54, 0xa, 0x8d, 0xd3, 0x31, 0x6f, 0xab, 0xf5, 0x17, 0x49, 0xce, 0x90, 0x72, 0x2c, 0x61, 0x3f, 0xdd, 0x83, 0x4, 0x5a, 0xb8, 0xe6, 0x2d, 0x73, 0x91, 0xcf, 0x48, 0x16, 0xf4, 0xaa, 0xe7, 0xb9, 0x5b, 0x5, 0x82, 0xdc, 0x3e, 0x60, 0xa4, 0xfa, 0x18, 0x46, 0xc1, 0x9f, 0x7d, 0x23, 0x6e, 0x30, 0xd2, 0x8c, 0xb, 0x55, 0xb7, 0xe9},
- {0x0, 0x5f, 0xbe, 0xe1, 0x61, 0x3e, 0xdf, 0x80, 0xc2, 0x9d, 0x7c, 0x23, 0xa3, 0xfc, 0x1d, 0x42, 0x99, 0xc6, 0x27, 0x78, 0xf8, 0xa7, 0x46, 0x19, 0x5b, 0x4, 0xe5, 0xba, 0x3a, 0x65, 0x84, 0xdb, 0x2f, 0x70, 0x91, 0xce, 0x4e, 0x11, 0xf0, 0xaf, 0xed, 0xb2, 0x53, 0xc, 0x8c, 0xd3, 0x32, 0x6d, 0xb6, 0xe9, 0x8, 0x57, 0xd7, 0x88, 0x69, 0x36, 0x74, 0x2b, 0xca, 0x95, 0x15, 0x4a, 0xab, 0xf4, 0x5e, 0x1, 0xe0, 0xbf, 0x3f, 0x60, 0x81, 0xde, 0x9c, 0xc3, 0x22, 0x7d, 0xfd, 0xa2, 0x43, 0x1c, 0xc7, 0x98, 0x79, 0x26, 0xa6, 0xf9, 0x18, 0x47, 0x5, 0x5a, 0xbb, 0xe4, 0x64, 0x3b, 0xda, 0x85, 0x71, 0x2e, 0xcf, 0x90, 0x10, 0x4f, 0xae, 0xf1, 0xb3, 0xec, 0xd, 0x52, 0xd2, 0x8d, 0x6c, 0x33, 0xe8, 0xb7, 0x56, 0x9, 0x89, 0xd6, 0x37, 0x68, 0x2a, 0x75, 0x94, 0xcb, 0x4b, 0x14, 0xf5, 0xaa, 0xbc, 0xe3, 0x2, 0x5d, 0xdd, 0x82, 0x63, 0x3c, 0x7e, 0x21, 0xc0, 0x9f, 0x1f, 0x40, 0xa1, 0xfe, 0x25, 0x7a, 0x9b, 0xc4, 0x44, 0x1b, 0xfa, 0xa5, 0xe7, 0xb8, 0x59, 0x6, 0x86, 0xd9, 0x38, 0x67, 0x93, 0xcc, 0x2d, 0x72, 0xf2, 0xad, 0x4c, 0x13, 0x51, 0xe, 0xef, 0xb0, 0x30, 0x6f, 0x8e, 0xd1, 0xa, 0x55, 0xb4, 0xeb, 0x6b, 0x34, 0xd5, 0x8a, 0xc8, 0x97, 0x76, 0x29, 0xa9, 0xf6, 0x17, 0x48, 0xe2, 0xbd, 0x5c, 0x3, 0x83, 0xdc, 0x3d, 0x62, 0x20, 0x7f, 0x9e, 0xc1, 0x41, 0x1e, 0xff, 0xa0, 0x7b, 0x24, 0xc5, 0x9a, 0x1a, 0x45, 0xa4, 0xfb, 0xb9, 0xe6, 0x7, 0x58, 0xd8, 0x87, 0x66, 0x39, 0xcd, 0x92, 0x73, 0x2c, 0xac, 0xf3, 0x12, 0x4d, 0xf, 0x50, 0xb1, 0xee, 0x6e, 0x31, 0xd0, 0x8f, 0x54, 0xb, 0xea, 0xb5, 0x35, 0x6a, 0x8b, 0xd4, 0x96, 0xc9, 0x28, 0x77, 0xf7, 0xa8, 0x49, 0x16},
- {0x0, 0x60, 0xc0, 0xa0, 0x9d, 0xfd, 0x5d, 0x3d, 0x27, 0x47, 0xe7, 0x87, 0xba, 0xda, 0x7a, 0x1a, 0x4e, 0x2e, 0x8e, 0xee, 0xd3, 0xb3, 0x13, 0x73, 0x69, 0x9, 0xa9, 0xc9, 0xf4, 0x94, 0x34, 0x54, 0x9c, 0xfc, 0x5c, 0x3c, 0x1, 0x61, 0xc1, 0xa1, 0xbb, 0xdb, 0x7b, 0x1b, 0x26, 0x46, 0xe6, 0x86, 0xd2, 0xb2, 0x12, 0x72, 0x4f, 0x2f, 0x8f, 0xef, 0xf5, 0x95, 0x35, 0x55, 0x68, 0x8, 0xa8, 0xc8, 0x25, 0x45, 0xe5, 0x85, 0xb8, 0xd8, 0x78, 0x18, 0x2, 0x62, 0xc2, 0xa2, 0x9f, 0xff, 0x5f, 0x3f, 0x6b, 0xb, 0xab, 0xcb, 0xf6, 0x96, 0x36, 0x56, 0x4c, 0x2c, 0x8c, 0xec, 0xd1, 0xb1, 0x11, 0x71, 0xb9, 0xd9, 0x79, 0x19, 0x24, 0x44, 0xe4, 0x84, 0x9e, 0xfe, 0x5e, 0x3e, 0x3, 0x63, 0xc3, 0xa3, 0xf7, 0x97, 0x37, 0x57, 0x6a, 0xa, 0xaa, 0xca, 0xd0, 0xb0, 0x10, 0x70, 0x4d, 0x2d, 0x8d, 0xed, 0x4a, 0x2a, 0x8a, 0xea, 0xd7, 0xb7, 0x17, 0x77, 0x6d, 0xd, 0xad, 0xcd, 0xf0, 0x90, 0x30, 0x50, 0x4, 0x64, 0xc4, 0xa4, 0x99, 0xf9, 0x59, 0x39, 0x23, 0x43, 0xe3, 0x83, 0xbe, 0xde, 0x7e, 0x1e, 0xd6, 0xb6, 0x16, 0x76, 0x4b, 0x2b, 0x8b, 0xeb, 0xf1, 0x91, 0x31, 0x51, 0x6c, 0xc, 0xac, 0xcc, 0x98, 0xf8, 0x58, 0x38, 0x5, 0x65, 0xc5, 0xa5, 0xbf, 0xdf, 0x7f, 0x1f, 0x22, 0x42, 0xe2, 0x82, 0x6f, 0xf, 0xaf, 0xcf, 0xf2, 0x92, 0x32, 0x52, 0x48, 0x28, 0x88, 0xe8, 0xd5, 0xb5, 0x15, 0x75, 0x21, 0x41, 0xe1, 0x81, 0xbc, 0xdc, 0x7c, 0x1c, 0x6, 0x66, 0xc6, 0xa6, 0x9b, 0xfb, 0x5b, 0x3b, 0xf3, 0x93, 0x33, 0x53, 0x6e, 0xe, 0xae, 0xce, 0xd4, 0xb4, 0x14, 0x74, 0x49, 0x29, 0x89, 0xe9, 0xbd, 0xdd, 0x7d, 0x1d, 0x20, 0x40, 0xe0, 0x80, 0x9a, 0xfa, 0x5a, 0x3a, 0x7, 0x67, 0xc7, 0xa7},
- {0x0, 0x61, 0xc2, 0xa3, 0x99, 0xf8, 0x5b, 0x3a, 0x2f, 0x4e, 0xed, 0x8c, 0xb6, 0xd7, 0x74, 0x15, 0x5e, 0x3f, 0x9c, 0xfd, 0xc7, 0xa6, 0x5, 0x64, 0x71, 0x10, 0xb3, 0xd2, 0xe8, 0x89, 0x2a, 0x4b, 0xbc, 0xdd, 0x7e, 0x1f, 0x25, 0x44, 0xe7, 0x86, 0x93, 0xf2, 0x51, 0x30, 0xa, 0x6b, 0xc8, 0xa9, 0xe2, 0x83, 0x20, 0x41, 0x7b, 0x1a, 0xb9, 0xd8, 0xcd, 0xac, 0xf, 0x6e, 0x54, 0x35, 0x96, 0xf7, 0x65, 0x4, 0xa7, 0xc6, 0xfc, 0x9d, 0x3e, 0x5f, 0x4a, 0x2b, 0x88, 0xe9, 0xd3, 0xb2, 0x11, 0x70, 0x3b, 0x5a, 0xf9, 0x98, 0xa2, 0xc3, 0x60, 0x1, 0x14, 0x75, 0xd6, 0xb7, 0x8d, 0xec, 0x4f, 0x2e, 0xd9, 0xb8, 0x1b, 0x7a, 0x40, 0x21, 0x82, 0xe3, 0xf6, 0x97, 0x34, 0x55, 0x6f, 0xe, 0xad, 0xcc, 0x87, 0xe6, 0x45, 0x24, 0x1e, 0x7f, 0xdc, 0xbd, 0xa8, 0xc9, 0x6a, 0xb, 0x31, 0x50, 0xf3, 0x92, 0xca, 0xab, 0x8, 0x69, 0x53, 0x32, 0x91, 0xf0, 0xe5, 0x84, 0x27, 0x46, 0x7c, 0x1d, 0xbe, 0xdf, 0x94, 0xf5, 0x56, 0x37, 0xd, 0x6c, 0xcf, 0xae, 0xbb, 0xda, 0x79, 0x18, 0x22, 0x43, 0xe0, 0x81, 0x76, 0x17, 0xb4, 0xd5, 0xef, 0x8e, 0x2d, 0x4c, 0x59, 0x38, 0x9b, 0xfa, 0xc0, 0xa1, 0x2, 0x63, 0x28, 0x49, 0xea, 0x8b, 0xb1, 0xd0, 0x73, 0x12, 0x7, 0x66, 0xc5, 0xa4, 0x9e, 0xff, 0x5c, 0x3d, 0xaf, 0xce, 0x6d, 0xc, 0x36, 0x57, 0xf4, 0x95, 0x80, 0xe1, 0x42, 0x23, 0x19, 0x78, 0xdb, 0xba, 0xf1, 0x90, 0x33, 0x52, 0x68, 0x9, 0xaa, 0xcb, 0xde, 0xbf, 0x1c, 0x7d, 0x47, 0x26, 0x85, 0xe4, 0x13, 0x72, 0xd1, 0xb0, 0x8a, 0xeb, 0x48, 0x29, 0x3c, 0x5d, 0xfe, 0x9f, 0xa5, 0xc4, 0x67, 0x6, 0x4d, 0x2c, 0x8f, 0xee, 0xd4, 0xb5, 0x16, 0x77, 0x62, 0x3, 0xa0, 0xc1, 0xfb, 0x9a, 0x39, 0x58},
- {0x0, 0x62, 0xc4, 0xa6, 0x95, 0xf7, 0x51, 0x33, 0x37, 0x55, 0xf3, 0x91, 0xa2, 0xc0, 0x66, 0x4, 0x6e, 0xc, 0xaa, 0xc8, 0xfb, 0x99, 0x3f, 0x5d, 0x59, 0x3b, 0x9d, 0xff, 0xcc, 0xae, 0x8, 0x6a, 0xdc, 0xbe, 0x18, 0x7a, 0x49, 0x2b, 0x8d, 0xef, 0xeb, 0x89, 0x2f, 0x4d, 0x7e, 0x1c, 0xba, 0xd8, 0xb2, 0xd0, 0x76, 0x14, 0x27, 0x45, 0xe3, 0x81, 0x85, 0xe7, 0x41, 0x23, 0x10, 0x72, 0xd4, 0xb6, 0xa5, 0xc7, 0x61, 0x3, 0x30, 0x52, 0xf4, 0x96, 0x92, 0xf0, 0x56, 0x34, 0x7, 0x65, 0xc3, 0xa1, 0xcb, 0xa9, 0xf, 0x6d, 0x5e, 0x3c, 0x9a, 0xf8, 0xfc, 0x9e, 0x38, 0x5a, 0x69, 0xb, 0xad, 0xcf, 0x79, 0x1b, 0xbd, 0xdf, 0xec, 0x8e, 0x28, 0x4a, 0x4e, 0x2c, 0x8a, 0xe8, 0xdb, 0xb9, 0x1f, 0x7d, 0x17, 0x75, 0xd3, 0xb1, 0x82, 0xe0, 0x46, 0x24, 0x20, 0x42, 0xe4, 0x86, 0xb5, 0xd7, 0x71, 0x13, 0x57, 0x35, 0x93, 0xf1, 0xc2, 0xa0, 0x6, 0x64, 0x60, 0x2, 0xa4, 0xc6, 0xf5, 0x97, 0x31, 0x53, 0x39, 0x5b, 0xfd, 0x9f, 0xac, 0xce, 0x68, 0xa, 0xe, 0x6c, 0xca, 0xa8, 0x9b, 0xf9, 0x5f, 0x3d, 0x8b, 0xe9, 0x4f, 0x2d, 0x1e, 0x7c, 0xda, 0xb8, 0xbc, 0xde, 0x78, 0x1a, 0x29, 0x4b, 0xed, 0x8f, 0xe5, 0x87, 0x21, 0x43, 0x70, 0x12, 0xb4, 0xd6, 0xd2, 0xb0, 0x16, 0x74, 0x47, 0x25, 0x83, 0xe1, 0xf2, 0x90, 0x36, 0x54, 0x67, 0x5, 0xa3, 0xc1, 0xc5, 0xa7, 0x1, 0x63, 0x50, 0x32, 0x94, 0xf6, 0x9c, 0xfe, 0x58, 0x3a, 0x9, 0x6b, 0xcd, 0xaf, 0xab, 0xc9, 0x6f, 0xd, 0x3e, 0x5c, 0xfa, 0x98, 0x2e, 0x4c, 0xea, 0x88, 0xbb, 0xd9, 0x7f, 0x1d, 0x19, 0x7b, 0xdd, 0xbf, 0x8c, 0xee, 0x48, 0x2a, 0x40, 0x22, 0x84, 0xe6, 0xd5, 0xb7, 0x11, 0x73, 0x77, 0x15, 0xb3, 0xd1, 0xe2, 0x80, 0x26, 0x44},
- {0x0, 0x63, 0xc6, 0xa5, 0x91, 0xf2, 0x57, 0x34, 0x3f, 0x5c, 0xf9, 0x9a, 0xae, 0xcd, 0x68, 0xb, 0x7e, 0x1d, 0xb8, 0xdb, 0xef, 0x8c, 0x29, 0x4a, 0x41, 0x22, 0x87, 0xe4, 0xd0, 0xb3, 0x16, 0x75, 0xfc, 0x9f, 0x3a, 0x59, 0x6d, 0xe, 0xab, 0xc8, 0xc3, 0xa0, 0x5, 0x66, 0x52, 0x31, 0x94, 0xf7, 0x82, 0xe1, 0x44, 0x27, 0x13, 0x70, 0xd5, 0xb6, 0xbd, 0xde, 0x7b, 0x18, 0x2c, 0x4f, 0xea, 0x89, 0xe5, 0x86, 0x23, 0x40, 0x74, 0x17, 0xb2, 0xd1, 0xda, 0xb9, 0x1c, 0x7f, 0x4b, 0x28, 0x8d, 0xee, 0x9b, 0xf8, 0x5d, 0x3e, 0xa, 0x69, 0xcc, 0xaf, 0xa4, 0xc7, 0x62, 0x1, 0x35, 0x56, 0xf3, 0x90, 0x19, 0x7a, 0xdf, 0xbc, 0x88, 0xeb, 0x4e, 0x2d, 0x26, 0x45, 0xe0, 0x83, 0xb7, 0xd4, 0x71, 0x12, 0x67, 0x4, 0xa1, 0xc2, 0xf6, 0x95, 0x30, 0x53, 0x58, 0x3b, 0x9e, 0xfd, 0xc9, 0xaa, 0xf, 0x6c, 0xd7, 0xb4, 0x11, 0x72, 0x46, 0x25, 0x80, 0xe3, 0xe8, 0x8b, 0x2e, 0x4d, 0x79, 0x1a, 0xbf, 0xdc, 0xa9, 0xca, 0x6f, 0xc, 0x38, 0x5b, 0xfe, 0x9d, 0x96, 0xf5, 0x50, 0x33, 0x7, 0x64, 0xc1, 0xa2, 0x2b, 0x48, 0xed, 0x8e, 0xba, 0xd9, 0x7c, 0x1f, 0x14, 0x77, 0xd2, 0xb1, 0x85, 0xe6, 0x43, 0x20, 0x55, 0x36, 0x93, 0xf0, 0xc4, 0xa7, 0x2, 0x61, 0x6a, 0x9, 0xac, 0xcf, 0xfb, 0x98, 0x3d, 0x5e, 0x32, 0x51, 0xf4, 0x97, 0xa3, 0xc0, 0x65, 0x6, 0xd, 0x6e, 0xcb, 0xa8, 0x9c, 0xff, 0x5a, 0x39, 0x4c, 0x2f, 0x8a, 0xe9, 0xdd, 0xbe, 0x1b, 0x78, 0x73, 0x10, 0xb5, 0xd6, 0xe2, 0x81, 0x24, 0x47, 0xce, 0xad, 0x8, 0x6b, 0x5f, 0x3c, 0x99, 0xfa, 0xf1, 0x92, 0x37, 0x54, 0x60, 0x3, 0xa6, 0xc5, 0xb0, 0xd3, 0x76, 0x15, 0x21, 0x42, 0xe7, 0x84, 0x8f, 0xec, 0x49, 0x2a, 0x1e, 0x7d, 0xd8, 0xbb},
- {0x0, 0x64, 0xc8, 0xac, 0x8d, 0xe9, 0x45, 0x21, 0x7, 0x63, 0xcf, 0xab, 0x8a, 0xee, 0x42, 0x26, 0xe, 0x6a, 0xc6, 0xa2, 0x83, 0xe7, 0x4b, 0x2f, 0x9, 0x6d, 0xc1, 0xa5, 0x84, 0xe0, 0x4c, 0x28, 0x1c, 0x78, 0xd4, 0xb0, 0x91, 0xf5, 0x59, 0x3d, 0x1b, 0x7f, 0xd3, 0xb7, 0x96, 0xf2, 0x5e, 0x3a, 0x12, 0x76, 0xda, 0xbe, 0x9f, 0xfb, 0x57, 0x33, 0x15, 0x71, 0xdd, 0xb9, 0x98, 0xfc, 0x50, 0x34, 0x38, 0x5c, 0xf0, 0x94, 0xb5, 0xd1, 0x7d, 0x19, 0x3f, 0x5b, 0xf7, 0x93, 0xb2, 0xd6, 0x7a, 0x1e, 0x36, 0x52, 0xfe, 0x9a, 0xbb, 0xdf, 0x73, 0x17, 0x31, 0x55, 0xf9, 0x9d, 0xbc, 0xd8, 0x74, 0x10, 0x24, 0x40, 0xec, 0x88, 0xa9, 0xcd, 0x61, 0x5, 0x23, 0x47, 0xeb, 0x8f, 0xae, 0xca, 0x66, 0x2, 0x2a, 0x4e, 0xe2, 0x86, 0xa7, 0xc3, 0x6f, 0xb, 0x2d, 0x49, 0xe5, 0x81, 0xa0, 0xc4, 0x68, 0xc, 0x70, 0x14, 0xb8, 0xdc, 0xfd, 0x99, 0x35, 0x51, 0x77, 0x13, 0xbf, 0xdb, 0xfa, 0x9e, 0x32, 0x56, 0x7e, 0x1a, 0xb6, 0xd2, 0xf3, 0x97, 0x3b, 0x5f, 0x79, 0x1d, 0xb1, 0xd5, 0xf4, 0x90, 0x3c, 0x58, 0x6c, 0x8, 0xa4, 0xc0, 0xe1, 0x85, 0x29, 0x4d, 0x6b, 0xf, 0xa3, 0xc7, 0xe6, 0x82, 0x2e, 0x4a, 0x62, 0x6, 0xaa, 0xce, 0xef, 0x8b, 0x27, 0x43, 0x65, 0x1, 0xad, 0xc9, 0xe8, 0x8c, 0x20, 0x44, 0x48, 0x2c, 0x80, 0xe4, 0xc5, 0xa1, 0xd, 0x69, 0x4f, 0x2b, 0x87, 0xe3, 0xc2, 0xa6, 0xa, 0x6e, 0x46, 0x22, 0x8e, 0xea, 0xcb, 0xaf, 0x3, 0x67, 0x41, 0x25, 0x89, 0xed, 0xcc, 0xa8, 0x4, 0x60, 0x54, 0x30, 0x9c, 0xf8, 0xd9, 0xbd, 0x11, 0x75, 0x53, 0x37, 0x9b, 0xff, 0xde, 0xba, 0x16, 0x72, 0x5a, 0x3e, 0x92, 0xf6, 0xd7, 0xb3, 0x1f, 0x7b, 0x5d, 0x39, 0x95, 0xf1, 0xd0, 0xb4, 0x18, 0x7c},
- {0x0, 0x65, 0xca, 0xaf, 0x89, 0xec, 0x43, 0x26, 0xf, 0x6a, 0xc5, 0xa0, 0x86, 0xe3, 0x4c, 0x29, 0x1e, 0x7b, 0xd4, 0xb1, 0x97, 0xf2, 0x5d, 0x38, 0x11, 0x74, 0xdb, 0xbe, 0x98, 0xfd, 0x52, 0x37, 0x3c, 0x59, 0xf6, 0x93, 0xb5, 0xd0, 0x7f, 0x1a, 0x33, 0x56, 0xf9, 0x9c, 0xba, 0xdf, 0x70, 0x15, 0x22, 0x47, 0xe8, 0x8d, 0xab, 0xce, 0x61, 0x4, 0x2d, 0x48, 0xe7, 0x82, 0xa4, 0xc1, 0x6e, 0xb, 0x78, 0x1d, 0xb2, 0xd7, 0xf1, 0x94, 0x3b, 0x5e, 0x77, 0x12, 0xbd, 0xd8, 0xfe, 0x9b, 0x34, 0x51, 0x66, 0x3, 0xac, 0xc9, 0xef, 0x8a, 0x25, 0x40, 0x69, 0xc, 0xa3, 0xc6, 0xe0, 0x85, 0x2a, 0x4f, 0x44, 0x21, 0x8e, 0xeb, 0xcd, 0xa8, 0x7, 0x62, 0x4b, 0x2e, 0x81, 0xe4, 0xc2, 0xa7, 0x8, 0x6d, 0x5a, 0x3f, 0x90, 0xf5, 0xd3, 0xb6, 0x19, 0x7c, 0x55, 0x30, 0x9f, 0xfa, 0xdc, 0xb9, 0x16, 0x73, 0xf0, 0x95, 0x3a, 0x5f, 0x79, 0x1c, 0xb3, 0xd6, 0xff, 0x9a, 0x35, 0x50, 0x76, 0x13, 0xbc, 0xd9, 0xee, 0x8b, 0x24, 0x41, 0x67, 0x2, 0xad, 0xc8, 0xe1, 0x84, 0x2b, 0x4e, 0x68, 0xd, 0xa2, 0xc7, 0xcc, 0xa9, 0x6, 0x63, 0x45, 0x20, 0x8f, 0xea, 0xc3, 0xa6, 0x9, 0x6c, 0x4a, 0x2f, 0x80, 0xe5, 0xd2, 0xb7, 0x18, 0x7d, 0x5b, 0x3e, 0x91, 0xf4, 0xdd, 0xb8, 0x17, 0x72, 0x54, 0x31, 0x9e, 0xfb, 0x88, 0xed, 0x42, 0x27, 0x1, 0x64, 0xcb, 0xae, 0x87, 0xe2, 0x4d, 0x28, 0xe, 0x6b, 0xc4, 0xa1, 0x96, 0xf3, 0x5c, 0x39, 0x1f, 0x7a, 0xd5, 0xb0, 0x99, 0xfc, 0x53, 0x36, 0x10, 0x75, 0xda, 0xbf, 0xb4, 0xd1, 0x7e, 0x1b, 0x3d, 0x58, 0xf7, 0x92, 0xbb, 0xde, 0x71, 0x14, 0x32, 0x57, 0xf8, 0x9d, 0xaa, 0xcf, 0x60, 0x5, 0x23, 0x46, 0xe9, 0x8c, 0xa5, 0xc0, 0x6f, 0xa, 0x2c, 0x49, 0xe6, 0x83},
- {0x0, 0x66, 0xcc, 0xaa, 0x85, 0xe3, 0x49, 0x2f, 0x17, 0x71, 0xdb, 0xbd, 0x92, 0xf4, 0x5e, 0x38, 0x2e, 0x48, 0xe2, 0x84, 0xab, 0xcd, 0x67, 0x1, 0x39, 0x5f, 0xf5, 0x93, 0xbc, 0xda, 0x70, 0x16, 0x5c, 0x3a, 0x90, 0xf6, 0xd9, 0xbf, 0x15, 0x73, 0x4b, 0x2d, 0x87, 0xe1, 0xce, 0xa8, 0x2, 0x64, 0x72, 0x14, 0xbe, 0xd8, 0xf7, 0x91, 0x3b, 0x5d, 0x65, 0x3, 0xa9, 0xcf, 0xe0, 0x86, 0x2c, 0x4a, 0xb8, 0xde, 0x74, 0x12, 0x3d, 0x5b, 0xf1, 0x97, 0xaf, 0xc9, 0x63, 0x5, 0x2a, 0x4c, 0xe6, 0x80, 0x96, 0xf0, 0x5a, 0x3c, 0x13, 0x75, 0xdf, 0xb9, 0x81, 0xe7, 0x4d, 0x2b, 0x4, 0x62, 0xc8, 0xae, 0xe4, 0x82, 0x28, 0x4e, 0x61, 0x7, 0xad, 0xcb, 0xf3, 0x95, 0x3f, 0x59, 0x76, 0x10, 0xba, 0xdc, 0xca, 0xac, 0x6, 0x60, 0x4f, 0x29, 0x83, 0xe5, 0xdd, 0xbb, 0x11, 0x77, 0x58, 0x3e, 0x94, 0xf2, 0x6d, 0xb, 0xa1, 0xc7, 0xe8, 0x8e, 0x24, 0x42, 0x7a, 0x1c, 0xb6, 0xd0, 0xff, 0x99, 0x33, 0x55, 0x43, 0x25, 0x8f, 0xe9, 0xc6, 0xa0, 0xa, 0x6c, 0x54, 0x32, 0x98, 0xfe, 0xd1, 0xb7, 0x1d, 0x7b, 0x31, 0x57, 0xfd, 0x9b, 0xb4, 0xd2, 0x78, 0x1e, 0x26, 0x40, 0xea, 0x8c, 0xa3, 0xc5, 0x6f, 0x9, 0x1f, 0x79, 0xd3, 0xb5, 0x9a, 0xfc, 0x56, 0x30, 0x8, 0x6e, 0xc4, 0xa2, 0x8d, 0xeb, 0x41, 0x27, 0xd5, 0xb3, 0x19, 0x7f, 0x50, 0x36, 0x9c, 0xfa, 0xc2, 0xa4, 0xe, 0x68, 0x47, 0x21, 0x8b, 0xed, 0xfb, 0x9d, 0x37, 0x51, 0x7e, 0x18, 0xb2, 0xd4, 0xec, 0x8a, 0x20, 0x46, 0x69, 0xf, 0xa5, 0xc3, 0x89, 0xef, 0x45, 0x23, 0xc, 0x6a, 0xc0, 0xa6, 0x9e, 0xf8, 0x52, 0x34, 0x1b, 0x7d, 0xd7, 0xb1, 0xa7, 0xc1, 0x6b, 0xd, 0x22, 0x44, 0xee, 0x88, 0xb0, 0xd6, 0x7c, 0x1a, 0x35, 0x53, 0xf9, 0x9f},
- {0x0, 0x67, 0xce, 0xa9, 0x81, 0xe6, 0x4f, 0x28, 0x1f, 0x78, 0xd1, 0xb6, 0x9e, 0xf9, 0x50, 0x37, 0x3e, 0x59, 0xf0, 0x97, 0xbf, 0xd8, 0x71, 0x16, 0x21, 0x46, 0xef, 0x88, 0xa0, 0xc7, 0x6e, 0x9, 0x7c, 0x1b, 0xb2, 0xd5, 0xfd, 0x9a, 0x33, 0x54, 0x63, 0x4, 0xad, 0xca, 0xe2, 0x85, 0x2c, 0x4b, 0x42, 0x25, 0x8c, 0xeb, 0xc3, 0xa4, 0xd, 0x6a, 0x5d, 0x3a, 0x93, 0xf4, 0xdc, 0xbb, 0x12, 0x75, 0xf8, 0x9f, 0x36, 0x51, 0x79, 0x1e, 0xb7, 0xd0, 0xe7, 0x80, 0x29, 0x4e, 0x66, 0x1, 0xa8, 0xcf, 0xc6, 0xa1, 0x8, 0x6f, 0x47, 0x20, 0x89, 0xee, 0xd9, 0xbe, 0x17, 0x70, 0x58, 0x3f, 0x96, 0xf1, 0x84, 0xe3, 0x4a, 0x2d, 0x5, 0x62, 0xcb, 0xac, 0x9b, 0xfc, 0x55, 0x32, 0x1a, 0x7d, 0xd4, 0xb3, 0xba, 0xdd, 0x74, 0x13, 0x3b, 0x5c, 0xf5, 0x92, 0xa5, 0xc2, 0x6b, 0xc, 0x24, 0x43, 0xea, 0x8d, 0xed, 0x8a, 0x23, 0x44, 0x6c, 0xb, 0xa2, 0xc5, 0xf2, 0x95, 0x3c, 0x5b, 0x73, 0x14, 0xbd, 0xda, 0xd3, 0xb4, 0x1d, 0x7a, 0x52, 0x35, 0x9c, 0xfb, 0xcc, 0xab, 0x2, 0x65, 0x4d, 0x2a, 0x83, 0xe4, 0x91, 0xf6, 0x5f, 0x38, 0x10, 0x77, 0xde, 0xb9, 0x8e, 0xe9, 0x40, 0x27, 0xf, 0x68, 0xc1, 0xa6, 0xaf, 0xc8, 0x61, 0x6, 0x2e, 0x49, 0xe0, 0x87, 0xb0, 0xd7, 0x7e, 0x19, 0x31, 0x56, 0xff, 0x98, 0x15, 0x72, 0xdb, 0xbc, 0x94, 0xf3, 0x5a, 0x3d, 0xa, 0x6d, 0xc4, 0xa3, 0x8b, 0xec, 0x45, 0x22, 0x2b, 0x4c, 0xe5, 0x82, 0xaa, 0xcd, 0x64, 0x3, 0x34, 0x53, 0xfa, 0x9d, 0xb5, 0xd2, 0x7b, 0x1c, 0x69, 0xe, 0xa7, 0xc0, 0xe8, 0x8f, 0x26, 0x41, 0x76, 0x11, 0xb8, 0xdf, 0xf7, 0x90, 0x39, 0x5e, 0x57, 0x30, 0x99, 0xfe, 0xd6, 0xb1, 0x18, 0x7f, 0x48, 0x2f, 0x86, 0xe1, 0xc9, 0xae, 0x7, 0x60},
- {0x0, 0x68, 0xd0, 0xb8, 0xbd, 0xd5, 0x6d, 0x5, 0x67, 0xf, 0xb7, 0xdf, 0xda, 0xb2, 0xa, 0x62, 0xce, 0xa6, 0x1e, 0x76, 0x73, 0x1b, 0xa3, 0xcb, 0xa9, 0xc1, 0x79, 0x11, 0x14, 0x7c, 0xc4, 0xac, 0x81, 0xe9, 0x51, 0x39, 0x3c, 0x54, 0xec, 0x84, 0xe6, 0x8e, 0x36, 0x5e, 0x5b, 0x33, 0x8b, 0xe3, 0x4f, 0x27, 0x9f, 0xf7, 0xf2, 0x9a, 0x22, 0x4a, 0x28, 0x40, 0xf8, 0x90, 0x95, 0xfd, 0x45, 0x2d, 0x1f, 0x77, 0xcf, 0xa7, 0xa2, 0xca, 0x72, 0x1a, 0x78, 0x10, 0xa8, 0xc0, 0xc5, 0xad, 0x15, 0x7d, 0xd1, 0xb9, 0x1, 0x69, 0x6c, 0x4, 0xbc, 0xd4, 0xb6, 0xde, 0x66, 0xe, 0xb, 0x63, 0xdb, 0xb3, 0x9e, 0xf6, 0x4e, 0x26, 0x23, 0x4b, 0xf3, 0x9b, 0xf9, 0x91, 0x29, 0x41, 0x44, 0x2c, 0x94, 0xfc, 0x50, 0x38, 0x80, 0xe8, 0xed, 0x85, 0x3d, 0x55, 0x37, 0x5f, 0xe7, 0x8f, 0x8a, 0xe2, 0x5a, 0x32, 0x3e, 0x56, 0xee, 0x86, 0x83, 0xeb, 0x53, 0x3b, 0x59, 0x31, 0x89, 0xe1, 0xe4, 0x8c, 0x34, 0x5c, 0xf0, 0x98, 0x20, 0x48, 0x4d, 0x25, 0x9d, 0xf5, 0x97, 0xff, 0x47, 0x2f, 0x2a, 0x42, 0xfa, 0x92, 0xbf, 0xd7, 0x6f, 0x7, 0x2, 0x6a, 0xd2, 0xba, 0xd8, 0xb0, 0x8, 0x60, 0x65, 0xd, 0xb5, 0xdd, 0x71, 0x19, 0xa1, 0xc9, 0xcc, 0xa4, 0x1c, 0x74, 0x16, 0x7e, 0xc6, 0xae, 0xab, 0xc3, 0x7b, 0x13, 0x21, 0x49, 0xf1, 0x99, 0x9c, 0xf4, 0x4c, 0x24, 0x46, 0x2e, 0x96, 0xfe, 0xfb, 0x93, 0x2b, 0x43, 0xef, 0x87, 0x3f, 0x57, 0x52, 0x3a, 0x82, 0xea, 0x88, 0xe0, 0x58, 0x30, 0x35, 0x5d, 0xe5, 0x8d, 0xa0, 0xc8, 0x70, 0x18, 0x1d, 0x75, 0xcd, 0xa5, 0xc7, 0xaf, 0x17, 0x7f, 0x7a, 0x12, 0xaa, 0xc2, 0x6e, 0x6, 0xbe, 0xd6, 0xd3, 0xbb, 0x3, 0x6b, 0x9, 0x61, 0xd9, 0xb1, 0xb4, 0xdc, 0x64, 0xc},
- {0x0, 0x69, 0xd2, 0xbb, 0xb9, 0xd0, 0x6b, 0x2, 0x6f, 0x6, 0xbd, 0xd4, 0xd6, 0xbf, 0x4, 0x6d, 0xde, 0xb7, 0xc, 0x65, 0x67, 0xe, 0xb5, 0xdc, 0xb1, 0xd8, 0x63, 0xa, 0x8, 0x61, 0xda, 0xb3, 0xa1, 0xc8, 0x73, 0x1a, 0x18, 0x71, 0xca, 0xa3, 0xce, 0xa7, 0x1c, 0x75, 0x77, 0x1e, 0xa5, 0xcc, 0x7f, 0x16, 0xad, 0xc4, 0xc6, 0xaf, 0x14, 0x7d, 0x10, 0x79, 0xc2, 0xab, 0xa9, 0xc0, 0x7b, 0x12, 0x5f, 0x36, 0x8d, 0xe4, 0xe6, 0x8f, 0x34, 0x5d, 0x30, 0x59, 0xe2, 0x8b, 0x89, 0xe0, 0x5b, 0x32, 0x81, 0xe8, 0x53, 0x3a, 0x38, 0x51, 0xea, 0x83, 0xee, 0x87, 0x3c, 0x55, 0x57, 0x3e, 0x85, 0xec, 0xfe, 0x97, 0x2c, 0x45, 0x47, 0x2e, 0x95, 0xfc, 0x91, 0xf8, 0x43, 0x2a, 0x28, 0x41, 0xfa, 0x93, 0x20, 0x49, 0xf2, 0x9b, 0x99, 0xf0, 0x4b, 0x22, 0x4f, 0x26, 0x9d, 0xf4, 0xf6, 0x9f, 0x24, 0x4d, 0xbe, 0xd7, 0x6c, 0x5, 0x7, 0x6e, 0xd5, 0xbc, 0xd1, 0xb8, 0x3, 0x6a, 0x68, 0x1, 0xba, 0xd3, 0x60, 0x9, 0xb2, 0xdb, 0xd9, 0xb0, 0xb, 0x62, 0xf, 0x66, 0xdd, 0xb4, 0xb6, 0xdf, 0x64, 0xd, 0x1f, 0x76, 0xcd, 0xa4, 0xa6, 0xcf, 0x74, 0x1d, 0x70, 0x19, 0xa2, 0xcb, 0xc9, 0xa0, 0x1b, 0x72, 0xc1, 0xa8, 0x13, 0x7a, 0x78, 0x11, 0xaa, 0xc3, 0xae, 0xc7, 0x7c, 0x15, 0x17, 0x7e, 0xc5, 0xac, 0xe1, 0x88, 0x33, 0x5a, 0x58, 0x31, 0x8a, 0xe3, 0x8e, 0xe7, 0x5c, 0x35, 0x37, 0x5e, 0xe5, 0x8c, 0x3f, 0x56, 0xed, 0x84, 0x86, 0xef, 0x54, 0x3d, 0x50, 0x39, 0x82, 0xeb, 0xe9, 0x80, 0x3b, 0x52, 0x40, 0x29, 0x92, 0xfb, 0xf9, 0x90, 0x2b, 0x42, 0x2f, 0x46, 0xfd, 0x94, 0x96, 0xff, 0x44, 0x2d, 0x9e, 0xf7, 0x4c, 0x25, 0x27, 0x4e, 0xf5, 0x9c, 0xf1, 0x98, 0x23, 0x4a, 0x48, 0x21, 0x9a, 0xf3},
- {0x0, 0x6a, 0xd4, 0xbe, 0xb5, 0xdf, 0x61, 0xb, 0x77, 0x1d, 0xa3, 0xc9, 0xc2, 0xa8, 0x16, 0x7c, 0xee, 0x84, 0x3a, 0x50, 0x5b, 0x31, 0x8f, 0xe5, 0x99, 0xf3, 0x4d, 0x27, 0x2c, 0x46, 0xf8, 0x92, 0xc1, 0xab, 0x15, 0x7f, 0x74, 0x1e, 0xa0, 0xca, 0xb6, 0xdc, 0x62, 0x8, 0x3, 0x69, 0xd7, 0xbd, 0x2f, 0x45, 0xfb, 0x91, 0x9a, 0xf0, 0x4e, 0x24, 0x58, 0x32, 0x8c, 0xe6, 0xed, 0x87, 0x39, 0x53, 0x9f, 0xf5, 0x4b, 0x21, 0x2a, 0x40, 0xfe, 0x94, 0xe8, 0x82, 0x3c, 0x56, 0x5d, 0x37, 0x89, 0xe3, 0x71, 0x1b, 0xa5, 0xcf, 0xc4, 0xae, 0x10, 0x7a, 0x6, 0x6c, 0xd2, 0xb8, 0xb3, 0xd9, 0x67, 0xd, 0x5e, 0x34, 0x8a, 0xe0, 0xeb, 0x81, 0x3f, 0x55, 0x29, 0x43, 0xfd, 0x97, 0x9c, 0xf6, 0x48, 0x22, 0xb0, 0xda, 0x64, 0xe, 0x5, 0x6f, 0xd1, 0xbb, 0xc7, 0xad, 0x13, 0x79, 0x72, 0x18, 0xa6, 0xcc, 0x23, 0x49, 0xf7, 0x9d, 0x96, 0xfc, 0x42, 0x28, 0x54, 0x3e, 0x80, 0xea, 0xe1, 0x8b, 0x35, 0x5f, 0xcd, 0xa7, 0x19, 0x73, 0x78, 0x12, 0xac, 0xc6, 0xba, 0xd0, 0x6e, 0x4, 0xf, 0x65, 0xdb, 0xb1, 0xe2, 0x88, 0x36, 0x5c, 0x57, 0x3d, 0x83, 0xe9, 0x95, 0xff, 0x41, 0x2b, 0x20, 0x4a, 0xf4, 0x9e, 0xc, 0x66, 0xd8, 0xb2, 0xb9, 0xd3, 0x6d, 0x7, 0x7b, 0x11, 0xaf, 0xc5, 0xce, 0xa4, 0x1a, 0x70, 0xbc, 0xd6, 0x68, 0x2, 0x9, 0x63, 0xdd, 0xb7, 0xcb, 0xa1, 0x1f, 0x75, 0x7e, 0x14, 0xaa, 0xc0, 0x52, 0x38, 0x86, 0xec, 0xe7, 0x8d, 0x33, 0x59, 0x25, 0x4f, 0xf1, 0x9b, 0x90, 0xfa, 0x44, 0x2e, 0x7d, 0x17, 0xa9, 0xc3, 0xc8, 0xa2, 0x1c, 0x76, 0xa, 0x60, 0xde, 0xb4, 0xbf, 0xd5, 0x6b, 0x1, 0x93, 0xf9, 0x47, 0x2d, 0x26, 0x4c, 0xf2, 0x98, 0xe4, 0x8e, 0x30, 0x5a, 0x51, 0x3b, 0x85, 0xef},
- {0x0, 0x6b, 0xd6, 0xbd, 0xb1, 0xda, 0x67, 0xc, 0x7f, 0x14, 0xa9, 0xc2, 0xce, 0xa5, 0x18, 0x73, 0xfe, 0x95, 0x28, 0x43, 0x4f, 0x24, 0x99, 0xf2, 0x81, 0xea, 0x57, 0x3c, 0x30, 0x5b, 0xe6, 0x8d, 0xe1, 0x8a, 0x37, 0x5c, 0x50, 0x3b, 0x86, 0xed, 0x9e, 0xf5, 0x48, 0x23, 0x2f, 0x44, 0xf9, 0x92, 0x1f, 0x74, 0xc9, 0xa2, 0xae, 0xc5, 0x78, 0x13, 0x60, 0xb, 0xb6, 0xdd, 0xd1, 0xba, 0x7, 0x6c, 0xdf, 0xb4, 0x9, 0x62, 0x6e, 0x5, 0xb8, 0xd3, 0xa0, 0xcb, 0x76, 0x1d, 0x11, 0x7a, 0xc7, 0xac, 0x21, 0x4a, 0xf7, 0x9c, 0x90, 0xfb, 0x46, 0x2d, 0x5e, 0x35, 0x88, 0xe3, 0xef, 0x84, 0x39, 0x52, 0x3e, 0x55, 0xe8, 0x83, 0x8f, 0xe4, 0x59, 0x32, 0x41, 0x2a, 0x97, 0xfc, 0xf0, 0x9b, 0x26, 0x4d, 0xc0, 0xab, 0x16, 0x7d, 0x71, 0x1a, 0xa7, 0xcc, 0xbf, 0xd4, 0x69, 0x2, 0xe, 0x65, 0xd8, 0xb3, 0xa3, 0xc8, 0x75, 0x1e, 0x12, 0x79, 0xc4, 0xaf, 0xdc, 0xb7, 0xa, 0x61, 0x6d, 0x6, 0xbb, 0xd0, 0x5d, 0x36, 0x8b, 0xe0, 0xec, 0x87, 0x3a, 0x51, 0x22, 0x49, 0xf4, 0x9f, 0x93, 0xf8, 0x45, 0x2e, 0x42, 0x29, 0x94, 0xff, 0xf3, 0x98, 0x25, 0x4e, 0x3d, 0x56, 0xeb, 0x80, 0x8c, 0xe7, 0x5a, 0x31, 0xbc, 0xd7, 0x6a, 0x1, 0xd, 0x66, 0xdb, 0xb0, 0xc3, 0xa8, 0x15, 0x7e, 0x72, 0x19, 0xa4, 0xcf, 0x7c, 0x17, 0xaa, 0xc1, 0xcd, 0xa6, 0x1b, 0x70, 0x3, 0x68, 0xd5, 0xbe, 0xb2, 0xd9, 0x64, 0xf, 0x82, 0xe9, 0x54, 0x3f, 0x33, 0x58, 0xe5, 0x8e, 0xfd, 0x96, 0x2b, 0x40, 0x4c, 0x27, 0x9a, 0xf1, 0x9d, 0xf6, 0x4b, 0x20, 0x2c, 0x47, 0xfa, 0x91, 0xe2, 0x89, 0x34, 0x5f, 0x53, 0x38, 0x85, 0xee, 0x63, 0x8, 0xb5, 0xde, 0xd2, 0xb9, 0x4, 0x6f, 0x1c, 0x77, 0xca, 0xa1, 0xad, 0xc6, 0x7b, 0x10},
- {0x0, 0x6c, 0xd8, 0xb4, 0xad, 0xc1, 0x75, 0x19, 0x47, 0x2b, 0x9f, 0xf3, 0xea, 0x86, 0x32, 0x5e, 0x8e, 0xe2, 0x56, 0x3a, 0x23, 0x4f, 0xfb, 0x97, 0xc9, 0xa5, 0x11, 0x7d, 0x64, 0x8, 0xbc, 0xd0, 0x1, 0x6d, 0xd9, 0xb5, 0xac, 0xc0, 0x74, 0x18, 0x46, 0x2a, 0x9e, 0xf2, 0xeb, 0x87, 0x33, 0x5f, 0x8f, 0xe3, 0x57, 0x3b, 0x22, 0x4e, 0xfa, 0x96, 0xc8, 0xa4, 0x10, 0x7c, 0x65, 0x9, 0xbd, 0xd1, 0x2, 0x6e, 0xda, 0xb6, 0xaf, 0xc3, 0x77, 0x1b, 0x45, 0x29, 0x9d, 0xf1, 0xe8, 0x84, 0x30, 0x5c, 0x8c, 0xe0, 0x54, 0x38, 0x21, 0x4d, 0xf9, 0x95, 0xcb, 0xa7, 0x13, 0x7f, 0x66, 0xa, 0xbe, 0xd2, 0x3, 0x6f, 0xdb, 0xb7, 0xae, 0xc2, 0x76, 0x1a, 0x44, 0x28, 0x9c, 0xf0, 0xe9, 0x85, 0x31, 0x5d, 0x8d, 0xe1, 0x55, 0x39, 0x20, 0x4c, 0xf8, 0x94, 0xca, 0xa6, 0x12, 0x7e, 0x67, 0xb, 0xbf, 0xd3, 0x4, 0x68, 0xdc, 0xb0, 0xa9, 0xc5, 0x71, 0x1d, 0x43, 0x2f, 0x9b, 0xf7, 0xee, 0x82, 0x36, 0x5a, 0x8a, 0xe6, 0x52, 0x3e, 0x27, 0x4b, 0xff, 0x93, 0xcd, 0xa1, 0x15, 0x79, 0x60, 0xc, 0xb8, 0xd4, 0x5, 0x69, 0xdd, 0xb1, 0xa8, 0xc4, 0x70, 0x1c, 0x42, 0x2e, 0x9a, 0xf6, 0xef, 0x83, 0x37, 0x5b, 0x8b, 0xe7, 0x53, 0x3f, 0x26, 0x4a, 0xfe, 0x92, 0xcc, 0xa0, 0x14, 0x78, 0x61, 0xd, 0xb9, 0xd5, 0x6, 0x6a, 0xde, 0xb2, 0xab, 0xc7, 0x73, 0x1f, 0x41, 0x2d, 0x99, 0xf5, 0xec, 0x80, 0x34, 0x58, 0x88, 0xe4, 0x50, 0x3c, 0x25, 0x49, 0xfd, 0x91, 0xcf, 0xa3, 0x17, 0x7b, 0x62, 0xe, 0xba, 0xd6, 0x7, 0x6b, 0xdf, 0xb3, 0xaa, 0xc6, 0x72, 0x1e, 0x40, 0x2c, 0x98, 0xf4, 0xed, 0x81, 0x35, 0x59, 0x89, 0xe5, 0x51, 0x3d, 0x24, 0x48, 0xfc, 0x90, 0xce, 0xa2, 0x16, 0x7a, 0x63, 0xf, 0xbb, 0xd7},
- {0x0, 0x6d, 0xda, 0xb7, 0xa9, 0xc4, 0x73, 0x1e, 0x4f, 0x22, 0x95, 0xf8, 0xe6, 0x8b, 0x3c, 0x51, 0x9e, 0xf3, 0x44, 0x29, 0x37, 0x5a, 0xed, 0x80, 0xd1, 0xbc, 0xb, 0x66, 0x78, 0x15, 0xa2, 0xcf, 0x21, 0x4c, 0xfb, 0x96, 0x88, 0xe5, 0x52, 0x3f, 0x6e, 0x3, 0xb4, 0xd9, 0xc7, 0xaa, 0x1d, 0x70, 0xbf, 0xd2, 0x65, 0x8, 0x16, 0x7b, 0xcc, 0xa1, 0xf0, 0x9d, 0x2a, 0x47, 0x59, 0x34, 0x83, 0xee, 0x42, 0x2f, 0x98, 0xf5, 0xeb, 0x86, 0x31, 0x5c, 0xd, 0x60, 0xd7, 0xba, 0xa4, 0xc9, 0x7e, 0x13, 0xdc, 0xb1, 0x6, 0x6b, 0x75, 0x18, 0xaf, 0xc2, 0x93, 0xfe, 0x49, 0x24, 0x3a, 0x57, 0xe0, 0x8d, 0x63, 0xe, 0xb9, 0xd4, 0xca, 0xa7, 0x10, 0x7d, 0x2c, 0x41, 0xf6, 0x9b, 0x85, 0xe8, 0x5f, 0x32, 0xfd, 0x90, 0x27, 0x4a, 0x54, 0x39, 0x8e, 0xe3, 0xb2, 0xdf, 0x68, 0x5, 0x1b, 0x76, 0xc1, 0xac, 0x84, 0xe9, 0x5e, 0x33, 0x2d, 0x40, 0xf7, 0x9a, 0xcb, 0xa6, 0x11, 0x7c, 0x62, 0xf, 0xb8, 0xd5, 0x1a, 0x77, 0xc0, 0xad, 0xb3, 0xde, 0x69, 0x4, 0x55, 0x38, 0x8f, 0xe2, 0xfc, 0x91, 0x26, 0x4b, 0xa5, 0xc8, 0x7f, 0x12, 0xc, 0x61, 0xd6, 0xbb, 0xea, 0x87, 0x30, 0x5d, 0x43, 0x2e, 0x99, 0xf4, 0x3b, 0x56, 0xe1, 0x8c, 0x92, 0xff, 0x48, 0x25, 0x74, 0x19, 0xae, 0xc3, 0xdd, 0xb0, 0x7, 0x6a, 0xc6, 0xab, 0x1c, 0x71, 0x6f, 0x2, 0xb5, 0xd8, 0x89, 0xe4, 0x53, 0x3e, 0x20, 0x4d, 0xfa, 0x97, 0x58, 0x35, 0x82, 0xef, 0xf1, 0x9c, 0x2b, 0x46, 0x17, 0x7a, 0xcd, 0xa0, 0xbe, 0xd3, 0x64, 0x9, 0xe7, 0x8a, 0x3d, 0x50, 0x4e, 0x23, 0x94, 0xf9, 0xa8, 0xc5, 0x72, 0x1f, 0x1, 0x6c, 0xdb, 0xb6, 0x79, 0x14, 0xa3, 0xce, 0xd0, 0xbd, 0xa, 0x67, 0x36, 0x5b, 0xec, 0x81, 0x9f, 0xf2, 0x45, 0x28},
- {0x0, 0x6e, 0xdc, 0xb2, 0xa5, 0xcb, 0x79, 0x17, 0x57, 0x39, 0x8b, 0xe5, 0xf2, 0x9c, 0x2e, 0x40, 0xae, 0xc0, 0x72, 0x1c, 0xb, 0x65, 0xd7, 0xb9, 0xf9, 0x97, 0x25, 0x4b, 0x5c, 0x32, 0x80, 0xee, 0x41, 0x2f, 0x9d, 0xf3, 0xe4, 0x8a, 0x38, 0x56, 0x16, 0x78, 0xca, 0xa4, 0xb3, 0xdd, 0x6f, 0x1, 0xef, 0x81, 0x33, 0x5d, 0x4a, 0x24, 0x96, 0xf8, 0xb8, 0xd6, 0x64, 0xa, 0x1d, 0x73, 0xc1, 0xaf, 0x82, 0xec, 0x5e, 0x30, 0x27, 0x49, 0xfb, 0x95, 0xd5, 0xbb, 0x9, 0x67, 0x70, 0x1e, 0xac, 0xc2, 0x2c, 0x42, 0xf0, 0x9e, 0x89, 0xe7, 0x55, 0x3b, 0x7b, 0x15, 0xa7, 0xc9, 0xde, 0xb0, 0x2, 0x6c, 0xc3, 0xad, 0x1f, 0x71, 0x66, 0x8, 0xba, 0xd4, 0x94, 0xfa, 0x48, 0x26, 0x31, 0x5f, 0xed, 0x83, 0x6d, 0x3, 0xb1, 0xdf, 0xc8, 0xa6, 0x14, 0x7a, 0x3a, 0x54, 0xe6, 0x88, 0x9f, 0xf1, 0x43, 0x2d, 0x19, 0x77, 0xc5, 0xab, 0xbc, 0xd2, 0x60, 0xe, 0x4e, 0x20, 0x92, 0xfc, 0xeb, 0x85, 0x37, 0x59, 0xb7, 0xd9, 0x6b, 0x5, 0x12, 0x7c, 0xce, 0xa0, 0xe0, 0x8e, 0x3c, 0x52, 0x45, 0x2b, 0x99, 0xf7, 0x58, 0x36, 0x84, 0xea, 0xfd, 0x93, 0x21, 0x4f, 0xf, 0x61, 0xd3, 0xbd, 0xaa, 0xc4, 0x76, 0x18, 0xf6, 0x98, 0x2a, 0x44, 0x53, 0x3d, 0x8f, 0xe1, 0xa1, 0xcf, 0x7d, 0x13, 0x4, 0x6a, 0xd8, 0xb6, 0x9b, 0xf5, 0x47, 0x29, 0x3e, 0x50, 0xe2, 0x8c, 0xcc, 0xa2, 0x10, 0x7e, 0x69, 0x7, 0xb5, 0xdb, 0x35, 0x5b, 0xe9, 0x87, 0x90, 0xfe, 0x4c, 0x22, 0x62, 0xc, 0xbe, 0xd0, 0xc7, 0xa9, 0x1b, 0x75, 0xda, 0xb4, 0x6, 0x68, 0x7f, 0x11, 0xa3, 0xcd, 0x8d, 0xe3, 0x51, 0x3f, 0x28, 0x46, 0xf4, 0x9a, 0x74, 0x1a, 0xa8, 0xc6, 0xd1, 0xbf, 0xd, 0x63, 0x23, 0x4d, 0xff, 0x91, 0x86, 0xe8, 0x5a, 0x34},
- {0x0, 0x6f, 0xde, 0xb1, 0xa1, 0xce, 0x7f, 0x10, 0x5f, 0x30, 0x81, 0xee, 0xfe, 0x91, 0x20, 0x4f, 0xbe, 0xd1, 0x60, 0xf, 0x1f, 0x70, 0xc1, 0xae, 0xe1, 0x8e, 0x3f, 0x50, 0x40, 0x2f, 0x9e, 0xf1, 0x61, 0xe, 0xbf, 0xd0, 0xc0, 0xaf, 0x1e, 0x71, 0x3e, 0x51, 0xe0, 0x8f, 0x9f, 0xf0, 0x41, 0x2e, 0xdf, 0xb0, 0x1, 0x6e, 0x7e, 0x11, 0xa0, 0xcf, 0x80, 0xef, 0x5e, 0x31, 0x21, 0x4e, 0xff, 0x90, 0xc2, 0xad, 0x1c, 0x73, 0x63, 0xc, 0xbd, 0xd2, 0x9d, 0xf2, 0x43, 0x2c, 0x3c, 0x53, 0xe2, 0x8d, 0x7c, 0x13, 0xa2, 0xcd, 0xdd, 0xb2, 0x3, 0x6c, 0x23, 0x4c, 0xfd, 0x92, 0x82, 0xed, 0x5c, 0x33, 0xa3, 0xcc, 0x7d, 0x12, 0x2, 0x6d, 0xdc, 0xb3, 0xfc, 0x93, 0x22, 0x4d, 0x5d, 0x32, 0x83, 0xec, 0x1d, 0x72, 0xc3, 0xac, 0xbc, 0xd3, 0x62, 0xd, 0x42, 0x2d, 0x9c, 0xf3, 0xe3, 0x8c, 0x3d, 0x52, 0x99, 0xf6, 0x47, 0x28, 0x38, 0x57, 0xe6, 0x89, 0xc6, 0xa9, 0x18, 0x77, 0x67, 0x8, 0xb9, 0xd6, 0x27, 0x48, 0xf9, 0x96, 0x86, 0xe9, 0x58, 0x37, 0x78, 0x17, 0xa6, 0xc9, 0xd9, 0xb6, 0x7, 0x68, 0xf8, 0x97, 0x26, 0x49, 0x59, 0x36, 0x87, 0xe8, 0xa7, 0xc8, 0x79, 0x16, 0x6, 0x69, 0xd8, 0xb7, 0x46, 0x29, 0x98, 0xf7, 0xe7, 0x88, 0x39, 0x56, 0x19, 0x76, 0xc7, 0xa8, 0xb8, 0xd7, 0x66, 0x9, 0x5b, 0x34, 0x85, 0xea, 0xfa, 0x95, 0x24, 0x4b, 0x4, 0x6b, 0xda, 0xb5, 0xa5, 0xca, 0x7b, 0x14, 0xe5, 0x8a, 0x3b, 0x54, 0x44, 0x2b, 0x9a, 0xf5, 0xba, 0xd5, 0x64, 0xb, 0x1b, 0x74, 0xc5, 0xaa, 0x3a, 0x55, 0xe4, 0x8b, 0x9b, 0xf4, 0x45, 0x2a, 0x65, 0xa, 0xbb, 0xd4, 0xc4, 0xab, 0x1a, 0x75, 0x84, 0xeb, 0x5a, 0x35, 0x25, 0x4a, 0xfb, 0x94, 0xdb, 0xb4, 0x5, 0x6a, 0x7a, 0x15, 0xa4, 0xcb},
- {0x0, 0x70, 0xe0, 0x90, 0xdd, 0xad, 0x3d, 0x4d, 0xa7, 0xd7, 0x47, 0x37, 0x7a, 0xa, 0x9a, 0xea, 0x53, 0x23, 0xb3, 0xc3, 0x8e, 0xfe, 0x6e, 0x1e, 0xf4, 0x84, 0x14, 0x64, 0x29, 0x59, 0xc9, 0xb9, 0xa6, 0xd6, 0x46, 0x36, 0x7b, 0xb, 0x9b, 0xeb, 0x1, 0x71, 0xe1, 0x91, 0xdc, 0xac, 0x3c, 0x4c, 0xf5, 0x85, 0x15, 0x65, 0x28, 0x58, 0xc8, 0xb8, 0x52, 0x22, 0xb2, 0xc2, 0x8f, 0xff, 0x6f, 0x1f, 0x51, 0x21, 0xb1, 0xc1, 0x8c, 0xfc, 0x6c, 0x1c, 0xf6, 0x86, 0x16, 0x66, 0x2b, 0x5b, 0xcb, 0xbb, 0x2, 0x72, 0xe2, 0x92, 0xdf, 0xaf, 0x3f, 0x4f, 0xa5, 0xd5, 0x45, 0x35, 0x78, 0x8, 0x98, 0xe8, 0xf7, 0x87, 0x17, 0x67, 0x2a, 0x5a, 0xca, 0xba, 0x50, 0x20, 0xb0, 0xc0, 0x8d, 0xfd, 0x6d, 0x1d, 0xa4, 0xd4, 0x44, 0x34, 0x79, 0x9, 0x99, 0xe9, 0x3, 0x73, 0xe3, 0x93, 0xde, 0xae, 0x3e, 0x4e, 0xa2, 0xd2, 0x42, 0x32, 0x7f, 0xf, 0x9f, 0xef, 0x5, 0x75, 0xe5, 0x95, 0xd8, 0xa8, 0x38, 0x48, 0xf1, 0x81, 0x11, 0x61, 0x2c, 0x5c, 0xcc, 0xbc, 0x56, 0x26, 0xb6, 0xc6, 0x8b, 0xfb, 0x6b, 0x1b, 0x4, 0x74, 0xe4, 0x94, 0xd9, 0xa9, 0x39, 0x49, 0xa3, 0xd3, 0x43, 0x33, 0x7e, 0xe, 0x9e, 0xee, 0x57, 0x27, 0xb7, 0xc7, 0x8a, 0xfa, 0x6a, 0x1a, 0xf0, 0x80, 0x10, 0x60, 0x2d, 0x5d, 0xcd, 0xbd, 0xf3, 0x83, 0x13, 0x63, 0x2e, 0x5e, 0xce, 0xbe, 0x54, 0x24, 0xb4, 0xc4, 0x89, 0xf9, 0x69, 0x19, 0xa0, 0xd0, 0x40, 0x30, 0x7d, 0xd, 0x9d, 0xed, 0x7, 0x77, 0xe7, 0x97, 0xda, 0xaa, 0x3a, 0x4a, 0x55, 0x25, 0xb5, 0xc5, 0x88, 0xf8, 0x68, 0x18, 0xf2, 0x82, 0x12, 0x62, 0x2f, 0x5f, 0xcf, 0xbf, 0x6, 0x76, 0xe6, 0x96, 0xdb, 0xab, 0x3b, 0x4b, 0xa1, 0xd1, 0x41, 0x31, 0x7c, 0xc, 0x9c, 0xec},
- {0x0, 0x71, 0xe2, 0x93, 0xd9, 0xa8, 0x3b, 0x4a, 0xaf, 0xde, 0x4d, 0x3c, 0x76, 0x7, 0x94, 0xe5, 0x43, 0x32, 0xa1, 0xd0, 0x9a, 0xeb, 0x78, 0x9, 0xec, 0x9d, 0xe, 0x7f, 0x35, 0x44, 0xd7, 0xa6, 0x86, 0xf7, 0x64, 0x15, 0x5f, 0x2e, 0xbd, 0xcc, 0x29, 0x58, 0xcb, 0xba, 0xf0, 0x81, 0x12, 0x63, 0xc5, 0xb4, 0x27, 0x56, 0x1c, 0x6d, 0xfe, 0x8f, 0x6a, 0x1b, 0x88, 0xf9, 0xb3, 0xc2, 0x51, 0x20, 0x11, 0x60, 0xf3, 0x82, 0xc8, 0xb9, 0x2a, 0x5b, 0xbe, 0xcf, 0x5c, 0x2d, 0x67, 0x16, 0x85, 0xf4, 0x52, 0x23, 0xb0, 0xc1, 0x8b, 0xfa, 0x69, 0x18, 0xfd, 0x8c, 0x1f, 0x6e, 0x24, 0x55, 0xc6, 0xb7, 0x97, 0xe6, 0x75, 0x4, 0x4e, 0x3f, 0xac, 0xdd, 0x38, 0x49, 0xda, 0xab, 0xe1, 0x90, 0x3, 0x72, 0xd4, 0xa5, 0x36, 0x47, 0xd, 0x7c, 0xef, 0x9e, 0x7b, 0xa, 0x99, 0xe8, 0xa2, 0xd3, 0x40, 0x31, 0x22, 0x53, 0xc0, 0xb1, 0xfb, 0x8a, 0x19, 0x68, 0x8d, 0xfc, 0x6f, 0x1e, 0x54, 0x25, 0xb6, 0xc7, 0x61, 0x10, 0x83, 0xf2, 0xb8, 0xc9, 0x5a, 0x2b, 0xce, 0xbf, 0x2c, 0x5d, 0x17, 0x66, 0xf5, 0x84, 0xa4, 0xd5, 0x46, 0x37, 0x7d, 0xc, 0x9f, 0xee, 0xb, 0x7a, 0xe9, 0x98, 0xd2, 0xa3, 0x30, 0x41, 0xe7, 0x96, 0x5, 0x74, 0x3e, 0x4f, 0xdc, 0xad, 0x48, 0x39, 0xaa, 0xdb, 0x91, 0xe0, 0x73, 0x2, 0x33, 0x42, 0xd1, 0xa0, 0xea, 0x9b, 0x8, 0x79, 0x9c, 0xed, 0x7e, 0xf, 0x45, 0x34, 0xa7, 0xd6, 0x70, 0x1, 0x92, 0xe3, 0xa9, 0xd8, 0x4b, 0x3a, 0xdf, 0xae, 0x3d, 0x4c, 0x6, 0x77, 0xe4, 0x95, 0xb5, 0xc4, 0x57, 0x26, 0x6c, 0x1d, 0x8e, 0xff, 0x1a, 0x6b, 0xf8, 0x89, 0xc3, 0xb2, 0x21, 0x50, 0xf6, 0x87, 0x14, 0x65, 0x2f, 0x5e, 0xcd, 0xbc, 0x59, 0x28, 0xbb, 0xca, 0x80, 0xf1, 0x62, 0x13},
- {0x0, 0x72, 0xe4, 0x96, 0xd5, 0xa7, 0x31, 0x43, 0xb7, 0xc5, 0x53, 0x21, 0x62, 0x10, 0x86, 0xf4, 0x73, 0x1, 0x97, 0xe5, 0xa6, 0xd4, 0x42, 0x30, 0xc4, 0xb6, 0x20, 0x52, 0x11, 0x63, 0xf5, 0x87, 0xe6, 0x94, 0x2, 0x70, 0x33, 0x41, 0xd7, 0xa5, 0x51, 0x23, 0xb5, 0xc7, 0x84, 0xf6, 0x60, 0x12, 0x95, 0xe7, 0x71, 0x3, 0x40, 0x32, 0xa4, 0xd6, 0x22, 0x50, 0xc6, 0xb4, 0xf7, 0x85, 0x13, 0x61, 0xd1, 0xa3, 0x35, 0x47, 0x4, 0x76, 0xe0, 0x92, 0x66, 0x14, 0x82, 0xf0, 0xb3, 0xc1, 0x57, 0x25, 0xa2, 0xd0, 0x46, 0x34, 0x77, 0x5, 0x93, 0xe1, 0x15, 0x67, 0xf1, 0x83, 0xc0, 0xb2, 0x24, 0x56, 0x37, 0x45, 0xd3, 0xa1, 0xe2, 0x90, 0x6, 0x74, 0x80, 0xf2, 0x64, 0x16, 0x55, 0x27, 0xb1, 0xc3, 0x44, 0x36, 0xa0, 0xd2, 0x91, 0xe3, 0x75, 0x7, 0xf3, 0x81, 0x17, 0x65, 0x26, 0x54, 0xc2, 0xb0, 0xbf, 0xcd, 0x5b, 0x29, 0x6a, 0x18, 0x8e, 0xfc, 0x8, 0x7a, 0xec, 0x9e, 0xdd, 0xaf, 0x39, 0x4b, 0xcc, 0xbe, 0x28, 0x5a, 0x19, 0x6b, 0xfd, 0x8f, 0x7b, 0x9, 0x9f, 0xed, 0xae, 0xdc, 0x4a, 0x38, 0x59, 0x2b, 0xbd, 0xcf, 0x8c, 0xfe, 0x68, 0x1a, 0xee, 0x9c, 0xa, 0x78, 0x3b, 0x49, 0xdf, 0xad, 0x2a, 0x58, 0xce, 0xbc, 0xff, 0x8d, 0x1b, 0x69, 0x9d, 0xef, 0x79, 0xb, 0x48, 0x3a, 0xac, 0xde, 0x6e, 0x1c, 0x8a, 0xf8, 0xbb, 0xc9, 0x5f, 0x2d, 0xd9, 0xab, 0x3d, 0x4f, 0xc, 0x7e, 0xe8, 0x9a, 0x1d, 0x6f, 0xf9, 0x8b, 0xc8, 0xba, 0x2c, 0x5e, 0xaa, 0xd8, 0x4e, 0x3c, 0x7f, 0xd, 0x9b, 0xe9, 0x88, 0xfa, 0x6c, 0x1e, 0x5d, 0x2f, 0xb9, 0xcb, 0x3f, 0x4d, 0xdb, 0xa9, 0xea, 0x98, 0xe, 0x7c, 0xfb, 0x89, 0x1f, 0x6d, 0x2e, 0x5c, 0xca, 0xb8, 0x4c, 0x3e, 0xa8, 0xda, 0x99, 0xeb, 0x7d, 0xf},
- {0x0, 0x73, 0xe6, 0x95, 0xd1, 0xa2, 0x37, 0x44, 0xbf, 0xcc, 0x59, 0x2a, 0x6e, 0x1d, 0x88, 0xfb, 0x63, 0x10, 0x85, 0xf6, 0xb2, 0xc1, 0x54, 0x27, 0xdc, 0xaf, 0x3a, 0x49, 0xd, 0x7e, 0xeb, 0x98, 0xc6, 0xb5, 0x20, 0x53, 0x17, 0x64, 0xf1, 0x82, 0x79, 0xa, 0x9f, 0xec, 0xa8, 0xdb, 0x4e, 0x3d, 0xa5, 0xd6, 0x43, 0x30, 0x74, 0x7, 0x92, 0xe1, 0x1a, 0x69, 0xfc, 0x8f, 0xcb, 0xb8, 0x2d, 0x5e, 0x91, 0xe2, 0x77, 0x4, 0x40, 0x33, 0xa6, 0xd5, 0x2e, 0x5d, 0xc8, 0xbb, 0xff, 0x8c, 0x19, 0x6a, 0xf2, 0x81, 0x14, 0x67, 0x23, 0x50, 0xc5, 0xb6, 0x4d, 0x3e, 0xab, 0xd8, 0x9c, 0xef, 0x7a, 0x9, 0x57, 0x24, 0xb1, 0xc2, 0x86, 0xf5, 0x60, 0x13, 0xe8, 0x9b, 0xe, 0x7d, 0x39, 0x4a, 0xdf, 0xac, 0x34, 0x47, 0xd2, 0xa1, 0xe5, 0x96, 0x3, 0x70, 0x8b, 0xf8, 0x6d, 0x1e, 0x5a, 0x29, 0xbc, 0xcf, 0x3f, 0x4c, 0xd9, 0xaa, 0xee, 0x9d, 0x8, 0x7b, 0x80, 0xf3, 0x66, 0x15, 0x51, 0x22, 0xb7, 0xc4, 0x5c, 0x2f, 0xba, 0xc9, 0x8d, 0xfe, 0x6b, 0x18, 0xe3, 0x90, 0x5, 0x76, 0x32, 0x41, 0xd4, 0xa7, 0xf9, 0x8a, 0x1f, 0x6c, 0x28, 0x5b, 0xce, 0xbd, 0x46, 0x35, 0xa0, 0xd3, 0x97, 0xe4, 0x71, 0x2, 0x9a, 0xe9, 0x7c, 0xf, 0x4b, 0x38, 0xad, 0xde, 0x25, 0x56, 0xc3, 0xb0, 0xf4, 0x87, 0x12, 0x61, 0xae, 0xdd, 0x48, 0x3b, 0x7f, 0xc, 0x99, 0xea, 0x11, 0x62, 0xf7, 0x84, 0xc0, 0xb3, 0x26, 0x55, 0xcd, 0xbe, 0x2b, 0x58, 0x1c, 0x6f, 0xfa, 0x89, 0x72, 0x1, 0x94, 0xe7, 0xa3, 0xd0, 0x45, 0x36, 0x68, 0x1b, 0x8e, 0xfd, 0xb9, 0xca, 0x5f, 0x2c, 0xd7, 0xa4, 0x31, 0x42, 0x6, 0x75, 0xe0, 0x93, 0xb, 0x78, 0xed, 0x9e, 0xda, 0xa9, 0x3c, 0x4f, 0xb4, 0xc7, 0x52, 0x21, 0x65, 0x16, 0x83, 0xf0},
- {0x0, 0x74, 0xe8, 0x9c, 0xcd, 0xb9, 0x25, 0x51, 0x87, 0xf3, 0x6f, 0x1b, 0x4a, 0x3e, 0xa2, 0xd6, 0x13, 0x67, 0xfb, 0x8f, 0xde, 0xaa, 0x36, 0x42, 0x94, 0xe0, 0x7c, 0x8, 0x59, 0x2d, 0xb1, 0xc5, 0x26, 0x52, 0xce, 0xba, 0xeb, 0x9f, 0x3, 0x77, 0xa1, 0xd5, 0x49, 0x3d, 0x6c, 0x18, 0x84, 0xf0, 0x35, 0x41, 0xdd, 0xa9, 0xf8, 0x8c, 0x10, 0x64, 0xb2, 0xc6, 0x5a, 0x2e, 0x7f, 0xb, 0x97, 0xe3, 0x4c, 0x38, 0xa4, 0xd0, 0x81, 0xf5, 0x69, 0x1d, 0xcb, 0xbf, 0x23, 0x57, 0x6, 0x72, 0xee, 0x9a, 0x5f, 0x2b, 0xb7, 0xc3, 0x92, 0xe6, 0x7a, 0xe, 0xd8, 0xac, 0x30, 0x44, 0x15, 0x61, 0xfd, 0x89, 0x6a, 0x1e, 0x82, 0xf6, 0xa7, 0xd3, 0x4f, 0x3b, 0xed, 0x99, 0x5, 0x71, 0x20, 0x54, 0xc8, 0xbc, 0x79, 0xd, 0x91, 0xe5, 0xb4, 0xc0, 0x5c, 0x28, 0xfe, 0x8a, 0x16, 0x62, 0x33, 0x47, 0xdb, 0xaf, 0x98, 0xec, 0x70, 0x4, 0x55, 0x21, 0xbd, 0xc9, 0x1f, 0x6b, 0xf7, 0x83, 0xd2, 0xa6, 0x3a, 0x4e, 0x8b, 0xff, 0x63, 0x17, 0x46, 0x32, 0xae, 0xda, 0xc, 0x78, 0xe4, 0x90, 0xc1, 0xb5, 0x29, 0x5d, 0xbe, 0xca, 0x56, 0x22, 0x73, 0x7, 0x9b, 0xef, 0x39, 0x4d, 0xd1, 0xa5, 0xf4, 0x80, 0x1c, 0x68, 0xad, 0xd9, 0x45, 0x31, 0x60, 0x14, 0x88, 0xfc, 0x2a, 0x5e, 0xc2, 0xb6, 0xe7, 0x93, 0xf, 0x7b, 0xd4, 0xa0, 0x3c, 0x48, 0x19, 0x6d, 0xf1, 0x85, 0x53, 0x27, 0xbb, 0xcf, 0x9e, 0xea, 0x76, 0x2, 0xc7, 0xb3, 0x2f, 0x5b, 0xa, 0x7e, 0xe2, 0x96, 0x40, 0x34, 0xa8, 0xdc, 0x8d, 0xf9, 0x65, 0x11, 0xf2, 0x86, 0x1a, 0x6e, 0x3f, 0x4b, 0xd7, 0xa3, 0x75, 0x1, 0x9d, 0xe9, 0xb8, 0xcc, 0x50, 0x24, 0xe1, 0x95, 0x9, 0x7d, 0x2c, 0x58, 0xc4, 0xb0, 0x66, 0x12, 0x8e, 0xfa, 0xab, 0xdf, 0x43, 0x37},
- {0x0, 0x75, 0xea, 0x9f, 0xc9, 0xbc, 0x23, 0x56, 0x8f, 0xfa, 0x65, 0x10, 0x46, 0x33, 0xac, 0xd9, 0x3, 0x76, 0xe9, 0x9c, 0xca, 0xbf, 0x20, 0x55, 0x8c, 0xf9, 0x66, 0x13, 0x45, 0x30, 0xaf, 0xda, 0x6, 0x73, 0xec, 0x99, 0xcf, 0xba, 0x25, 0x50, 0x89, 0xfc, 0x63, 0x16, 0x40, 0x35, 0xaa, 0xdf, 0x5, 0x70, 0xef, 0x9a, 0xcc, 0xb9, 0x26, 0x53, 0x8a, 0xff, 0x60, 0x15, 0x43, 0x36, 0xa9, 0xdc, 0xc, 0x79, 0xe6, 0x93, 0xc5, 0xb0, 0x2f, 0x5a, 0x83, 0xf6, 0x69, 0x1c, 0x4a, 0x3f, 0xa0, 0xd5, 0xf, 0x7a, 0xe5, 0x90, 0xc6, 0xb3, 0x2c, 0x59, 0x80, 0xf5, 0x6a, 0x1f, 0x49, 0x3c, 0xa3, 0xd6, 0xa, 0x7f, 0xe0, 0x95, 0xc3, 0xb6, 0x29, 0x5c, 0x85, 0xf0, 0x6f, 0x1a, 0x4c, 0x39, 0xa6, 0xd3, 0x9, 0x7c, 0xe3, 0x96, 0xc0, 0xb5, 0x2a, 0x5f, 0x86, 0xf3, 0x6c, 0x19, 0x4f, 0x3a, 0xa5, 0xd0, 0x18, 0x6d, 0xf2, 0x87, 0xd1, 0xa4, 0x3b, 0x4e, 0x97, 0xe2, 0x7d, 0x8, 0x5e, 0x2b, 0xb4, 0xc1, 0x1b, 0x6e, 0xf1, 0x84, 0xd2, 0xa7, 0x38, 0x4d, 0x94, 0xe1, 0x7e, 0xb, 0x5d, 0x28, 0xb7, 0xc2, 0x1e, 0x6b, 0xf4, 0x81, 0xd7, 0xa2, 0x3d, 0x48, 0x91, 0xe4, 0x7b, 0xe, 0x58, 0x2d, 0xb2, 0xc7, 0x1d, 0x68, 0xf7, 0x82, 0xd4, 0xa1, 0x3e, 0x4b, 0x92, 0xe7, 0x78, 0xd, 0x5b, 0x2e, 0xb1, 0xc4, 0x14, 0x61, 0xfe, 0x8b, 0xdd, 0xa8, 0x37, 0x42, 0x9b, 0xee, 0x71, 0x4, 0x52, 0x27, 0xb8, 0xcd, 0x17, 0x62, 0xfd, 0x88, 0xde, 0xab, 0x34, 0x41, 0x98, 0xed, 0x72, 0x7, 0x51, 0x24, 0xbb, 0xce, 0x12, 0x67, 0xf8, 0x8d, 0xdb, 0xae, 0x31, 0x44, 0x9d, 0xe8, 0x77, 0x2, 0x54, 0x21, 0xbe, 0xcb, 0x11, 0x64, 0xfb, 0x8e, 0xd8, 0xad, 0x32, 0x47, 0x9e, 0xeb, 0x74, 0x1, 0x57, 0x22, 0xbd, 0xc8},
- {0x0, 0x76, 0xec, 0x9a, 0xc5, 0xb3, 0x29, 0x5f, 0x97, 0xe1, 0x7b, 0xd, 0x52, 0x24, 0xbe, 0xc8, 0x33, 0x45, 0xdf, 0xa9, 0xf6, 0x80, 0x1a, 0x6c, 0xa4, 0xd2, 0x48, 0x3e, 0x61, 0x17, 0x8d, 0xfb, 0x66, 0x10, 0x8a, 0xfc, 0xa3, 0xd5, 0x4f, 0x39, 0xf1, 0x87, 0x1d, 0x6b, 0x34, 0x42, 0xd8, 0xae, 0x55, 0x23, 0xb9, 0xcf, 0x90, 0xe6, 0x7c, 0xa, 0xc2, 0xb4, 0x2e, 0x58, 0x7, 0x71, 0xeb, 0x9d, 0xcc, 0xba, 0x20, 0x56, 0x9, 0x7f, 0xe5, 0x93, 0x5b, 0x2d, 0xb7, 0xc1, 0x9e, 0xe8, 0x72, 0x4, 0xff, 0x89, 0x13, 0x65, 0x3a, 0x4c, 0xd6, 0xa0, 0x68, 0x1e, 0x84, 0xf2, 0xad, 0xdb, 0x41, 0x37, 0xaa, 0xdc, 0x46, 0x30, 0x6f, 0x19, 0x83, 0xf5, 0x3d, 0x4b, 0xd1, 0xa7, 0xf8, 0x8e, 0x14, 0x62, 0x99, 0xef, 0x75, 0x3, 0x5c, 0x2a, 0xb0, 0xc6, 0xe, 0x78, 0xe2, 0x94, 0xcb, 0xbd, 0x27, 0x51, 0x85, 0xf3, 0x69, 0x1f, 0x40, 0x36, 0xac, 0xda, 0x12, 0x64, 0xfe, 0x88, 0xd7, 0xa1, 0x3b, 0x4d, 0xb6, 0xc0, 0x5a, 0x2c, 0x73, 0x5, 0x9f, 0xe9, 0x21, 0x57, 0xcd, 0xbb, 0xe4, 0x92, 0x8, 0x7e, 0xe3, 0x95, 0xf, 0x79, 0x26, 0x50, 0xca, 0xbc, 0x74, 0x2, 0x98, 0xee, 0xb1, 0xc7, 0x5d, 0x2b, 0xd0, 0xa6, 0x3c, 0x4a, 0x15, 0x63, 0xf9, 0x8f, 0x47, 0x31, 0xab, 0xdd, 0x82, 0xf4, 0x6e, 0x18, 0x49, 0x3f, 0xa5, 0xd3, 0x8c, 0xfa, 0x60, 0x16, 0xde, 0xa8, 0x32, 0x44, 0x1b, 0x6d, 0xf7, 0x81, 0x7a, 0xc, 0x96, 0xe0, 0xbf, 0xc9, 0x53, 0x25, 0xed, 0x9b, 0x1, 0x77, 0x28, 0x5e, 0xc4, 0xb2, 0x2f, 0x59, 0xc3, 0xb5, 0xea, 0x9c, 0x6, 0x70, 0xb8, 0xce, 0x54, 0x22, 0x7d, 0xb, 0x91, 0xe7, 0x1c, 0x6a, 0xf0, 0x86, 0xd9, 0xaf, 0x35, 0x43, 0x8b, 0xfd, 0x67, 0x11, 0x4e, 0x38, 0xa2, 0xd4},
- {0x0, 0x77, 0xee, 0x99, 0xc1, 0xb6, 0x2f, 0x58, 0x9f, 0xe8, 0x71, 0x6, 0x5e, 0x29, 0xb0, 0xc7, 0x23, 0x54, 0xcd, 0xba, 0xe2, 0x95, 0xc, 0x7b, 0xbc, 0xcb, 0x52, 0x25, 0x7d, 0xa, 0x93, 0xe4, 0x46, 0x31, 0xa8, 0xdf, 0x87, 0xf0, 0x69, 0x1e, 0xd9, 0xae, 0x37, 0x40, 0x18, 0x6f, 0xf6, 0x81, 0x65, 0x12, 0x8b, 0xfc, 0xa4, 0xd3, 0x4a, 0x3d, 0xfa, 0x8d, 0x14, 0x63, 0x3b, 0x4c, 0xd5, 0xa2, 0x8c, 0xfb, 0x62, 0x15, 0x4d, 0x3a, 0xa3, 0xd4, 0x13, 0x64, 0xfd, 0x8a, 0xd2, 0xa5, 0x3c, 0x4b, 0xaf, 0xd8, 0x41, 0x36, 0x6e, 0x19, 0x80, 0xf7, 0x30, 0x47, 0xde, 0xa9, 0xf1, 0x86, 0x1f, 0x68, 0xca, 0xbd, 0x24, 0x53, 0xb, 0x7c, 0xe5, 0x92, 0x55, 0x22, 0xbb, 0xcc, 0x94, 0xe3, 0x7a, 0xd, 0xe9, 0x9e, 0x7, 0x70, 0x28, 0x5f, 0xc6, 0xb1, 0x76, 0x1, 0x98, 0xef, 0xb7, 0xc0, 0x59, 0x2e, 0x5, 0x72, 0xeb, 0x9c, 0xc4, 0xb3, 0x2a, 0x5d, 0x9a, 0xed, 0x74, 0x3, 0x5b, 0x2c, 0xb5, 0xc2, 0x26, 0x51, 0xc8, 0xbf, 0xe7, 0x90, 0x9, 0x7e, 0xb9, 0xce, 0x57, 0x20, 0x78, 0xf, 0x96, 0xe1, 0x43, 0x34, 0xad, 0xda, 0x82, 0xf5, 0x6c, 0x1b, 0xdc, 0xab, 0x32, 0x45, 0x1d, 0x6a, 0xf3, 0x84, 0x60, 0x17, 0x8e, 0xf9, 0xa1, 0xd6, 0x4f, 0x38, 0xff, 0x88, 0x11, 0x66, 0x3e, 0x49, 0xd0, 0xa7, 0x89, 0xfe, 0x67, 0x10, 0x48, 0x3f, 0xa6, 0xd1, 0x16, 0x61, 0xf8, 0x8f, 0xd7, 0xa0, 0x39, 0x4e, 0xaa, 0xdd, 0x44, 0x33, 0x6b, 0x1c, 0x85, 0xf2, 0x35, 0x42, 0xdb, 0xac, 0xf4, 0x83, 0x1a, 0x6d, 0xcf, 0xb8, 0x21, 0x56, 0xe, 0x79, 0xe0, 0x97, 0x50, 0x27, 0xbe, 0xc9, 0x91, 0xe6, 0x7f, 0x8, 0xec, 0x9b, 0x2, 0x75, 0x2d, 0x5a, 0xc3, 0xb4, 0x73, 0x4, 0x9d, 0xea, 0xb2, 0xc5, 0x5c, 0x2b},
- {0x0, 0x78, 0xf0, 0x88, 0xfd, 0x85, 0xd, 0x75, 0xe7, 0x9f, 0x17, 0x6f, 0x1a, 0x62, 0xea, 0x92, 0xd3, 0xab, 0x23, 0x5b, 0x2e, 0x56, 0xde, 0xa6, 0x34, 0x4c, 0xc4, 0xbc, 0xc9, 0xb1, 0x39, 0x41, 0xbb, 0xc3, 0x4b, 0x33, 0x46, 0x3e, 0xb6, 0xce, 0x5c, 0x24, 0xac, 0xd4, 0xa1, 0xd9, 0x51, 0x29, 0x68, 0x10, 0x98, 0xe0, 0x95, 0xed, 0x65, 0x1d, 0x8f, 0xf7, 0x7f, 0x7, 0x72, 0xa, 0x82, 0xfa, 0x6b, 0x13, 0x9b, 0xe3, 0x96, 0xee, 0x66, 0x1e, 0x8c, 0xf4, 0x7c, 0x4, 0x71, 0x9, 0x81, 0xf9, 0xb8, 0xc0, 0x48, 0x30, 0x45, 0x3d, 0xb5, 0xcd, 0x5f, 0x27, 0xaf, 0xd7, 0xa2, 0xda, 0x52, 0x2a, 0xd0, 0xa8, 0x20, 0x58, 0x2d, 0x55, 0xdd, 0xa5, 0x37, 0x4f, 0xc7, 0xbf, 0xca, 0xb2, 0x3a, 0x42, 0x3, 0x7b, 0xf3, 0x8b, 0xfe, 0x86, 0xe, 0x76, 0xe4, 0x9c, 0x14, 0x6c, 0x19, 0x61, 0xe9, 0x91, 0xd6, 0xae, 0x26, 0x5e, 0x2b, 0x53, 0xdb, 0xa3, 0x31, 0x49, 0xc1, 0xb9, 0xcc, 0xb4, 0x3c, 0x44, 0x5, 0x7d, 0xf5, 0x8d, 0xf8, 0x80, 0x8, 0x70, 0xe2, 0x9a, 0x12, 0x6a, 0x1f, 0x67, 0xef, 0x97, 0x6d, 0x15, 0x9d, 0xe5, 0x90, 0xe8, 0x60, 0x18, 0x8a, 0xf2, 0x7a, 0x2, 0x77, 0xf, 0x87, 0xff, 0xbe, 0xc6, 0x4e, 0x36, 0x43, 0x3b, 0xb3, 0xcb, 0x59, 0x21, 0xa9, 0xd1, 0xa4, 0xdc, 0x54, 0x2c, 0xbd, 0xc5, 0x4d, 0x35, 0x40, 0x38, 0xb0, 0xc8, 0x5a, 0x22, 0xaa, 0xd2, 0xa7, 0xdf, 0x57, 0x2f, 0x6e, 0x16, 0x9e, 0xe6, 0x93, 0xeb, 0x63, 0x1b, 0x89, 0xf1, 0x79, 0x1, 0x74, 0xc, 0x84, 0xfc, 0x6, 0x7e, 0xf6, 0x8e, 0xfb, 0x83, 0xb, 0x73, 0xe1, 0x99, 0x11, 0x69, 0x1c, 0x64, 0xec, 0x94, 0xd5, 0xad, 0x25, 0x5d, 0x28, 0x50, 0xd8, 0xa0, 0x32, 0x4a, 0xc2, 0xba, 0xcf, 0xb7, 0x3f, 0x47},
- {0x0, 0x79, 0xf2, 0x8b, 0xf9, 0x80, 0xb, 0x72, 0xef, 0x96, 0x1d, 0x64, 0x16, 0x6f, 0xe4, 0x9d, 0xc3, 0xba, 0x31, 0x48, 0x3a, 0x43, 0xc8, 0xb1, 0x2c, 0x55, 0xde, 0xa7, 0xd5, 0xac, 0x27, 0x5e, 0x9b, 0xe2, 0x69, 0x10, 0x62, 0x1b, 0x90, 0xe9, 0x74, 0xd, 0x86, 0xff, 0x8d, 0xf4, 0x7f, 0x6, 0x58, 0x21, 0xaa, 0xd3, 0xa1, 0xd8, 0x53, 0x2a, 0xb7, 0xce, 0x45, 0x3c, 0x4e, 0x37, 0xbc, 0xc5, 0x2b, 0x52, 0xd9, 0xa0, 0xd2, 0xab, 0x20, 0x59, 0xc4, 0xbd, 0x36, 0x4f, 0x3d, 0x44, 0xcf, 0xb6, 0xe8, 0x91, 0x1a, 0x63, 0x11, 0x68, 0xe3, 0x9a, 0x7, 0x7e, 0xf5, 0x8c, 0xfe, 0x87, 0xc, 0x75, 0xb0, 0xc9, 0x42, 0x3b, 0x49, 0x30, 0xbb, 0xc2, 0x5f, 0x26, 0xad, 0xd4, 0xa6, 0xdf, 0x54, 0x2d, 0x73, 0xa, 0x81, 0xf8, 0x8a, 0xf3, 0x78, 0x1, 0x9c, 0xe5, 0x6e, 0x17, 0x65, 0x1c, 0x97, 0xee, 0x56, 0x2f, 0xa4, 0xdd, 0xaf, 0xd6, 0x5d, 0x24, 0xb9, 0xc0, 0x4b, 0x32, 0x40, 0x39, 0xb2, 0xcb, 0x95, 0xec, 0x67, 0x1e, 0x6c, 0x15, 0x9e, 0xe7, 0x7a, 0x3, 0x88, 0xf1, 0x83, 0xfa, 0x71, 0x8, 0xcd, 0xb4, 0x3f, 0x46, 0x34, 0x4d, 0xc6, 0xbf, 0x22, 0x5b, 0xd0, 0xa9, 0xdb, 0xa2, 0x29, 0x50, 0xe, 0x77, 0xfc, 0x85, 0xf7, 0x8e, 0x5, 0x7c, 0xe1, 0x98, 0x13, 0x6a, 0x18, 0x61, 0xea, 0x93, 0x7d, 0x4, 0x8f, 0xf6, 0x84, 0xfd, 0x76, 0xf, 0x92, 0xeb, 0x60, 0x19, 0x6b, 0x12, 0x99, 0xe0, 0xbe, 0xc7, 0x4c, 0x35, 0x47, 0x3e, 0xb5, 0xcc, 0x51, 0x28, 0xa3, 0xda, 0xa8, 0xd1, 0x5a, 0x23, 0xe6, 0x9f, 0x14, 0x6d, 0x1f, 0x66, 0xed, 0x94, 0x9, 0x70, 0xfb, 0x82, 0xf0, 0x89, 0x2, 0x7b, 0x25, 0x5c, 0xd7, 0xae, 0xdc, 0xa5, 0x2e, 0x57, 0xca, 0xb3, 0x38, 0x41, 0x33, 0x4a, 0xc1, 0xb8},
- {0x0, 0x7a, 0xf4, 0x8e, 0xf5, 0x8f, 0x1, 0x7b, 0xf7, 0x8d, 0x3, 0x79, 0x2, 0x78, 0xf6, 0x8c, 0xf3, 0x89, 0x7, 0x7d, 0x6, 0x7c, 0xf2, 0x88, 0x4, 0x7e, 0xf0, 0x8a, 0xf1, 0x8b, 0x5, 0x7f, 0xfb, 0x81, 0xf, 0x75, 0xe, 0x74, 0xfa, 0x80, 0xc, 0x76, 0xf8, 0x82, 0xf9, 0x83, 0xd, 0x77, 0x8, 0x72, 0xfc, 0x86, 0xfd, 0x87, 0x9, 0x73, 0xff, 0x85, 0xb, 0x71, 0xa, 0x70, 0xfe, 0x84, 0xeb, 0x91, 0x1f, 0x65, 0x1e, 0x64, 0xea, 0x90, 0x1c, 0x66, 0xe8, 0x92, 0xe9, 0x93, 0x1d, 0x67, 0x18, 0x62, 0xec, 0x96, 0xed, 0x97, 0x19, 0x63, 0xef, 0x95, 0x1b, 0x61, 0x1a, 0x60, 0xee, 0x94, 0x10, 0x6a, 0xe4, 0x9e, 0xe5, 0x9f, 0x11, 0x6b, 0xe7, 0x9d, 0x13, 0x69, 0x12, 0x68, 0xe6, 0x9c, 0xe3, 0x99, 0x17, 0x6d, 0x16, 0x6c, 0xe2, 0x98, 0x14, 0x6e, 0xe0, 0x9a, 0xe1, 0x9b, 0x15, 0x6f, 0xcb, 0xb1, 0x3f, 0x45, 0x3e, 0x44, 0xca, 0xb0, 0x3c, 0x46, 0xc8, 0xb2, 0xc9, 0xb3, 0x3d, 0x47, 0x38, 0x42, 0xcc, 0xb6, 0xcd, 0xb7, 0x39, 0x43, 0xcf, 0xb5, 0x3b, 0x41, 0x3a, 0x40, 0xce, 0xb4, 0x30, 0x4a, 0xc4, 0xbe, 0xc5, 0xbf, 0x31, 0x4b, 0xc7, 0xbd, 0x33, 0x49, 0x32, 0x48, 0xc6, 0xbc, 0xc3, 0xb9, 0x37, 0x4d, 0x36, 0x4c, 0xc2, 0xb8, 0x34, 0x4e, 0xc0, 0xba, 0xc1, 0xbb, 0x35, 0x4f, 0x20, 0x5a, 0xd4, 0xae, 0xd5, 0xaf, 0x21, 0x5b, 0xd7, 0xad, 0x23, 0x59, 0x22, 0x58, 0xd6, 0xac, 0xd3, 0xa9, 0x27, 0x5d, 0x26, 0x5c, 0xd2, 0xa8, 0x24, 0x5e, 0xd0, 0xaa, 0xd1, 0xab, 0x25, 0x5f, 0xdb, 0xa1, 0x2f, 0x55, 0x2e, 0x54, 0xda, 0xa0, 0x2c, 0x56, 0xd8, 0xa2, 0xd9, 0xa3, 0x2d, 0x57, 0x28, 0x52, 0xdc, 0xa6, 0xdd, 0xa7, 0x29, 0x53, 0xdf, 0xa5, 0x2b, 0x51, 0x2a, 0x50, 0xde, 0xa4},
- {0x0, 0x7b, 0xf6, 0x8d, 0xf1, 0x8a, 0x7, 0x7c, 0xff, 0x84, 0x9, 0x72, 0xe, 0x75, 0xf8, 0x83, 0xe3, 0x98, 0x15, 0x6e, 0x12, 0x69, 0xe4, 0x9f, 0x1c, 0x67, 0xea, 0x91, 0xed, 0x96, 0x1b, 0x60, 0xdb, 0xa0, 0x2d, 0x56, 0x2a, 0x51, 0xdc, 0xa7, 0x24, 0x5f, 0xd2, 0xa9, 0xd5, 0xae, 0x23, 0x58, 0x38, 0x43, 0xce, 0xb5, 0xc9, 0xb2, 0x3f, 0x44, 0xc7, 0xbc, 0x31, 0x4a, 0x36, 0x4d, 0xc0, 0xbb, 0xab, 0xd0, 0x5d, 0x26, 0x5a, 0x21, 0xac, 0xd7, 0x54, 0x2f, 0xa2, 0xd9, 0xa5, 0xde, 0x53, 0x28, 0x48, 0x33, 0xbe, 0xc5, 0xb9, 0xc2, 0x4f, 0x34, 0xb7, 0xcc, 0x41, 0x3a, 0x46, 0x3d, 0xb0, 0xcb, 0x70, 0xb, 0x86, 0xfd, 0x81, 0xfa, 0x77, 0xc, 0x8f, 0xf4, 0x79, 0x2, 0x7e, 0x5, 0x88, 0xf3, 0x93, 0xe8, 0x65, 0x1e, 0x62, 0x19, 0x94, 0xef, 0x6c, 0x17, 0x9a, 0xe1, 0x9d, 0xe6, 0x6b, 0x10, 0x4b, 0x30, 0xbd, 0xc6, 0xba, 0xc1, 0x4c, 0x37, 0xb4, 0xcf, 0x42, 0x39, 0x45, 0x3e, 0xb3, 0xc8, 0xa8, 0xd3, 0x5e, 0x25, 0x59, 0x22, 0xaf, 0xd4, 0x57, 0x2c, 0xa1, 0xda, 0xa6, 0xdd, 0x50, 0x2b, 0x90, 0xeb, 0x66, 0x1d, 0x61, 0x1a, 0x97, 0xec, 0x6f, 0x14, 0x99, 0xe2, 0x9e, 0xe5, 0x68, 0x13, 0x73, 0x8, 0x85, 0xfe, 0x82, 0xf9, 0x74, 0xf, 0x8c, 0xf7, 0x7a, 0x1, 0x7d, 0x6, 0x8b, 0xf0, 0xe0, 0x9b, 0x16, 0x6d, 0x11, 0x6a, 0xe7, 0x9c, 0x1f, 0x64, 0xe9, 0x92, 0xee, 0x95, 0x18, 0x63, 0x3, 0x78, 0xf5, 0x8e, 0xf2, 0x89, 0x4, 0x7f, 0xfc, 0x87, 0xa, 0x71, 0xd, 0x76, 0xfb, 0x80, 0x3b, 0x40, 0xcd, 0xb6, 0xca, 0xb1, 0x3c, 0x47, 0xc4, 0xbf, 0x32, 0x49, 0x35, 0x4e, 0xc3, 0xb8, 0xd8, 0xa3, 0x2e, 0x55, 0x29, 0x52, 0xdf, 0xa4, 0x27, 0x5c, 0xd1, 0xaa, 0xd6, 0xad, 0x20, 0x5b},
- {0x0, 0x7c, 0xf8, 0x84, 0xed, 0x91, 0x15, 0x69, 0xc7, 0xbb, 0x3f, 0x43, 0x2a, 0x56, 0xd2, 0xae, 0x93, 0xef, 0x6b, 0x17, 0x7e, 0x2, 0x86, 0xfa, 0x54, 0x28, 0xac, 0xd0, 0xb9, 0xc5, 0x41, 0x3d, 0x3b, 0x47, 0xc3, 0xbf, 0xd6, 0xaa, 0x2e, 0x52, 0xfc, 0x80, 0x4, 0x78, 0x11, 0x6d, 0xe9, 0x95, 0xa8, 0xd4, 0x50, 0x2c, 0x45, 0x39, 0xbd, 0xc1, 0x6f, 0x13, 0x97, 0xeb, 0x82, 0xfe, 0x7a, 0x6, 0x76, 0xa, 0x8e, 0xf2, 0x9b, 0xe7, 0x63, 0x1f, 0xb1, 0xcd, 0x49, 0x35, 0x5c, 0x20, 0xa4, 0xd8, 0xe5, 0x99, 0x1d, 0x61, 0x8, 0x74, 0xf0, 0x8c, 0x22, 0x5e, 0xda, 0xa6, 0xcf, 0xb3, 0x37, 0x4b, 0x4d, 0x31, 0xb5, 0xc9, 0xa0, 0xdc, 0x58, 0x24, 0x8a, 0xf6, 0x72, 0xe, 0x67, 0x1b, 0x9f, 0xe3, 0xde, 0xa2, 0x26, 0x5a, 0x33, 0x4f, 0xcb, 0xb7, 0x19, 0x65, 0xe1, 0x9d, 0xf4, 0x88, 0xc, 0x70, 0xec, 0x90, 0x14, 0x68, 0x1, 0x7d, 0xf9, 0x85, 0x2b, 0x57, 0xd3, 0xaf, 0xc6, 0xba, 0x3e, 0x42, 0x7f, 0x3, 0x87, 0xfb, 0x92, 0xee, 0x6a, 0x16, 0xb8, 0xc4, 0x40, 0x3c, 0x55, 0x29, 0xad, 0xd1, 0xd7, 0xab, 0x2f, 0x53, 0x3a, 0x46, 0xc2, 0xbe, 0x10, 0x6c, 0xe8, 0x94, 0xfd, 0x81, 0x5, 0x79, 0x44, 0x38, 0xbc, 0xc0, 0xa9, 0xd5, 0x51, 0x2d, 0x83, 0xff, 0x7b, 0x7, 0x6e, 0x12, 0x96, 0xea, 0x9a, 0xe6, 0x62, 0x1e, 0x77, 0xb, 0x8f, 0xf3, 0x5d, 0x21, 0xa5, 0xd9, 0xb0, 0xcc, 0x48, 0x34, 0x9, 0x75, 0xf1, 0x8d, 0xe4, 0x98, 0x1c, 0x60, 0xce, 0xb2, 0x36, 0x4a, 0x23, 0x5f, 0xdb, 0xa7, 0xa1, 0xdd, 0x59, 0x25, 0x4c, 0x30, 0xb4, 0xc8, 0x66, 0x1a, 0x9e, 0xe2, 0x8b, 0xf7, 0x73, 0xf, 0x32, 0x4e, 0xca, 0xb6, 0xdf, 0xa3, 0x27, 0x5b, 0xf5, 0x89, 0xd, 0x71, 0x18, 0x64, 0xe0, 0x9c},
- {0x0, 0x7d, 0xfa, 0x87, 0xe9, 0x94, 0x13, 0x6e, 0xcf, 0xb2, 0x35, 0x48, 0x26, 0x5b, 0xdc, 0xa1, 0x83, 0xfe, 0x79, 0x4, 0x6a, 0x17, 0x90, 0xed, 0x4c, 0x31, 0xb6, 0xcb, 0xa5, 0xd8, 0x5f, 0x22, 0x1b, 0x66, 0xe1, 0x9c, 0xf2, 0x8f, 0x8, 0x75, 0xd4, 0xa9, 0x2e, 0x53, 0x3d, 0x40, 0xc7, 0xba, 0x98, 0xe5, 0x62, 0x1f, 0x71, 0xc, 0x8b, 0xf6, 0x57, 0x2a, 0xad, 0xd0, 0xbe, 0xc3, 0x44, 0x39, 0x36, 0x4b, 0xcc, 0xb1, 0xdf, 0xa2, 0x25, 0x58, 0xf9, 0x84, 0x3, 0x7e, 0x10, 0x6d, 0xea, 0x97, 0xb5, 0xc8, 0x4f, 0x32, 0x5c, 0x21, 0xa6, 0xdb, 0x7a, 0x7, 0x80, 0xfd, 0x93, 0xee, 0x69, 0x14, 0x2d, 0x50, 0xd7, 0xaa, 0xc4, 0xb9, 0x3e, 0x43, 0xe2, 0x9f, 0x18, 0x65, 0xb, 0x76, 0xf1, 0x8c, 0xae, 0xd3, 0x54, 0x29, 0x47, 0x3a, 0xbd, 0xc0, 0x61, 0x1c, 0x9b, 0xe6, 0x88, 0xf5, 0x72, 0xf, 0x6c, 0x11, 0x96, 0xeb, 0x85, 0xf8, 0x7f, 0x2, 0xa3, 0xde, 0x59, 0x24, 0x4a, 0x37, 0xb0, 0xcd, 0xef, 0x92, 0x15, 0x68, 0x6, 0x7b, 0xfc, 0x81, 0x20, 0x5d, 0xda, 0xa7, 0xc9, 0xb4, 0x33, 0x4e, 0x77, 0xa, 0x8d, 0xf0, 0x9e, 0xe3, 0x64, 0x19, 0xb8, 0xc5, 0x42, 0x3f, 0x51, 0x2c, 0xab, 0xd6, 0xf4, 0x89, 0xe, 0x73, 0x1d, 0x60, 0xe7, 0x9a, 0x3b, 0x46, 0xc1, 0xbc, 0xd2, 0xaf, 0x28, 0x55, 0x5a, 0x27, 0xa0, 0xdd, 0xb3, 0xce, 0x49, 0x34, 0x95, 0xe8, 0x6f, 0x12, 0x7c, 0x1, 0x86, 0xfb, 0xd9, 0xa4, 0x23, 0x5e, 0x30, 0x4d, 0xca, 0xb7, 0x16, 0x6b, 0xec, 0x91, 0xff, 0x82, 0x5, 0x78, 0x41, 0x3c, 0xbb, 0xc6, 0xa8, 0xd5, 0x52, 0x2f, 0x8e, 0xf3, 0x74, 0x9, 0x67, 0x1a, 0x9d, 0xe0, 0xc2, 0xbf, 0x38, 0x45, 0x2b, 0x56, 0xd1, 0xac, 0xd, 0x70, 0xf7, 0x8a, 0xe4, 0x99, 0x1e, 0x63},
- {0x0, 0x7e, 0xfc, 0x82, 0xe5, 0x9b, 0x19, 0x67, 0xd7, 0xa9, 0x2b, 0x55, 0x32, 0x4c, 0xce, 0xb0, 0xb3, 0xcd, 0x4f, 0x31, 0x56, 0x28, 0xaa, 0xd4, 0x64, 0x1a, 0x98, 0xe6, 0x81, 0xff, 0x7d, 0x3, 0x7b, 0x5, 0x87, 0xf9, 0x9e, 0xe0, 0x62, 0x1c, 0xac, 0xd2, 0x50, 0x2e, 0x49, 0x37, 0xb5, 0xcb, 0xc8, 0xb6, 0x34, 0x4a, 0x2d, 0x53, 0xd1, 0xaf, 0x1f, 0x61, 0xe3, 0x9d, 0xfa, 0x84, 0x6, 0x78, 0xf6, 0x88, 0xa, 0x74, 0x13, 0x6d, 0xef, 0x91, 0x21, 0x5f, 0xdd, 0xa3, 0xc4, 0xba, 0x38, 0x46, 0x45, 0x3b, 0xb9, 0xc7, 0xa0, 0xde, 0x5c, 0x22, 0x92, 0xec, 0x6e, 0x10, 0x77, 0x9, 0x8b, 0xf5, 0x8d, 0xf3, 0x71, 0xf, 0x68, 0x16, 0x94, 0xea, 0x5a, 0x24, 0xa6, 0xd8, 0xbf, 0xc1, 0x43, 0x3d, 0x3e, 0x40, 0xc2, 0xbc, 0xdb, 0xa5, 0x27, 0x59, 0xe9, 0x97, 0x15, 0x6b, 0xc, 0x72, 0xf0, 0x8e, 0xf1, 0x8f, 0xd, 0x73, 0x14, 0x6a, 0xe8, 0x96, 0x26, 0x58, 0xda, 0xa4, 0xc3, 0xbd, 0x3f, 0x41, 0x42, 0x3c, 0xbe, 0xc0, 0xa7, 0xd9, 0x5b, 0x25, 0x95, 0xeb, 0x69, 0x17, 0x70, 0xe, 0x8c, 0xf2, 0x8a, 0xf4, 0x76, 0x8, 0x6f, 0x11, 0x93, 0xed, 0x5d, 0x23, 0xa1, 0xdf, 0xb8, 0xc6, 0x44, 0x3a, 0x39, 0x47, 0xc5, 0xbb, 0xdc, 0xa2, 0x20, 0x5e, 0xee, 0x90, 0x12, 0x6c, 0xb, 0x75, 0xf7, 0x89, 0x7, 0x79, 0xfb, 0x85, 0xe2, 0x9c, 0x1e, 0x60, 0xd0, 0xae, 0x2c, 0x52, 0x35, 0x4b, 0xc9, 0xb7, 0xb4, 0xca, 0x48, 0x36, 0x51, 0x2f, 0xad, 0xd3, 0x63, 0x1d, 0x9f, 0xe1, 0x86, 0xf8, 0x7a, 0x4, 0x7c, 0x2, 0x80, 0xfe, 0x99, 0xe7, 0x65, 0x1b, 0xab, 0xd5, 0x57, 0x29, 0x4e, 0x30, 0xb2, 0xcc, 0xcf, 0xb1, 0x33, 0x4d, 0x2a, 0x54, 0xd6, 0xa8, 0x18, 0x66, 0xe4, 0x9a, 0xfd, 0x83, 0x1, 0x7f},
- {0x0, 0x7f, 0xfe, 0x81, 0xe1, 0x9e, 0x1f, 0x60, 0xdf, 0xa0, 0x21, 0x5e, 0x3e, 0x41, 0xc0, 0xbf, 0xa3, 0xdc, 0x5d, 0x22, 0x42, 0x3d, 0xbc, 0xc3, 0x7c, 0x3, 0x82, 0xfd, 0x9d, 0xe2, 0x63, 0x1c, 0x5b, 0x24, 0xa5, 0xda, 0xba, 0xc5, 0x44, 0x3b, 0x84, 0xfb, 0x7a, 0x5, 0x65, 0x1a, 0x9b, 0xe4, 0xf8, 0x87, 0x6, 0x79, 0x19, 0x66, 0xe7, 0x98, 0x27, 0x58, 0xd9, 0xa6, 0xc6, 0xb9, 0x38, 0x47, 0xb6, 0xc9, 0x48, 0x37, 0x57, 0x28, 0xa9, 0xd6, 0x69, 0x16, 0x97, 0xe8, 0x88, 0xf7, 0x76, 0x9, 0x15, 0x6a, 0xeb, 0x94, 0xf4, 0x8b, 0xa, 0x75, 0xca, 0xb5, 0x34, 0x4b, 0x2b, 0x54, 0xd5, 0xaa, 0xed, 0x92, 0x13, 0x6c, 0xc, 0x73, 0xf2, 0x8d, 0x32, 0x4d, 0xcc, 0xb3, 0xd3, 0xac, 0x2d, 0x52, 0x4e, 0x31, 0xb0, 0xcf, 0xaf, 0xd0, 0x51, 0x2e, 0x91, 0xee, 0x6f, 0x10, 0x70, 0xf, 0x8e, 0xf1, 0x71, 0xe, 0x8f, 0xf0, 0x90, 0xef, 0x6e, 0x11, 0xae, 0xd1, 0x50, 0x2f, 0x4f, 0x30, 0xb1, 0xce, 0xd2, 0xad, 0x2c, 0x53, 0x33, 0x4c, 0xcd, 0xb2, 0xd, 0x72, 0xf3, 0x8c, 0xec, 0x93, 0x12, 0x6d, 0x2a, 0x55, 0xd4, 0xab, 0xcb, 0xb4, 0x35, 0x4a, 0xf5, 0x8a, 0xb, 0x74, 0x14, 0x6b, 0xea, 0x95, 0x89, 0xf6, 0x77, 0x8, 0x68, 0x17, 0x96, 0xe9, 0x56, 0x29, 0xa8, 0xd7, 0xb7, 0xc8, 0x49, 0x36, 0xc7, 0xb8, 0x39, 0x46, 0x26, 0x59, 0xd8, 0xa7, 0x18, 0x67, 0xe6, 0x99, 0xf9, 0x86, 0x7, 0x78, 0x64, 0x1b, 0x9a, 0xe5, 0x85, 0xfa, 0x7b, 0x4, 0xbb, 0xc4, 0x45, 0x3a, 0x5a, 0x25, 0xa4, 0xdb, 0x9c, 0xe3, 0x62, 0x1d, 0x7d, 0x2, 0x83, 0xfc, 0x43, 0x3c, 0xbd, 0xc2, 0xa2, 0xdd, 0x5c, 0x23, 0x3f, 0x40, 0xc1, 0xbe, 0xde, 0xa1, 0x20, 0x5f, 0xe0, 0x9f, 0x1e, 0x61, 0x1, 0x7e, 0xff, 0x80},
- {0x0, 0x80, 0x1d, 0x9d, 0x3a, 0xba, 0x27, 0xa7, 0x74, 0xf4, 0x69, 0xe9, 0x4e, 0xce, 0x53, 0xd3, 0xe8, 0x68, 0xf5, 0x75, 0xd2, 0x52, 0xcf, 0x4f, 0x9c, 0x1c, 0x81, 0x1, 0xa6, 0x26, 0xbb, 0x3b, 0xcd, 0x4d, 0xd0, 0x50, 0xf7, 0x77, 0xea, 0x6a, 0xb9, 0x39, 0xa4, 0x24, 0x83, 0x3, 0x9e, 0x1e, 0x25, 0xa5, 0x38, 0xb8, 0x1f, 0x9f, 0x2, 0x82, 0x51, 0xd1, 0x4c, 0xcc, 0x6b, 0xeb, 0x76, 0xf6, 0x87, 0x7, 0x9a, 0x1a, 0xbd, 0x3d, 0xa0, 0x20, 0xf3, 0x73, 0xee, 0x6e, 0xc9, 0x49, 0xd4, 0x54, 0x6f, 0xef, 0x72, 0xf2, 0x55, 0xd5, 0x48, 0xc8, 0x1b, 0x9b, 0x6, 0x86, 0x21, 0xa1, 0x3c, 0xbc, 0x4a, 0xca, 0x57, 0xd7, 0x70, 0xf0, 0x6d, 0xed, 0x3e, 0xbe, 0x23, 0xa3, 0x4, 0x84, 0x19, 0x99, 0xa2, 0x22, 0xbf, 0x3f, 0x98, 0x18, 0x85, 0x5, 0xd6, 0x56, 0xcb, 0x4b, 0xec, 0x6c, 0xf1, 0x71, 0x13, 0x93, 0xe, 0x8e, 0x29, 0xa9, 0x34, 0xb4, 0x67, 0xe7, 0x7a, 0xfa, 0x5d, 0xdd, 0x40, 0xc0, 0xfb, 0x7b, 0xe6, 0x66, 0xc1, 0x41, 0xdc, 0x5c, 0x8f, 0xf, 0x92, 0x12, 0xb5, 0x35, 0xa8, 0x28, 0xde, 0x5e, 0xc3, 0x43, 0xe4, 0x64, 0xf9, 0x79, 0xaa, 0x2a, 0xb7, 0x37, 0x90, 0x10, 0x8d, 0xd, 0x36, 0xb6, 0x2b, 0xab, 0xc, 0x8c, 0x11, 0x91, 0x42, 0xc2, 0x5f, 0xdf, 0x78, 0xf8, 0x65, 0xe5, 0x94, 0x14, 0x89, 0x9, 0xae, 0x2e, 0xb3, 0x33, 0xe0, 0x60, 0xfd, 0x7d, 0xda, 0x5a, 0xc7, 0x47, 0x7c, 0xfc, 0x61, 0xe1, 0x46, 0xc6, 0x5b, 0xdb, 0x8, 0x88, 0x15, 0x95, 0x32, 0xb2, 0x2f, 0xaf, 0x59, 0xd9, 0x44, 0xc4, 0x63, 0xe3, 0x7e, 0xfe, 0x2d, 0xad, 0x30, 0xb0, 0x17, 0x97, 0xa, 0x8a, 0xb1, 0x31, 0xac, 0x2c, 0x8b, 0xb, 0x96, 0x16, 0xc5, 0x45, 0xd8, 0x58, 0xff, 0x7f, 0xe2, 0x62},
- {0x0, 0x81, 0x1f, 0x9e, 0x3e, 0xbf, 0x21, 0xa0, 0x7c, 0xfd, 0x63, 0xe2, 0x42, 0xc3, 0x5d, 0xdc, 0xf8, 0x79, 0xe7, 0x66, 0xc6, 0x47, 0xd9, 0x58, 0x84, 0x5, 0x9b, 0x1a, 0xba, 0x3b, 0xa5, 0x24, 0xed, 0x6c, 0xf2, 0x73, 0xd3, 0x52, 0xcc, 0x4d, 0x91, 0x10, 0x8e, 0xf, 0xaf, 0x2e, 0xb0, 0x31, 0x15, 0x94, 0xa, 0x8b, 0x2b, 0xaa, 0x34, 0xb5, 0x69, 0xe8, 0x76, 0xf7, 0x57, 0xd6, 0x48, 0xc9, 0xc7, 0x46, 0xd8, 0x59, 0xf9, 0x78, 0xe6, 0x67, 0xbb, 0x3a, 0xa4, 0x25, 0x85, 0x4, 0x9a, 0x1b, 0x3f, 0xbe, 0x20, 0xa1, 0x1, 0x80, 0x1e, 0x9f, 0x43, 0xc2, 0x5c, 0xdd, 0x7d, 0xfc, 0x62, 0xe3, 0x2a, 0xab, 0x35, 0xb4, 0x14, 0x95, 0xb, 0x8a, 0x56, 0xd7, 0x49, 0xc8, 0x68, 0xe9, 0x77, 0xf6, 0xd2, 0x53, 0xcd, 0x4c, 0xec, 0x6d, 0xf3, 0x72, 0xae, 0x2f, 0xb1, 0x30, 0x90, 0x11, 0x8f, 0xe, 0x93, 0x12, 0x8c, 0xd, 0xad, 0x2c, 0xb2, 0x33, 0xef, 0x6e, 0xf0, 0x71, 0xd1, 0x50, 0xce, 0x4f, 0x6b, 0xea, 0x74, 0xf5, 0x55, 0xd4, 0x4a, 0xcb, 0x17, 0x96, 0x8, 0x89, 0x29, 0xa8, 0x36, 0xb7, 0x7e, 0xff, 0x61, 0xe0, 0x40, 0xc1, 0x5f, 0xde, 0x2, 0x83, 0x1d, 0x9c, 0x3c, 0xbd, 0x23, 0xa2, 0x86, 0x7, 0x99, 0x18, 0xb8, 0x39, 0xa7, 0x26, 0xfa, 0x7b, 0xe5, 0x64, 0xc4, 0x45, 0xdb, 0x5a, 0x54, 0xd5, 0x4b, 0xca, 0x6a, 0xeb, 0x75, 0xf4, 0x28, 0xa9, 0x37, 0xb6, 0x16, 0x97, 0x9, 0x88, 0xac, 0x2d, 0xb3, 0x32, 0x92, 0x13, 0x8d, 0xc, 0xd0, 0x51, 0xcf, 0x4e, 0xee, 0x6f, 0xf1, 0x70, 0xb9, 0x38, 0xa6, 0x27, 0x87, 0x6, 0x98, 0x19, 0xc5, 0x44, 0xda, 0x5b, 0xfb, 0x7a, 0xe4, 0x65, 0x41, 0xc0, 0x5e, 0xdf, 0x7f, 0xfe, 0x60, 0xe1, 0x3d, 0xbc, 0x22, 0xa3, 0x3, 0x82, 0x1c, 0x9d},
- {0x0, 0x82, 0x19, 0x9b, 0x32, 0xb0, 0x2b, 0xa9, 0x64, 0xe6, 0x7d, 0xff, 0x56, 0xd4, 0x4f, 0xcd, 0xc8, 0x4a, 0xd1, 0x53, 0xfa, 0x78, 0xe3, 0x61, 0xac, 0x2e, 0xb5, 0x37, 0x9e, 0x1c, 0x87, 0x5, 0x8d, 0xf, 0x94, 0x16, 0xbf, 0x3d, 0xa6, 0x24, 0xe9, 0x6b, 0xf0, 0x72, 0xdb, 0x59, 0xc2, 0x40, 0x45, 0xc7, 0x5c, 0xde, 0x77, 0xf5, 0x6e, 0xec, 0x21, 0xa3, 0x38, 0xba, 0x13, 0x91, 0xa, 0x88, 0x7, 0x85, 0x1e, 0x9c, 0x35, 0xb7, 0x2c, 0xae, 0x63, 0xe1, 0x7a, 0xf8, 0x51, 0xd3, 0x48, 0xca, 0xcf, 0x4d, 0xd6, 0x54, 0xfd, 0x7f, 0xe4, 0x66, 0xab, 0x29, 0xb2, 0x30, 0x99, 0x1b, 0x80, 0x2, 0x8a, 0x8, 0x93, 0x11, 0xb8, 0x3a, 0xa1, 0x23, 0xee, 0x6c, 0xf7, 0x75, 0xdc, 0x5e, 0xc5, 0x47, 0x42, 0xc0, 0x5b, 0xd9, 0x70, 0xf2, 0x69, 0xeb, 0x26, 0xa4, 0x3f, 0xbd, 0x14, 0x96, 0xd, 0x8f, 0xe, 0x8c, 0x17, 0x95, 0x3c, 0xbe, 0x25, 0xa7, 0x6a, 0xe8, 0x73, 0xf1, 0x58, 0xda, 0x41, 0xc3, 0xc6, 0x44, 0xdf, 0x5d, 0xf4, 0x76, 0xed, 0x6f, 0xa2, 0x20, 0xbb, 0x39, 0x90, 0x12, 0x89, 0xb, 0x83, 0x1, 0x9a, 0x18, 0xb1, 0x33, 0xa8, 0x2a, 0xe7, 0x65, 0xfe, 0x7c, 0xd5, 0x57, 0xcc, 0x4e, 0x4b, 0xc9, 0x52, 0xd0, 0x79, 0xfb, 0x60, 0xe2, 0x2f, 0xad, 0x36, 0xb4, 0x1d, 0x9f, 0x4, 0x86, 0x9, 0x8b, 0x10, 0x92, 0x3b, 0xb9, 0x22, 0xa0, 0x6d, 0xef, 0x74, 0xf6, 0x5f, 0xdd, 0x46, 0xc4, 0xc1, 0x43, 0xd8, 0x5a, 0xf3, 0x71, 0xea, 0x68, 0xa5, 0x27, 0xbc, 0x3e, 0x97, 0x15, 0x8e, 0xc, 0x84, 0x6, 0x9d, 0x1f, 0xb6, 0x34, 0xaf, 0x2d, 0xe0, 0x62, 0xf9, 0x7b, 0xd2, 0x50, 0xcb, 0x49, 0x4c, 0xce, 0x55, 0xd7, 0x7e, 0xfc, 0x67, 0xe5, 0x28, 0xaa, 0x31, 0xb3, 0x1a, 0x98, 0x3, 0x81},
- {0x0, 0x83, 0x1b, 0x98, 0x36, 0xb5, 0x2d, 0xae, 0x6c, 0xef, 0x77, 0xf4, 0x5a, 0xd9, 0x41, 0xc2, 0xd8, 0x5b, 0xc3, 0x40, 0xee, 0x6d, 0xf5, 0x76, 0xb4, 0x37, 0xaf, 0x2c, 0x82, 0x1, 0x99, 0x1a, 0xad, 0x2e, 0xb6, 0x35, 0x9b, 0x18, 0x80, 0x3, 0xc1, 0x42, 0xda, 0x59, 0xf7, 0x74, 0xec, 0x6f, 0x75, 0xf6, 0x6e, 0xed, 0x43, 0xc0, 0x58, 0xdb, 0x19, 0x9a, 0x2, 0x81, 0x2f, 0xac, 0x34, 0xb7, 0x47, 0xc4, 0x5c, 0xdf, 0x71, 0xf2, 0x6a, 0xe9, 0x2b, 0xa8, 0x30, 0xb3, 0x1d, 0x9e, 0x6, 0x85, 0x9f, 0x1c, 0x84, 0x7, 0xa9, 0x2a, 0xb2, 0x31, 0xf3, 0x70, 0xe8, 0x6b, 0xc5, 0x46, 0xde, 0x5d, 0xea, 0x69, 0xf1, 0x72, 0xdc, 0x5f, 0xc7, 0x44, 0x86, 0x5, 0x9d, 0x1e, 0xb0, 0x33, 0xab, 0x28, 0x32, 0xb1, 0x29, 0xaa, 0x4, 0x87, 0x1f, 0x9c, 0x5e, 0xdd, 0x45, 0xc6, 0x68, 0xeb, 0x73, 0xf0, 0x8e, 0xd, 0x95, 0x16, 0xb8, 0x3b, 0xa3, 0x20, 0xe2, 0x61, 0xf9, 0x7a, 0xd4, 0x57, 0xcf, 0x4c, 0x56, 0xd5, 0x4d, 0xce, 0x60, 0xe3, 0x7b, 0xf8, 0x3a, 0xb9, 0x21, 0xa2, 0xc, 0x8f, 0x17, 0x94, 0x23, 0xa0, 0x38, 0xbb, 0x15, 0x96, 0xe, 0x8d, 0x4f, 0xcc, 0x54, 0xd7, 0x79, 0xfa, 0x62, 0xe1, 0xfb, 0x78, 0xe0, 0x63, 0xcd, 0x4e, 0xd6, 0x55, 0x97, 0x14, 0x8c, 0xf, 0xa1, 0x22, 0xba, 0x39, 0xc9, 0x4a, 0xd2, 0x51, 0xff, 0x7c, 0xe4, 0x67, 0xa5, 0x26, 0xbe, 0x3d, 0x93, 0x10, 0x88, 0xb, 0x11, 0x92, 0xa, 0x89, 0x27, 0xa4, 0x3c, 0xbf, 0x7d, 0xfe, 0x66, 0xe5, 0x4b, 0xc8, 0x50, 0xd3, 0x64, 0xe7, 0x7f, 0xfc, 0x52, 0xd1, 0x49, 0xca, 0x8, 0x8b, 0x13, 0x90, 0x3e, 0xbd, 0x25, 0xa6, 0xbc, 0x3f, 0xa7, 0x24, 0x8a, 0x9, 0x91, 0x12, 0xd0, 0x53, 0xcb, 0x48, 0xe6, 0x65, 0xfd, 0x7e},
- {0x0, 0x84, 0x15, 0x91, 0x2a, 0xae, 0x3f, 0xbb, 0x54, 0xd0, 0x41, 0xc5, 0x7e, 0xfa, 0x6b, 0xef, 0xa8, 0x2c, 0xbd, 0x39, 0x82, 0x6, 0x97, 0x13, 0xfc, 0x78, 0xe9, 0x6d, 0xd6, 0x52, 0xc3, 0x47, 0x4d, 0xc9, 0x58, 0xdc, 0x67, 0xe3, 0x72, 0xf6, 0x19, 0x9d, 0xc, 0x88, 0x33, 0xb7, 0x26, 0xa2, 0xe5, 0x61, 0xf0, 0x74, 0xcf, 0x4b, 0xda, 0x5e, 0xb1, 0x35, 0xa4, 0x20, 0x9b, 0x1f, 0x8e, 0xa, 0x9a, 0x1e, 0x8f, 0xb, 0xb0, 0x34, 0xa5, 0x21, 0xce, 0x4a, 0xdb, 0x5f, 0xe4, 0x60, 0xf1, 0x75, 0x32, 0xb6, 0x27, 0xa3, 0x18, 0x9c, 0xd, 0x89, 0x66, 0xe2, 0x73, 0xf7, 0x4c, 0xc8, 0x59, 0xdd, 0xd7, 0x53, 0xc2, 0x46, 0xfd, 0x79, 0xe8, 0x6c, 0x83, 0x7, 0x96, 0x12, 0xa9, 0x2d, 0xbc, 0x38, 0x7f, 0xfb, 0x6a, 0xee, 0x55, 0xd1, 0x40, 0xc4, 0x2b, 0xaf, 0x3e, 0xba, 0x1, 0x85, 0x14, 0x90, 0x29, 0xad, 0x3c, 0xb8, 0x3, 0x87, 0x16, 0x92, 0x7d, 0xf9, 0x68, 0xec, 0x57, 0xd3, 0x42, 0xc6, 0x81, 0x5, 0x94, 0x10, 0xab, 0x2f, 0xbe, 0x3a, 0xd5, 0x51, 0xc0, 0x44, 0xff, 0x7b, 0xea, 0x6e, 0x64, 0xe0, 0x71, 0xf5, 0x4e, 0xca, 0x5b, 0xdf, 0x30, 0xb4, 0x25, 0xa1, 0x1a, 0x9e, 0xf, 0x8b, 0xcc, 0x48, 0xd9, 0x5d, 0xe6, 0x62, 0xf3, 0x77, 0x98, 0x1c, 0x8d, 0x9, 0xb2, 0x36, 0xa7, 0x23, 0xb3, 0x37, 0xa6, 0x22, 0x99, 0x1d, 0x8c, 0x8, 0xe7, 0x63, 0xf2, 0x76, 0xcd, 0x49, 0xd8, 0x5c, 0x1b, 0x9f, 0xe, 0x8a, 0x31, 0xb5, 0x24, 0xa0, 0x4f, 0xcb, 0x5a, 0xde, 0x65, 0xe1, 0x70, 0xf4, 0xfe, 0x7a, 0xeb, 0x6f, 0xd4, 0x50, 0xc1, 0x45, 0xaa, 0x2e, 0xbf, 0x3b, 0x80, 0x4, 0x95, 0x11, 0x56, 0xd2, 0x43, 0xc7, 0x7c, 0xf8, 0x69, 0xed, 0x2, 0x86, 0x17, 0x93, 0x28, 0xac, 0x3d, 0xb9},
- {0x0, 0x85, 0x17, 0x92, 0x2e, 0xab, 0x39, 0xbc, 0x5c, 0xd9, 0x4b, 0xce, 0x72, 0xf7, 0x65, 0xe0, 0xb8, 0x3d, 0xaf, 0x2a, 0x96, 0x13, 0x81, 0x4, 0xe4, 0x61, 0xf3, 0x76, 0xca, 0x4f, 0xdd, 0x58, 0x6d, 0xe8, 0x7a, 0xff, 0x43, 0xc6, 0x54, 0xd1, 0x31, 0xb4, 0x26, 0xa3, 0x1f, 0x9a, 0x8, 0x8d, 0xd5, 0x50, 0xc2, 0x47, 0xfb, 0x7e, 0xec, 0x69, 0x89, 0xc, 0x9e, 0x1b, 0xa7, 0x22, 0xb0, 0x35, 0xda, 0x5f, 0xcd, 0x48, 0xf4, 0x71, 0xe3, 0x66, 0x86, 0x3, 0x91, 0x14, 0xa8, 0x2d, 0xbf, 0x3a, 0x62, 0xe7, 0x75, 0xf0, 0x4c, 0xc9, 0x5b, 0xde, 0x3e, 0xbb, 0x29, 0xac, 0x10, 0x95, 0x7, 0x82, 0xb7, 0x32, 0xa0, 0x25, 0x99, 0x1c, 0x8e, 0xb, 0xeb, 0x6e, 0xfc, 0x79, 0xc5, 0x40, 0xd2, 0x57, 0xf, 0x8a, 0x18, 0x9d, 0x21, 0xa4, 0x36, 0xb3, 0x53, 0xd6, 0x44, 0xc1, 0x7d, 0xf8, 0x6a, 0xef, 0xa9, 0x2c, 0xbe, 0x3b, 0x87, 0x2, 0x90, 0x15, 0xf5, 0x70, 0xe2, 0x67, 0xdb, 0x5e, 0xcc, 0x49, 0x11, 0x94, 0x6, 0x83, 0x3f, 0xba, 0x28, 0xad, 0x4d, 0xc8, 0x5a, 0xdf, 0x63, 0xe6, 0x74, 0xf1, 0xc4, 0x41, 0xd3, 0x56, 0xea, 0x6f, 0xfd, 0x78, 0x98, 0x1d, 0x8f, 0xa, 0xb6, 0x33, 0xa1, 0x24, 0x7c, 0xf9, 0x6b, 0xee, 0x52, 0xd7, 0x45, 0xc0, 0x20, 0xa5, 0x37, 0xb2, 0xe, 0x8b, 0x19, 0x9c, 0x73, 0xf6, 0x64, 0xe1, 0x5d, 0xd8, 0x4a, 0xcf, 0x2f, 0xaa, 0x38, 0xbd, 0x1, 0x84, 0x16, 0x93, 0xcb, 0x4e, 0xdc, 0x59, 0xe5, 0x60, 0xf2, 0x77, 0x97, 0x12, 0x80, 0x5, 0xb9, 0x3c, 0xae, 0x2b, 0x1e, 0x9b, 0x9, 0x8c, 0x30, 0xb5, 0x27, 0xa2, 0x42, 0xc7, 0x55, 0xd0, 0x6c, 0xe9, 0x7b, 0xfe, 0xa6, 0x23, 0xb1, 0x34, 0x88, 0xd, 0x9f, 0x1a, 0xfa, 0x7f, 0xed, 0x68, 0xd4, 0x51, 0xc3, 0x46},
- {0x0, 0x86, 0x11, 0x97, 0x22, 0xa4, 0x33, 0xb5, 0x44, 0xc2, 0x55, 0xd3, 0x66, 0xe0, 0x77, 0xf1, 0x88, 0xe, 0x99, 0x1f, 0xaa, 0x2c, 0xbb, 0x3d, 0xcc, 0x4a, 0xdd, 0x5b, 0xee, 0x68, 0xff, 0x79, 0xd, 0x8b, 0x1c, 0x9a, 0x2f, 0xa9, 0x3e, 0xb8, 0x49, 0xcf, 0x58, 0xde, 0x6b, 0xed, 0x7a, 0xfc, 0x85, 0x3, 0x94, 0x12, 0xa7, 0x21, 0xb6, 0x30, 0xc1, 0x47, 0xd0, 0x56, 0xe3, 0x65, 0xf2, 0x74, 0x1a, 0x9c, 0xb, 0x8d, 0x38, 0xbe, 0x29, 0xaf, 0x5e, 0xd8, 0x4f, 0xc9, 0x7c, 0xfa, 0x6d, 0xeb, 0x92, 0x14, 0x83, 0x5, 0xb0, 0x36, 0xa1, 0x27, 0xd6, 0x50, 0xc7, 0x41, 0xf4, 0x72, 0xe5, 0x63, 0x17, 0x91, 0x6, 0x80, 0x35, 0xb3, 0x24, 0xa2, 0x53, 0xd5, 0x42, 0xc4, 0x71, 0xf7, 0x60, 0xe6, 0x9f, 0x19, 0x8e, 0x8, 0xbd, 0x3b, 0xac, 0x2a, 0xdb, 0x5d, 0xca, 0x4c, 0xf9, 0x7f, 0xe8, 0x6e, 0x34, 0xb2, 0x25, 0xa3, 0x16, 0x90, 0x7, 0x81, 0x70, 0xf6, 0x61, 0xe7, 0x52, 0xd4, 0x43, 0xc5, 0xbc, 0x3a, 0xad, 0x2b, 0x9e, 0x18, 0x8f, 0x9, 0xf8, 0x7e, 0xe9, 0x6f, 0xda, 0x5c, 0xcb, 0x4d, 0x39, 0xbf, 0x28, 0xae, 0x1b, 0x9d, 0xa, 0x8c, 0x7d, 0xfb, 0x6c, 0xea, 0x5f, 0xd9, 0x4e, 0xc8, 0xb1, 0x37, 0xa0, 0x26, 0x93, 0x15, 0x82, 0x4, 0xf5, 0x73, 0xe4, 0x62, 0xd7, 0x51, 0xc6, 0x40, 0x2e, 0xa8, 0x3f, 0xb9, 0xc, 0x8a, 0x1d, 0x9b, 0x6a, 0xec, 0x7b, 0xfd, 0x48, 0xce, 0x59, 0xdf, 0xa6, 0x20, 0xb7, 0x31, 0x84, 0x2, 0x95, 0x13, 0xe2, 0x64, 0xf3, 0x75, 0xc0, 0x46, 0xd1, 0x57, 0x23, 0xa5, 0x32, 0xb4, 0x1, 0x87, 0x10, 0x96, 0x67, 0xe1, 0x76, 0xf0, 0x45, 0xc3, 0x54, 0xd2, 0xab, 0x2d, 0xba, 0x3c, 0x89, 0xf, 0x98, 0x1e, 0xef, 0x69, 0xfe, 0x78, 0xcd, 0x4b, 0xdc, 0x5a},
- {0x0, 0x87, 0x13, 0x94, 0x26, 0xa1, 0x35, 0xb2, 0x4c, 0xcb, 0x5f, 0xd8, 0x6a, 0xed, 0x79, 0xfe, 0x98, 0x1f, 0x8b, 0xc, 0xbe, 0x39, 0xad, 0x2a, 0xd4, 0x53, 0xc7, 0x40, 0xf2, 0x75, 0xe1, 0x66, 0x2d, 0xaa, 0x3e, 0xb9, 0xb, 0x8c, 0x18, 0x9f, 0x61, 0xe6, 0x72, 0xf5, 0x47, 0xc0, 0x54, 0xd3, 0xb5, 0x32, 0xa6, 0x21, 0x93, 0x14, 0x80, 0x7, 0xf9, 0x7e, 0xea, 0x6d, 0xdf, 0x58, 0xcc, 0x4b, 0x5a, 0xdd, 0x49, 0xce, 0x7c, 0xfb, 0x6f, 0xe8, 0x16, 0x91, 0x5, 0x82, 0x30, 0xb7, 0x23, 0xa4, 0xc2, 0x45, 0xd1, 0x56, 0xe4, 0x63, 0xf7, 0x70, 0x8e, 0x9, 0x9d, 0x1a, 0xa8, 0x2f, 0xbb, 0x3c, 0x77, 0xf0, 0x64, 0xe3, 0x51, 0xd6, 0x42, 0xc5, 0x3b, 0xbc, 0x28, 0xaf, 0x1d, 0x9a, 0xe, 0x89, 0xef, 0x68, 0xfc, 0x7b, 0xc9, 0x4e, 0xda, 0x5d, 0xa3, 0x24, 0xb0, 0x37, 0x85, 0x2, 0x96, 0x11, 0xb4, 0x33, 0xa7, 0x20, 0x92, 0x15, 0x81, 0x6, 0xf8, 0x7f, 0xeb, 0x6c, 0xde, 0x59, 0xcd, 0x4a, 0x2c, 0xab, 0x3f, 0xb8, 0xa, 0x8d, 0x19, 0x9e, 0x60, 0xe7, 0x73, 0xf4, 0x46, 0xc1, 0x55, 0xd2, 0x99, 0x1e, 0x8a, 0xd, 0xbf, 0x38, 0xac, 0x2b, 0xd5, 0x52, 0xc6, 0x41, 0xf3, 0x74, 0xe0, 0x67, 0x1, 0x86, 0x12, 0x95, 0x27, 0xa0, 0x34, 0xb3, 0x4d, 0xca, 0x5e, 0xd9, 0x6b, 0xec, 0x78, 0xff, 0xee, 0x69, 0xfd, 0x7a, 0xc8, 0x4f, 0xdb, 0x5c, 0xa2, 0x25, 0xb1, 0x36, 0x84, 0x3, 0x97, 0x10, 0x76, 0xf1, 0x65, 0xe2, 0x50, 0xd7, 0x43, 0xc4, 0x3a, 0xbd, 0x29, 0xae, 0x1c, 0x9b, 0xf, 0x88, 0xc3, 0x44, 0xd0, 0x57, 0xe5, 0x62, 0xf6, 0x71, 0x8f, 0x8, 0x9c, 0x1b, 0xa9, 0x2e, 0xba, 0x3d, 0x5b, 0xdc, 0x48, 0xcf, 0x7d, 0xfa, 0x6e, 0xe9, 0x17, 0x90, 0x4, 0x83, 0x31, 0xb6, 0x22, 0xa5},
- {0x0, 0x88, 0xd, 0x85, 0x1a, 0x92, 0x17, 0x9f, 0x34, 0xbc, 0x39, 0xb1, 0x2e, 0xa6, 0x23, 0xab, 0x68, 0xe0, 0x65, 0xed, 0x72, 0xfa, 0x7f, 0xf7, 0x5c, 0xd4, 0x51, 0xd9, 0x46, 0xce, 0x4b, 0xc3, 0xd0, 0x58, 0xdd, 0x55, 0xca, 0x42, 0xc7, 0x4f, 0xe4, 0x6c, 0xe9, 0x61, 0xfe, 0x76, 0xf3, 0x7b, 0xb8, 0x30, 0xb5, 0x3d, 0xa2, 0x2a, 0xaf, 0x27, 0x8c, 0x4, 0x81, 0x9, 0x96, 0x1e, 0x9b, 0x13, 0xbd, 0x35, 0xb0, 0x38, 0xa7, 0x2f, 0xaa, 0x22, 0x89, 0x1, 0x84, 0xc, 0x93, 0x1b, 0x9e, 0x16, 0xd5, 0x5d, 0xd8, 0x50, 0xcf, 0x47, 0xc2, 0x4a, 0xe1, 0x69, 0xec, 0x64, 0xfb, 0x73, 0xf6, 0x7e, 0x6d, 0xe5, 0x60, 0xe8, 0x77, 0xff, 0x7a, 0xf2, 0x59, 0xd1, 0x54, 0xdc, 0x43, 0xcb, 0x4e, 0xc6, 0x5, 0x8d, 0x8, 0x80, 0x1f, 0x97, 0x12, 0x9a, 0x31, 0xb9, 0x3c, 0xb4, 0x2b, 0xa3, 0x26, 0xae, 0x67, 0xef, 0x6a, 0xe2, 0x7d, 0xf5, 0x70, 0xf8, 0x53, 0xdb, 0x5e, 0xd6, 0x49, 0xc1, 0x44, 0xcc, 0xf, 0x87, 0x2, 0x8a, 0x15, 0x9d, 0x18, 0x90, 0x3b, 0xb3, 0x36, 0xbe, 0x21, 0xa9, 0x2c, 0xa4, 0xb7, 0x3f, 0xba, 0x32, 0xad, 0x25, 0xa0, 0x28, 0x83, 0xb, 0x8e, 0x6, 0x99, 0x11, 0x94, 0x1c, 0xdf, 0x57, 0xd2, 0x5a, 0xc5, 0x4d, 0xc8, 0x40, 0xeb, 0x63, 0xe6, 0x6e, 0xf1, 0x79, 0xfc, 0x74, 0xda, 0x52, 0xd7, 0x5f, 0xc0, 0x48, 0xcd, 0x45, 0xee, 0x66, 0xe3, 0x6b, 0xf4, 0x7c, 0xf9, 0x71, 0xb2, 0x3a, 0xbf, 0x37, 0xa8, 0x20, 0xa5, 0x2d, 0x86, 0xe, 0x8b, 0x3, 0x9c, 0x14, 0x91, 0x19, 0xa, 0x82, 0x7, 0x8f, 0x10, 0x98, 0x1d, 0x95, 0x3e, 0xb6, 0x33, 0xbb, 0x24, 0xac, 0x29, 0xa1, 0x62, 0xea, 0x6f, 0xe7, 0x78, 0xf0, 0x75, 0xfd, 0x56, 0xde, 0x5b, 0xd3, 0x4c, 0xc4, 0x41, 0xc9},
- {0x0, 0x89, 0xf, 0x86, 0x1e, 0x97, 0x11, 0x98, 0x3c, 0xb5, 0x33, 0xba, 0x22, 0xab, 0x2d, 0xa4, 0x78, 0xf1, 0x77, 0xfe, 0x66, 0xef, 0x69, 0xe0, 0x44, 0xcd, 0x4b, 0xc2, 0x5a, 0xd3, 0x55, 0xdc, 0xf0, 0x79, 0xff, 0x76, 0xee, 0x67, 0xe1, 0x68, 0xcc, 0x45, 0xc3, 0x4a, 0xd2, 0x5b, 0xdd, 0x54, 0x88, 0x1, 0x87, 0xe, 0x96, 0x1f, 0x99, 0x10, 0xb4, 0x3d, 0xbb, 0x32, 0xaa, 0x23, 0xa5, 0x2c, 0xfd, 0x74, 0xf2, 0x7b, 0xe3, 0x6a, 0xec, 0x65, 0xc1, 0x48, 0xce, 0x47, 0xdf, 0x56, 0xd0, 0x59, 0x85, 0xc, 0x8a, 0x3, 0x9b, 0x12, 0x94, 0x1d, 0xb9, 0x30, 0xb6, 0x3f, 0xa7, 0x2e, 0xa8, 0x21, 0xd, 0x84, 0x2, 0x8b, 0x13, 0x9a, 0x1c, 0x95, 0x31, 0xb8, 0x3e, 0xb7, 0x2f, 0xa6, 0x20, 0xa9, 0x75, 0xfc, 0x7a, 0xf3, 0x6b, 0xe2, 0x64, 0xed, 0x49, 0xc0, 0x46, 0xcf, 0x57, 0xde, 0x58, 0xd1, 0xe7, 0x6e, 0xe8, 0x61, 0xf9, 0x70, 0xf6, 0x7f, 0xdb, 0x52, 0xd4, 0x5d, 0xc5, 0x4c, 0xca, 0x43, 0x9f, 0x16, 0x90, 0x19, 0x81, 0x8, 0x8e, 0x7, 0xa3, 0x2a, 0xac, 0x25, 0xbd, 0x34, 0xb2, 0x3b, 0x17, 0x9e, 0x18, 0x91, 0x9, 0x80, 0x6, 0x8f, 0x2b, 0xa2, 0x24, 0xad, 0x35, 0xbc, 0x3a, 0xb3, 0x6f, 0xe6, 0x60, 0xe9, 0x71, 0xf8, 0x7e, 0xf7, 0x53, 0xda, 0x5c, 0xd5, 0x4d, 0xc4, 0x42, 0xcb, 0x1a, 0x93, 0x15, 0x9c, 0x4, 0x8d, 0xb, 0x82, 0x26, 0xaf, 0x29, 0xa0, 0x38, 0xb1, 0x37, 0xbe, 0x62, 0xeb, 0x6d, 0xe4, 0x7c, 0xf5, 0x73, 0xfa, 0x5e, 0xd7, 0x51, 0xd8, 0x40, 0xc9, 0x4f, 0xc6, 0xea, 0x63, 0xe5, 0x6c, 0xf4, 0x7d, 0xfb, 0x72, 0xd6, 0x5f, 0xd9, 0x50, 0xc8, 0x41, 0xc7, 0x4e, 0x92, 0x1b, 0x9d, 0x14, 0x8c, 0x5, 0x83, 0xa, 0xae, 0x27, 0xa1, 0x28, 0xb0, 0x39, 0xbf, 0x36},
- {0x0, 0x8a, 0x9, 0x83, 0x12, 0x98, 0x1b, 0x91, 0x24, 0xae, 0x2d, 0xa7, 0x36, 0xbc, 0x3f, 0xb5, 0x48, 0xc2, 0x41, 0xcb, 0x5a, 0xd0, 0x53, 0xd9, 0x6c, 0xe6, 0x65, 0xef, 0x7e, 0xf4, 0x77, 0xfd, 0x90, 0x1a, 0x99, 0x13, 0x82, 0x8, 0x8b, 0x1, 0xb4, 0x3e, 0xbd, 0x37, 0xa6, 0x2c, 0xaf, 0x25, 0xd8, 0x52, 0xd1, 0x5b, 0xca, 0x40, 0xc3, 0x49, 0xfc, 0x76, 0xf5, 0x7f, 0xee, 0x64, 0xe7, 0x6d, 0x3d, 0xb7, 0x34, 0xbe, 0x2f, 0xa5, 0x26, 0xac, 0x19, 0x93, 0x10, 0x9a, 0xb, 0x81, 0x2, 0x88, 0x75, 0xff, 0x7c, 0xf6, 0x67, 0xed, 0x6e, 0xe4, 0x51, 0xdb, 0x58, 0xd2, 0x43, 0xc9, 0x4a, 0xc0, 0xad, 0x27, 0xa4, 0x2e, 0xbf, 0x35, 0xb6, 0x3c, 0x89, 0x3, 0x80, 0xa, 0x9b, 0x11, 0x92, 0x18, 0xe5, 0x6f, 0xec, 0x66, 0xf7, 0x7d, 0xfe, 0x74, 0xc1, 0x4b, 0xc8, 0x42, 0xd3, 0x59, 0xda, 0x50, 0x7a, 0xf0, 0x73, 0xf9, 0x68, 0xe2, 0x61, 0xeb, 0x5e, 0xd4, 0x57, 0xdd, 0x4c, 0xc6, 0x45, 0xcf, 0x32, 0xb8, 0x3b, 0xb1, 0x20, 0xaa, 0x29, 0xa3, 0x16, 0x9c, 0x1f, 0x95, 0x4, 0x8e, 0xd, 0x87, 0xea, 0x60, 0xe3, 0x69, 0xf8, 0x72, 0xf1, 0x7b, 0xce, 0x44, 0xc7, 0x4d, 0xdc, 0x56, 0xd5, 0x5f, 0xa2, 0x28, 0xab, 0x21, 0xb0, 0x3a, 0xb9, 0x33, 0x86, 0xc, 0x8f, 0x5, 0x94, 0x1e, 0x9d, 0x17, 0x47, 0xcd, 0x4e, 0xc4, 0x55, 0xdf, 0x5c, 0xd6, 0x63, 0xe9, 0x6a, 0xe0, 0x71, 0xfb, 0x78, 0xf2, 0xf, 0x85, 0x6, 0x8c, 0x1d, 0x97, 0x14, 0x9e, 0x2b, 0xa1, 0x22, 0xa8, 0x39, 0xb3, 0x30, 0xba, 0xd7, 0x5d, 0xde, 0x54, 0xc5, 0x4f, 0xcc, 0x46, 0xf3, 0x79, 0xfa, 0x70, 0xe1, 0x6b, 0xe8, 0x62, 0x9f, 0x15, 0x96, 0x1c, 0x8d, 0x7, 0x84, 0xe, 0xbb, 0x31, 0xb2, 0x38, 0xa9, 0x23, 0xa0, 0x2a},
- {0x0, 0x8b, 0xb, 0x80, 0x16, 0x9d, 0x1d, 0x96, 0x2c, 0xa7, 0x27, 0xac, 0x3a, 0xb1, 0x31, 0xba, 0x58, 0xd3, 0x53, 0xd8, 0x4e, 0xc5, 0x45, 0xce, 0x74, 0xff, 0x7f, 0xf4, 0x62, 0xe9, 0x69, 0xe2, 0xb0, 0x3b, 0xbb, 0x30, 0xa6, 0x2d, 0xad, 0x26, 0x9c, 0x17, 0x97, 0x1c, 0x8a, 0x1, 0x81, 0xa, 0xe8, 0x63, 0xe3, 0x68, 0xfe, 0x75, 0xf5, 0x7e, 0xc4, 0x4f, 0xcf, 0x44, 0xd2, 0x59, 0xd9, 0x52, 0x7d, 0xf6, 0x76, 0xfd, 0x6b, 0xe0, 0x60, 0xeb, 0x51, 0xda, 0x5a, 0xd1, 0x47, 0xcc, 0x4c, 0xc7, 0x25, 0xae, 0x2e, 0xa5, 0x33, 0xb8, 0x38, 0xb3, 0x9, 0x82, 0x2, 0x89, 0x1f, 0x94, 0x14, 0x9f, 0xcd, 0x46, 0xc6, 0x4d, 0xdb, 0x50, 0xd0, 0x5b, 0xe1, 0x6a, 0xea, 0x61, 0xf7, 0x7c, 0xfc, 0x77, 0x95, 0x1e, 0x9e, 0x15, 0x83, 0x8, 0x88, 0x3, 0xb9, 0x32, 0xb2, 0x39, 0xaf, 0x24, 0xa4, 0x2f, 0xfa, 0x71, 0xf1, 0x7a, 0xec, 0x67, 0xe7, 0x6c, 0xd6, 0x5d, 0xdd, 0x56, 0xc0, 0x4b, 0xcb, 0x40, 0xa2, 0x29, 0xa9, 0x22, 0xb4, 0x3f, 0xbf, 0x34, 0x8e, 0x5, 0x85, 0xe, 0x98, 0x13, 0x93, 0x18, 0x4a, 0xc1, 0x41, 0xca, 0x5c, 0xd7, 0x57, 0xdc, 0x66, 0xed, 0x6d, 0xe6, 0x70, 0xfb, 0x7b, 0xf0, 0x12, 0x99, 0x19, 0x92, 0x4, 0x8f, 0xf, 0x84, 0x3e, 0xb5, 0x35, 0xbe, 0x28, 0xa3, 0x23, 0xa8, 0x87, 0xc, 0x8c, 0x7, 0x91, 0x1a, 0x9a, 0x11, 0xab, 0x20, 0xa0, 0x2b, 0xbd, 0x36, 0xb6, 0x3d, 0xdf, 0x54, 0xd4, 0x5f, 0xc9, 0x42, 0xc2, 0x49, 0xf3, 0x78, 0xf8, 0x73, 0xe5, 0x6e, 0xee, 0x65, 0x37, 0xbc, 0x3c, 0xb7, 0x21, 0xaa, 0x2a, 0xa1, 0x1b, 0x90, 0x10, 0x9b, 0xd, 0x86, 0x6, 0x8d, 0x6f, 0xe4, 0x64, 0xef, 0x79, 0xf2, 0x72, 0xf9, 0x43, 0xc8, 0x48, 0xc3, 0x55, 0xde, 0x5e, 0xd5},
- {0x0, 0x8c, 0x5, 0x89, 0xa, 0x86, 0xf, 0x83, 0x14, 0x98, 0x11, 0x9d, 0x1e, 0x92, 0x1b, 0x97, 0x28, 0xa4, 0x2d, 0xa1, 0x22, 0xae, 0x27, 0xab, 0x3c, 0xb0, 0x39, 0xb5, 0x36, 0xba, 0x33, 0xbf, 0x50, 0xdc, 0x55, 0xd9, 0x5a, 0xd6, 0x5f, 0xd3, 0x44, 0xc8, 0x41, 0xcd, 0x4e, 0xc2, 0x4b, 0xc7, 0x78, 0xf4, 0x7d, 0xf1, 0x72, 0xfe, 0x77, 0xfb, 0x6c, 0xe0, 0x69, 0xe5, 0x66, 0xea, 0x63, 0xef, 0xa0, 0x2c, 0xa5, 0x29, 0xaa, 0x26, 0xaf, 0x23, 0xb4, 0x38, 0xb1, 0x3d, 0xbe, 0x32, 0xbb, 0x37, 0x88, 0x4, 0x8d, 0x1, 0x82, 0xe, 0x87, 0xb, 0x9c, 0x10, 0x99, 0x15, 0x96, 0x1a, 0x93, 0x1f, 0xf0, 0x7c, 0xf5, 0x79, 0xfa, 0x76, 0xff, 0x73, 0xe4, 0x68, 0xe1, 0x6d, 0xee, 0x62, 0xeb, 0x67, 0xd8, 0x54, 0xdd, 0x51, 0xd2, 0x5e, 0xd7, 0x5b, 0xcc, 0x40, 0xc9, 0x45, 0xc6, 0x4a, 0xc3, 0x4f, 0x5d, 0xd1, 0x58, 0xd4, 0x57, 0xdb, 0x52, 0xde, 0x49, 0xc5, 0x4c, 0xc0, 0x43, 0xcf, 0x46, 0xca, 0x75, 0xf9, 0x70, 0xfc, 0x7f, 0xf3, 0x7a, 0xf6, 0x61, 0xed, 0x64, 0xe8, 0x6b, 0xe7, 0x6e, 0xe2, 0xd, 0x81, 0x8, 0x84, 0x7, 0x8b, 0x2, 0x8e, 0x19, 0x95, 0x1c, 0x90, 0x13, 0x9f, 0x16, 0x9a, 0x25, 0xa9, 0x20, 0xac, 0x2f, 0xa3, 0x2a, 0xa6, 0x31, 0xbd, 0x34, 0xb8, 0x3b, 0xb7, 0x3e, 0xb2, 0xfd, 0x71, 0xf8, 0x74, 0xf7, 0x7b, 0xf2, 0x7e, 0xe9, 0x65, 0xec, 0x60, 0xe3, 0x6f, 0xe6, 0x6a, 0xd5, 0x59, 0xd0, 0x5c, 0xdf, 0x53, 0xda, 0x56, 0xc1, 0x4d, 0xc4, 0x48, 0xcb, 0x47, 0xce, 0x42, 0xad, 0x21, 0xa8, 0x24, 0xa7, 0x2b, 0xa2, 0x2e, 0xb9, 0x35, 0xbc, 0x30, 0xb3, 0x3f, 0xb6, 0x3a, 0x85, 0x9, 0x80, 0xc, 0x8f, 0x3, 0x8a, 0x6, 0x91, 0x1d, 0x94, 0x18, 0x9b, 0x17, 0x9e, 0x12},
- {0x0, 0x8d, 0x7, 0x8a, 0xe, 0x83, 0x9, 0x84, 0x1c, 0x91, 0x1b, 0x96, 0x12, 0x9f, 0x15, 0x98, 0x38, 0xb5, 0x3f, 0xb2, 0x36, 0xbb, 0x31, 0xbc, 0x24, 0xa9, 0x23, 0xae, 0x2a, 0xa7, 0x2d, 0xa0, 0x70, 0xfd, 0x77, 0xfa, 0x7e, 0xf3, 0x79, 0xf4, 0x6c, 0xe1, 0x6b, 0xe6, 0x62, 0xef, 0x65, 0xe8, 0x48, 0xc5, 0x4f, 0xc2, 0x46, 0xcb, 0x41, 0xcc, 0x54, 0xd9, 0x53, 0xde, 0x5a, 0xd7, 0x5d, 0xd0, 0xe0, 0x6d, 0xe7, 0x6a, 0xee, 0x63, 0xe9, 0x64, 0xfc, 0x71, 0xfb, 0x76, 0xf2, 0x7f, 0xf5, 0x78, 0xd8, 0x55, 0xdf, 0x52, 0xd6, 0x5b, 0xd1, 0x5c, 0xc4, 0x49, 0xc3, 0x4e, 0xca, 0x47, 0xcd, 0x40, 0x90, 0x1d, 0x97, 0x1a, 0x9e, 0x13, 0x99, 0x14, 0x8c, 0x1, 0x8b, 0x6, 0x82, 0xf, 0x85, 0x8, 0xa8, 0x25, 0xaf, 0x22, 0xa6, 0x2b, 0xa1, 0x2c, 0xb4, 0x39, 0xb3, 0x3e, 0xba, 0x37, 0xbd, 0x30, 0xdd, 0x50, 0xda, 0x57, 0xd3, 0x5e, 0xd4, 0x59, 0xc1, 0x4c, 0xc6, 0x4b, 0xcf, 0x42, 0xc8, 0x45, 0xe5, 0x68, 0xe2, 0x6f, 0xeb, 0x66, 0xec, 0x61, 0xf9, 0x74, 0xfe, 0x73, 0xf7, 0x7a, 0xf0, 0x7d, 0xad, 0x20, 0xaa, 0x27, 0xa3, 0x2e, 0xa4, 0x29, 0xb1, 0x3c, 0xb6, 0x3b, 0xbf, 0x32, 0xb8, 0x35, 0x95, 0x18, 0x92, 0x1f, 0x9b, 0x16, 0x9c, 0x11, 0x89, 0x4, 0x8e, 0x3, 0x87, 0xa, 0x80, 0xd, 0x3d, 0xb0, 0x3a, 0xb7, 0x33, 0xbe, 0x34, 0xb9, 0x21, 0xac, 0x26, 0xab, 0x2f, 0xa2, 0x28, 0xa5, 0x5, 0x88, 0x2, 0x8f, 0xb, 0x86, 0xc, 0x81, 0x19, 0x94, 0x1e, 0x93, 0x17, 0x9a, 0x10, 0x9d, 0x4d, 0xc0, 0x4a, 0xc7, 0x43, 0xce, 0x44, 0xc9, 0x51, 0xdc, 0x56, 0xdb, 0x5f, 0xd2, 0x58, 0xd5, 0x75, 0xf8, 0x72, 0xff, 0x7b, 0xf6, 0x7c, 0xf1, 0x69, 0xe4, 0x6e, 0xe3, 0x67, 0xea, 0x60, 0xed},
- {0x0, 0x8e, 0x1, 0x8f, 0x2, 0x8c, 0x3, 0x8d, 0x4, 0x8a, 0x5, 0x8b, 0x6, 0x88, 0x7, 0x89, 0x8, 0x86, 0x9, 0x87, 0xa, 0x84, 0xb, 0x85, 0xc, 0x82, 0xd, 0x83, 0xe, 0x80, 0xf, 0x81, 0x10, 0x9e, 0x11, 0x9f, 0x12, 0x9c, 0x13, 0x9d, 0x14, 0x9a, 0x15, 0x9b, 0x16, 0x98, 0x17, 0x99, 0x18, 0x96, 0x19, 0x97, 0x1a, 0x94, 0x1b, 0x95, 0x1c, 0x92, 0x1d, 0x93, 0x1e, 0x90, 0x1f, 0x91, 0x20, 0xae, 0x21, 0xaf, 0x22, 0xac, 0x23, 0xad, 0x24, 0xaa, 0x25, 0xab, 0x26, 0xa8, 0x27, 0xa9, 0x28, 0xa6, 0x29, 0xa7, 0x2a, 0xa4, 0x2b, 0xa5, 0x2c, 0xa2, 0x2d, 0xa3, 0x2e, 0xa0, 0x2f, 0xa1, 0x30, 0xbe, 0x31, 0xbf, 0x32, 0xbc, 0x33, 0xbd, 0x34, 0xba, 0x35, 0xbb, 0x36, 0xb8, 0x37, 0xb9, 0x38, 0xb6, 0x39, 0xb7, 0x3a, 0xb4, 0x3b, 0xb5, 0x3c, 0xb2, 0x3d, 0xb3, 0x3e, 0xb0, 0x3f, 0xb1, 0x40, 0xce, 0x41, 0xcf, 0x42, 0xcc, 0x43, 0xcd, 0x44, 0xca, 0x45, 0xcb, 0x46, 0xc8, 0x47, 0xc9, 0x48, 0xc6, 0x49, 0xc7, 0x4a, 0xc4, 0x4b, 0xc5, 0x4c, 0xc2, 0x4d, 0xc3, 0x4e, 0xc0, 0x4f, 0xc1, 0x50, 0xde, 0x51, 0xdf, 0x52, 0xdc, 0x53, 0xdd, 0x54, 0xda, 0x55, 0xdb, 0x56, 0xd8, 0x57, 0xd9, 0x58, 0xd6, 0x59, 0xd7, 0x5a, 0xd4, 0x5b, 0xd5, 0x5c, 0xd2, 0x5d, 0xd3, 0x5e, 0xd0, 0x5f, 0xd1, 0x60, 0xee, 0x61, 0xef, 0x62, 0xec, 0x63, 0xed, 0x64, 0xea, 0x65, 0xeb, 0x66, 0xe8, 0x67, 0xe9, 0x68, 0xe6, 0x69, 0xe7, 0x6a, 0xe4, 0x6b, 0xe5, 0x6c, 0xe2, 0x6d, 0xe3, 0x6e, 0xe0, 0x6f, 0xe1, 0x70, 0xfe, 0x71, 0xff, 0x72, 0xfc, 0x73, 0xfd, 0x74, 0xfa, 0x75, 0xfb, 0x76, 0xf8, 0x77, 0xf9, 0x78, 0xf6, 0x79, 0xf7, 0x7a, 0xf4, 0x7b, 0xf5, 0x7c, 0xf2, 0x7d, 0xf3, 0x7e, 0xf0, 0x7f, 0xf1},
- {0x0, 0x8f, 0x3, 0x8c, 0x6, 0x89, 0x5, 0x8a, 0xc, 0x83, 0xf, 0x80, 0xa, 0x85, 0x9, 0x86, 0x18, 0x97, 0x1b, 0x94, 0x1e, 0x91, 0x1d, 0x92, 0x14, 0x9b, 0x17, 0x98, 0x12, 0x9d, 0x11, 0x9e, 0x30, 0xbf, 0x33, 0xbc, 0x36, 0xb9, 0x35, 0xba, 0x3c, 0xb3, 0x3f, 0xb0, 0x3a, 0xb5, 0x39, 0xb6, 0x28, 0xa7, 0x2b, 0xa4, 0x2e, 0xa1, 0x2d, 0xa2, 0x24, 0xab, 0x27, 0xa8, 0x22, 0xad, 0x21, 0xae, 0x60, 0xef, 0x63, 0xec, 0x66, 0xe9, 0x65, 0xea, 0x6c, 0xe3, 0x6f, 0xe0, 0x6a, 0xe5, 0x69, 0xe6, 0x78, 0xf7, 0x7b, 0xf4, 0x7e, 0xf1, 0x7d, 0xf2, 0x74, 0xfb, 0x77, 0xf8, 0x72, 0xfd, 0x71, 0xfe, 0x50, 0xdf, 0x53, 0xdc, 0x56, 0xd9, 0x55, 0xda, 0x5c, 0xd3, 0x5f, 0xd0, 0x5a, 0xd5, 0x59, 0xd6, 0x48, 0xc7, 0x4b, 0xc4, 0x4e, 0xc1, 0x4d, 0xc2, 0x44, 0xcb, 0x47, 0xc8, 0x42, 0xcd, 0x41, 0xce, 0xc0, 0x4f, 0xc3, 0x4c, 0xc6, 0x49, 0xc5, 0x4a, 0xcc, 0x43, 0xcf, 0x40, 0xca, 0x45, 0xc9, 0x46, 0xd8, 0x57, 0xdb, 0x54, 0xde, 0x51, 0xdd, 0x52, 0xd4, 0x5b, 0xd7, 0x58, 0xd2, 0x5d, 0xd1, 0x5e, 0xf0, 0x7f, 0xf3, 0x7c, 0xf6, 0x79, 0xf5, 0x7a, 0xfc, 0x73, 0xff, 0x70, 0xfa, 0x75, 0xf9, 0x76, 0xe8, 0x67, 0xeb, 0x64, 0xee, 0x61, 0xed, 0x62, 0xe4, 0x6b, 0xe7, 0x68, 0xe2, 0x6d, 0xe1, 0x6e, 0xa0, 0x2f, 0xa3, 0x2c, 0xa6, 0x29, 0xa5, 0x2a, 0xac, 0x23, 0xaf, 0x20, 0xaa, 0x25, 0xa9, 0x26, 0xb8, 0x37, 0xbb, 0x34, 0xbe, 0x31, 0xbd, 0x32, 0xb4, 0x3b, 0xb7, 0x38, 0xb2, 0x3d, 0xb1, 0x3e, 0x90, 0x1f, 0x93, 0x1c, 0x96, 0x19, 0x95, 0x1a, 0x9c, 0x13, 0x9f, 0x10, 0x9a, 0x15, 0x99, 0x16, 0x88, 0x7, 0x8b, 0x4, 0x8e, 0x1, 0x8d, 0x2, 0x84, 0xb, 0x87, 0x8, 0x82, 0xd, 0x81, 0xe},
- {0x0, 0x90, 0x3d, 0xad, 0x7a, 0xea, 0x47, 0xd7, 0xf4, 0x64, 0xc9, 0x59, 0x8e, 0x1e, 0xb3, 0x23, 0xf5, 0x65, 0xc8, 0x58, 0x8f, 0x1f, 0xb2, 0x22, 0x1, 0x91, 0x3c, 0xac, 0x7b, 0xeb, 0x46, 0xd6, 0xf7, 0x67, 0xca, 0x5a, 0x8d, 0x1d, 0xb0, 0x20, 0x3, 0x93, 0x3e, 0xae, 0x79, 0xe9, 0x44, 0xd4, 0x2, 0x92, 0x3f, 0xaf, 0x78, 0xe8, 0x45, 0xd5, 0xf6, 0x66, 0xcb, 0x5b, 0x8c, 0x1c, 0xb1, 0x21, 0xf3, 0x63, 0xce, 0x5e, 0x89, 0x19, 0xb4, 0x24, 0x7, 0x97, 0x3a, 0xaa, 0x7d, 0xed, 0x40, 0xd0, 0x6, 0x96, 0x3b, 0xab, 0x7c, 0xec, 0x41, 0xd1, 0xf2, 0x62, 0xcf, 0x5f, 0x88, 0x18, 0xb5, 0x25, 0x4, 0x94, 0x39, 0xa9, 0x7e, 0xee, 0x43, 0xd3, 0xf0, 0x60, 0xcd, 0x5d, 0x8a, 0x1a, 0xb7, 0x27, 0xf1, 0x61, 0xcc, 0x5c, 0x8b, 0x1b, 0xb6, 0x26, 0x5, 0x95, 0x38, 0xa8, 0x7f, 0xef, 0x42, 0xd2, 0xfb, 0x6b, 0xc6, 0x56, 0x81, 0x11, 0xbc, 0x2c, 0xf, 0x9f, 0x32, 0xa2, 0x75, 0xe5, 0x48, 0xd8, 0xe, 0x9e, 0x33, 0xa3, 0x74, 0xe4, 0x49, 0xd9, 0xfa, 0x6a, 0xc7, 0x57, 0x80, 0x10, 0xbd, 0x2d, 0xc, 0x9c, 0x31, 0xa1, 0x76, 0xe6, 0x4b, 0xdb, 0xf8, 0x68, 0xc5, 0x55, 0x82, 0x12, 0xbf, 0x2f, 0xf9, 0x69, 0xc4, 0x54, 0x83, 0x13, 0xbe, 0x2e, 0xd, 0x9d, 0x30, 0xa0, 0x77, 0xe7, 0x4a, 0xda, 0x8, 0x98, 0x35, 0xa5, 0x72, 0xe2, 0x4f, 0xdf, 0xfc, 0x6c, 0xc1, 0x51, 0x86, 0x16, 0xbb, 0x2b, 0xfd, 0x6d, 0xc0, 0x50, 0x87, 0x17, 0xba, 0x2a, 0x9, 0x99, 0x34, 0xa4, 0x73, 0xe3, 0x4e, 0xde, 0xff, 0x6f, 0xc2, 0x52, 0x85, 0x15, 0xb8, 0x28, 0xb, 0x9b, 0x36, 0xa6, 0x71, 0xe1, 0x4c, 0xdc, 0xa, 0x9a, 0x37, 0xa7, 0x70, 0xe0, 0x4d, 0xdd, 0xfe, 0x6e, 0xc3, 0x53, 0x84, 0x14, 0xb9, 0x29},
- {0x0, 0x91, 0x3f, 0xae, 0x7e, 0xef, 0x41, 0xd0, 0xfc, 0x6d, 0xc3, 0x52, 0x82, 0x13, 0xbd, 0x2c, 0xe5, 0x74, 0xda, 0x4b, 0x9b, 0xa, 0xa4, 0x35, 0x19, 0x88, 0x26, 0xb7, 0x67, 0xf6, 0x58, 0xc9, 0xd7, 0x46, 0xe8, 0x79, 0xa9, 0x38, 0x96, 0x7, 0x2b, 0xba, 0x14, 0x85, 0x55, 0xc4, 0x6a, 0xfb, 0x32, 0xa3, 0xd, 0x9c, 0x4c, 0xdd, 0x73, 0xe2, 0xce, 0x5f, 0xf1, 0x60, 0xb0, 0x21, 0x8f, 0x1e, 0xb3, 0x22, 0x8c, 0x1d, 0xcd, 0x5c, 0xf2, 0x63, 0x4f, 0xde, 0x70, 0xe1, 0x31, 0xa0, 0xe, 0x9f, 0x56, 0xc7, 0x69, 0xf8, 0x28, 0xb9, 0x17, 0x86, 0xaa, 0x3b, 0x95, 0x4, 0xd4, 0x45, 0xeb, 0x7a, 0x64, 0xf5, 0x5b, 0xca, 0x1a, 0x8b, 0x25, 0xb4, 0x98, 0x9, 0xa7, 0x36, 0xe6, 0x77, 0xd9, 0x48, 0x81, 0x10, 0xbe, 0x2f, 0xff, 0x6e, 0xc0, 0x51, 0x7d, 0xec, 0x42, 0xd3, 0x3, 0x92, 0x3c, 0xad, 0x7b, 0xea, 0x44, 0xd5, 0x5, 0x94, 0x3a, 0xab, 0x87, 0x16, 0xb8, 0x29, 0xf9, 0x68, 0xc6, 0x57, 0x9e, 0xf, 0xa1, 0x30, 0xe0, 0x71, 0xdf, 0x4e, 0x62, 0xf3, 0x5d, 0xcc, 0x1c, 0x8d, 0x23, 0xb2, 0xac, 0x3d, 0x93, 0x2, 0xd2, 0x43, 0xed, 0x7c, 0x50, 0xc1, 0x6f, 0xfe, 0x2e, 0xbf, 0x11, 0x80, 0x49, 0xd8, 0x76, 0xe7, 0x37, 0xa6, 0x8, 0x99, 0xb5, 0x24, 0x8a, 0x1b, 0xcb, 0x5a, 0xf4, 0x65, 0xc8, 0x59, 0xf7, 0x66, 0xb6, 0x27, 0x89, 0x18, 0x34, 0xa5, 0xb, 0x9a, 0x4a, 0xdb, 0x75, 0xe4, 0x2d, 0xbc, 0x12, 0x83, 0x53, 0xc2, 0x6c, 0xfd, 0xd1, 0x40, 0xee, 0x7f, 0xaf, 0x3e, 0x90, 0x1, 0x1f, 0x8e, 0x20, 0xb1, 0x61, 0xf0, 0x5e, 0xcf, 0xe3, 0x72, 0xdc, 0x4d, 0x9d, 0xc, 0xa2, 0x33, 0xfa, 0x6b, 0xc5, 0x54, 0x84, 0x15, 0xbb, 0x2a, 0x6, 0x97, 0x39, 0xa8, 0x78, 0xe9, 0x47, 0xd6},
- {0x0, 0x92, 0x39, 0xab, 0x72, 0xe0, 0x4b, 0xd9, 0xe4, 0x76, 0xdd, 0x4f, 0x96, 0x4, 0xaf, 0x3d, 0xd5, 0x47, 0xec, 0x7e, 0xa7, 0x35, 0x9e, 0xc, 0x31, 0xa3, 0x8, 0x9a, 0x43, 0xd1, 0x7a, 0xe8, 0xb7, 0x25, 0x8e, 0x1c, 0xc5, 0x57, 0xfc, 0x6e, 0x53, 0xc1, 0x6a, 0xf8, 0x21, 0xb3, 0x18, 0x8a, 0x62, 0xf0, 0x5b, 0xc9, 0x10, 0x82, 0x29, 0xbb, 0x86, 0x14, 0xbf, 0x2d, 0xf4, 0x66, 0xcd, 0x5f, 0x73, 0xe1, 0x4a, 0xd8, 0x1, 0x93, 0x38, 0xaa, 0x97, 0x5, 0xae, 0x3c, 0xe5, 0x77, 0xdc, 0x4e, 0xa6, 0x34, 0x9f, 0xd, 0xd4, 0x46, 0xed, 0x7f, 0x42, 0xd0, 0x7b, 0xe9, 0x30, 0xa2, 0x9, 0x9b, 0xc4, 0x56, 0xfd, 0x6f, 0xb6, 0x24, 0x8f, 0x1d, 0x20, 0xb2, 0x19, 0x8b, 0x52, 0xc0, 0x6b, 0xf9, 0x11, 0x83, 0x28, 0xba, 0x63, 0xf1, 0x5a, 0xc8, 0xf5, 0x67, 0xcc, 0x5e, 0x87, 0x15, 0xbe, 0x2c, 0xe6, 0x74, 0xdf, 0x4d, 0x94, 0x6, 0xad, 0x3f, 0x2, 0x90, 0x3b, 0xa9, 0x70, 0xe2, 0x49, 0xdb, 0x33, 0xa1, 0xa, 0x98, 0x41, 0xd3, 0x78, 0xea, 0xd7, 0x45, 0xee, 0x7c, 0xa5, 0x37, 0x9c, 0xe, 0x51, 0xc3, 0x68, 0xfa, 0x23, 0xb1, 0x1a, 0x88, 0xb5, 0x27, 0x8c, 0x1e, 0xc7, 0x55, 0xfe, 0x6c, 0x84, 0x16, 0xbd, 0x2f, 0xf6, 0x64, 0xcf, 0x5d, 0x60, 0xf2, 0x59, 0xcb, 0x12, 0x80, 0x2b, 0xb9, 0x95, 0x7, 0xac, 0x3e, 0xe7, 0x75, 0xde, 0x4c, 0x71, 0xe3, 0x48, 0xda, 0x3, 0x91, 0x3a, 0xa8, 0x40, 0xd2, 0x79, 0xeb, 0x32, 0xa0, 0xb, 0x99, 0xa4, 0x36, 0x9d, 0xf, 0xd6, 0x44, 0xef, 0x7d, 0x22, 0xb0, 0x1b, 0x89, 0x50, 0xc2, 0x69, 0xfb, 0xc6, 0x54, 0xff, 0x6d, 0xb4, 0x26, 0x8d, 0x1f, 0xf7, 0x65, 0xce, 0x5c, 0x85, 0x17, 0xbc, 0x2e, 0x13, 0x81, 0x2a, 0xb8, 0x61, 0xf3, 0x58, 0xca},
- {0x0, 0x93, 0x3b, 0xa8, 0x76, 0xe5, 0x4d, 0xde, 0xec, 0x7f, 0xd7, 0x44, 0x9a, 0x9, 0xa1, 0x32, 0xc5, 0x56, 0xfe, 0x6d, 0xb3, 0x20, 0x88, 0x1b, 0x29, 0xba, 0x12, 0x81, 0x5f, 0xcc, 0x64, 0xf7, 0x97, 0x4, 0xac, 0x3f, 0xe1, 0x72, 0xda, 0x49, 0x7b, 0xe8, 0x40, 0xd3, 0xd, 0x9e, 0x36, 0xa5, 0x52, 0xc1, 0x69, 0xfa, 0x24, 0xb7, 0x1f, 0x8c, 0xbe, 0x2d, 0x85, 0x16, 0xc8, 0x5b, 0xf3, 0x60, 0x33, 0xa0, 0x8, 0x9b, 0x45, 0xd6, 0x7e, 0xed, 0xdf, 0x4c, 0xe4, 0x77, 0xa9, 0x3a, 0x92, 0x1, 0xf6, 0x65, 0xcd, 0x5e, 0x80, 0x13, 0xbb, 0x28, 0x1a, 0x89, 0x21, 0xb2, 0x6c, 0xff, 0x57, 0xc4, 0xa4, 0x37, 0x9f, 0xc, 0xd2, 0x41, 0xe9, 0x7a, 0x48, 0xdb, 0x73, 0xe0, 0x3e, 0xad, 0x5, 0x96, 0x61, 0xf2, 0x5a, 0xc9, 0x17, 0x84, 0x2c, 0xbf, 0x8d, 0x1e, 0xb6, 0x25, 0xfb, 0x68, 0xc0, 0x53, 0x66, 0xf5, 0x5d, 0xce, 0x10, 0x83, 0x2b, 0xb8, 0x8a, 0x19, 0xb1, 0x22, 0xfc, 0x6f, 0xc7, 0x54, 0xa3, 0x30, 0x98, 0xb, 0xd5, 0x46, 0xee, 0x7d, 0x4f, 0xdc, 0x74, 0xe7, 0x39, 0xaa, 0x2, 0x91, 0xf1, 0x62, 0xca, 0x59, 0x87, 0x14, 0xbc, 0x2f, 0x1d, 0x8e, 0x26, 0xb5, 0x6b, 0xf8, 0x50, 0xc3, 0x34, 0xa7, 0xf, 0x9c, 0x42, 0xd1, 0x79, 0xea, 0xd8, 0x4b, 0xe3, 0x70, 0xae, 0x3d, 0x95, 0x6, 0x55, 0xc6, 0x6e, 0xfd, 0x23, 0xb0, 0x18, 0x8b, 0xb9, 0x2a, 0x82, 0x11, 0xcf, 0x5c, 0xf4, 0x67, 0x90, 0x3, 0xab, 0x38, 0xe6, 0x75, 0xdd, 0x4e, 0x7c, 0xef, 0x47, 0xd4, 0xa, 0x99, 0x31, 0xa2, 0xc2, 0x51, 0xf9, 0x6a, 0xb4, 0x27, 0x8f, 0x1c, 0x2e, 0xbd, 0x15, 0x86, 0x58, 0xcb, 0x63, 0xf0, 0x7, 0x94, 0x3c, 0xaf, 0x71, 0xe2, 0x4a, 0xd9, 0xeb, 0x78, 0xd0, 0x43, 0x9d, 0xe, 0xa6, 0x35},
- {0x0, 0x94, 0x35, 0xa1, 0x6a, 0xfe, 0x5f, 0xcb, 0xd4, 0x40, 0xe1, 0x75, 0xbe, 0x2a, 0x8b, 0x1f, 0xb5, 0x21, 0x80, 0x14, 0xdf, 0x4b, 0xea, 0x7e, 0x61, 0xf5, 0x54, 0xc0, 0xb, 0x9f, 0x3e, 0xaa, 0x77, 0xe3, 0x42, 0xd6, 0x1d, 0x89, 0x28, 0xbc, 0xa3, 0x37, 0x96, 0x2, 0xc9, 0x5d, 0xfc, 0x68, 0xc2, 0x56, 0xf7, 0x63, 0xa8, 0x3c, 0x9d, 0x9, 0x16, 0x82, 0x23, 0xb7, 0x7c, 0xe8, 0x49, 0xdd, 0xee, 0x7a, 0xdb, 0x4f, 0x84, 0x10, 0xb1, 0x25, 0x3a, 0xae, 0xf, 0x9b, 0x50, 0xc4, 0x65, 0xf1, 0x5b, 0xcf, 0x6e, 0xfa, 0x31, 0xa5, 0x4, 0x90, 0x8f, 0x1b, 0xba, 0x2e, 0xe5, 0x71, 0xd0, 0x44, 0x99, 0xd, 0xac, 0x38, 0xf3, 0x67, 0xc6, 0x52, 0x4d, 0xd9, 0x78, 0xec, 0x27, 0xb3, 0x12, 0x86, 0x2c, 0xb8, 0x19, 0x8d, 0x46, 0xd2, 0x73, 0xe7, 0xf8, 0x6c, 0xcd, 0x59, 0x92, 0x6, 0xa7, 0x33, 0xc1, 0x55, 0xf4, 0x60, 0xab, 0x3f, 0x9e, 0xa, 0x15, 0x81, 0x20, 0xb4, 0x7f, 0xeb, 0x4a, 0xde, 0x74, 0xe0, 0x41, 0xd5, 0x1e, 0x8a, 0x2b, 0xbf, 0xa0, 0x34, 0x95, 0x1, 0xca, 0x5e, 0xff, 0x6b, 0xb6, 0x22, 0x83, 0x17, 0xdc, 0x48, 0xe9, 0x7d, 0x62, 0xf6, 0x57, 0xc3, 0x8, 0x9c, 0x3d, 0xa9, 0x3, 0x97, 0x36, 0xa2, 0x69, 0xfd, 0x5c, 0xc8, 0xd7, 0x43, 0xe2, 0x76, 0xbd, 0x29, 0x88, 0x1c, 0x2f, 0xbb, 0x1a, 0x8e, 0x45, 0xd1, 0x70, 0xe4, 0xfb, 0x6f, 0xce, 0x5a, 0x91, 0x5, 0xa4, 0x30, 0x9a, 0xe, 0xaf, 0x3b, 0xf0, 0x64, 0xc5, 0x51, 0x4e, 0xda, 0x7b, 0xef, 0x24, 0xb0, 0x11, 0x85, 0x58, 0xcc, 0x6d, 0xf9, 0x32, 0xa6, 0x7, 0x93, 0x8c, 0x18, 0xb9, 0x2d, 0xe6, 0x72, 0xd3, 0x47, 0xed, 0x79, 0xd8, 0x4c, 0x87, 0x13, 0xb2, 0x26, 0x39, 0xad, 0xc, 0x98, 0x53, 0xc7, 0x66, 0xf2},
- {0x0, 0x95, 0x37, 0xa2, 0x6e, 0xfb, 0x59, 0xcc, 0xdc, 0x49, 0xeb, 0x7e, 0xb2, 0x27, 0x85, 0x10, 0xa5, 0x30, 0x92, 0x7, 0xcb, 0x5e, 0xfc, 0x69, 0x79, 0xec, 0x4e, 0xdb, 0x17, 0x82, 0x20, 0xb5, 0x57, 0xc2, 0x60, 0xf5, 0x39, 0xac, 0xe, 0x9b, 0x8b, 0x1e, 0xbc, 0x29, 0xe5, 0x70, 0xd2, 0x47, 0xf2, 0x67, 0xc5, 0x50, 0x9c, 0x9, 0xab, 0x3e, 0x2e, 0xbb, 0x19, 0x8c, 0x40, 0xd5, 0x77, 0xe2, 0xae, 0x3b, 0x99, 0xc, 0xc0, 0x55, 0xf7, 0x62, 0x72, 0xe7, 0x45, 0xd0, 0x1c, 0x89, 0x2b, 0xbe, 0xb, 0x9e, 0x3c, 0xa9, 0x65, 0xf0, 0x52, 0xc7, 0xd7, 0x42, 0xe0, 0x75, 0xb9, 0x2c, 0x8e, 0x1b, 0xf9, 0x6c, 0xce, 0x5b, 0x97, 0x2, 0xa0, 0x35, 0x25, 0xb0, 0x12, 0x87, 0x4b, 0xde, 0x7c, 0xe9, 0x5c, 0xc9, 0x6b, 0xfe, 0x32, 0xa7, 0x5, 0x90, 0x80, 0x15, 0xb7, 0x22, 0xee, 0x7b, 0xd9, 0x4c, 0x41, 0xd4, 0x76, 0xe3, 0x2f, 0xba, 0x18, 0x8d, 0x9d, 0x8, 0xaa, 0x3f, 0xf3, 0x66, 0xc4, 0x51, 0xe4, 0x71, 0xd3, 0x46, 0x8a, 0x1f, 0xbd, 0x28, 0x38, 0xad, 0xf, 0x9a, 0x56, 0xc3, 0x61, 0xf4, 0x16, 0x83, 0x21, 0xb4, 0x78, 0xed, 0x4f, 0xda, 0xca, 0x5f, 0xfd, 0x68, 0xa4, 0x31, 0x93, 0x6, 0xb3, 0x26, 0x84, 0x11, 0xdd, 0x48, 0xea, 0x7f, 0x6f, 0xfa, 0x58, 0xcd, 0x1, 0x94, 0x36, 0xa3, 0xef, 0x7a, 0xd8, 0x4d, 0x81, 0x14, 0xb6, 0x23, 0x33, 0xa6, 0x4, 0x91, 0x5d, 0xc8, 0x6a, 0xff, 0x4a, 0xdf, 0x7d, 0xe8, 0x24, 0xb1, 0x13, 0x86, 0x96, 0x3, 0xa1, 0x34, 0xf8, 0x6d, 0xcf, 0x5a, 0xb8, 0x2d, 0x8f, 0x1a, 0xd6, 0x43, 0xe1, 0x74, 0x64, 0xf1, 0x53, 0xc6, 0xa, 0x9f, 0x3d, 0xa8, 0x1d, 0x88, 0x2a, 0xbf, 0x73, 0xe6, 0x44, 0xd1, 0xc1, 0x54, 0xf6, 0x63, 0xaf, 0x3a, 0x98, 0xd},
- {0x0, 0x96, 0x31, 0xa7, 0x62, 0xf4, 0x53, 0xc5, 0xc4, 0x52, 0xf5, 0x63, 0xa6, 0x30, 0x97, 0x1, 0x95, 0x3, 0xa4, 0x32, 0xf7, 0x61, 0xc6, 0x50, 0x51, 0xc7, 0x60, 0xf6, 0x33, 0xa5, 0x2, 0x94, 0x37, 0xa1, 0x6, 0x90, 0x55, 0xc3, 0x64, 0xf2, 0xf3, 0x65, 0xc2, 0x54, 0x91, 0x7, 0xa0, 0x36, 0xa2, 0x34, 0x93, 0x5, 0xc0, 0x56, 0xf1, 0x67, 0x66, 0xf0, 0x57, 0xc1, 0x4, 0x92, 0x35, 0xa3, 0x6e, 0xf8, 0x5f, 0xc9, 0xc, 0x9a, 0x3d, 0xab, 0xaa, 0x3c, 0x9b, 0xd, 0xc8, 0x5e, 0xf9, 0x6f, 0xfb, 0x6d, 0xca, 0x5c, 0x99, 0xf, 0xa8, 0x3e, 0x3f, 0xa9, 0xe, 0x98, 0x5d, 0xcb, 0x6c, 0xfa, 0x59, 0xcf, 0x68, 0xfe, 0x3b, 0xad, 0xa, 0x9c, 0x9d, 0xb, 0xac, 0x3a, 0xff, 0x69, 0xce, 0x58, 0xcc, 0x5a, 0xfd, 0x6b, 0xae, 0x38, 0x9f, 0x9, 0x8, 0x9e, 0x39, 0xaf, 0x6a, 0xfc, 0x5b, 0xcd, 0xdc, 0x4a, 0xed, 0x7b, 0xbe, 0x28, 0x8f, 0x19, 0x18, 0x8e, 0x29, 0xbf, 0x7a, 0xec, 0x4b, 0xdd, 0x49, 0xdf, 0x78, 0xee, 0x2b, 0xbd, 0x1a, 0x8c, 0x8d, 0x1b, 0xbc, 0x2a, 0xef, 0x79, 0xde, 0x48, 0xeb, 0x7d, 0xda, 0x4c, 0x89, 0x1f, 0xb8, 0x2e, 0x2f, 0xb9, 0x1e, 0x88, 0x4d, 0xdb, 0x7c, 0xea, 0x7e, 0xe8, 0x4f, 0xd9, 0x1c, 0x8a, 0x2d, 0xbb, 0xba, 0x2c, 0x8b, 0x1d, 0xd8, 0x4e, 0xe9, 0x7f, 0xb2, 0x24, 0x83, 0x15, 0xd0, 0x46, 0xe1, 0x77, 0x76, 0xe0, 0x47, 0xd1, 0x14, 0x82, 0x25, 0xb3, 0x27, 0xb1, 0x16, 0x80, 0x45, 0xd3, 0x74, 0xe2, 0xe3, 0x75, 0xd2, 0x44, 0x81, 0x17, 0xb0, 0x26, 0x85, 0x13, 0xb4, 0x22, 0xe7, 0x71, 0xd6, 0x40, 0x41, 0xd7, 0x70, 0xe6, 0x23, 0xb5, 0x12, 0x84, 0x10, 0x86, 0x21, 0xb7, 0x72, 0xe4, 0x43, 0xd5, 0xd4, 0x42, 0xe5, 0x73, 0xb6, 0x20, 0x87, 0x11},
- {0x0, 0x97, 0x33, 0xa4, 0x66, 0xf1, 0x55, 0xc2, 0xcc, 0x5b, 0xff, 0x68, 0xaa, 0x3d, 0x99, 0xe, 0x85, 0x12, 0xb6, 0x21, 0xe3, 0x74, 0xd0, 0x47, 0x49, 0xde, 0x7a, 0xed, 0x2f, 0xb8, 0x1c, 0x8b, 0x17, 0x80, 0x24, 0xb3, 0x71, 0xe6, 0x42, 0xd5, 0xdb, 0x4c, 0xe8, 0x7f, 0xbd, 0x2a, 0x8e, 0x19, 0x92, 0x5, 0xa1, 0x36, 0xf4, 0x63, 0xc7, 0x50, 0x5e, 0xc9, 0x6d, 0xfa, 0x38, 0xaf, 0xb, 0x9c, 0x2e, 0xb9, 0x1d, 0x8a, 0x48, 0xdf, 0x7b, 0xec, 0xe2, 0x75, 0xd1, 0x46, 0x84, 0x13, 0xb7, 0x20, 0xab, 0x3c, 0x98, 0xf, 0xcd, 0x5a, 0xfe, 0x69, 0x67, 0xf0, 0x54, 0xc3, 0x1, 0x96, 0x32, 0xa5, 0x39, 0xae, 0xa, 0x9d, 0x5f, 0xc8, 0x6c, 0xfb, 0xf5, 0x62, 0xc6, 0x51, 0x93, 0x4, 0xa0, 0x37, 0xbc, 0x2b, 0x8f, 0x18, 0xda, 0x4d, 0xe9, 0x7e, 0x70, 0xe7, 0x43, 0xd4, 0x16, 0x81, 0x25, 0xb2, 0x5c, 0xcb, 0x6f, 0xf8, 0x3a, 0xad, 0x9, 0x9e, 0x90, 0x7, 0xa3, 0x34, 0xf6, 0x61, 0xc5, 0x52, 0xd9, 0x4e, 0xea, 0x7d, 0xbf, 0x28, 0x8c, 0x1b, 0x15, 0x82, 0x26, 0xb1, 0x73, 0xe4, 0x40, 0xd7, 0x4b, 0xdc, 0x78, 0xef, 0x2d, 0xba, 0x1e, 0x89, 0x87, 0x10, 0xb4, 0x23, 0xe1, 0x76, 0xd2, 0x45, 0xce, 0x59, 0xfd, 0x6a, 0xa8, 0x3f, 0x9b, 0xc, 0x2, 0x95, 0x31, 0xa6, 0x64, 0xf3, 0x57, 0xc0, 0x72, 0xe5, 0x41, 0xd6, 0x14, 0x83, 0x27, 0xb0, 0xbe, 0x29, 0x8d, 0x1a, 0xd8, 0x4f, 0xeb, 0x7c, 0xf7, 0x60, 0xc4, 0x53, 0x91, 0x6, 0xa2, 0x35, 0x3b, 0xac, 0x8, 0x9f, 0x5d, 0xca, 0x6e, 0xf9, 0x65, 0xf2, 0x56, 0xc1, 0x3, 0x94, 0x30, 0xa7, 0xa9, 0x3e, 0x9a, 0xd, 0xcf, 0x58, 0xfc, 0x6b, 0xe0, 0x77, 0xd3, 0x44, 0x86, 0x11, 0xb5, 0x22, 0x2c, 0xbb, 0x1f, 0x88, 0x4a, 0xdd, 0x79, 0xee},
- {0x0, 0x98, 0x2d, 0xb5, 0x5a, 0xc2, 0x77, 0xef, 0xb4, 0x2c, 0x99, 0x1, 0xee, 0x76, 0xc3, 0x5b, 0x75, 0xed, 0x58, 0xc0, 0x2f, 0xb7, 0x2, 0x9a, 0xc1, 0x59, 0xec, 0x74, 0x9b, 0x3, 0xb6, 0x2e, 0xea, 0x72, 0xc7, 0x5f, 0xb0, 0x28, 0x9d, 0x5, 0x5e, 0xc6, 0x73, 0xeb, 0x4, 0x9c, 0x29, 0xb1, 0x9f, 0x7, 0xb2, 0x2a, 0xc5, 0x5d, 0xe8, 0x70, 0x2b, 0xb3, 0x6, 0x9e, 0x71, 0xe9, 0x5c, 0xc4, 0xc9, 0x51, 0xe4, 0x7c, 0x93, 0xb, 0xbe, 0x26, 0x7d, 0xe5, 0x50, 0xc8, 0x27, 0xbf, 0xa, 0x92, 0xbc, 0x24, 0x91, 0x9, 0xe6, 0x7e, 0xcb, 0x53, 0x8, 0x90, 0x25, 0xbd, 0x52, 0xca, 0x7f, 0xe7, 0x23, 0xbb, 0xe, 0x96, 0x79, 0xe1, 0x54, 0xcc, 0x97, 0xf, 0xba, 0x22, 0xcd, 0x55, 0xe0, 0x78, 0x56, 0xce, 0x7b, 0xe3, 0xc, 0x94, 0x21, 0xb9, 0xe2, 0x7a, 0xcf, 0x57, 0xb8, 0x20, 0x95, 0xd, 0x8f, 0x17, 0xa2, 0x3a, 0xd5, 0x4d, 0xf8, 0x60, 0x3b, 0xa3, 0x16, 0x8e, 0x61, 0xf9, 0x4c, 0xd4, 0xfa, 0x62, 0xd7, 0x4f, 0xa0, 0x38, 0x8d, 0x15, 0x4e, 0xd6, 0x63, 0xfb, 0x14, 0x8c, 0x39, 0xa1, 0x65, 0xfd, 0x48, 0xd0, 0x3f, 0xa7, 0x12, 0x8a, 0xd1, 0x49, 0xfc, 0x64, 0x8b, 0x13, 0xa6, 0x3e, 0x10, 0x88, 0x3d, 0xa5, 0x4a, 0xd2, 0x67, 0xff, 0xa4, 0x3c, 0x89, 0x11, 0xfe, 0x66, 0xd3, 0x4b, 0x46, 0xde, 0x6b, 0xf3, 0x1c, 0x84, 0x31, 0xa9, 0xf2, 0x6a, 0xdf, 0x47, 0xa8, 0x30, 0x85, 0x1d, 0x33, 0xab, 0x1e, 0x86, 0x69, 0xf1, 0x44, 0xdc, 0x87, 0x1f, 0xaa, 0x32, 0xdd, 0x45, 0xf0, 0x68, 0xac, 0x34, 0x81, 0x19, 0xf6, 0x6e, 0xdb, 0x43, 0x18, 0x80, 0x35, 0xad, 0x42, 0xda, 0x6f, 0xf7, 0xd9, 0x41, 0xf4, 0x6c, 0x83, 0x1b, 0xae, 0x36, 0x6d, 0xf5, 0x40, 0xd8, 0x37, 0xaf, 0x1a, 0x82},
- {0x0, 0x99, 0x2f, 0xb6, 0x5e, 0xc7, 0x71, 0xe8, 0xbc, 0x25, 0x93, 0xa, 0xe2, 0x7b, 0xcd, 0x54, 0x65, 0xfc, 0x4a, 0xd3, 0x3b, 0xa2, 0x14, 0x8d, 0xd9, 0x40, 0xf6, 0x6f, 0x87, 0x1e, 0xa8, 0x31, 0xca, 0x53, 0xe5, 0x7c, 0x94, 0xd, 0xbb, 0x22, 0x76, 0xef, 0x59, 0xc0, 0x28, 0xb1, 0x7, 0x9e, 0xaf, 0x36, 0x80, 0x19, 0xf1, 0x68, 0xde, 0x47, 0x13, 0x8a, 0x3c, 0xa5, 0x4d, 0xd4, 0x62, 0xfb, 0x89, 0x10, 0xa6, 0x3f, 0xd7, 0x4e, 0xf8, 0x61, 0x35, 0xac, 0x1a, 0x83, 0x6b, 0xf2, 0x44, 0xdd, 0xec, 0x75, 0xc3, 0x5a, 0xb2, 0x2b, 0x9d, 0x4, 0x50, 0xc9, 0x7f, 0xe6, 0xe, 0x97, 0x21, 0xb8, 0x43, 0xda, 0x6c, 0xf5, 0x1d, 0x84, 0x32, 0xab, 0xff, 0x66, 0xd0, 0x49, 0xa1, 0x38, 0x8e, 0x17, 0x26, 0xbf, 0x9, 0x90, 0x78, 0xe1, 0x57, 0xce, 0x9a, 0x3, 0xb5, 0x2c, 0xc4, 0x5d, 0xeb, 0x72, 0xf, 0x96, 0x20, 0xb9, 0x51, 0xc8, 0x7e, 0xe7, 0xb3, 0x2a, 0x9c, 0x5, 0xed, 0x74, 0xc2, 0x5b, 0x6a, 0xf3, 0x45, 0xdc, 0x34, 0xad, 0x1b, 0x82, 0xd6, 0x4f, 0xf9, 0x60, 0x88, 0x11, 0xa7, 0x3e, 0xc5, 0x5c, 0xea, 0x73, 0x9b, 0x2, 0xb4, 0x2d, 0x79, 0xe0, 0x56, 0xcf, 0x27, 0xbe, 0x8, 0x91, 0xa0, 0x39, 0x8f, 0x16, 0xfe, 0x67, 0xd1, 0x48, 0x1c, 0x85, 0x33, 0xaa, 0x42, 0xdb, 0x6d, 0xf4, 0x86, 0x1f, 0xa9, 0x30, 0xd8, 0x41, 0xf7, 0x6e, 0x3a, 0xa3, 0x15, 0x8c, 0x64, 0xfd, 0x4b, 0xd2, 0xe3, 0x7a, 0xcc, 0x55, 0xbd, 0x24, 0x92, 0xb, 0x5f, 0xc6, 0x70, 0xe9, 0x1, 0x98, 0x2e, 0xb7, 0x4c, 0xd5, 0x63, 0xfa, 0x12, 0x8b, 0x3d, 0xa4, 0xf0, 0x69, 0xdf, 0x46, 0xae, 0x37, 0x81, 0x18, 0x29, 0xb0, 0x6, 0x9f, 0x77, 0xee, 0x58, 0xc1, 0x95, 0xc, 0xba, 0x23, 0xcb, 0x52, 0xe4, 0x7d},
- {0x0, 0x9a, 0x29, 0xb3, 0x52, 0xc8, 0x7b, 0xe1, 0xa4, 0x3e, 0x8d, 0x17, 0xf6, 0x6c, 0xdf, 0x45, 0x55, 0xcf, 0x7c, 0xe6, 0x7, 0x9d, 0x2e, 0xb4, 0xf1, 0x6b, 0xd8, 0x42, 0xa3, 0x39, 0x8a, 0x10, 0xaa, 0x30, 0x83, 0x19, 0xf8, 0x62, 0xd1, 0x4b, 0xe, 0x94, 0x27, 0xbd, 0x5c, 0xc6, 0x75, 0xef, 0xff, 0x65, 0xd6, 0x4c, 0xad, 0x37, 0x84, 0x1e, 0x5b, 0xc1, 0x72, 0xe8, 0x9, 0x93, 0x20, 0xba, 0x49, 0xd3, 0x60, 0xfa, 0x1b, 0x81, 0x32, 0xa8, 0xed, 0x77, 0xc4, 0x5e, 0xbf, 0x25, 0x96, 0xc, 0x1c, 0x86, 0x35, 0xaf, 0x4e, 0xd4, 0x67, 0xfd, 0xb8, 0x22, 0x91, 0xb, 0xea, 0x70, 0xc3, 0x59, 0xe3, 0x79, 0xca, 0x50, 0xb1, 0x2b, 0x98, 0x2, 0x47, 0xdd, 0x6e, 0xf4, 0x15, 0x8f, 0x3c, 0xa6, 0xb6, 0x2c, 0x9f, 0x5, 0xe4, 0x7e, 0xcd, 0x57, 0x12, 0x88, 0x3b, 0xa1, 0x40, 0xda, 0x69, 0xf3, 0x92, 0x8, 0xbb, 0x21, 0xc0, 0x5a, 0xe9, 0x73, 0x36, 0xac, 0x1f, 0x85, 0x64, 0xfe, 0x4d, 0xd7, 0xc7, 0x5d, 0xee, 0x74, 0x95, 0xf, 0xbc, 0x26, 0x63, 0xf9, 0x4a, 0xd0, 0x31, 0xab, 0x18, 0x82, 0x38, 0xa2, 0x11, 0x8b, 0x6a, 0xf0, 0x43, 0xd9, 0x9c, 0x6, 0xb5, 0x2f, 0xce, 0x54, 0xe7, 0x7d, 0x6d, 0xf7, 0x44, 0xde, 0x3f, 0xa5, 0x16, 0x8c, 0xc9, 0x53, 0xe0, 0x7a, 0x9b, 0x1, 0xb2, 0x28, 0xdb, 0x41, 0xf2, 0x68, 0x89, 0x13, 0xa0, 0x3a, 0x7f, 0xe5, 0x56, 0xcc, 0x2d, 0xb7, 0x4, 0x9e, 0x8e, 0x14, 0xa7, 0x3d, 0xdc, 0x46, 0xf5, 0x6f, 0x2a, 0xb0, 0x3, 0x99, 0x78, 0xe2, 0x51, 0xcb, 0x71, 0xeb, 0x58, 0xc2, 0x23, 0xb9, 0xa, 0x90, 0xd5, 0x4f, 0xfc, 0x66, 0x87, 0x1d, 0xae, 0x34, 0x24, 0xbe, 0xd, 0x97, 0x76, 0xec, 0x5f, 0xc5, 0x80, 0x1a, 0xa9, 0x33, 0xd2, 0x48, 0xfb, 0x61},
- {0x0, 0x9b, 0x2b, 0xb0, 0x56, 0xcd, 0x7d, 0xe6, 0xac, 0x37, 0x87, 0x1c, 0xfa, 0x61, 0xd1, 0x4a, 0x45, 0xde, 0x6e, 0xf5, 0x13, 0x88, 0x38, 0xa3, 0xe9, 0x72, 0xc2, 0x59, 0xbf, 0x24, 0x94, 0xf, 0x8a, 0x11, 0xa1, 0x3a, 0xdc, 0x47, 0xf7, 0x6c, 0x26, 0xbd, 0xd, 0x96, 0x70, 0xeb, 0x5b, 0xc0, 0xcf, 0x54, 0xe4, 0x7f, 0x99, 0x2, 0xb2, 0x29, 0x63, 0xf8, 0x48, 0xd3, 0x35, 0xae, 0x1e, 0x85, 0x9, 0x92, 0x22, 0xb9, 0x5f, 0xc4, 0x74, 0xef, 0xa5, 0x3e, 0x8e, 0x15, 0xf3, 0x68, 0xd8, 0x43, 0x4c, 0xd7, 0x67, 0xfc, 0x1a, 0x81, 0x31, 0xaa, 0xe0, 0x7b, 0xcb, 0x50, 0xb6, 0x2d, 0x9d, 0x6, 0x83, 0x18, 0xa8, 0x33, 0xd5, 0x4e, 0xfe, 0x65, 0x2f, 0xb4, 0x4, 0x9f, 0x79, 0xe2, 0x52, 0xc9, 0xc6, 0x5d, 0xed, 0x76, 0x90, 0xb, 0xbb, 0x20, 0x6a, 0xf1, 0x41, 0xda, 0x3c, 0xa7, 0x17, 0x8c, 0x12, 0x89, 0x39, 0xa2, 0x44, 0xdf, 0x6f, 0xf4, 0xbe, 0x25, 0x95, 0xe, 0xe8, 0x73, 0xc3, 0x58, 0x57, 0xcc, 0x7c, 0xe7, 0x1, 0x9a, 0x2a, 0xb1, 0xfb, 0x60, 0xd0, 0x4b, 0xad, 0x36, 0x86, 0x1d, 0x98, 0x3, 0xb3, 0x28, 0xce, 0x55, 0xe5, 0x7e, 0x34, 0xaf, 0x1f, 0x84, 0x62, 0xf9, 0x49, 0xd2, 0xdd, 0x46, 0xf6, 0x6d, 0x8b, 0x10, 0xa0, 0x3b, 0x71, 0xea, 0x5a, 0xc1, 0x27, 0xbc, 0xc, 0x97, 0x1b, 0x80, 0x30, 0xab, 0x4d, 0xd6, 0x66, 0xfd, 0xb7, 0x2c, 0x9c, 0x7, 0xe1, 0x7a, 0xca, 0x51, 0x5e, 0xc5, 0x75, 0xee, 0x8, 0x93, 0x23, 0xb8, 0xf2, 0x69, 0xd9, 0x42, 0xa4, 0x3f, 0x8f, 0x14, 0x91, 0xa, 0xba, 0x21, 0xc7, 0x5c, 0xec, 0x77, 0x3d, 0xa6, 0x16, 0x8d, 0x6b, 0xf0, 0x40, 0xdb, 0xd4, 0x4f, 0xff, 0x64, 0x82, 0x19, 0xa9, 0x32, 0x78, 0xe3, 0x53, 0xc8, 0x2e, 0xb5, 0x5, 0x9e},
- {0x0, 0x9c, 0x25, 0xb9, 0x4a, 0xd6, 0x6f, 0xf3, 0x94, 0x8, 0xb1, 0x2d, 0xde, 0x42, 0xfb, 0x67, 0x35, 0xa9, 0x10, 0x8c, 0x7f, 0xe3, 0x5a, 0xc6, 0xa1, 0x3d, 0x84, 0x18, 0xeb, 0x77, 0xce, 0x52, 0x6a, 0xf6, 0x4f, 0xd3, 0x20, 0xbc, 0x5, 0x99, 0xfe, 0x62, 0xdb, 0x47, 0xb4, 0x28, 0x91, 0xd, 0x5f, 0xc3, 0x7a, 0xe6, 0x15, 0x89, 0x30, 0xac, 0xcb, 0x57, 0xee, 0x72, 0x81, 0x1d, 0xa4, 0x38, 0xd4, 0x48, 0xf1, 0x6d, 0x9e, 0x2, 0xbb, 0x27, 0x40, 0xdc, 0x65, 0xf9, 0xa, 0x96, 0x2f, 0xb3, 0xe1, 0x7d, 0xc4, 0x58, 0xab, 0x37, 0x8e, 0x12, 0x75, 0xe9, 0x50, 0xcc, 0x3f, 0xa3, 0x1a, 0x86, 0xbe, 0x22, 0x9b, 0x7, 0xf4, 0x68, 0xd1, 0x4d, 0x2a, 0xb6, 0xf, 0x93, 0x60, 0xfc, 0x45, 0xd9, 0x8b, 0x17, 0xae, 0x32, 0xc1, 0x5d, 0xe4, 0x78, 0x1f, 0x83, 0x3a, 0xa6, 0x55, 0xc9, 0x70, 0xec, 0xb5, 0x29, 0x90, 0xc, 0xff, 0x63, 0xda, 0x46, 0x21, 0xbd, 0x4, 0x98, 0x6b, 0xf7, 0x4e, 0xd2, 0x80, 0x1c, 0xa5, 0x39, 0xca, 0x56, 0xef, 0x73, 0x14, 0x88, 0x31, 0xad, 0x5e, 0xc2, 0x7b, 0xe7, 0xdf, 0x43, 0xfa, 0x66, 0x95, 0x9, 0xb0, 0x2c, 0x4b, 0xd7, 0x6e, 0xf2, 0x1, 0x9d, 0x24, 0xb8, 0xea, 0x76, 0xcf, 0x53, 0xa0, 0x3c, 0x85, 0x19, 0x7e, 0xe2, 0x5b, 0xc7, 0x34, 0xa8, 0x11, 0x8d, 0x61, 0xfd, 0x44, 0xd8, 0x2b, 0xb7, 0xe, 0x92, 0xf5, 0x69, 0xd0, 0x4c, 0xbf, 0x23, 0x9a, 0x6, 0x54, 0xc8, 0x71, 0xed, 0x1e, 0x82, 0x3b, 0xa7, 0xc0, 0x5c, 0xe5, 0x79, 0x8a, 0x16, 0xaf, 0x33, 0xb, 0x97, 0x2e, 0xb2, 0x41, 0xdd, 0x64, 0xf8, 0x9f, 0x3, 0xba, 0x26, 0xd5, 0x49, 0xf0, 0x6c, 0x3e, 0xa2, 0x1b, 0x87, 0x74, 0xe8, 0x51, 0xcd, 0xaa, 0x36, 0x8f, 0x13, 0xe0, 0x7c, 0xc5, 0x59},
- {0x0, 0x9d, 0x27, 0xba, 0x4e, 0xd3, 0x69, 0xf4, 0x9c, 0x1, 0xbb, 0x26, 0xd2, 0x4f, 0xf5, 0x68, 0x25, 0xb8, 0x2, 0x9f, 0x6b, 0xf6, 0x4c, 0xd1, 0xb9, 0x24, 0x9e, 0x3, 0xf7, 0x6a, 0xd0, 0x4d, 0x4a, 0xd7, 0x6d, 0xf0, 0x4, 0x99, 0x23, 0xbe, 0xd6, 0x4b, 0xf1, 0x6c, 0x98, 0x5, 0xbf, 0x22, 0x6f, 0xf2, 0x48, 0xd5, 0x21, 0xbc, 0x6, 0x9b, 0xf3, 0x6e, 0xd4, 0x49, 0xbd, 0x20, 0x9a, 0x7, 0x94, 0x9, 0xb3, 0x2e, 0xda, 0x47, 0xfd, 0x60, 0x8, 0x95, 0x2f, 0xb2, 0x46, 0xdb, 0x61, 0xfc, 0xb1, 0x2c, 0x96, 0xb, 0xff, 0x62, 0xd8, 0x45, 0x2d, 0xb0, 0xa, 0x97, 0x63, 0xfe, 0x44, 0xd9, 0xde, 0x43, 0xf9, 0x64, 0x90, 0xd, 0xb7, 0x2a, 0x42, 0xdf, 0x65, 0xf8, 0xc, 0x91, 0x2b, 0xb6, 0xfb, 0x66, 0xdc, 0x41, 0xb5, 0x28, 0x92, 0xf, 0x67, 0xfa, 0x40, 0xdd, 0x29, 0xb4, 0xe, 0x93, 0x35, 0xa8, 0x12, 0x8f, 0x7b, 0xe6, 0x5c, 0xc1, 0xa9, 0x34, 0x8e, 0x13, 0xe7, 0x7a, 0xc0, 0x5d, 0x10, 0x8d, 0x37, 0xaa, 0x5e, 0xc3, 0x79, 0xe4, 0x8c, 0x11, 0xab, 0x36, 0xc2, 0x5f, 0xe5, 0x78, 0x7f, 0xe2, 0x58, 0xc5, 0x31, 0xac, 0x16, 0x8b, 0xe3, 0x7e, 0xc4, 0x59, 0xad, 0x30, 0x8a, 0x17, 0x5a, 0xc7, 0x7d, 0xe0, 0x14, 0x89, 0x33, 0xae, 0xc6, 0x5b, 0xe1, 0x7c, 0x88, 0x15, 0xaf, 0x32, 0xa1, 0x3c, 0x86, 0x1b, 0xef, 0x72, 0xc8, 0x55, 0x3d, 0xa0, 0x1a, 0x87, 0x73, 0xee, 0x54, 0xc9, 0x84, 0x19, 0xa3, 0x3e, 0xca, 0x57, 0xed, 0x70, 0x18, 0x85, 0x3f, 0xa2, 0x56, 0xcb, 0x71, 0xec, 0xeb, 0x76, 0xcc, 0x51, 0xa5, 0x38, 0x82, 0x1f, 0x77, 0xea, 0x50, 0xcd, 0x39, 0xa4, 0x1e, 0x83, 0xce, 0x53, 0xe9, 0x74, 0x80, 0x1d, 0xa7, 0x3a, 0x52, 0xcf, 0x75, 0xe8, 0x1c, 0x81, 0x3b, 0xa6},
- {0x0, 0x9e, 0x21, 0xbf, 0x42, 0xdc, 0x63, 0xfd, 0x84, 0x1a, 0xa5, 0x3b, 0xc6, 0x58, 0xe7, 0x79, 0x15, 0x8b, 0x34, 0xaa, 0x57, 0xc9, 0x76, 0xe8, 0x91, 0xf, 0xb0, 0x2e, 0xd3, 0x4d, 0xf2, 0x6c, 0x2a, 0xb4, 0xb, 0x95, 0x68, 0xf6, 0x49, 0xd7, 0xae, 0x30, 0x8f, 0x11, 0xec, 0x72, 0xcd, 0x53, 0x3f, 0xa1, 0x1e, 0x80, 0x7d, 0xe3, 0x5c, 0xc2, 0xbb, 0x25, 0x9a, 0x4, 0xf9, 0x67, 0xd8, 0x46, 0x54, 0xca, 0x75, 0xeb, 0x16, 0x88, 0x37, 0xa9, 0xd0, 0x4e, 0xf1, 0x6f, 0x92, 0xc, 0xb3, 0x2d, 0x41, 0xdf, 0x60, 0xfe, 0x3, 0x9d, 0x22, 0xbc, 0xc5, 0x5b, 0xe4, 0x7a, 0x87, 0x19, 0xa6, 0x38, 0x7e, 0xe0, 0x5f, 0xc1, 0x3c, 0xa2, 0x1d, 0x83, 0xfa, 0x64, 0xdb, 0x45, 0xb8, 0x26, 0x99, 0x7, 0x6b, 0xf5, 0x4a, 0xd4, 0x29, 0xb7, 0x8, 0x96, 0xef, 0x71, 0xce, 0x50, 0xad, 0x33, 0x8c, 0x12, 0xa8, 0x36, 0x89, 0x17, 0xea, 0x74, 0xcb, 0x55, 0x2c, 0xb2, 0xd, 0x93, 0x6e, 0xf0, 0x4f, 0xd1, 0xbd, 0x23, 0x9c, 0x2, 0xff, 0x61, 0xde, 0x40, 0x39, 0xa7, 0x18, 0x86, 0x7b, 0xe5, 0x5a, 0xc4, 0x82, 0x1c, 0xa3, 0x3d, 0xc0, 0x5e, 0xe1, 0x7f, 0x6, 0x98, 0x27, 0xb9, 0x44, 0xda, 0x65, 0xfb, 0x97, 0x9, 0xb6, 0x28, 0xd5, 0x4b, 0xf4, 0x6a, 0x13, 0x8d, 0x32, 0xac, 0x51, 0xcf, 0x70, 0xee, 0xfc, 0x62, 0xdd, 0x43, 0xbe, 0x20, 0x9f, 0x1, 0x78, 0xe6, 0x59, 0xc7, 0x3a, 0xa4, 0x1b, 0x85, 0xe9, 0x77, 0xc8, 0x56, 0xab, 0x35, 0x8a, 0x14, 0x6d, 0xf3, 0x4c, 0xd2, 0x2f, 0xb1, 0xe, 0x90, 0xd6, 0x48, 0xf7, 0x69, 0x94, 0xa, 0xb5, 0x2b, 0x52, 0xcc, 0x73, 0xed, 0x10, 0x8e, 0x31, 0xaf, 0xc3, 0x5d, 0xe2, 0x7c, 0x81, 0x1f, 0xa0, 0x3e, 0x47, 0xd9, 0x66, 0xf8, 0x5, 0x9b, 0x24, 0xba},
- {0x0, 0x9f, 0x23, 0xbc, 0x46, 0xd9, 0x65, 0xfa, 0x8c, 0x13, 0xaf, 0x30, 0xca, 0x55, 0xe9, 0x76, 0x5, 0x9a, 0x26, 0xb9, 0x43, 0xdc, 0x60, 0xff, 0x89, 0x16, 0xaa, 0x35, 0xcf, 0x50, 0xec, 0x73, 0xa, 0x95, 0x29, 0xb6, 0x4c, 0xd3, 0x6f, 0xf0, 0x86, 0x19, 0xa5, 0x3a, 0xc0, 0x5f, 0xe3, 0x7c, 0xf, 0x90, 0x2c, 0xb3, 0x49, 0xd6, 0x6a, 0xf5, 0x83, 0x1c, 0xa0, 0x3f, 0xc5, 0x5a, 0xe6, 0x79, 0x14, 0x8b, 0x37, 0xa8, 0x52, 0xcd, 0x71, 0xee, 0x98, 0x7, 0xbb, 0x24, 0xde, 0x41, 0xfd, 0x62, 0x11, 0x8e, 0x32, 0xad, 0x57, 0xc8, 0x74, 0xeb, 0x9d, 0x2, 0xbe, 0x21, 0xdb, 0x44, 0xf8, 0x67, 0x1e, 0x81, 0x3d, 0xa2, 0x58, 0xc7, 0x7b, 0xe4, 0x92, 0xd, 0xb1, 0x2e, 0xd4, 0x4b, 0xf7, 0x68, 0x1b, 0x84, 0x38, 0xa7, 0x5d, 0xc2, 0x7e, 0xe1, 0x97, 0x8, 0xb4, 0x2b, 0xd1, 0x4e, 0xf2, 0x6d, 0x28, 0xb7, 0xb, 0x94, 0x6e, 0xf1, 0x4d, 0xd2, 0xa4, 0x3b, 0x87, 0x18, 0xe2, 0x7d, 0xc1, 0x5e, 0x2d, 0xb2, 0xe, 0x91, 0x6b, 0xf4, 0x48, 0xd7, 0xa1, 0x3e, 0x82, 0x1d, 0xe7, 0x78, 0xc4, 0x5b, 0x22, 0xbd, 0x1, 0x9e, 0x64, 0xfb, 0x47, 0xd8, 0xae, 0x31, 0x8d, 0x12, 0xe8, 0x77, 0xcb, 0x54, 0x27, 0xb8, 0x4, 0x9b, 0x61, 0xfe, 0x42, 0xdd, 0xab, 0x34, 0x88, 0x17, 0xed, 0x72, 0xce, 0x51, 0x3c, 0xa3, 0x1f, 0x80, 0x7a, 0xe5, 0x59, 0xc6, 0xb0, 0x2f, 0x93, 0xc, 0xf6, 0x69, 0xd5, 0x4a, 0x39, 0xa6, 0x1a, 0x85, 0x7f, 0xe0, 0x5c, 0xc3, 0xb5, 0x2a, 0x96, 0x9, 0xf3, 0x6c, 0xd0, 0x4f, 0x36, 0xa9, 0x15, 0x8a, 0x70, 0xef, 0x53, 0xcc, 0xba, 0x25, 0x99, 0x6, 0xfc, 0x63, 0xdf, 0x40, 0x33, 0xac, 0x10, 0x8f, 0x75, 0xea, 0x56, 0xc9, 0xbf, 0x20, 0x9c, 0x3, 0xf9, 0x66, 0xda, 0x45},
- {0x0, 0xa0, 0x5d, 0xfd, 0xba, 0x1a, 0xe7, 0x47, 0x69, 0xc9, 0x34, 0x94, 0xd3, 0x73, 0x8e, 0x2e, 0xd2, 0x72, 0x8f, 0x2f, 0x68, 0xc8, 0x35, 0x95, 0xbb, 0x1b, 0xe6, 0x46, 0x1, 0xa1, 0x5c, 0xfc, 0xb9, 0x19, 0xe4, 0x44, 0x3, 0xa3, 0x5e, 0xfe, 0xd0, 0x70, 0x8d, 0x2d, 0x6a, 0xca, 0x37, 0x97, 0x6b, 0xcb, 0x36, 0x96, 0xd1, 0x71, 0x8c, 0x2c, 0x2, 0xa2, 0x5f, 0xff, 0xb8, 0x18, 0xe5, 0x45, 0x6f, 0xcf, 0x32, 0x92, 0xd5, 0x75, 0x88, 0x28, 0x6, 0xa6, 0x5b, 0xfb, 0xbc, 0x1c, 0xe1, 0x41, 0xbd, 0x1d, 0xe0, 0x40, 0x7, 0xa7, 0x5a, 0xfa, 0xd4, 0x74, 0x89, 0x29, 0x6e, 0xce, 0x33, 0x93, 0xd6, 0x76, 0x8b, 0x2b, 0x6c, 0xcc, 0x31, 0x91, 0xbf, 0x1f, 0xe2, 0x42, 0x5, 0xa5, 0x58, 0xf8, 0x4, 0xa4, 0x59, 0xf9, 0xbe, 0x1e, 0xe3, 0x43, 0x6d, 0xcd, 0x30, 0x90, 0xd7, 0x77, 0x8a, 0x2a, 0xde, 0x7e, 0x83, 0x23, 0x64, 0xc4, 0x39, 0x99, 0xb7, 0x17, 0xea, 0x4a, 0xd, 0xad, 0x50, 0xf0, 0xc, 0xac, 0x51, 0xf1, 0xb6, 0x16, 0xeb, 0x4b, 0x65, 0xc5, 0x38, 0x98, 0xdf, 0x7f, 0x82, 0x22, 0x67, 0xc7, 0x3a, 0x9a, 0xdd, 0x7d, 0x80, 0x20, 0xe, 0xae, 0x53, 0xf3, 0xb4, 0x14, 0xe9, 0x49, 0xb5, 0x15, 0xe8, 0x48, 0xf, 0xaf, 0x52, 0xf2, 0xdc, 0x7c, 0x81, 0x21, 0x66, 0xc6, 0x3b, 0x9b, 0xb1, 0x11, 0xec, 0x4c, 0xb, 0xab, 0x56, 0xf6, 0xd8, 0x78, 0x85, 0x25, 0x62, 0xc2, 0x3f, 0x9f, 0x63, 0xc3, 0x3e, 0x9e, 0xd9, 0x79, 0x84, 0x24, 0xa, 0xaa, 0x57, 0xf7, 0xb0, 0x10, 0xed, 0x4d, 0x8, 0xa8, 0x55, 0xf5, 0xb2, 0x12, 0xef, 0x4f, 0x61, 0xc1, 0x3c, 0x9c, 0xdb, 0x7b, 0x86, 0x26, 0xda, 0x7a, 0x87, 0x27, 0x60, 0xc0, 0x3d, 0x9d, 0xb3, 0x13, 0xee, 0x4e, 0x9, 0xa9, 0x54, 0xf4},
- {0x0, 0xa1, 0x5f, 0xfe, 0xbe, 0x1f, 0xe1, 0x40, 0x61, 0xc0, 0x3e, 0x9f, 0xdf, 0x7e, 0x80, 0x21, 0xc2, 0x63, 0x9d, 0x3c, 0x7c, 0xdd, 0x23, 0x82, 0xa3, 0x2, 0xfc, 0x5d, 0x1d, 0xbc, 0x42, 0xe3, 0x99, 0x38, 0xc6, 0x67, 0x27, 0x86, 0x78, 0xd9, 0xf8, 0x59, 0xa7, 0x6, 0x46, 0xe7, 0x19, 0xb8, 0x5b, 0xfa, 0x4, 0xa5, 0xe5, 0x44, 0xba, 0x1b, 0x3a, 0x9b, 0x65, 0xc4, 0x84, 0x25, 0xdb, 0x7a, 0x2f, 0x8e, 0x70, 0xd1, 0x91, 0x30, 0xce, 0x6f, 0x4e, 0xef, 0x11, 0xb0, 0xf0, 0x51, 0xaf, 0xe, 0xed, 0x4c, 0xb2, 0x13, 0x53, 0xf2, 0xc, 0xad, 0x8c, 0x2d, 0xd3, 0x72, 0x32, 0x93, 0x6d, 0xcc, 0xb6, 0x17, 0xe9, 0x48, 0x8, 0xa9, 0x57, 0xf6, 0xd7, 0x76, 0x88, 0x29, 0x69, 0xc8, 0x36, 0x97, 0x74, 0xd5, 0x2b, 0x8a, 0xca, 0x6b, 0x95, 0x34, 0x15, 0xb4, 0x4a, 0xeb, 0xab, 0xa, 0xf4, 0x55, 0x5e, 0xff, 0x1, 0xa0, 0xe0, 0x41, 0xbf, 0x1e, 0x3f, 0x9e, 0x60, 0xc1, 0x81, 0x20, 0xde, 0x7f, 0x9c, 0x3d, 0xc3, 0x62, 0x22, 0x83, 0x7d, 0xdc, 0xfd, 0x5c, 0xa2, 0x3, 0x43, 0xe2, 0x1c, 0xbd, 0xc7, 0x66, 0x98, 0x39, 0x79, 0xd8, 0x26, 0x87, 0xa6, 0x7, 0xf9, 0x58, 0x18, 0xb9, 0x47, 0xe6, 0x5, 0xa4, 0x5a, 0xfb, 0xbb, 0x1a, 0xe4, 0x45, 0x64, 0xc5, 0x3b, 0x9a, 0xda, 0x7b, 0x85, 0x24, 0x71, 0xd0, 0x2e, 0x8f, 0xcf, 0x6e, 0x90, 0x31, 0x10, 0xb1, 0x4f, 0xee, 0xae, 0xf, 0xf1, 0x50, 0xb3, 0x12, 0xec, 0x4d, 0xd, 0xac, 0x52, 0xf3, 0xd2, 0x73, 0x8d, 0x2c, 0x6c, 0xcd, 0x33, 0x92, 0xe8, 0x49, 0xb7, 0x16, 0x56, 0xf7, 0x9, 0xa8, 0x89, 0x28, 0xd6, 0x77, 0x37, 0x96, 0x68, 0xc9, 0x2a, 0x8b, 0x75, 0xd4, 0x94, 0x35, 0xcb, 0x6a, 0x4b, 0xea, 0x14, 0xb5, 0xf5, 0x54, 0xaa, 0xb},
- {0x0, 0xa2, 0x59, 0xfb, 0xb2, 0x10, 0xeb, 0x49, 0x79, 0xdb, 0x20, 0x82, 0xcb, 0x69, 0x92, 0x30, 0xf2, 0x50, 0xab, 0x9, 0x40, 0xe2, 0x19, 0xbb, 0x8b, 0x29, 0xd2, 0x70, 0x39, 0x9b, 0x60, 0xc2, 0xf9, 0x5b, 0xa0, 0x2, 0x4b, 0xe9, 0x12, 0xb0, 0x80, 0x22, 0xd9, 0x7b, 0x32, 0x90, 0x6b, 0xc9, 0xb, 0xa9, 0x52, 0xf0, 0xb9, 0x1b, 0xe0, 0x42, 0x72, 0xd0, 0x2b, 0x89, 0xc0, 0x62, 0x99, 0x3b, 0xef, 0x4d, 0xb6, 0x14, 0x5d, 0xff, 0x4, 0xa6, 0x96, 0x34, 0xcf, 0x6d, 0x24, 0x86, 0x7d, 0xdf, 0x1d, 0xbf, 0x44, 0xe6, 0xaf, 0xd, 0xf6, 0x54, 0x64, 0xc6, 0x3d, 0x9f, 0xd6, 0x74, 0x8f, 0x2d, 0x16, 0xb4, 0x4f, 0xed, 0xa4, 0x6, 0xfd, 0x5f, 0x6f, 0xcd, 0x36, 0x94, 0xdd, 0x7f, 0x84, 0x26, 0xe4, 0x46, 0xbd, 0x1f, 0x56, 0xf4, 0xf, 0xad, 0x9d, 0x3f, 0xc4, 0x66, 0x2f, 0x8d, 0x76, 0xd4, 0xc3, 0x61, 0x9a, 0x38, 0x71, 0xd3, 0x28, 0x8a, 0xba, 0x18, 0xe3, 0x41, 0x8, 0xaa, 0x51, 0xf3, 0x31, 0x93, 0x68, 0xca, 0x83, 0x21, 0xda, 0x78, 0x48, 0xea, 0x11, 0xb3, 0xfa, 0x58, 0xa3, 0x1, 0x3a, 0x98, 0x63, 0xc1, 0x88, 0x2a, 0xd1, 0x73, 0x43, 0xe1, 0x1a, 0xb8, 0xf1, 0x53, 0xa8, 0xa, 0xc8, 0x6a, 0x91, 0x33, 0x7a, 0xd8, 0x23, 0x81, 0xb1, 0x13, 0xe8, 0x4a, 0x3, 0xa1, 0x5a, 0xf8, 0x2c, 0x8e, 0x75, 0xd7, 0x9e, 0x3c, 0xc7, 0x65, 0x55, 0xf7, 0xc, 0xae, 0xe7, 0x45, 0xbe, 0x1c, 0xde, 0x7c, 0x87, 0x25, 0x6c, 0xce, 0x35, 0x97, 0xa7, 0x5, 0xfe, 0x5c, 0x15, 0xb7, 0x4c, 0xee, 0xd5, 0x77, 0x8c, 0x2e, 0x67, 0xc5, 0x3e, 0x9c, 0xac, 0xe, 0xf5, 0x57, 0x1e, 0xbc, 0x47, 0xe5, 0x27, 0x85, 0x7e, 0xdc, 0x95, 0x37, 0xcc, 0x6e, 0x5e, 0xfc, 0x7, 0xa5, 0xec, 0x4e, 0xb5, 0x17},
- {0x0, 0xa3, 0x5b, 0xf8, 0xb6, 0x15, 0xed, 0x4e, 0x71, 0xd2, 0x2a, 0x89, 0xc7, 0x64, 0x9c, 0x3f, 0xe2, 0x41, 0xb9, 0x1a, 0x54, 0xf7, 0xf, 0xac, 0x93, 0x30, 0xc8, 0x6b, 0x25, 0x86, 0x7e, 0xdd, 0xd9, 0x7a, 0x82, 0x21, 0x6f, 0xcc, 0x34, 0x97, 0xa8, 0xb, 0xf3, 0x50, 0x1e, 0xbd, 0x45, 0xe6, 0x3b, 0x98, 0x60, 0xc3, 0x8d, 0x2e, 0xd6, 0x75, 0x4a, 0xe9, 0x11, 0xb2, 0xfc, 0x5f, 0xa7, 0x4, 0xaf, 0xc, 0xf4, 0x57, 0x19, 0xba, 0x42, 0xe1, 0xde, 0x7d, 0x85, 0x26, 0x68, 0xcb, 0x33, 0x90, 0x4d, 0xee, 0x16, 0xb5, 0xfb, 0x58, 0xa0, 0x3, 0x3c, 0x9f, 0x67, 0xc4, 0x8a, 0x29, 0xd1, 0x72, 0x76, 0xd5, 0x2d, 0x8e, 0xc0, 0x63, 0x9b, 0x38, 0x7, 0xa4, 0x5c, 0xff, 0xb1, 0x12, 0xea, 0x49, 0x94, 0x37, 0xcf, 0x6c, 0x22, 0x81, 0x79, 0xda, 0xe5, 0x46, 0xbe, 0x1d, 0x53, 0xf0, 0x8, 0xab, 0x43, 0xe0, 0x18, 0xbb, 0xf5, 0x56, 0xae, 0xd, 0x32, 0x91, 0x69, 0xca, 0x84, 0x27, 0xdf, 0x7c, 0xa1, 0x2, 0xfa, 0x59, 0x17, 0xb4, 0x4c, 0xef, 0xd0, 0x73, 0x8b, 0x28, 0x66, 0xc5, 0x3d, 0x9e, 0x9a, 0x39, 0xc1, 0x62, 0x2c, 0x8f, 0x77, 0xd4, 0xeb, 0x48, 0xb0, 0x13, 0x5d, 0xfe, 0x6, 0xa5, 0x78, 0xdb, 0x23, 0x80, 0xce, 0x6d, 0x95, 0x36, 0x9, 0xaa, 0x52, 0xf1, 0xbf, 0x1c, 0xe4, 0x47, 0xec, 0x4f, 0xb7, 0x14, 0x5a, 0xf9, 0x1, 0xa2, 0x9d, 0x3e, 0xc6, 0x65, 0x2b, 0x88, 0x70, 0xd3, 0xe, 0xad, 0x55, 0xf6, 0xb8, 0x1b, 0xe3, 0x40, 0x7f, 0xdc, 0x24, 0x87, 0xc9, 0x6a, 0x92, 0x31, 0x35, 0x96, 0x6e, 0xcd, 0x83, 0x20, 0xd8, 0x7b, 0x44, 0xe7, 0x1f, 0xbc, 0xf2, 0x51, 0xa9, 0xa, 0xd7, 0x74, 0x8c, 0x2f, 0x61, 0xc2, 0x3a, 0x99, 0xa6, 0x5, 0xfd, 0x5e, 0x10, 0xb3, 0x4b, 0xe8},
- {0x0, 0xa4, 0x55, 0xf1, 0xaa, 0xe, 0xff, 0x5b, 0x49, 0xed, 0x1c, 0xb8, 0xe3, 0x47, 0xb6, 0x12, 0x92, 0x36, 0xc7, 0x63, 0x38, 0x9c, 0x6d, 0xc9, 0xdb, 0x7f, 0x8e, 0x2a, 0x71, 0xd5, 0x24, 0x80, 0x39, 0x9d, 0x6c, 0xc8, 0x93, 0x37, 0xc6, 0x62, 0x70, 0xd4, 0x25, 0x81, 0xda, 0x7e, 0x8f, 0x2b, 0xab, 0xf, 0xfe, 0x5a, 0x1, 0xa5, 0x54, 0xf0, 0xe2, 0x46, 0xb7, 0x13, 0x48, 0xec, 0x1d, 0xb9, 0x72, 0xd6, 0x27, 0x83, 0xd8, 0x7c, 0x8d, 0x29, 0x3b, 0x9f, 0x6e, 0xca, 0x91, 0x35, 0xc4, 0x60, 0xe0, 0x44, 0xb5, 0x11, 0x4a, 0xee, 0x1f, 0xbb, 0xa9, 0xd, 0xfc, 0x58, 0x3, 0xa7, 0x56, 0xf2, 0x4b, 0xef, 0x1e, 0xba, 0xe1, 0x45, 0xb4, 0x10, 0x2, 0xa6, 0x57, 0xf3, 0xa8, 0xc, 0xfd, 0x59, 0xd9, 0x7d, 0x8c, 0x28, 0x73, 0xd7, 0x26, 0x82, 0x90, 0x34, 0xc5, 0x61, 0x3a, 0x9e, 0x6f, 0xcb, 0xe4, 0x40, 0xb1, 0x15, 0x4e, 0xea, 0x1b, 0xbf, 0xad, 0x9, 0xf8, 0x5c, 0x7, 0xa3, 0x52, 0xf6, 0x76, 0xd2, 0x23, 0x87, 0xdc, 0x78, 0x89, 0x2d, 0x3f, 0x9b, 0x6a, 0xce, 0x95, 0x31, 0xc0, 0x64, 0xdd, 0x79, 0x88, 0x2c, 0x77, 0xd3, 0x22, 0x86, 0x94, 0x30, 0xc1, 0x65, 0x3e, 0x9a, 0x6b, 0xcf, 0x4f, 0xeb, 0x1a, 0xbe, 0xe5, 0x41, 0xb0, 0x14, 0x6, 0xa2, 0x53, 0xf7, 0xac, 0x8, 0xf9, 0x5d, 0x96, 0x32, 0xc3, 0x67, 0x3c, 0x98, 0x69, 0xcd, 0xdf, 0x7b, 0x8a, 0x2e, 0x75, 0xd1, 0x20, 0x84, 0x4, 0xa0, 0x51, 0xf5, 0xae, 0xa, 0xfb, 0x5f, 0x4d, 0xe9, 0x18, 0xbc, 0xe7, 0x43, 0xb2, 0x16, 0xaf, 0xb, 0xfa, 0x5e, 0x5, 0xa1, 0x50, 0xf4, 0xe6, 0x42, 0xb3, 0x17, 0x4c, 0xe8, 0x19, 0xbd, 0x3d, 0x99, 0x68, 0xcc, 0x97, 0x33, 0xc2, 0x66, 0x74, 0xd0, 0x21, 0x85, 0xde, 0x7a, 0x8b, 0x2f},
- {0x0, 0xa5, 0x57, 0xf2, 0xae, 0xb, 0xf9, 0x5c, 0x41, 0xe4, 0x16, 0xb3, 0xef, 0x4a, 0xb8, 0x1d, 0x82, 0x27, 0xd5, 0x70, 0x2c, 0x89, 0x7b, 0xde, 0xc3, 0x66, 0x94, 0x31, 0x6d, 0xc8, 0x3a, 0x9f, 0x19, 0xbc, 0x4e, 0xeb, 0xb7, 0x12, 0xe0, 0x45, 0x58, 0xfd, 0xf, 0xaa, 0xf6, 0x53, 0xa1, 0x4, 0x9b, 0x3e, 0xcc, 0x69, 0x35, 0x90, 0x62, 0xc7, 0xda, 0x7f, 0x8d, 0x28, 0x74, 0xd1, 0x23, 0x86, 0x32, 0x97, 0x65, 0xc0, 0x9c, 0x39, 0xcb, 0x6e, 0x73, 0xd6, 0x24, 0x81, 0xdd, 0x78, 0x8a, 0x2f, 0xb0, 0x15, 0xe7, 0x42, 0x1e, 0xbb, 0x49, 0xec, 0xf1, 0x54, 0xa6, 0x3, 0x5f, 0xfa, 0x8, 0xad, 0x2b, 0x8e, 0x7c, 0xd9, 0x85, 0x20, 0xd2, 0x77, 0x6a, 0xcf, 0x3d, 0x98, 0xc4, 0x61, 0x93, 0x36, 0xa9, 0xc, 0xfe, 0x5b, 0x7, 0xa2, 0x50, 0xf5, 0xe8, 0x4d, 0xbf, 0x1a, 0x46, 0xe3, 0x11, 0xb4, 0x64, 0xc1, 0x33, 0x96, 0xca, 0x6f, 0x9d, 0x38, 0x25, 0x80, 0x72, 0xd7, 0x8b, 0x2e, 0xdc, 0x79, 0xe6, 0x43, 0xb1, 0x14, 0x48, 0xed, 0x1f, 0xba, 0xa7, 0x2, 0xf0, 0x55, 0x9, 0xac, 0x5e, 0xfb, 0x7d, 0xd8, 0x2a, 0x8f, 0xd3, 0x76, 0x84, 0x21, 0x3c, 0x99, 0x6b, 0xce, 0x92, 0x37, 0xc5, 0x60, 0xff, 0x5a, 0xa8, 0xd, 0x51, 0xf4, 0x6, 0xa3, 0xbe, 0x1b, 0xe9, 0x4c, 0x10, 0xb5, 0x47, 0xe2, 0x56, 0xf3, 0x1, 0xa4, 0xf8, 0x5d, 0xaf, 0xa, 0x17, 0xb2, 0x40, 0xe5, 0xb9, 0x1c, 0xee, 0x4b, 0xd4, 0x71, 0x83, 0x26, 0x7a, 0xdf, 0x2d, 0x88, 0x95, 0x30, 0xc2, 0x67, 0x3b, 0x9e, 0x6c, 0xc9, 0x4f, 0xea, 0x18, 0xbd, 0xe1, 0x44, 0xb6, 0x13, 0xe, 0xab, 0x59, 0xfc, 0xa0, 0x5, 0xf7, 0x52, 0xcd, 0x68, 0x9a, 0x3f, 0x63, 0xc6, 0x34, 0x91, 0x8c, 0x29, 0xdb, 0x7e, 0x22, 0x87, 0x75, 0xd0},
- {0x0, 0xa6, 0x51, 0xf7, 0xa2, 0x4, 0xf3, 0x55, 0x59, 0xff, 0x8, 0xae, 0xfb, 0x5d, 0xaa, 0xc, 0xb2, 0x14, 0xe3, 0x45, 0x10, 0xb6, 0x41, 0xe7, 0xeb, 0x4d, 0xba, 0x1c, 0x49, 0xef, 0x18, 0xbe, 0x79, 0xdf, 0x28, 0x8e, 0xdb, 0x7d, 0x8a, 0x2c, 0x20, 0x86, 0x71, 0xd7, 0x82, 0x24, 0xd3, 0x75, 0xcb, 0x6d, 0x9a, 0x3c, 0x69, 0xcf, 0x38, 0x9e, 0x92, 0x34, 0xc3, 0x65, 0x30, 0x96, 0x61, 0xc7, 0xf2, 0x54, 0xa3, 0x5, 0x50, 0xf6, 0x1, 0xa7, 0xab, 0xd, 0xfa, 0x5c, 0x9, 0xaf, 0x58, 0xfe, 0x40, 0xe6, 0x11, 0xb7, 0xe2, 0x44, 0xb3, 0x15, 0x19, 0xbf, 0x48, 0xee, 0xbb, 0x1d, 0xea, 0x4c, 0x8b, 0x2d, 0xda, 0x7c, 0x29, 0x8f, 0x78, 0xde, 0xd2, 0x74, 0x83, 0x25, 0x70, 0xd6, 0x21, 0x87, 0x39, 0x9f, 0x68, 0xce, 0x9b, 0x3d, 0xca, 0x6c, 0x60, 0xc6, 0x31, 0x97, 0xc2, 0x64, 0x93, 0x35, 0xf9, 0x5f, 0xa8, 0xe, 0x5b, 0xfd, 0xa, 0xac, 0xa0, 0x6, 0xf1, 0x57, 0x2, 0xa4, 0x53, 0xf5, 0x4b, 0xed, 0x1a, 0xbc, 0xe9, 0x4f, 0xb8, 0x1e, 0x12, 0xb4, 0x43, 0xe5, 0xb0, 0x16, 0xe1, 0x47, 0x80, 0x26, 0xd1, 0x77, 0x22, 0x84, 0x73, 0xd5, 0xd9, 0x7f, 0x88, 0x2e, 0x7b, 0xdd, 0x2a, 0x8c, 0x32, 0x94, 0x63, 0xc5, 0x90, 0x36, 0xc1, 0x67, 0x6b, 0xcd, 0x3a, 0x9c, 0xc9, 0x6f, 0x98, 0x3e, 0xb, 0xad, 0x5a, 0xfc, 0xa9, 0xf, 0xf8, 0x5e, 0x52, 0xf4, 0x3, 0xa5, 0xf0, 0x56, 0xa1, 0x7, 0xb9, 0x1f, 0xe8, 0x4e, 0x1b, 0xbd, 0x4a, 0xec, 0xe0, 0x46, 0xb1, 0x17, 0x42, 0xe4, 0x13, 0xb5, 0x72, 0xd4, 0x23, 0x85, 0xd0, 0x76, 0x81, 0x27, 0x2b, 0x8d, 0x7a, 0xdc, 0x89, 0x2f, 0xd8, 0x7e, 0xc0, 0x66, 0x91, 0x37, 0x62, 0xc4, 0x33, 0x95, 0x99, 0x3f, 0xc8, 0x6e, 0x3b, 0x9d, 0x6a, 0xcc},
- {0x0, 0xa7, 0x53, 0xf4, 0xa6, 0x1, 0xf5, 0x52, 0x51, 0xf6, 0x2, 0xa5, 0xf7, 0x50, 0xa4, 0x3, 0xa2, 0x5, 0xf1, 0x56, 0x4, 0xa3, 0x57, 0xf0, 0xf3, 0x54, 0xa0, 0x7, 0x55, 0xf2, 0x6, 0xa1, 0x59, 0xfe, 0xa, 0xad, 0xff, 0x58, 0xac, 0xb, 0x8, 0xaf, 0x5b, 0xfc, 0xae, 0x9, 0xfd, 0x5a, 0xfb, 0x5c, 0xa8, 0xf, 0x5d, 0xfa, 0xe, 0xa9, 0xaa, 0xd, 0xf9, 0x5e, 0xc, 0xab, 0x5f, 0xf8, 0xb2, 0x15, 0xe1, 0x46, 0x14, 0xb3, 0x47, 0xe0, 0xe3, 0x44, 0xb0, 0x17, 0x45, 0xe2, 0x16, 0xb1, 0x10, 0xb7, 0x43, 0xe4, 0xb6, 0x11, 0xe5, 0x42, 0x41, 0xe6, 0x12, 0xb5, 0xe7, 0x40, 0xb4, 0x13, 0xeb, 0x4c, 0xb8, 0x1f, 0x4d, 0xea, 0x1e, 0xb9, 0xba, 0x1d, 0xe9, 0x4e, 0x1c, 0xbb, 0x4f, 0xe8, 0x49, 0xee, 0x1a, 0xbd, 0xef, 0x48, 0xbc, 0x1b, 0x18, 0xbf, 0x4b, 0xec, 0xbe, 0x19, 0xed, 0x4a, 0x79, 0xde, 0x2a, 0x8d, 0xdf, 0x78, 0x8c, 0x2b, 0x28, 0x8f, 0x7b, 0xdc, 0x8e, 0x29, 0xdd, 0x7a, 0xdb, 0x7c, 0x88, 0x2f, 0x7d, 0xda, 0x2e, 0x89, 0x8a, 0x2d, 0xd9, 0x7e, 0x2c, 0x8b, 0x7f, 0xd8, 0x20, 0x87, 0x73, 0xd4, 0x86, 0x21, 0xd5, 0x72, 0x71, 0xd6, 0x22, 0x85, 0xd7, 0x70, 0x84, 0x23, 0x82, 0x25, 0xd1, 0x76, 0x24, 0x83, 0x77, 0xd0, 0xd3, 0x74, 0x80, 0x27, 0x75, 0xd2, 0x26, 0x81, 0xcb, 0x6c, 0x98, 0x3f, 0x6d, 0xca, 0x3e, 0x99, 0x9a, 0x3d, 0xc9, 0x6e, 0x3c, 0x9b, 0x6f, 0xc8, 0x69, 0xce, 0x3a, 0x9d, 0xcf, 0x68, 0x9c, 0x3b, 0x38, 0x9f, 0x6b, 0xcc, 0x9e, 0x39, 0xcd, 0x6a, 0x92, 0x35, 0xc1, 0x66, 0x34, 0x93, 0x67, 0xc0, 0xc3, 0x64, 0x90, 0x37, 0x65, 0xc2, 0x36, 0x91, 0x30, 0x97, 0x63, 0xc4, 0x96, 0x31, 0xc5, 0x62, 0x61, 0xc6, 0x32, 0x95, 0xc7, 0x60, 0x94, 0x33},
- {0x0, 0xa8, 0x4d, 0xe5, 0x9a, 0x32, 0xd7, 0x7f, 0x29, 0x81, 0x64, 0xcc, 0xb3, 0x1b, 0xfe, 0x56, 0x52, 0xfa, 0x1f, 0xb7, 0xc8, 0x60, 0x85, 0x2d, 0x7b, 0xd3, 0x36, 0x9e, 0xe1, 0x49, 0xac, 0x4, 0xa4, 0xc, 0xe9, 0x41, 0x3e, 0x96, 0x73, 0xdb, 0x8d, 0x25, 0xc0, 0x68, 0x17, 0xbf, 0x5a, 0xf2, 0xf6, 0x5e, 0xbb, 0x13, 0x6c, 0xc4, 0x21, 0x89, 0xdf, 0x77, 0x92, 0x3a, 0x45, 0xed, 0x8, 0xa0, 0x55, 0xfd, 0x18, 0xb0, 0xcf, 0x67, 0x82, 0x2a, 0x7c, 0xd4, 0x31, 0x99, 0xe6, 0x4e, 0xab, 0x3, 0x7, 0xaf, 0x4a, 0xe2, 0x9d, 0x35, 0xd0, 0x78, 0x2e, 0x86, 0x63, 0xcb, 0xb4, 0x1c, 0xf9, 0x51, 0xf1, 0x59, 0xbc, 0x14, 0x6b, 0xc3, 0x26, 0x8e, 0xd8, 0x70, 0x95, 0x3d, 0x42, 0xea, 0xf, 0xa7, 0xa3, 0xb, 0xee, 0x46, 0x39, 0x91, 0x74, 0xdc, 0x8a, 0x22, 0xc7, 0x6f, 0x10, 0xb8, 0x5d, 0xf5, 0xaa, 0x2, 0xe7, 0x4f, 0x30, 0x98, 0x7d, 0xd5, 0x83, 0x2b, 0xce, 0x66, 0x19, 0xb1, 0x54, 0xfc, 0xf8, 0x50, 0xb5, 0x1d, 0x62, 0xca, 0x2f, 0x87, 0xd1, 0x79, 0x9c, 0x34, 0x4b, 0xe3, 0x6, 0xae, 0xe, 0xa6, 0x43, 0xeb, 0x94, 0x3c, 0xd9, 0x71, 0x27, 0x8f, 0x6a, 0xc2, 0xbd, 0x15, 0xf0, 0x58, 0x5c, 0xf4, 0x11, 0xb9, 0xc6, 0x6e, 0x8b, 0x23, 0x75, 0xdd, 0x38, 0x90, 0xef, 0x47, 0xa2, 0xa, 0xff, 0x57, 0xb2, 0x1a, 0x65, 0xcd, 0x28, 0x80, 0xd6, 0x7e, 0x9b, 0x33, 0x4c, 0xe4, 0x1, 0xa9, 0xad, 0x5, 0xe0, 0x48, 0x37, 0x9f, 0x7a, 0xd2, 0x84, 0x2c, 0xc9, 0x61, 0x1e, 0xb6, 0x53, 0xfb, 0x5b, 0xf3, 0x16, 0xbe, 0xc1, 0x69, 0x8c, 0x24, 0x72, 0xda, 0x3f, 0x97, 0xe8, 0x40, 0xa5, 0xd, 0x9, 0xa1, 0x44, 0xec, 0x93, 0x3b, 0xde, 0x76, 0x20, 0x88, 0x6d, 0xc5, 0xba, 0x12, 0xf7, 0x5f},
- {0x0, 0xa9, 0x4f, 0xe6, 0x9e, 0x37, 0xd1, 0x78, 0x21, 0x88, 0x6e, 0xc7, 0xbf, 0x16, 0xf0, 0x59, 0x42, 0xeb, 0xd, 0xa4, 0xdc, 0x75, 0x93, 0x3a, 0x63, 0xca, 0x2c, 0x85, 0xfd, 0x54, 0xb2, 0x1b, 0x84, 0x2d, 0xcb, 0x62, 0x1a, 0xb3, 0x55, 0xfc, 0xa5, 0xc, 0xea, 0x43, 0x3b, 0x92, 0x74, 0xdd, 0xc6, 0x6f, 0x89, 0x20, 0x58, 0xf1, 0x17, 0xbe, 0xe7, 0x4e, 0xa8, 0x1, 0x79, 0xd0, 0x36, 0x9f, 0x15, 0xbc, 0x5a, 0xf3, 0x8b, 0x22, 0xc4, 0x6d, 0x34, 0x9d, 0x7b, 0xd2, 0xaa, 0x3, 0xe5, 0x4c, 0x57, 0xfe, 0x18, 0xb1, 0xc9, 0x60, 0x86, 0x2f, 0x76, 0xdf, 0x39, 0x90, 0xe8, 0x41, 0xa7, 0xe, 0x91, 0x38, 0xde, 0x77, 0xf, 0xa6, 0x40, 0xe9, 0xb0, 0x19, 0xff, 0x56, 0x2e, 0x87, 0x61, 0xc8, 0xd3, 0x7a, 0x9c, 0x35, 0x4d, 0xe4, 0x2, 0xab, 0xf2, 0x5b, 0xbd, 0x14, 0x6c, 0xc5, 0x23, 0x8a, 0x2a, 0x83, 0x65, 0xcc, 0xb4, 0x1d, 0xfb, 0x52, 0xb, 0xa2, 0x44, 0xed, 0x95, 0x3c, 0xda, 0x73, 0x68, 0xc1, 0x27, 0x8e, 0xf6, 0x5f, 0xb9, 0x10, 0x49, 0xe0, 0x6, 0xaf, 0xd7, 0x7e, 0x98, 0x31, 0xae, 0x7, 0xe1, 0x48, 0x30, 0x99, 0x7f, 0xd6, 0x8f, 0x26, 0xc0, 0x69, 0x11, 0xb8, 0x5e, 0xf7, 0xec, 0x45, 0xa3, 0xa, 0x72, 0xdb, 0x3d, 0x94, 0xcd, 0x64, 0x82, 0x2b, 0x53, 0xfa, 0x1c, 0xb5, 0x3f, 0x96, 0x70, 0xd9, 0xa1, 0x8, 0xee, 0x47, 0x1e, 0xb7, 0x51, 0xf8, 0x80, 0x29, 0xcf, 0x66, 0x7d, 0xd4, 0x32, 0x9b, 0xe3, 0x4a, 0xac, 0x5, 0x5c, 0xf5, 0x13, 0xba, 0xc2, 0x6b, 0x8d, 0x24, 0xbb, 0x12, 0xf4, 0x5d, 0x25, 0x8c, 0x6a, 0xc3, 0x9a, 0x33, 0xd5, 0x7c, 0x4, 0xad, 0x4b, 0xe2, 0xf9, 0x50, 0xb6, 0x1f, 0x67, 0xce, 0x28, 0x81, 0xd8, 0x71, 0x97, 0x3e, 0x46, 0xef, 0x9, 0xa0},
- {0x0, 0xaa, 0x49, 0xe3, 0x92, 0x38, 0xdb, 0x71, 0x39, 0x93, 0x70, 0xda, 0xab, 0x1, 0xe2, 0x48, 0x72, 0xd8, 0x3b, 0x91, 0xe0, 0x4a, 0xa9, 0x3, 0x4b, 0xe1, 0x2, 0xa8, 0xd9, 0x73, 0x90, 0x3a, 0xe4, 0x4e, 0xad, 0x7, 0x76, 0xdc, 0x3f, 0x95, 0xdd, 0x77, 0x94, 0x3e, 0x4f, 0xe5, 0x6, 0xac, 0x96, 0x3c, 0xdf, 0x75, 0x4, 0xae, 0x4d, 0xe7, 0xaf, 0x5, 0xe6, 0x4c, 0x3d, 0x97, 0x74, 0xde, 0xd5, 0x7f, 0x9c, 0x36, 0x47, 0xed, 0xe, 0xa4, 0xec, 0x46, 0xa5, 0xf, 0x7e, 0xd4, 0x37, 0x9d, 0xa7, 0xd, 0xee, 0x44, 0x35, 0x9f, 0x7c, 0xd6, 0x9e, 0x34, 0xd7, 0x7d, 0xc, 0xa6, 0x45, 0xef, 0x31, 0x9b, 0x78, 0xd2, 0xa3, 0x9, 0xea, 0x40, 0x8, 0xa2, 0x41, 0xeb, 0x9a, 0x30, 0xd3, 0x79, 0x43, 0xe9, 0xa, 0xa0, 0xd1, 0x7b, 0x98, 0x32, 0x7a, 0xd0, 0x33, 0x99, 0xe8, 0x42, 0xa1, 0xb, 0xb7, 0x1d, 0xfe, 0x54, 0x25, 0x8f, 0x6c, 0xc6, 0x8e, 0x24, 0xc7, 0x6d, 0x1c, 0xb6, 0x55, 0xff, 0xc5, 0x6f, 0x8c, 0x26, 0x57, 0xfd, 0x1e, 0xb4, 0xfc, 0x56, 0xb5, 0x1f, 0x6e, 0xc4, 0x27, 0x8d, 0x53, 0xf9, 0x1a, 0xb0, 0xc1, 0x6b, 0x88, 0x22, 0x6a, 0xc0, 0x23, 0x89, 0xf8, 0x52, 0xb1, 0x1b, 0x21, 0x8b, 0x68, 0xc2, 0xb3, 0x19, 0xfa, 0x50, 0x18, 0xb2, 0x51, 0xfb, 0x8a, 0x20, 0xc3, 0x69, 0x62, 0xc8, 0x2b, 0x81, 0xf0, 0x5a, 0xb9, 0x13, 0x5b, 0xf1, 0x12, 0xb8, 0xc9, 0x63, 0x80, 0x2a, 0x10, 0xba, 0x59, 0xf3, 0x82, 0x28, 0xcb, 0x61, 0x29, 0x83, 0x60, 0xca, 0xbb, 0x11, 0xf2, 0x58, 0x86, 0x2c, 0xcf, 0x65, 0x14, 0xbe, 0x5d, 0xf7, 0xbf, 0x15, 0xf6, 0x5c, 0x2d, 0x87, 0x64, 0xce, 0xf4, 0x5e, 0xbd, 0x17, 0x66, 0xcc, 0x2f, 0x85, 0xcd, 0x67, 0x84, 0x2e, 0x5f, 0xf5, 0x16, 0xbc},
- {0x0, 0xab, 0x4b, 0xe0, 0x96, 0x3d, 0xdd, 0x76, 0x31, 0x9a, 0x7a, 0xd1, 0xa7, 0xc, 0xec, 0x47, 0x62, 0xc9, 0x29, 0x82, 0xf4, 0x5f, 0xbf, 0x14, 0x53, 0xf8, 0x18, 0xb3, 0xc5, 0x6e, 0x8e, 0x25, 0xc4, 0x6f, 0x8f, 0x24, 0x52, 0xf9, 0x19, 0xb2, 0xf5, 0x5e, 0xbe, 0x15, 0x63, 0xc8, 0x28, 0x83, 0xa6, 0xd, 0xed, 0x46, 0x30, 0x9b, 0x7b, 0xd0, 0x97, 0x3c, 0xdc, 0x77, 0x1, 0xaa, 0x4a, 0xe1, 0x95, 0x3e, 0xde, 0x75, 0x3, 0xa8, 0x48, 0xe3, 0xa4, 0xf, 0xef, 0x44, 0x32, 0x99, 0x79, 0xd2, 0xf7, 0x5c, 0xbc, 0x17, 0x61, 0xca, 0x2a, 0x81, 0xc6, 0x6d, 0x8d, 0x26, 0x50, 0xfb, 0x1b, 0xb0, 0x51, 0xfa, 0x1a, 0xb1, 0xc7, 0x6c, 0x8c, 0x27, 0x60, 0xcb, 0x2b, 0x80, 0xf6, 0x5d, 0xbd, 0x16, 0x33, 0x98, 0x78, 0xd3, 0xa5, 0xe, 0xee, 0x45, 0x2, 0xa9, 0x49, 0xe2, 0x94, 0x3f, 0xdf, 0x74, 0x37, 0x9c, 0x7c, 0xd7, 0xa1, 0xa, 0xea, 0x41, 0x6, 0xad, 0x4d, 0xe6, 0x90, 0x3b, 0xdb, 0x70, 0x55, 0xfe, 0x1e, 0xb5, 0xc3, 0x68, 0x88, 0x23, 0x64, 0xcf, 0x2f, 0x84, 0xf2, 0x59, 0xb9, 0x12, 0xf3, 0x58, 0xb8, 0x13, 0x65, 0xce, 0x2e, 0x85, 0xc2, 0x69, 0x89, 0x22, 0x54, 0xff, 0x1f, 0xb4, 0x91, 0x3a, 0xda, 0x71, 0x7, 0xac, 0x4c, 0xe7, 0xa0, 0xb, 0xeb, 0x40, 0x36, 0x9d, 0x7d, 0xd6, 0xa2, 0x9, 0xe9, 0x42, 0x34, 0x9f, 0x7f, 0xd4, 0x93, 0x38, 0xd8, 0x73, 0x5, 0xae, 0x4e, 0xe5, 0xc0, 0x6b, 0x8b, 0x20, 0x56, 0xfd, 0x1d, 0xb6, 0xf1, 0x5a, 0xba, 0x11, 0x67, 0xcc, 0x2c, 0x87, 0x66, 0xcd, 0x2d, 0x86, 0xf0, 0x5b, 0xbb, 0x10, 0x57, 0xfc, 0x1c, 0xb7, 0xc1, 0x6a, 0x8a, 0x21, 0x4, 0xaf, 0x4f, 0xe4, 0x92, 0x39, 0xd9, 0x72, 0x35, 0x9e, 0x7e, 0xd5, 0xa3, 0x8, 0xe8, 0x43},
- {0x0, 0xac, 0x45, 0xe9, 0x8a, 0x26, 0xcf, 0x63, 0x9, 0xa5, 0x4c, 0xe0, 0x83, 0x2f, 0xc6, 0x6a, 0x12, 0xbe, 0x57, 0xfb, 0x98, 0x34, 0xdd, 0x71, 0x1b, 0xb7, 0x5e, 0xf2, 0x91, 0x3d, 0xd4, 0x78, 0x24, 0x88, 0x61, 0xcd, 0xae, 0x2, 0xeb, 0x47, 0x2d, 0x81, 0x68, 0xc4, 0xa7, 0xb, 0xe2, 0x4e, 0x36, 0x9a, 0x73, 0xdf, 0xbc, 0x10, 0xf9, 0x55, 0x3f, 0x93, 0x7a, 0xd6, 0xb5, 0x19, 0xf0, 0x5c, 0x48, 0xe4, 0xd, 0xa1, 0xc2, 0x6e, 0x87, 0x2b, 0x41, 0xed, 0x4, 0xa8, 0xcb, 0x67, 0x8e, 0x22, 0x5a, 0xf6, 0x1f, 0xb3, 0xd0, 0x7c, 0x95, 0x39, 0x53, 0xff, 0x16, 0xba, 0xd9, 0x75, 0x9c, 0x30, 0x6c, 0xc0, 0x29, 0x85, 0xe6, 0x4a, 0xa3, 0xf, 0x65, 0xc9, 0x20, 0x8c, 0xef, 0x43, 0xaa, 0x6, 0x7e, 0xd2, 0x3b, 0x97, 0xf4, 0x58, 0xb1, 0x1d, 0x77, 0xdb, 0x32, 0x9e, 0xfd, 0x51, 0xb8, 0x14, 0x90, 0x3c, 0xd5, 0x79, 0x1a, 0xb6, 0x5f, 0xf3, 0x99, 0x35, 0xdc, 0x70, 0x13, 0xbf, 0x56, 0xfa, 0x82, 0x2e, 0xc7, 0x6b, 0x8, 0xa4, 0x4d, 0xe1, 0x8b, 0x27, 0xce, 0x62, 0x1, 0xad, 0x44, 0xe8, 0xb4, 0x18, 0xf1, 0x5d, 0x3e, 0x92, 0x7b, 0xd7, 0xbd, 0x11, 0xf8, 0x54, 0x37, 0x9b, 0x72, 0xde, 0xa6, 0xa, 0xe3, 0x4f, 0x2c, 0x80, 0x69, 0xc5, 0xaf, 0x3, 0xea, 0x46, 0x25, 0x89, 0x60, 0xcc, 0xd8, 0x74, 0x9d, 0x31, 0x52, 0xfe, 0x17, 0xbb, 0xd1, 0x7d, 0x94, 0x38, 0x5b, 0xf7, 0x1e, 0xb2, 0xca, 0x66, 0x8f, 0x23, 0x40, 0xec, 0x5, 0xa9, 0xc3, 0x6f, 0x86, 0x2a, 0x49, 0xe5, 0xc, 0xa0, 0xfc, 0x50, 0xb9, 0x15, 0x76, 0xda, 0x33, 0x9f, 0xf5, 0x59, 0xb0, 0x1c, 0x7f, 0xd3, 0x3a, 0x96, 0xee, 0x42, 0xab, 0x7, 0x64, 0xc8, 0x21, 0x8d, 0xe7, 0x4b, 0xa2, 0xe, 0x6d, 0xc1, 0x28, 0x84},
- {0x0, 0xad, 0x47, 0xea, 0x8e, 0x23, 0xc9, 0x64, 0x1, 0xac, 0x46, 0xeb, 0x8f, 0x22, 0xc8, 0x65, 0x2, 0xaf, 0x45, 0xe8, 0x8c, 0x21, 0xcb, 0x66, 0x3, 0xae, 0x44, 0xe9, 0x8d, 0x20, 0xca, 0x67, 0x4, 0xa9, 0x43, 0xee, 0x8a, 0x27, 0xcd, 0x60, 0x5, 0xa8, 0x42, 0xef, 0x8b, 0x26, 0xcc, 0x61, 0x6, 0xab, 0x41, 0xec, 0x88, 0x25, 0xcf, 0x62, 0x7, 0xaa, 0x40, 0xed, 0x89, 0x24, 0xce, 0x63, 0x8, 0xa5, 0x4f, 0xe2, 0x86, 0x2b, 0xc1, 0x6c, 0x9, 0xa4, 0x4e, 0xe3, 0x87, 0x2a, 0xc0, 0x6d, 0xa, 0xa7, 0x4d, 0xe0, 0x84, 0x29, 0xc3, 0x6e, 0xb, 0xa6, 0x4c, 0xe1, 0x85, 0x28, 0xc2, 0x6f, 0xc, 0xa1, 0x4b, 0xe6, 0x82, 0x2f, 0xc5, 0x68, 0xd, 0xa0, 0x4a, 0xe7, 0x83, 0x2e, 0xc4, 0x69, 0xe, 0xa3, 0x49, 0xe4, 0x80, 0x2d, 0xc7, 0x6a, 0xf, 0xa2, 0x48, 0xe5, 0x81, 0x2c, 0xc6, 0x6b, 0x10, 0xbd, 0x57, 0xfa, 0x9e, 0x33, 0xd9, 0x74, 0x11, 0xbc, 0x56, 0xfb, 0x9f, 0x32, 0xd8, 0x75, 0x12, 0xbf, 0x55, 0xf8, 0x9c, 0x31, 0xdb, 0x76, 0x13, 0xbe, 0x54, 0xf9, 0x9d, 0x30, 0xda, 0x77, 0x14, 0xb9, 0x53, 0xfe, 0x9a, 0x37, 0xdd, 0x70, 0x15, 0xb8, 0x52, 0xff, 0x9b, 0x36, 0xdc, 0x71, 0x16, 0xbb, 0x51, 0xfc, 0x98, 0x35, 0xdf, 0x72, 0x17, 0xba, 0x50, 0xfd, 0x99, 0x34, 0xde, 0x73, 0x18, 0xb5, 0x5f, 0xf2, 0x96, 0x3b, 0xd1, 0x7c, 0x19, 0xb4, 0x5e, 0xf3, 0x97, 0x3a, 0xd0, 0x7d, 0x1a, 0xb7, 0x5d, 0xf0, 0x94, 0x39, 0xd3, 0x7e, 0x1b, 0xb6, 0x5c, 0xf1, 0x95, 0x38, 0xd2, 0x7f, 0x1c, 0xb1, 0x5b, 0xf6, 0x92, 0x3f, 0xd5, 0x78, 0x1d, 0xb0, 0x5a, 0xf7, 0x93, 0x3e, 0xd4, 0x79, 0x1e, 0xb3, 0x59, 0xf4, 0x90, 0x3d, 0xd7, 0x7a, 0x1f, 0xb2, 0x58, 0xf5, 0x91, 0x3c, 0xd6, 0x7b},
- {0x0, 0xae, 0x41, 0xef, 0x82, 0x2c, 0xc3, 0x6d, 0x19, 0xb7, 0x58, 0xf6, 0x9b, 0x35, 0xda, 0x74, 0x32, 0x9c, 0x73, 0xdd, 0xb0, 0x1e, 0xf1, 0x5f, 0x2b, 0x85, 0x6a, 0xc4, 0xa9, 0x7, 0xe8, 0x46, 0x64, 0xca, 0x25, 0x8b, 0xe6, 0x48, 0xa7, 0x9, 0x7d, 0xd3, 0x3c, 0x92, 0xff, 0x51, 0xbe, 0x10, 0x56, 0xf8, 0x17, 0xb9, 0xd4, 0x7a, 0x95, 0x3b, 0x4f, 0xe1, 0xe, 0xa0, 0xcd, 0x63, 0x8c, 0x22, 0xc8, 0x66, 0x89, 0x27, 0x4a, 0xe4, 0xb, 0xa5, 0xd1, 0x7f, 0x90, 0x3e, 0x53, 0xfd, 0x12, 0xbc, 0xfa, 0x54, 0xbb, 0x15, 0x78, 0xd6, 0x39, 0x97, 0xe3, 0x4d, 0xa2, 0xc, 0x61, 0xcf, 0x20, 0x8e, 0xac, 0x2, 0xed, 0x43, 0x2e, 0x80, 0x6f, 0xc1, 0xb5, 0x1b, 0xf4, 0x5a, 0x37, 0x99, 0x76, 0xd8, 0x9e, 0x30, 0xdf, 0x71, 0x1c, 0xb2, 0x5d, 0xf3, 0x87, 0x29, 0xc6, 0x68, 0x5, 0xab, 0x44, 0xea, 0x8d, 0x23, 0xcc, 0x62, 0xf, 0xa1, 0x4e, 0xe0, 0x94, 0x3a, 0xd5, 0x7b, 0x16, 0xb8, 0x57, 0xf9, 0xbf, 0x11, 0xfe, 0x50, 0x3d, 0x93, 0x7c, 0xd2, 0xa6, 0x8, 0xe7, 0x49, 0x24, 0x8a, 0x65, 0xcb, 0xe9, 0x47, 0xa8, 0x6, 0x6b, 0xc5, 0x2a, 0x84, 0xf0, 0x5e, 0xb1, 0x1f, 0x72, 0xdc, 0x33, 0x9d, 0xdb, 0x75, 0x9a, 0x34, 0x59, 0xf7, 0x18, 0xb6, 0xc2, 0x6c, 0x83, 0x2d, 0x40, 0xee, 0x1, 0xaf, 0x45, 0xeb, 0x4, 0xaa, 0xc7, 0x69, 0x86, 0x28, 0x5c, 0xf2, 0x1d, 0xb3, 0xde, 0x70, 0x9f, 0x31, 0x77, 0xd9, 0x36, 0x98, 0xf5, 0x5b, 0xb4, 0x1a, 0x6e, 0xc0, 0x2f, 0x81, 0xec, 0x42, 0xad, 0x3, 0x21, 0x8f, 0x60, 0xce, 0xa3, 0xd, 0xe2, 0x4c, 0x38, 0x96, 0x79, 0xd7, 0xba, 0x14, 0xfb, 0x55, 0x13, 0xbd, 0x52, 0xfc, 0x91, 0x3f, 0xd0, 0x7e, 0xa, 0xa4, 0x4b, 0xe5, 0x88, 0x26, 0xc9, 0x67},
- {0x0, 0xaf, 0x43, 0xec, 0x86, 0x29, 0xc5, 0x6a, 0x11, 0xbe, 0x52, 0xfd, 0x97, 0x38, 0xd4, 0x7b, 0x22, 0x8d, 0x61, 0xce, 0xa4, 0xb, 0xe7, 0x48, 0x33, 0x9c, 0x70, 0xdf, 0xb5, 0x1a, 0xf6, 0x59, 0x44, 0xeb, 0x7, 0xa8, 0xc2, 0x6d, 0x81, 0x2e, 0x55, 0xfa, 0x16, 0xb9, 0xd3, 0x7c, 0x90, 0x3f, 0x66, 0xc9, 0x25, 0x8a, 0xe0, 0x4f, 0xa3, 0xc, 0x77, 0xd8, 0x34, 0x9b, 0xf1, 0x5e, 0xb2, 0x1d, 0x88, 0x27, 0xcb, 0x64, 0xe, 0xa1, 0x4d, 0xe2, 0x99, 0x36, 0xda, 0x75, 0x1f, 0xb0, 0x5c, 0xf3, 0xaa, 0x5, 0xe9, 0x46, 0x2c, 0x83, 0x6f, 0xc0, 0xbb, 0x14, 0xf8, 0x57, 0x3d, 0x92, 0x7e, 0xd1, 0xcc, 0x63, 0x8f, 0x20, 0x4a, 0xe5, 0x9, 0xa6, 0xdd, 0x72, 0x9e, 0x31, 0x5b, 0xf4, 0x18, 0xb7, 0xee, 0x41, 0xad, 0x2, 0x68, 0xc7, 0x2b, 0x84, 0xff, 0x50, 0xbc, 0x13, 0x79, 0xd6, 0x3a, 0x95, 0xd, 0xa2, 0x4e, 0xe1, 0x8b, 0x24, 0xc8, 0x67, 0x1c, 0xb3, 0x5f, 0xf0, 0x9a, 0x35, 0xd9, 0x76, 0x2f, 0x80, 0x6c, 0xc3, 0xa9, 0x6, 0xea, 0x45, 0x3e, 0x91, 0x7d, 0xd2, 0xb8, 0x17, 0xfb, 0x54, 0x49, 0xe6, 0xa, 0xa5, 0xcf, 0x60, 0x8c, 0x23, 0x58, 0xf7, 0x1b, 0xb4, 0xde, 0x71, 0x9d, 0x32, 0x6b, 0xc4, 0x28, 0x87, 0xed, 0x42, 0xae, 0x1, 0x7a, 0xd5, 0x39, 0x96, 0xfc, 0x53, 0xbf, 0x10, 0x85, 0x2a, 0xc6, 0x69, 0x3, 0xac, 0x40, 0xef, 0x94, 0x3b, 0xd7, 0x78, 0x12, 0xbd, 0x51, 0xfe, 0xa7, 0x8, 0xe4, 0x4b, 0x21, 0x8e, 0x62, 0xcd, 0xb6, 0x19, 0xf5, 0x5a, 0x30, 0x9f, 0x73, 0xdc, 0xc1, 0x6e, 0x82, 0x2d, 0x47, 0xe8, 0x4, 0xab, 0xd0, 0x7f, 0x93, 0x3c, 0x56, 0xf9, 0x15, 0xba, 0xe3, 0x4c, 0xa0, 0xf, 0x65, 0xca, 0x26, 0x89, 0xf2, 0x5d, 0xb1, 0x1e, 0x74, 0xdb, 0x37, 0x98},
- {0x0, 0xb0, 0x7d, 0xcd, 0xfa, 0x4a, 0x87, 0x37, 0xe9, 0x59, 0x94, 0x24, 0x13, 0xa3, 0x6e, 0xde, 0xcf, 0x7f, 0xb2, 0x2, 0x35, 0x85, 0x48, 0xf8, 0x26, 0x96, 0x5b, 0xeb, 0xdc, 0x6c, 0xa1, 0x11, 0x83, 0x33, 0xfe, 0x4e, 0x79, 0xc9, 0x4, 0xb4, 0x6a, 0xda, 0x17, 0xa7, 0x90, 0x20, 0xed, 0x5d, 0x4c, 0xfc, 0x31, 0x81, 0xb6, 0x6, 0xcb, 0x7b, 0xa5, 0x15, 0xd8, 0x68, 0x5f, 0xef, 0x22, 0x92, 0x1b, 0xab, 0x66, 0xd6, 0xe1, 0x51, 0x9c, 0x2c, 0xf2, 0x42, 0x8f, 0x3f, 0x8, 0xb8, 0x75, 0xc5, 0xd4, 0x64, 0xa9, 0x19, 0x2e, 0x9e, 0x53, 0xe3, 0x3d, 0x8d, 0x40, 0xf0, 0xc7, 0x77, 0xba, 0xa, 0x98, 0x28, 0xe5, 0x55, 0x62, 0xd2, 0x1f, 0xaf, 0x71, 0xc1, 0xc, 0xbc, 0x8b, 0x3b, 0xf6, 0x46, 0x57, 0xe7, 0x2a, 0x9a, 0xad, 0x1d, 0xd0, 0x60, 0xbe, 0xe, 0xc3, 0x73, 0x44, 0xf4, 0x39, 0x89, 0x36, 0x86, 0x4b, 0xfb, 0xcc, 0x7c, 0xb1, 0x1, 0xdf, 0x6f, 0xa2, 0x12, 0x25, 0x95, 0x58, 0xe8, 0xf9, 0x49, 0x84, 0x34, 0x3, 0xb3, 0x7e, 0xce, 0x10, 0xa0, 0x6d, 0xdd, 0xea, 0x5a, 0x97, 0x27, 0xb5, 0x5, 0xc8, 0x78, 0x4f, 0xff, 0x32, 0x82, 0x5c, 0xec, 0x21, 0x91, 0xa6, 0x16, 0xdb, 0x6b, 0x7a, 0xca, 0x7, 0xb7, 0x80, 0x30, 0xfd, 0x4d, 0x93, 0x23, 0xee, 0x5e, 0x69, 0xd9, 0x14, 0xa4, 0x2d, 0x9d, 0x50, 0xe0, 0xd7, 0x67, 0xaa, 0x1a, 0xc4, 0x74, 0xb9, 0x9, 0x3e, 0x8e, 0x43, 0xf3, 0xe2, 0x52, 0x9f, 0x2f, 0x18, 0xa8, 0x65, 0xd5, 0xb, 0xbb, 0x76, 0xc6, 0xf1, 0x41, 0x8c, 0x3c, 0xae, 0x1e, 0xd3, 0x63, 0x54, 0xe4, 0x29, 0x99, 0x47, 0xf7, 0x3a, 0x8a, 0xbd, 0xd, 0xc0, 0x70, 0x61, 0xd1, 0x1c, 0xac, 0x9b, 0x2b, 0xe6, 0x56, 0x88, 0x38, 0xf5, 0x45, 0x72, 0xc2, 0xf, 0xbf},
- {0x0, 0xb1, 0x7f, 0xce, 0xfe, 0x4f, 0x81, 0x30, 0xe1, 0x50, 0x9e, 0x2f, 0x1f, 0xae, 0x60, 0xd1, 0xdf, 0x6e, 0xa0, 0x11, 0x21, 0x90, 0x5e, 0xef, 0x3e, 0x8f, 0x41, 0xf0, 0xc0, 0x71, 0xbf, 0xe, 0xa3, 0x12, 0xdc, 0x6d, 0x5d, 0xec, 0x22, 0x93, 0x42, 0xf3, 0x3d, 0x8c, 0xbc, 0xd, 0xc3, 0x72, 0x7c, 0xcd, 0x3, 0xb2, 0x82, 0x33, 0xfd, 0x4c, 0x9d, 0x2c, 0xe2, 0x53, 0x63, 0xd2, 0x1c, 0xad, 0x5b, 0xea, 0x24, 0x95, 0xa5, 0x14, 0xda, 0x6b, 0xba, 0xb, 0xc5, 0x74, 0x44, 0xf5, 0x3b, 0x8a, 0x84, 0x35, 0xfb, 0x4a, 0x7a, 0xcb, 0x5, 0xb4, 0x65, 0xd4, 0x1a, 0xab, 0x9b, 0x2a, 0xe4, 0x55, 0xf8, 0x49, 0x87, 0x36, 0x6, 0xb7, 0x79, 0xc8, 0x19, 0xa8, 0x66, 0xd7, 0xe7, 0x56, 0x98, 0x29, 0x27, 0x96, 0x58, 0xe9, 0xd9, 0x68, 0xa6, 0x17, 0xc6, 0x77, 0xb9, 0x8, 0x38, 0x89, 0x47, 0xf6, 0xb6, 0x7, 0xc9, 0x78, 0x48, 0xf9, 0x37, 0x86, 0x57, 0xe6, 0x28, 0x99, 0xa9, 0x18, 0xd6, 0x67, 0x69, 0xd8, 0x16, 0xa7, 0x97, 0x26, 0xe8, 0x59, 0x88, 0x39, 0xf7, 0x46, 0x76, 0xc7, 0x9, 0xb8, 0x15, 0xa4, 0x6a, 0xdb, 0xeb, 0x5a, 0x94, 0x25, 0xf4, 0x45, 0x8b, 0x3a, 0xa, 0xbb, 0x75, 0xc4, 0xca, 0x7b, 0xb5, 0x4, 0x34, 0x85, 0x4b, 0xfa, 0x2b, 0x9a, 0x54, 0xe5, 0xd5, 0x64, 0xaa, 0x1b, 0xed, 0x5c, 0x92, 0x23, 0x13, 0xa2, 0x6c, 0xdd, 0xc, 0xbd, 0x73, 0xc2, 0xf2, 0x43, 0x8d, 0x3c, 0x32, 0x83, 0x4d, 0xfc, 0xcc, 0x7d, 0xb3, 0x2, 0xd3, 0x62, 0xac, 0x1d, 0x2d, 0x9c, 0x52, 0xe3, 0x4e, 0xff, 0x31, 0x80, 0xb0, 0x1, 0xcf, 0x7e, 0xaf, 0x1e, 0xd0, 0x61, 0x51, 0xe0, 0x2e, 0x9f, 0x91, 0x20, 0xee, 0x5f, 0x6f, 0xde, 0x10, 0xa1, 0x70, 0xc1, 0xf, 0xbe, 0x8e, 0x3f, 0xf1, 0x40},
- {0x0, 0xb2, 0x79, 0xcb, 0xf2, 0x40, 0x8b, 0x39, 0xf9, 0x4b, 0x80, 0x32, 0xb, 0xb9, 0x72, 0xc0, 0xef, 0x5d, 0x96, 0x24, 0x1d, 0xaf, 0x64, 0xd6, 0x16, 0xa4, 0x6f, 0xdd, 0xe4, 0x56, 0x9d, 0x2f, 0xc3, 0x71, 0xba, 0x8, 0x31, 0x83, 0x48, 0xfa, 0x3a, 0x88, 0x43, 0xf1, 0xc8, 0x7a, 0xb1, 0x3, 0x2c, 0x9e, 0x55, 0xe7, 0xde, 0x6c, 0xa7, 0x15, 0xd5, 0x67, 0xac, 0x1e, 0x27, 0x95, 0x5e, 0xec, 0x9b, 0x29, 0xe2, 0x50, 0x69, 0xdb, 0x10, 0xa2, 0x62, 0xd0, 0x1b, 0xa9, 0x90, 0x22, 0xe9, 0x5b, 0x74, 0xc6, 0xd, 0xbf, 0x86, 0x34, 0xff, 0x4d, 0x8d, 0x3f, 0xf4, 0x46, 0x7f, 0xcd, 0x6, 0xb4, 0x58, 0xea, 0x21, 0x93, 0xaa, 0x18, 0xd3, 0x61, 0xa1, 0x13, 0xd8, 0x6a, 0x53, 0xe1, 0x2a, 0x98, 0xb7, 0x5, 0xce, 0x7c, 0x45, 0xf7, 0x3c, 0x8e, 0x4e, 0xfc, 0x37, 0x85, 0xbc, 0xe, 0xc5, 0x77, 0x2b, 0x99, 0x52, 0xe0, 0xd9, 0x6b, 0xa0, 0x12, 0xd2, 0x60, 0xab, 0x19, 0x20, 0x92, 0x59, 0xeb, 0xc4, 0x76, 0xbd, 0xf, 0x36, 0x84, 0x4f, 0xfd, 0x3d, 0x8f, 0x44, 0xf6, 0xcf, 0x7d, 0xb6, 0x4, 0xe8, 0x5a, 0x91, 0x23, 0x1a, 0xa8, 0x63, 0xd1, 0x11, 0xa3, 0x68, 0xda, 0xe3, 0x51, 0x9a, 0x28, 0x7, 0xb5, 0x7e, 0xcc, 0xf5, 0x47, 0x8c, 0x3e, 0xfe, 0x4c, 0x87, 0x35, 0xc, 0xbe, 0x75, 0xc7, 0xb0, 0x2, 0xc9, 0x7b, 0x42, 0xf0, 0x3b, 0x89, 0x49, 0xfb, 0x30, 0x82, 0xbb, 0x9, 0xc2, 0x70, 0x5f, 0xed, 0x26, 0x94, 0xad, 0x1f, 0xd4, 0x66, 0xa6, 0x14, 0xdf, 0x6d, 0x54, 0xe6, 0x2d, 0x9f, 0x73, 0xc1, 0xa, 0xb8, 0x81, 0x33, 0xf8, 0x4a, 0x8a, 0x38, 0xf3, 0x41, 0x78, 0xca, 0x1, 0xb3, 0x9c, 0x2e, 0xe5, 0x57, 0x6e, 0xdc, 0x17, 0xa5, 0x65, 0xd7, 0x1c, 0xae, 0x97, 0x25, 0xee, 0x5c},
- {0x0, 0xb3, 0x7b, 0xc8, 0xf6, 0x45, 0x8d, 0x3e, 0xf1, 0x42, 0x8a, 0x39, 0x7, 0xb4, 0x7c, 0xcf, 0xff, 0x4c, 0x84, 0x37, 0x9, 0xba, 0x72, 0xc1, 0xe, 0xbd, 0x75, 0xc6, 0xf8, 0x4b, 0x83, 0x30, 0xe3, 0x50, 0x98, 0x2b, 0x15, 0xa6, 0x6e, 0xdd, 0x12, 0xa1, 0x69, 0xda, 0xe4, 0x57, 0x9f, 0x2c, 0x1c, 0xaf, 0x67, 0xd4, 0xea, 0x59, 0x91, 0x22, 0xed, 0x5e, 0x96, 0x25, 0x1b, 0xa8, 0x60, 0xd3, 0xdb, 0x68, 0xa0, 0x13, 0x2d, 0x9e, 0x56, 0xe5, 0x2a, 0x99, 0x51, 0xe2, 0xdc, 0x6f, 0xa7, 0x14, 0x24, 0x97, 0x5f, 0xec, 0xd2, 0x61, 0xa9, 0x1a, 0xd5, 0x66, 0xae, 0x1d, 0x23, 0x90, 0x58, 0xeb, 0x38, 0x8b, 0x43, 0xf0, 0xce, 0x7d, 0xb5, 0x6, 0xc9, 0x7a, 0xb2, 0x1, 0x3f, 0x8c, 0x44, 0xf7, 0xc7, 0x74, 0xbc, 0xf, 0x31, 0x82, 0x4a, 0xf9, 0x36, 0x85, 0x4d, 0xfe, 0xc0, 0x73, 0xbb, 0x8, 0xab, 0x18, 0xd0, 0x63, 0x5d, 0xee, 0x26, 0x95, 0x5a, 0xe9, 0x21, 0x92, 0xac, 0x1f, 0xd7, 0x64, 0x54, 0xe7, 0x2f, 0x9c, 0xa2, 0x11, 0xd9, 0x6a, 0xa5, 0x16, 0xde, 0x6d, 0x53, 0xe0, 0x28, 0x9b, 0x48, 0xfb, 0x33, 0x80, 0xbe, 0xd, 0xc5, 0x76, 0xb9, 0xa, 0xc2, 0x71, 0x4f, 0xfc, 0x34, 0x87, 0xb7, 0x4, 0xcc, 0x7f, 0x41, 0xf2, 0x3a, 0x89, 0x46, 0xf5, 0x3d, 0x8e, 0xb0, 0x3, 0xcb, 0x78, 0x70, 0xc3, 0xb, 0xb8, 0x86, 0x35, 0xfd, 0x4e, 0x81, 0x32, 0xfa, 0x49, 0x77, 0xc4, 0xc, 0xbf, 0x8f, 0x3c, 0xf4, 0x47, 0x79, 0xca, 0x2, 0xb1, 0x7e, 0xcd, 0x5, 0xb6, 0x88, 0x3b, 0xf3, 0x40, 0x93, 0x20, 0xe8, 0x5b, 0x65, 0xd6, 0x1e, 0xad, 0x62, 0xd1, 0x19, 0xaa, 0x94, 0x27, 0xef, 0x5c, 0x6c, 0xdf, 0x17, 0xa4, 0x9a, 0x29, 0xe1, 0x52, 0x9d, 0x2e, 0xe6, 0x55, 0x6b, 0xd8, 0x10, 0xa3},
- {0x0, 0xb4, 0x75, 0xc1, 0xea, 0x5e, 0x9f, 0x2b, 0xc9, 0x7d, 0xbc, 0x8, 0x23, 0x97, 0x56, 0xe2, 0x8f, 0x3b, 0xfa, 0x4e, 0x65, 0xd1, 0x10, 0xa4, 0x46, 0xf2, 0x33, 0x87, 0xac, 0x18, 0xd9, 0x6d, 0x3, 0xb7, 0x76, 0xc2, 0xe9, 0x5d, 0x9c, 0x28, 0xca, 0x7e, 0xbf, 0xb, 0x20, 0x94, 0x55, 0xe1, 0x8c, 0x38, 0xf9, 0x4d, 0x66, 0xd2, 0x13, 0xa7, 0x45, 0xf1, 0x30, 0x84, 0xaf, 0x1b, 0xda, 0x6e, 0x6, 0xb2, 0x73, 0xc7, 0xec, 0x58, 0x99, 0x2d, 0xcf, 0x7b, 0xba, 0xe, 0x25, 0x91, 0x50, 0xe4, 0x89, 0x3d, 0xfc, 0x48, 0x63, 0xd7, 0x16, 0xa2, 0x40, 0xf4, 0x35, 0x81, 0xaa, 0x1e, 0xdf, 0x6b, 0x5, 0xb1, 0x70, 0xc4, 0xef, 0x5b, 0x9a, 0x2e, 0xcc, 0x78, 0xb9, 0xd, 0x26, 0x92, 0x53, 0xe7, 0x8a, 0x3e, 0xff, 0x4b, 0x60, 0xd4, 0x15, 0xa1, 0x43, 0xf7, 0x36, 0x82, 0xa9, 0x1d, 0xdc, 0x68, 0xc, 0xb8, 0x79, 0xcd, 0xe6, 0x52, 0x93, 0x27, 0xc5, 0x71, 0xb0, 0x4, 0x2f, 0x9b, 0x5a, 0xee, 0x83, 0x37, 0xf6, 0x42, 0x69, 0xdd, 0x1c, 0xa8, 0x4a, 0xfe, 0x3f, 0x8b, 0xa0, 0x14, 0xd5, 0x61, 0xf, 0xbb, 0x7a, 0xce, 0xe5, 0x51, 0x90, 0x24, 0xc6, 0x72, 0xb3, 0x7, 0x2c, 0x98, 0x59, 0xed, 0x80, 0x34, 0xf5, 0x41, 0x6a, 0xde, 0x1f, 0xab, 0x49, 0xfd, 0x3c, 0x88, 0xa3, 0x17, 0xd6, 0x62, 0xa, 0xbe, 0x7f, 0xcb, 0xe0, 0x54, 0x95, 0x21, 0xc3, 0x77, 0xb6, 0x2, 0x29, 0x9d, 0x5c, 0xe8, 0x85, 0x31, 0xf0, 0x44, 0x6f, 0xdb, 0x1a, 0xae, 0x4c, 0xf8, 0x39, 0x8d, 0xa6, 0x12, 0xd3, 0x67, 0x9, 0xbd, 0x7c, 0xc8, 0xe3, 0x57, 0x96, 0x22, 0xc0, 0x74, 0xb5, 0x1, 0x2a, 0x9e, 0x5f, 0xeb, 0x86, 0x32, 0xf3, 0x47, 0x6c, 0xd8, 0x19, 0xad, 0x4f, 0xfb, 0x3a, 0x8e, 0xa5, 0x11, 0xd0, 0x64},
- {0x0, 0xb5, 0x77, 0xc2, 0xee, 0x5b, 0x99, 0x2c, 0xc1, 0x74, 0xb6, 0x3, 0x2f, 0x9a, 0x58, 0xed, 0x9f, 0x2a, 0xe8, 0x5d, 0x71, 0xc4, 0x6, 0xb3, 0x5e, 0xeb, 0x29, 0x9c, 0xb0, 0x5, 0xc7, 0x72, 0x23, 0x96, 0x54, 0xe1, 0xcd, 0x78, 0xba, 0xf, 0xe2, 0x57, 0x95, 0x20, 0xc, 0xb9, 0x7b, 0xce, 0xbc, 0x9, 0xcb, 0x7e, 0x52, 0xe7, 0x25, 0x90, 0x7d, 0xc8, 0xa, 0xbf, 0x93, 0x26, 0xe4, 0x51, 0x46, 0xf3, 0x31, 0x84, 0xa8, 0x1d, 0xdf, 0x6a, 0x87, 0x32, 0xf0, 0x45, 0x69, 0xdc, 0x1e, 0xab, 0xd9, 0x6c, 0xae, 0x1b, 0x37, 0x82, 0x40, 0xf5, 0x18, 0xad, 0x6f, 0xda, 0xf6, 0x43, 0x81, 0x34, 0x65, 0xd0, 0x12, 0xa7, 0x8b, 0x3e, 0xfc, 0x49, 0xa4, 0x11, 0xd3, 0x66, 0x4a, 0xff, 0x3d, 0x88, 0xfa, 0x4f, 0x8d, 0x38, 0x14, 0xa1, 0x63, 0xd6, 0x3b, 0x8e, 0x4c, 0xf9, 0xd5, 0x60, 0xa2, 0x17, 0x8c, 0x39, 0xfb, 0x4e, 0x62, 0xd7, 0x15, 0xa0, 0x4d, 0xf8, 0x3a, 0x8f, 0xa3, 0x16, 0xd4, 0x61, 0x13, 0xa6, 0x64, 0xd1, 0xfd, 0x48, 0x8a, 0x3f, 0xd2, 0x67, 0xa5, 0x10, 0x3c, 0x89, 0x4b, 0xfe, 0xaf, 0x1a, 0xd8, 0x6d, 0x41, 0xf4, 0x36, 0x83, 0x6e, 0xdb, 0x19, 0xac, 0x80, 0x35, 0xf7, 0x42, 0x30, 0x85, 0x47, 0xf2, 0xde, 0x6b, 0xa9, 0x1c, 0xf1, 0x44, 0x86, 0x33, 0x1f, 0xaa, 0x68, 0xdd, 0xca, 0x7f, 0xbd, 0x8, 0x24, 0x91, 0x53, 0xe6, 0xb, 0xbe, 0x7c, 0xc9, 0xe5, 0x50, 0x92, 0x27, 0x55, 0xe0, 0x22, 0x97, 0xbb, 0xe, 0xcc, 0x79, 0x94, 0x21, 0xe3, 0x56, 0x7a, 0xcf, 0xd, 0xb8, 0xe9, 0x5c, 0x9e, 0x2b, 0x7, 0xb2, 0x70, 0xc5, 0x28, 0x9d, 0x5f, 0xea, 0xc6, 0x73, 0xb1, 0x4, 0x76, 0xc3, 0x1, 0xb4, 0x98, 0x2d, 0xef, 0x5a, 0xb7, 0x2, 0xc0, 0x75, 0x59, 0xec, 0x2e, 0x9b},
- {0x0, 0xb6, 0x71, 0xc7, 0xe2, 0x54, 0x93, 0x25, 0xd9, 0x6f, 0xa8, 0x1e, 0x3b, 0x8d, 0x4a, 0xfc, 0xaf, 0x19, 0xde, 0x68, 0x4d, 0xfb, 0x3c, 0x8a, 0x76, 0xc0, 0x7, 0xb1, 0x94, 0x22, 0xe5, 0x53, 0x43, 0xf5, 0x32, 0x84, 0xa1, 0x17, 0xd0, 0x66, 0x9a, 0x2c, 0xeb, 0x5d, 0x78, 0xce, 0x9, 0xbf, 0xec, 0x5a, 0x9d, 0x2b, 0xe, 0xb8, 0x7f, 0xc9, 0x35, 0x83, 0x44, 0xf2, 0xd7, 0x61, 0xa6, 0x10, 0x86, 0x30, 0xf7, 0x41, 0x64, 0xd2, 0x15, 0xa3, 0x5f, 0xe9, 0x2e, 0x98, 0xbd, 0xb, 0xcc, 0x7a, 0x29, 0x9f, 0x58, 0xee, 0xcb, 0x7d, 0xba, 0xc, 0xf0, 0x46, 0x81, 0x37, 0x12, 0xa4, 0x63, 0xd5, 0xc5, 0x73, 0xb4, 0x2, 0x27, 0x91, 0x56, 0xe0, 0x1c, 0xaa, 0x6d, 0xdb, 0xfe, 0x48, 0x8f, 0x39, 0x6a, 0xdc, 0x1b, 0xad, 0x88, 0x3e, 0xf9, 0x4f, 0xb3, 0x5, 0xc2, 0x74, 0x51, 0xe7, 0x20, 0x96, 0x11, 0xa7, 0x60, 0xd6, 0xf3, 0x45, 0x82, 0x34, 0xc8, 0x7e, 0xb9, 0xf, 0x2a, 0x9c, 0x5b, 0xed, 0xbe, 0x8, 0xcf, 0x79, 0x5c, 0xea, 0x2d, 0x9b, 0x67, 0xd1, 0x16, 0xa0, 0x85, 0x33, 0xf4, 0x42, 0x52, 0xe4, 0x23, 0x95, 0xb0, 0x6, 0xc1, 0x77, 0x8b, 0x3d, 0xfa, 0x4c, 0x69, 0xdf, 0x18, 0xae, 0xfd, 0x4b, 0x8c, 0x3a, 0x1f, 0xa9, 0x6e, 0xd8, 0x24, 0x92, 0x55, 0xe3, 0xc6, 0x70, 0xb7, 0x1, 0x97, 0x21, 0xe6, 0x50, 0x75, 0xc3, 0x4, 0xb2, 0x4e, 0xf8, 0x3f, 0x89, 0xac, 0x1a, 0xdd, 0x6b, 0x38, 0x8e, 0x49, 0xff, 0xda, 0x6c, 0xab, 0x1d, 0xe1, 0x57, 0x90, 0x26, 0x3, 0xb5, 0x72, 0xc4, 0xd4, 0x62, 0xa5, 0x13, 0x36, 0x80, 0x47, 0xf1, 0xd, 0xbb, 0x7c, 0xca, 0xef, 0x59, 0x9e, 0x28, 0x7b, 0xcd, 0xa, 0xbc, 0x99, 0x2f, 0xe8, 0x5e, 0xa2, 0x14, 0xd3, 0x65, 0x40, 0xf6, 0x31, 0x87},
- {0x0, 0xb7, 0x73, 0xc4, 0xe6, 0x51, 0x95, 0x22, 0xd1, 0x66, 0xa2, 0x15, 0x37, 0x80, 0x44, 0xf3, 0xbf, 0x8, 0xcc, 0x7b, 0x59, 0xee, 0x2a, 0x9d, 0x6e, 0xd9, 0x1d, 0xaa, 0x88, 0x3f, 0xfb, 0x4c, 0x63, 0xd4, 0x10, 0xa7, 0x85, 0x32, 0xf6, 0x41, 0xb2, 0x5, 0xc1, 0x76, 0x54, 0xe3, 0x27, 0x90, 0xdc, 0x6b, 0xaf, 0x18, 0x3a, 0x8d, 0x49, 0xfe, 0xd, 0xba, 0x7e, 0xc9, 0xeb, 0x5c, 0x98, 0x2f, 0xc6, 0x71, 0xb5, 0x2, 0x20, 0x97, 0x53, 0xe4, 0x17, 0xa0, 0x64, 0xd3, 0xf1, 0x46, 0x82, 0x35, 0x79, 0xce, 0xa, 0xbd, 0x9f, 0x28, 0xec, 0x5b, 0xa8, 0x1f, 0xdb, 0x6c, 0x4e, 0xf9, 0x3d, 0x8a, 0xa5, 0x12, 0xd6, 0x61, 0x43, 0xf4, 0x30, 0x87, 0x74, 0xc3, 0x7, 0xb0, 0x92, 0x25, 0xe1, 0x56, 0x1a, 0xad, 0x69, 0xde, 0xfc, 0x4b, 0x8f, 0x38, 0xcb, 0x7c, 0xb8, 0xf, 0x2d, 0x9a, 0x5e, 0xe9, 0x91, 0x26, 0xe2, 0x55, 0x77, 0xc0, 0x4, 0xb3, 0x40, 0xf7, 0x33, 0x84, 0xa6, 0x11, 0xd5, 0x62, 0x2e, 0x99, 0x5d, 0xea, 0xc8, 0x7f, 0xbb, 0xc, 0xff, 0x48, 0x8c, 0x3b, 0x19, 0xae, 0x6a, 0xdd, 0xf2, 0x45, 0x81, 0x36, 0x14, 0xa3, 0x67, 0xd0, 0x23, 0x94, 0x50, 0xe7, 0xc5, 0x72, 0xb6, 0x1, 0x4d, 0xfa, 0x3e, 0x89, 0xab, 0x1c, 0xd8, 0x6f, 0x9c, 0x2b, 0xef, 0x58, 0x7a, 0xcd, 0x9, 0xbe, 0x57, 0xe0, 0x24, 0x93, 0xb1, 0x6, 0xc2, 0x75, 0x86, 0x31, 0xf5, 0x42, 0x60, 0xd7, 0x13, 0xa4, 0xe8, 0x5f, 0x9b, 0x2c, 0xe, 0xb9, 0x7d, 0xca, 0x39, 0x8e, 0x4a, 0xfd, 0xdf, 0x68, 0xac, 0x1b, 0x34, 0x83, 0x47, 0xf0, 0xd2, 0x65, 0xa1, 0x16, 0xe5, 0x52, 0x96, 0x21, 0x3, 0xb4, 0x70, 0xc7, 0x8b, 0x3c, 0xf8, 0x4f, 0x6d, 0xda, 0x1e, 0xa9, 0x5a, 0xed, 0x29, 0x9e, 0xbc, 0xb, 0xcf, 0x78},
- {0x0, 0xb8, 0x6d, 0xd5, 0xda, 0x62, 0xb7, 0xf, 0xa9, 0x11, 0xc4, 0x7c, 0x73, 0xcb, 0x1e, 0xa6, 0x4f, 0xf7, 0x22, 0x9a, 0x95, 0x2d, 0xf8, 0x40, 0xe6, 0x5e, 0x8b, 0x33, 0x3c, 0x84, 0x51, 0xe9, 0x9e, 0x26, 0xf3, 0x4b, 0x44, 0xfc, 0x29, 0x91, 0x37, 0x8f, 0x5a, 0xe2, 0xed, 0x55, 0x80, 0x38, 0xd1, 0x69, 0xbc, 0x4, 0xb, 0xb3, 0x66, 0xde, 0x78, 0xc0, 0x15, 0xad, 0xa2, 0x1a, 0xcf, 0x77, 0x21, 0x99, 0x4c, 0xf4, 0xfb, 0x43, 0x96, 0x2e, 0x88, 0x30, 0xe5, 0x5d, 0x52, 0xea, 0x3f, 0x87, 0x6e, 0xd6, 0x3, 0xbb, 0xb4, 0xc, 0xd9, 0x61, 0xc7, 0x7f, 0xaa, 0x12, 0x1d, 0xa5, 0x70, 0xc8, 0xbf, 0x7, 0xd2, 0x6a, 0x65, 0xdd, 0x8, 0xb0, 0x16, 0xae, 0x7b, 0xc3, 0xcc, 0x74, 0xa1, 0x19, 0xf0, 0x48, 0x9d, 0x25, 0x2a, 0x92, 0x47, 0xff, 0x59, 0xe1, 0x34, 0x8c, 0x83, 0x3b, 0xee, 0x56, 0x42, 0xfa, 0x2f, 0x97, 0x98, 0x20, 0xf5, 0x4d, 0xeb, 0x53, 0x86, 0x3e, 0x31, 0x89, 0x5c, 0xe4, 0xd, 0xb5, 0x60, 0xd8, 0xd7, 0x6f, 0xba, 0x2, 0xa4, 0x1c, 0xc9, 0x71, 0x7e, 0xc6, 0x13, 0xab, 0xdc, 0x64, 0xb1, 0x9, 0x6, 0xbe, 0x6b, 0xd3, 0x75, 0xcd, 0x18, 0xa0, 0xaf, 0x17, 0xc2, 0x7a, 0x93, 0x2b, 0xfe, 0x46, 0x49, 0xf1, 0x24, 0x9c, 0x3a, 0x82, 0x57, 0xef, 0xe0, 0x58, 0x8d, 0x35, 0x63, 0xdb, 0xe, 0xb6, 0xb9, 0x1, 0xd4, 0x6c, 0xca, 0x72, 0xa7, 0x1f, 0x10, 0xa8, 0x7d, 0xc5, 0x2c, 0x94, 0x41, 0xf9, 0xf6, 0x4e, 0x9b, 0x23, 0x85, 0x3d, 0xe8, 0x50, 0x5f, 0xe7, 0x32, 0x8a, 0xfd, 0x45, 0x90, 0x28, 0x27, 0x9f, 0x4a, 0xf2, 0x54, 0xec, 0x39, 0x81, 0x8e, 0x36, 0xe3, 0x5b, 0xb2, 0xa, 0xdf, 0x67, 0x68, 0xd0, 0x5, 0xbd, 0x1b, 0xa3, 0x76, 0xce, 0xc1, 0x79, 0xac, 0x14},
- {0x0, 0xb9, 0x6f, 0xd6, 0xde, 0x67, 0xb1, 0x8, 0xa1, 0x18, 0xce, 0x77, 0x7f, 0xc6, 0x10, 0xa9, 0x5f, 0xe6, 0x30, 0x89, 0x81, 0x38, 0xee, 0x57, 0xfe, 0x47, 0x91, 0x28, 0x20, 0x99, 0x4f, 0xf6, 0xbe, 0x7, 0xd1, 0x68, 0x60, 0xd9, 0xf, 0xb6, 0x1f, 0xa6, 0x70, 0xc9, 0xc1, 0x78, 0xae, 0x17, 0xe1, 0x58, 0x8e, 0x37, 0x3f, 0x86, 0x50, 0xe9, 0x40, 0xf9, 0x2f, 0x96, 0x9e, 0x27, 0xf1, 0x48, 0x61, 0xd8, 0xe, 0xb7, 0xbf, 0x6, 0xd0, 0x69, 0xc0, 0x79, 0xaf, 0x16, 0x1e, 0xa7, 0x71, 0xc8, 0x3e, 0x87, 0x51, 0xe8, 0xe0, 0x59, 0x8f, 0x36, 0x9f, 0x26, 0xf0, 0x49, 0x41, 0xf8, 0x2e, 0x97, 0xdf, 0x66, 0xb0, 0x9, 0x1, 0xb8, 0x6e, 0xd7, 0x7e, 0xc7, 0x11, 0xa8, 0xa0, 0x19, 0xcf, 0x76, 0x80, 0x39, 0xef, 0x56, 0x5e, 0xe7, 0x31, 0x88, 0x21, 0x98, 0x4e, 0xf7, 0xff, 0x46, 0x90, 0x29, 0xc2, 0x7b, 0xad, 0x14, 0x1c, 0xa5, 0x73, 0xca, 0x63, 0xda, 0xc, 0xb5, 0xbd, 0x4, 0xd2, 0x6b, 0x9d, 0x24, 0xf2, 0x4b, 0x43, 0xfa, 0x2c, 0x95, 0x3c, 0x85, 0x53, 0xea, 0xe2, 0x5b, 0x8d, 0x34, 0x7c, 0xc5, 0x13, 0xaa, 0xa2, 0x1b, 0xcd, 0x74, 0xdd, 0x64, 0xb2, 0xb, 0x3, 0xba, 0x6c, 0xd5, 0x23, 0x9a, 0x4c, 0xf5, 0xfd, 0x44, 0x92, 0x2b, 0x82, 0x3b, 0xed, 0x54, 0x5c, 0xe5, 0x33, 0x8a, 0xa3, 0x1a, 0xcc, 0x75, 0x7d, 0xc4, 0x12, 0xab, 0x2, 0xbb, 0x6d, 0xd4, 0xdc, 0x65, 0xb3, 0xa, 0xfc, 0x45, 0x93, 0x2a, 0x22, 0x9b, 0x4d, 0xf4, 0x5d, 0xe4, 0x32, 0x8b, 0x83, 0x3a, 0xec, 0x55, 0x1d, 0xa4, 0x72, 0xcb, 0xc3, 0x7a, 0xac, 0x15, 0xbc, 0x5, 0xd3, 0x6a, 0x62, 0xdb, 0xd, 0xb4, 0x42, 0xfb, 0x2d, 0x94, 0x9c, 0x25, 0xf3, 0x4a, 0xe3, 0x5a, 0x8c, 0x35, 0x3d, 0x84, 0x52, 0xeb},
- {0x0, 0xba, 0x69, 0xd3, 0xd2, 0x68, 0xbb, 0x1, 0xb9, 0x3, 0xd0, 0x6a, 0x6b, 0xd1, 0x2, 0xb8, 0x6f, 0xd5, 0x6, 0xbc, 0xbd, 0x7, 0xd4, 0x6e, 0xd6, 0x6c, 0xbf, 0x5, 0x4, 0xbe, 0x6d, 0xd7, 0xde, 0x64, 0xb7, 0xd, 0xc, 0xb6, 0x65, 0xdf, 0x67, 0xdd, 0xe, 0xb4, 0xb5, 0xf, 0xdc, 0x66, 0xb1, 0xb, 0xd8, 0x62, 0x63, 0xd9, 0xa, 0xb0, 0x8, 0xb2, 0x61, 0xdb, 0xda, 0x60, 0xb3, 0x9, 0xa1, 0x1b, 0xc8, 0x72, 0x73, 0xc9, 0x1a, 0xa0, 0x18, 0xa2, 0x71, 0xcb, 0xca, 0x70, 0xa3, 0x19, 0xce, 0x74, 0xa7, 0x1d, 0x1c, 0xa6, 0x75, 0xcf, 0x77, 0xcd, 0x1e, 0xa4, 0xa5, 0x1f, 0xcc, 0x76, 0x7f, 0xc5, 0x16, 0xac, 0xad, 0x17, 0xc4, 0x7e, 0xc6, 0x7c, 0xaf, 0x15, 0x14, 0xae, 0x7d, 0xc7, 0x10, 0xaa, 0x79, 0xc3, 0xc2, 0x78, 0xab, 0x11, 0xa9, 0x13, 0xc0, 0x7a, 0x7b, 0xc1, 0x12, 0xa8, 0x5f, 0xe5, 0x36, 0x8c, 0x8d, 0x37, 0xe4, 0x5e, 0xe6, 0x5c, 0x8f, 0x35, 0x34, 0x8e, 0x5d, 0xe7, 0x30, 0x8a, 0x59, 0xe3, 0xe2, 0x58, 0x8b, 0x31, 0x89, 0x33, 0xe0, 0x5a, 0x5b, 0xe1, 0x32, 0x88, 0x81, 0x3b, 0xe8, 0x52, 0x53, 0xe9, 0x3a, 0x80, 0x38, 0x82, 0x51, 0xeb, 0xea, 0x50, 0x83, 0x39, 0xee, 0x54, 0x87, 0x3d, 0x3c, 0x86, 0x55, 0xef, 0x57, 0xed, 0x3e, 0x84, 0x85, 0x3f, 0xec, 0x56, 0xfe, 0x44, 0x97, 0x2d, 0x2c, 0x96, 0x45, 0xff, 0x47, 0xfd, 0x2e, 0x94, 0x95, 0x2f, 0xfc, 0x46, 0x91, 0x2b, 0xf8, 0x42, 0x43, 0xf9, 0x2a, 0x90, 0x28, 0x92, 0x41, 0xfb, 0xfa, 0x40, 0x93, 0x29, 0x20, 0x9a, 0x49, 0xf3, 0xf2, 0x48, 0x9b, 0x21, 0x99, 0x23, 0xf0, 0x4a, 0x4b, 0xf1, 0x22, 0x98, 0x4f, 0xf5, 0x26, 0x9c, 0x9d, 0x27, 0xf4, 0x4e, 0xf6, 0x4c, 0x9f, 0x25, 0x24, 0x9e, 0x4d, 0xf7},
- {0x0, 0xbb, 0x6b, 0xd0, 0xd6, 0x6d, 0xbd, 0x6, 0xb1, 0xa, 0xda, 0x61, 0x67, 0xdc, 0xc, 0xb7, 0x7f, 0xc4, 0x14, 0xaf, 0xa9, 0x12, 0xc2, 0x79, 0xce, 0x75, 0xa5, 0x1e, 0x18, 0xa3, 0x73, 0xc8, 0xfe, 0x45, 0x95, 0x2e, 0x28, 0x93, 0x43, 0xf8, 0x4f, 0xf4, 0x24, 0x9f, 0x99, 0x22, 0xf2, 0x49, 0x81, 0x3a, 0xea, 0x51, 0x57, 0xec, 0x3c, 0x87, 0x30, 0x8b, 0x5b, 0xe0, 0xe6, 0x5d, 0x8d, 0x36, 0xe1, 0x5a, 0x8a, 0x31, 0x37, 0x8c, 0x5c, 0xe7, 0x50, 0xeb, 0x3b, 0x80, 0x86, 0x3d, 0xed, 0x56, 0x9e, 0x25, 0xf5, 0x4e, 0x48, 0xf3, 0x23, 0x98, 0x2f, 0x94, 0x44, 0xff, 0xf9, 0x42, 0x92, 0x29, 0x1f, 0xa4, 0x74, 0xcf, 0xc9, 0x72, 0xa2, 0x19, 0xae, 0x15, 0xc5, 0x7e, 0x78, 0xc3, 0x13, 0xa8, 0x60, 0xdb, 0xb, 0xb0, 0xb6, 0xd, 0xdd, 0x66, 0xd1, 0x6a, 0xba, 0x1, 0x7, 0xbc, 0x6c, 0xd7, 0xdf, 0x64, 0xb4, 0xf, 0x9, 0xb2, 0x62, 0xd9, 0x6e, 0xd5, 0x5, 0xbe, 0xb8, 0x3, 0xd3, 0x68, 0xa0, 0x1b, 0xcb, 0x70, 0x76, 0xcd, 0x1d, 0xa6, 0x11, 0xaa, 0x7a, 0xc1, 0xc7, 0x7c, 0xac, 0x17, 0x21, 0x9a, 0x4a, 0xf1, 0xf7, 0x4c, 0x9c, 0x27, 0x90, 0x2b, 0xfb, 0x40, 0x46, 0xfd, 0x2d, 0x96, 0x5e, 0xe5, 0x35, 0x8e, 0x88, 0x33, 0xe3, 0x58, 0xef, 0x54, 0x84, 0x3f, 0x39, 0x82, 0x52, 0xe9, 0x3e, 0x85, 0x55, 0xee, 0xe8, 0x53, 0x83, 0x38, 0x8f, 0x34, 0xe4, 0x5f, 0x59, 0xe2, 0x32, 0x89, 0x41, 0xfa, 0x2a, 0x91, 0x97, 0x2c, 0xfc, 0x47, 0xf0, 0x4b, 0x9b, 0x20, 0x26, 0x9d, 0x4d, 0xf6, 0xc0, 0x7b, 0xab, 0x10, 0x16, 0xad, 0x7d, 0xc6, 0x71, 0xca, 0x1a, 0xa1, 0xa7, 0x1c, 0xcc, 0x77, 0xbf, 0x4, 0xd4, 0x6f, 0x69, 0xd2, 0x2, 0xb9, 0xe, 0xb5, 0x65, 0xde, 0xd8, 0x63, 0xb3, 0x8},
- {0x0, 0xbc, 0x65, 0xd9, 0xca, 0x76, 0xaf, 0x13, 0x89, 0x35, 0xec, 0x50, 0x43, 0xff, 0x26, 0x9a, 0xf, 0xb3, 0x6a, 0xd6, 0xc5, 0x79, 0xa0, 0x1c, 0x86, 0x3a, 0xe3, 0x5f, 0x4c, 0xf0, 0x29, 0x95, 0x1e, 0xa2, 0x7b, 0xc7, 0xd4, 0x68, 0xb1, 0xd, 0x97, 0x2b, 0xf2, 0x4e, 0x5d, 0xe1, 0x38, 0x84, 0x11, 0xad, 0x74, 0xc8, 0xdb, 0x67, 0xbe, 0x2, 0x98, 0x24, 0xfd, 0x41, 0x52, 0xee, 0x37, 0x8b, 0x3c, 0x80, 0x59, 0xe5, 0xf6, 0x4a, 0x93, 0x2f, 0xb5, 0x9, 0xd0, 0x6c, 0x7f, 0xc3, 0x1a, 0xa6, 0x33, 0x8f, 0x56, 0xea, 0xf9, 0x45, 0x9c, 0x20, 0xba, 0x6, 0xdf, 0x63, 0x70, 0xcc, 0x15, 0xa9, 0x22, 0x9e, 0x47, 0xfb, 0xe8, 0x54, 0x8d, 0x31, 0xab, 0x17, 0xce, 0x72, 0x61, 0xdd, 0x4, 0xb8, 0x2d, 0x91, 0x48, 0xf4, 0xe7, 0x5b, 0x82, 0x3e, 0xa4, 0x18, 0xc1, 0x7d, 0x6e, 0xd2, 0xb, 0xb7, 0x78, 0xc4, 0x1d, 0xa1, 0xb2, 0xe, 0xd7, 0x6b, 0xf1, 0x4d, 0x94, 0x28, 0x3b, 0x87, 0x5e, 0xe2, 0x77, 0xcb, 0x12, 0xae, 0xbd, 0x1, 0xd8, 0x64, 0xfe, 0x42, 0x9b, 0x27, 0x34, 0x88, 0x51, 0xed, 0x66, 0xda, 0x3, 0xbf, 0xac, 0x10, 0xc9, 0x75, 0xef, 0x53, 0x8a, 0x36, 0x25, 0x99, 0x40, 0xfc, 0x69, 0xd5, 0xc, 0xb0, 0xa3, 0x1f, 0xc6, 0x7a, 0xe0, 0x5c, 0x85, 0x39, 0x2a, 0x96, 0x4f, 0xf3, 0x44, 0xf8, 0x21, 0x9d, 0x8e, 0x32, 0xeb, 0x57, 0xcd, 0x71, 0xa8, 0x14, 0x7, 0xbb, 0x62, 0xde, 0x4b, 0xf7, 0x2e, 0x92, 0x81, 0x3d, 0xe4, 0x58, 0xc2, 0x7e, 0xa7, 0x1b, 0x8, 0xb4, 0x6d, 0xd1, 0x5a, 0xe6, 0x3f, 0x83, 0x90, 0x2c, 0xf5, 0x49, 0xd3, 0x6f, 0xb6, 0xa, 0x19, 0xa5, 0x7c, 0xc0, 0x55, 0xe9, 0x30, 0x8c, 0x9f, 0x23, 0xfa, 0x46, 0xdc, 0x60, 0xb9, 0x5, 0x16, 0xaa, 0x73, 0xcf},
- {0x0, 0xbd, 0x67, 0xda, 0xce, 0x73, 0xa9, 0x14, 0x81, 0x3c, 0xe6, 0x5b, 0x4f, 0xf2, 0x28, 0x95, 0x1f, 0xa2, 0x78, 0xc5, 0xd1, 0x6c, 0xb6, 0xb, 0x9e, 0x23, 0xf9, 0x44, 0x50, 0xed, 0x37, 0x8a, 0x3e, 0x83, 0x59, 0xe4, 0xf0, 0x4d, 0x97, 0x2a, 0xbf, 0x2, 0xd8, 0x65, 0x71, 0xcc, 0x16, 0xab, 0x21, 0x9c, 0x46, 0xfb, 0xef, 0x52, 0x88, 0x35, 0xa0, 0x1d, 0xc7, 0x7a, 0x6e, 0xd3, 0x9, 0xb4, 0x7c, 0xc1, 0x1b, 0xa6, 0xb2, 0xf, 0xd5, 0x68, 0xfd, 0x40, 0x9a, 0x27, 0x33, 0x8e, 0x54, 0xe9, 0x63, 0xde, 0x4, 0xb9, 0xad, 0x10, 0xca, 0x77, 0xe2, 0x5f, 0x85, 0x38, 0x2c, 0x91, 0x4b, 0xf6, 0x42, 0xff, 0x25, 0x98, 0x8c, 0x31, 0xeb, 0x56, 0xc3, 0x7e, 0xa4, 0x19, 0xd, 0xb0, 0x6a, 0xd7, 0x5d, 0xe0, 0x3a, 0x87, 0x93, 0x2e, 0xf4, 0x49, 0xdc, 0x61, 0xbb, 0x6, 0x12, 0xaf, 0x75, 0xc8, 0xf8, 0x45, 0x9f, 0x22, 0x36, 0x8b, 0x51, 0xec, 0x79, 0xc4, 0x1e, 0xa3, 0xb7, 0xa, 0xd0, 0x6d, 0xe7, 0x5a, 0x80, 0x3d, 0x29, 0x94, 0x4e, 0xf3, 0x66, 0xdb, 0x1, 0xbc, 0xa8, 0x15, 0xcf, 0x72, 0xc6, 0x7b, 0xa1, 0x1c, 0x8, 0xb5, 0x6f, 0xd2, 0x47, 0xfa, 0x20, 0x9d, 0x89, 0x34, 0xee, 0x53, 0xd9, 0x64, 0xbe, 0x3, 0x17, 0xaa, 0x70, 0xcd, 0x58, 0xe5, 0x3f, 0x82, 0x96, 0x2b, 0xf1, 0x4c, 0x84, 0x39, 0xe3, 0x5e, 0x4a, 0xf7, 0x2d, 0x90, 0x5, 0xb8, 0x62, 0xdf, 0xcb, 0x76, 0xac, 0x11, 0x9b, 0x26, 0xfc, 0x41, 0x55, 0xe8, 0x32, 0x8f, 0x1a, 0xa7, 0x7d, 0xc0, 0xd4, 0x69, 0xb3, 0xe, 0xba, 0x7, 0xdd, 0x60, 0x74, 0xc9, 0x13, 0xae, 0x3b, 0x86, 0x5c, 0xe1, 0xf5, 0x48, 0x92, 0x2f, 0xa5, 0x18, 0xc2, 0x7f, 0x6b, 0xd6, 0xc, 0xb1, 0x24, 0x99, 0x43, 0xfe, 0xea, 0x57, 0x8d, 0x30},
- {0x0, 0xbe, 0x61, 0xdf, 0xc2, 0x7c, 0xa3, 0x1d, 0x99, 0x27, 0xf8, 0x46, 0x5b, 0xe5, 0x3a, 0x84, 0x2f, 0x91, 0x4e, 0xf0, 0xed, 0x53, 0x8c, 0x32, 0xb6, 0x8, 0xd7, 0x69, 0x74, 0xca, 0x15, 0xab, 0x5e, 0xe0, 0x3f, 0x81, 0x9c, 0x22, 0xfd, 0x43, 0xc7, 0x79, 0xa6, 0x18, 0x5, 0xbb, 0x64, 0xda, 0x71, 0xcf, 0x10, 0xae, 0xb3, 0xd, 0xd2, 0x6c, 0xe8, 0x56, 0x89, 0x37, 0x2a, 0x94, 0x4b, 0xf5, 0xbc, 0x2, 0xdd, 0x63, 0x7e, 0xc0, 0x1f, 0xa1, 0x25, 0x9b, 0x44, 0xfa, 0xe7, 0x59, 0x86, 0x38, 0x93, 0x2d, 0xf2, 0x4c, 0x51, 0xef, 0x30, 0x8e, 0xa, 0xb4, 0x6b, 0xd5, 0xc8, 0x76, 0xa9, 0x17, 0xe2, 0x5c, 0x83, 0x3d, 0x20, 0x9e, 0x41, 0xff, 0x7b, 0xc5, 0x1a, 0xa4, 0xb9, 0x7, 0xd8, 0x66, 0xcd, 0x73, 0xac, 0x12, 0xf, 0xb1, 0x6e, 0xd0, 0x54, 0xea, 0x35, 0x8b, 0x96, 0x28, 0xf7, 0x49, 0x65, 0xdb, 0x4, 0xba, 0xa7, 0x19, 0xc6, 0x78, 0xfc, 0x42, 0x9d, 0x23, 0x3e, 0x80, 0x5f, 0xe1, 0x4a, 0xf4, 0x2b, 0x95, 0x88, 0x36, 0xe9, 0x57, 0xd3, 0x6d, 0xb2, 0xc, 0x11, 0xaf, 0x70, 0xce, 0x3b, 0x85, 0x5a, 0xe4, 0xf9, 0x47, 0x98, 0x26, 0xa2, 0x1c, 0xc3, 0x7d, 0x60, 0xde, 0x1, 0xbf, 0x14, 0xaa, 0x75, 0xcb, 0xd6, 0x68, 0xb7, 0x9, 0x8d, 0x33, 0xec, 0x52, 0x4f, 0xf1, 0x2e, 0x90, 0xd9, 0x67, 0xb8, 0x6, 0x1b, 0xa5, 0x7a, 0xc4, 0x40, 0xfe, 0x21, 0x9f, 0x82, 0x3c, 0xe3, 0x5d, 0xf6, 0x48, 0x97, 0x29, 0x34, 0x8a, 0x55, 0xeb, 0x6f, 0xd1, 0xe, 0xb0, 0xad, 0x13, 0xcc, 0x72, 0x87, 0x39, 0xe6, 0x58, 0x45, 0xfb, 0x24, 0x9a, 0x1e, 0xa0, 0x7f, 0xc1, 0xdc, 0x62, 0xbd, 0x3, 0xa8, 0x16, 0xc9, 0x77, 0x6a, 0xd4, 0xb, 0xb5, 0x31, 0x8f, 0x50, 0xee, 0xf3, 0x4d, 0x92, 0x2c},
- {0x0, 0xbf, 0x63, 0xdc, 0xc6, 0x79, 0xa5, 0x1a, 0x91, 0x2e, 0xf2, 0x4d, 0x57, 0xe8, 0x34, 0x8b, 0x3f, 0x80, 0x5c, 0xe3, 0xf9, 0x46, 0x9a, 0x25, 0xae, 0x11, 0xcd, 0x72, 0x68, 0xd7, 0xb, 0xb4, 0x7e, 0xc1, 0x1d, 0xa2, 0xb8, 0x7, 0xdb, 0x64, 0xef, 0x50, 0x8c, 0x33, 0x29, 0x96, 0x4a, 0xf5, 0x41, 0xfe, 0x22, 0x9d, 0x87, 0x38, 0xe4, 0x5b, 0xd0, 0x6f, 0xb3, 0xc, 0x16, 0xa9, 0x75, 0xca, 0xfc, 0x43, 0x9f, 0x20, 0x3a, 0x85, 0x59, 0xe6, 0x6d, 0xd2, 0xe, 0xb1, 0xab, 0x14, 0xc8, 0x77, 0xc3, 0x7c, 0xa0, 0x1f, 0x5, 0xba, 0x66, 0xd9, 0x52, 0xed, 0x31, 0x8e, 0x94, 0x2b, 0xf7, 0x48, 0x82, 0x3d, 0xe1, 0x5e, 0x44, 0xfb, 0x27, 0x98, 0x13, 0xac, 0x70, 0xcf, 0xd5, 0x6a, 0xb6, 0x9, 0xbd, 0x2, 0xde, 0x61, 0x7b, 0xc4, 0x18, 0xa7, 0x2c, 0x93, 0x4f, 0xf0, 0xea, 0x55, 0x89, 0x36, 0xe5, 0x5a, 0x86, 0x39, 0x23, 0x9c, 0x40, 0xff, 0x74, 0xcb, 0x17, 0xa8, 0xb2, 0xd, 0xd1, 0x6e, 0xda, 0x65, 0xb9, 0x6, 0x1c, 0xa3, 0x7f, 0xc0, 0x4b, 0xf4, 0x28, 0x97, 0x8d, 0x32, 0xee, 0x51, 0x9b, 0x24, 0xf8, 0x47, 0x5d, 0xe2, 0x3e, 0x81, 0xa, 0xb5, 0x69, 0xd6, 0xcc, 0x73, 0xaf, 0x10, 0xa4, 0x1b, 0xc7, 0x78, 0x62, 0xdd, 0x1, 0xbe, 0x35, 0x8a, 0x56, 0xe9, 0xf3, 0x4c, 0x90, 0x2f, 0x19, 0xa6, 0x7a, 0xc5, 0xdf, 0x60, 0xbc, 0x3, 0x88, 0x37, 0xeb, 0x54, 0x4e, 0xf1, 0x2d, 0x92, 0x26, 0x99, 0x45, 0xfa, 0xe0, 0x5f, 0x83, 0x3c, 0xb7, 0x8, 0xd4, 0x6b, 0x71, 0xce, 0x12, 0xad, 0x67, 0xd8, 0x4, 0xbb, 0xa1, 0x1e, 0xc2, 0x7d, 0xf6, 0x49, 0x95, 0x2a, 0x30, 0x8f, 0x53, 0xec, 0x58, 0xe7, 0x3b, 0x84, 0x9e, 0x21, 0xfd, 0x42, 0xc9, 0x76, 0xaa, 0x15, 0xf, 0xb0, 0x6c, 0xd3},
- {0x0, 0xc0, 0x9d, 0x5d, 0x27, 0xe7, 0xba, 0x7a, 0x4e, 0x8e, 0xd3, 0x13, 0x69, 0xa9, 0xf4, 0x34, 0x9c, 0x5c, 0x1, 0xc1, 0xbb, 0x7b, 0x26, 0xe6, 0xd2, 0x12, 0x4f, 0x8f, 0xf5, 0x35, 0x68, 0xa8, 0x25, 0xe5, 0xb8, 0x78, 0x2, 0xc2, 0x9f, 0x5f, 0x6b, 0xab, 0xf6, 0x36, 0x4c, 0x8c, 0xd1, 0x11, 0xb9, 0x79, 0x24, 0xe4, 0x9e, 0x5e, 0x3, 0xc3, 0xf7, 0x37, 0x6a, 0xaa, 0xd0, 0x10, 0x4d, 0x8d, 0x4a, 0x8a, 0xd7, 0x17, 0x6d, 0xad, 0xf0, 0x30, 0x4, 0xc4, 0x99, 0x59, 0x23, 0xe3, 0xbe, 0x7e, 0xd6, 0x16, 0x4b, 0x8b, 0xf1, 0x31, 0x6c, 0xac, 0x98, 0x58, 0x5, 0xc5, 0xbf, 0x7f, 0x22, 0xe2, 0x6f, 0xaf, 0xf2, 0x32, 0x48, 0x88, 0xd5, 0x15, 0x21, 0xe1, 0xbc, 0x7c, 0x6, 0xc6, 0x9b, 0x5b, 0xf3, 0x33, 0x6e, 0xae, 0xd4, 0x14, 0x49, 0x89, 0xbd, 0x7d, 0x20, 0xe0, 0x9a, 0x5a, 0x7, 0xc7, 0x94, 0x54, 0x9, 0xc9, 0xb3, 0x73, 0x2e, 0xee, 0xda, 0x1a, 0x47, 0x87, 0xfd, 0x3d, 0x60, 0xa0, 0x8, 0xc8, 0x95, 0x55, 0x2f, 0xef, 0xb2, 0x72, 0x46, 0x86, 0xdb, 0x1b, 0x61, 0xa1, 0xfc, 0x3c, 0xb1, 0x71, 0x2c, 0xec, 0x96, 0x56, 0xb, 0xcb, 0xff, 0x3f, 0x62, 0xa2, 0xd8, 0x18, 0x45, 0x85, 0x2d, 0xed, 0xb0, 0x70, 0xa, 0xca, 0x97, 0x57, 0x63, 0xa3, 0xfe, 0x3e, 0x44, 0x84, 0xd9, 0x19, 0xde, 0x1e, 0x43, 0x83, 0xf9, 0x39, 0x64, 0xa4, 0x90, 0x50, 0xd, 0xcd, 0xb7, 0x77, 0x2a, 0xea, 0x42, 0x82, 0xdf, 0x1f, 0x65, 0xa5, 0xf8, 0x38, 0xc, 0xcc, 0x91, 0x51, 0x2b, 0xeb, 0xb6, 0x76, 0xfb, 0x3b, 0x66, 0xa6, 0xdc, 0x1c, 0x41, 0x81, 0xb5, 0x75, 0x28, 0xe8, 0x92, 0x52, 0xf, 0xcf, 0x67, 0xa7, 0xfa, 0x3a, 0x40, 0x80, 0xdd, 0x1d, 0x29, 0xe9, 0xb4, 0x74, 0xe, 0xce, 0x93, 0x53},
- {0x0, 0xc1, 0x9f, 0x5e, 0x23, 0xe2, 0xbc, 0x7d, 0x46, 0x87, 0xd9, 0x18, 0x65, 0xa4, 0xfa, 0x3b, 0x8c, 0x4d, 0x13, 0xd2, 0xaf, 0x6e, 0x30, 0xf1, 0xca, 0xb, 0x55, 0x94, 0xe9, 0x28, 0x76, 0xb7, 0x5, 0xc4, 0x9a, 0x5b, 0x26, 0xe7, 0xb9, 0x78, 0x43, 0x82, 0xdc, 0x1d, 0x60, 0xa1, 0xff, 0x3e, 0x89, 0x48, 0x16, 0xd7, 0xaa, 0x6b, 0x35, 0xf4, 0xcf, 0xe, 0x50, 0x91, 0xec, 0x2d, 0x73, 0xb2, 0xa, 0xcb, 0x95, 0x54, 0x29, 0xe8, 0xb6, 0x77, 0x4c, 0x8d, 0xd3, 0x12, 0x6f, 0xae, 0xf0, 0x31, 0x86, 0x47, 0x19, 0xd8, 0xa5, 0x64, 0x3a, 0xfb, 0xc0, 0x1, 0x5f, 0x9e, 0xe3, 0x22, 0x7c, 0xbd, 0xf, 0xce, 0x90, 0x51, 0x2c, 0xed, 0xb3, 0x72, 0x49, 0x88, 0xd6, 0x17, 0x6a, 0xab, 0xf5, 0x34, 0x83, 0x42, 0x1c, 0xdd, 0xa0, 0x61, 0x3f, 0xfe, 0xc5, 0x4, 0x5a, 0x9b, 0xe6, 0x27, 0x79, 0xb8, 0x14, 0xd5, 0x8b, 0x4a, 0x37, 0xf6, 0xa8, 0x69, 0x52, 0x93, 0xcd, 0xc, 0x71, 0xb0, 0xee, 0x2f, 0x98, 0x59, 0x7, 0xc6, 0xbb, 0x7a, 0x24, 0xe5, 0xde, 0x1f, 0x41, 0x80, 0xfd, 0x3c, 0x62, 0xa3, 0x11, 0xd0, 0x8e, 0x4f, 0x32, 0xf3, 0xad, 0x6c, 0x57, 0x96, 0xc8, 0x9, 0x74, 0xb5, 0xeb, 0x2a, 0x9d, 0x5c, 0x2, 0xc3, 0xbe, 0x7f, 0x21, 0xe0, 0xdb, 0x1a, 0x44, 0x85, 0xf8, 0x39, 0x67, 0xa6, 0x1e, 0xdf, 0x81, 0x40, 0x3d, 0xfc, 0xa2, 0x63, 0x58, 0x99, 0xc7, 0x6, 0x7b, 0xba, 0xe4, 0x25, 0x92, 0x53, 0xd, 0xcc, 0xb1, 0x70, 0x2e, 0xef, 0xd4, 0x15, 0x4b, 0x8a, 0xf7, 0x36, 0x68, 0xa9, 0x1b, 0xda, 0x84, 0x45, 0x38, 0xf9, 0xa7, 0x66, 0x5d, 0x9c, 0xc2, 0x3, 0x7e, 0xbf, 0xe1, 0x20, 0x97, 0x56, 0x8, 0xc9, 0xb4, 0x75, 0x2b, 0xea, 0xd1, 0x10, 0x4e, 0x8f, 0xf2, 0x33, 0x6d, 0xac},
- {0x0, 0xc2, 0x99, 0x5b, 0x2f, 0xed, 0xb6, 0x74, 0x5e, 0x9c, 0xc7, 0x5, 0x71, 0xb3, 0xe8, 0x2a, 0xbc, 0x7e, 0x25, 0xe7, 0x93, 0x51, 0xa, 0xc8, 0xe2, 0x20, 0x7b, 0xb9, 0xcd, 0xf, 0x54, 0x96, 0x65, 0xa7, 0xfc, 0x3e, 0x4a, 0x88, 0xd3, 0x11, 0x3b, 0xf9, 0xa2, 0x60, 0x14, 0xd6, 0x8d, 0x4f, 0xd9, 0x1b, 0x40, 0x82, 0xf6, 0x34, 0x6f, 0xad, 0x87, 0x45, 0x1e, 0xdc, 0xa8, 0x6a, 0x31, 0xf3, 0xca, 0x8, 0x53, 0x91, 0xe5, 0x27, 0x7c, 0xbe, 0x94, 0x56, 0xd, 0xcf, 0xbb, 0x79, 0x22, 0xe0, 0x76, 0xb4, 0xef, 0x2d, 0x59, 0x9b, 0xc0, 0x2, 0x28, 0xea, 0xb1, 0x73, 0x7, 0xc5, 0x9e, 0x5c, 0xaf, 0x6d, 0x36, 0xf4, 0x80, 0x42, 0x19, 0xdb, 0xf1, 0x33, 0x68, 0xaa, 0xde, 0x1c, 0x47, 0x85, 0x13, 0xd1, 0x8a, 0x48, 0x3c, 0xfe, 0xa5, 0x67, 0x4d, 0x8f, 0xd4, 0x16, 0x62, 0xa0, 0xfb, 0x39, 0x89, 0x4b, 0x10, 0xd2, 0xa6, 0x64, 0x3f, 0xfd, 0xd7, 0x15, 0x4e, 0x8c, 0xf8, 0x3a, 0x61, 0xa3, 0x35, 0xf7, 0xac, 0x6e, 0x1a, 0xd8, 0x83, 0x41, 0x6b, 0xa9, 0xf2, 0x30, 0x44, 0x86, 0xdd, 0x1f, 0xec, 0x2e, 0x75, 0xb7, 0xc3, 0x1, 0x5a, 0x98, 0xb2, 0x70, 0x2b, 0xe9, 0x9d, 0x5f, 0x4, 0xc6, 0x50, 0x92, 0xc9, 0xb, 0x7f, 0xbd, 0xe6, 0x24, 0xe, 0xcc, 0x97, 0x55, 0x21, 0xe3, 0xb8, 0x7a, 0x43, 0x81, 0xda, 0x18, 0x6c, 0xae, 0xf5, 0x37, 0x1d, 0xdf, 0x84, 0x46, 0x32, 0xf0, 0xab, 0x69, 0xff, 0x3d, 0x66, 0xa4, 0xd0, 0x12, 0x49, 0x8b, 0xa1, 0x63, 0x38, 0xfa, 0x8e, 0x4c, 0x17, 0xd5, 0x26, 0xe4, 0xbf, 0x7d, 0x9, 0xcb, 0x90, 0x52, 0x78, 0xba, 0xe1, 0x23, 0x57, 0x95, 0xce, 0xc, 0x9a, 0x58, 0x3, 0xc1, 0xb5, 0x77, 0x2c, 0xee, 0xc4, 0x6, 0x5d, 0x9f, 0xeb, 0x29, 0x72, 0xb0},
- {0x0, 0xc3, 0x9b, 0x58, 0x2b, 0xe8, 0xb0, 0x73, 0x56, 0x95, 0xcd, 0xe, 0x7d, 0xbe, 0xe6, 0x25, 0xac, 0x6f, 0x37, 0xf4, 0x87, 0x44, 0x1c, 0xdf, 0xfa, 0x39, 0x61, 0xa2, 0xd1, 0x12, 0x4a, 0x89, 0x45, 0x86, 0xde, 0x1d, 0x6e, 0xad, 0xf5, 0x36, 0x13, 0xd0, 0x88, 0x4b, 0x38, 0xfb, 0xa3, 0x60, 0xe9, 0x2a, 0x72, 0xb1, 0xc2, 0x1, 0x59, 0x9a, 0xbf, 0x7c, 0x24, 0xe7, 0x94, 0x57, 0xf, 0xcc, 0x8a, 0x49, 0x11, 0xd2, 0xa1, 0x62, 0x3a, 0xf9, 0xdc, 0x1f, 0x47, 0x84, 0xf7, 0x34, 0x6c, 0xaf, 0x26, 0xe5, 0xbd, 0x7e, 0xd, 0xce, 0x96, 0x55, 0x70, 0xb3, 0xeb, 0x28, 0x5b, 0x98, 0xc0, 0x3, 0xcf, 0xc, 0x54, 0x97, 0xe4, 0x27, 0x7f, 0xbc, 0x99, 0x5a, 0x2, 0xc1, 0xb2, 0x71, 0x29, 0xea, 0x63, 0xa0, 0xf8, 0x3b, 0x48, 0x8b, 0xd3, 0x10, 0x35, 0xf6, 0xae, 0x6d, 0x1e, 0xdd, 0x85, 0x46, 0x9, 0xca, 0x92, 0x51, 0x22, 0xe1, 0xb9, 0x7a, 0x5f, 0x9c, 0xc4, 0x7, 0x74, 0xb7, 0xef, 0x2c, 0xa5, 0x66, 0x3e, 0xfd, 0x8e, 0x4d, 0x15, 0xd6, 0xf3, 0x30, 0x68, 0xab, 0xd8, 0x1b, 0x43, 0x80, 0x4c, 0x8f, 0xd7, 0x14, 0x67, 0xa4, 0xfc, 0x3f, 0x1a, 0xd9, 0x81, 0x42, 0x31, 0xf2, 0xaa, 0x69, 0xe0, 0x23, 0x7b, 0xb8, 0xcb, 0x8, 0x50, 0x93, 0xb6, 0x75, 0x2d, 0xee, 0x9d, 0x5e, 0x6, 0xc5, 0x83, 0x40, 0x18, 0xdb, 0xa8, 0x6b, 0x33, 0xf0, 0xd5, 0x16, 0x4e, 0x8d, 0xfe, 0x3d, 0x65, 0xa6, 0x2f, 0xec, 0xb4, 0x77, 0x4, 0xc7, 0x9f, 0x5c, 0x79, 0xba, 0xe2, 0x21, 0x52, 0x91, 0xc9, 0xa, 0xc6, 0x5, 0x5d, 0x9e, 0xed, 0x2e, 0x76, 0xb5, 0x90, 0x53, 0xb, 0xc8, 0xbb, 0x78, 0x20, 0xe3, 0x6a, 0xa9, 0xf1, 0x32, 0x41, 0x82, 0xda, 0x19, 0x3c, 0xff, 0xa7, 0x64, 0x17, 0xd4, 0x8c, 0x4f},
- {0x0, 0xc4, 0x95, 0x51, 0x37, 0xf3, 0xa2, 0x66, 0x6e, 0xaa, 0xfb, 0x3f, 0x59, 0x9d, 0xcc, 0x8, 0xdc, 0x18, 0x49, 0x8d, 0xeb, 0x2f, 0x7e, 0xba, 0xb2, 0x76, 0x27, 0xe3, 0x85, 0x41, 0x10, 0xd4, 0xa5, 0x61, 0x30, 0xf4, 0x92, 0x56, 0x7, 0xc3, 0xcb, 0xf, 0x5e, 0x9a, 0xfc, 0x38, 0x69, 0xad, 0x79, 0xbd, 0xec, 0x28, 0x4e, 0x8a, 0xdb, 0x1f, 0x17, 0xd3, 0x82, 0x46, 0x20, 0xe4, 0xb5, 0x71, 0x57, 0x93, 0xc2, 0x6, 0x60, 0xa4, 0xf5, 0x31, 0x39, 0xfd, 0xac, 0x68, 0xe, 0xca, 0x9b, 0x5f, 0x8b, 0x4f, 0x1e, 0xda, 0xbc, 0x78, 0x29, 0xed, 0xe5, 0x21, 0x70, 0xb4, 0xd2, 0x16, 0x47, 0x83, 0xf2, 0x36, 0x67, 0xa3, 0xc5, 0x1, 0x50, 0x94, 0x9c, 0x58, 0x9, 0xcd, 0xab, 0x6f, 0x3e, 0xfa, 0x2e, 0xea, 0xbb, 0x7f, 0x19, 0xdd, 0x8c, 0x48, 0x40, 0x84, 0xd5, 0x11, 0x77, 0xb3, 0xe2, 0x26, 0xae, 0x6a, 0x3b, 0xff, 0x99, 0x5d, 0xc, 0xc8, 0xc0, 0x4, 0x55, 0x91, 0xf7, 0x33, 0x62, 0xa6, 0x72, 0xb6, 0xe7, 0x23, 0x45, 0x81, 0xd0, 0x14, 0x1c, 0xd8, 0x89, 0x4d, 0x2b, 0xef, 0xbe, 0x7a, 0xb, 0xcf, 0x9e, 0x5a, 0x3c, 0xf8, 0xa9, 0x6d, 0x65, 0xa1, 0xf0, 0x34, 0x52, 0x96, 0xc7, 0x3, 0xd7, 0x13, 0x42, 0x86, 0xe0, 0x24, 0x75, 0xb1, 0xb9, 0x7d, 0x2c, 0xe8, 0x8e, 0x4a, 0x1b, 0xdf, 0xf9, 0x3d, 0x6c, 0xa8, 0xce, 0xa, 0x5b, 0x9f, 0x97, 0x53, 0x2, 0xc6, 0xa0, 0x64, 0x35, 0xf1, 0x25, 0xe1, 0xb0, 0x74, 0x12, 0xd6, 0x87, 0x43, 0x4b, 0x8f, 0xde, 0x1a, 0x7c, 0xb8, 0xe9, 0x2d, 0x5c, 0x98, 0xc9, 0xd, 0x6b, 0xaf, 0xfe, 0x3a, 0x32, 0xf6, 0xa7, 0x63, 0x5, 0xc1, 0x90, 0x54, 0x80, 0x44, 0x15, 0xd1, 0xb7, 0x73, 0x22, 0xe6, 0xee, 0x2a, 0x7b, 0xbf, 0xd9, 0x1d, 0x4c, 0x88},
- {0x0, 0xc5, 0x97, 0x52, 0x33, 0xf6, 0xa4, 0x61, 0x66, 0xa3, 0xf1, 0x34, 0x55, 0x90, 0xc2, 0x7, 0xcc, 0x9, 0x5b, 0x9e, 0xff, 0x3a, 0x68, 0xad, 0xaa, 0x6f, 0x3d, 0xf8, 0x99, 0x5c, 0xe, 0xcb, 0x85, 0x40, 0x12, 0xd7, 0xb6, 0x73, 0x21, 0xe4, 0xe3, 0x26, 0x74, 0xb1, 0xd0, 0x15, 0x47, 0x82, 0x49, 0x8c, 0xde, 0x1b, 0x7a, 0xbf, 0xed, 0x28, 0x2f, 0xea, 0xb8, 0x7d, 0x1c, 0xd9, 0x8b, 0x4e, 0x17, 0xd2, 0x80, 0x45, 0x24, 0xe1, 0xb3, 0x76, 0x71, 0xb4, 0xe6, 0x23, 0x42, 0x87, 0xd5, 0x10, 0xdb, 0x1e, 0x4c, 0x89, 0xe8, 0x2d, 0x7f, 0xba, 0xbd, 0x78, 0x2a, 0xef, 0x8e, 0x4b, 0x19, 0xdc, 0x92, 0x57, 0x5, 0xc0, 0xa1, 0x64, 0x36, 0xf3, 0xf4, 0x31, 0x63, 0xa6, 0xc7, 0x2, 0x50, 0x95, 0x5e, 0x9b, 0xc9, 0xc, 0x6d, 0xa8, 0xfa, 0x3f, 0x38, 0xfd, 0xaf, 0x6a, 0xb, 0xce, 0x9c, 0x59, 0x2e, 0xeb, 0xb9, 0x7c, 0x1d, 0xd8, 0x8a, 0x4f, 0x48, 0x8d, 0xdf, 0x1a, 0x7b, 0xbe, 0xec, 0x29, 0xe2, 0x27, 0x75, 0xb0, 0xd1, 0x14, 0x46, 0x83, 0x84, 0x41, 0x13, 0xd6, 0xb7, 0x72, 0x20, 0xe5, 0xab, 0x6e, 0x3c, 0xf9, 0x98, 0x5d, 0xf, 0xca, 0xcd, 0x8, 0x5a, 0x9f, 0xfe, 0x3b, 0x69, 0xac, 0x67, 0xa2, 0xf0, 0x35, 0x54, 0x91, 0xc3, 0x6, 0x1, 0xc4, 0x96, 0x53, 0x32, 0xf7, 0xa5, 0x60, 0x39, 0xfc, 0xae, 0x6b, 0xa, 0xcf, 0x9d, 0x58, 0x5f, 0x9a, 0xc8, 0xd, 0x6c, 0xa9, 0xfb, 0x3e, 0xf5, 0x30, 0x62, 0xa7, 0xc6, 0x3, 0x51, 0x94, 0x93, 0x56, 0x4, 0xc1, 0xa0, 0x65, 0x37, 0xf2, 0xbc, 0x79, 0x2b, 0xee, 0x8f, 0x4a, 0x18, 0xdd, 0xda, 0x1f, 0x4d, 0x88, 0xe9, 0x2c, 0x7e, 0xbb, 0x70, 0xb5, 0xe7, 0x22, 0x43, 0x86, 0xd4, 0x11, 0x16, 0xd3, 0x81, 0x44, 0x25, 0xe0, 0xb2, 0x77},
- {0x0, 0xc6, 0x91, 0x57, 0x3f, 0xf9, 0xae, 0x68, 0x7e, 0xb8, 0xef, 0x29, 0x41, 0x87, 0xd0, 0x16, 0xfc, 0x3a, 0x6d, 0xab, 0xc3, 0x5, 0x52, 0x94, 0x82, 0x44, 0x13, 0xd5, 0xbd, 0x7b, 0x2c, 0xea, 0xe5, 0x23, 0x74, 0xb2, 0xda, 0x1c, 0x4b, 0x8d, 0x9b, 0x5d, 0xa, 0xcc, 0xa4, 0x62, 0x35, 0xf3, 0x19, 0xdf, 0x88, 0x4e, 0x26, 0xe0, 0xb7, 0x71, 0x67, 0xa1, 0xf6, 0x30, 0x58, 0x9e, 0xc9, 0xf, 0xd7, 0x11, 0x46, 0x80, 0xe8, 0x2e, 0x79, 0xbf, 0xa9, 0x6f, 0x38, 0xfe, 0x96, 0x50, 0x7, 0xc1, 0x2b, 0xed, 0xba, 0x7c, 0x14, 0xd2, 0x85, 0x43, 0x55, 0x93, 0xc4, 0x2, 0x6a, 0xac, 0xfb, 0x3d, 0x32, 0xf4, 0xa3, 0x65, 0xd, 0xcb, 0x9c, 0x5a, 0x4c, 0x8a, 0xdd, 0x1b, 0x73, 0xb5, 0xe2, 0x24, 0xce, 0x8, 0x5f, 0x99, 0xf1, 0x37, 0x60, 0xa6, 0xb0, 0x76, 0x21, 0xe7, 0x8f, 0x49, 0x1e, 0xd8, 0xb3, 0x75, 0x22, 0xe4, 0x8c, 0x4a, 0x1d, 0xdb, 0xcd, 0xb, 0x5c, 0x9a, 0xf2, 0x34, 0x63, 0xa5, 0x4f, 0x89, 0xde, 0x18, 0x70, 0xb6, 0xe1, 0x27, 0x31, 0xf7, 0xa0, 0x66, 0xe, 0xc8, 0x9f, 0x59, 0x56, 0x90, 0xc7, 0x1, 0x69, 0xaf, 0xf8, 0x3e, 0x28, 0xee, 0xb9, 0x7f, 0x17, 0xd1, 0x86, 0x40, 0xaa, 0x6c, 0x3b, 0xfd, 0x95, 0x53, 0x4, 0xc2, 0xd4, 0x12, 0x45, 0x83, 0xeb, 0x2d, 0x7a, 0xbc, 0x64, 0xa2, 0xf5, 0x33, 0x5b, 0x9d, 0xca, 0xc, 0x1a, 0xdc, 0x8b, 0x4d, 0x25, 0xe3, 0xb4, 0x72, 0x98, 0x5e, 0x9, 0xcf, 0xa7, 0x61, 0x36, 0xf0, 0xe6, 0x20, 0x77, 0xb1, 0xd9, 0x1f, 0x48, 0x8e, 0x81, 0x47, 0x10, 0xd6, 0xbe, 0x78, 0x2f, 0xe9, 0xff, 0x39, 0x6e, 0xa8, 0xc0, 0x6, 0x51, 0x97, 0x7d, 0xbb, 0xec, 0x2a, 0x42, 0x84, 0xd3, 0x15, 0x3, 0xc5, 0x92, 0x54, 0x3c, 0xfa, 0xad, 0x6b},
- {0x0, 0xc7, 0x93, 0x54, 0x3b, 0xfc, 0xa8, 0x6f, 0x76, 0xb1, 0xe5, 0x22, 0x4d, 0x8a, 0xde, 0x19, 0xec, 0x2b, 0x7f, 0xb8, 0xd7, 0x10, 0x44, 0x83, 0x9a, 0x5d, 0x9, 0xce, 0xa1, 0x66, 0x32, 0xf5, 0xc5, 0x2, 0x56, 0x91, 0xfe, 0x39, 0x6d, 0xaa, 0xb3, 0x74, 0x20, 0xe7, 0x88, 0x4f, 0x1b, 0xdc, 0x29, 0xee, 0xba, 0x7d, 0x12, 0xd5, 0x81, 0x46, 0x5f, 0x98, 0xcc, 0xb, 0x64, 0xa3, 0xf7, 0x30, 0x97, 0x50, 0x4, 0xc3, 0xac, 0x6b, 0x3f, 0xf8, 0xe1, 0x26, 0x72, 0xb5, 0xda, 0x1d, 0x49, 0x8e, 0x7b, 0xbc, 0xe8, 0x2f, 0x40, 0x87, 0xd3, 0x14, 0xd, 0xca, 0x9e, 0x59, 0x36, 0xf1, 0xa5, 0x62, 0x52, 0x95, 0xc1, 0x6, 0x69, 0xae, 0xfa, 0x3d, 0x24, 0xe3, 0xb7, 0x70, 0x1f, 0xd8, 0x8c, 0x4b, 0xbe, 0x79, 0x2d, 0xea, 0x85, 0x42, 0x16, 0xd1, 0xc8, 0xf, 0x5b, 0x9c, 0xf3, 0x34, 0x60, 0xa7, 0x33, 0xf4, 0xa0, 0x67, 0x8, 0xcf, 0x9b, 0x5c, 0x45, 0x82, 0xd6, 0x11, 0x7e, 0xb9, 0xed, 0x2a, 0xdf, 0x18, 0x4c, 0x8b, 0xe4, 0x23, 0x77, 0xb0, 0xa9, 0x6e, 0x3a, 0xfd, 0x92, 0x55, 0x1, 0xc6, 0xf6, 0x31, 0x65, 0xa2, 0xcd, 0xa, 0x5e, 0x99, 0x80, 0x47, 0x13, 0xd4, 0xbb, 0x7c, 0x28, 0xef, 0x1a, 0xdd, 0x89, 0x4e, 0x21, 0xe6, 0xb2, 0x75, 0x6c, 0xab, 0xff, 0x38, 0x57, 0x90, 0xc4, 0x3, 0xa4, 0x63, 0x37, 0xf0, 0x9f, 0x58, 0xc, 0xcb, 0xd2, 0x15, 0x41, 0x86, 0xe9, 0x2e, 0x7a, 0xbd, 0x48, 0x8f, 0xdb, 0x1c, 0x73, 0xb4, 0xe0, 0x27, 0x3e, 0xf9, 0xad, 0x6a, 0x5, 0xc2, 0x96, 0x51, 0x61, 0xa6, 0xf2, 0x35, 0x5a, 0x9d, 0xc9, 0xe, 0x17, 0xd0, 0x84, 0x43, 0x2c, 0xeb, 0xbf, 0x78, 0x8d, 0x4a, 0x1e, 0xd9, 0xb6, 0x71, 0x25, 0xe2, 0xfb, 0x3c, 0x68, 0xaf, 0xc0, 0x7, 0x53, 0x94},
- {0x0, 0xc8, 0x8d, 0x45, 0x7, 0xcf, 0x8a, 0x42, 0xe, 0xc6, 0x83, 0x4b, 0x9, 0xc1, 0x84, 0x4c, 0x1c, 0xd4, 0x91, 0x59, 0x1b, 0xd3, 0x96, 0x5e, 0x12, 0xda, 0x9f, 0x57, 0x15, 0xdd, 0x98, 0x50, 0x38, 0xf0, 0xb5, 0x7d, 0x3f, 0xf7, 0xb2, 0x7a, 0x36, 0xfe, 0xbb, 0x73, 0x31, 0xf9, 0xbc, 0x74, 0x24, 0xec, 0xa9, 0x61, 0x23, 0xeb, 0xae, 0x66, 0x2a, 0xe2, 0xa7, 0x6f, 0x2d, 0xe5, 0xa0, 0x68, 0x70, 0xb8, 0xfd, 0x35, 0x77, 0xbf, 0xfa, 0x32, 0x7e, 0xb6, 0xf3, 0x3b, 0x79, 0xb1, 0xf4, 0x3c, 0x6c, 0xa4, 0xe1, 0x29, 0x6b, 0xa3, 0xe6, 0x2e, 0x62, 0xaa, 0xef, 0x27, 0x65, 0xad, 0xe8, 0x20, 0x48, 0x80, 0xc5, 0xd, 0x4f, 0x87, 0xc2, 0xa, 0x46, 0x8e, 0xcb, 0x3, 0x41, 0x89, 0xcc, 0x4, 0x54, 0x9c, 0xd9, 0x11, 0x53, 0x9b, 0xde, 0x16, 0x5a, 0x92, 0xd7, 0x1f, 0x5d, 0x95, 0xd0, 0x18, 0xe0, 0x28, 0x6d, 0xa5, 0xe7, 0x2f, 0x6a, 0xa2, 0xee, 0x26, 0x63, 0xab, 0xe9, 0x21, 0x64, 0xac, 0xfc, 0x34, 0x71, 0xb9, 0xfb, 0x33, 0x76, 0xbe, 0xf2, 0x3a, 0x7f, 0xb7, 0xf5, 0x3d, 0x78, 0xb0, 0xd8, 0x10, 0x55, 0x9d, 0xdf, 0x17, 0x52, 0x9a, 0xd6, 0x1e, 0x5b, 0x93, 0xd1, 0x19, 0x5c, 0x94, 0xc4, 0xc, 0x49, 0x81, 0xc3, 0xb, 0x4e, 0x86, 0xca, 0x2, 0x47, 0x8f, 0xcd, 0x5, 0x40, 0x88, 0x90, 0x58, 0x1d, 0xd5, 0x97, 0x5f, 0x1a, 0xd2, 0x9e, 0x56, 0x13, 0xdb, 0x99, 0x51, 0x14, 0xdc, 0x8c, 0x44, 0x1, 0xc9, 0x8b, 0x43, 0x6, 0xce, 0x82, 0x4a, 0xf, 0xc7, 0x85, 0x4d, 0x8, 0xc0, 0xa8, 0x60, 0x25, 0xed, 0xaf, 0x67, 0x22, 0xea, 0xa6, 0x6e, 0x2b, 0xe3, 0xa1, 0x69, 0x2c, 0xe4, 0xb4, 0x7c, 0x39, 0xf1, 0xb3, 0x7b, 0x3e, 0xf6, 0xba, 0x72, 0x37, 0xff, 0xbd, 0x75, 0x30, 0xf8},
- {0x0, 0xc9, 0x8f, 0x46, 0x3, 0xca, 0x8c, 0x45, 0x6, 0xcf, 0x89, 0x40, 0x5, 0xcc, 0x8a, 0x43, 0xc, 0xc5, 0x83, 0x4a, 0xf, 0xc6, 0x80, 0x49, 0xa, 0xc3, 0x85, 0x4c, 0x9, 0xc0, 0x86, 0x4f, 0x18, 0xd1, 0x97, 0x5e, 0x1b, 0xd2, 0x94, 0x5d, 0x1e, 0xd7, 0x91, 0x58, 0x1d, 0xd4, 0x92, 0x5b, 0x14, 0xdd, 0x9b, 0x52, 0x17, 0xde, 0x98, 0x51, 0x12, 0xdb, 0x9d, 0x54, 0x11, 0xd8, 0x9e, 0x57, 0x30, 0xf9, 0xbf, 0x76, 0x33, 0xfa, 0xbc, 0x75, 0x36, 0xff, 0xb9, 0x70, 0x35, 0xfc, 0xba, 0x73, 0x3c, 0xf5, 0xb3, 0x7a, 0x3f, 0xf6, 0xb0, 0x79, 0x3a, 0xf3, 0xb5, 0x7c, 0x39, 0xf0, 0xb6, 0x7f, 0x28, 0xe1, 0xa7, 0x6e, 0x2b, 0xe2, 0xa4, 0x6d, 0x2e, 0xe7, 0xa1, 0x68, 0x2d, 0xe4, 0xa2, 0x6b, 0x24, 0xed, 0xab, 0x62, 0x27, 0xee, 0xa8, 0x61, 0x22, 0xeb, 0xad, 0x64, 0x21, 0xe8, 0xae, 0x67, 0x60, 0xa9, 0xef, 0x26, 0x63, 0xaa, 0xec, 0x25, 0x66, 0xaf, 0xe9, 0x20, 0x65, 0xac, 0xea, 0x23, 0x6c, 0xa5, 0xe3, 0x2a, 0x6f, 0xa6, 0xe0, 0x29, 0x6a, 0xa3, 0xe5, 0x2c, 0x69, 0xa0, 0xe6, 0x2f, 0x78, 0xb1, 0xf7, 0x3e, 0x7b, 0xb2, 0xf4, 0x3d, 0x7e, 0xb7, 0xf1, 0x38, 0x7d, 0xb4, 0xf2, 0x3b, 0x74, 0xbd, 0xfb, 0x32, 0x77, 0xbe, 0xf8, 0x31, 0x72, 0xbb, 0xfd, 0x34, 0x71, 0xb8, 0xfe, 0x37, 0x50, 0x99, 0xdf, 0x16, 0x53, 0x9a, 0xdc, 0x15, 0x56, 0x9f, 0xd9, 0x10, 0x55, 0x9c, 0xda, 0x13, 0x5c, 0x95, 0xd3, 0x1a, 0x5f, 0x96, 0xd0, 0x19, 0x5a, 0x93, 0xd5, 0x1c, 0x59, 0x90, 0xd6, 0x1f, 0x48, 0x81, 0xc7, 0xe, 0x4b, 0x82, 0xc4, 0xd, 0x4e, 0x87, 0xc1, 0x8, 0x4d, 0x84, 0xc2, 0xb, 0x44, 0x8d, 0xcb, 0x2, 0x47, 0x8e, 0xc8, 0x1, 0x42, 0x8b, 0xcd, 0x4, 0x41, 0x88, 0xce, 0x7},
- {0x0, 0xca, 0x89, 0x43, 0xf, 0xc5, 0x86, 0x4c, 0x1e, 0xd4, 0x97, 0x5d, 0x11, 0xdb, 0x98, 0x52, 0x3c, 0xf6, 0xb5, 0x7f, 0x33, 0xf9, 0xba, 0x70, 0x22, 0xe8, 0xab, 0x61, 0x2d, 0xe7, 0xa4, 0x6e, 0x78, 0xb2, 0xf1, 0x3b, 0x77, 0xbd, 0xfe, 0x34, 0x66, 0xac, 0xef, 0x25, 0x69, 0xa3, 0xe0, 0x2a, 0x44, 0x8e, 0xcd, 0x7, 0x4b, 0x81, 0xc2, 0x8, 0x5a, 0x90, 0xd3, 0x19, 0x55, 0x9f, 0xdc, 0x16, 0xf0, 0x3a, 0x79, 0xb3, 0xff, 0x35, 0x76, 0xbc, 0xee, 0x24, 0x67, 0xad, 0xe1, 0x2b, 0x68, 0xa2, 0xcc, 0x6, 0x45, 0x8f, 0xc3, 0x9, 0x4a, 0x80, 0xd2, 0x18, 0x5b, 0x91, 0xdd, 0x17, 0x54, 0x9e, 0x88, 0x42, 0x1, 0xcb, 0x87, 0x4d, 0xe, 0xc4, 0x96, 0x5c, 0x1f, 0xd5, 0x99, 0x53, 0x10, 0xda, 0xb4, 0x7e, 0x3d, 0xf7, 0xbb, 0x71, 0x32, 0xf8, 0xaa, 0x60, 0x23, 0xe9, 0xa5, 0x6f, 0x2c, 0xe6, 0xfd, 0x37, 0x74, 0xbe, 0xf2, 0x38, 0x7b, 0xb1, 0xe3, 0x29, 0x6a, 0xa0, 0xec, 0x26, 0x65, 0xaf, 0xc1, 0xb, 0x48, 0x82, 0xce, 0x4, 0x47, 0x8d, 0xdf, 0x15, 0x56, 0x9c, 0xd0, 0x1a, 0x59, 0x93, 0x85, 0x4f, 0xc, 0xc6, 0x8a, 0x40, 0x3, 0xc9, 0x9b, 0x51, 0x12, 0xd8, 0x94, 0x5e, 0x1d, 0xd7, 0xb9, 0x73, 0x30, 0xfa, 0xb6, 0x7c, 0x3f, 0xf5, 0xa7, 0x6d, 0x2e, 0xe4, 0xa8, 0x62, 0x21, 0xeb, 0xd, 0xc7, 0x84, 0x4e, 0x2, 0xc8, 0x8b, 0x41, 0x13, 0xd9, 0x9a, 0x50, 0x1c, 0xd6, 0x95, 0x5f, 0x31, 0xfb, 0xb8, 0x72, 0x3e, 0xf4, 0xb7, 0x7d, 0x2f, 0xe5, 0xa6, 0x6c, 0x20, 0xea, 0xa9, 0x63, 0x75, 0xbf, 0xfc, 0x36, 0x7a, 0xb0, 0xf3, 0x39, 0x6b, 0xa1, 0xe2, 0x28, 0x64, 0xae, 0xed, 0x27, 0x49, 0x83, 0xc0, 0xa, 0x46, 0x8c, 0xcf, 0x5, 0x57, 0x9d, 0xde, 0x14, 0x58, 0x92, 0xd1, 0x1b},
- {0x0, 0xcb, 0x8b, 0x40, 0xb, 0xc0, 0x80, 0x4b, 0x16, 0xdd, 0x9d, 0x56, 0x1d, 0xd6, 0x96, 0x5d, 0x2c, 0xe7, 0xa7, 0x6c, 0x27, 0xec, 0xac, 0x67, 0x3a, 0xf1, 0xb1, 0x7a, 0x31, 0xfa, 0xba, 0x71, 0x58, 0x93, 0xd3, 0x18, 0x53, 0x98, 0xd8, 0x13, 0x4e, 0x85, 0xc5, 0xe, 0x45, 0x8e, 0xce, 0x5, 0x74, 0xbf, 0xff, 0x34, 0x7f, 0xb4, 0xf4, 0x3f, 0x62, 0xa9, 0xe9, 0x22, 0x69, 0xa2, 0xe2, 0x29, 0xb0, 0x7b, 0x3b, 0xf0, 0xbb, 0x70, 0x30, 0xfb, 0xa6, 0x6d, 0x2d, 0xe6, 0xad, 0x66, 0x26, 0xed, 0x9c, 0x57, 0x17, 0xdc, 0x97, 0x5c, 0x1c, 0xd7, 0x8a, 0x41, 0x1, 0xca, 0x81, 0x4a, 0xa, 0xc1, 0xe8, 0x23, 0x63, 0xa8, 0xe3, 0x28, 0x68, 0xa3, 0xfe, 0x35, 0x75, 0xbe, 0xf5, 0x3e, 0x7e, 0xb5, 0xc4, 0xf, 0x4f, 0x84, 0xcf, 0x4, 0x44, 0x8f, 0xd2, 0x19, 0x59, 0x92, 0xd9, 0x12, 0x52, 0x99, 0x7d, 0xb6, 0xf6, 0x3d, 0x76, 0xbd, 0xfd, 0x36, 0x6b, 0xa0, 0xe0, 0x2b, 0x60, 0xab, 0xeb, 0x20, 0x51, 0x9a, 0xda, 0x11, 0x5a, 0x91, 0xd1, 0x1a, 0x47, 0x8c, 0xcc, 0x7, 0x4c, 0x87, 0xc7, 0xc, 0x25, 0xee, 0xae, 0x65, 0x2e, 0xe5, 0xa5, 0x6e, 0x33, 0xf8, 0xb8, 0x73, 0x38, 0xf3, 0xb3, 0x78, 0x9, 0xc2, 0x82, 0x49, 0x2, 0xc9, 0x89, 0x42, 0x1f, 0xd4, 0x94, 0x5f, 0x14, 0xdf, 0x9f, 0x54, 0xcd, 0x6, 0x46, 0x8d, 0xc6, 0xd, 0x4d, 0x86, 0xdb, 0x10, 0x50, 0x9b, 0xd0, 0x1b, 0x5b, 0x90, 0xe1, 0x2a, 0x6a, 0xa1, 0xea, 0x21, 0x61, 0xaa, 0xf7, 0x3c, 0x7c, 0xb7, 0xfc, 0x37, 0x77, 0xbc, 0x95, 0x5e, 0x1e, 0xd5, 0x9e, 0x55, 0x15, 0xde, 0x83, 0x48, 0x8, 0xc3, 0x88, 0x43, 0x3, 0xc8, 0xb9, 0x72, 0x32, 0xf9, 0xb2, 0x79, 0x39, 0xf2, 0xaf, 0x64, 0x24, 0xef, 0xa4, 0x6f, 0x2f, 0xe4},
- {0x0, 0xcc, 0x85, 0x49, 0x17, 0xdb, 0x92, 0x5e, 0x2e, 0xe2, 0xab, 0x67, 0x39, 0xf5, 0xbc, 0x70, 0x5c, 0x90, 0xd9, 0x15, 0x4b, 0x87, 0xce, 0x2, 0x72, 0xbe, 0xf7, 0x3b, 0x65, 0xa9, 0xe0, 0x2c, 0xb8, 0x74, 0x3d, 0xf1, 0xaf, 0x63, 0x2a, 0xe6, 0x96, 0x5a, 0x13, 0xdf, 0x81, 0x4d, 0x4, 0xc8, 0xe4, 0x28, 0x61, 0xad, 0xf3, 0x3f, 0x76, 0xba, 0xca, 0x6, 0x4f, 0x83, 0xdd, 0x11, 0x58, 0x94, 0x6d, 0xa1, 0xe8, 0x24, 0x7a, 0xb6, 0xff, 0x33, 0x43, 0x8f, 0xc6, 0xa, 0x54, 0x98, 0xd1, 0x1d, 0x31, 0xfd, 0xb4, 0x78, 0x26, 0xea, 0xa3, 0x6f, 0x1f, 0xd3, 0x9a, 0x56, 0x8, 0xc4, 0x8d, 0x41, 0xd5, 0x19, 0x50, 0x9c, 0xc2, 0xe, 0x47, 0x8b, 0xfb, 0x37, 0x7e, 0xb2, 0xec, 0x20, 0x69, 0xa5, 0x89, 0x45, 0xc, 0xc0, 0x9e, 0x52, 0x1b, 0xd7, 0xa7, 0x6b, 0x22, 0xee, 0xb0, 0x7c, 0x35, 0xf9, 0xda, 0x16, 0x5f, 0x93, 0xcd, 0x1, 0x48, 0x84, 0xf4, 0x38, 0x71, 0xbd, 0xe3, 0x2f, 0x66, 0xaa, 0x86, 0x4a, 0x3, 0xcf, 0x91, 0x5d, 0x14, 0xd8, 0xa8, 0x64, 0x2d, 0xe1, 0xbf, 0x73, 0x3a, 0xf6, 0x62, 0xae, 0xe7, 0x2b, 0x75, 0xb9, 0xf0, 0x3c, 0x4c, 0x80, 0xc9, 0x5, 0x5b, 0x97, 0xde, 0x12, 0x3e, 0xf2, 0xbb, 0x77, 0x29, 0xe5, 0xac, 0x60, 0x10, 0xdc, 0x95, 0x59, 0x7, 0xcb, 0x82, 0x4e, 0xb7, 0x7b, 0x32, 0xfe, 0xa0, 0x6c, 0x25, 0xe9, 0x99, 0x55, 0x1c, 0xd0, 0x8e, 0x42, 0xb, 0xc7, 0xeb, 0x27, 0x6e, 0xa2, 0xfc, 0x30, 0x79, 0xb5, 0xc5, 0x9, 0x40, 0x8c, 0xd2, 0x1e, 0x57, 0x9b, 0xf, 0xc3, 0x8a, 0x46, 0x18, 0xd4, 0x9d, 0x51, 0x21, 0xed, 0xa4, 0x68, 0x36, 0xfa, 0xb3, 0x7f, 0x53, 0x9f, 0xd6, 0x1a, 0x44, 0x88, 0xc1, 0xd, 0x7d, 0xb1, 0xf8, 0x34, 0x6a, 0xa6, 0xef, 0x23},
- {0x0, 0xcd, 0x87, 0x4a, 0x13, 0xde, 0x94, 0x59, 0x26, 0xeb, 0xa1, 0x6c, 0x35, 0xf8, 0xb2, 0x7f, 0x4c, 0x81, 0xcb, 0x6, 0x5f, 0x92, 0xd8, 0x15, 0x6a, 0xa7, 0xed, 0x20, 0x79, 0xb4, 0xfe, 0x33, 0x98, 0x55, 0x1f, 0xd2, 0x8b, 0x46, 0xc, 0xc1, 0xbe, 0x73, 0x39, 0xf4, 0xad, 0x60, 0x2a, 0xe7, 0xd4, 0x19, 0x53, 0x9e, 0xc7, 0xa, 0x40, 0x8d, 0xf2, 0x3f, 0x75, 0xb8, 0xe1, 0x2c, 0x66, 0xab, 0x2d, 0xe0, 0xaa, 0x67, 0x3e, 0xf3, 0xb9, 0x74, 0xb, 0xc6, 0x8c, 0x41, 0x18, 0xd5, 0x9f, 0x52, 0x61, 0xac, 0xe6, 0x2b, 0x72, 0xbf, 0xf5, 0x38, 0x47, 0x8a, 0xc0, 0xd, 0x54, 0x99, 0xd3, 0x1e, 0xb5, 0x78, 0x32, 0xff, 0xa6, 0x6b, 0x21, 0xec, 0x93, 0x5e, 0x14, 0xd9, 0x80, 0x4d, 0x7, 0xca, 0xf9, 0x34, 0x7e, 0xb3, 0xea, 0x27, 0x6d, 0xa0, 0xdf, 0x12, 0x58, 0x95, 0xcc, 0x1, 0x4b, 0x86, 0x5a, 0x97, 0xdd, 0x10, 0x49, 0x84, 0xce, 0x3, 0x7c, 0xb1, 0xfb, 0x36, 0x6f, 0xa2, 0xe8, 0x25, 0x16, 0xdb, 0x91, 0x5c, 0x5, 0xc8, 0x82, 0x4f, 0x30, 0xfd, 0xb7, 0x7a, 0x23, 0xee, 0xa4, 0x69, 0xc2, 0xf, 0x45, 0x88, 0xd1, 0x1c, 0x56, 0x9b, 0xe4, 0x29, 0x63, 0xae, 0xf7, 0x3a, 0x70, 0xbd, 0x8e, 0x43, 0x9, 0xc4, 0x9d, 0x50, 0x1a, 0xd7, 0xa8, 0x65, 0x2f, 0xe2, 0xbb, 0x76, 0x3c, 0xf1, 0x77, 0xba, 0xf0, 0x3d, 0x64, 0xa9, 0xe3, 0x2e, 0x51, 0x9c, 0xd6, 0x1b, 0x42, 0x8f, 0xc5, 0x8, 0x3b, 0xf6, 0xbc, 0x71, 0x28, 0xe5, 0xaf, 0x62, 0x1d, 0xd0, 0x9a, 0x57, 0xe, 0xc3, 0x89, 0x44, 0xef, 0x22, 0x68, 0xa5, 0xfc, 0x31, 0x7b, 0xb6, 0xc9, 0x4, 0x4e, 0x83, 0xda, 0x17, 0x5d, 0x90, 0xa3, 0x6e, 0x24, 0xe9, 0xb0, 0x7d, 0x37, 0xfa, 0x85, 0x48, 0x2, 0xcf, 0x96, 0x5b, 0x11, 0xdc},
- {0x0, 0xce, 0x81, 0x4f, 0x1f, 0xd1, 0x9e, 0x50, 0x3e, 0xf0, 0xbf, 0x71, 0x21, 0xef, 0xa0, 0x6e, 0x7c, 0xb2, 0xfd, 0x33, 0x63, 0xad, 0xe2, 0x2c, 0x42, 0x8c, 0xc3, 0xd, 0x5d, 0x93, 0xdc, 0x12, 0xf8, 0x36, 0x79, 0xb7, 0xe7, 0x29, 0x66, 0xa8, 0xc6, 0x8, 0x47, 0x89, 0xd9, 0x17, 0x58, 0x96, 0x84, 0x4a, 0x5, 0xcb, 0x9b, 0x55, 0x1a, 0xd4, 0xba, 0x74, 0x3b, 0xf5, 0xa5, 0x6b, 0x24, 0xea, 0xed, 0x23, 0x6c, 0xa2, 0xf2, 0x3c, 0x73, 0xbd, 0xd3, 0x1d, 0x52, 0x9c, 0xcc, 0x2, 0x4d, 0x83, 0x91, 0x5f, 0x10, 0xde, 0x8e, 0x40, 0xf, 0xc1, 0xaf, 0x61, 0x2e, 0xe0, 0xb0, 0x7e, 0x31, 0xff, 0x15, 0xdb, 0x94, 0x5a, 0xa, 0xc4, 0x8b, 0x45, 0x2b, 0xe5, 0xaa, 0x64, 0x34, 0xfa, 0xb5, 0x7b, 0x69, 0xa7, 0xe8, 0x26, 0x76, 0xb8, 0xf7, 0x39, 0x57, 0x99, 0xd6, 0x18, 0x48, 0x86, 0xc9, 0x7, 0xc7, 0x9, 0x46, 0x88, 0xd8, 0x16, 0x59, 0x97, 0xf9, 0x37, 0x78, 0xb6, 0xe6, 0x28, 0x67, 0xa9, 0xbb, 0x75, 0x3a, 0xf4, 0xa4, 0x6a, 0x25, 0xeb, 0x85, 0x4b, 0x4, 0xca, 0x9a, 0x54, 0x1b, 0xd5, 0x3f, 0xf1, 0xbe, 0x70, 0x20, 0xee, 0xa1, 0x6f, 0x1, 0xcf, 0x80, 0x4e, 0x1e, 0xd0, 0x9f, 0x51, 0x43, 0x8d, 0xc2, 0xc, 0x5c, 0x92, 0xdd, 0x13, 0x7d, 0xb3, 0xfc, 0x32, 0x62, 0xac, 0xe3, 0x2d, 0x2a, 0xe4, 0xab, 0x65, 0x35, 0xfb, 0xb4, 0x7a, 0x14, 0xda, 0x95, 0x5b, 0xb, 0xc5, 0x8a, 0x44, 0x56, 0x98, 0xd7, 0x19, 0x49, 0x87, 0xc8, 0x6, 0x68, 0xa6, 0xe9, 0x27, 0x77, 0xb9, 0xf6, 0x38, 0xd2, 0x1c, 0x53, 0x9d, 0xcd, 0x3, 0x4c, 0x82, 0xec, 0x22, 0x6d, 0xa3, 0xf3, 0x3d, 0x72, 0xbc, 0xae, 0x60, 0x2f, 0xe1, 0xb1, 0x7f, 0x30, 0xfe, 0x90, 0x5e, 0x11, 0xdf, 0x8f, 0x41, 0xe, 0xc0},
- {0x0, 0xcf, 0x83, 0x4c, 0x1b, 0xd4, 0x98, 0x57, 0x36, 0xf9, 0xb5, 0x7a, 0x2d, 0xe2, 0xae, 0x61, 0x6c, 0xa3, 0xef, 0x20, 0x77, 0xb8, 0xf4, 0x3b, 0x5a, 0x95, 0xd9, 0x16, 0x41, 0x8e, 0xc2, 0xd, 0xd8, 0x17, 0x5b, 0x94, 0xc3, 0xc, 0x40, 0x8f, 0xee, 0x21, 0x6d, 0xa2, 0xf5, 0x3a, 0x76, 0xb9, 0xb4, 0x7b, 0x37, 0xf8, 0xaf, 0x60, 0x2c, 0xe3, 0x82, 0x4d, 0x1, 0xce, 0x99, 0x56, 0x1a, 0xd5, 0xad, 0x62, 0x2e, 0xe1, 0xb6, 0x79, 0x35, 0xfa, 0x9b, 0x54, 0x18, 0xd7, 0x80, 0x4f, 0x3, 0xcc, 0xc1, 0xe, 0x42, 0x8d, 0xda, 0x15, 0x59, 0x96, 0xf7, 0x38, 0x74, 0xbb, 0xec, 0x23, 0x6f, 0xa0, 0x75, 0xba, 0xf6, 0x39, 0x6e, 0xa1, 0xed, 0x22, 0x43, 0x8c, 0xc0, 0xf, 0x58, 0x97, 0xdb, 0x14, 0x19, 0xd6, 0x9a, 0x55, 0x2, 0xcd, 0x81, 0x4e, 0x2f, 0xe0, 0xac, 0x63, 0x34, 0xfb, 0xb7, 0x78, 0x47, 0x88, 0xc4, 0xb, 0x5c, 0x93, 0xdf, 0x10, 0x71, 0xbe, 0xf2, 0x3d, 0x6a, 0xa5, 0xe9, 0x26, 0x2b, 0xe4, 0xa8, 0x67, 0x30, 0xff, 0xb3, 0x7c, 0x1d, 0xd2, 0x9e, 0x51, 0x6, 0xc9, 0x85, 0x4a, 0x9f, 0x50, 0x1c, 0xd3, 0x84, 0x4b, 0x7, 0xc8, 0xa9, 0x66, 0x2a, 0xe5, 0xb2, 0x7d, 0x31, 0xfe, 0xf3, 0x3c, 0x70, 0xbf, 0xe8, 0x27, 0x6b, 0xa4, 0xc5, 0xa, 0x46, 0x89, 0xde, 0x11, 0x5d, 0x92, 0xea, 0x25, 0x69, 0xa6, 0xf1, 0x3e, 0x72, 0xbd, 0xdc, 0x13, 0x5f, 0x90, 0xc7, 0x8, 0x44, 0x8b, 0x86, 0x49, 0x5, 0xca, 0x9d, 0x52, 0x1e, 0xd1, 0xb0, 0x7f, 0x33, 0xfc, 0xab, 0x64, 0x28, 0xe7, 0x32, 0xfd, 0xb1, 0x7e, 0x29, 0xe6, 0xaa, 0x65, 0x4, 0xcb, 0x87, 0x48, 0x1f, 0xd0, 0x9c, 0x53, 0x5e, 0x91, 0xdd, 0x12, 0x45, 0x8a, 0xc6, 0x9, 0x68, 0xa7, 0xeb, 0x24, 0x73, 0xbc, 0xf0, 0x3f},
- {0x0, 0xd0, 0xbd, 0x6d, 0x67, 0xb7, 0xda, 0xa, 0xce, 0x1e, 0x73, 0xa3, 0xa9, 0x79, 0x14, 0xc4, 0x81, 0x51, 0x3c, 0xec, 0xe6, 0x36, 0x5b, 0x8b, 0x4f, 0x9f, 0xf2, 0x22, 0x28, 0xf8, 0x95, 0x45, 0x1f, 0xcf, 0xa2, 0x72, 0x78, 0xa8, 0xc5, 0x15, 0xd1, 0x1, 0x6c, 0xbc, 0xb6, 0x66, 0xb, 0xdb, 0x9e, 0x4e, 0x23, 0xf3, 0xf9, 0x29, 0x44, 0x94, 0x50, 0x80, 0xed, 0x3d, 0x37, 0xe7, 0x8a, 0x5a, 0x3e, 0xee, 0x83, 0x53, 0x59, 0x89, 0xe4, 0x34, 0xf0, 0x20, 0x4d, 0x9d, 0x97, 0x47, 0x2a, 0xfa, 0xbf, 0x6f, 0x2, 0xd2, 0xd8, 0x8, 0x65, 0xb5, 0x71, 0xa1, 0xcc, 0x1c, 0x16, 0xc6, 0xab, 0x7b, 0x21, 0xf1, 0x9c, 0x4c, 0x46, 0x96, 0xfb, 0x2b, 0xef, 0x3f, 0x52, 0x82, 0x88, 0x58, 0x35, 0xe5, 0xa0, 0x70, 0x1d, 0xcd, 0xc7, 0x17, 0x7a, 0xaa, 0x6e, 0xbe, 0xd3, 0x3, 0x9, 0xd9, 0xb4, 0x64, 0x7c, 0xac, 0xc1, 0x11, 0x1b, 0xcb, 0xa6, 0x76, 0xb2, 0x62, 0xf, 0xdf, 0xd5, 0x5, 0x68, 0xb8, 0xfd, 0x2d, 0x40, 0x90, 0x9a, 0x4a, 0x27, 0xf7, 0x33, 0xe3, 0x8e, 0x5e, 0x54, 0x84, 0xe9, 0x39, 0x63, 0xb3, 0xde, 0xe, 0x4, 0xd4, 0xb9, 0x69, 0xad, 0x7d, 0x10, 0xc0, 0xca, 0x1a, 0x77, 0xa7, 0xe2, 0x32, 0x5f, 0x8f, 0x85, 0x55, 0x38, 0xe8, 0x2c, 0xfc, 0x91, 0x41, 0x4b, 0x9b, 0xf6, 0x26, 0x42, 0x92, 0xff, 0x2f, 0x25, 0xf5, 0x98, 0x48, 0x8c, 0x5c, 0x31, 0xe1, 0xeb, 0x3b, 0x56, 0x86, 0xc3, 0x13, 0x7e, 0xae, 0xa4, 0x74, 0x19, 0xc9, 0xd, 0xdd, 0xb0, 0x60, 0x6a, 0xba, 0xd7, 0x7, 0x5d, 0x8d, 0xe0, 0x30, 0x3a, 0xea, 0x87, 0x57, 0x93, 0x43, 0x2e, 0xfe, 0xf4, 0x24, 0x49, 0x99, 0xdc, 0xc, 0x61, 0xb1, 0xbb, 0x6b, 0x6, 0xd6, 0x12, 0xc2, 0xaf, 0x7f, 0x75, 0xa5, 0xc8, 0x18},
- {0x0, 0xd1, 0xbf, 0x6e, 0x63, 0xb2, 0xdc, 0xd, 0xc6, 0x17, 0x79, 0xa8, 0xa5, 0x74, 0x1a, 0xcb, 0x91, 0x40, 0x2e, 0xff, 0xf2, 0x23, 0x4d, 0x9c, 0x57, 0x86, 0xe8, 0x39, 0x34, 0xe5, 0x8b, 0x5a, 0x3f, 0xee, 0x80, 0x51, 0x5c, 0x8d, 0xe3, 0x32, 0xf9, 0x28, 0x46, 0x97, 0x9a, 0x4b, 0x25, 0xf4, 0xae, 0x7f, 0x11, 0xc0, 0xcd, 0x1c, 0x72, 0xa3, 0x68, 0xb9, 0xd7, 0x6, 0xb, 0xda, 0xb4, 0x65, 0x7e, 0xaf, 0xc1, 0x10, 0x1d, 0xcc, 0xa2, 0x73, 0xb8, 0x69, 0x7, 0xd6, 0xdb, 0xa, 0x64, 0xb5, 0xef, 0x3e, 0x50, 0x81, 0x8c, 0x5d, 0x33, 0xe2, 0x29, 0xf8, 0x96, 0x47, 0x4a, 0x9b, 0xf5, 0x24, 0x41, 0x90, 0xfe, 0x2f, 0x22, 0xf3, 0x9d, 0x4c, 0x87, 0x56, 0x38, 0xe9, 0xe4, 0x35, 0x5b, 0x8a, 0xd0, 0x1, 0x6f, 0xbe, 0xb3, 0x62, 0xc, 0xdd, 0x16, 0xc7, 0xa9, 0x78, 0x75, 0xa4, 0xca, 0x1b, 0xfc, 0x2d, 0x43, 0x92, 0x9f, 0x4e, 0x20, 0xf1, 0x3a, 0xeb, 0x85, 0x54, 0x59, 0x88, 0xe6, 0x37, 0x6d, 0xbc, 0xd2, 0x3, 0xe, 0xdf, 0xb1, 0x60, 0xab, 0x7a, 0x14, 0xc5, 0xc8, 0x19, 0x77, 0xa6, 0xc3, 0x12, 0x7c, 0xad, 0xa0, 0x71, 0x1f, 0xce, 0x5, 0xd4, 0xba, 0x6b, 0x66, 0xb7, 0xd9, 0x8, 0x52, 0x83, 0xed, 0x3c, 0x31, 0xe0, 0x8e, 0x5f, 0x94, 0x45, 0x2b, 0xfa, 0xf7, 0x26, 0x48, 0x99, 0x82, 0x53, 0x3d, 0xec, 0xe1, 0x30, 0x5e, 0x8f, 0x44, 0x95, 0xfb, 0x2a, 0x27, 0xf6, 0x98, 0x49, 0x13, 0xc2, 0xac, 0x7d, 0x70, 0xa1, 0xcf, 0x1e, 0xd5, 0x4, 0x6a, 0xbb, 0xb6, 0x67, 0x9, 0xd8, 0xbd, 0x6c, 0x2, 0xd3, 0xde, 0xf, 0x61, 0xb0, 0x7b, 0xaa, 0xc4, 0x15, 0x18, 0xc9, 0xa7, 0x76, 0x2c, 0xfd, 0x93, 0x42, 0x4f, 0x9e, 0xf0, 0x21, 0xea, 0x3b, 0x55, 0x84, 0x89, 0x58, 0x36, 0xe7},
- {0x0, 0xd2, 0xb9, 0x6b, 0x6f, 0xbd, 0xd6, 0x4, 0xde, 0xc, 0x67, 0xb5, 0xb1, 0x63, 0x8, 0xda, 0xa1, 0x73, 0x18, 0xca, 0xce, 0x1c, 0x77, 0xa5, 0x7f, 0xad, 0xc6, 0x14, 0x10, 0xc2, 0xa9, 0x7b, 0x5f, 0x8d, 0xe6, 0x34, 0x30, 0xe2, 0x89, 0x5b, 0x81, 0x53, 0x38, 0xea, 0xee, 0x3c, 0x57, 0x85, 0xfe, 0x2c, 0x47, 0x95, 0x91, 0x43, 0x28, 0xfa, 0x20, 0xf2, 0x99, 0x4b, 0x4f, 0x9d, 0xf6, 0x24, 0xbe, 0x6c, 0x7, 0xd5, 0xd1, 0x3, 0x68, 0xba, 0x60, 0xb2, 0xd9, 0xb, 0xf, 0xdd, 0xb6, 0x64, 0x1f, 0xcd, 0xa6, 0x74, 0x70, 0xa2, 0xc9, 0x1b, 0xc1, 0x13, 0x78, 0xaa, 0xae, 0x7c, 0x17, 0xc5, 0xe1, 0x33, 0x58, 0x8a, 0x8e, 0x5c, 0x37, 0xe5, 0x3f, 0xed, 0x86, 0x54, 0x50, 0x82, 0xe9, 0x3b, 0x40, 0x92, 0xf9, 0x2b, 0x2f, 0xfd, 0x96, 0x44, 0x9e, 0x4c, 0x27, 0xf5, 0xf1, 0x23, 0x48, 0x9a, 0x61, 0xb3, 0xd8, 0xa, 0xe, 0xdc, 0xb7, 0x65, 0xbf, 0x6d, 0x6, 0xd4, 0xd0, 0x2, 0x69, 0xbb, 0xc0, 0x12, 0x79, 0xab, 0xaf, 0x7d, 0x16, 0xc4, 0x1e, 0xcc, 0xa7, 0x75, 0x71, 0xa3, 0xc8, 0x1a, 0x3e, 0xec, 0x87, 0x55, 0x51, 0x83, 0xe8, 0x3a, 0xe0, 0x32, 0x59, 0x8b, 0x8f, 0x5d, 0x36, 0xe4, 0x9f, 0x4d, 0x26, 0xf4, 0xf0, 0x22, 0x49, 0x9b, 0x41, 0x93, 0xf8, 0x2a, 0x2e, 0xfc, 0x97, 0x45, 0xdf, 0xd, 0x66, 0xb4, 0xb0, 0x62, 0x9, 0xdb, 0x1, 0xd3, 0xb8, 0x6a, 0x6e, 0xbc, 0xd7, 0x5, 0x7e, 0xac, 0xc7, 0x15, 0x11, 0xc3, 0xa8, 0x7a, 0xa0, 0x72, 0x19, 0xcb, 0xcf, 0x1d, 0x76, 0xa4, 0x80, 0x52, 0x39, 0xeb, 0xef, 0x3d, 0x56, 0x84, 0x5e, 0x8c, 0xe7, 0x35, 0x31, 0xe3, 0x88, 0x5a, 0x21, 0xf3, 0x98, 0x4a, 0x4e, 0x9c, 0xf7, 0x25, 0xff, 0x2d, 0x46, 0x94, 0x90, 0x42, 0x29, 0xfb},
- {0x0, 0xd3, 0xbb, 0x68, 0x6b, 0xb8, 0xd0, 0x3, 0xd6, 0x5, 0x6d, 0xbe, 0xbd, 0x6e, 0x6, 0xd5, 0xb1, 0x62, 0xa, 0xd9, 0xda, 0x9, 0x61, 0xb2, 0x67, 0xb4, 0xdc, 0xf, 0xc, 0xdf, 0xb7, 0x64, 0x7f, 0xac, 0xc4, 0x17, 0x14, 0xc7, 0xaf, 0x7c, 0xa9, 0x7a, 0x12, 0xc1, 0xc2, 0x11, 0x79, 0xaa, 0xce, 0x1d, 0x75, 0xa6, 0xa5, 0x76, 0x1e, 0xcd, 0x18, 0xcb, 0xa3, 0x70, 0x73, 0xa0, 0xc8, 0x1b, 0xfe, 0x2d, 0x45, 0x96, 0x95, 0x46, 0x2e, 0xfd, 0x28, 0xfb, 0x93, 0x40, 0x43, 0x90, 0xf8, 0x2b, 0x4f, 0x9c, 0xf4, 0x27, 0x24, 0xf7, 0x9f, 0x4c, 0x99, 0x4a, 0x22, 0xf1, 0xf2, 0x21, 0x49, 0x9a, 0x81, 0x52, 0x3a, 0xe9, 0xea, 0x39, 0x51, 0x82, 0x57, 0x84, 0xec, 0x3f, 0x3c, 0xef, 0x87, 0x54, 0x30, 0xe3, 0x8b, 0x58, 0x5b, 0x88, 0xe0, 0x33, 0xe6, 0x35, 0x5d, 0x8e, 0x8d, 0x5e, 0x36, 0xe5, 0xe1, 0x32, 0x5a, 0x89, 0x8a, 0x59, 0x31, 0xe2, 0x37, 0xe4, 0x8c, 0x5f, 0x5c, 0x8f, 0xe7, 0x34, 0x50, 0x83, 0xeb, 0x38, 0x3b, 0xe8, 0x80, 0x53, 0x86, 0x55, 0x3d, 0xee, 0xed, 0x3e, 0x56, 0x85, 0x9e, 0x4d, 0x25, 0xf6, 0xf5, 0x26, 0x4e, 0x9d, 0x48, 0x9b, 0xf3, 0x20, 0x23, 0xf0, 0x98, 0x4b, 0x2f, 0xfc, 0x94, 0x47, 0x44, 0x97, 0xff, 0x2c, 0xf9, 0x2a, 0x42, 0x91, 0x92, 0x41, 0x29, 0xfa, 0x1f, 0xcc, 0xa4, 0x77, 0x74, 0xa7, 0xcf, 0x1c, 0xc9, 0x1a, 0x72, 0xa1, 0xa2, 0x71, 0x19, 0xca, 0xae, 0x7d, 0x15, 0xc6, 0xc5, 0x16, 0x7e, 0xad, 0x78, 0xab, 0xc3, 0x10, 0x13, 0xc0, 0xa8, 0x7b, 0x60, 0xb3, 0xdb, 0x8, 0xb, 0xd8, 0xb0, 0x63, 0xb6, 0x65, 0xd, 0xde, 0xdd, 0xe, 0x66, 0xb5, 0xd1, 0x2, 0x6a, 0xb9, 0xba, 0x69, 0x1, 0xd2, 0x7, 0xd4, 0xbc, 0x6f, 0x6c, 0xbf, 0xd7, 0x4},
- {0x0, 0xd4, 0xb5, 0x61, 0x77, 0xa3, 0xc2, 0x16, 0xee, 0x3a, 0x5b, 0x8f, 0x99, 0x4d, 0x2c, 0xf8, 0xc1, 0x15, 0x74, 0xa0, 0xb6, 0x62, 0x3, 0xd7, 0x2f, 0xfb, 0x9a, 0x4e, 0x58, 0x8c, 0xed, 0x39, 0x9f, 0x4b, 0x2a, 0xfe, 0xe8, 0x3c, 0x5d, 0x89, 0x71, 0xa5, 0xc4, 0x10, 0x6, 0xd2, 0xb3, 0x67, 0x5e, 0x8a, 0xeb, 0x3f, 0x29, 0xfd, 0x9c, 0x48, 0xb0, 0x64, 0x5, 0xd1, 0xc7, 0x13, 0x72, 0xa6, 0x23, 0xf7, 0x96, 0x42, 0x54, 0x80, 0xe1, 0x35, 0xcd, 0x19, 0x78, 0xac, 0xba, 0x6e, 0xf, 0xdb, 0xe2, 0x36, 0x57, 0x83, 0x95, 0x41, 0x20, 0xf4, 0xc, 0xd8, 0xb9, 0x6d, 0x7b, 0xaf, 0xce, 0x1a, 0xbc, 0x68, 0x9, 0xdd, 0xcb, 0x1f, 0x7e, 0xaa, 0x52, 0x86, 0xe7, 0x33, 0x25, 0xf1, 0x90, 0x44, 0x7d, 0xa9, 0xc8, 0x1c, 0xa, 0xde, 0xbf, 0x6b, 0x93, 0x47, 0x26, 0xf2, 0xe4, 0x30, 0x51, 0x85, 0x46, 0x92, 0xf3, 0x27, 0x31, 0xe5, 0x84, 0x50, 0xa8, 0x7c, 0x1d, 0xc9, 0xdf, 0xb, 0x6a, 0xbe, 0x87, 0x53, 0x32, 0xe6, 0xf0, 0x24, 0x45, 0x91, 0x69, 0xbd, 0xdc, 0x8, 0x1e, 0xca, 0xab, 0x7f, 0xd9, 0xd, 0x6c, 0xb8, 0xae, 0x7a, 0x1b, 0xcf, 0x37, 0xe3, 0x82, 0x56, 0x40, 0x94, 0xf5, 0x21, 0x18, 0xcc, 0xad, 0x79, 0x6f, 0xbb, 0xda, 0xe, 0xf6, 0x22, 0x43, 0x97, 0x81, 0x55, 0x34, 0xe0, 0x65, 0xb1, 0xd0, 0x4, 0x12, 0xc6, 0xa7, 0x73, 0x8b, 0x5f, 0x3e, 0xea, 0xfc, 0x28, 0x49, 0x9d, 0xa4, 0x70, 0x11, 0xc5, 0xd3, 0x7, 0x66, 0xb2, 0x4a, 0x9e, 0xff, 0x2b, 0x3d, 0xe9, 0x88, 0x5c, 0xfa, 0x2e, 0x4f, 0x9b, 0x8d, 0x59, 0x38, 0xec, 0x14, 0xc0, 0xa1, 0x75, 0x63, 0xb7, 0xd6, 0x2, 0x3b, 0xef, 0x8e, 0x5a, 0x4c, 0x98, 0xf9, 0x2d, 0xd5, 0x1, 0x60, 0xb4, 0xa2, 0x76, 0x17, 0xc3},
- {0x0, 0xd5, 0xb7, 0x62, 0x73, 0xa6, 0xc4, 0x11, 0xe6, 0x33, 0x51, 0x84, 0x95, 0x40, 0x22, 0xf7, 0xd1, 0x4, 0x66, 0xb3, 0xa2, 0x77, 0x15, 0xc0, 0x37, 0xe2, 0x80, 0x55, 0x44, 0x91, 0xf3, 0x26, 0xbf, 0x6a, 0x8, 0xdd, 0xcc, 0x19, 0x7b, 0xae, 0x59, 0x8c, 0xee, 0x3b, 0x2a, 0xff, 0x9d, 0x48, 0x6e, 0xbb, 0xd9, 0xc, 0x1d, 0xc8, 0xaa, 0x7f, 0x88, 0x5d, 0x3f, 0xea, 0xfb, 0x2e, 0x4c, 0x99, 0x63, 0xb6, 0xd4, 0x1, 0x10, 0xc5, 0xa7, 0x72, 0x85, 0x50, 0x32, 0xe7, 0xf6, 0x23, 0x41, 0x94, 0xb2, 0x67, 0x5, 0xd0, 0xc1, 0x14, 0x76, 0xa3, 0x54, 0x81, 0xe3, 0x36, 0x27, 0xf2, 0x90, 0x45, 0xdc, 0x9, 0x6b, 0xbe, 0xaf, 0x7a, 0x18, 0xcd, 0x3a, 0xef, 0x8d, 0x58, 0x49, 0x9c, 0xfe, 0x2b, 0xd, 0xd8, 0xba, 0x6f, 0x7e, 0xab, 0xc9, 0x1c, 0xeb, 0x3e, 0x5c, 0x89, 0x98, 0x4d, 0x2f, 0xfa, 0xc6, 0x13, 0x71, 0xa4, 0xb5, 0x60, 0x2, 0xd7, 0x20, 0xf5, 0x97, 0x42, 0x53, 0x86, 0xe4, 0x31, 0x17, 0xc2, 0xa0, 0x75, 0x64, 0xb1, 0xd3, 0x6, 0xf1, 0x24, 0x46, 0x93, 0x82, 0x57, 0x35, 0xe0, 0x79, 0xac, 0xce, 0x1b, 0xa, 0xdf, 0xbd, 0x68, 0x9f, 0x4a, 0x28, 0xfd, 0xec, 0x39, 0x5b, 0x8e, 0xa8, 0x7d, 0x1f, 0xca, 0xdb, 0xe, 0x6c, 0xb9, 0x4e, 0x9b, 0xf9, 0x2c, 0x3d, 0xe8, 0x8a, 0x5f, 0xa5, 0x70, 0x12, 0xc7, 0xd6, 0x3, 0x61, 0xb4, 0x43, 0x96, 0xf4, 0x21, 0x30, 0xe5, 0x87, 0x52, 0x74, 0xa1, 0xc3, 0x16, 0x7, 0xd2, 0xb0, 0x65, 0x92, 0x47, 0x25, 0xf0, 0xe1, 0x34, 0x56, 0x83, 0x1a, 0xcf, 0xad, 0x78, 0x69, 0xbc, 0xde, 0xb, 0xfc, 0x29, 0x4b, 0x9e, 0x8f, 0x5a, 0x38, 0xed, 0xcb, 0x1e, 0x7c, 0xa9, 0xb8, 0x6d, 0xf, 0xda, 0x2d, 0xf8, 0x9a, 0x4f, 0x5e, 0x8b, 0xe9, 0x3c},
- {0x0, 0xd6, 0xb1, 0x67, 0x7f, 0xa9, 0xce, 0x18, 0xfe, 0x28, 0x4f, 0x99, 0x81, 0x57, 0x30, 0xe6, 0xe1, 0x37, 0x50, 0x86, 0x9e, 0x48, 0x2f, 0xf9, 0x1f, 0xc9, 0xae, 0x78, 0x60, 0xb6, 0xd1, 0x7, 0xdf, 0x9, 0x6e, 0xb8, 0xa0, 0x76, 0x11, 0xc7, 0x21, 0xf7, 0x90, 0x46, 0x5e, 0x88, 0xef, 0x39, 0x3e, 0xe8, 0x8f, 0x59, 0x41, 0x97, 0xf0, 0x26, 0xc0, 0x16, 0x71, 0xa7, 0xbf, 0x69, 0xe, 0xd8, 0xa3, 0x75, 0x12, 0xc4, 0xdc, 0xa, 0x6d, 0xbb, 0x5d, 0x8b, 0xec, 0x3a, 0x22, 0xf4, 0x93, 0x45, 0x42, 0x94, 0xf3, 0x25, 0x3d, 0xeb, 0x8c, 0x5a, 0xbc, 0x6a, 0xd, 0xdb, 0xc3, 0x15, 0x72, 0xa4, 0x7c, 0xaa, 0xcd, 0x1b, 0x3, 0xd5, 0xb2, 0x64, 0x82, 0x54, 0x33, 0xe5, 0xfd, 0x2b, 0x4c, 0x9a, 0x9d, 0x4b, 0x2c, 0xfa, 0xe2, 0x34, 0x53, 0x85, 0x63, 0xb5, 0xd2, 0x4, 0x1c, 0xca, 0xad, 0x7b, 0x5b, 0x8d, 0xea, 0x3c, 0x24, 0xf2, 0x95, 0x43, 0xa5, 0x73, 0x14, 0xc2, 0xda, 0xc, 0x6b, 0xbd, 0xba, 0x6c, 0xb, 0xdd, 0xc5, 0x13, 0x74, 0xa2, 0x44, 0x92, 0xf5, 0x23, 0x3b, 0xed, 0x8a, 0x5c, 0x84, 0x52, 0x35, 0xe3, 0xfb, 0x2d, 0x4a, 0x9c, 0x7a, 0xac, 0xcb, 0x1d, 0x5, 0xd3, 0xb4, 0x62, 0x65, 0xb3, 0xd4, 0x2, 0x1a, 0xcc, 0xab, 0x7d, 0x9b, 0x4d, 0x2a, 0xfc, 0xe4, 0x32, 0x55, 0x83, 0xf8, 0x2e, 0x49, 0x9f, 0x87, 0x51, 0x36, 0xe0, 0x6, 0xd0, 0xb7, 0x61, 0x79, 0xaf, 0xc8, 0x1e, 0x19, 0xcf, 0xa8, 0x7e, 0x66, 0xb0, 0xd7, 0x1, 0xe7, 0x31, 0x56, 0x80, 0x98, 0x4e, 0x29, 0xff, 0x27, 0xf1, 0x96, 0x40, 0x58, 0x8e, 0xe9, 0x3f, 0xd9, 0xf, 0x68, 0xbe, 0xa6, 0x70, 0x17, 0xc1, 0xc6, 0x10, 0x77, 0xa1, 0xb9, 0x6f, 0x8, 0xde, 0x38, 0xee, 0x89, 0x5f, 0x47, 0x91, 0xf6, 0x20},
- {0x0, 0xd7, 0xb3, 0x64, 0x7b, 0xac, 0xc8, 0x1f, 0xf6, 0x21, 0x45, 0x92, 0x8d, 0x5a, 0x3e, 0xe9, 0xf1, 0x26, 0x42, 0x95, 0x8a, 0x5d, 0x39, 0xee, 0x7, 0xd0, 0xb4, 0x63, 0x7c, 0xab, 0xcf, 0x18, 0xff, 0x28, 0x4c, 0x9b, 0x84, 0x53, 0x37, 0xe0, 0x9, 0xde, 0xba, 0x6d, 0x72, 0xa5, 0xc1, 0x16, 0xe, 0xd9, 0xbd, 0x6a, 0x75, 0xa2, 0xc6, 0x11, 0xf8, 0x2f, 0x4b, 0x9c, 0x83, 0x54, 0x30, 0xe7, 0xe3, 0x34, 0x50, 0x87, 0x98, 0x4f, 0x2b, 0xfc, 0x15, 0xc2, 0xa6, 0x71, 0x6e, 0xb9, 0xdd, 0xa, 0x12, 0xc5, 0xa1, 0x76, 0x69, 0xbe, 0xda, 0xd, 0xe4, 0x33, 0x57, 0x80, 0x9f, 0x48, 0x2c, 0xfb, 0x1c, 0xcb, 0xaf, 0x78, 0x67, 0xb0, 0xd4, 0x3, 0xea, 0x3d, 0x59, 0x8e, 0x91, 0x46, 0x22, 0xf5, 0xed, 0x3a, 0x5e, 0x89, 0x96, 0x41, 0x25, 0xf2, 0x1b, 0xcc, 0xa8, 0x7f, 0x60, 0xb7, 0xd3, 0x4, 0xdb, 0xc, 0x68, 0xbf, 0xa0, 0x77, 0x13, 0xc4, 0x2d, 0xfa, 0x9e, 0x49, 0x56, 0x81, 0xe5, 0x32, 0x2a, 0xfd, 0x99, 0x4e, 0x51, 0x86, 0xe2, 0x35, 0xdc, 0xb, 0x6f, 0xb8, 0xa7, 0x70, 0x14, 0xc3, 0x24, 0xf3, 0x97, 0x40, 0x5f, 0x88, 0xec, 0x3b, 0xd2, 0x5, 0x61, 0xb6, 0xa9, 0x7e, 0x1a, 0xcd, 0xd5, 0x2, 0x66, 0xb1, 0xae, 0x79, 0x1d, 0xca, 0x23, 0xf4, 0x90, 0x47, 0x58, 0x8f, 0xeb, 0x3c, 0x38, 0xef, 0x8b, 0x5c, 0x43, 0x94, 0xf0, 0x27, 0xce, 0x19, 0x7d, 0xaa, 0xb5, 0x62, 0x6, 0xd1, 0xc9, 0x1e, 0x7a, 0xad, 0xb2, 0x65, 0x1, 0xd6, 0x3f, 0xe8, 0x8c, 0x5b, 0x44, 0x93, 0xf7, 0x20, 0xc7, 0x10, 0x74, 0xa3, 0xbc, 0x6b, 0xf, 0xd8, 0x31, 0xe6, 0x82, 0x55, 0x4a, 0x9d, 0xf9, 0x2e, 0x36, 0xe1, 0x85, 0x52, 0x4d, 0x9a, 0xfe, 0x29, 0xc0, 0x17, 0x73, 0xa4, 0xbb, 0x6c, 0x8, 0xdf},
- {0x0, 0xd8, 0xad, 0x75, 0x47, 0x9f, 0xea, 0x32, 0x8e, 0x56, 0x23, 0xfb, 0xc9, 0x11, 0x64, 0xbc, 0x1, 0xd9, 0xac, 0x74, 0x46, 0x9e, 0xeb, 0x33, 0x8f, 0x57, 0x22, 0xfa, 0xc8, 0x10, 0x65, 0xbd, 0x2, 0xda, 0xaf, 0x77, 0x45, 0x9d, 0xe8, 0x30, 0x8c, 0x54, 0x21, 0xf9, 0xcb, 0x13, 0x66, 0xbe, 0x3, 0xdb, 0xae, 0x76, 0x44, 0x9c, 0xe9, 0x31, 0x8d, 0x55, 0x20, 0xf8, 0xca, 0x12, 0x67, 0xbf, 0x4, 0xdc, 0xa9, 0x71, 0x43, 0x9b, 0xee, 0x36, 0x8a, 0x52, 0x27, 0xff, 0xcd, 0x15, 0x60, 0xb8, 0x5, 0xdd, 0xa8, 0x70, 0x42, 0x9a, 0xef, 0x37, 0x8b, 0x53, 0x26, 0xfe, 0xcc, 0x14, 0x61, 0xb9, 0x6, 0xde, 0xab, 0x73, 0x41, 0x99, 0xec, 0x34, 0x88, 0x50, 0x25, 0xfd, 0xcf, 0x17, 0x62, 0xba, 0x7, 0xdf, 0xaa, 0x72, 0x40, 0x98, 0xed, 0x35, 0x89, 0x51, 0x24, 0xfc, 0xce, 0x16, 0x63, 0xbb, 0x8, 0xd0, 0xa5, 0x7d, 0x4f, 0x97, 0xe2, 0x3a, 0x86, 0x5e, 0x2b, 0xf3, 0xc1, 0x19, 0x6c, 0xb4, 0x9, 0xd1, 0xa4, 0x7c, 0x4e, 0x96, 0xe3, 0x3b, 0x87, 0x5f, 0x2a, 0xf2, 0xc0, 0x18, 0x6d, 0xb5, 0xa, 0xd2, 0xa7, 0x7f, 0x4d, 0x95, 0xe0, 0x38, 0x84, 0x5c, 0x29, 0xf1, 0xc3, 0x1b, 0x6e, 0xb6, 0xb, 0xd3, 0xa6, 0x7e, 0x4c, 0x94, 0xe1, 0x39, 0x85, 0x5d, 0x28, 0xf0, 0xc2, 0x1a, 0x6f, 0xb7, 0xc, 0xd4, 0xa1, 0x79, 0x4b, 0x93, 0xe6, 0x3e, 0x82, 0x5a, 0x2f, 0xf7, 0xc5, 0x1d, 0x68, 0xb0, 0xd, 0xd5, 0xa0, 0x78, 0x4a, 0x92, 0xe7, 0x3f, 0x83, 0x5b, 0x2e, 0xf6, 0xc4, 0x1c, 0x69, 0xb1, 0xe, 0xd6, 0xa3, 0x7b, 0x49, 0x91, 0xe4, 0x3c, 0x80, 0x58, 0x2d, 0xf5, 0xc7, 0x1f, 0x6a, 0xb2, 0xf, 0xd7, 0xa2, 0x7a, 0x48, 0x90, 0xe5, 0x3d, 0x81, 0x59, 0x2c, 0xf4, 0xc6, 0x1e, 0x6b, 0xb3},
- {0x0, 0xd9, 0xaf, 0x76, 0x43, 0x9a, 0xec, 0x35, 0x86, 0x5f, 0x29, 0xf0, 0xc5, 0x1c, 0x6a, 0xb3, 0x11, 0xc8, 0xbe, 0x67, 0x52, 0x8b, 0xfd, 0x24, 0x97, 0x4e, 0x38, 0xe1, 0xd4, 0xd, 0x7b, 0xa2, 0x22, 0xfb, 0x8d, 0x54, 0x61, 0xb8, 0xce, 0x17, 0xa4, 0x7d, 0xb, 0xd2, 0xe7, 0x3e, 0x48, 0x91, 0x33, 0xea, 0x9c, 0x45, 0x70, 0xa9, 0xdf, 0x6, 0xb5, 0x6c, 0x1a, 0xc3, 0xf6, 0x2f, 0x59, 0x80, 0x44, 0x9d, 0xeb, 0x32, 0x7, 0xde, 0xa8, 0x71, 0xc2, 0x1b, 0x6d, 0xb4, 0x81, 0x58, 0x2e, 0xf7, 0x55, 0x8c, 0xfa, 0x23, 0x16, 0xcf, 0xb9, 0x60, 0xd3, 0xa, 0x7c, 0xa5, 0x90, 0x49, 0x3f, 0xe6, 0x66, 0xbf, 0xc9, 0x10, 0x25, 0xfc, 0x8a, 0x53, 0xe0, 0x39, 0x4f, 0x96, 0xa3, 0x7a, 0xc, 0xd5, 0x77, 0xae, 0xd8, 0x1, 0x34, 0xed, 0x9b, 0x42, 0xf1, 0x28, 0x5e, 0x87, 0xb2, 0x6b, 0x1d, 0xc4, 0x88, 0x51, 0x27, 0xfe, 0xcb, 0x12, 0x64, 0xbd, 0xe, 0xd7, 0xa1, 0x78, 0x4d, 0x94, 0xe2, 0x3b, 0x99, 0x40, 0x36, 0xef, 0xda, 0x3, 0x75, 0xac, 0x1f, 0xc6, 0xb0, 0x69, 0x5c, 0x85, 0xf3, 0x2a, 0xaa, 0x73, 0x5, 0xdc, 0xe9, 0x30, 0x46, 0x9f, 0x2c, 0xf5, 0x83, 0x5a, 0x6f, 0xb6, 0xc0, 0x19, 0xbb, 0x62, 0x14, 0xcd, 0xf8, 0x21, 0x57, 0x8e, 0x3d, 0xe4, 0x92, 0x4b, 0x7e, 0xa7, 0xd1, 0x8, 0xcc, 0x15, 0x63, 0xba, 0x8f, 0x56, 0x20, 0xf9, 0x4a, 0x93, 0xe5, 0x3c, 0x9, 0xd0, 0xa6, 0x7f, 0xdd, 0x4, 0x72, 0xab, 0x9e, 0x47, 0x31, 0xe8, 0x5b, 0x82, 0xf4, 0x2d, 0x18, 0xc1, 0xb7, 0x6e, 0xee, 0x37, 0x41, 0x98, 0xad, 0x74, 0x2, 0xdb, 0x68, 0xb1, 0xc7, 0x1e, 0x2b, 0xf2, 0x84, 0x5d, 0xff, 0x26, 0x50, 0x89, 0xbc, 0x65, 0x13, 0xca, 0x79, 0xa0, 0xd6, 0xf, 0x3a, 0xe3, 0x95, 0x4c},
- {0x0, 0xda, 0xa9, 0x73, 0x4f, 0x95, 0xe6, 0x3c, 0x9e, 0x44, 0x37, 0xed, 0xd1, 0xb, 0x78, 0xa2, 0x21, 0xfb, 0x88, 0x52, 0x6e, 0xb4, 0xc7, 0x1d, 0xbf, 0x65, 0x16, 0xcc, 0xf0, 0x2a, 0x59, 0x83, 0x42, 0x98, 0xeb, 0x31, 0xd, 0xd7, 0xa4, 0x7e, 0xdc, 0x6, 0x75, 0xaf, 0x93, 0x49, 0x3a, 0xe0, 0x63, 0xb9, 0xca, 0x10, 0x2c, 0xf6, 0x85, 0x5f, 0xfd, 0x27, 0x54, 0x8e, 0xb2, 0x68, 0x1b, 0xc1, 0x84, 0x5e, 0x2d, 0xf7, 0xcb, 0x11, 0x62, 0xb8, 0x1a, 0xc0, 0xb3, 0x69, 0x55, 0x8f, 0xfc, 0x26, 0xa5, 0x7f, 0xc, 0xd6, 0xea, 0x30, 0x43, 0x99, 0x3b, 0xe1, 0x92, 0x48, 0x74, 0xae, 0xdd, 0x7, 0xc6, 0x1c, 0x6f, 0xb5, 0x89, 0x53, 0x20, 0xfa, 0x58, 0x82, 0xf1, 0x2b, 0x17, 0xcd, 0xbe, 0x64, 0xe7, 0x3d, 0x4e, 0x94, 0xa8, 0x72, 0x1, 0xdb, 0x79, 0xa3, 0xd0, 0xa, 0x36, 0xec, 0x9f, 0x45, 0x15, 0xcf, 0xbc, 0x66, 0x5a, 0x80, 0xf3, 0x29, 0x8b, 0x51, 0x22, 0xf8, 0xc4, 0x1e, 0x6d, 0xb7, 0x34, 0xee, 0x9d, 0x47, 0x7b, 0xa1, 0xd2, 0x8, 0xaa, 0x70, 0x3, 0xd9, 0xe5, 0x3f, 0x4c, 0x96, 0x57, 0x8d, 0xfe, 0x24, 0x18, 0xc2, 0xb1, 0x6b, 0xc9, 0x13, 0x60, 0xba, 0x86, 0x5c, 0x2f, 0xf5, 0x76, 0xac, 0xdf, 0x5, 0x39, 0xe3, 0x90, 0x4a, 0xe8, 0x32, 0x41, 0x9b, 0xa7, 0x7d, 0xe, 0xd4, 0x91, 0x4b, 0x38, 0xe2, 0xde, 0x4, 0x77, 0xad, 0xf, 0xd5, 0xa6, 0x7c, 0x40, 0x9a, 0xe9, 0x33, 0xb0, 0x6a, 0x19, 0xc3, 0xff, 0x25, 0x56, 0x8c, 0x2e, 0xf4, 0x87, 0x5d, 0x61, 0xbb, 0xc8, 0x12, 0xd3, 0x9, 0x7a, 0xa0, 0x9c, 0x46, 0x35, 0xef, 0x4d, 0x97, 0xe4, 0x3e, 0x2, 0xd8, 0xab, 0x71, 0xf2, 0x28, 0x5b, 0x81, 0xbd, 0x67, 0x14, 0xce, 0x6c, 0xb6, 0xc5, 0x1f, 0x23, 0xf9, 0x8a, 0x50},
- {0x0, 0xdb, 0xab, 0x70, 0x4b, 0x90, 0xe0, 0x3b, 0x96, 0x4d, 0x3d, 0xe6, 0xdd, 0x6, 0x76, 0xad, 0x31, 0xea, 0x9a, 0x41, 0x7a, 0xa1, 0xd1, 0xa, 0xa7, 0x7c, 0xc, 0xd7, 0xec, 0x37, 0x47, 0x9c, 0x62, 0xb9, 0xc9, 0x12, 0x29, 0xf2, 0x82, 0x59, 0xf4, 0x2f, 0x5f, 0x84, 0xbf, 0x64, 0x14, 0xcf, 0x53, 0x88, 0xf8, 0x23, 0x18, 0xc3, 0xb3, 0x68, 0xc5, 0x1e, 0x6e, 0xb5, 0x8e, 0x55, 0x25, 0xfe, 0xc4, 0x1f, 0x6f, 0xb4, 0x8f, 0x54, 0x24, 0xff, 0x52, 0x89, 0xf9, 0x22, 0x19, 0xc2, 0xb2, 0x69, 0xf5, 0x2e, 0x5e, 0x85, 0xbe, 0x65, 0x15, 0xce, 0x63, 0xb8, 0xc8, 0x13, 0x28, 0xf3, 0x83, 0x58, 0xa6, 0x7d, 0xd, 0xd6, 0xed, 0x36, 0x46, 0x9d, 0x30, 0xeb, 0x9b, 0x40, 0x7b, 0xa0, 0xd0, 0xb, 0x97, 0x4c, 0x3c, 0xe7, 0xdc, 0x7, 0x77, 0xac, 0x1, 0xda, 0xaa, 0x71, 0x4a, 0x91, 0xe1, 0x3a, 0x95, 0x4e, 0x3e, 0xe5, 0xde, 0x5, 0x75, 0xae, 0x3, 0xd8, 0xa8, 0x73, 0x48, 0x93, 0xe3, 0x38, 0xa4, 0x7f, 0xf, 0xd4, 0xef, 0x34, 0x44, 0x9f, 0x32, 0xe9, 0x99, 0x42, 0x79, 0xa2, 0xd2, 0x9, 0xf7, 0x2c, 0x5c, 0x87, 0xbc, 0x67, 0x17, 0xcc, 0x61, 0xba, 0xca, 0x11, 0x2a, 0xf1, 0x81, 0x5a, 0xc6, 0x1d, 0x6d, 0xb6, 0x8d, 0x56, 0x26, 0xfd, 0x50, 0x8b, 0xfb, 0x20, 0x1b, 0xc0, 0xb0, 0x6b, 0x51, 0x8a, 0xfa, 0x21, 0x1a, 0xc1, 0xb1, 0x6a, 0xc7, 0x1c, 0x6c, 0xb7, 0x8c, 0x57, 0x27, 0xfc, 0x60, 0xbb, 0xcb, 0x10, 0x2b, 0xf0, 0x80, 0x5b, 0xf6, 0x2d, 0x5d, 0x86, 0xbd, 0x66, 0x16, 0xcd, 0x33, 0xe8, 0x98, 0x43, 0x78, 0xa3, 0xd3, 0x8, 0xa5, 0x7e, 0xe, 0xd5, 0xee, 0x35, 0x45, 0x9e, 0x2, 0xd9, 0xa9, 0x72, 0x49, 0x92, 0xe2, 0x39, 0x94, 0x4f, 0x3f, 0xe4, 0xdf, 0x4, 0x74, 0xaf},
- {0x0, 0xdc, 0xa5, 0x79, 0x57, 0x8b, 0xf2, 0x2e, 0xae, 0x72, 0xb, 0xd7, 0xf9, 0x25, 0x5c, 0x80, 0x41, 0x9d, 0xe4, 0x38, 0x16, 0xca, 0xb3, 0x6f, 0xef, 0x33, 0x4a, 0x96, 0xb8, 0x64, 0x1d, 0xc1, 0x82, 0x5e, 0x27, 0xfb, 0xd5, 0x9, 0x70, 0xac, 0x2c, 0xf0, 0x89, 0x55, 0x7b, 0xa7, 0xde, 0x2, 0xc3, 0x1f, 0x66, 0xba, 0x94, 0x48, 0x31, 0xed, 0x6d, 0xb1, 0xc8, 0x14, 0x3a, 0xe6, 0x9f, 0x43, 0x19, 0xc5, 0xbc, 0x60, 0x4e, 0x92, 0xeb, 0x37, 0xb7, 0x6b, 0x12, 0xce, 0xe0, 0x3c, 0x45, 0x99, 0x58, 0x84, 0xfd, 0x21, 0xf, 0xd3, 0xaa, 0x76, 0xf6, 0x2a, 0x53, 0x8f, 0xa1, 0x7d, 0x4, 0xd8, 0x9b, 0x47, 0x3e, 0xe2, 0xcc, 0x10, 0x69, 0xb5, 0x35, 0xe9, 0x90, 0x4c, 0x62, 0xbe, 0xc7, 0x1b, 0xda, 0x6, 0x7f, 0xa3, 0x8d, 0x51, 0x28, 0xf4, 0x74, 0xa8, 0xd1, 0xd, 0x23, 0xff, 0x86, 0x5a, 0x32, 0xee, 0x97, 0x4b, 0x65, 0xb9, 0xc0, 0x1c, 0x9c, 0x40, 0x39, 0xe5, 0xcb, 0x17, 0x6e, 0xb2, 0x73, 0xaf, 0xd6, 0xa, 0x24, 0xf8, 0x81, 0x5d, 0xdd, 0x1, 0x78, 0xa4, 0x8a, 0x56, 0x2f, 0xf3, 0xb0, 0x6c, 0x15, 0xc9, 0xe7, 0x3b, 0x42, 0x9e, 0x1e, 0xc2, 0xbb, 0x67, 0x49, 0x95, 0xec, 0x30, 0xf1, 0x2d, 0x54, 0x88, 0xa6, 0x7a, 0x3, 0xdf, 0x5f, 0x83, 0xfa, 0x26, 0x8, 0xd4, 0xad, 0x71, 0x2b, 0xf7, 0x8e, 0x52, 0x7c, 0xa0, 0xd9, 0x5, 0x85, 0x59, 0x20, 0xfc, 0xd2, 0xe, 0x77, 0xab, 0x6a, 0xb6, 0xcf, 0x13, 0x3d, 0xe1, 0x98, 0x44, 0xc4, 0x18, 0x61, 0xbd, 0x93, 0x4f, 0x36, 0xea, 0xa9, 0x75, 0xc, 0xd0, 0xfe, 0x22, 0x5b, 0x87, 0x7, 0xdb, 0xa2, 0x7e, 0x50, 0x8c, 0xf5, 0x29, 0xe8, 0x34, 0x4d, 0x91, 0xbf, 0x63, 0x1a, 0xc6, 0x46, 0x9a, 0xe3, 0x3f, 0x11, 0xcd, 0xb4, 0x68},
- {0x0, 0xdd, 0xa7, 0x7a, 0x53, 0x8e, 0xf4, 0x29, 0xa6, 0x7b, 0x1, 0xdc, 0xf5, 0x28, 0x52, 0x8f, 0x51, 0x8c, 0xf6, 0x2b, 0x2, 0xdf, 0xa5, 0x78, 0xf7, 0x2a, 0x50, 0x8d, 0xa4, 0x79, 0x3, 0xde, 0xa2, 0x7f, 0x5, 0xd8, 0xf1, 0x2c, 0x56, 0x8b, 0x4, 0xd9, 0xa3, 0x7e, 0x57, 0x8a, 0xf0, 0x2d, 0xf3, 0x2e, 0x54, 0x89, 0xa0, 0x7d, 0x7, 0xda, 0x55, 0x88, 0xf2, 0x2f, 0x6, 0xdb, 0xa1, 0x7c, 0x59, 0x84, 0xfe, 0x23, 0xa, 0xd7, 0xad, 0x70, 0xff, 0x22, 0x58, 0x85, 0xac, 0x71, 0xb, 0xd6, 0x8, 0xd5, 0xaf, 0x72, 0x5b, 0x86, 0xfc, 0x21, 0xae, 0x73, 0x9, 0xd4, 0xfd, 0x20, 0x5a, 0x87, 0xfb, 0x26, 0x5c, 0x81, 0xa8, 0x75, 0xf, 0xd2, 0x5d, 0x80, 0xfa, 0x27, 0xe, 0xd3, 0xa9, 0x74, 0xaa, 0x77, 0xd, 0xd0, 0xf9, 0x24, 0x5e, 0x83, 0xc, 0xd1, 0xab, 0x76, 0x5f, 0x82, 0xf8, 0x25, 0xb2, 0x6f, 0x15, 0xc8, 0xe1, 0x3c, 0x46, 0x9b, 0x14, 0xc9, 0xb3, 0x6e, 0x47, 0x9a, 0xe0, 0x3d, 0xe3, 0x3e, 0x44, 0x99, 0xb0, 0x6d, 0x17, 0xca, 0x45, 0x98, 0xe2, 0x3f, 0x16, 0xcb, 0xb1, 0x6c, 0x10, 0xcd, 0xb7, 0x6a, 0x43, 0x9e, 0xe4, 0x39, 0xb6, 0x6b, 0x11, 0xcc, 0xe5, 0x38, 0x42, 0x9f, 0x41, 0x9c, 0xe6, 0x3b, 0x12, 0xcf, 0xb5, 0x68, 0xe7, 0x3a, 0x40, 0x9d, 0xb4, 0x69, 0x13, 0xce, 0xeb, 0x36, 0x4c, 0x91, 0xb8, 0x65, 0x1f, 0xc2, 0x4d, 0x90, 0xea, 0x37, 0x1e, 0xc3, 0xb9, 0x64, 0xba, 0x67, 0x1d, 0xc0, 0xe9, 0x34, 0x4e, 0x93, 0x1c, 0xc1, 0xbb, 0x66, 0x4f, 0x92, 0xe8, 0x35, 0x49, 0x94, 0xee, 0x33, 0x1a, 0xc7, 0xbd, 0x60, 0xef, 0x32, 0x48, 0x95, 0xbc, 0x61, 0x1b, 0xc6, 0x18, 0xc5, 0xbf, 0x62, 0x4b, 0x96, 0xec, 0x31, 0xbe, 0x63, 0x19, 0xc4, 0xed, 0x30, 0x4a, 0x97},
- {0x0, 0xde, 0xa1, 0x7f, 0x5f, 0x81, 0xfe, 0x20, 0xbe, 0x60, 0x1f, 0xc1, 0xe1, 0x3f, 0x40, 0x9e, 0x61, 0xbf, 0xc0, 0x1e, 0x3e, 0xe0, 0x9f, 0x41, 0xdf, 0x1, 0x7e, 0xa0, 0x80, 0x5e, 0x21, 0xff, 0xc2, 0x1c, 0x63, 0xbd, 0x9d, 0x43, 0x3c, 0xe2, 0x7c, 0xa2, 0xdd, 0x3, 0x23, 0xfd, 0x82, 0x5c, 0xa3, 0x7d, 0x2, 0xdc, 0xfc, 0x22, 0x5d, 0x83, 0x1d, 0xc3, 0xbc, 0x62, 0x42, 0x9c, 0xe3, 0x3d, 0x99, 0x47, 0x38, 0xe6, 0xc6, 0x18, 0x67, 0xb9, 0x27, 0xf9, 0x86, 0x58, 0x78, 0xa6, 0xd9, 0x7, 0xf8, 0x26, 0x59, 0x87, 0xa7, 0x79, 0x6, 0xd8, 0x46, 0x98, 0xe7, 0x39, 0x19, 0xc7, 0xb8, 0x66, 0x5b, 0x85, 0xfa, 0x24, 0x4, 0xda, 0xa5, 0x7b, 0xe5, 0x3b, 0x44, 0x9a, 0xba, 0x64, 0x1b, 0xc5, 0x3a, 0xe4, 0x9b, 0x45, 0x65, 0xbb, 0xc4, 0x1a, 0x84, 0x5a, 0x25, 0xfb, 0xdb, 0x5, 0x7a, 0xa4, 0x2f, 0xf1, 0x8e, 0x50, 0x70, 0xae, 0xd1, 0xf, 0x91, 0x4f, 0x30, 0xee, 0xce, 0x10, 0x6f, 0xb1, 0x4e, 0x90, 0xef, 0x31, 0x11, 0xcf, 0xb0, 0x6e, 0xf0, 0x2e, 0x51, 0x8f, 0xaf, 0x71, 0xe, 0xd0, 0xed, 0x33, 0x4c, 0x92, 0xb2, 0x6c, 0x13, 0xcd, 0x53, 0x8d, 0xf2, 0x2c, 0xc, 0xd2, 0xad, 0x73, 0x8c, 0x52, 0x2d, 0xf3, 0xd3, 0xd, 0x72, 0xac, 0x32, 0xec, 0x93, 0x4d, 0x6d, 0xb3, 0xcc, 0x12, 0xb6, 0x68, 0x17, 0xc9, 0xe9, 0x37, 0x48, 0x96, 0x8, 0xd6, 0xa9, 0x77, 0x57, 0x89, 0xf6, 0x28, 0xd7, 0x9, 0x76, 0xa8, 0x88, 0x56, 0x29, 0xf7, 0x69, 0xb7, 0xc8, 0x16, 0x36, 0xe8, 0x97, 0x49, 0x74, 0xaa, 0xd5, 0xb, 0x2b, 0xf5, 0x8a, 0x54, 0xca, 0x14, 0x6b, 0xb5, 0x95, 0x4b, 0x34, 0xea, 0x15, 0xcb, 0xb4, 0x6a, 0x4a, 0x94, 0xeb, 0x35, 0xab, 0x75, 0xa, 0xd4, 0xf4, 0x2a, 0x55, 0x8b},
- {0x0, 0xdf, 0xa3, 0x7c, 0x5b, 0x84, 0xf8, 0x27, 0xb6, 0x69, 0x15, 0xca, 0xed, 0x32, 0x4e, 0x91, 0x71, 0xae, 0xd2, 0xd, 0x2a, 0xf5, 0x89, 0x56, 0xc7, 0x18, 0x64, 0xbb, 0x9c, 0x43, 0x3f, 0xe0, 0xe2, 0x3d, 0x41, 0x9e, 0xb9, 0x66, 0x1a, 0xc5, 0x54, 0x8b, 0xf7, 0x28, 0xf, 0xd0, 0xac, 0x73, 0x93, 0x4c, 0x30, 0xef, 0xc8, 0x17, 0x6b, 0xb4, 0x25, 0xfa, 0x86, 0x59, 0x7e, 0xa1, 0xdd, 0x2, 0xd9, 0x6, 0x7a, 0xa5, 0x82, 0x5d, 0x21, 0xfe, 0x6f, 0xb0, 0xcc, 0x13, 0x34, 0xeb, 0x97, 0x48, 0xa8, 0x77, 0xb, 0xd4, 0xf3, 0x2c, 0x50, 0x8f, 0x1e, 0xc1, 0xbd, 0x62, 0x45, 0x9a, 0xe6, 0x39, 0x3b, 0xe4, 0x98, 0x47, 0x60, 0xbf, 0xc3, 0x1c, 0x8d, 0x52, 0x2e, 0xf1, 0xd6, 0x9, 0x75, 0xaa, 0x4a, 0x95, 0xe9, 0x36, 0x11, 0xce, 0xb2, 0x6d, 0xfc, 0x23, 0x5f, 0x80, 0xa7, 0x78, 0x4, 0xdb, 0xaf, 0x70, 0xc, 0xd3, 0xf4, 0x2b, 0x57, 0x88, 0x19, 0xc6, 0xba, 0x65, 0x42, 0x9d, 0xe1, 0x3e, 0xde, 0x1, 0x7d, 0xa2, 0x85, 0x5a, 0x26, 0xf9, 0x68, 0xb7, 0xcb, 0x14, 0x33, 0xec, 0x90, 0x4f, 0x4d, 0x92, 0xee, 0x31, 0x16, 0xc9, 0xb5, 0x6a, 0xfb, 0x24, 0x58, 0x87, 0xa0, 0x7f, 0x3, 0xdc, 0x3c, 0xe3, 0x9f, 0x40, 0x67, 0xb8, 0xc4, 0x1b, 0x8a, 0x55, 0x29, 0xf6, 0xd1, 0xe, 0x72, 0xad, 0x76, 0xa9, 0xd5, 0xa, 0x2d, 0xf2, 0x8e, 0x51, 0xc0, 0x1f, 0x63, 0xbc, 0x9b, 0x44, 0x38, 0xe7, 0x7, 0xd8, 0xa4, 0x7b, 0x5c, 0x83, 0xff, 0x20, 0xb1, 0x6e, 0x12, 0xcd, 0xea, 0x35, 0x49, 0x96, 0x94, 0x4b, 0x37, 0xe8, 0xcf, 0x10, 0x6c, 0xb3, 0x22, 0xfd, 0x81, 0x5e, 0x79, 0xa6, 0xda, 0x5, 0xe5, 0x3a, 0x46, 0x99, 0xbe, 0x61, 0x1d, 0xc2, 0x53, 0x8c, 0xf0, 0x2f, 0x8, 0xd7, 0xab, 0x74},
- {0x0, 0xe0, 0xdd, 0x3d, 0xa7, 0x47, 0x7a, 0x9a, 0x53, 0xb3, 0x8e, 0x6e, 0xf4, 0x14, 0x29, 0xc9, 0xa6, 0x46, 0x7b, 0x9b, 0x1, 0xe1, 0xdc, 0x3c, 0xf5, 0x15, 0x28, 0xc8, 0x52, 0xb2, 0x8f, 0x6f, 0x51, 0xb1, 0x8c, 0x6c, 0xf6, 0x16, 0x2b, 0xcb, 0x2, 0xe2, 0xdf, 0x3f, 0xa5, 0x45, 0x78, 0x98, 0xf7, 0x17, 0x2a, 0xca, 0x50, 0xb0, 0x8d, 0x6d, 0xa4, 0x44, 0x79, 0x99, 0x3, 0xe3, 0xde, 0x3e, 0xa2, 0x42, 0x7f, 0x9f, 0x5, 0xe5, 0xd8, 0x38, 0xf1, 0x11, 0x2c, 0xcc, 0x56, 0xb6, 0x8b, 0x6b, 0x4, 0xe4, 0xd9, 0x39, 0xa3, 0x43, 0x7e, 0x9e, 0x57, 0xb7, 0x8a, 0x6a, 0xf0, 0x10, 0x2d, 0xcd, 0xf3, 0x13, 0x2e, 0xce, 0x54, 0xb4, 0x89, 0x69, 0xa0, 0x40, 0x7d, 0x9d, 0x7, 0xe7, 0xda, 0x3a, 0x55, 0xb5, 0x88, 0x68, 0xf2, 0x12, 0x2f, 0xcf, 0x6, 0xe6, 0xdb, 0x3b, 0xa1, 0x41, 0x7c, 0x9c, 0x59, 0xb9, 0x84, 0x64, 0xfe, 0x1e, 0x23, 0xc3, 0xa, 0xea, 0xd7, 0x37, 0xad, 0x4d, 0x70, 0x90, 0xff, 0x1f, 0x22, 0xc2, 0x58, 0xb8, 0x85, 0x65, 0xac, 0x4c, 0x71, 0x91, 0xb, 0xeb, 0xd6, 0x36, 0x8, 0xe8, 0xd5, 0x35, 0xaf, 0x4f, 0x72, 0x92, 0x5b, 0xbb, 0x86, 0x66, 0xfc, 0x1c, 0x21, 0xc1, 0xae, 0x4e, 0x73, 0x93, 0x9, 0xe9, 0xd4, 0x34, 0xfd, 0x1d, 0x20, 0xc0, 0x5a, 0xba, 0x87, 0x67, 0xfb, 0x1b, 0x26, 0xc6, 0x5c, 0xbc, 0x81, 0x61, 0xa8, 0x48, 0x75, 0x95, 0xf, 0xef, 0xd2, 0x32, 0x5d, 0xbd, 0x80, 0x60, 0xfa, 0x1a, 0x27, 0xc7, 0xe, 0xee, 0xd3, 0x33, 0xa9, 0x49, 0x74, 0x94, 0xaa, 0x4a, 0x77, 0x97, 0xd, 0xed, 0xd0, 0x30, 0xf9, 0x19, 0x24, 0xc4, 0x5e, 0xbe, 0x83, 0x63, 0xc, 0xec, 0xd1, 0x31, 0xab, 0x4b, 0x76, 0x96, 0x5f, 0xbf, 0x82, 0x62, 0xf8, 0x18, 0x25, 0xc5},
- {0x0, 0xe1, 0xdf, 0x3e, 0xa3, 0x42, 0x7c, 0x9d, 0x5b, 0xba, 0x84, 0x65, 0xf8, 0x19, 0x27, 0xc6, 0xb6, 0x57, 0x69, 0x88, 0x15, 0xf4, 0xca, 0x2b, 0xed, 0xc, 0x32, 0xd3, 0x4e, 0xaf, 0x91, 0x70, 0x71, 0x90, 0xae, 0x4f, 0xd2, 0x33, 0xd, 0xec, 0x2a, 0xcb, 0xf5, 0x14, 0x89, 0x68, 0x56, 0xb7, 0xc7, 0x26, 0x18, 0xf9, 0x64, 0x85, 0xbb, 0x5a, 0x9c, 0x7d, 0x43, 0xa2, 0x3f, 0xde, 0xe0, 0x1, 0xe2, 0x3, 0x3d, 0xdc, 0x41, 0xa0, 0x9e, 0x7f, 0xb9, 0x58, 0x66, 0x87, 0x1a, 0xfb, 0xc5, 0x24, 0x54, 0xb5, 0x8b, 0x6a, 0xf7, 0x16, 0x28, 0xc9, 0xf, 0xee, 0xd0, 0x31, 0xac, 0x4d, 0x73, 0x92, 0x93, 0x72, 0x4c, 0xad, 0x30, 0xd1, 0xef, 0xe, 0xc8, 0x29, 0x17, 0xf6, 0x6b, 0x8a, 0xb4, 0x55, 0x25, 0xc4, 0xfa, 0x1b, 0x86, 0x67, 0x59, 0xb8, 0x7e, 0x9f, 0xa1, 0x40, 0xdd, 0x3c, 0x2, 0xe3, 0xd9, 0x38, 0x6, 0xe7, 0x7a, 0x9b, 0xa5, 0x44, 0x82, 0x63, 0x5d, 0xbc, 0x21, 0xc0, 0xfe, 0x1f, 0x6f, 0x8e, 0xb0, 0x51, 0xcc, 0x2d, 0x13, 0xf2, 0x34, 0xd5, 0xeb, 0xa, 0x97, 0x76, 0x48, 0xa9, 0xa8, 0x49, 0x77, 0x96, 0xb, 0xea, 0xd4, 0x35, 0xf3, 0x12, 0x2c, 0xcd, 0x50, 0xb1, 0x8f, 0x6e, 0x1e, 0xff, 0xc1, 0x20, 0xbd, 0x5c, 0x62, 0x83, 0x45, 0xa4, 0x9a, 0x7b, 0xe6, 0x7, 0x39, 0xd8, 0x3b, 0xda, 0xe4, 0x5, 0x98, 0x79, 0x47, 0xa6, 0x60, 0x81, 0xbf, 0x5e, 0xc3, 0x22, 0x1c, 0xfd, 0x8d, 0x6c, 0x52, 0xb3, 0x2e, 0xcf, 0xf1, 0x10, 0xd6, 0x37, 0x9, 0xe8, 0x75, 0x94, 0xaa, 0x4b, 0x4a, 0xab, 0x95, 0x74, 0xe9, 0x8, 0x36, 0xd7, 0x11, 0xf0, 0xce, 0x2f, 0xb2, 0x53, 0x6d, 0x8c, 0xfc, 0x1d, 0x23, 0xc2, 0x5f, 0xbe, 0x80, 0x61, 0xa7, 0x46, 0x78, 0x99, 0x4, 0xe5, 0xdb, 0x3a},
- {0x0, 0xe2, 0xd9, 0x3b, 0xaf, 0x4d, 0x76, 0x94, 0x43, 0xa1, 0x9a, 0x78, 0xec, 0xe, 0x35, 0xd7, 0x86, 0x64, 0x5f, 0xbd, 0x29, 0xcb, 0xf0, 0x12, 0xc5, 0x27, 0x1c, 0xfe, 0x6a, 0x88, 0xb3, 0x51, 0x11, 0xf3, 0xc8, 0x2a, 0xbe, 0x5c, 0x67, 0x85, 0x52, 0xb0, 0x8b, 0x69, 0xfd, 0x1f, 0x24, 0xc6, 0x97, 0x75, 0x4e, 0xac, 0x38, 0xda, 0xe1, 0x3, 0xd4, 0x36, 0xd, 0xef, 0x7b, 0x99, 0xa2, 0x40, 0x22, 0xc0, 0xfb, 0x19, 0x8d, 0x6f, 0x54, 0xb6, 0x61, 0x83, 0xb8, 0x5a, 0xce, 0x2c, 0x17, 0xf5, 0xa4, 0x46, 0x7d, 0x9f, 0xb, 0xe9, 0xd2, 0x30, 0xe7, 0x5, 0x3e, 0xdc, 0x48, 0xaa, 0x91, 0x73, 0x33, 0xd1, 0xea, 0x8, 0x9c, 0x7e, 0x45, 0xa7, 0x70, 0x92, 0xa9, 0x4b, 0xdf, 0x3d, 0x6, 0xe4, 0xb5, 0x57, 0x6c, 0x8e, 0x1a, 0xf8, 0xc3, 0x21, 0xf6, 0x14, 0x2f, 0xcd, 0x59, 0xbb, 0x80, 0x62, 0x44, 0xa6, 0x9d, 0x7f, 0xeb, 0x9, 0x32, 0xd0, 0x7, 0xe5, 0xde, 0x3c, 0xa8, 0x4a, 0x71, 0x93, 0xc2, 0x20, 0x1b, 0xf9, 0x6d, 0x8f, 0xb4, 0x56, 0x81, 0x63, 0x58, 0xba, 0x2e, 0xcc, 0xf7, 0x15, 0x55, 0xb7, 0x8c, 0x6e, 0xfa, 0x18, 0x23, 0xc1, 0x16, 0xf4, 0xcf, 0x2d, 0xb9, 0x5b, 0x60, 0x82, 0xd3, 0x31, 0xa, 0xe8, 0x7c, 0x9e, 0xa5, 0x47, 0x90, 0x72, 0x49, 0xab, 0x3f, 0xdd, 0xe6, 0x4, 0x66, 0x84, 0xbf, 0x5d, 0xc9, 0x2b, 0x10, 0xf2, 0x25, 0xc7, 0xfc, 0x1e, 0x8a, 0x68, 0x53, 0xb1, 0xe0, 0x2, 0x39, 0xdb, 0x4f, 0xad, 0x96, 0x74, 0xa3, 0x41, 0x7a, 0x98, 0xc, 0xee, 0xd5, 0x37, 0x77, 0x95, 0xae, 0x4c, 0xd8, 0x3a, 0x1, 0xe3, 0x34, 0xd6, 0xed, 0xf, 0x9b, 0x79, 0x42, 0xa0, 0xf1, 0x13, 0x28, 0xca, 0x5e, 0xbc, 0x87, 0x65, 0xb2, 0x50, 0x6b, 0x89, 0x1d, 0xff, 0xc4, 0x26},
- {0x0, 0xe3, 0xdb, 0x38, 0xab, 0x48, 0x70, 0x93, 0x4b, 0xa8, 0x90, 0x73, 0xe0, 0x3, 0x3b, 0xd8, 0x96, 0x75, 0x4d, 0xae, 0x3d, 0xde, 0xe6, 0x5, 0xdd, 0x3e, 0x6, 0xe5, 0x76, 0x95, 0xad, 0x4e, 0x31, 0xd2, 0xea, 0x9, 0x9a, 0x79, 0x41, 0xa2, 0x7a, 0x99, 0xa1, 0x42, 0xd1, 0x32, 0xa, 0xe9, 0xa7, 0x44, 0x7c, 0x9f, 0xc, 0xef, 0xd7, 0x34, 0xec, 0xf, 0x37, 0xd4, 0x47, 0xa4, 0x9c, 0x7f, 0x62, 0x81, 0xb9, 0x5a, 0xc9, 0x2a, 0x12, 0xf1, 0x29, 0xca, 0xf2, 0x11, 0x82, 0x61, 0x59, 0xba, 0xf4, 0x17, 0x2f, 0xcc, 0x5f, 0xbc, 0x84, 0x67, 0xbf, 0x5c, 0x64, 0x87, 0x14, 0xf7, 0xcf, 0x2c, 0x53, 0xb0, 0x88, 0x6b, 0xf8, 0x1b, 0x23, 0xc0, 0x18, 0xfb, 0xc3, 0x20, 0xb3, 0x50, 0x68, 0x8b, 0xc5, 0x26, 0x1e, 0xfd, 0x6e, 0x8d, 0xb5, 0x56, 0x8e, 0x6d, 0x55, 0xb6, 0x25, 0xc6, 0xfe, 0x1d, 0xc4, 0x27, 0x1f, 0xfc, 0x6f, 0x8c, 0xb4, 0x57, 0x8f, 0x6c, 0x54, 0xb7, 0x24, 0xc7, 0xff, 0x1c, 0x52, 0xb1, 0x89, 0x6a, 0xf9, 0x1a, 0x22, 0xc1, 0x19, 0xfa, 0xc2, 0x21, 0xb2, 0x51, 0x69, 0x8a, 0xf5, 0x16, 0x2e, 0xcd, 0x5e, 0xbd, 0x85, 0x66, 0xbe, 0x5d, 0x65, 0x86, 0x15, 0xf6, 0xce, 0x2d, 0x63, 0x80, 0xb8, 0x5b, 0xc8, 0x2b, 0x13, 0xf0, 0x28, 0xcb, 0xf3, 0x10, 0x83, 0x60, 0x58, 0xbb, 0xa6, 0x45, 0x7d, 0x9e, 0xd, 0xee, 0xd6, 0x35, 0xed, 0xe, 0x36, 0xd5, 0x46, 0xa5, 0x9d, 0x7e, 0x30, 0xd3, 0xeb, 0x8, 0x9b, 0x78, 0x40, 0xa3, 0x7b, 0x98, 0xa0, 0x43, 0xd0, 0x33, 0xb, 0xe8, 0x97, 0x74, 0x4c, 0xaf, 0x3c, 0xdf, 0xe7, 0x4, 0xdc, 0x3f, 0x7, 0xe4, 0x77, 0x94, 0xac, 0x4f, 0x1, 0xe2, 0xda, 0x39, 0xaa, 0x49, 0x71, 0x92, 0x4a, 0xa9, 0x91, 0x72, 0xe1, 0x2, 0x3a, 0xd9},
- {0x0, 0xe4, 0xd5, 0x31, 0xb7, 0x53, 0x62, 0x86, 0x73, 0x97, 0xa6, 0x42, 0xc4, 0x20, 0x11, 0xf5, 0xe6, 0x2, 0x33, 0xd7, 0x51, 0xb5, 0x84, 0x60, 0x95, 0x71, 0x40, 0xa4, 0x22, 0xc6, 0xf7, 0x13, 0xd1, 0x35, 0x4, 0xe0, 0x66, 0x82, 0xb3, 0x57, 0xa2, 0x46, 0x77, 0x93, 0x15, 0xf1, 0xc0, 0x24, 0x37, 0xd3, 0xe2, 0x6, 0x80, 0x64, 0x55, 0xb1, 0x44, 0xa0, 0x91, 0x75, 0xf3, 0x17, 0x26, 0xc2, 0xbf, 0x5b, 0x6a, 0x8e, 0x8, 0xec, 0xdd, 0x39, 0xcc, 0x28, 0x19, 0xfd, 0x7b, 0x9f, 0xae, 0x4a, 0x59, 0xbd, 0x8c, 0x68, 0xee, 0xa, 0x3b, 0xdf, 0x2a, 0xce, 0xff, 0x1b, 0x9d, 0x79, 0x48, 0xac, 0x6e, 0x8a, 0xbb, 0x5f, 0xd9, 0x3d, 0xc, 0xe8, 0x1d, 0xf9, 0xc8, 0x2c, 0xaa, 0x4e, 0x7f, 0x9b, 0x88, 0x6c, 0x5d, 0xb9, 0x3f, 0xdb, 0xea, 0xe, 0xfb, 0x1f, 0x2e, 0xca, 0x4c, 0xa8, 0x99, 0x7d, 0x63, 0x87, 0xb6, 0x52, 0xd4, 0x30, 0x1, 0xe5, 0x10, 0xf4, 0xc5, 0x21, 0xa7, 0x43, 0x72, 0x96, 0x85, 0x61, 0x50, 0xb4, 0x32, 0xd6, 0xe7, 0x3, 0xf6, 0x12, 0x23, 0xc7, 0x41, 0xa5, 0x94, 0x70, 0xb2, 0x56, 0x67, 0x83, 0x5, 0xe1, 0xd0, 0x34, 0xc1, 0x25, 0x14, 0xf0, 0x76, 0x92, 0xa3, 0x47, 0x54, 0xb0, 0x81, 0x65, 0xe3, 0x7, 0x36, 0xd2, 0x27, 0xc3, 0xf2, 0x16, 0x90, 0x74, 0x45, 0xa1, 0xdc, 0x38, 0x9, 0xed, 0x6b, 0x8f, 0xbe, 0x5a, 0xaf, 0x4b, 0x7a, 0x9e, 0x18, 0xfc, 0xcd, 0x29, 0x3a, 0xde, 0xef, 0xb, 0x8d, 0x69, 0x58, 0xbc, 0x49, 0xad, 0x9c, 0x78, 0xfe, 0x1a, 0x2b, 0xcf, 0xd, 0xe9, 0xd8, 0x3c, 0xba, 0x5e, 0x6f, 0x8b, 0x7e, 0x9a, 0xab, 0x4f, 0xc9, 0x2d, 0x1c, 0xf8, 0xeb, 0xf, 0x3e, 0xda, 0x5c, 0xb8, 0x89, 0x6d, 0x98, 0x7c, 0x4d, 0xa9, 0x2f, 0xcb, 0xfa, 0x1e},
- {0x0, 0xe5, 0xd7, 0x32, 0xb3, 0x56, 0x64, 0x81, 0x7b, 0x9e, 0xac, 0x49, 0xc8, 0x2d, 0x1f, 0xfa, 0xf6, 0x13, 0x21, 0xc4, 0x45, 0xa0, 0x92, 0x77, 0x8d, 0x68, 0x5a, 0xbf, 0x3e, 0xdb, 0xe9, 0xc, 0xf1, 0x14, 0x26, 0xc3, 0x42, 0xa7, 0x95, 0x70, 0x8a, 0x6f, 0x5d, 0xb8, 0x39, 0xdc, 0xee, 0xb, 0x7, 0xe2, 0xd0, 0x35, 0xb4, 0x51, 0x63, 0x86, 0x7c, 0x99, 0xab, 0x4e, 0xcf, 0x2a, 0x18, 0xfd, 0xff, 0x1a, 0x28, 0xcd, 0x4c, 0xa9, 0x9b, 0x7e, 0x84, 0x61, 0x53, 0xb6, 0x37, 0xd2, 0xe0, 0x5, 0x9, 0xec, 0xde, 0x3b, 0xba, 0x5f, 0x6d, 0x88, 0x72, 0x97, 0xa5, 0x40, 0xc1, 0x24, 0x16, 0xf3, 0xe, 0xeb, 0xd9, 0x3c, 0xbd, 0x58, 0x6a, 0x8f, 0x75, 0x90, 0xa2, 0x47, 0xc6, 0x23, 0x11, 0xf4, 0xf8, 0x1d, 0x2f, 0xca, 0x4b, 0xae, 0x9c, 0x79, 0x83, 0x66, 0x54, 0xb1, 0x30, 0xd5, 0xe7, 0x2, 0xe3, 0x6, 0x34, 0xd1, 0x50, 0xb5, 0x87, 0x62, 0x98, 0x7d, 0x4f, 0xaa, 0x2b, 0xce, 0xfc, 0x19, 0x15, 0xf0, 0xc2, 0x27, 0xa6, 0x43, 0x71, 0x94, 0x6e, 0x8b, 0xb9, 0x5c, 0xdd, 0x38, 0xa, 0xef, 0x12, 0xf7, 0xc5, 0x20, 0xa1, 0x44, 0x76, 0x93, 0x69, 0x8c, 0xbe, 0x5b, 0xda, 0x3f, 0xd, 0xe8, 0xe4, 0x1, 0x33, 0xd6, 0x57, 0xb2, 0x80, 0x65, 0x9f, 0x7a, 0x48, 0xad, 0x2c, 0xc9, 0xfb, 0x1e, 0x1c, 0xf9, 0xcb, 0x2e, 0xaf, 0x4a, 0x78, 0x9d, 0x67, 0x82, 0xb0, 0x55, 0xd4, 0x31, 0x3, 0xe6, 0xea, 0xf, 0x3d, 0xd8, 0x59, 0xbc, 0x8e, 0x6b, 0x91, 0x74, 0x46, 0xa3, 0x22, 0xc7, 0xf5, 0x10, 0xed, 0x8, 0x3a, 0xdf, 0x5e, 0xbb, 0x89, 0x6c, 0x96, 0x73, 0x41, 0xa4, 0x25, 0xc0, 0xf2, 0x17, 0x1b, 0xfe, 0xcc, 0x29, 0xa8, 0x4d, 0x7f, 0x9a, 0x60, 0x85, 0xb7, 0x52, 0xd3, 0x36, 0x4, 0xe1},
- {0x0, 0xe6, 0xd1, 0x37, 0xbf, 0x59, 0x6e, 0x88, 0x63, 0x85, 0xb2, 0x54, 0xdc, 0x3a, 0xd, 0xeb, 0xc6, 0x20, 0x17, 0xf1, 0x79, 0x9f, 0xa8, 0x4e, 0xa5, 0x43, 0x74, 0x92, 0x1a, 0xfc, 0xcb, 0x2d, 0x91, 0x77, 0x40, 0xa6, 0x2e, 0xc8, 0xff, 0x19, 0xf2, 0x14, 0x23, 0xc5, 0x4d, 0xab, 0x9c, 0x7a, 0x57, 0xb1, 0x86, 0x60, 0xe8, 0xe, 0x39, 0xdf, 0x34, 0xd2, 0xe5, 0x3, 0x8b, 0x6d, 0x5a, 0xbc, 0x3f, 0xd9, 0xee, 0x8, 0x80, 0x66, 0x51, 0xb7, 0x5c, 0xba, 0x8d, 0x6b, 0xe3, 0x5, 0x32, 0xd4, 0xf9, 0x1f, 0x28, 0xce, 0x46, 0xa0, 0x97, 0x71, 0x9a, 0x7c, 0x4b, 0xad, 0x25, 0xc3, 0xf4, 0x12, 0xae, 0x48, 0x7f, 0x99, 0x11, 0xf7, 0xc0, 0x26, 0xcd, 0x2b, 0x1c, 0xfa, 0x72, 0x94, 0xa3, 0x45, 0x68, 0x8e, 0xb9, 0x5f, 0xd7, 0x31, 0x6, 0xe0, 0xb, 0xed, 0xda, 0x3c, 0xb4, 0x52, 0x65, 0x83, 0x7e, 0x98, 0xaf, 0x49, 0xc1, 0x27, 0x10, 0xf6, 0x1d, 0xfb, 0xcc, 0x2a, 0xa2, 0x44, 0x73, 0x95, 0xb8, 0x5e, 0x69, 0x8f, 0x7, 0xe1, 0xd6, 0x30, 0xdb, 0x3d, 0xa, 0xec, 0x64, 0x82, 0xb5, 0x53, 0xef, 0x9, 0x3e, 0xd8, 0x50, 0xb6, 0x81, 0x67, 0x8c, 0x6a, 0x5d, 0xbb, 0x33, 0xd5, 0xe2, 0x4, 0x29, 0xcf, 0xf8, 0x1e, 0x96, 0x70, 0x47, 0xa1, 0x4a, 0xac, 0x9b, 0x7d, 0xf5, 0x13, 0x24, 0xc2, 0x41, 0xa7, 0x90, 0x76, 0xfe, 0x18, 0x2f, 0xc9, 0x22, 0xc4, 0xf3, 0x15, 0x9d, 0x7b, 0x4c, 0xaa, 0x87, 0x61, 0x56, 0xb0, 0x38, 0xde, 0xe9, 0xf, 0xe4, 0x2, 0x35, 0xd3, 0x5b, 0xbd, 0x8a, 0x6c, 0xd0, 0x36, 0x1, 0xe7, 0x6f, 0x89, 0xbe, 0x58, 0xb3, 0x55, 0x62, 0x84, 0xc, 0xea, 0xdd, 0x3b, 0x16, 0xf0, 0xc7, 0x21, 0xa9, 0x4f, 0x78, 0x9e, 0x75, 0x93, 0xa4, 0x42, 0xca, 0x2c, 0x1b, 0xfd},
- {0x0, 0xe7, 0xd3, 0x34, 0xbb, 0x5c, 0x68, 0x8f, 0x6b, 0x8c, 0xb8, 0x5f, 0xd0, 0x37, 0x3, 0xe4, 0xd6, 0x31, 0x5, 0xe2, 0x6d, 0x8a, 0xbe, 0x59, 0xbd, 0x5a, 0x6e, 0x89, 0x6, 0xe1, 0xd5, 0x32, 0xb1, 0x56, 0x62, 0x85, 0xa, 0xed, 0xd9, 0x3e, 0xda, 0x3d, 0x9, 0xee, 0x61, 0x86, 0xb2, 0x55, 0x67, 0x80, 0xb4, 0x53, 0xdc, 0x3b, 0xf, 0xe8, 0xc, 0xeb, 0xdf, 0x38, 0xb7, 0x50, 0x64, 0x83, 0x7f, 0x98, 0xac, 0x4b, 0xc4, 0x23, 0x17, 0xf0, 0x14, 0xf3, 0xc7, 0x20, 0xaf, 0x48, 0x7c, 0x9b, 0xa9, 0x4e, 0x7a, 0x9d, 0x12, 0xf5, 0xc1, 0x26, 0xc2, 0x25, 0x11, 0xf6, 0x79, 0x9e, 0xaa, 0x4d, 0xce, 0x29, 0x1d, 0xfa, 0x75, 0x92, 0xa6, 0x41, 0xa5, 0x42, 0x76, 0x91, 0x1e, 0xf9, 0xcd, 0x2a, 0x18, 0xff, 0xcb, 0x2c, 0xa3, 0x44, 0x70, 0x97, 0x73, 0x94, 0xa0, 0x47, 0xc8, 0x2f, 0x1b, 0xfc, 0xfe, 0x19, 0x2d, 0xca, 0x45, 0xa2, 0x96, 0x71, 0x95, 0x72, 0x46, 0xa1, 0x2e, 0xc9, 0xfd, 0x1a, 0x28, 0xcf, 0xfb, 0x1c, 0x93, 0x74, 0x40, 0xa7, 0x43, 0xa4, 0x90, 0x77, 0xf8, 0x1f, 0x2b, 0xcc, 0x4f, 0xa8, 0x9c, 0x7b, 0xf4, 0x13, 0x27, 0xc0, 0x24, 0xc3, 0xf7, 0x10, 0x9f, 0x78, 0x4c, 0xab, 0x99, 0x7e, 0x4a, 0xad, 0x22, 0xc5, 0xf1, 0x16, 0xf2, 0x15, 0x21, 0xc6, 0x49, 0xae, 0x9a, 0x7d, 0x81, 0x66, 0x52, 0xb5, 0x3a, 0xdd, 0xe9, 0xe, 0xea, 0xd, 0x39, 0xde, 0x51, 0xb6, 0x82, 0x65, 0x57, 0xb0, 0x84, 0x63, 0xec, 0xb, 0x3f, 0xd8, 0x3c, 0xdb, 0xef, 0x8, 0x87, 0x60, 0x54, 0xb3, 0x30, 0xd7, 0xe3, 0x4, 0x8b, 0x6c, 0x58, 0xbf, 0x5b, 0xbc, 0x88, 0x6f, 0xe0, 0x7, 0x33, 0xd4, 0xe6, 0x1, 0x35, 0xd2, 0x5d, 0xba, 0x8e, 0x69, 0x8d, 0x6a, 0x5e, 0xb9, 0x36, 0xd1, 0xe5, 0x2},
- {0x0, 0xe8, 0xcd, 0x25, 0x87, 0x6f, 0x4a, 0xa2, 0x13, 0xfb, 0xde, 0x36, 0x94, 0x7c, 0x59, 0xb1, 0x26, 0xce, 0xeb, 0x3, 0xa1, 0x49, 0x6c, 0x84, 0x35, 0xdd, 0xf8, 0x10, 0xb2, 0x5a, 0x7f, 0x97, 0x4c, 0xa4, 0x81, 0x69, 0xcb, 0x23, 0x6, 0xee, 0x5f, 0xb7, 0x92, 0x7a, 0xd8, 0x30, 0x15, 0xfd, 0x6a, 0x82, 0xa7, 0x4f, 0xed, 0x5, 0x20, 0xc8, 0x79, 0x91, 0xb4, 0x5c, 0xfe, 0x16, 0x33, 0xdb, 0x98, 0x70, 0x55, 0xbd, 0x1f, 0xf7, 0xd2, 0x3a, 0x8b, 0x63, 0x46, 0xae, 0xc, 0xe4, 0xc1, 0x29, 0xbe, 0x56, 0x73, 0x9b, 0x39, 0xd1, 0xf4, 0x1c, 0xad, 0x45, 0x60, 0x88, 0x2a, 0xc2, 0xe7, 0xf, 0xd4, 0x3c, 0x19, 0xf1, 0x53, 0xbb, 0x9e, 0x76, 0xc7, 0x2f, 0xa, 0xe2, 0x40, 0xa8, 0x8d, 0x65, 0xf2, 0x1a, 0x3f, 0xd7, 0x75, 0x9d, 0xb8, 0x50, 0xe1, 0x9, 0x2c, 0xc4, 0x66, 0x8e, 0xab, 0x43, 0x2d, 0xc5, 0xe0, 0x8, 0xaa, 0x42, 0x67, 0x8f, 0x3e, 0xd6, 0xf3, 0x1b, 0xb9, 0x51, 0x74, 0x9c, 0xb, 0xe3, 0xc6, 0x2e, 0x8c, 0x64, 0x41, 0xa9, 0x18, 0xf0, 0xd5, 0x3d, 0x9f, 0x77, 0x52, 0xba, 0x61, 0x89, 0xac, 0x44, 0xe6, 0xe, 0x2b, 0xc3, 0x72, 0x9a, 0xbf, 0x57, 0xf5, 0x1d, 0x38, 0xd0, 0x47, 0xaf, 0x8a, 0x62, 0xc0, 0x28, 0xd, 0xe5, 0x54, 0xbc, 0x99, 0x71, 0xd3, 0x3b, 0x1e, 0xf6, 0xb5, 0x5d, 0x78, 0x90, 0x32, 0xda, 0xff, 0x17, 0xa6, 0x4e, 0x6b, 0x83, 0x21, 0xc9, 0xec, 0x4, 0x93, 0x7b, 0x5e, 0xb6, 0x14, 0xfc, 0xd9, 0x31, 0x80, 0x68, 0x4d, 0xa5, 0x7, 0xef, 0xca, 0x22, 0xf9, 0x11, 0x34, 0xdc, 0x7e, 0x96, 0xb3, 0x5b, 0xea, 0x2, 0x27, 0xcf, 0x6d, 0x85, 0xa0, 0x48, 0xdf, 0x37, 0x12, 0xfa, 0x58, 0xb0, 0x95, 0x7d, 0xcc, 0x24, 0x1, 0xe9, 0x4b, 0xa3, 0x86, 0x6e},
- {0x0, 0xe9, 0xcf, 0x26, 0x83, 0x6a, 0x4c, 0xa5, 0x1b, 0xf2, 0xd4, 0x3d, 0x98, 0x71, 0x57, 0xbe, 0x36, 0xdf, 0xf9, 0x10, 0xb5, 0x5c, 0x7a, 0x93, 0x2d, 0xc4, 0xe2, 0xb, 0xae, 0x47, 0x61, 0x88, 0x6c, 0x85, 0xa3, 0x4a, 0xef, 0x6, 0x20, 0xc9, 0x77, 0x9e, 0xb8, 0x51, 0xf4, 0x1d, 0x3b, 0xd2, 0x5a, 0xb3, 0x95, 0x7c, 0xd9, 0x30, 0x16, 0xff, 0x41, 0xa8, 0x8e, 0x67, 0xc2, 0x2b, 0xd, 0xe4, 0xd8, 0x31, 0x17, 0xfe, 0x5b, 0xb2, 0x94, 0x7d, 0xc3, 0x2a, 0xc, 0xe5, 0x40, 0xa9, 0x8f, 0x66, 0xee, 0x7, 0x21, 0xc8, 0x6d, 0x84, 0xa2, 0x4b, 0xf5, 0x1c, 0x3a, 0xd3, 0x76, 0x9f, 0xb9, 0x50, 0xb4, 0x5d, 0x7b, 0x92, 0x37, 0xde, 0xf8, 0x11, 0xaf, 0x46, 0x60, 0x89, 0x2c, 0xc5, 0xe3, 0xa, 0x82, 0x6b, 0x4d, 0xa4, 0x1, 0xe8, 0xce, 0x27, 0x99, 0x70, 0x56, 0xbf, 0x1a, 0xf3, 0xd5, 0x3c, 0xad, 0x44, 0x62, 0x8b, 0x2e, 0xc7, 0xe1, 0x8, 0xb6, 0x5f, 0x79, 0x90, 0x35, 0xdc, 0xfa, 0x13, 0x9b, 0x72, 0x54, 0xbd, 0x18, 0xf1, 0xd7, 0x3e, 0x80, 0x69, 0x4f, 0xa6, 0x3, 0xea, 0xcc, 0x25, 0xc1, 0x28, 0xe, 0xe7, 0x42, 0xab, 0x8d, 0x64, 0xda, 0x33, 0x15, 0xfc, 0x59, 0xb0, 0x96, 0x7f, 0xf7, 0x1e, 0x38, 0xd1, 0x74, 0x9d, 0xbb, 0x52, 0xec, 0x5, 0x23, 0xca, 0x6f, 0x86, 0xa0, 0x49, 0x75, 0x9c, 0xba, 0x53, 0xf6, 0x1f, 0x39, 0xd0, 0x6e, 0x87, 0xa1, 0x48, 0xed, 0x4, 0x22, 0xcb, 0x43, 0xaa, 0x8c, 0x65, 0xc0, 0x29, 0xf, 0xe6, 0x58, 0xb1, 0x97, 0x7e, 0xdb, 0x32, 0x14, 0xfd, 0x19, 0xf0, 0xd6, 0x3f, 0x9a, 0x73, 0x55, 0xbc, 0x2, 0xeb, 0xcd, 0x24, 0x81, 0x68, 0x4e, 0xa7, 0x2f, 0xc6, 0xe0, 0x9, 0xac, 0x45, 0x63, 0x8a, 0x34, 0xdd, 0xfb, 0x12, 0xb7, 0x5e, 0x78, 0x91},
- {0x0, 0xea, 0xc9, 0x23, 0x8f, 0x65, 0x46, 0xac, 0x3, 0xe9, 0xca, 0x20, 0x8c, 0x66, 0x45, 0xaf, 0x6, 0xec, 0xcf, 0x25, 0x89, 0x63, 0x40, 0xaa, 0x5, 0xef, 0xcc, 0x26, 0x8a, 0x60, 0x43, 0xa9, 0xc, 0xe6, 0xc5, 0x2f, 0x83, 0x69, 0x4a, 0xa0, 0xf, 0xe5, 0xc6, 0x2c, 0x80, 0x6a, 0x49, 0xa3, 0xa, 0xe0, 0xc3, 0x29, 0x85, 0x6f, 0x4c, 0xa6, 0x9, 0xe3, 0xc0, 0x2a, 0x86, 0x6c, 0x4f, 0xa5, 0x18, 0xf2, 0xd1, 0x3b, 0x97, 0x7d, 0x5e, 0xb4, 0x1b, 0xf1, 0xd2, 0x38, 0x94, 0x7e, 0x5d, 0xb7, 0x1e, 0xf4, 0xd7, 0x3d, 0x91, 0x7b, 0x58, 0xb2, 0x1d, 0xf7, 0xd4, 0x3e, 0x92, 0x78, 0x5b, 0xb1, 0x14, 0xfe, 0xdd, 0x37, 0x9b, 0x71, 0x52, 0xb8, 0x17, 0xfd, 0xde, 0x34, 0x98, 0x72, 0x51, 0xbb, 0x12, 0xf8, 0xdb, 0x31, 0x9d, 0x77, 0x54, 0xbe, 0x11, 0xfb, 0xd8, 0x32, 0x9e, 0x74, 0x57, 0xbd, 0x30, 0xda, 0xf9, 0x13, 0xbf, 0x55, 0x76, 0x9c, 0x33, 0xd9, 0xfa, 0x10, 0xbc, 0x56, 0x75, 0x9f, 0x36, 0xdc, 0xff, 0x15, 0xb9, 0x53, 0x70, 0x9a, 0x35, 0xdf, 0xfc, 0x16, 0xba, 0x50, 0x73, 0x99, 0x3c, 0xd6, 0xf5, 0x1f, 0xb3, 0x59, 0x7a, 0x90, 0x3f, 0xd5, 0xf6, 0x1c, 0xb0, 0x5a, 0x79, 0x93, 0x3a, 0xd0, 0xf3, 0x19, 0xb5, 0x5f, 0x7c, 0x96, 0x39, 0xd3, 0xf0, 0x1a, 0xb6, 0x5c, 0x7f, 0x95, 0x28, 0xc2, 0xe1, 0xb, 0xa7, 0x4d, 0x6e, 0x84, 0x2b, 0xc1, 0xe2, 0x8, 0xa4, 0x4e, 0x6d, 0x87, 0x2e, 0xc4, 0xe7, 0xd, 0xa1, 0x4b, 0x68, 0x82, 0x2d, 0xc7, 0xe4, 0xe, 0xa2, 0x48, 0x6b, 0x81, 0x24, 0xce, 0xed, 0x7, 0xab, 0x41, 0x62, 0x88, 0x27, 0xcd, 0xee, 0x4, 0xa8, 0x42, 0x61, 0x8b, 0x22, 0xc8, 0xeb, 0x1, 0xad, 0x47, 0x64, 0x8e, 0x21, 0xcb, 0xe8, 0x2, 0xae, 0x44, 0x67, 0x8d},
- {0x0, 0xeb, 0xcb, 0x20, 0x8b, 0x60, 0x40, 0xab, 0xb, 0xe0, 0xc0, 0x2b, 0x80, 0x6b, 0x4b, 0xa0, 0x16, 0xfd, 0xdd, 0x36, 0x9d, 0x76, 0x56, 0xbd, 0x1d, 0xf6, 0xd6, 0x3d, 0x96, 0x7d, 0x5d, 0xb6, 0x2c, 0xc7, 0xe7, 0xc, 0xa7, 0x4c, 0x6c, 0x87, 0x27, 0xcc, 0xec, 0x7, 0xac, 0x47, 0x67, 0x8c, 0x3a, 0xd1, 0xf1, 0x1a, 0xb1, 0x5a, 0x7a, 0x91, 0x31, 0xda, 0xfa, 0x11, 0xba, 0x51, 0x71, 0x9a, 0x58, 0xb3, 0x93, 0x78, 0xd3, 0x38, 0x18, 0xf3, 0x53, 0xb8, 0x98, 0x73, 0xd8, 0x33, 0x13, 0xf8, 0x4e, 0xa5, 0x85, 0x6e, 0xc5, 0x2e, 0xe, 0xe5, 0x45, 0xae, 0x8e, 0x65, 0xce, 0x25, 0x5, 0xee, 0x74, 0x9f, 0xbf, 0x54, 0xff, 0x14, 0x34, 0xdf, 0x7f, 0x94, 0xb4, 0x5f, 0xf4, 0x1f, 0x3f, 0xd4, 0x62, 0x89, 0xa9, 0x42, 0xe9, 0x2, 0x22, 0xc9, 0x69, 0x82, 0xa2, 0x49, 0xe2, 0x9, 0x29, 0xc2, 0xb0, 0x5b, 0x7b, 0x90, 0x3b, 0xd0, 0xf0, 0x1b, 0xbb, 0x50, 0x70, 0x9b, 0x30, 0xdb, 0xfb, 0x10, 0xa6, 0x4d, 0x6d, 0x86, 0x2d, 0xc6, 0xe6, 0xd, 0xad, 0x46, 0x66, 0x8d, 0x26, 0xcd, 0xed, 0x6, 0x9c, 0x77, 0x57, 0xbc, 0x17, 0xfc, 0xdc, 0x37, 0x97, 0x7c, 0x5c, 0xb7, 0x1c, 0xf7, 0xd7, 0x3c, 0x8a, 0x61, 0x41, 0xaa, 0x1, 0xea, 0xca, 0x21, 0x81, 0x6a, 0x4a, 0xa1, 0xa, 0xe1, 0xc1, 0x2a, 0xe8, 0x3, 0x23, 0xc8, 0x63, 0x88, 0xa8, 0x43, 0xe3, 0x8, 0x28, 0xc3, 0x68, 0x83, 0xa3, 0x48, 0xfe, 0x15, 0x35, 0xde, 0x75, 0x9e, 0xbe, 0x55, 0xf5, 0x1e, 0x3e, 0xd5, 0x7e, 0x95, 0xb5, 0x5e, 0xc4, 0x2f, 0xf, 0xe4, 0x4f, 0xa4, 0x84, 0x6f, 0xcf, 0x24, 0x4, 0xef, 0x44, 0xaf, 0x8f, 0x64, 0xd2, 0x39, 0x19, 0xf2, 0x59, 0xb2, 0x92, 0x79, 0xd9, 0x32, 0x12, 0xf9, 0x52, 0xb9, 0x99, 0x72},
- {0x0, 0xec, 0xc5, 0x29, 0x97, 0x7b, 0x52, 0xbe, 0x33, 0xdf, 0xf6, 0x1a, 0xa4, 0x48, 0x61, 0x8d, 0x66, 0x8a, 0xa3, 0x4f, 0xf1, 0x1d, 0x34, 0xd8, 0x55, 0xb9, 0x90, 0x7c, 0xc2, 0x2e, 0x7, 0xeb, 0xcc, 0x20, 0x9, 0xe5, 0x5b, 0xb7, 0x9e, 0x72, 0xff, 0x13, 0x3a, 0xd6, 0x68, 0x84, 0xad, 0x41, 0xaa, 0x46, 0x6f, 0x83, 0x3d, 0xd1, 0xf8, 0x14, 0x99, 0x75, 0x5c, 0xb0, 0xe, 0xe2, 0xcb, 0x27, 0x85, 0x69, 0x40, 0xac, 0x12, 0xfe, 0xd7, 0x3b, 0xb6, 0x5a, 0x73, 0x9f, 0x21, 0xcd, 0xe4, 0x8, 0xe3, 0xf, 0x26, 0xca, 0x74, 0x98, 0xb1, 0x5d, 0xd0, 0x3c, 0x15, 0xf9, 0x47, 0xab, 0x82, 0x6e, 0x49, 0xa5, 0x8c, 0x60, 0xde, 0x32, 0x1b, 0xf7, 0x7a, 0x96, 0xbf, 0x53, 0xed, 0x1, 0x28, 0xc4, 0x2f, 0xc3, 0xea, 0x6, 0xb8, 0x54, 0x7d, 0x91, 0x1c, 0xf0, 0xd9, 0x35, 0x8b, 0x67, 0x4e, 0xa2, 0x17, 0xfb, 0xd2, 0x3e, 0x80, 0x6c, 0x45, 0xa9, 0x24, 0xc8, 0xe1, 0xd, 0xb3, 0x5f, 0x76, 0x9a, 0x71, 0x9d, 0xb4, 0x58, 0xe6, 0xa, 0x23, 0xcf, 0x42, 0xae, 0x87, 0x6b, 0xd5, 0x39, 0x10, 0xfc, 0xdb, 0x37, 0x1e, 0xf2, 0x4c, 0xa0, 0x89, 0x65, 0xe8, 0x4, 0x2d, 0xc1, 0x7f, 0x93, 0xba, 0x56, 0xbd, 0x51, 0x78, 0x94, 0x2a, 0xc6, 0xef, 0x3, 0x8e, 0x62, 0x4b, 0xa7, 0x19, 0xf5, 0xdc, 0x30, 0x92, 0x7e, 0x57, 0xbb, 0x5, 0xe9, 0xc0, 0x2c, 0xa1, 0x4d, 0x64, 0x88, 0x36, 0xda, 0xf3, 0x1f, 0xf4, 0x18, 0x31, 0xdd, 0x63, 0x8f, 0xa6, 0x4a, 0xc7, 0x2b, 0x2, 0xee, 0x50, 0xbc, 0x95, 0x79, 0x5e, 0xb2, 0x9b, 0x77, 0xc9, 0x25, 0xc, 0xe0, 0x6d, 0x81, 0xa8, 0x44, 0xfa, 0x16, 0x3f, 0xd3, 0x38, 0xd4, 0xfd, 0x11, 0xaf, 0x43, 0x6a, 0x86, 0xb, 0xe7, 0xce, 0x22, 0x9c, 0x70, 0x59, 0xb5},
- {0x0, 0xed, 0xc7, 0x2a, 0x93, 0x7e, 0x54, 0xb9, 0x3b, 0xd6, 0xfc, 0x11, 0xa8, 0x45, 0x6f, 0x82, 0x76, 0x9b, 0xb1, 0x5c, 0xe5, 0x8, 0x22, 0xcf, 0x4d, 0xa0, 0x8a, 0x67, 0xde, 0x33, 0x19, 0xf4, 0xec, 0x1, 0x2b, 0xc6, 0x7f, 0x92, 0xb8, 0x55, 0xd7, 0x3a, 0x10, 0xfd, 0x44, 0xa9, 0x83, 0x6e, 0x9a, 0x77, 0x5d, 0xb0, 0x9, 0xe4, 0xce, 0x23, 0xa1, 0x4c, 0x66, 0x8b, 0x32, 0xdf, 0xf5, 0x18, 0xc5, 0x28, 0x2, 0xef, 0x56, 0xbb, 0x91, 0x7c, 0xfe, 0x13, 0x39, 0xd4, 0x6d, 0x80, 0xaa, 0x47, 0xb3, 0x5e, 0x74, 0x99, 0x20, 0xcd, 0xe7, 0xa, 0x88, 0x65, 0x4f, 0xa2, 0x1b, 0xf6, 0xdc, 0x31, 0x29, 0xc4, 0xee, 0x3, 0xba, 0x57, 0x7d, 0x90, 0x12, 0xff, 0xd5, 0x38, 0x81, 0x6c, 0x46, 0xab, 0x5f, 0xb2, 0x98, 0x75, 0xcc, 0x21, 0xb, 0xe6, 0x64, 0x89, 0xa3, 0x4e, 0xf7, 0x1a, 0x30, 0xdd, 0x97, 0x7a, 0x50, 0xbd, 0x4, 0xe9, 0xc3, 0x2e, 0xac, 0x41, 0x6b, 0x86, 0x3f, 0xd2, 0xf8, 0x15, 0xe1, 0xc, 0x26, 0xcb, 0x72, 0x9f, 0xb5, 0x58, 0xda, 0x37, 0x1d, 0xf0, 0x49, 0xa4, 0x8e, 0x63, 0x7b, 0x96, 0xbc, 0x51, 0xe8, 0x5, 0x2f, 0xc2, 0x40, 0xad, 0x87, 0x6a, 0xd3, 0x3e, 0x14, 0xf9, 0xd, 0xe0, 0xca, 0x27, 0x9e, 0x73, 0x59, 0xb4, 0x36, 0xdb, 0xf1, 0x1c, 0xa5, 0x48, 0x62, 0x8f, 0x52, 0xbf, 0x95, 0x78, 0xc1, 0x2c, 0x6, 0xeb, 0x69, 0x84, 0xae, 0x43, 0xfa, 0x17, 0x3d, 0xd0, 0x24, 0xc9, 0xe3, 0xe, 0xb7, 0x5a, 0x70, 0x9d, 0x1f, 0xf2, 0xd8, 0x35, 0x8c, 0x61, 0x4b, 0xa6, 0xbe, 0x53, 0x79, 0x94, 0x2d, 0xc0, 0xea, 0x7, 0x85, 0x68, 0x42, 0xaf, 0x16, 0xfb, 0xd1, 0x3c, 0xc8, 0x25, 0xf, 0xe2, 0x5b, 0xb6, 0x9c, 0x71, 0xf3, 0x1e, 0x34, 0xd9, 0x60, 0x8d, 0xa7, 0x4a},
- {0x0, 0xee, 0xc1, 0x2f, 0x9f, 0x71, 0x5e, 0xb0, 0x23, 0xcd, 0xe2, 0xc, 0xbc, 0x52, 0x7d, 0x93, 0x46, 0xa8, 0x87, 0x69, 0xd9, 0x37, 0x18, 0xf6, 0x65, 0x8b, 0xa4, 0x4a, 0xfa, 0x14, 0x3b, 0xd5, 0x8c, 0x62, 0x4d, 0xa3, 0x13, 0xfd, 0xd2, 0x3c, 0xaf, 0x41, 0x6e, 0x80, 0x30, 0xde, 0xf1, 0x1f, 0xca, 0x24, 0xb, 0xe5, 0x55, 0xbb, 0x94, 0x7a, 0xe9, 0x7, 0x28, 0xc6, 0x76, 0x98, 0xb7, 0x59, 0x5, 0xeb, 0xc4, 0x2a, 0x9a, 0x74, 0x5b, 0xb5, 0x26, 0xc8, 0xe7, 0x9, 0xb9, 0x57, 0x78, 0x96, 0x43, 0xad, 0x82, 0x6c, 0xdc, 0x32, 0x1d, 0xf3, 0x60, 0x8e, 0xa1, 0x4f, 0xff, 0x11, 0x3e, 0xd0, 0x89, 0x67, 0x48, 0xa6, 0x16, 0xf8, 0xd7, 0x39, 0xaa, 0x44, 0x6b, 0x85, 0x35, 0xdb, 0xf4, 0x1a, 0xcf, 0x21, 0xe, 0xe0, 0x50, 0xbe, 0x91, 0x7f, 0xec, 0x2, 0x2d, 0xc3, 0x73, 0x9d, 0xb2, 0x5c, 0xa, 0xe4, 0xcb, 0x25, 0x95, 0x7b, 0x54, 0xba, 0x29, 0xc7, 0xe8, 0x6, 0xb6, 0x58, 0x77, 0x99, 0x4c, 0xa2, 0x8d, 0x63, 0xd3, 0x3d, 0x12, 0xfc, 0x6f, 0x81, 0xae, 0x40, 0xf0, 0x1e, 0x31, 0xdf, 0x86, 0x68, 0x47, 0xa9, 0x19, 0xf7, 0xd8, 0x36, 0xa5, 0x4b, 0x64, 0x8a, 0x3a, 0xd4, 0xfb, 0x15, 0xc0, 0x2e, 0x1, 0xef, 0x5f, 0xb1, 0x9e, 0x70, 0xe3, 0xd, 0x22, 0xcc, 0x7c, 0x92, 0xbd, 0x53, 0xf, 0xe1, 0xce, 0x20, 0x90, 0x7e, 0x51, 0xbf, 0x2c, 0xc2, 0xed, 0x3, 0xb3, 0x5d, 0x72, 0x9c, 0x49, 0xa7, 0x88, 0x66, 0xd6, 0x38, 0x17, 0xf9, 0x6a, 0x84, 0xab, 0x45, 0xf5, 0x1b, 0x34, 0xda, 0x83, 0x6d, 0x42, 0xac, 0x1c, 0xf2, 0xdd, 0x33, 0xa0, 0x4e, 0x61, 0x8f, 0x3f, 0xd1, 0xfe, 0x10, 0xc5, 0x2b, 0x4, 0xea, 0x5a, 0xb4, 0x9b, 0x75, 0xe6, 0x8, 0x27, 0xc9, 0x79, 0x97, 0xb8, 0x56},
- {0x0, 0xef, 0xc3, 0x2c, 0x9b, 0x74, 0x58, 0xb7, 0x2b, 0xc4, 0xe8, 0x7, 0xb0, 0x5f, 0x73, 0x9c, 0x56, 0xb9, 0x95, 0x7a, 0xcd, 0x22, 0xe, 0xe1, 0x7d, 0x92, 0xbe, 0x51, 0xe6, 0x9, 0x25, 0xca, 0xac, 0x43, 0x6f, 0x80, 0x37, 0xd8, 0xf4, 0x1b, 0x87, 0x68, 0x44, 0xab, 0x1c, 0xf3, 0xdf, 0x30, 0xfa, 0x15, 0x39, 0xd6, 0x61, 0x8e, 0xa2, 0x4d, 0xd1, 0x3e, 0x12, 0xfd, 0x4a, 0xa5, 0x89, 0x66, 0x45, 0xaa, 0x86, 0x69, 0xde, 0x31, 0x1d, 0xf2, 0x6e, 0x81, 0xad, 0x42, 0xf5, 0x1a, 0x36, 0xd9, 0x13, 0xfc, 0xd0, 0x3f, 0x88, 0x67, 0x4b, 0xa4, 0x38, 0xd7, 0xfb, 0x14, 0xa3, 0x4c, 0x60, 0x8f, 0xe9, 0x6, 0x2a, 0xc5, 0x72, 0x9d, 0xb1, 0x5e, 0xc2, 0x2d, 0x1, 0xee, 0x59, 0xb6, 0x9a, 0x75, 0xbf, 0x50, 0x7c, 0x93, 0x24, 0xcb, 0xe7, 0x8, 0x94, 0x7b, 0x57, 0xb8, 0xf, 0xe0, 0xcc, 0x23, 0x8a, 0x65, 0x49, 0xa6, 0x11, 0xfe, 0xd2, 0x3d, 0xa1, 0x4e, 0x62, 0x8d, 0x3a, 0xd5, 0xf9, 0x16, 0xdc, 0x33, 0x1f, 0xf0, 0x47, 0xa8, 0x84, 0x6b, 0xf7, 0x18, 0x34, 0xdb, 0x6c, 0x83, 0xaf, 0x40, 0x26, 0xc9, 0xe5, 0xa, 0xbd, 0x52, 0x7e, 0x91, 0xd, 0xe2, 0xce, 0x21, 0x96, 0x79, 0x55, 0xba, 0x70, 0x9f, 0xb3, 0x5c, 0xeb, 0x4, 0x28, 0xc7, 0x5b, 0xb4, 0x98, 0x77, 0xc0, 0x2f, 0x3, 0xec, 0xcf, 0x20, 0xc, 0xe3, 0x54, 0xbb, 0x97, 0x78, 0xe4, 0xb, 0x27, 0xc8, 0x7f, 0x90, 0xbc, 0x53, 0x99, 0x76, 0x5a, 0xb5, 0x2, 0xed, 0xc1, 0x2e, 0xb2, 0x5d, 0x71, 0x9e, 0x29, 0xc6, 0xea, 0x5, 0x63, 0x8c, 0xa0, 0x4f, 0xf8, 0x17, 0x3b, 0xd4, 0x48, 0xa7, 0x8b, 0x64, 0xd3, 0x3c, 0x10, 0xff, 0x35, 0xda, 0xf6, 0x19, 0xae, 0x41, 0x6d, 0x82, 0x1e, 0xf1, 0xdd, 0x32, 0x85, 0x6a, 0x46, 0xa9},
- {0x0, 0xf0, 0xfd, 0xd, 0xe7, 0x17, 0x1a, 0xea, 0xd3, 0x23, 0x2e, 0xde, 0x34, 0xc4, 0xc9, 0x39, 0xbb, 0x4b, 0x46, 0xb6, 0x5c, 0xac, 0xa1, 0x51, 0x68, 0x98, 0x95, 0x65, 0x8f, 0x7f, 0x72, 0x82, 0x6b, 0x9b, 0x96, 0x66, 0x8c, 0x7c, 0x71, 0x81, 0xb8, 0x48, 0x45, 0xb5, 0x5f, 0xaf, 0xa2, 0x52, 0xd0, 0x20, 0x2d, 0xdd, 0x37, 0xc7, 0xca, 0x3a, 0x3, 0xf3, 0xfe, 0xe, 0xe4, 0x14, 0x19, 0xe9, 0xd6, 0x26, 0x2b, 0xdb, 0x31, 0xc1, 0xcc, 0x3c, 0x5, 0xf5, 0xf8, 0x8, 0xe2, 0x12, 0x1f, 0xef, 0x6d, 0x9d, 0x90, 0x60, 0x8a, 0x7a, 0x77, 0x87, 0xbe, 0x4e, 0x43, 0xb3, 0x59, 0xa9, 0xa4, 0x54, 0xbd, 0x4d, 0x40, 0xb0, 0x5a, 0xaa, 0xa7, 0x57, 0x6e, 0x9e, 0x93, 0x63, 0x89, 0x79, 0x74, 0x84, 0x6, 0xf6, 0xfb, 0xb, 0xe1, 0x11, 0x1c, 0xec, 0xd5, 0x25, 0x28, 0xd8, 0x32, 0xc2, 0xcf, 0x3f, 0xb1, 0x41, 0x4c, 0xbc, 0x56, 0xa6, 0xab, 0x5b, 0x62, 0x92, 0x9f, 0x6f, 0x85, 0x75, 0x78, 0x88, 0xa, 0xfa, 0xf7, 0x7, 0xed, 0x1d, 0x10, 0xe0, 0xd9, 0x29, 0x24, 0xd4, 0x3e, 0xce, 0xc3, 0x33, 0xda, 0x2a, 0x27, 0xd7, 0x3d, 0xcd, 0xc0, 0x30, 0x9, 0xf9, 0xf4, 0x4, 0xee, 0x1e, 0x13, 0xe3, 0x61, 0x91, 0x9c, 0x6c, 0x86, 0x76, 0x7b, 0x8b, 0xb2, 0x42, 0x4f, 0xbf, 0x55, 0xa5, 0xa8, 0x58, 0x67, 0x97, 0x9a, 0x6a, 0x80, 0x70, 0x7d, 0x8d, 0xb4, 0x44, 0x49, 0xb9, 0x53, 0xa3, 0xae, 0x5e, 0xdc, 0x2c, 0x21, 0xd1, 0x3b, 0xcb, 0xc6, 0x36, 0xf, 0xff, 0xf2, 0x2, 0xe8, 0x18, 0x15, 0xe5, 0xc, 0xfc, 0xf1, 0x1, 0xeb, 0x1b, 0x16, 0xe6, 0xdf, 0x2f, 0x22, 0xd2, 0x38, 0xc8, 0xc5, 0x35, 0xb7, 0x47, 0x4a, 0xba, 0x50, 0xa0, 0xad, 0x5d, 0x64, 0x94, 0x99, 0x69, 0x83, 0x73, 0x7e, 0x8e},
- {0x0, 0xf1, 0xff, 0xe, 0xe3, 0x12, 0x1c, 0xed, 0xdb, 0x2a, 0x24, 0xd5, 0x38, 0xc9, 0xc7, 0x36, 0xab, 0x5a, 0x54, 0xa5, 0x48, 0xb9, 0xb7, 0x46, 0x70, 0x81, 0x8f, 0x7e, 0x93, 0x62, 0x6c, 0x9d, 0x4b, 0xba, 0xb4, 0x45, 0xa8, 0x59, 0x57, 0xa6, 0x90, 0x61, 0x6f, 0x9e, 0x73, 0x82, 0x8c, 0x7d, 0xe0, 0x11, 0x1f, 0xee, 0x3, 0xf2, 0xfc, 0xd, 0x3b, 0xca, 0xc4, 0x35, 0xd8, 0x29, 0x27, 0xd6, 0x96, 0x67, 0x69, 0x98, 0x75, 0x84, 0x8a, 0x7b, 0x4d, 0xbc, 0xb2, 0x43, 0xae, 0x5f, 0x51, 0xa0, 0x3d, 0xcc, 0xc2, 0x33, 0xde, 0x2f, 0x21, 0xd0, 0xe6, 0x17, 0x19, 0xe8, 0x5, 0xf4, 0xfa, 0xb, 0xdd, 0x2c, 0x22, 0xd3, 0x3e, 0xcf, 0xc1, 0x30, 0x6, 0xf7, 0xf9, 0x8, 0xe5, 0x14, 0x1a, 0xeb, 0x76, 0x87, 0x89, 0x78, 0x95, 0x64, 0x6a, 0x9b, 0xad, 0x5c, 0x52, 0xa3, 0x4e, 0xbf, 0xb1, 0x40, 0x31, 0xc0, 0xce, 0x3f, 0xd2, 0x23, 0x2d, 0xdc, 0xea, 0x1b, 0x15, 0xe4, 0x9, 0xf8, 0xf6, 0x7, 0x9a, 0x6b, 0x65, 0x94, 0x79, 0x88, 0x86, 0x77, 0x41, 0xb0, 0xbe, 0x4f, 0xa2, 0x53, 0x5d, 0xac, 0x7a, 0x8b, 0x85, 0x74, 0x99, 0x68, 0x66, 0x97, 0xa1, 0x50, 0x5e, 0xaf, 0x42, 0xb3, 0xbd, 0x4c, 0xd1, 0x20, 0x2e, 0xdf, 0x32, 0xc3, 0xcd, 0x3c, 0xa, 0xfb, 0xf5, 0x4, 0xe9, 0x18, 0x16, 0xe7, 0xa7, 0x56, 0x58, 0xa9, 0x44, 0xb5, 0xbb, 0x4a, 0x7c, 0x8d, 0x83, 0x72, 0x9f, 0x6e, 0x60, 0x91, 0xc, 0xfd, 0xf3, 0x2, 0xef, 0x1e, 0x10, 0xe1, 0xd7, 0x26, 0x28, 0xd9, 0x34, 0xc5, 0xcb, 0x3a, 0xec, 0x1d, 0x13, 0xe2, 0xf, 0xfe, 0xf0, 0x1, 0x37, 0xc6, 0xc8, 0x39, 0xd4, 0x25, 0x2b, 0xda, 0x47, 0xb6, 0xb8, 0x49, 0xa4, 0x55, 0x5b, 0xaa, 0x9c, 0x6d, 0x63, 0x92, 0x7f, 0x8e, 0x80, 0x71},
- {0x0, 0xf2, 0xf9, 0xb, 0xef, 0x1d, 0x16, 0xe4, 0xc3, 0x31, 0x3a, 0xc8, 0x2c, 0xde, 0xd5, 0x27, 0x9b, 0x69, 0x62, 0x90, 0x74, 0x86, 0x8d, 0x7f, 0x58, 0xaa, 0xa1, 0x53, 0xb7, 0x45, 0x4e, 0xbc, 0x2b, 0xd9, 0xd2, 0x20, 0xc4, 0x36, 0x3d, 0xcf, 0xe8, 0x1a, 0x11, 0xe3, 0x7, 0xf5, 0xfe, 0xc, 0xb0, 0x42, 0x49, 0xbb, 0x5f, 0xad, 0xa6, 0x54, 0x73, 0x81, 0x8a, 0x78, 0x9c, 0x6e, 0x65, 0x97, 0x56, 0xa4, 0xaf, 0x5d, 0xb9, 0x4b, 0x40, 0xb2, 0x95, 0x67, 0x6c, 0x9e, 0x7a, 0x88, 0x83, 0x71, 0xcd, 0x3f, 0x34, 0xc6, 0x22, 0xd0, 0xdb, 0x29, 0xe, 0xfc, 0xf7, 0x5, 0xe1, 0x13, 0x18, 0xea, 0x7d, 0x8f, 0x84, 0x76, 0x92, 0x60, 0x6b, 0x99, 0xbe, 0x4c, 0x47, 0xb5, 0x51, 0xa3, 0xa8, 0x5a, 0xe6, 0x14, 0x1f, 0xed, 0x9, 0xfb, 0xf0, 0x2, 0x25, 0xd7, 0xdc, 0x2e, 0xca, 0x38, 0x33, 0xc1, 0xac, 0x5e, 0x55, 0xa7, 0x43, 0xb1, 0xba, 0x48, 0x6f, 0x9d, 0x96, 0x64, 0x80, 0x72, 0x79, 0x8b, 0x37, 0xc5, 0xce, 0x3c, 0xd8, 0x2a, 0x21, 0xd3, 0xf4, 0x6, 0xd, 0xff, 0x1b, 0xe9, 0xe2, 0x10, 0x87, 0x75, 0x7e, 0x8c, 0x68, 0x9a, 0x91, 0x63, 0x44, 0xb6, 0xbd, 0x4f, 0xab, 0x59, 0x52, 0xa0, 0x1c, 0xee, 0xe5, 0x17, 0xf3, 0x1, 0xa, 0xf8, 0xdf, 0x2d, 0x26, 0xd4, 0x30, 0xc2, 0xc9, 0x3b, 0xfa, 0x8, 0x3, 0xf1, 0x15, 0xe7, 0xec, 0x1e, 0x39, 0xcb, 0xc0, 0x32, 0xd6, 0x24, 0x2f, 0xdd, 0x61, 0x93, 0x98, 0x6a, 0x8e, 0x7c, 0x77, 0x85, 0xa2, 0x50, 0x5b, 0xa9, 0x4d, 0xbf, 0xb4, 0x46, 0xd1, 0x23, 0x28, 0xda, 0x3e, 0xcc, 0xc7, 0x35, 0x12, 0xe0, 0xeb, 0x19, 0xfd, 0xf, 0x4, 0xf6, 0x4a, 0xb8, 0xb3, 0x41, 0xa5, 0x57, 0x5c, 0xae, 0x89, 0x7b, 0x70, 0x82, 0x66, 0x94, 0x9f, 0x6d},
- {0x0, 0xf3, 0xfb, 0x8, 0xeb, 0x18, 0x10, 0xe3, 0xcb, 0x38, 0x30, 0xc3, 0x20, 0xd3, 0xdb, 0x28, 0x8b, 0x78, 0x70, 0x83, 0x60, 0x93, 0x9b, 0x68, 0x40, 0xb3, 0xbb, 0x48, 0xab, 0x58, 0x50, 0xa3, 0xb, 0xf8, 0xf0, 0x3, 0xe0, 0x13, 0x1b, 0xe8, 0xc0, 0x33, 0x3b, 0xc8, 0x2b, 0xd8, 0xd0, 0x23, 0x80, 0x73, 0x7b, 0x88, 0x6b, 0x98, 0x90, 0x63, 0x4b, 0xb8, 0xb0, 0x43, 0xa0, 0x53, 0x5b, 0xa8, 0x16, 0xe5, 0xed, 0x1e, 0xfd, 0xe, 0x6, 0xf5, 0xdd, 0x2e, 0x26, 0xd5, 0x36, 0xc5, 0xcd, 0x3e, 0x9d, 0x6e, 0x66, 0x95, 0x76, 0x85, 0x8d, 0x7e, 0x56, 0xa5, 0xad, 0x5e, 0xbd, 0x4e, 0x46, 0xb5, 0x1d, 0xee, 0xe6, 0x15, 0xf6, 0x5, 0xd, 0xfe, 0xd6, 0x25, 0x2d, 0xde, 0x3d, 0xce, 0xc6, 0x35, 0x96, 0x65, 0x6d, 0x9e, 0x7d, 0x8e, 0x86, 0x75, 0x5d, 0xae, 0xa6, 0x55, 0xb6, 0x45, 0x4d, 0xbe, 0x2c, 0xdf, 0xd7, 0x24, 0xc7, 0x34, 0x3c, 0xcf, 0xe7, 0x14, 0x1c, 0xef, 0xc, 0xff, 0xf7, 0x4, 0xa7, 0x54, 0x5c, 0xaf, 0x4c, 0xbf, 0xb7, 0x44, 0x6c, 0x9f, 0x97, 0x64, 0x87, 0x74, 0x7c, 0x8f, 0x27, 0xd4, 0xdc, 0x2f, 0xcc, 0x3f, 0x37, 0xc4, 0xec, 0x1f, 0x17, 0xe4, 0x7, 0xf4, 0xfc, 0xf, 0xac, 0x5f, 0x57, 0xa4, 0x47, 0xb4, 0xbc, 0x4f, 0x67, 0x94, 0x9c, 0x6f, 0x8c, 0x7f, 0x77, 0x84, 0x3a, 0xc9, 0xc1, 0x32, 0xd1, 0x22, 0x2a, 0xd9, 0xf1, 0x2, 0xa, 0xf9, 0x1a, 0xe9, 0xe1, 0x12, 0xb1, 0x42, 0x4a, 0xb9, 0x5a, 0xa9, 0xa1, 0x52, 0x7a, 0x89, 0x81, 0x72, 0x91, 0x62, 0x6a, 0x99, 0x31, 0xc2, 0xca, 0x39, 0xda, 0x29, 0x21, 0xd2, 0xfa, 0x9, 0x1, 0xf2, 0x11, 0xe2, 0xea, 0x19, 0xba, 0x49, 0x41, 0xb2, 0x51, 0xa2, 0xaa, 0x59, 0x71, 0x82, 0x8a, 0x79, 0x9a, 0x69, 0x61, 0x92},
- {0x0, 0xf4, 0xf5, 0x1, 0xf7, 0x3, 0x2, 0xf6, 0xf3, 0x7, 0x6, 0xf2, 0x4, 0xf0, 0xf1, 0x5, 0xfb, 0xf, 0xe, 0xfa, 0xc, 0xf8, 0xf9, 0xd, 0x8, 0xfc, 0xfd, 0x9, 0xff, 0xb, 0xa, 0xfe, 0xeb, 0x1f, 0x1e, 0xea, 0x1c, 0xe8, 0xe9, 0x1d, 0x18, 0xec, 0xed, 0x19, 0xef, 0x1b, 0x1a, 0xee, 0x10, 0xe4, 0xe5, 0x11, 0xe7, 0x13, 0x12, 0xe6, 0xe3, 0x17, 0x16, 0xe2, 0x14, 0xe0, 0xe1, 0x15, 0xcb, 0x3f, 0x3e, 0xca, 0x3c, 0xc8, 0xc9, 0x3d, 0x38, 0xcc, 0xcd, 0x39, 0xcf, 0x3b, 0x3a, 0xce, 0x30, 0xc4, 0xc5, 0x31, 0xc7, 0x33, 0x32, 0xc6, 0xc3, 0x37, 0x36, 0xc2, 0x34, 0xc0, 0xc1, 0x35, 0x20, 0xd4, 0xd5, 0x21, 0xd7, 0x23, 0x22, 0xd6, 0xd3, 0x27, 0x26, 0xd2, 0x24, 0xd0, 0xd1, 0x25, 0xdb, 0x2f, 0x2e, 0xda, 0x2c, 0xd8, 0xd9, 0x2d, 0x28, 0xdc, 0xdd, 0x29, 0xdf, 0x2b, 0x2a, 0xde, 0x8b, 0x7f, 0x7e, 0x8a, 0x7c, 0x88, 0x89, 0x7d, 0x78, 0x8c, 0x8d, 0x79, 0x8f, 0x7b, 0x7a, 0x8e, 0x70, 0x84, 0x85, 0x71, 0x87, 0x73, 0x72, 0x86, 0x83, 0x77, 0x76, 0x82, 0x74, 0x80, 0x81, 0x75, 0x60, 0x94, 0x95, 0x61, 0x97, 0x63, 0x62, 0x96, 0x93, 0x67, 0x66, 0x92, 0x64, 0x90, 0x91, 0x65, 0x9b, 0x6f, 0x6e, 0x9a, 0x6c, 0x98, 0x99, 0x6d, 0x68, 0x9c, 0x9d, 0x69, 0x9f, 0x6b, 0x6a, 0x9e, 0x40, 0xb4, 0xb5, 0x41, 0xb7, 0x43, 0x42, 0xb6, 0xb3, 0x47, 0x46, 0xb2, 0x44, 0xb0, 0xb1, 0x45, 0xbb, 0x4f, 0x4e, 0xba, 0x4c, 0xb8, 0xb9, 0x4d, 0x48, 0xbc, 0xbd, 0x49, 0xbf, 0x4b, 0x4a, 0xbe, 0xab, 0x5f, 0x5e, 0xaa, 0x5c, 0xa8, 0xa9, 0x5d, 0x58, 0xac, 0xad, 0x59, 0xaf, 0x5b, 0x5a, 0xae, 0x50, 0xa4, 0xa5, 0x51, 0xa7, 0x53, 0x52, 0xa6, 0xa3, 0x57, 0x56, 0xa2, 0x54, 0xa0, 0xa1, 0x55},
- {0x0, 0xf5, 0xf7, 0x2, 0xf3, 0x6, 0x4, 0xf1, 0xfb, 0xe, 0xc, 0xf9, 0x8, 0xfd, 0xff, 0xa, 0xeb, 0x1e, 0x1c, 0xe9, 0x18, 0xed, 0xef, 0x1a, 0x10, 0xe5, 0xe7, 0x12, 0xe3, 0x16, 0x14, 0xe1, 0xcb, 0x3e, 0x3c, 0xc9, 0x38, 0xcd, 0xcf, 0x3a, 0x30, 0xc5, 0xc7, 0x32, 0xc3, 0x36, 0x34, 0xc1, 0x20, 0xd5, 0xd7, 0x22, 0xd3, 0x26, 0x24, 0xd1, 0xdb, 0x2e, 0x2c, 0xd9, 0x28, 0xdd, 0xdf, 0x2a, 0x8b, 0x7e, 0x7c, 0x89, 0x78, 0x8d, 0x8f, 0x7a, 0x70, 0x85, 0x87, 0x72, 0x83, 0x76, 0x74, 0x81, 0x60, 0x95, 0x97, 0x62, 0x93, 0x66, 0x64, 0x91, 0x9b, 0x6e, 0x6c, 0x99, 0x68, 0x9d, 0x9f, 0x6a, 0x40, 0xb5, 0xb7, 0x42, 0xb3, 0x46, 0x44, 0xb1, 0xbb, 0x4e, 0x4c, 0xb9, 0x48, 0xbd, 0xbf, 0x4a, 0xab, 0x5e, 0x5c, 0xa9, 0x58, 0xad, 0xaf, 0x5a, 0x50, 0xa5, 0xa7, 0x52, 0xa3, 0x56, 0x54, 0xa1, 0xb, 0xfe, 0xfc, 0x9, 0xf8, 0xd, 0xf, 0xfa, 0xf0, 0x5, 0x7, 0xf2, 0x3, 0xf6, 0xf4, 0x1, 0xe0, 0x15, 0x17, 0xe2, 0x13, 0xe6, 0xe4, 0x11, 0x1b, 0xee, 0xec, 0x19, 0xe8, 0x1d, 0x1f, 0xea, 0xc0, 0x35, 0x37, 0xc2, 0x33, 0xc6, 0xc4, 0x31, 0x3b, 0xce, 0xcc, 0x39, 0xc8, 0x3d, 0x3f, 0xca, 0x2b, 0xde, 0xdc, 0x29, 0xd8, 0x2d, 0x2f, 0xda, 0xd0, 0x25, 0x27, 0xd2, 0x23, 0xd6, 0xd4, 0x21, 0x80, 0x75, 0x77, 0x82, 0x73, 0x86, 0x84, 0x71, 0x7b, 0x8e, 0x8c, 0x79, 0x88, 0x7d, 0x7f, 0x8a, 0x6b, 0x9e, 0x9c, 0x69, 0x98, 0x6d, 0x6f, 0x9a, 0x90, 0x65, 0x67, 0x92, 0x63, 0x96, 0x94, 0x61, 0x4b, 0xbe, 0xbc, 0x49, 0xb8, 0x4d, 0x4f, 0xba, 0xb0, 0x45, 0x47, 0xb2, 0x43, 0xb6, 0xb4, 0x41, 0xa0, 0x55, 0x57, 0xa2, 0x53, 0xa6, 0xa4, 0x51, 0x5b, 0xae, 0xac, 0x59, 0xa8, 0x5d, 0x5f, 0xaa},
- {0x0, 0xf6, 0xf1, 0x7, 0xff, 0x9, 0xe, 0xf8, 0xe3, 0x15, 0x12, 0xe4, 0x1c, 0xea, 0xed, 0x1b, 0xdb, 0x2d, 0x2a, 0xdc, 0x24, 0xd2, 0xd5, 0x23, 0x38, 0xce, 0xc9, 0x3f, 0xc7, 0x31, 0x36, 0xc0, 0xab, 0x5d, 0x5a, 0xac, 0x54, 0xa2, 0xa5, 0x53, 0x48, 0xbe, 0xb9, 0x4f, 0xb7, 0x41, 0x46, 0xb0, 0x70, 0x86, 0x81, 0x77, 0x8f, 0x79, 0x7e, 0x88, 0x93, 0x65, 0x62, 0x94, 0x6c, 0x9a, 0x9d, 0x6b, 0x4b, 0xbd, 0xba, 0x4c, 0xb4, 0x42, 0x45, 0xb3, 0xa8, 0x5e, 0x59, 0xaf, 0x57, 0xa1, 0xa6, 0x50, 0x90, 0x66, 0x61, 0x97, 0x6f, 0x99, 0x9e, 0x68, 0x73, 0x85, 0x82, 0x74, 0x8c, 0x7a, 0x7d, 0x8b, 0xe0, 0x16, 0x11, 0xe7, 0x1f, 0xe9, 0xee, 0x18, 0x3, 0xf5, 0xf2, 0x4, 0xfc, 0xa, 0xd, 0xfb, 0x3b, 0xcd, 0xca, 0x3c, 0xc4, 0x32, 0x35, 0xc3, 0xd8, 0x2e, 0x29, 0xdf, 0x27, 0xd1, 0xd6, 0x20, 0x96, 0x60, 0x67, 0x91, 0x69, 0x9f, 0x98, 0x6e, 0x75, 0x83, 0x84, 0x72, 0x8a, 0x7c, 0x7b, 0x8d, 0x4d, 0xbb, 0xbc, 0x4a, 0xb2, 0x44, 0x43, 0xb5, 0xae, 0x58, 0x5f, 0xa9, 0x51, 0xa7, 0xa0, 0x56, 0x3d, 0xcb, 0xcc, 0x3a, 0xc2, 0x34, 0x33, 0xc5, 0xde, 0x28, 0x2f, 0xd9, 0x21, 0xd7, 0xd0, 0x26, 0xe6, 0x10, 0x17, 0xe1, 0x19, 0xef, 0xe8, 0x1e, 0x5, 0xf3, 0xf4, 0x2, 0xfa, 0xc, 0xb, 0xfd, 0xdd, 0x2b, 0x2c, 0xda, 0x22, 0xd4, 0xd3, 0x25, 0x3e, 0xc8, 0xcf, 0x39, 0xc1, 0x37, 0x30, 0xc6, 0x6, 0xf0, 0xf7, 0x1, 0xf9, 0xf, 0x8, 0xfe, 0xe5, 0x13, 0x14, 0xe2, 0x1a, 0xec, 0xeb, 0x1d, 0x76, 0x80, 0x87, 0x71, 0x89, 0x7f, 0x78, 0x8e, 0x95, 0x63, 0x64, 0x92, 0x6a, 0x9c, 0x9b, 0x6d, 0xad, 0x5b, 0x5c, 0xaa, 0x52, 0xa4, 0xa3, 0x55, 0x4e, 0xb8, 0xbf, 0x49, 0xb1, 0x47, 0x40, 0xb6},
- {0x0, 0xf7, 0xf3, 0x4, 0xfb, 0xc, 0x8, 0xff, 0xeb, 0x1c, 0x18, 0xef, 0x10, 0xe7, 0xe3, 0x14, 0xcb, 0x3c, 0x38, 0xcf, 0x30, 0xc7, 0xc3, 0x34, 0x20, 0xd7, 0xd3, 0x24, 0xdb, 0x2c, 0x28, 0xdf, 0x8b, 0x7c, 0x78, 0x8f, 0x70, 0x87, 0x83, 0x74, 0x60, 0x97, 0x93, 0x64, 0x9b, 0x6c, 0x68, 0x9f, 0x40, 0xb7, 0xb3, 0x44, 0xbb, 0x4c, 0x48, 0xbf, 0xab, 0x5c, 0x58, 0xaf, 0x50, 0xa7, 0xa3, 0x54, 0xb, 0xfc, 0xf8, 0xf, 0xf0, 0x7, 0x3, 0xf4, 0xe0, 0x17, 0x13, 0xe4, 0x1b, 0xec, 0xe8, 0x1f, 0xc0, 0x37, 0x33, 0xc4, 0x3b, 0xcc, 0xc8, 0x3f, 0x2b, 0xdc, 0xd8, 0x2f, 0xd0, 0x27, 0x23, 0xd4, 0x80, 0x77, 0x73, 0x84, 0x7b, 0x8c, 0x88, 0x7f, 0x6b, 0x9c, 0x98, 0x6f, 0x90, 0x67, 0x63, 0x94, 0x4b, 0xbc, 0xb8, 0x4f, 0xb0, 0x47, 0x43, 0xb4, 0xa0, 0x57, 0x53, 0xa4, 0x5b, 0xac, 0xa8, 0x5f, 0x16, 0xe1, 0xe5, 0x12, 0xed, 0x1a, 0x1e, 0xe9, 0xfd, 0xa, 0xe, 0xf9, 0x6, 0xf1, 0xf5, 0x2, 0xdd, 0x2a, 0x2e, 0xd9, 0x26, 0xd1, 0xd5, 0x22, 0x36, 0xc1, 0xc5, 0x32, 0xcd, 0x3a, 0x3e, 0xc9, 0x9d, 0x6a, 0x6e, 0x99, 0x66, 0x91, 0x95, 0x62, 0x76, 0x81, 0x85, 0x72, 0x8d, 0x7a, 0x7e, 0x89, 0x56, 0xa1, 0xa5, 0x52, 0xad, 0x5a, 0x5e, 0xa9, 0xbd, 0x4a, 0x4e, 0xb9, 0x46, 0xb1, 0xb5, 0x42, 0x1d, 0xea, 0xee, 0x19, 0xe6, 0x11, 0x15, 0xe2, 0xf6, 0x1, 0x5, 0xf2, 0xd, 0xfa, 0xfe, 0x9, 0xd6, 0x21, 0x25, 0xd2, 0x2d, 0xda, 0xde, 0x29, 0x3d, 0xca, 0xce, 0x39, 0xc6, 0x31, 0x35, 0xc2, 0x96, 0x61, 0x65, 0x92, 0x6d, 0x9a, 0x9e, 0x69, 0x7d, 0x8a, 0x8e, 0x79, 0x86, 0x71, 0x75, 0x82, 0x5d, 0xaa, 0xae, 0x59, 0xa6, 0x51, 0x55, 0xa2, 0xb6, 0x41, 0x45, 0xb2, 0x4d, 0xba, 0xbe, 0x49},
- {0x0, 0xf8, 0xed, 0x15, 0xc7, 0x3f, 0x2a, 0xd2, 0x93, 0x6b, 0x7e, 0x86, 0x54, 0xac, 0xb9, 0x41, 0x3b, 0xc3, 0xd6, 0x2e, 0xfc, 0x4, 0x11, 0xe9, 0xa8, 0x50, 0x45, 0xbd, 0x6f, 0x97, 0x82, 0x7a, 0x76, 0x8e, 0x9b, 0x63, 0xb1, 0x49, 0x5c, 0xa4, 0xe5, 0x1d, 0x8, 0xf0, 0x22, 0xda, 0xcf, 0x37, 0x4d, 0xb5, 0xa0, 0x58, 0x8a, 0x72, 0x67, 0x9f, 0xde, 0x26, 0x33, 0xcb, 0x19, 0xe1, 0xf4, 0xc, 0xec, 0x14, 0x1, 0xf9, 0x2b, 0xd3, 0xc6, 0x3e, 0x7f, 0x87, 0x92, 0x6a, 0xb8, 0x40, 0x55, 0xad, 0xd7, 0x2f, 0x3a, 0xc2, 0x10, 0xe8, 0xfd, 0x5, 0x44, 0xbc, 0xa9, 0x51, 0x83, 0x7b, 0x6e, 0x96, 0x9a, 0x62, 0x77, 0x8f, 0x5d, 0xa5, 0xb0, 0x48, 0x9, 0xf1, 0xe4, 0x1c, 0xce, 0x36, 0x23, 0xdb, 0xa1, 0x59, 0x4c, 0xb4, 0x66, 0x9e, 0x8b, 0x73, 0x32, 0xca, 0xdf, 0x27, 0xf5, 0xd, 0x18, 0xe0, 0xc5, 0x3d, 0x28, 0xd0, 0x2, 0xfa, 0xef, 0x17, 0x56, 0xae, 0xbb, 0x43, 0x91, 0x69, 0x7c, 0x84, 0xfe, 0x6, 0x13, 0xeb, 0x39, 0xc1, 0xd4, 0x2c, 0x6d, 0x95, 0x80, 0x78, 0xaa, 0x52, 0x47, 0xbf, 0xb3, 0x4b, 0x5e, 0xa6, 0x74, 0x8c, 0x99, 0x61, 0x20, 0xd8, 0xcd, 0x35, 0xe7, 0x1f, 0xa, 0xf2, 0x88, 0x70, 0x65, 0x9d, 0x4f, 0xb7, 0xa2, 0x5a, 0x1b, 0xe3, 0xf6, 0xe, 0xdc, 0x24, 0x31, 0xc9, 0x29, 0xd1, 0xc4, 0x3c, 0xee, 0x16, 0x3, 0xfb, 0xba, 0x42, 0x57, 0xaf, 0x7d, 0x85, 0x90, 0x68, 0x12, 0xea, 0xff, 0x7, 0xd5, 0x2d, 0x38, 0xc0, 0x81, 0x79, 0x6c, 0x94, 0x46, 0xbe, 0xab, 0x53, 0x5f, 0xa7, 0xb2, 0x4a, 0x98, 0x60, 0x75, 0x8d, 0xcc, 0x34, 0x21, 0xd9, 0xb, 0xf3, 0xe6, 0x1e, 0x64, 0x9c, 0x89, 0x71, 0xa3, 0x5b, 0x4e, 0xb6, 0xf7, 0xf, 0x1a, 0xe2, 0x30, 0xc8, 0xdd, 0x25},
- {0x0, 0xf9, 0xef, 0x16, 0xc3, 0x3a, 0x2c, 0xd5, 0x9b, 0x62, 0x74, 0x8d, 0x58, 0xa1, 0xb7, 0x4e, 0x2b, 0xd2, 0xc4, 0x3d, 0xe8, 0x11, 0x7, 0xfe, 0xb0, 0x49, 0x5f, 0xa6, 0x73, 0x8a, 0x9c, 0x65, 0x56, 0xaf, 0xb9, 0x40, 0x95, 0x6c, 0x7a, 0x83, 0xcd, 0x34, 0x22, 0xdb, 0xe, 0xf7, 0xe1, 0x18, 0x7d, 0x84, 0x92, 0x6b, 0xbe, 0x47, 0x51, 0xa8, 0xe6, 0x1f, 0x9, 0xf0, 0x25, 0xdc, 0xca, 0x33, 0xac, 0x55, 0x43, 0xba, 0x6f, 0x96, 0x80, 0x79, 0x37, 0xce, 0xd8, 0x21, 0xf4, 0xd, 0x1b, 0xe2, 0x87, 0x7e, 0x68, 0x91, 0x44, 0xbd, 0xab, 0x52, 0x1c, 0xe5, 0xf3, 0xa, 0xdf, 0x26, 0x30, 0xc9, 0xfa, 0x3, 0x15, 0xec, 0x39, 0xc0, 0xd6, 0x2f, 0x61, 0x98, 0x8e, 0x77, 0xa2, 0x5b, 0x4d, 0xb4, 0xd1, 0x28, 0x3e, 0xc7, 0x12, 0xeb, 0xfd, 0x4, 0x4a, 0xb3, 0xa5, 0x5c, 0x89, 0x70, 0x66, 0x9f, 0x45, 0xbc, 0xaa, 0x53, 0x86, 0x7f, 0x69, 0x90, 0xde, 0x27, 0x31, 0xc8, 0x1d, 0xe4, 0xf2, 0xb, 0x6e, 0x97, 0x81, 0x78, 0xad, 0x54, 0x42, 0xbb, 0xf5, 0xc, 0x1a, 0xe3, 0x36, 0xcf, 0xd9, 0x20, 0x13, 0xea, 0xfc, 0x5, 0xd0, 0x29, 0x3f, 0xc6, 0x88, 0x71, 0x67, 0x9e, 0x4b, 0xb2, 0xa4, 0x5d, 0x38, 0xc1, 0xd7, 0x2e, 0xfb, 0x2, 0x14, 0xed, 0xa3, 0x5a, 0x4c, 0xb5, 0x60, 0x99, 0x8f, 0x76, 0xe9, 0x10, 0x6, 0xff, 0x2a, 0xd3, 0xc5, 0x3c, 0x72, 0x8b, 0x9d, 0x64, 0xb1, 0x48, 0x5e, 0xa7, 0xc2, 0x3b, 0x2d, 0xd4, 0x1, 0xf8, 0xee, 0x17, 0x59, 0xa0, 0xb6, 0x4f, 0x9a, 0x63, 0x75, 0x8c, 0xbf, 0x46, 0x50, 0xa9, 0x7c, 0x85, 0x93, 0x6a, 0x24, 0xdd, 0xcb, 0x32, 0xe7, 0x1e, 0x8, 0xf1, 0x94, 0x6d, 0x7b, 0x82, 0x57, 0xae, 0xb8, 0x41, 0xf, 0xf6, 0xe0, 0x19, 0xcc, 0x35, 0x23, 0xda},
- {0x0, 0xfa, 0xe9, 0x13, 0xcf, 0x35, 0x26, 0xdc, 0x83, 0x79, 0x6a, 0x90, 0x4c, 0xb6, 0xa5, 0x5f, 0x1b, 0xe1, 0xf2, 0x8, 0xd4, 0x2e, 0x3d, 0xc7, 0x98, 0x62, 0x71, 0x8b, 0x57, 0xad, 0xbe, 0x44, 0x36, 0xcc, 0xdf, 0x25, 0xf9, 0x3, 0x10, 0xea, 0xb5, 0x4f, 0x5c, 0xa6, 0x7a, 0x80, 0x93, 0x69, 0x2d, 0xd7, 0xc4, 0x3e, 0xe2, 0x18, 0xb, 0xf1, 0xae, 0x54, 0x47, 0xbd, 0x61, 0x9b, 0x88, 0x72, 0x6c, 0x96, 0x85, 0x7f, 0xa3, 0x59, 0x4a, 0xb0, 0xef, 0x15, 0x6, 0xfc, 0x20, 0xda, 0xc9, 0x33, 0x77, 0x8d, 0x9e, 0x64, 0xb8, 0x42, 0x51, 0xab, 0xf4, 0xe, 0x1d, 0xe7, 0x3b, 0xc1, 0xd2, 0x28, 0x5a, 0xa0, 0xb3, 0x49, 0x95, 0x6f, 0x7c, 0x86, 0xd9, 0x23, 0x30, 0xca, 0x16, 0xec, 0xff, 0x5, 0x41, 0xbb, 0xa8, 0x52, 0x8e, 0x74, 0x67, 0x9d, 0xc2, 0x38, 0x2b, 0xd1, 0xd, 0xf7, 0xe4, 0x1e, 0xd8, 0x22, 0x31, 0xcb, 0x17, 0xed, 0xfe, 0x4, 0x5b, 0xa1, 0xb2, 0x48, 0x94, 0x6e, 0x7d, 0x87, 0xc3, 0x39, 0x2a, 0xd0, 0xc, 0xf6, 0xe5, 0x1f, 0x40, 0xba, 0xa9, 0x53, 0x8f, 0x75, 0x66, 0x9c, 0xee, 0x14, 0x7, 0xfd, 0x21, 0xdb, 0xc8, 0x32, 0x6d, 0x97, 0x84, 0x7e, 0xa2, 0x58, 0x4b, 0xb1, 0xf5, 0xf, 0x1c, 0xe6, 0x3a, 0xc0, 0xd3, 0x29, 0x76, 0x8c, 0x9f, 0x65, 0xb9, 0x43, 0x50, 0xaa, 0xb4, 0x4e, 0x5d, 0xa7, 0x7b, 0x81, 0x92, 0x68, 0x37, 0xcd, 0xde, 0x24, 0xf8, 0x2, 0x11, 0xeb, 0xaf, 0x55, 0x46, 0xbc, 0x60, 0x9a, 0x89, 0x73, 0x2c, 0xd6, 0xc5, 0x3f, 0xe3, 0x19, 0xa, 0xf0, 0x82, 0x78, 0x6b, 0x91, 0x4d, 0xb7, 0xa4, 0x5e, 0x1, 0xfb, 0xe8, 0x12, 0xce, 0x34, 0x27, 0xdd, 0x99, 0x63, 0x70, 0x8a, 0x56, 0xac, 0xbf, 0x45, 0x1a, 0xe0, 0xf3, 0x9, 0xd5, 0x2f, 0x3c, 0xc6},
- {0x0, 0xfb, 0xeb, 0x10, 0xcb, 0x30, 0x20, 0xdb, 0x8b, 0x70, 0x60, 0x9b, 0x40, 0xbb, 0xab, 0x50, 0xb, 0xf0, 0xe0, 0x1b, 0xc0, 0x3b, 0x2b, 0xd0, 0x80, 0x7b, 0x6b, 0x90, 0x4b, 0xb0, 0xa0, 0x5b, 0x16, 0xed, 0xfd, 0x6, 0xdd, 0x26, 0x36, 0xcd, 0x9d, 0x66, 0x76, 0x8d, 0x56, 0xad, 0xbd, 0x46, 0x1d, 0xe6, 0xf6, 0xd, 0xd6, 0x2d, 0x3d, 0xc6, 0x96, 0x6d, 0x7d, 0x86, 0x5d, 0xa6, 0xb6, 0x4d, 0x2c, 0xd7, 0xc7, 0x3c, 0xe7, 0x1c, 0xc, 0xf7, 0xa7, 0x5c, 0x4c, 0xb7, 0x6c, 0x97, 0x87, 0x7c, 0x27, 0xdc, 0xcc, 0x37, 0xec, 0x17, 0x7, 0xfc, 0xac, 0x57, 0x47, 0xbc, 0x67, 0x9c, 0x8c, 0x77, 0x3a, 0xc1, 0xd1, 0x2a, 0xf1, 0xa, 0x1a, 0xe1, 0xb1, 0x4a, 0x5a, 0xa1, 0x7a, 0x81, 0x91, 0x6a, 0x31, 0xca, 0xda, 0x21, 0xfa, 0x1, 0x11, 0xea, 0xba, 0x41, 0x51, 0xaa, 0x71, 0x8a, 0x9a, 0x61, 0x58, 0xa3, 0xb3, 0x48, 0x93, 0x68, 0x78, 0x83, 0xd3, 0x28, 0x38, 0xc3, 0x18, 0xe3, 0xf3, 0x8, 0x53, 0xa8, 0xb8, 0x43, 0x98, 0x63, 0x73, 0x88, 0xd8, 0x23, 0x33, 0xc8, 0x13, 0xe8, 0xf8, 0x3, 0x4e, 0xb5, 0xa5, 0x5e, 0x85, 0x7e, 0x6e, 0x95, 0xc5, 0x3e, 0x2e, 0xd5, 0xe, 0xf5, 0xe5, 0x1e, 0x45, 0xbe, 0xae, 0x55, 0x8e, 0x75, 0x65, 0x9e, 0xce, 0x35, 0x25, 0xde, 0x5, 0xfe, 0xee, 0x15, 0x74, 0x8f, 0x9f, 0x64, 0xbf, 0x44, 0x54, 0xaf, 0xff, 0x4, 0x14, 0xef, 0x34, 0xcf, 0xdf, 0x24, 0x7f, 0x84, 0x94, 0x6f, 0xb4, 0x4f, 0x5f, 0xa4, 0xf4, 0xf, 0x1f, 0xe4, 0x3f, 0xc4, 0xd4, 0x2f, 0x62, 0x99, 0x89, 0x72, 0xa9, 0x52, 0x42, 0xb9, 0xe9, 0x12, 0x2, 0xf9, 0x22, 0xd9, 0xc9, 0x32, 0x69, 0x92, 0x82, 0x79, 0xa2, 0x59, 0x49, 0xb2, 0xe2, 0x19, 0x9, 0xf2, 0x29, 0xd2, 0xc2, 0x39},
- {0x0, 0xfc, 0xe5, 0x19, 0xd7, 0x2b, 0x32, 0xce, 0xb3, 0x4f, 0x56, 0xaa, 0x64, 0x98, 0x81, 0x7d, 0x7b, 0x87, 0x9e, 0x62, 0xac, 0x50, 0x49, 0xb5, 0xc8, 0x34, 0x2d, 0xd1, 0x1f, 0xe3, 0xfa, 0x6, 0xf6, 0xa, 0x13, 0xef, 0x21, 0xdd, 0xc4, 0x38, 0x45, 0xb9, 0xa0, 0x5c, 0x92, 0x6e, 0x77, 0x8b, 0x8d, 0x71, 0x68, 0x94, 0x5a, 0xa6, 0xbf, 0x43, 0x3e, 0xc2, 0xdb, 0x27, 0xe9, 0x15, 0xc, 0xf0, 0xf1, 0xd, 0x14, 0xe8, 0x26, 0xda, 0xc3, 0x3f, 0x42, 0xbe, 0xa7, 0x5b, 0x95, 0x69, 0x70, 0x8c, 0x8a, 0x76, 0x6f, 0x93, 0x5d, 0xa1, 0xb8, 0x44, 0x39, 0xc5, 0xdc, 0x20, 0xee, 0x12, 0xb, 0xf7, 0x7, 0xfb, 0xe2, 0x1e, 0xd0, 0x2c, 0x35, 0xc9, 0xb4, 0x48, 0x51, 0xad, 0x63, 0x9f, 0x86, 0x7a, 0x7c, 0x80, 0x99, 0x65, 0xab, 0x57, 0x4e, 0xb2, 0xcf, 0x33, 0x2a, 0xd6, 0x18, 0xe4, 0xfd, 0x1, 0xff, 0x3, 0x1a, 0xe6, 0x28, 0xd4, 0xcd, 0x31, 0x4c, 0xb0, 0xa9, 0x55, 0x9b, 0x67, 0x7e, 0x82, 0x84, 0x78, 0x61, 0x9d, 0x53, 0xaf, 0xb6, 0x4a, 0x37, 0xcb, 0xd2, 0x2e, 0xe0, 0x1c, 0x5, 0xf9, 0x9, 0xf5, 0xec, 0x10, 0xde, 0x22, 0x3b, 0xc7, 0xba, 0x46, 0x5f, 0xa3, 0x6d, 0x91, 0x88, 0x74, 0x72, 0x8e, 0x97, 0x6b, 0xa5, 0x59, 0x40, 0xbc, 0xc1, 0x3d, 0x24, 0xd8, 0x16, 0xea, 0xf3, 0xf, 0xe, 0xf2, 0xeb, 0x17, 0xd9, 0x25, 0x3c, 0xc0, 0xbd, 0x41, 0x58, 0xa4, 0x6a, 0x96, 0x8f, 0x73, 0x75, 0x89, 0x90, 0x6c, 0xa2, 0x5e, 0x47, 0xbb, 0xc6, 0x3a, 0x23, 0xdf, 0x11, 0xed, 0xf4, 0x8, 0xf8, 0x4, 0x1d, 0xe1, 0x2f, 0xd3, 0xca, 0x36, 0x4b, 0xb7, 0xae, 0x52, 0x9c, 0x60, 0x79, 0x85, 0x83, 0x7f, 0x66, 0x9a, 0x54, 0xa8, 0xb1, 0x4d, 0x30, 0xcc, 0xd5, 0x29, 0xe7, 0x1b, 0x2, 0xfe},
- {0x0, 0xfd, 0xe7, 0x1a, 0xd3, 0x2e, 0x34, 0xc9, 0xbb, 0x46, 0x5c, 0xa1, 0x68, 0x95, 0x8f, 0x72, 0x6b, 0x96, 0x8c, 0x71, 0xb8, 0x45, 0x5f, 0xa2, 0xd0, 0x2d, 0x37, 0xca, 0x3, 0xfe, 0xe4, 0x19, 0xd6, 0x2b, 0x31, 0xcc, 0x5, 0xf8, 0xe2, 0x1f, 0x6d, 0x90, 0x8a, 0x77, 0xbe, 0x43, 0x59, 0xa4, 0xbd, 0x40, 0x5a, 0xa7, 0x6e, 0x93, 0x89, 0x74, 0x6, 0xfb, 0xe1, 0x1c, 0xd5, 0x28, 0x32, 0xcf, 0xb1, 0x4c, 0x56, 0xab, 0x62, 0x9f, 0x85, 0x78, 0xa, 0xf7, 0xed, 0x10, 0xd9, 0x24, 0x3e, 0xc3, 0xda, 0x27, 0x3d, 0xc0, 0x9, 0xf4, 0xee, 0x13, 0x61, 0x9c, 0x86, 0x7b, 0xb2, 0x4f, 0x55, 0xa8, 0x67, 0x9a, 0x80, 0x7d, 0xb4, 0x49, 0x53, 0xae, 0xdc, 0x21, 0x3b, 0xc6, 0xf, 0xf2, 0xe8, 0x15, 0xc, 0xf1, 0xeb, 0x16, 0xdf, 0x22, 0x38, 0xc5, 0xb7, 0x4a, 0x50, 0xad, 0x64, 0x99, 0x83, 0x7e, 0x7f, 0x82, 0x98, 0x65, 0xac, 0x51, 0x4b, 0xb6, 0xc4, 0x39, 0x23, 0xde, 0x17, 0xea, 0xf0, 0xd, 0x14, 0xe9, 0xf3, 0xe, 0xc7, 0x3a, 0x20, 0xdd, 0xaf, 0x52, 0x48, 0xb5, 0x7c, 0x81, 0x9b, 0x66, 0xa9, 0x54, 0x4e, 0xb3, 0x7a, 0x87, 0x9d, 0x60, 0x12, 0xef, 0xf5, 0x8, 0xc1, 0x3c, 0x26, 0xdb, 0xc2, 0x3f, 0x25, 0xd8, 0x11, 0xec, 0xf6, 0xb, 0x79, 0x84, 0x9e, 0x63, 0xaa, 0x57, 0x4d, 0xb0, 0xce, 0x33, 0x29, 0xd4, 0x1d, 0xe0, 0xfa, 0x7, 0x75, 0x88, 0x92, 0x6f, 0xa6, 0x5b, 0x41, 0xbc, 0xa5, 0x58, 0x42, 0xbf, 0x76, 0x8b, 0x91, 0x6c, 0x1e, 0xe3, 0xf9, 0x4, 0xcd, 0x30, 0x2a, 0xd7, 0x18, 0xe5, 0xff, 0x2, 0xcb, 0x36, 0x2c, 0xd1, 0xa3, 0x5e, 0x44, 0xb9, 0x70, 0x8d, 0x97, 0x6a, 0x73, 0x8e, 0x94, 0x69, 0xa0, 0x5d, 0x47, 0xba, 0xc8, 0x35, 0x2f, 0xd2, 0x1b, 0xe6, 0xfc, 0x1},
- {0x0, 0xfe, 0xe1, 0x1f, 0xdf, 0x21, 0x3e, 0xc0, 0xa3, 0x5d, 0x42, 0xbc, 0x7c, 0x82, 0x9d, 0x63, 0x5b, 0xa5, 0xba, 0x44, 0x84, 0x7a, 0x65, 0x9b, 0xf8, 0x6, 0x19, 0xe7, 0x27, 0xd9, 0xc6, 0x38, 0xb6, 0x48, 0x57, 0xa9, 0x69, 0x97, 0x88, 0x76, 0x15, 0xeb, 0xf4, 0xa, 0xca, 0x34, 0x2b, 0xd5, 0xed, 0x13, 0xc, 0xf2, 0x32, 0xcc, 0xd3, 0x2d, 0x4e, 0xb0, 0xaf, 0x51, 0x91, 0x6f, 0x70, 0x8e, 0x71, 0x8f, 0x90, 0x6e, 0xae, 0x50, 0x4f, 0xb1, 0xd2, 0x2c, 0x33, 0xcd, 0xd, 0xf3, 0xec, 0x12, 0x2a, 0xd4, 0xcb, 0x35, 0xf5, 0xb, 0x14, 0xea, 0x89, 0x77, 0x68, 0x96, 0x56, 0xa8, 0xb7, 0x49, 0xc7, 0x39, 0x26, 0xd8, 0x18, 0xe6, 0xf9, 0x7, 0x64, 0x9a, 0x85, 0x7b, 0xbb, 0x45, 0x5a, 0xa4, 0x9c, 0x62, 0x7d, 0x83, 0x43, 0xbd, 0xa2, 0x5c, 0x3f, 0xc1, 0xde, 0x20, 0xe0, 0x1e, 0x1, 0xff, 0xe2, 0x1c, 0x3, 0xfd, 0x3d, 0xc3, 0xdc, 0x22, 0x41, 0xbf, 0xa0, 0x5e, 0x9e, 0x60, 0x7f, 0x81, 0xb9, 0x47, 0x58, 0xa6, 0x66, 0x98, 0x87, 0x79, 0x1a, 0xe4, 0xfb, 0x5, 0xc5, 0x3b, 0x24, 0xda, 0x54, 0xaa, 0xb5, 0x4b, 0x8b, 0x75, 0x6a, 0x94, 0xf7, 0x9, 0x16, 0xe8, 0x28, 0xd6, 0xc9, 0x37, 0xf, 0xf1, 0xee, 0x10, 0xd0, 0x2e, 0x31, 0xcf, 0xac, 0x52, 0x4d, 0xb3, 0x73, 0x8d, 0x92, 0x6c, 0x93, 0x6d, 0x72, 0x8c, 0x4c, 0xb2, 0xad, 0x53, 0x30, 0xce, 0xd1, 0x2f, 0xef, 0x11, 0xe, 0xf0, 0xc8, 0x36, 0x29, 0xd7, 0x17, 0xe9, 0xf6, 0x8, 0x6b, 0x95, 0x8a, 0x74, 0xb4, 0x4a, 0x55, 0xab, 0x25, 0xdb, 0xc4, 0x3a, 0xfa, 0x4, 0x1b, 0xe5, 0x86, 0x78, 0x67, 0x99, 0x59, 0xa7, 0xb8, 0x46, 0x7e, 0x80, 0x9f, 0x61, 0xa1, 0x5f, 0x40, 0xbe, 0xdd, 0x23, 0x3c, 0xc2, 0x2, 0xfc, 0xe3, 0x1d},
- {0x0, 0xff, 0xe3, 0x1c, 0xdb, 0x24, 0x38, 0xc7, 0xab, 0x54, 0x48, 0xb7, 0x70, 0x8f, 0x93, 0x6c, 0x4b, 0xb4, 0xa8, 0x57, 0x90, 0x6f, 0x73, 0x8c, 0xe0, 0x1f, 0x3, 0xfc, 0x3b, 0xc4, 0xd8, 0x27, 0x96, 0x69, 0x75, 0x8a, 0x4d, 0xb2, 0xae, 0x51, 0x3d, 0xc2, 0xde, 0x21, 0xe6, 0x19, 0x5, 0xfa, 0xdd, 0x22, 0x3e, 0xc1, 0x6, 0xf9, 0xe5, 0x1a, 0x76, 0x89, 0x95, 0x6a, 0xad, 0x52, 0x4e, 0xb1, 0x31, 0xce, 0xd2, 0x2d, 0xea, 0x15, 0x9, 0xf6, 0x9a, 0x65, 0x79, 0x86, 0x41, 0xbe, 0xa2, 0x5d, 0x7a, 0x85, 0x99, 0x66, 0xa1, 0x5e, 0x42, 0xbd, 0xd1, 0x2e, 0x32, 0xcd, 0xa, 0xf5, 0xe9, 0x16, 0xa7, 0x58, 0x44, 0xbb, 0x7c, 0x83, 0x9f, 0x60, 0xc, 0xf3, 0xef, 0x10, 0xd7, 0x28, 0x34, 0xcb, 0xec, 0x13, 0xf, 0xf0, 0x37, 0xc8, 0xd4, 0x2b, 0x47, 0xb8, 0xa4, 0x5b, 0x9c, 0x63, 0x7f, 0x80, 0x62, 0x9d, 0x81, 0x7e, 0xb9, 0x46, 0x5a, 0xa5, 0xc9, 0x36, 0x2a, 0xd5, 0x12, 0xed, 0xf1, 0xe, 0x29, 0xd6, 0xca, 0x35, 0xf2, 0xd, 0x11, 0xee, 0x82, 0x7d, 0x61, 0x9e, 0x59, 0xa6, 0xba, 0x45, 0xf4, 0xb, 0x17, 0xe8, 0x2f, 0xd0, 0xcc, 0x33, 0x5f, 0xa0, 0xbc, 0x43, 0x84, 0x7b, 0x67, 0x98, 0xbf, 0x40, 0x5c, 0xa3, 0x64, 0x9b, 0x87, 0x78, 0x14, 0xeb, 0xf7, 0x8, 0xcf, 0x30, 0x2c, 0xd3, 0x53, 0xac, 0xb0, 0x4f, 0x88, 0x77, 0x6b, 0x94, 0xf8, 0x7, 0x1b, 0xe4, 0x23, 0xdc, 0xc0, 0x3f, 0x18, 0xe7, 0xfb, 0x4, 0xc3, 0x3c, 0x20, 0xdf, 0xb3, 0x4c, 0x50, 0xaf, 0x68, 0x97, 0x8b, 0x74, 0xc5, 0x3a, 0x26, 0xd9, 0x1e, 0xe1, 0xfd, 0x2, 0x6e, 0x91, 0x8d, 0x72, 0xb5, 0x4a, 0x56, 0xa9, 0x8e, 0x71, 0x6d, 0x92, 0x55, 0xaa, 0xb6, 0x49, 0x25, 0xda, 0xc6, 0x39, 0xfe, 0x1, 0x1d, 0xe2}}
-
-var mulTableLow = [256][16]uint8{{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
- {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf},
- {0x0, 0x2, 0x4, 0x6, 0x8, 0xa, 0xc, 0xe, 0x10, 0x12, 0x14, 0x16, 0x18, 0x1a, 0x1c, 0x1e},
- {0x0, 0x3, 0x6, 0x5, 0xc, 0xf, 0xa, 0x9, 0x18, 0x1b, 0x1e, 0x1d, 0x14, 0x17, 0x12, 0x11},
- {0x0, 0x4, 0x8, 0xc, 0x10, 0x14, 0x18, 0x1c, 0x20, 0x24, 0x28, 0x2c, 0x30, 0x34, 0x38, 0x3c},
- {0x0, 0x5, 0xa, 0xf, 0x14, 0x11, 0x1e, 0x1b, 0x28, 0x2d, 0x22, 0x27, 0x3c, 0x39, 0x36, 0x33},
- {0x0, 0x6, 0xc, 0xa, 0x18, 0x1e, 0x14, 0x12, 0x30, 0x36, 0x3c, 0x3a, 0x28, 0x2e, 0x24, 0x22},
- {0x0, 0x7, 0xe, 0x9, 0x1c, 0x1b, 0x12, 0x15, 0x38, 0x3f, 0x36, 0x31, 0x24, 0x23, 0x2a, 0x2d},
- {0x0, 0x8, 0x10, 0x18, 0x20, 0x28, 0x30, 0x38, 0x40, 0x48, 0x50, 0x58, 0x60, 0x68, 0x70, 0x78},
- {0x0, 0x9, 0x12, 0x1b, 0x24, 0x2d, 0x36, 0x3f, 0x48, 0x41, 0x5a, 0x53, 0x6c, 0x65, 0x7e, 0x77},
- {0x0, 0xa, 0x14, 0x1e, 0x28, 0x22, 0x3c, 0x36, 0x50, 0x5a, 0x44, 0x4e, 0x78, 0x72, 0x6c, 0x66},
- {0x0, 0xb, 0x16, 0x1d, 0x2c, 0x27, 0x3a, 0x31, 0x58, 0x53, 0x4e, 0x45, 0x74, 0x7f, 0x62, 0x69},
- {0x0, 0xc, 0x18, 0x14, 0x30, 0x3c, 0x28, 0x24, 0x60, 0x6c, 0x78, 0x74, 0x50, 0x5c, 0x48, 0x44},
- {0x0, 0xd, 0x1a, 0x17, 0x34, 0x39, 0x2e, 0x23, 0x68, 0x65, 0x72, 0x7f, 0x5c, 0x51, 0x46, 0x4b},
- {0x0, 0xe, 0x1c, 0x12, 0x38, 0x36, 0x24, 0x2a, 0x70, 0x7e, 0x6c, 0x62, 0x48, 0x46, 0x54, 0x5a},
- {0x0, 0xf, 0x1e, 0x11, 0x3c, 0x33, 0x22, 0x2d, 0x78, 0x77, 0x66, 0x69, 0x44, 0x4b, 0x5a, 0x55},
- {0x0, 0x10, 0x20, 0x30, 0x40, 0x50, 0x60, 0x70, 0x80, 0x90, 0xa0, 0xb0, 0xc0, 0xd0, 0xe0, 0xf0},
- {0x0, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff},
- {0x0, 0x12, 0x24, 0x36, 0x48, 0x5a, 0x6c, 0x7e, 0x90, 0x82, 0xb4, 0xa6, 0xd8, 0xca, 0xfc, 0xee},
- {0x0, 0x13, 0x26, 0x35, 0x4c, 0x5f, 0x6a, 0x79, 0x98, 0x8b, 0xbe, 0xad, 0xd4, 0xc7, 0xf2, 0xe1},
- {0x0, 0x14, 0x28, 0x3c, 0x50, 0x44, 0x78, 0x6c, 0xa0, 0xb4, 0x88, 0x9c, 0xf0, 0xe4, 0xd8, 0xcc},
- {0x0, 0x15, 0x2a, 0x3f, 0x54, 0x41, 0x7e, 0x6b, 0xa8, 0xbd, 0x82, 0x97, 0xfc, 0xe9, 0xd6, 0xc3},
- {0x0, 0x16, 0x2c, 0x3a, 0x58, 0x4e, 0x74, 0x62, 0xb0, 0xa6, 0x9c, 0x8a, 0xe8, 0xfe, 0xc4, 0xd2},
- {0x0, 0x17, 0x2e, 0x39, 0x5c, 0x4b, 0x72, 0x65, 0xb8, 0xaf, 0x96, 0x81, 0xe4, 0xf3, 0xca, 0xdd},
- {0x0, 0x18, 0x30, 0x28, 0x60, 0x78, 0x50, 0x48, 0xc0, 0xd8, 0xf0, 0xe8, 0xa0, 0xb8, 0x90, 0x88},
- {0x0, 0x19, 0x32, 0x2b, 0x64, 0x7d, 0x56, 0x4f, 0xc8, 0xd1, 0xfa, 0xe3, 0xac, 0xb5, 0x9e, 0x87},
- {0x0, 0x1a, 0x34, 0x2e, 0x68, 0x72, 0x5c, 0x46, 0xd0, 0xca, 0xe4, 0xfe, 0xb8, 0xa2, 0x8c, 0x96},
- {0x0, 0x1b, 0x36, 0x2d, 0x6c, 0x77, 0x5a, 0x41, 0xd8, 0xc3, 0xee, 0xf5, 0xb4, 0xaf, 0x82, 0x99},
- {0x0, 0x1c, 0x38, 0x24, 0x70, 0x6c, 0x48, 0x54, 0xe0, 0xfc, 0xd8, 0xc4, 0x90, 0x8c, 0xa8, 0xb4},
- {0x0, 0x1d, 0x3a, 0x27, 0x74, 0x69, 0x4e, 0x53, 0xe8, 0xf5, 0xd2, 0xcf, 0x9c, 0x81, 0xa6, 0xbb},
- {0x0, 0x1e, 0x3c, 0x22, 0x78, 0x66, 0x44, 0x5a, 0xf0, 0xee, 0xcc, 0xd2, 0x88, 0x96, 0xb4, 0xaa},
- {0x0, 0x1f, 0x3e, 0x21, 0x7c, 0x63, 0x42, 0x5d, 0xf8, 0xe7, 0xc6, 0xd9, 0x84, 0x9b, 0xba, 0xa5},
- {0x0, 0x20, 0x40, 0x60, 0x80, 0xa0, 0xc0, 0xe0, 0x1d, 0x3d, 0x5d, 0x7d, 0x9d, 0xbd, 0xdd, 0xfd},
- {0x0, 0x21, 0x42, 0x63, 0x84, 0xa5, 0xc6, 0xe7, 0x15, 0x34, 0x57, 0x76, 0x91, 0xb0, 0xd3, 0xf2},
- {0x0, 0x22, 0x44, 0x66, 0x88, 0xaa, 0xcc, 0xee, 0xd, 0x2f, 0x49, 0x6b, 0x85, 0xa7, 0xc1, 0xe3},
- {0x0, 0x23, 0x46, 0x65, 0x8c, 0xaf, 0xca, 0xe9, 0x5, 0x26, 0x43, 0x60, 0x89, 0xaa, 0xcf, 0xec},
- {0x0, 0x24, 0x48, 0x6c, 0x90, 0xb4, 0xd8, 0xfc, 0x3d, 0x19, 0x75, 0x51, 0xad, 0x89, 0xe5, 0xc1},
- {0x0, 0x25, 0x4a, 0x6f, 0x94, 0xb1, 0xde, 0xfb, 0x35, 0x10, 0x7f, 0x5a, 0xa1, 0x84, 0xeb, 0xce},
- {0x0, 0x26, 0x4c, 0x6a, 0x98, 0xbe, 0xd4, 0xf2, 0x2d, 0xb, 0x61, 0x47, 0xb5, 0x93, 0xf9, 0xdf},
- {0x0, 0x27, 0x4e, 0x69, 0x9c, 0xbb, 0xd2, 0xf5, 0x25, 0x2, 0x6b, 0x4c, 0xb9, 0x9e, 0xf7, 0xd0},
- {0x0, 0x28, 0x50, 0x78, 0xa0, 0x88, 0xf0, 0xd8, 0x5d, 0x75, 0xd, 0x25, 0xfd, 0xd5, 0xad, 0x85},
- {0x0, 0x29, 0x52, 0x7b, 0xa4, 0x8d, 0xf6, 0xdf, 0x55, 0x7c, 0x7, 0x2e, 0xf1, 0xd8, 0xa3, 0x8a},
- {0x0, 0x2a, 0x54, 0x7e, 0xa8, 0x82, 0xfc, 0xd6, 0x4d, 0x67, 0x19, 0x33, 0xe5, 0xcf, 0xb1, 0x9b},
- {0x0, 0x2b, 0x56, 0x7d, 0xac, 0x87, 0xfa, 0xd1, 0x45, 0x6e, 0x13, 0x38, 0xe9, 0xc2, 0xbf, 0x94},
- {0x0, 0x2c, 0x58, 0x74, 0xb0, 0x9c, 0xe8, 0xc4, 0x7d, 0x51, 0x25, 0x9, 0xcd, 0xe1, 0x95, 0xb9},
- {0x0, 0x2d, 0x5a, 0x77, 0xb4, 0x99, 0xee, 0xc3, 0x75, 0x58, 0x2f, 0x2, 0xc1, 0xec, 0x9b, 0xb6},
- {0x0, 0x2e, 0x5c, 0x72, 0xb8, 0x96, 0xe4, 0xca, 0x6d, 0x43, 0x31, 0x1f, 0xd5, 0xfb, 0x89, 0xa7},
- {0x0, 0x2f, 0x5e, 0x71, 0xbc, 0x93, 0xe2, 0xcd, 0x65, 0x4a, 0x3b, 0x14, 0xd9, 0xf6, 0x87, 0xa8},
- {0x0, 0x30, 0x60, 0x50, 0xc0, 0xf0, 0xa0, 0x90, 0x9d, 0xad, 0xfd, 0xcd, 0x5d, 0x6d, 0x3d, 0xd},
- {0x0, 0x31, 0x62, 0x53, 0xc4, 0xf5, 0xa6, 0x97, 0x95, 0xa4, 0xf7, 0xc6, 0x51, 0x60, 0x33, 0x2},
- {0x0, 0x32, 0x64, 0x56, 0xc8, 0xfa, 0xac, 0x9e, 0x8d, 0xbf, 0xe9, 0xdb, 0x45, 0x77, 0x21, 0x13},
- {0x0, 0x33, 0x66, 0x55, 0xcc, 0xff, 0xaa, 0x99, 0x85, 0xb6, 0xe3, 0xd0, 0x49, 0x7a, 0x2f, 0x1c},
- {0x0, 0x34, 0x68, 0x5c, 0xd0, 0xe4, 0xb8, 0x8c, 0xbd, 0x89, 0xd5, 0xe1, 0x6d, 0x59, 0x5, 0x31},
- {0x0, 0x35, 0x6a, 0x5f, 0xd4, 0xe1, 0xbe, 0x8b, 0xb5, 0x80, 0xdf, 0xea, 0x61, 0x54, 0xb, 0x3e},
- {0x0, 0x36, 0x6c, 0x5a, 0xd8, 0xee, 0xb4, 0x82, 0xad, 0x9b, 0xc1, 0xf7, 0x75, 0x43, 0x19, 0x2f},
- {0x0, 0x37, 0x6e, 0x59, 0xdc, 0xeb, 0xb2, 0x85, 0xa5, 0x92, 0xcb, 0xfc, 0x79, 0x4e, 0x17, 0x20},
- {0x0, 0x38, 0x70, 0x48, 0xe0, 0xd8, 0x90, 0xa8, 0xdd, 0xe5, 0xad, 0x95, 0x3d, 0x5, 0x4d, 0x75},
- {0x0, 0x39, 0x72, 0x4b, 0xe4, 0xdd, 0x96, 0xaf, 0xd5, 0xec, 0xa7, 0x9e, 0x31, 0x8, 0x43, 0x7a},
- {0x0, 0x3a, 0x74, 0x4e, 0xe8, 0xd2, 0x9c, 0xa6, 0xcd, 0xf7, 0xb9, 0x83, 0x25, 0x1f, 0x51, 0x6b},
- {0x0, 0x3b, 0x76, 0x4d, 0xec, 0xd7, 0x9a, 0xa1, 0xc5, 0xfe, 0xb3, 0x88, 0x29, 0x12, 0x5f, 0x64},
- {0x0, 0x3c, 0x78, 0x44, 0xf0, 0xcc, 0x88, 0xb4, 0xfd, 0xc1, 0x85, 0xb9, 0xd, 0x31, 0x75, 0x49},
- {0x0, 0x3d, 0x7a, 0x47, 0xf4, 0xc9, 0x8e, 0xb3, 0xf5, 0xc8, 0x8f, 0xb2, 0x1, 0x3c, 0x7b, 0x46},
- {0x0, 0x3e, 0x7c, 0x42, 0xf8, 0xc6, 0x84, 0xba, 0xed, 0xd3, 0x91, 0xaf, 0x15, 0x2b, 0x69, 0x57},
- {0x0, 0x3f, 0x7e, 0x41, 0xfc, 0xc3, 0x82, 0xbd, 0xe5, 0xda, 0x9b, 0xa4, 0x19, 0x26, 0x67, 0x58},
- {0x0, 0x40, 0x80, 0xc0, 0x1d, 0x5d, 0x9d, 0xdd, 0x3a, 0x7a, 0xba, 0xfa, 0x27, 0x67, 0xa7, 0xe7},
- {0x0, 0x41, 0x82, 0xc3, 0x19, 0x58, 0x9b, 0xda, 0x32, 0x73, 0xb0, 0xf1, 0x2b, 0x6a, 0xa9, 0xe8},
- {0x0, 0x42, 0x84, 0xc6, 0x15, 0x57, 0x91, 0xd3, 0x2a, 0x68, 0xae, 0xec, 0x3f, 0x7d, 0xbb, 0xf9},
- {0x0, 0x43, 0x86, 0xc5, 0x11, 0x52, 0x97, 0xd4, 0x22, 0x61, 0xa4, 0xe7, 0x33, 0x70, 0xb5, 0xf6},
- {0x0, 0x44, 0x88, 0xcc, 0xd, 0x49, 0x85, 0xc1, 0x1a, 0x5e, 0x92, 0xd6, 0x17, 0x53, 0x9f, 0xdb},
- {0x0, 0x45, 0x8a, 0xcf, 0x9, 0x4c, 0x83, 0xc6, 0x12, 0x57, 0x98, 0xdd, 0x1b, 0x5e, 0x91, 0xd4},
- {0x0, 0x46, 0x8c, 0xca, 0x5, 0x43, 0x89, 0xcf, 0xa, 0x4c, 0x86, 0xc0, 0xf, 0x49, 0x83, 0xc5},
- {0x0, 0x47, 0x8e, 0xc9, 0x1, 0x46, 0x8f, 0xc8, 0x2, 0x45, 0x8c, 0xcb, 0x3, 0x44, 0x8d, 0xca},
- {0x0, 0x48, 0x90, 0xd8, 0x3d, 0x75, 0xad, 0xe5, 0x7a, 0x32, 0xea, 0xa2, 0x47, 0xf, 0xd7, 0x9f},
- {0x0, 0x49, 0x92, 0xdb, 0x39, 0x70, 0xab, 0xe2, 0x72, 0x3b, 0xe0, 0xa9, 0x4b, 0x2, 0xd9, 0x90},
- {0x0, 0x4a, 0x94, 0xde, 0x35, 0x7f, 0xa1, 0xeb, 0x6a, 0x20, 0xfe, 0xb4, 0x5f, 0x15, 0xcb, 0x81},
- {0x0, 0x4b, 0x96, 0xdd, 0x31, 0x7a, 0xa7, 0xec, 0x62, 0x29, 0xf4, 0xbf, 0x53, 0x18, 0xc5, 0x8e},
- {0x0, 0x4c, 0x98, 0xd4, 0x2d, 0x61, 0xb5, 0xf9, 0x5a, 0x16, 0xc2, 0x8e, 0x77, 0x3b, 0xef, 0xa3},
- {0x0, 0x4d, 0x9a, 0xd7, 0x29, 0x64, 0xb3, 0xfe, 0x52, 0x1f, 0xc8, 0x85, 0x7b, 0x36, 0xe1, 0xac},
- {0x0, 0x4e, 0x9c, 0xd2, 0x25, 0x6b, 0xb9, 0xf7, 0x4a, 0x4, 0xd6, 0x98, 0x6f, 0x21, 0xf3, 0xbd},
- {0x0, 0x4f, 0x9e, 0xd1, 0x21, 0x6e, 0xbf, 0xf0, 0x42, 0xd, 0xdc, 0x93, 0x63, 0x2c, 0xfd, 0xb2},
- {0x0, 0x50, 0xa0, 0xf0, 0x5d, 0xd, 0xfd, 0xad, 0xba, 0xea, 0x1a, 0x4a, 0xe7, 0xb7, 0x47, 0x17},
- {0x0, 0x51, 0xa2, 0xf3, 0x59, 0x8, 0xfb, 0xaa, 0xb2, 0xe3, 0x10, 0x41, 0xeb, 0xba, 0x49, 0x18},
- {0x0, 0x52, 0xa4, 0xf6, 0x55, 0x7, 0xf1, 0xa3, 0xaa, 0xf8, 0xe, 0x5c, 0xff, 0xad, 0x5b, 0x9},
- {0x0, 0x53, 0xa6, 0xf5, 0x51, 0x2, 0xf7, 0xa4, 0xa2, 0xf1, 0x4, 0x57, 0xf3, 0xa0, 0x55, 0x6},
- {0x0, 0x54, 0xa8, 0xfc, 0x4d, 0x19, 0xe5, 0xb1, 0x9a, 0xce, 0x32, 0x66, 0xd7, 0x83, 0x7f, 0x2b},
- {0x0, 0x55, 0xaa, 0xff, 0x49, 0x1c, 0xe3, 0xb6, 0x92, 0xc7, 0x38, 0x6d, 0xdb, 0x8e, 0x71, 0x24},
- {0x0, 0x56, 0xac, 0xfa, 0x45, 0x13, 0xe9, 0xbf, 0x8a, 0xdc, 0x26, 0x70, 0xcf, 0x99, 0x63, 0x35},
- {0x0, 0x57, 0xae, 0xf9, 0x41, 0x16, 0xef, 0xb8, 0x82, 0xd5, 0x2c, 0x7b, 0xc3, 0x94, 0x6d, 0x3a},
- {0x0, 0x58, 0xb0, 0xe8, 0x7d, 0x25, 0xcd, 0x95, 0xfa, 0xa2, 0x4a, 0x12, 0x87, 0xdf, 0x37, 0x6f},
- {0x0, 0x59, 0xb2, 0xeb, 0x79, 0x20, 0xcb, 0x92, 0xf2, 0xab, 0x40, 0x19, 0x8b, 0xd2, 0x39, 0x60},
- {0x0, 0x5a, 0xb4, 0xee, 0x75, 0x2f, 0xc1, 0x9b, 0xea, 0xb0, 0x5e, 0x4, 0x9f, 0xc5, 0x2b, 0x71},
- {0x0, 0x5b, 0xb6, 0xed, 0x71, 0x2a, 0xc7, 0x9c, 0xe2, 0xb9, 0x54, 0xf, 0x93, 0xc8, 0x25, 0x7e},
- {0x0, 0x5c, 0xb8, 0xe4, 0x6d, 0x31, 0xd5, 0x89, 0xda, 0x86, 0x62, 0x3e, 0xb7, 0xeb, 0xf, 0x53},
- {0x0, 0x5d, 0xba, 0xe7, 0x69, 0x34, 0xd3, 0x8e, 0xd2, 0x8f, 0x68, 0x35, 0xbb, 0xe6, 0x1, 0x5c},
- {0x0, 0x5e, 0xbc, 0xe2, 0x65, 0x3b, 0xd9, 0x87, 0xca, 0x94, 0x76, 0x28, 0xaf, 0xf1, 0x13, 0x4d},
- {0x0, 0x5f, 0xbe, 0xe1, 0x61, 0x3e, 0xdf, 0x80, 0xc2, 0x9d, 0x7c, 0x23, 0xa3, 0xfc, 0x1d, 0x42},
- {0x0, 0x60, 0xc0, 0xa0, 0x9d, 0xfd, 0x5d, 0x3d, 0x27, 0x47, 0xe7, 0x87, 0xba, 0xda, 0x7a, 0x1a},
- {0x0, 0x61, 0xc2, 0xa3, 0x99, 0xf8, 0x5b, 0x3a, 0x2f, 0x4e, 0xed, 0x8c, 0xb6, 0xd7, 0x74, 0x15},
- {0x0, 0x62, 0xc4, 0xa6, 0x95, 0xf7, 0x51, 0x33, 0x37, 0x55, 0xf3, 0x91, 0xa2, 0xc0, 0x66, 0x4},
- {0x0, 0x63, 0xc6, 0xa5, 0x91, 0xf2, 0x57, 0x34, 0x3f, 0x5c, 0xf9, 0x9a, 0xae, 0xcd, 0x68, 0xb},
- {0x0, 0x64, 0xc8, 0xac, 0x8d, 0xe9, 0x45, 0x21, 0x7, 0x63, 0xcf, 0xab, 0x8a, 0xee, 0x42, 0x26},
- {0x0, 0x65, 0xca, 0xaf, 0x89, 0xec, 0x43, 0x26, 0xf, 0x6a, 0xc5, 0xa0, 0x86, 0xe3, 0x4c, 0x29},
- {0x0, 0x66, 0xcc, 0xaa, 0x85, 0xe3, 0x49, 0x2f, 0x17, 0x71, 0xdb, 0xbd, 0x92, 0xf4, 0x5e, 0x38},
- {0x0, 0x67, 0xce, 0xa9, 0x81, 0xe6, 0x4f, 0x28, 0x1f, 0x78, 0xd1, 0xb6, 0x9e, 0xf9, 0x50, 0x37},
- {0x0, 0x68, 0xd0, 0xb8, 0xbd, 0xd5, 0x6d, 0x5, 0x67, 0xf, 0xb7, 0xdf, 0xda, 0xb2, 0xa, 0x62},
- {0x0, 0x69, 0xd2, 0xbb, 0xb9, 0xd0, 0x6b, 0x2, 0x6f, 0x6, 0xbd, 0xd4, 0xd6, 0xbf, 0x4, 0x6d},
- {0x0, 0x6a, 0xd4, 0xbe, 0xb5, 0xdf, 0x61, 0xb, 0x77, 0x1d, 0xa3, 0xc9, 0xc2, 0xa8, 0x16, 0x7c},
- {0x0, 0x6b, 0xd6, 0xbd, 0xb1, 0xda, 0x67, 0xc, 0x7f, 0x14, 0xa9, 0xc2, 0xce, 0xa5, 0x18, 0x73},
- {0x0, 0x6c, 0xd8, 0xb4, 0xad, 0xc1, 0x75, 0x19, 0x47, 0x2b, 0x9f, 0xf3, 0xea, 0x86, 0x32, 0x5e},
- {0x0, 0x6d, 0xda, 0xb7, 0xa9, 0xc4, 0x73, 0x1e, 0x4f, 0x22, 0x95, 0xf8, 0xe6, 0x8b, 0x3c, 0x51},
- {0x0, 0x6e, 0xdc, 0xb2, 0xa5, 0xcb, 0x79, 0x17, 0x57, 0x39, 0x8b, 0xe5, 0xf2, 0x9c, 0x2e, 0x40},
- {0x0, 0x6f, 0xde, 0xb1, 0xa1, 0xce, 0x7f, 0x10, 0x5f, 0x30, 0x81, 0xee, 0xfe, 0x91, 0x20, 0x4f},
- {0x0, 0x70, 0xe0, 0x90, 0xdd, 0xad, 0x3d, 0x4d, 0xa7, 0xd7, 0x47, 0x37, 0x7a, 0xa, 0x9a, 0xea},
- {0x0, 0x71, 0xe2, 0x93, 0xd9, 0xa8, 0x3b, 0x4a, 0xaf, 0xde, 0x4d, 0x3c, 0x76, 0x7, 0x94, 0xe5},
- {0x0, 0x72, 0xe4, 0x96, 0xd5, 0xa7, 0x31, 0x43, 0xb7, 0xc5, 0x53, 0x21, 0x62, 0x10, 0x86, 0xf4},
- {0x0, 0x73, 0xe6, 0x95, 0xd1, 0xa2, 0x37, 0x44, 0xbf, 0xcc, 0x59, 0x2a, 0x6e, 0x1d, 0x88, 0xfb},
- {0x0, 0x74, 0xe8, 0x9c, 0xcd, 0xb9, 0x25, 0x51, 0x87, 0xf3, 0x6f, 0x1b, 0x4a, 0x3e, 0xa2, 0xd6},
- {0x0, 0x75, 0xea, 0x9f, 0xc9, 0xbc, 0x23, 0x56, 0x8f, 0xfa, 0x65, 0x10, 0x46, 0x33, 0xac, 0xd9},
- {0x0, 0x76, 0xec, 0x9a, 0xc5, 0xb3, 0x29, 0x5f, 0x97, 0xe1, 0x7b, 0xd, 0x52, 0x24, 0xbe, 0xc8},
- {0x0, 0x77, 0xee, 0x99, 0xc1, 0xb6, 0x2f, 0x58, 0x9f, 0xe8, 0x71, 0x6, 0x5e, 0x29, 0xb0, 0xc7},
- {0x0, 0x78, 0xf0, 0x88, 0xfd, 0x85, 0xd, 0x75, 0xe7, 0x9f, 0x17, 0x6f, 0x1a, 0x62, 0xea, 0x92},
- {0x0, 0x79, 0xf2, 0x8b, 0xf9, 0x80, 0xb, 0x72, 0xef, 0x96, 0x1d, 0x64, 0x16, 0x6f, 0xe4, 0x9d},
- {0x0, 0x7a, 0xf4, 0x8e, 0xf5, 0x8f, 0x1, 0x7b, 0xf7, 0x8d, 0x3, 0x79, 0x2, 0x78, 0xf6, 0x8c},
- {0x0, 0x7b, 0xf6, 0x8d, 0xf1, 0x8a, 0x7, 0x7c, 0xff, 0x84, 0x9, 0x72, 0xe, 0x75, 0xf8, 0x83},
- {0x0, 0x7c, 0xf8, 0x84, 0xed, 0x91, 0x15, 0x69, 0xc7, 0xbb, 0x3f, 0x43, 0x2a, 0x56, 0xd2, 0xae},
- {0x0, 0x7d, 0xfa, 0x87, 0xe9, 0x94, 0x13, 0x6e, 0xcf, 0xb2, 0x35, 0x48, 0x26, 0x5b, 0xdc, 0xa1},
- {0x0, 0x7e, 0xfc, 0x82, 0xe5, 0x9b, 0x19, 0x67, 0xd7, 0xa9, 0x2b, 0x55, 0x32, 0x4c, 0xce, 0xb0},
- {0x0, 0x7f, 0xfe, 0x81, 0xe1, 0x9e, 0x1f, 0x60, 0xdf, 0xa0, 0x21, 0x5e, 0x3e, 0x41, 0xc0, 0xbf},
- {0x0, 0x80, 0x1d, 0x9d, 0x3a, 0xba, 0x27, 0xa7, 0x74, 0xf4, 0x69, 0xe9, 0x4e, 0xce, 0x53, 0xd3},
- {0x0, 0x81, 0x1f, 0x9e, 0x3e, 0xbf, 0x21, 0xa0, 0x7c, 0xfd, 0x63, 0xe2, 0x42, 0xc3, 0x5d, 0xdc},
- {0x0, 0x82, 0x19, 0x9b, 0x32, 0xb0, 0x2b, 0xa9, 0x64, 0xe6, 0x7d, 0xff, 0x56, 0xd4, 0x4f, 0xcd},
- {0x0, 0x83, 0x1b, 0x98, 0x36, 0xb5, 0x2d, 0xae, 0x6c, 0xef, 0x77, 0xf4, 0x5a, 0xd9, 0x41, 0xc2},
- {0x0, 0x84, 0x15, 0x91, 0x2a, 0xae, 0x3f, 0xbb, 0x54, 0xd0, 0x41, 0xc5, 0x7e, 0xfa, 0x6b, 0xef},
- {0x0, 0x85, 0x17, 0x92, 0x2e, 0xab, 0x39, 0xbc, 0x5c, 0xd9, 0x4b, 0xce, 0x72, 0xf7, 0x65, 0xe0},
- {0x0, 0x86, 0x11, 0x97, 0x22, 0xa4, 0x33, 0xb5, 0x44, 0xc2, 0x55, 0xd3, 0x66, 0xe0, 0x77, 0xf1},
- {0x0, 0x87, 0x13, 0x94, 0x26, 0xa1, 0x35, 0xb2, 0x4c, 0xcb, 0x5f, 0xd8, 0x6a, 0xed, 0x79, 0xfe},
- {0x0, 0x88, 0xd, 0x85, 0x1a, 0x92, 0x17, 0x9f, 0x34, 0xbc, 0x39, 0xb1, 0x2e, 0xa6, 0x23, 0xab},
- {0x0, 0x89, 0xf, 0x86, 0x1e, 0x97, 0x11, 0x98, 0x3c, 0xb5, 0x33, 0xba, 0x22, 0xab, 0x2d, 0xa4},
- {0x0, 0x8a, 0x9, 0x83, 0x12, 0x98, 0x1b, 0x91, 0x24, 0xae, 0x2d, 0xa7, 0x36, 0xbc, 0x3f, 0xb5},
- {0x0, 0x8b, 0xb, 0x80, 0x16, 0x9d, 0x1d, 0x96, 0x2c, 0xa7, 0x27, 0xac, 0x3a, 0xb1, 0x31, 0xba},
- {0x0, 0x8c, 0x5, 0x89, 0xa, 0x86, 0xf, 0x83, 0x14, 0x98, 0x11, 0x9d, 0x1e, 0x92, 0x1b, 0x97},
- {0x0, 0x8d, 0x7, 0x8a, 0xe, 0x83, 0x9, 0x84, 0x1c, 0x91, 0x1b, 0x96, 0x12, 0x9f, 0x15, 0x98},
- {0x0, 0x8e, 0x1, 0x8f, 0x2, 0x8c, 0x3, 0x8d, 0x4, 0x8a, 0x5, 0x8b, 0x6, 0x88, 0x7, 0x89},
- {0x0, 0x8f, 0x3, 0x8c, 0x6, 0x89, 0x5, 0x8a, 0xc, 0x83, 0xf, 0x80, 0xa, 0x85, 0x9, 0x86},
- {0x0, 0x90, 0x3d, 0xad, 0x7a, 0xea, 0x47, 0xd7, 0xf4, 0x64, 0xc9, 0x59, 0x8e, 0x1e, 0xb3, 0x23},
- {0x0, 0x91, 0x3f, 0xae, 0x7e, 0xef, 0x41, 0xd0, 0xfc, 0x6d, 0xc3, 0x52, 0x82, 0x13, 0xbd, 0x2c},
- {0x0, 0x92, 0x39, 0xab, 0x72, 0xe0, 0x4b, 0xd9, 0xe4, 0x76, 0xdd, 0x4f, 0x96, 0x4, 0xaf, 0x3d},
- {0x0, 0x93, 0x3b, 0xa8, 0x76, 0xe5, 0x4d, 0xde, 0xec, 0x7f, 0xd7, 0x44, 0x9a, 0x9, 0xa1, 0x32},
- {0x0, 0x94, 0x35, 0xa1, 0x6a, 0xfe, 0x5f, 0xcb, 0xd4, 0x40, 0xe1, 0x75, 0xbe, 0x2a, 0x8b, 0x1f},
- {0x0, 0x95, 0x37, 0xa2, 0x6e, 0xfb, 0x59, 0xcc, 0xdc, 0x49, 0xeb, 0x7e, 0xb2, 0x27, 0x85, 0x10},
- {0x0, 0x96, 0x31, 0xa7, 0x62, 0xf4, 0x53, 0xc5, 0xc4, 0x52, 0xf5, 0x63, 0xa6, 0x30, 0x97, 0x1},
- {0x0, 0x97, 0x33, 0xa4, 0x66, 0xf1, 0x55, 0xc2, 0xcc, 0x5b, 0xff, 0x68, 0xaa, 0x3d, 0x99, 0xe},
- {0x0, 0x98, 0x2d, 0xb5, 0x5a, 0xc2, 0x77, 0xef, 0xb4, 0x2c, 0x99, 0x1, 0xee, 0x76, 0xc3, 0x5b},
- {0x0, 0x99, 0x2f, 0xb6, 0x5e, 0xc7, 0x71, 0xe8, 0xbc, 0x25, 0x93, 0xa, 0xe2, 0x7b, 0xcd, 0x54},
- {0x0, 0x9a, 0x29, 0xb3, 0x52, 0xc8, 0x7b, 0xe1, 0xa4, 0x3e, 0x8d, 0x17, 0xf6, 0x6c, 0xdf, 0x45},
- {0x0, 0x9b, 0x2b, 0xb0, 0x56, 0xcd, 0x7d, 0xe6, 0xac, 0x37, 0x87, 0x1c, 0xfa, 0x61, 0xd1, 0x4a},
- {0x0, 0x9c, 0x25, 0xb9, 0x4a, 0xd6, 0x6f, 0xf3, 0x94, 0x8, 0xb1, 0x2d, 0xde, 0x42, 0xfb, 0x67},
- {0x0, 0x9d, 0x27, 0xba, 0x4e, 0xd3, 0x69, 0xf4, 0x9c, 0x1, 0xbb, 0x26, 0xd2, 0x4f, 0xf5, 0x68},
- {0x0, 0x9e, 0x21, 0xbf, 0x42, 0xdc, 0x63, 0xfd, 0x84, 0x1a, 0xa5, 0x3b, 0xc6, 0x58, 0xe7, 0x79},
- {0x0, 0x9f, 0x23, 0xbc, 0x46, 0xd9, 0x65, 0xfa, 0x8c, 0x13, 0xaf, 0x30, 0xca, 0x55, 0xe9, 0x76},
- {0x0, 0xa0, 0x5d, 0xfd, 0xba, 0x1a, 0xe7, 0x47, 0x69, 0xc9, 0x34, 0x94, 0xd3, 0x73, 0x8e, 0x2e},
- {0x0, 0xa1, 0x5f, 0xfe, 0xbe, 0x1f, 0xe1, 0x40, 0x61, 0xc0, 0x3e, 0x9f, 0xdf, 0x7e, 0x80, 0x21},
- {0x0, 0xa2, 0x59, 0xfb, 0xb2, 0x10, 0xeb, 0x49, 0x79, 0xdb, 0x20, 0x82, 0xcb, 0x69, 0x92, 0x30},
- {0x0, 0xa3, 0x5b, 0xf8, 0xb6, 0x15, 0xed, 0x4e, 0x71, 0xd2, 0x2a, 0x89, 0xc7, 0x64, 0x9c, 0x3f},
- {0x0, 0xa4, 0x55, 0xf1, 0xaa, 0xe, 0xff, 0x5b, 0x49, 0xed, 0x1c, 0xb8, 0xe3, 0x47, 0xb6, 0x12},
- {0x0, 0xa5, 0x57, 0xf2, 0xae, 0xb, 0xf9, 0x5c, 0x41, 0xe4, 0x16, 0xb3, 0xef, 0x4a, 0xb8, 0x1d},
- {0x0, 0xa6, 0x51, 0xf7, 0xa2, 0x4, 0xf3, 0x55, 0x59, 0xff, 0x8, 0xae, 0xfb, 0x5d, 0xaa, 0xc},
- {0x0, 0xa7, 0x53, 0xf4, 0xa6, 0x1, 0xf5, 0x52, 0x51, 0xf6, 0x2, 0xa5, 0xf7, 0x50, 0xa4, 0x3},
- {0x0, 0xa8, 0x4d, 0xe5, 0x9a, 0x32, 0xd7, 0x7f, 0x29, 0x81, 0x64, 0xcc, 0xb3, 0x1b, 0xfe, 0x56},
- {0x0, 0xa9, 0x4f, 0xe6, 0x9e, 0x37, 0xd1, 0x78, 0x21, 0x88, 0x6e, 0xc7, 0xbf, 0x16, 0xf0, 0x59},
- {0x0, 0xaa, 0x49, 0xe3, 0x92, 0x38, 0xdb, 0x71, 0x39, 0x93, 0x70, 0xda, 0xab, 0x1, 0xe2, 0x48},
- {0x0, 0xab, 0x4b, 0xe0, 0x96, 0x3d, 0xdd, 0x76, 0x31, 0x9a, 0x7a, 0xd1, 0xa7, 0xc, 0xec, 0x47},
- {0x0, 0xac, 0x45, 0xe9, 0x8a, 0x26, 0xcf, 0x63, 0x9, 0xa5, 0x4c, 0xe0, 0x83, 0x2f, 0xc6, 0x6a},
- {0x0, 0xad, 0x47, 0xea, 0x8e, 0x23, 0xc9, 0x64, 0x1, 0xac, 0x46, 0xeb, 0x8f, 0x22, 0xc8, 0x65},
- {0x0, 0xae, 0x41, 0xef, 0x82, 0x2c, 0xc3, 0x6d, 0x19, 0xb7, 0x58, 0xf6, 0x9b, 0x35, 0xda, 0x74},
- {0x0, 0xaf, 0x43, 0xec, 0x86, 0x29, 0xc5, 0x6a, 0x11, 0xbe, 0x52, 0xfd, 0x97, 0x38, 0xd4, 0x7b},
- {0x0, 0xb0, 0x7d, 0xcd, 0xfa, 0x4a, 0x87, 0x37, 0xe9, 0x59, 0x94, 0x24, 0x13, 0xa3, 0x6e, 0xde},
- {0x0, 0xb1, 0x7f, 0xce, 0xfe, 0x4f, 0x81, 0x30, 0xe1, 0x50, 0x9e, 0x2f, 0x1f, 0xae, 0x60, 0xd1},
- {0x0, 0xb2, 0x79, 0xcb, 0xf2, 0x40, 0x8b, 0x39, 0xf9, 0x4b, 0x80, 0x32, 0xb, 0xb9, 0x72, 0xc0},
- {0x0, 0xb3, 0x7b, 0xc8, 0xf6, 0x45, 0x8d, 0x3e, 0xf1, 0x42, 0x8a, 0x39, 0x7, 0xb4, 0x7c, 0xcf},
- {0x0, 0xb4, 0x75, 0xc1, 0xea, 0x5e, 0x9f, 0x2b, 0xc9, 0x7d, 0xbc, 0x8, 0x23, 0x97, 0x56, 0xe2},
- {0x0, 0xb5, 0x77, 0xc2, 0xee, 0x5b, 0x99, 0x2c, 0xc1, 0x74, 0xb6, 0x3, 0x2f, 0x9a, 0x58, 0xed},
- {0x0, 0xb6, 0x71, 0xc7, 0xe2, 0x54, 0x93, 0x25, 0xd9, 0x6f, 0xa8, 0x1e, 0x3b, 0x8d, 0x4a, 0xfc},
- {0x0, 0xb7, 0x73, 0xc4, 0xe6, 0x51, 0x95, 0x22, 0xd1, 0x66, 0xa2, 0x15, 0x37, 0x80, 0x44, 0xf3},
- {0x0, 0xb8, 0x6d, 0xd5, 0xda, 0x62, 0xb7, 0xf, 0xa9, 0x11, 0xc4, 0x7c, 0x73, 0xcb, 0x1e, 0xa6},
- {0x0, 0xb9, 0x6f, 0xd6, 0xde, 0x67, 0xb1, 0x8, 0xa1, 0x18, 0xce, 0x77, 0x7f, 0xc6, 0x10, 0xa9},
- {0x0, 0xba, 0x69, 0xd3, 0xd2, 0x68, 0xbb, 0x1, 0xb9, 0x3, 0xd0, 0x6a, 0x6b, 0xd1, 0x2, 0xb8},
- {0x0, 0xbb, 0x6b, 0xd0, 0xd6, 0x6d, 0xbd, 0x6, 0xb1, 0xa, 0xda, 0x61, 0x67, 0xdc, 0xc, 0xb7},
- {0x0, 0xbc, 0x65, 0xd9, 0xca, 0x76, 0xaf, 0x13, 0x89, 0x35, 0xec, 0x50, 0x43, 0xff, 0x26, 0x9a},
- {0x0, 0xbd, 0x67, 0xda, 0xce, 0x73, 0xa9, 0x14, 0x81, 0x3c, 0xe6, 0x5b, 0x4f, 0xf2, 0x28, 0x95},
- {0x0, 0xbe, 0x61, 0xdf, 0xc2, 0x7c, 0xa3, 0x1d, 0x99, 0x27, 0xf8, 0x46, 0x5b, 0xe5, 0x3a, 0x84},
- {0x0, 0xbf, 0x63, 0xdc, 0xc6, 0x79, 0xa5, 0x1a, 0x91, 0x2e, 0xf2, 0x4d, 0x57, 0xe8, 0x34, 0x8b},
- {0x0, 0xc0, 0x9d, 0x5d, 0x27, 0xe7, 0xba, 0x7a, 0x4e, 0x8e, 0xd3, 0x13, 0x69, 0xa9, 0xf4, 0x34},
- {0x0, 0xc1, 0x9f, 0x5e, 0x23, 0xe2, 0xbc, 0x7d, 0x46, 0x87, 0xd9, 0x18, 0x65, 0xa4, 0xfa, 0x3b},
- {0x0, 0xc2, 0x99, 0x5b, 0x2f, 0xed, 0xb6, 0x74, 0x5e, 0x9c, 0xc7, 0x5, 0x71, 0xb3, 0xe8, 0x2a},
- {0x0, 0xc3, 0x9b, 0x58, 0x2b, 0xe8, 0xb0, 0x73, 0x56, 0x95, 0xcd, 0xe, 0x7d, 0xbe, 0xe6, 0x25},
- {0x0, 0xc4, 0x95, 0x51, 0x37, 0xf3, 0xa2, 0x66, 0x6e, 0xaa, 0xfb, 0x3f, 0x59, 0x9d, 0xcc, 0x8},
- {0x0, 0xc5, 0x97, 0x52, 0x33, 0xf6, 0xa4, 0x61, 0x66, 0xa3, 0xf1, 0x34, 0x55, 0x90, 0xc2, 0x7},
- {0x0, 0xc6, 0x91, 0x57, 0x3f, 0xf9, 0xae, 0x68, 0x7e, 0xb8, 0xef, 0x29, 0x41, 0x87, 0xd0, 0x16},
- {0x0, 0xc7, 0x93, 0x54, 0x3b, 0xfc, 0xa8, 0x6f, 0x76, 0xb1, 0xe5, 0x22, 0x4d, 0x8a, 0xde, 0x19},
- {0x0, 0xc8, 0x8d, 0x45, 0x7, 0xcf, 0x8a, 0x42, 0xe, 0xc6, 0x83, 0x4b, 0x9, 0xc1, 0x84, 0x4c},
- {0x0, 0xc9, 0x8f, 0x46, 0x3, 0xca, 0x8c, 0x45, 0x6, 0xcf, 0x89, 0x40, 0x5, 0xcc, 0x8a, 0x43},
- {0x0, 0xca, 0x89, 0x43, 0xf, 0xc5, 0x86, 0x4c, 0x1e, 0xd4, 0x97, 0x5d, 0x11, 0xdb, 0x98, 0x52},
- {0x0, 0xcb, 0x8b, 0x40, 0xb, 0xc0, 0x80, 0x4b, 0x16, 0xdd, 0x9d, 0x56, 0x1d, 0xd6, 0x96, 0x5d},
- {0x0, 0xcc, 0x85, 0x49, 0x17, 0xdb, 0x92, 0x5e, 0x2e, 0xe2, 0xab, 0x67, 0x39, 0xf5, 0xbc, 0x70},
- {0x0, 0xcd, 0x87, 0x4a, 0x13, 0xde, 0x94, 0x59, 0x26, 0xeb, 0xa1, 0x6c, 0x35, 0xf8, 0xb2, 0x7f},
- {0x0, 0xce, 0x81, 0x4f, 0x1f, 0xd1, 0x9e, 0x50, 0x3e, 0xf0, 0xbf, 0x71, 0x21, 0xef, 0xa0, 0x6e},
- {0x0, 0xcf, 0x83, 0x4c, 0x1b, 0xd4, 0x98, 0x57, 0x36, 0xf9, 0xb5, 0x7a, 0x2d, 0xe2, 0xae, 0x61},
- {0x0, 0xd0, 0xbd, 0x6d, 0x67, 0xb7, 0xda, 0xa, 0xce, 0x1e, 0x73, 0xa3, 0xa9, 0x79, 0x14, 0xc4},
- {0x0, 0xd1, 0xbf, 0x6e, 0x63, 0xb2, 0xdc, 0xd, 0xc6, 0x17, 0x79, 0xa8, 0xa5, 0x74, 0x1a, 0xcb},
- {0x0, 0xd2, 0xb9, 0x6b, 0x6f, 0xbd, 0xd6, 0x4, 0xde, 0xc, 0x67, 0xb5, 0xb1, 0x63, 0x8, 0xda},
- {0x0, 0xd3, 0xbb, 0x68, 0x6b, 0xb8, 0xd0, 0x3, 0xd6, 0x5, 0x6d, 0xbe, 0xbd, 0x6e, 0x6, 0xd5},
- {0x0, 0xd4, 0xb5, 0x61, 0x77, 0xa3, 0xc2, 0x16, 0xee, 0x3a, 0x5b, 0x8f, 0x99, 0x4d, 0x2c, 0xf8},
- {0x0, 0xd5, 0xb7, 0x62, 0x73, 0xa6, 0xc4, 0x11, 0xe6, 0x33, 0x51, 0x84, 0x95, 0x40, 0x22, 0xf7},
- {0x0, 0xd6, 0xb1, 0x67, 0x7f, 0xa9, 0xce, 0x18, 0xfe, 0x28, 0x4f, 0x99, 0x81, 0x57, 0x30, 0xe6},
- {0x0, 0xd7, 0xb3, 0x64, 0x7b, 0xac, 0xc8, 0x1f, 0xf6, 0x21, 0x45, 0x92, 0x8d, 0x5a, 0x3e, 0xe9},
- {0x0, 0xd8, 0xad, 0x75, 0x47, 0x9f, 0xea, 0x32, 0x8e, 0x56, 0x23, 0xfb, 0xc9, 0x11, 0x64, 0xbc},
- {0x0, 0xd9, 0xaf, 0x76, 0x43, 0x9a, 0xec, 0x35, 0x86, 0x5f, 0x29, 0xf0, 0xc5, 0x1c, 0x6a, 0xb3},
- {0x0, 0xda, 0xa9, 0x73, 0x4f, 0x95, 0xe6, 0x3c, 0x9e, 0x44, 0x37, 0xed, 0xd1, 0xb, 0x78, 0xa2},
- {0x0, 0xdb, 0xab, 0x70, 0x4b, 0x90, 0xe0, 0x3b, 0x96, 0x4d, 0x3d, 0xe6, 0xdd, 0x6, 0x76, 0xad},
- {0x0, 0xdc, 0xa5, 0x79, 0x57, 0x8b, 0xf2, 0x2e, 0xae, 0x72, 0xb, 0xd7, 0xf9, 0x25, 0x5c, 0x80},
- {0x0, 0xdd, 0xa7, 0x7a, 0x53, 0x8e, 0xf4, 0x29, 0xa6, 0x7b, 0x1, 0xdc, 0xf5, 0x28, 0x52, 0x8f},
- {0x0, 0xde, 0xa1, 0x7f, 0x5f, 0x81, 0xfe, 0x20, 0xbe, 0x60, 0x1f, 0xc1, 0xe1, 0x3f, 0x40, 0x9e},
- {0x0, 0xdf, 0xa3, 0x7c, 0x5b, 0x84, 0xf8, 0x27, 0xb6, 0x69, 0x15, 0xca, 0xed, 0x32, 0x4e, 0x91},
- {0x0, 0xe0, 0xdd, 0x3d, 0xa7, 0x47, 0x7a, 0x9a, 0x53, 0xb3, 0x8e, 0x6e, 0xf4, 0x14, 0x29, 0xc9},
- {0x0, 0xe1, 0xdf, 0x3e, 0xa3, 0x42, 0x7c, 0x9d, 0x5b, 0xba, 0x84, 0x65, 0xf8, 0x19, 0x27, 0xc6},
- {0x0, 0xe2, 0xd9, 0x3b, 0xaf, 0x4d, 0x76, 0x94, 0x43, 0xa1, 0x9a, 0x78, 0xec, 0xe, 0x35, 0xd7},
- {0x0, 0xe3, 0xdb, 0x38, 0xab, 0x48, 0x70, 0x93, 0x4b, 0xa8, 0x90, 0x73, 0xe0, 0x3, 0x3b, 0xd8},
- {0x0, 0xe4, 0xd5, 0x31, 0xb7, 0x53, 0x62, 0x86, 0x73, 0x97, 0xa6, 0x42, 0xc4, 0x20, 0x11, 0xf5},
- {0x0, 0xe5, 0xd7, 0x32, 0xb3, 0x56, 0x64, 0x81, 0x7b, 0x9e, 0xac, 0x49, 0xc8, 0x2d, 0x1f, 0xfa},
- {0x0, 0xe6, 0xd1, 0x37, 0xbf, 0x59, 0x6e, 0x88, 0x63, 0x85, 0xb2, 0x54, 0xdc, 0x3a, 0xd, 0xeb},
- {0x0, 0xe7, 0xd3, 0x34, 0xbb, 0x5c, 0x68, 0x8f, 0x6b, 0x8c, 0xb8, 0x5f, 0xd0, 0x37, 0x3, 0xe4},
- {0x0, 0xe8, 0xcd, 0x25, 0x87, 0x6f, 0x4a, 0xa2, 0x13, 0xfb, 0xde, 0x36, 0x94, 0x7c, 0x59, 0xb1},
- {0x0, 0xe9, 0xcf, 0x26, 0x83, 0x6a, 0x4c, 0xa5, 0x1b, 0xf2, 0xd4, 0x3d, 0x98, 0x71, 0x57, 0xbe},
- {0x0, 0xea, 0xc9, 0x23, 0x8f, 0x65, 0x46, 0xac, 0x3, 0xe9, 0xca, 0x20, 0x8c, 0x66, 0x45, 0xaf},
- {0x0, 0xeb, 0xcb, 0x20, 0x8b, 0x60, 0x40, 0xab, 0xb, 0xe0, 0xc0, 0x2b, 0x80, 0x6b, 0x4b, 0xa0},
- {0x0, 0xec, 0xc5, 0x29, 0x97, 0x7b, 0x52, 0xbe, 0x33, 0xdf, 0xf6, 0x1a, 0xa4, 0x48, 0x61, 0x8d},
- {0x0, 0xed, 0xc7, 0x2a, 0x93, 0x7e, 0x54, 0xb9, 0x3b, 0xd6, 0xfc, 0x11, 0xa8, 0x45, 0x6f, 0x82},
- {0x0, 0xee, 0xc1, 0x2f, 0x9f, 0x71, 0x5e, 0xb0, 0x23, 0xcd, 0xe2, 0xc, 0xbc, 0x52, 0x7d, 0x93},
- {0x0, 0xef, 0xc3, 0x2c, 0x9b, 0x74, 0x58, 0xb7, 0x2b, 0xc4, 0xe8, 0x7, 0xb0, 0x5f, 0x73, 0x9c},
- {0x0, 0xf0, 0xfd, 0xd, 0xe7, 0x17, 0x1a, 0xea, 0xd3, 0x23, 0x2e, 0xde, 0x34, 0xc4, 0xc9, 0x39},
- {0x0, 0xf1, 0xff, 0xe, 0xe3, 0x12, 0x1c, 0xed, 0xdb, 0x2a, 0x24, 0xd5, 0x38, 0xc9, 0xc7, 0x36},
- {0x0, 0xf2, 0xf9, 0xb, 0xef, 0x1d, 0x16, 0xe4, 0xc3, 0x31, 0x3a, 0xc8, 0x2c, 0xde, 0xd5, 0x27},
- {0x0, 0xf3, 0xfb, 0x8, 0xeb, 0x18, 0x10, 0xe3, 0xcb, 0x38, 0x30, 0xc3, 0x20, 0xd3, 0xdb, 0x28},
- {0x0, 0xf4, 0xf5, 0x1, 0xf7, 0x3, 0x2, 0xf6, 0xf3, 0x7, 0x6, 0xf2, 0x4, 0xf0, 0xf1, 0x5},
- {0x0, 0xf5, 0xf7, 0x2, 0xf3, 0x6, 0x4, 0xf1, 0xfb, 0xe, 0xc, 0xf9, 0x8, 0xfd, 0xff, 0xa},
- {0x0, 0xf6, 0xf1, 0x7, 0xff, 0x9, 0xe, 0xf8, 0xe3, 0x15, 0x12, 0xe4, 0x1c, 0xea, 0xed, 0x1b},
- {0x0, 0xf7, 0xf3, 0x4, 0xfb, 0xc, 0x8, 0xff, 0xeb, 0x1c, 0x18, 0xef, 0x10, 0xe7, 0xe3, 0x14},
- {0x0, 0xf8, 0xed, 0x15, 0xc7, 0x3f, 0x2a, 0xd2, 0x93, 0x6b, 0x7e, 0x86, 0x54, 0xac, 0xb9, 0x41},
- {0x0, 0xf9, 0xef, 0x16, 0xc3, 0x3a, 0x2c, 0xd5, 0x9b, 0x62, 0x74, 0x8d, 0x58, 0xa1, 0xb7, 0x4e},
- {0x0, 0xfa, 0xe9, 0x13, 0xcf, 0x35, 0x26, 0xdc, 0x83, 0x79, 0x6a, 0x90, 0x4c, 0xb6, 0xa5, 0x5f},
- {0x0, 0xfb, 0xeb, 0x10, 0xcb, 0x30, 0x20, 0xdb, 0x8b, 0x70, 0x60, 0x9b, 0x40, 0xbb, 0xab, 0x50},
- {0x0, 0xfc, 0xe5, 0x19, 0xd7, 0x2b, 0x32, 0xce, 0xb3, 0x4f, 0x56, 0xaa, 0x64, 0x98, 0x81, 0x7d},
- {0x0, 0xfd, 0xe7, 0x1a, 0xd3, 0x2e, 0x34, 0xc9, 0xbb, 0x46, 0x5c, 0xa1, 0x68, 0x95, 0x8f, 0x72},
- {0x0, 0xfe, 0xe1, 0x1f, 0xdf, 0x21, 0x3e, 0xc0, 0xa3, 0x5d, 0x42, 0xbc, 0x7c, 0x82, 0x9d, 0x63},
- {0x0, 0xff, 0xe3, 0x1c, 0xdb, 0x24, 0x38, 0xc7, 0xab, 0x54, 0x48, 0xb7, 0x70, 0x8f, 0x93, 0x6c}}
-var mulTableHigh = [256][16]uint8{{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0},
- {0x0, 0x10, 0x20, 0x30, 0x40, 0x50, 0x60, 0x70, 0x80, 0x90, 0xa0, 0xb0, 0xc0, 0xd0, 0xe0, 0xf0},
- {0x0, 0x20, 0x40, 0x60, 0x80, 0xa0, 0xc0, 0xe0, 0x1d, 0x3d, 0x5d, 0x7d, 0x9d, 0xbd, 0xdd, 0xfd},
- {0x0, 0x30, 0x60, 0x50, 0xc0, 0xf0, 0xa0, 0x90, 0x9d, 0xad, 0xfd, 0xcd, 0x5d, 0x6d, 0x3d, 0xd},
- {0x0, 0x40, 0x80, 0xc0, 0x1d, 0x5d, 0x9d, 0xdd, 0x3a, 0x7a, 0xba, 0xfa, 0x27, 0x67, 0xa7, 0xe7},
- {0x0, 0x50, 0xa0, 0xf0, 0x5d, 0xd, 0xfd, 0xad, 0xba, 0xea, 0x1a, 0x4a, 0xe7, 0xb7, 0x47, 0x17},
- {0x0, 0x60, 0xc0, 0xa0, 0x9d, 0xfd, 0x5d, 0x3d, 0x27, 0x47, 0xe7, 0x87, 0xba, 0xda, 0x7a, 0x1a},
- {0x0, 0x70, 0xe0, 0x90, 0xdd, 0xad, 0x3d, 0x4d, 0xa7, 0xd7, 0x47, 0x37, 0x7a, 0xa, 0x9a, 0xea},
- {0x0, 0x80, 0x1d, 0x9d, 0x3a, 0xba, 0x27, 0xa7, 0x74, 0xf4, 0x69, 0xe9, 0x4e, 0xce, 0x53, 0xd3},
- {0x0, 0x90, 0x3d, 0xad, 0x7a, 0xea, 0x47, 0xd7, 0xf4, 0x64, 0xc9, 0x59, 0x8e, 0x1e, 0xb3, 0x23},
- {0x0, 0xa0, 0x5d, 0xfd, 0xba, 0x1a, 0xe7, 0x47, 0x69, 0xc9, 0x34, 0x94, 0xd3, 0x73, 0x8e, 0x2e},
- {0x0, 0xb0, 0x7d, 0xcd, 0xfa, 0x4a, 0x87, 0x37, 0xe9, 0x59, 0x94, 0x24, 0x13, 0xa3, 0x6e, 0xde},
- {0x0, 0xc0, 0x9d, 0x5d, 0x27, 0xe7, 0xba, 0x7a, 0x4e, 0x8e, 0xd3, 0x13, 0x69, 0xa9, 0xf4, 0x34},
- {0x0, 0xd0, 0xbd, 0x6d, 0x67, 0xb7, 0xda, 0xa, 0xce, 0x1e, 0x73, 0xa3, 0xa9, 0x79, 0x14, 0xc4},
- {0x0, 0xe0, 0xdd, 0x3d, 0xa7, 0x47, 0x7a, 0x9a, 0x53, 0xb3, 0x8e, 0x6e, 0xf4, 0x14, 0x29, 0xc9},
- {0x0, 0xf0, 0xfd, 0xd, 0xe7, 0x17, 0x1a, 0xea, 0xd3, 0x23, 0x2e, 0xde, 0x34, 0xc4, 0xc9, 0x39},
- {0x0, 0x1d, 0x3a, 0x27, 0x74, 0x69, 0x4e, 0x53, 0xe8, 0xf5, 0xd2, 0xcf, 0x9c, 0x81, 0xa6, 0xbb},
- {0x0, 0xd, 0x1a, 0x17, 0x34, 0x39, 0x2e, 0x23, 0x68, 0x65, 0x72, 0x7f, 0x5c, 0x51, 0x46, 0x4b},
- {0x0, 0x3d, 0x7a, 0x47, 0xf4, 0xc9, 0x8e, 0xb3, 0xf5, 0xc8, 0x8f, 0xb2, 0x1, 0x3c, 0x7b, 0x46},
- {0x0, 0x2d, 0x5a, 0x77, 0xb4, 0x99, 0xee, 0xc3, 0x75, 0x58, 0x2f, 0x2, 0xc1, 0xec, 0x9b, 0xb6},
- {0x0, 0x5d, 0xba, 0xe7, 0x69, 0x34, 0xd3, 0x8e, 0xd2, 0x8f, 0x68, 0x35, 0xbb, 0xe6, 0x1, 0x5c},
- {0x0, 0x4d, 0x9a, 0xd7, 0x29, 0x64, 0xb3, 0xfe, 0x52, 0x1f, 0xc8, 0x85, 0x7b, 0x36, 0xe1, 0xac},
- {0x0, 0x7d, 0xfa, 0x87, 0xe9, 0x94, 0x13, 0x6e, 0xcf, 0xb2, 0x35, 0x48, 0x26, 0x5b, 0xdc, 0xa1},
- {0x0, 0x6d, 0xda, 0xb7, 0xa9, 0xc4, 0x73, 0x1e, 0x4f, 0x22, 0x95, 0xf8, 0xe6, 0x8b, 0x3c, 0x51},
- {0x0, 0x9d, 0x27, 0xba, 0x4e, 0xd3, 0x69, 0xf4, 0x9c, 0x1, 0xbb, 0x26, 0xd2, 0x4f, 0xf5, 0x68},
- {0x0, 0x8d, 0x7, 0x8a, 0xe, 0x83, 0x9, 0x84, 0x1c, 0x91, 0x1b, 0x96, 0x12, 0x9f, 0x15, 0x98},
- {0x0, 0xbd, 0x67, 0xda, 0xce, 0x73, 0xa9, 0x14, 0x81, 0x3c, 0xe6, 0x5b, 0x4f, 0xf2, 0x28, 0x95},
- {0x0, 0xad, 0x47, 0xea, 0x8e, 0x23, 0xc9, 0x64, 0x1, 0xac, 0x46, 0xeb, 0x8f, 0x22, 0xc8, 0x65},
- {0x0, 0xdd, 0xa7, 0x7a, 0x53, 0x8e, 0xf4, 0x29, 0xa6, 0x7b, 0x1, 0xdc, 0xf5, 0x28, 0x52, 0x8f},
- {0x0, 0xcd, 0x87, 0x4a, 0x13, 0xde, 0x94, 0x59, 0x26, 0xeb, 0xa1, 0x6c, 0x35, 0xf8, 0xb2, 0x7f},
- {0x0, 0xfd, 0xe7, 0x1a, 0xd3, 0x2e, 0x34, 0xc9, 0xbb, 0x46, 0x5c, 0xa1, 0x68, 0x95, 0x8f, 0x72},
- {0x0, 0xed, 0xc7, 0x2a, 0x93, 0x7e, 0x54, 0xb9, 0x3b, 0xd6, 0xfc, 0x11, 0xa8, 0x45, 0x6f, 0x82},
- {0x0, 0x3a, 0x74, 0x4e, 0xe8, 0xd2, 0x9c, 0xa6, 0xcd, 0xf7, 0xb9, 0x83, 0x25, 0x1f, 0x51, 0x6b},
- {0x0, 0x2a, 0x54, 0x7e, 0xa8, 0x82, 0xfc, 0xd6, 0x4d, 0x67, 0x19, 0x33, 0xe5, 0xcf, 0xb1, 0x9b},
- {0x0, 0x1a, 0x34, 0x2e, 0x68, 0x72, 0x5c, 0x46, 0xd0, 0xca, 0xe4, 0xfe, 0xb8, 0xa2, 0x8c, 0x96},
- {0x0, 0xa, 0x14, 0x1e, 0x28, 0x22, 0x3c, 0x36, 0x50, 0x5a, 0x44, 0x4e, 0x78, 0x72, 0x6c, 0x66},
- {0x0, 0x7a, 0xf4, 0x8e, 0xf5, 0x8f, 0x1, 0x7b, 0xf7, 0x8d, 0x3, 0x79, 0x2, 0x78, 0xf6, 0x8c},
- {0x0, 0x6a, 0xd4, 0xbe, 0xb5, 0xdf, 0x61, 0xb, 0x77, 0x1d, 0xa3, 0xc9, 0xc2, 0xa8, 0x16, 0x7c},
- {0x0, 0x5a, 0xb4, 0xee, 0x75, 0x2f, 0xc1, 0x9b, 0xea, 0xb0, 0x5e, 0x4, 0x9f, 0xc5, 0x2b, 0x71},
- {0x0, 0x4a, 0x94, 0xde, 0x35, 0x7f, 0xa1, 0xeb, 0x6a, 0x20, 0xfe, 0xb4, 0x5f, 0x15, 0xcb, 0x81},
- {0x0, 0xba, 0x69, 0xd3, 0xd2, 0x68, 0xbb, 0x1, 0xb9, 0x3, 0xd0, 0x6a, 0x6b, 0xd1, 0x2, 0xb8},
- {0x0, 0xaa, 0x49, 0xe3, 0x92, 0x38, 0xdb, 0x71, 0x39, 0x93, 0x70, 0xda, 0xab, 0x1, 0xe2, 0x48},
- {0x0, 0x9a, 0x29, 0xb3, 0x52, 0xc8, 0x7b, 0xe1, 0xa4, 0x3e, 0x8d, 0x17, 0xf6, 0x6c, 0xdf, 0x45},
- {0x0, 0x8a, 0x9, 0x83, 0x12, 0x98, 0x1b, 0x91, 0x24, 0xae, 0x2d, 0xa7, 0x36, 0xbc, 0x3f, 0xb5},
- {0x0, 0xfa, 0xe9, 0x13, 0xcf, 0x35, 0x26, 0xdc, 0x83, 0x79, 0x6a, 0x90, 0x4c, 0xb6, 0xa5, 0x5f},
- {0x0, 0xea, 0xc9, 0x23, 0x8f, 0x65, 0x46, 0xac, 0x3, 0xe9, 0xca, 0x20, 0x8c, 0x66, 0x45, 0xaf},
- {0x0, 0xda, 0xa9, 0x73, 0x4f, 0x95, 0xe6, 0x3c, 0x9e, 0x44, 0x37, 0xed, 0xd1, 0xb, 0x78, 0xa2},
- {0x0, 0xca, 0x89, 0x43, 0xf, 0xc5, 0x86, 0x4c, 0x1e, 0xd4, 0x97, 0x5d, 0x11, 0xdb, 0x98, 0x52},
- {0x0, 0x27, 0x4e, 0x69, 0x9c, 0xbb, 0xd2, 0xf5, 0x25, 0x2, 0x6b, 0x4c, 0xb9, 0x9e, 0xf7, 0xd0},
- {0x0, 0x37, 0x6e, 0x59, 0xdc, 0xeb, 0xb2, 0x85, 0xa5, 0x92, 0xcb, 0xfc, 0x79, 0x4e, 0x17, 0x20},
- {0x0, 0x7, 0xe, 0x9, 0x1c, 0x1b, 0x12, 0x15, 0x38, 0x3f, 0x36, 0x31, 0x24, 0x23, 0x2a, 0x2d},
- {0x0, 0x17, 0x2e, 0x39, 0x5c, 0x4b, 0x72, 0x65, 0xb8, 0xaf, 0x96, 0x81, 0xe4, 0xf3, 0xca, 0xdd},
- {0x0, 0x67, 0xce, 0xa9, 0x81, 0xe6, 0x4f, 0x28, 0x1f, 0x78, 0xd1, 0xb6, 0x9e, 0xf9, 0x50, 0x37},
- {0x0, 0x77, 0xee, 0x99, 0xc1, 0xb6, 0x2f, 0x58, 0x9f, 0xe8, 0x71, 0x6, 0x5e, 0x29, 0xb0, 0xc7},
- {0x0, 0x47, 0x8e, 0xc9, 0x1, 0x46, 0x8f, 0xc8, 0x2, 0x45, 0x8c, 0xcb, 0x3, 0x44, 0x8d, 0xca},
- {0x0, 0x57, 0xae, 0xf9, 0x41, 0x16, 0xef, 0xb8, 0x82, 0xd5, 0x2c, 0x7b, 0xc3, 0x94, 0x6d, 0x3a},
- {0x0, 0xa7, 0x53, 0xf4, 0xa6, 0x1, 0xf5, 0x52, 0x51, 0xf6, 0x2, 0xa5, 0xf7, 0x50, 0xa4, 0x3},
- {0x0, 0xb7, 0x73, 0xc4, 0xe6, 0x51, 0x95, 0x22, 0xd1, 0x66, 0xa2, 0x15, 0x37, 0x80, 0x44, 0xf3},
- {0x0, 0x87, 0x13, 0x94, 0x26, 0xa1, 0x35, 0xb2, 0x4c, 0xcb, 0x5f, 0xd8, 0x6a, 0xed, 0x79, 0xfe},
- {0x0, 0x97, 0x33, 0xa4, 0x66, 0xf1, 0x55, 0xc2, 0xcc, 0x5b, 0xff, 0x68, 0xaa, 0x3d, 0x99, 0xe},
- {0x0, 0xe7, 0xd3, 0x34, 0xbb, 0x5c, 0x68, 0x8f, 0x6b, 0x8c, 0xb8, 0x5f, 0xd0, 0x37, 0x3, 0xe4},
- {0x0, 0xf7, 0xf3, 0x4, 0xfb, 0xc, 0x8, 0xff, 0xeb, 0x1c, 0x18, 0xef, 0x10, 0xe7, 0xe3, 0x14},
- {0x0, 0xc7, 0x93, 0x54, 0x3b, 0xfc, 0xa8, 0x6f, 0x76, 0xb1, 0xe5, 0x22, 0x4d, 0x8a, 0xde, 0x19},
- {0x0, 0xd7, 0xb3, 0x64, 0x7b, 0xac, 0xc8, 0x1f, 0xf6, 0x21, 0x45, 0x92, 0x8d, 0x5a, 0x3e, 0xe9},
- {0x0, 0x74, 0xe8, 0x9c, 0xcd, 0xb9, 0x25, 0x51, 0x87, 0xf3, 0x6f, 0x1b, 0x4a, 0x3e, 0xa2, 0xd6},
- {0x0, 0x64, 0xc8, 0xac, 0x8d, 0xe9, 0x45, 0x21, 0x7, 0x63, 0xcf, 0xab, 0x8a, 0xee, 0x42, 0x26},
- {0x0, 0x54, 0xa8, 0xfc, 0x4d, 0x19, 0xe5, 0xb1, 0x9a, 0xce, 0x32, 0x66, 0xd7, 0x83, 0x7f, 0x2b},
- {0x0, 0x44, 0x88, 0xcc, 0xd, 0x49, 0x85, 0xc1, 0x1a, 0x5e, 0x92, 0xd6, 0x17, 0x53, 0x9f, 0xdb},
- {0x0, 0x34, 0x68, 0x5c, 0xd0, 0xe4, 0xb8, 0x8c, 0xbd, 0x89, 0xd5, 0xe1, 0x6d, 0x59, 0x5, 0x31},
- {0x0, 0x24, 0x48, 0x6c, 0x90, 0xb4, 0xd8, 0xfc, 0x3d, 0x19, 0x75, 0x51, 0xad, 0x89, 0xe5, 0xc1},
- {0x0, 0x14, 0x28, 0x3c, 0x50, 0x44, 0x78, 0x6c, 0xa0, 0xb4, 0x88, 0x9c, 0xf0, 0xe4, 0xd8, 0xcc},
- {0x0, 0x4, 0x8, 0xc, 0x10, 0x14, 0x18, 0x1c, 0x20, 0x24, 0x28, 0x2c, 0x30, 0x34, 0x38, 0x3c},
- {0x0, 0xf4, 0xf5, 0x1, 0xf7, 0x3, 0x2, 0xf6, 0xf3, 0x7, 0x6, 0xf2, 0x4, 0xf0, 0xf1, 0x5},
- {0x0, 0xe4, 0xd5, 0x31, 0xb7, 0x53, 0x62, 0x86, 0x73, 0x97, 0xa6, 0x42, 0xc4, 0x20, 0x11, 0xf5},
- {0x0, 0xd4, 0xb5, 0x61, 0x77, 0xa3, 0xc2, 0x16, 0xee, 0x3a, 0x5b, 0x8f, 0x99, 0x4d, 0x2c, 0xf8},
- {0x0, 0xc4, 0x95, 0x51, 0x37, 0xf3, 0xa2, 0x66, 0x6e, 0xaa, 0xfb, 0x3f, 0x59, 0x9d, 0xcc, 0x8},
- {0x0, 0xb4, 0x75, 0xc1, 0xea, 0x5e, 0x9f, 0x2b, 0xc9, 0x7d, 0xbc, 0x8, 0x23, 0x97, 0x56, 0xe2},
- {0x0, 0xa4, 0x55, 0xf1, 0xaa, 0xe, 0xff, 0x5b, 0x49, 0xed, 0x1c, 0xb8, 0xe3, 0x47, 0xb6, 0x12},
- {0x0, 0x94, 0x35, 0xa1, 0x6a, 0xfe, 0x5f, 0xcb, 0xd4, 0x40, 0xe1, 0x75, 0xbe, 0x2a, 0x8b, 0x1f},
- {0x0, 0x84, 0x15, 0x91, 0x2a, 0xae, 0x3f, 0xbb, 0x54, 0xd0, 0x41, 0xc5, 0x7e, 0xfa, 0x6b, 0xef},
- {0x0, 0x69, 0xd2, 0xbb, 0xb9, 0xd0, 0x6b, 0x2, 0x6f, 0x6, 0xbd, 0xd4, 0xd6, 0xbf, 0x4, 0x6d},
- {0x0, 0x79, 0xf2, 0x8b, 0xf9, 0x80, 0xb, 0x72, 0xef, 0x96, 0x1d, 0x64, 0x16, 0x6f, 0xe4, 0x9d},
- {0x0, 0x49, 0x92, 0xdb, 0x39, 0x70, 0xab, 0xe2, 0x72, 0x3b, 0xe0, 0xa9, 0x4b, 0x2, 0xd9, 0x90},
- {0x0, 0x59, 0xb2, 0xeb, 0x79, 0x20, 0xcb, 0x92, 0xf2, 0xab, 0x40, 0x19, 0x8b, 0xd2, 0x39, 0x60},
- {0x0, 0x29, 0x52, 0x7b, 0xa4, 0x8d, 0xf6, 0xdf, 0x55, 0x7c, 0x7, 0x2e, 0xf1, 0xd8, 0xa3, 0x8a},
- {0x0, 0x39, 0x72, 0x4b, 0xe4, 0xdd, 0x96, 0xaf, 0xd5, 0xec, 0xa7, 0x9e, 0x31, 0x8, 0x43, 0x7a},
- {0x0, 0x9, 0x12, 0x1b, 0x24, 0x2d, 0x36, 0x3f, 0x48, 0x41, 0x5a, 0x53, 0x6c, 0x65, 0x7e, 0x77},
- {0x0, 0x19, 0x32, 0x2b, 0x64, 0x7d, 0x56, 0x4f, 0xc8, 0xd1, 0xfa, 0xe3, 0xac, 0xb5, 0x9e, 0x87},
- {0x0, 0xe9, 0xcf, 0x26, 0x83, 0x6a, 0x4c, 0xa5, 0x1b, 0xf2, 0xd4, 0x3d, 0x98, 0x71, 0x57, 0xbe},
- {0x0, 0xf9, 0xef, 0x16, 0xc3, 0x3a, 0x2c, 0xd5, 0x9b, 0x62, 0x74, 0x8d, 0x58, 0xa1, 0xb7, 0x4e},
- {0x0, 0xc9, 0x8f, 0x46, 0x3, 0xca, 0x8c, 0x45, 0x6, 0xcf, 0x89, 0x40, 0x5, 0xcc, 0x8a, 0x43},
- {0x0, 0xd9, 0xaf, 0x76, 0x43, 0x9a, 0xec, 0x35, 0x86, 0x5f, 0x29, 0xf0, 0xc5, 0x1c, 0x6a, 0xb3},
- {0x0, 0xa9, 0x4f, 0xe6, 0x9e, 0x37, 0xd1, 0x78, 0x21, 0x88, 0x6e, 0xc7, 0xbf, 0x16, 0xf0, 0x59},
- {0x0, 0xb9, 0x6f, 0xd6, 0xde, 0x67, 0xb1, 0x8, 0xa1, 0x18, 0xce, 0x77, 0x7f, 0xc6, 0x10, 0xa9},
- {0x0, 0x89, 0xf, 0x86, 0x1e, 0x97, 0x11, 0x98, 0x3c, 0xb5, 0x33, 0xba, 0x22, 0xab, 0x2d, 0xa4},
- {0x0, 0x99, 0x2f, 0xb6, 0x5e, 0xc7, 0x71, 0xe8, 0xbc, 0x25, 0x93, 0xa, 0xe2, 0x7b, 0xcd, 0x54},
- {0x0, 0x4e, 0x9c, 0xd2, 0x25, 0x6b, 0xb9, 0xf7, 0x4a, 0x4, 0xd6, 0x98, 0x6f, 0x21, 0xf3, 0xbd},
- {0x0, 0x5e, 0xbc, 0xe2, 0x65, 0x3b, 0xd9, 0x87, 0xca, 0x94, 0x76, 0x28, 0xaf, 0xf1, 0x13, 0x4d},
- {0x0, 0x6e, 0xdc, 0xb2, 0xa5, 0xcb, 0x79, 0x17, 0x57, 0x39, 0x8b, 0xe5, 0xf2, 0x9c, 0x2e, 0x40},
- {0x0, 0x7e, 0xfc, 0x82, 0xe5, 0x9b, 0x19, 0x67, 0xd7, 0xa9, 0x2b, 0x55, 0x32, 0x4c, 0xce, 0xb0},
- {0x0, 0xe, 0x1c, 0x12, 0x38, 0x36, 0x24, 0x2a, 0x70, 0x7e, 0x6c, 0x62, 0x48, 0x46, 0x54, 0x5a},
- {0x0, 0x1e, 0x3c, 0x22, 0x78, 0x66, 0x44, 0x5a, 0xf0, 0xee, 0xcc, 0xd2, 0x88, 0x96, 0xb4, 0xaa},
- {0x0, 0x2e, 0x5c, 0x72, 0xb8, 0x96, 0xe4, 0xca, 0x6d, 0x43, 0x31, 0x1f, 0xd5, 0xfb, 0x89, 0xa7},
- {0x0, 0x3e, 0x7c, 0x42, 0xf8, 0xc6, 0x84, 0xba, 0xed, 0xd3, 0x91, 0xaf, 0x15, 0x2b, 0x69, 0x57},
- {0x0, 0xce, 0x81, 0x4f, 0x1f, 0xd1, 0x9e, 0x50, 0x3e, 0xf0, 0xbf, 0x71, 0x21, 0xef, 0xa0, 0x6e},
- {0x0, 0xde, 0xa1, 0x7f, 0x5f, 0x81, 0xfe, 0x20, 0xbe, 0x60, 0x1f, 0xc1, 0xe1, 0x3f, 0x40, 0x9e},
- {0x0, 0xee, 0xc1, 0x2f, 0x9f, 0x71, 0x5e, 0xb0, 0x23, 0xcd, 0xe2, 0xc, 0xbc, 0x52, 0x7d, 0x93},
- {0x0, 0xfe, 0xe1, 0x1f, 0xdf, 0x21, 0x3e, 0xc0, 0xa3, 0x5d, 0x42, 0xbc, 0x7c, 0x82, 0x9d, 0x63},
- {0x0, 0x8e, 0x1, 0x8f, 0x2, 0x8c, 0x3, 0x8d, 0x4, 0x8a, 0x5, 0x8b, 0x6, 0x88, 0x7, 0x89},
- {0x0, 0x9e, 0x21, 0xbf, 0x42, 0xdc, 0x63, 0xfd, 0x84, 0x1a, 0xa5, 0x3b, 0xc6, 0x58, 0xe7, 0x79},
- {0x0, 0xae, 0x41, 0xef, 0x82, 0x2c, 0xc3, 0x6d, 0x19, 0xb7, 0x58, 0xf6, 0x9b, 0x35, 0xda, 0x74},
- {0x0, 0xbe, 0x61, 0xdf, 0xc2, 0x7c, 0xa3, 0x1d, 0x99, 0x27, 0xf8, 0x46, 0x5b, 0xe5, 0x3a, 0x84},
- {0x0, 0x53, 0xa6, 0xf5, 0x51, 0x2, 0xf7, 0xa4, 0xa2, 0xf1, 0x4, 0x57, 0xf3, 0xa0, 0x55, 0x6},
- {0x0, 0x43, 0x86, 0xc5, 0x11, 0x52, 0x97, 0xd4, 0x22, 0x61, 0xa4, 0xe7, 0x33, 0x70, 0xb5, 0xf6},
- {0x0, 0x73, 0xe6, 0x95, 0xd1, 0xa2, 0x37, 0x44, 0xbf, 0xcc, 0x59, 0x2a, 0x6e, 0x1d, 0x88, 0xfb},
- {0x0, 0x63, 0xc6, 0xa5, 0x91, 0xf2, 0x57, 0x34, 0x3f, 0x5c, 0xf9, 0x9a, 0xae, 0xcd, 0x68, 0xb},
- {0x0, 0x13, 0x26, 0x35, 0x4c, 0x5f, 0x6a, 0x79, 0x98, 0x8b, 0xbe, 0xad, 0xd4, 0xc7, 0xf2, 0xe1},
- {0x0, 0x3, 0x6, 0x5, 0xc, 0xf, 0xa, 0x9, 0x18, 0x1b, 0x1e, 0x1d, 0x14, 0x17, 0x12, 0x11},
- {0x0, 0x33, 0x66, 0x55, 0xcc, 0xff, 0xaa, 0x99, 0x85, 0xb6, 0xe3, 0xd0, 0x49, 0x7a, 0x2f, 0x1c},
- {0x0, 0x23, 0x46, 0x65, 0x8c, 0xaf, 0xca, 0xe9, 0x5, 0x26, 0x43, 0x60, 0x89, 0xaa, 0xcf, 0xec},
- {0x0, 0xd3, 0xbb, 0x68, 0x6b, 0xb8, 0xd0, 0x3, 0xd6, 0x5, 0x6d, 0xbe, 0xbd, 0x6e, 0x6, 0xd5},
- {0x0, 0xc3, 0x9b, 0x58, 0x2b, 0xe8, 0xb0, 0x73, 0x56, 0x95, 0xcd, 0xe, 0x7d, 0xbe, 0xe6, 0x25},
- {0x0, 0xf3, 0xfb, 0x8, 0xeb, 0x18, 0x10, 0xe3, 0xcb, 0x38, 0x30, 0xc3, 0x20, 0xd3, 0xdb, 0x28},
- {0x0, 0xe3, 0xdb, 0x38, 0xab, 0x48, 0x70, 0x93, 0x4b, 0xa8, 0x90, 0x73, 0xe0, 0x3, 0x3b, 0xd8},
- {0x0, 0x93, 0x3b, 0xa8, 0x76, 0xe5, 0x4d, 0xde, 0xec, 0x7f, 0xd7, 0x44, 0x9a, 0x9, 0xa1, 0x32},
- {0x0, 0x83, 0x1b, 0x98, 0x36, 0xb5, 0x2d, 0xae, 0x6c, 0xef, 0x77, 0xf4, 0x5a, 0xd9, 0x41, 0xc2},
- {0x0, 0xb3, 0x7b, 0xc8, 0xf6, 0x45, 0x8d, 0x3e, 0xf1, 0x42, 0x8a, 0x39, 0x7, 0xb4, 0x7c, 0xcf},
- {0x0, 0xa3, 0x5b, 0xf8, 0xb6, 0x15, 0xed, 0x4e, 0x71, 0xd2, 0x2a, 0x89, 0xc7, 0x64, 0x9c, 0x3f},
- {0x0, 0xe8, 0xcd, 0x25, 0x87, 0x6f, 0x4a, 0xa2, 0x13, 0xfb, 0xde, 0x36, 0x94, 0x7c, 0x59, 0xb1},
- {0x0, 0xf8, 0xed, 0x15, 0xc7, 0x3f, 0x2a, 0xd2, 0x93, 0x6b, 0x7e, 0x86, 0x54, 0xac, 0xb9, 0x41},
- {0x0, 0xc8, 0x8d, 0x45, 0x7, 0xcf, 0x8a, 0x42, 0xe, 0xc6, 0x83, 0x4b, 0x9, 0xc1, 0x84, 0x4c},
- {0x0, 0xd8, 0xad, 0x75, 0x47, 0x9f, 0xea, 0x32, 0x8e, 0x56, 0x23, 0xfb, 0xc9, 0x11, 0x64, 0xbc},
- {0x0, 0xa8, 0x4d, 0xe5, 0x9a, 0x32, 0xd7, 0x7f, 0x29, 0x81, 0x64, 0xcc, 0xb3, 0x1b, 0xfe, 0x56},
- {0x0, 0xb8, 0x6d, 0xd5, 0xda, 0x62, 0xb7, 0xf, 0xa9, 0x11, 0xc4, 0x7c, 0x73, 0xcb, 0x1e, 0xa6},
- {0x0, 0x88, 0xd, 0x85, 0x1a, 0x92, 0x17, 0x9f, 0x34, 0xbc, 0x39, 0xb1, 0x2e, 0xa6, 0x23, 0xab},
- {0x0, 0x98, 0x2d, 0xb5, 0x5a, 0xc2, 0x77, 0xef, 0xb4, 0x2c, 0x99, 0x1, 0xee, 0x76, 0xc3, 0x5b},
- {0x0, 0x68, 0xd0, 0xb8, 0xbd, 0xd5, 0x6d, 0x5, 0x67, 0xf, 0xb7, 0xdf, 0xda, 0xb2, 0xa, 0x62},
- {0x0, 0x78, 0xf0, 0x88, 0xfd, 0x85, 0xd, 0x75, 0xe7, 0x9f, 0x17, 0x6f, 0x1a, 0x62, 0xea, 0x92},
- {0x0, 0x48, 0x90, 0xd8, 0x3d, 0x75, 0xad, 0xe5, 0x7a, 0x32, 0xea, 0xa2, 0x47, 0xf, 0xd7, 0x9f},
- {0x0, 0x58, 0xb0, 0xe8, 0x7d, 0x25, 0xcd, 0x95, 0xfa, 0xa2, 0x4a, 0x12, 0x87, 0xdf, 0x37, 0x6f},
- {0x0, 0x28, 0x50, 0x78, 0xa0, 0x88, 0xf0, 0xd8, 0x5d, 0x75, 0xd, 0x25, 0xfd, 0xd5, 0xad, 0x85},
- {0x0, 0x38, 0x70, 0x48, 0xe0, 0xd8, 0x90, 0xa8, 0xdd, 0xe5, 0xad, 0x95, 0x3d, 0x5, 0x4d, 0x75},
- {0x0, 0x8, 0x10, 0x18, 0x20, 0x28, 0x30, 0x38, 0x40, 0x48, 0x50, 0x58, 0x60, 0x68, 0x70, 0x78},
- {0x0, 0x18, 0x30, 0x28, 0x60, 0x78, 0x50, 0x48, 0xc0, 0xd8, 0xf0, 0xe8, 0xa0, 0xb8, 0x90, 0x88},
- {0x0, 0xf5, 0xf7, 0x2, 0xf3, 0x6, 0x4, 0xf1, 0xfb, 0xe, 0xc, 0xf9, 0x8, 0xfd, 0xff, 0xa},
- {0x0, 0xe5, 0xd7, 0x32, 0xb3, 0x56, 0x64, 0x81, 0x7b, 0x9e, 0xac, 0x49, 0xc8, 0x2d, 0x1f, 0xfa},
- {0x0, 0xd5, 0xb7, 0x62, 0x73, 0xa6, 0xc4, 0x11, 0xe6, 0x33, 0x51, 0x84, 0x95, 0x40, 0x22, 0xf7},
- {0x0, 0xc5, 0x97, 0x52, 0x33, 0xf6, 0xa4, 0x61, 0x66, 0xa3, 0xf1, 0x34, 0x55, 0x90, 0xc2, 0x7},
- {0x0, 0xb5, 0x77, 0xc2, 0xee, 0x5b, 0x99, 0x2c, 0xc1, 0x74, 0xb6, 0x3, 0x2f, 0x9a, 0x58, 0xed},
- {0x0, 0xa5, 0x57, 0xf2, 0xae, 0xb, 0xf9, 0x5c, 0x41, 0xe4, 0x16, 0xb3, 0xef, 0x4a, 0xb8, 0x1d},
- {0x0, 0x95, 0x37, 0xa2, 0x6e, 0xfb, 0x59, 0xcc, 0xdc, 0x49, 0xeb, 0x7e, 0xb2, 0x27, 0x85, 0x10},
- {0x0, 0x85, 0x17, 0x92, 0x2e, 0xab, 0x39, 0xbc, 0x5c, 0xd9, 0x4b, 0xce, 0x72, 0xf7, 0x65, 0xe0},
- {0x0, 0x75, 0xea, 0x9f, 0xc9, 0xbc, 0x23, 0x56, 0x8f, 0xfa, 0x65, 0x10, 0x46, 0x33, 0xac, 0xd9},
- {0x0, 0x65, 0xca, 0xaf, 0x89, 0xec, 0x43, 0x26, 0xf, 0x6a, 0xc5, 0xa0, 0x86, 0xe3, 0x4c, 0x29},
- {0x0, 0x55, 0xaa, 0xff, 0x49, 0x1c, 0xe3, 0xb6, 0x92, 0xc7, 0x38, 0x6d, 0xdb, 0x8e, 0x71, 0x24},
- {0x0, 0x45, 0x8a, 0xcf, 0x9, 0x4c, 0x83, 0xc6, 0x12, 0x57, 0x98, 0xdd, 0x1b, 0x5e, 0x91, 0xd4},
- {0x0, 0x35, 0x6a, 0x5f, 0xd4, 0xe1, 0xbe, 0x8b, 0xb5, 0x80, 0xdf, 0xea, 0x61, 0x54, 0xb, 0x3e},
- {0x0, 0x25, 0x4a, 0x6f, 0x94, 0xb1, 0xde, 0xfb, 0x35, 0x10, 0x7f, 0x5a, 0xa1, 0x84, 0xeb, 0xce},
- {0x0, 0x15, 0x2a, 0x3f, 0x54, 0x41, 0x7e, 0x6b, 0xa8, 0xbd, 0x82, 0x97, 0xfc, 0xe9, 0xd6, 0xc3},
- {0x0, 0x5, 0xa, 0xf, 0x14, 0x11, 0x1e, 0x1b, 0x28, 0x2d, 0x22, 0x27, 0x3c, 0x39, 0x36, 0x33},
- {0x0, 0xd2, 0xb9, 0x6b, 0x6f, 0xbd, 0xd6, 0x4, 0xde, 0xc, 0x67, 0xb5, 0xb1, 0x63, 0x8, 0xda},
- {0x0, 0xc2, 0x99, 0x5b, 0x2f, 0xed, 0xb6, 0x74, 0x5e, 0x9c, 0xc7, 0x5, 0x71, 0xb3, 0xe8, 0x2a},
- {0x0, 0xf2, 0xf9, 0xb, 0xef, 0x1d, 0x16, 0xe4, 0xc3, 0x31, 0x3a, 0xc8, 0x2c, 0xde, 0xd5, 0x27},
- {0x0, 0xe2, 0xd9, 0x3b, 0xaf, 0x4d, 0x76, 0x94, 0x43, 0xa1, 0x9a, 0x78, 0xec, 0xe, 0x35, 0xd7},
- {0x0, 0x92, 0x39, 0xab, 0x72, 0xe0, 0x4b, 0xd9, 0xe4, 0x76, 0xdd, 0x4f, 0x96, 0x4, 0xaf, 0x3d},
- {0x0, 0x82, 0x19, 0x9b, 0x32, 0xb0, 0x2b, 0xa9, 0x64, 0xe6, 0x7d, 0xff, 0x56, 0xd4, 0x4f, 0xcd},
- {0x0, 0xb2, 0x79, 0xcb, 0xf2, 0x40, 0x8b, 0x39, 0xf9, 0x4b, 0x80, 0x32, 0xb, 0xb9, 0x72, 0xc0},
- {0x0, 0xa2, 0x59, 0xfb, 0xb2, 0x10, 0xeb, 0x49, 0x79, 0xdb, 0x20, 0x82, 0xcb, 0x69, 0x92, 0x30},
- {0x0, 0x52, 0xa4, 0xf6, 0x55, 0x7, 0xf1, 0xa3, 0xaa, 0xf8, 0xe, 0x5c, 0xff, 0xad, 0x5b, 0x9},
- {0x0, 0x42, 0x84, 0xc6, 0x15, 0x57, 0x91, 0xd3, 0x2a, 0x68, 0xae, 0xec, 0x3f, 0x7d, 0xbb, 0xf9},
- {0x0, 0x72, 0xe4, 0x96, 0xd5, 0xa7, 0x31, 0x43, 0xb7, 0xc5, 0x53, 0x21, 0x62, 0x10, 0x86, 0xf4},
- {0x0, 0x62, 0xc4, 0xa6, 0x95, 0xf7, 0x51, 0x33, 0x37, 0x55, 0xf3, 0x91, 0xa2, 0xc0, 0x66, 0x4},
- {0x0, 0x12, 0x24, 0x36, 0x48, 0x5a, 0x6c, 0x7e, 0x90, 0x82, 0xb4, 0xa6, 0xd8, 0xca, 0xfc, 0xee},
- {0x0, 0x2, 0x4, 0x6, 0x8, 0xa, 0xc, 0xe, 0x10, 0x12, 0x14, 0x16, 0x18, 0x1a, 0x1c, 0x1e},
- {0x0, 0x32, 0x64, 0x56, 0xc8, 0xfa, 0xac, 0x9e, 0x8d, 0xbf, 0xe9, 0xdb, 0x45, 0x77, 0x21, 0x13},
- {0x0, 0x22, 0x44, 0x66, 0x88, 0xaa, 0xcc, 0xee, 0xd, 0x2f, 0x49, 0x6b, 0x85, 0xa7, 0xc1, 0xe3},
- {0x0, 0xcf, 0x83, 0x4c, 0x1b, 0xd4, 0x98, 0x57, 0x36, 0xf9, 0xb5, 0x7a, 0x2d, 0xe2, 0xae, 0x61},
- {0x0, 0xdf, 0xa3, 0x7c, 0x5b, 0x84, 0xf8, 0x27, 0xb6, 0x69, 0x15, 0xca, 0xed, 0x32, 0x4e, 0x91},
- {0x0, 0xef, 0xc3, 0x2c, 0x9b, 0x74, 0x58, 0xb7, 0x2b, 0xc4, 0xe8, 0x7, 0xb0, 0x5f, 0x73, 0x9c},
- {0x0, 0xff, 0xe3, 0x1c, 0xdb, 0x24, 0x38, 0xc7, 0xab, 0x54, 0x48, 0xb7, 0x70, 0x8f, 0x93, 0x6c},
- {0x0, 0x8f, 0x3, 0x8c, 0x6, 0x89, 0x5, 0x8a, 0xc, 0x83, 0xf, 0x80, 0xa, 0x85, 0x9, 0x86},
- {0x0, 0x9f, 0x23, 0xbc, 0x46, 0xd9, 0x65, 0xfa, 0x8c, 0x13, 0xaf, 0x30, 0xca, 0x55, 0xe9, 0x76},
- {0x0, 0xaf, 0x43, 0xec, 0x86, 0x29, 0xc5, 0x6a, 0x11, 0xbe, 0x52, 0xfd, 0x97, 0x38, 0xd4, 0x7b},
- {0x0, 0xbf, 0x63, 0xdc, 0xc6, 0x79, 0xa5, 0x1a, 0x91, 0x2e, 0xf2, 0x4d, 0x57, 0xe8, 0x34, 0x8b},
- {0x0, 0x4f, 0x9e, 0xd1, 0x21, 0x6e, 0xbf, 0xf0, 0x42, 0xd, 0xdc, 0x93, 0x63, 0x2c, 0xfd, 0xb2},
- {0x0, 0x5f, 0xbe, 0xe1, 0x61, 0x3e, 0xdf, 0x80, 0xc2, 0x9d, 0x7c, 0x23, 0xa3, 0xfc, 0x1d, 0x42},
- {0x0, 0x6f, 0xde, 0xb1, 0xa1, 0xce, 0x7f, 0x10, 0x5f, 0x30, 0x81, 0xee, 0xfe, 0x91, 0x20, 0x4f},
- {0x0, 0x7f, 0xfe, 0x81, 0xe1, 0x9e, 0x1f, 0x60, 0xdf, 0xa0, 0x21, 0x5e, 0x3e, 0x41, 0xc0, 0xbf},
- {0x0, 0xf, 0x1e, 0x11, 0x3c, 0x33, 0x22, 0x2d, 0x78, 0x77, 0x66, 0x69, 0x44, 0x4b, 0x5a, 0x55},
- {0x0, 0x1f, 0x3e, 0x21, 0x7c, 0x63, 0x42, 0x5d, 0xf8, 0xe7, 0xc6, 0xd9, 0x84, 0x9b, 0xba, 0xa5},
- {0x0, 0x2f, 0x5e, 0x71, 0xbc, 0x93, 0xe2, 0xcd, 0x65, 0x4a, 0x3b, 0x14, 0xd9, 0xf6, 0x87, 0xa8},
- {0x0, 0x3f, 0x7e, 0x41, 0xfc, 0xc3, 0x82, 0xbd, 0xe5, 0xda, 0x9b, 0xa4, 0x19, 0x26, 0x67, 0x58},
- {0x0, 0x9c, 0x25, 0xb9, 0x4a, 0xd6, 0x6f, 0xf3, 0x94, 0x8, 0xb1, 0x2d, 0xde, 0x42, 0xfb, 0x67},
- {0x0, 0x8c, 0x5, 0x89, 0xa, 0x86, 0xf, 0x83, 0x14, 0x98, 0x11, 0x9d, 0x1e, 0x92, 0x1b, 0x97},
- {0x0, 0xbc, 0x65, 0xd9, 0xca, 0x76, 0xaf, 0x13, 0x89, 0x35, 0xec, 0x50, 0x43, 0xff, 0x26, 0x9a},
- {0x0, 0xac, 0x45, 0xe9, 0x8a, 0x26, 0xcf, 0x63, 0x9, 0xa5, 0x4c, 0xe0, 0x83, 0x2f, 0xc6, 0x6a},
- {0x0, 0xdc, 0xa5, 0x79, 0x57, 0x8b, 0xf2, 0x2e, 0xae, 0x72, 0xb, 0xd7, 0xf9, 0x25, 0x5c, 0x80},
- {0x0, 0xcc, 0x85, 0x49, 0x17, 0xdb, 0x92, 0x5e, 0x2e, 0xe2, 0xab, 0x67, 0x39, 0xf5, 0xbc, 0x70},
- {0x0, 0xfc, 0xe5, 0x19, 0xd7, 0x2b, 0x32, 0xce, 0xb3, 0x4f, 0x56, 0xaa, 0x64, 0x98, 0x81, 0x7d},
- {0x0, 0xec, 0xc5, 0x29, 0x97, 0x7b, 0x52, 0xbe, 0x33, 0xdf, 0xf6, 0x1a, 0xa4, 0x48, 0x61, 0x8d},
- {0x0, 0x1c, 0x38, 0x24, 0x70, 0x6c, 0x48, 0x54, 0xe0, 0xfc, 0xd8, 0xc4, 0x90, 0x8c, 0xa8, 0xb4},
- {0x0, 0xc, 0x18, 0x14, 0x30, 0x3c, 0x28, 0x24, 0x60, 0x6c, 0x78, 0x74, 0x50, 0x5c, 0x48, 0x44},
- {0x0, 0x3c, 0x78, 0x44, 0xf0, 0xcc, 0x88, 0xb4, 0xfd, 0xc1, 0x85, 0xb9, 0xd, 0x31, 0x75, 0x49},
- {0x0, 0x2c, 0x58, 0x74, 0xb0, 0x9c, 0xe8, 0xc4, 0x7d, 0x51, 0x25, 0x9, 0xcd, 0xe1, 0x95, 0xb9},
- {0x0, 0x5c, 0xb8, 0xe4, 0x6d, 0x31, 0xd5, 0x89, 0xda, 0x86, 0x62, 0x3e, 0xb7, 0xeb, 0xf, 0x53},
- {0x0, 0x4c, 0x98, 0xd4, 0x2d, 0x61, 0xb5, 0xf9, 0x5a, 0x16, 0xc2, 0x8e, 0x77, 0x3b, 0xef, 0xa3},
- {0x0, 0x7c, 0xf8, 0x84, 0xed, 0x91, 0x15, 0x69, 0xc7, 0xbb, 0x3f, 0x43, 0x2a, 0x56, 0xd2, 0xae},
- {0x0, 0x6c, 0xd8, 0xb4, 0xad, 0xc1, 0x75, 0x19, 0x47, 0x2b, 0x9f, 0xf3, 0xea, 0x86, 0x32, 0x5e},
- {0x0, 0x81, 0x1f, 0x9e, 0x3e, 0xbf, 0x21, 0xa0, 0x7c, 0xfd, 0x63, 0xe2, 0x42, 0xc3, 0x5d, 0xdc},
- {0x0, 0x91, 0x3f, 0xae, 0x7e, 0xef, 0x41, 0xd0, 0xfc, 0x6d, 0xc3, 0x52, 0x82, 0x13, 0xbd, 0x2c},
- {0x0, 0xa1, 0x5f, 0xfe, 0xbe, 0x1f, 0xe1, 0x40, 0x61, 0xc0, 0x3e, 0x9f, 0xdf, 0x7e, 0x80, 0x21},
- {0x0, 0xb1, 0x7f, 0xce, 0xfe, 0x4f, 0x81, 0x30, 0xe1, 0x50, 0x9e, 0x2f, 0x1f, 0xae, 0x60, 0xd1},
- {0x0, 0xc1, 0x9f, 0x5e, 0x23, 0xe2, 0xbc, 0x7d, 0x46, 0x87, 0xd9, 0x18, 0x65, 0xa4, 0xfa, 0x3b},
- {0x0, 0xd1, 0xbf, 0x6e, 0x63, 0xb2, 0xdc, 0xd, 0xc6, 0x17, 0x79, 0xa8, 0xa5, 0x74, 0x1a, 0xcb},
- {0x0, 0xe1, 0xdf, 0x3e, 0xa3, 0x42, 0x7c, 0x9d, 0x5b, 0xba, 0x84, 0x65, 0xf8, 0x19, 0x27, 0xc6},
- {0x0, 0xf1, 0xff, 0xe, 0xe3, 0x12, 0x1c, 0xed, 0xdb, 0x2a, 0x24, 0xd5, 0x38, 0xc9, 0xc7, 0x36},
- {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf},
- {0x0, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff},
- {0x0, 0x21, 0x42, 0x63, 0x84, 0xa5, 0xc6, 0xe7, 0x15, 0x34, 0x57, 0x76, 0x91, 0xb0, 0xd3, 0xf2},
- {0x0, 0x31, 0x62, 0x53, 0xc4, 0xf5, 0xa6, 0x97, 0x95, 0xa4, 0xf7, 0xc6, 0x51, 0x60, 0x33, 0x2},
- {0x0, 0x41, 0x82, 0xc3, 0x19, 0x58, 0x9b, 0xda, 0x32, 0x73, 0xb0, 0xf1, 0x2b, 0x6a, 0xa9, 0xe8},
- {0x0, 0x51, 0xa2, 0xf3, 0x59, 0x8, 0xfb, 0xaa, 0xb2, 0xe3, 0x10, 0x41, 0xeb, 0xba, 0x49, 0x18},
- {0x0, 0x61, 0xc2, 0xa3, 0x99, 0xf8, 0x5b, 0x3a, 0x2f, 0x4e, 0xed, 0x8c, 0xb6, 0xd7, 0x74, 0x15},
- {0x0, 0x71, 0xe2, 0x93, 0xd9, 0xa8, 0x3b, 0x4a, 0xaf, 0xde, 0x4d, 0x3c, 0x76, 0x7, 0x94, 0xe5},
- {0x0, 0xa6, 0x51, 0xf7, 0xa2, 0x4, 0xf3, 0x55, 0x59, 0xff, 0x8, 0xae, 0xfb, 0x5d, 0xaa, 0xc},
- {0x0, 0xb6, 0x71, 0xc7, 0xe2, 0x54, 0x93, 0x25, 0xd9, 0x6f, 0xa8, 0x1e, 0x3b, 0x8d, 0x4a, 0xfc},
- {0x0, 0x86, 0x11, 0x97, 0x22, 0xa4, 0x33, 0xb5, 0x44, 0xc2, 0x55, 0xd3, 0x66, 0xe0, 0x77, 0xf1},
- {0x0, 0x96, 0x31, 0xa7, 0x62, 0xf4, 0x53, 0xc5, 0xc4, 0x52, 0xf5, 0x63, 0xa6, 0x30, 0x97, 0x1},
- {0x0, 0xe6, 0xd1, 0x37, 0xbf, 0x59, 0x6e, 0x88, 0x63, 0x85, 0xb2, 0x54, 0xdc, 0x3a, 0xd, 0xeb},
- {0x0, 0xf6, 0xf1, 0x7, 0xff, 0x9, 0xe, 0xf8, 0xe3, 0x15, 0x12, 0xe4, 0x1c, 0xea, 0xed, 0x1b},
- {0x0, 0xc6, 0x91, 0x57, 0x3f, 0xf9, 0xae, 0x68, 0x7e, 0xb8, 0xef, 0x29, 0x41, 0x87, 0xd0, 0x16},
- {0x0, 0xd6, 0xb1, 0x67, 0x7f, 0xa9, 0xce, 0x18, 0xfe, 0x28, 0x4f, 0x99, 0x81, 0x57, 0x30, 0xe6},
- {0x0, 0x26, 0x4c, 0x6a, 0x98, 0xbe, 0xd4, 0xf2, 0x2d, 0xb, 0x61, 0x47, 0xb5, 0x93, 0xf9, 0xdf},
- {0x0, 0x36, 0x6c, 0x5a, 0xd8, 0xee, 0xb4, 0x82, 0xad, 0x9b, 0xc1, 0xf7, 0x75, 0x43, 0x19, 0x2f},
- {0x0, 0x6, 0xc, 0xa, 0x18, 0x1e, 0x14, 0x12, 0x30, 0x36, 0x3c, 0x3a, 0x28, 0x2e, 0x24, 0x22},
- {0x0, 0x16, 0x2c, 0x3a, 0x58, 0x4e, 0x74, 0x62, 0xb0, 0xa6, 0x9c, 0x8a, 0xe8, 0xfe, 0xc4, 0xd2},
- {0x0, 0x66, 0xcc, 0xaa, 0x85, 0xe3, 0x49, 0x2f, 0x17, 0x71, 0xdb, 0xbd, 0x92, 0xf4, 0x5e, 0x38},
- {0x0, 0x76, 0xec, 0x9a, 0xc5, 0xb3, 0x29, 0x5f, 0x97, 0xe1, 0x7b, 0xd, 0x52, 0x24, 0xbe, 0xc8},
- {0x0, 0x46, 0x8c, 0xca, 0x5, 0x43, 0x89, 0xcf, 0xa, 0x4c, 0x86, 0xc0, 0xf, 0x49, 0x83, 0xc5},
- {0x0, 0x56, 0xac, 0xfa, 0x45, 0x13, 0xe9, 0xbf, 0x8a, 0xdc, 0x26, 0x70, 0xcf, 0x99, 0x63, 0x35},
- {0x0, 0xbb, 0x6b, 0xd0, 0xd6, 0x6d, 0xbd, 0x6, 0xb1, 0xa, 0xda, 0x61, 0x67, 0xdc, 0xc, 0xb7},
- {0x0, 0xab, 0x4b, 0xe0, 0x96, 0x3d, 0xdd, 0x76, 0x31, 0x9a, 0x7a, 0xd1, 0xa7, 0xc, 0xec, 0x47},
- {0x0, 0x9b, 0x2b, 0xb0, 0x56, 0xcd, 0x7d, 0xe6, 0xac, 0x37, 0x87, 0x1c, 0xfa, 0x61, 0xd1, 0x4a},
- {0x0, 0x8b, 0xb, 0x80, 0x16, 0x9d, 0x1d, 0x96, 0x2c, 0xa7, 0x27, 0xac, 0x3a, 0xb1, 0x31, 0xba},
- {0x0, 0xfb, 0xeb, 0x10, 0xcb, 0x30, 0x20, 0xdb, 0x8b, 0x70, 0x60, 0x9b, 0x40, 0xbb, 0xab, 0x50},
- {0x0, 0xeb, 0xcb, 0x20, 0x8b, 0x60, 0x40, 0xab, 0xb, 0xe0, 0xc0, 0x2b, 0x80, 0x6b, 0x4b, 0xa0},
- {0x0, 0xdb, 0xab, 0x70, 0x4b, 0x90, 0xe0, 0x3b, 0x96, 0x4d, 0x3d, 0xe6, 0xdd, 0x6, 0x76, 0xad},
- {0x0, 0xcb, 0x8b, 0x40, 0xb, 0xc0, 0x80, 0x4b, 0x16, 0xdd, 0x9d, 0x56, 0x1d, 0xd6, 0x96, 0x5d},
- {0x0, 0x3b, 0x76, 0x4d, 0xec, 0xd7, 0x9a, 0xa1, 0xc5, 0xfe, 0xb3, 0x88, 0x29, 0x12, 0x5f, 0x64},
- {0x0, 0x2b, 0x56, 0x7d, 0xac, 0x87, 0xfa, 0xd1, 0x45, 0x6e, 0x13, 0x38, 0xe9, 0xc2, 0xbf, 0x94},
- {0x0, 0x1b, 0x36, 0x2d, 0x6c, 0x77, 0x5a, 0x41, 0xd8, 0xc3, 0xee, 0xf5, 0xb4, 0xaf, 0x82, 0x99},
- {0x0, 0xb, 0x16, 0x1d, 0x2c, 0x27, 0x3a, 0x31, 0x58, 0x53, 0x4e, 0x45, 0x74, 0x7f, 0x62, 0x69},
- {0x0, 0x7b, 0xf6, 0x8d, 0xf1, 0x8a, 0x7, 0x7c, 0xff, 0x84, 0x9, 0x72, 0xe, 0x75, 0xf8, 0x83},
- {0x0, 0x6b, 0xd6, 0xbd, 0xb1, 0xda, 0x67, 0xc, 0x7f, 0x14, 0xa9, 0xc2, 0xce, 0xa5, 0x18, 0x73},
- {0x0, 0x5b, 0xb6, 0xed, 0x71, 0x2a, 0xc7, 0x9c, 0xe2, 0xb9, 0x54, 0xf, 0x93, 0xc8, 0x25, 0x7e},
- {0x0, 0x4b, 0x96, 0xdd, 0x31, 0x7a, 0xa7, 0xec, 0x62, 0x29, 0xf4, 0xbf, 0x53, 0x18, 0xc5, 0x8e}}
-
-// galMultiply multiplies to elements of the field.
-// Uses lookup table ~40% faster
-func galMultiply(a, b byte) byte {
- return mulTable[a][b]
-}
-
-// Original function:
-/*
-// galMultiply multiplies to elements of the field.
-func galMultiply(a, b byte) byte {
- if a == 0 || b == 0 {
- return 0
- }
- logA := int(logTable[a])
- logB := int(logTable[b])
- return expTable[logA+logB]
-}
-*/
-
-// galDivide is inverse of galMultiply.
-func galDivide(a, b byte) byte {
- if a == 0 {
- return 0
- }
- if b == 0 {
- panic("Argument 'divisor' is 0")
- }
- logA := int(logTable[a])
- logB := int(logTable[b])
- logResult := logA - logB
- if logResult < 0 {
- logResult += 255
- }
- return expTable[logResult]
-}
-
-// Computes a**n.
-//
-// The result will be the same as multiplying a times itself n times.
-func galExp(a byte, n int) byte {
- if n == 0 {
- return 1
- }
- if a == 0 {
- return 0
- }
-
- logA := logTable[a]
- logResult := int(logA) * n
- for logResult >= 255 {
- logResult -= 255
- }
- return expTable[logResult]
-}
-
-func genAvx2Matrix(matrixRows [][]byte, inputs, inIdx, outputs int, dst []byte) []byte {
- if !avx2CodeGen {
- panic("codegen not enabled")
- }
- total := inputs * outputs
-
- // Duplicated in+out
- wantBytes := total * 32 * 2
- if cap(dst) < wantBytes {
- dst = make([]byte, wantBytes)
- } else {
- dst = dst[:wantBytes]
- }
- for i, row := range matrixRows[:outputs] {
- for j, idx := range row[inIdx : inIdx+inputs] {
- dstIdx := (j*outputs + i) * 64
- dstPart := dst[dstIdx:]
- dstPart = dstPart[:64]
- lo := mulTableLow[idx][:]
- hi := mulTableHigh[idx][:]
- copy(dstPart[:16], lo)
- copy(dstPart[16:32], lo)
- copy(dstPart[32:48], hi)
- copy(dstPart[48:64], hi)
- }
- }
- return dst
-}
-
-// xor slices writing to out.
-func sliceXorGo(in, out []byte, _ *options) {
- for len(out) >= 32 {
- inS := in[:32]
- v0 := binary.LittleEndian.Uint64(out[:8]) ^ binary.LittleEndian.Uint64(inS[:8])
- v1 := binary.LittleEndian.Uint64(out[8:16]) ^ binary.LittleEndian.Uint64(inS[8:16])
- v2 := binary.LittleEndian.Uint64(out[16:24]) ^ binary.LittleEndian.Uint64(inS[16:24])
- v3 := binary.LittleEndian.Uint64(out[24:32]) ^ binary.LittleEndian.Uint64(inS[24:32])
- binary.LittleEndian.PutUint64(out[:8], v0)
- binary.LittleEndian.PutUint64(out[8:16], v1)
- binary.LittleEndian.PutUint64(out[16:24], v2)
- binary.LittleEndian.PutUint64(out[24:32], v3)
- out = out[32:]
- in = in[32:]
- }
- out = out[:len(in)]
- for n, input := range in {
- out[n] ^= input
- }
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.go b/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.go
deleted file mode 100644
index 9a249d2..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.go
+++ /dev/null
@@ -1,339 +0,0 @@
-//go:build !noasm && !appengine && !gccgo
-// +build !noasm,!appengine,!gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2019, Minio, Inc.
-
-package reedsolomon
-
-import (
- "sync"
-)
-
-//go:noescape
-func _galMulAVX512Parallel81(in, out [][]byte, matrix *[matrixSize81]byte, addTo bool)
-
-//go:noescape
-func _galMulAVX512Parallel82(in, out [][]byte, matrix *[matrixSize82]byte, addTo bool)
-
-//go:noescape
-func _galMulAVX512Parallel84(in, out [][]byte, matrix *[matrixSize84]byte, addTo bool)
-
-const (
- dimIn = 8 // Number of input rows processed simultaneously
- dimOut81 = 1 // Number of output rows processed simultaneously for x1 routine
- dimOut82 = 2 // Number of output rows processed simultaneously for x2 routine
- dimOut84 = 4 // Number of output rows processed simultaneously for x4 routine
- matrixSize81 = (16 + 16) * dimIn * dimOut81 // Dimension of slice of matrix coefficient passed into x1 routine
- matrixSize82 = (16 + 16) * dimIn * dimOut82 // Dimension of slice of matrix coefficient passed into x2 routine
- matrixSize84 = (16 + 16) * dimIn * dimOut84 // Dimension of slice of matrix coefficient passed into x4 routine
-)
-
-// Construct block of matrix coefficients for single output row in parallel
-func setupMatrix81(matrixRows [][]byte, inputOffset, outputOffset int, matrix *[matrixSize81]byte) {
- offset := 0
- for c := inputOffset; c < inputOffset+dimIn; c++ {
- for iRow := outputOffset; iRow < outputOffset+dimOut81; iRow++ {
- if c < len(matrixRows[iRow]) {
- coeff := matrixRows[iRow][c]
- copy(matrix[offset*32:], mulTableLow[coeff][:])
- copy(matrix[offset*32+16:], mulTableHigh[coeff][:])
- } else {
- // coefficients not used for this input shard (so null out)
- v := matrix[offset*32 : offset*32+32]
- for i := range v {
- v[i] = 0
- }
- }
- offset += dimIn
- if offset >= dimIn*dimOut81 {
- offset -= dimIn*dimOut81 - 1
- }
- }
- }
-}
-
-// Construct block of matrix coefficients for 2 output rows in parallel
-func setupMatrix82(matrixRows [][]byte, inputOffset, outputOffset int, matrix *[matrixSize82]byte) {
- offset := 0
- for c := inputOffset; c < inputOffset+dimIn; c++ {
- for iRow := outputOffset; iRow < outputOffset+dimOut82; iRow++ {
- if c < len(matrixRows[iRow]) {
- coeff := matrixRows[iRow][c]
- copy(matrix[offset*32:], mulTableLow[coeff][:])
- copy(matrix[offset*32+16:], mulTableHigh[coeff][:])
- } else {
- // coefficients not used for this input shard (so null out)
- v := matrix[offset*32 : offset*32+32]
- for i := range v {
- v[i] = 0
- }
- }
- offset += dimIn
- if offset >= dimIn*dimOut82 {
- offset -= dimIn*dimOut82 - 1
- }
- }
- }
-}
-
-// Construct block of matrix coefficients for 4 output rows in parallel
-func setupMatrix84(matrixRows [][]byte, inputOffset, outputOffset int, matrix *[matrixSize84]byte) {
- offset := 0
- for c := inputOffset; c < inputOffset+dimIn; c++ {
- for iRow := outputOffset; iRow < outputOffset+dimOut84; iRow++ {
- if c < len(matrixRows[iRow]) {
- coeff := matrixRows[iRow][c]
- copy(matrix[offset*32:], mulTableLow[coeff][:])
- copy(matrix[offset*32+16:], mulTableHigh[coeff][:])
- } else {
- // coefficients not used for this input shard (so null out)
- v := matrix[offset*32 : offset*32+32]
- for i := range v {
- v[i] = 0
- }
- }
- offset += dimIn
- if offset >= dimIn*dimOut84 {
- offset -= dimIn*dimOut84 - 1
- }
- }
- }
-}
-
-// Invoke AVX512 routine for single output row in parallel
-func galMulAVX512Parallel81(in, out [][]byte, matrixRows [][]byte, inputOffset, outputOffset, start, stop int, matrix81 *[matrixSize81]byte) {
- done := stop - start
- if done <= 0 || len(in) == 0 || len(out) == 0 {
- return
- }
-
- inputEnd := inputOffset + dimIn
- if inputEnd > len(in) {
- inputEnd = len(in)
- }
- outputEnd := outputOffset + dimOut81
- if outputEnd > len(out) {
- outputEnd = len(out)
- }
-
- // We know the max size, alloc temp array.
- var inTmp [dimIn][]byte
- for i, v := range in[inputOffset:inputEnd] {
- inTmp[i] = v[start:stop]
- }
- var outTmp [dimOut81][]byte
- for i, v := range out[outputOffset:outputEnd] {
- outTmp[i] = v[start:stop]
- }
-
- addTo := inputOffset != 0 // Except for the first input column, add to previous results
- _galMulAVX512Parallel81(inTmp[:inputEnd-inputOffset], outTmp[:outputEnd-outputOffset], matrix81, addTo)
-
- done = start + ((done >> 6) << 6)
- if done < stop {
- galMulAVX512LastInput(inputOffset, inputEnd, outputOffset, outputEnd, matrixRows, done, stop, out, in)
- }
-}
-
-// Invoke AVX512 routine for 2 output rows in parallel
-func galMulAVX512Parallel82(in, out [][]byte, matrixRows [][]byte, inputOffset, outputOffset, start, stop int, matrix82 *[matrixSize82]byte) {
- done := stop - start
- if done <= 0 || len(in) == 0 || len(out) == 0 {
- return
- }
-
- inputEnd := inputOffset + dimIn
- if inputEnd > len(in) {
- inputEnd = len(in)
- }
- outputEnd := outputOffset + dimOut82
- if outputEnd > len(out) {
- outputEnd = len(out)
- }
-
- // We know the max size, alloc temp array.
- var inTmp [dimIn][]byte
- for i, v := range in[inputOffset:inputEnd] {
- inTmp[i] = v[start:stop]
- }
- var outTmp [dimOut82][]byte
- for i, v := range out[outputOffset:outputEnd] {
- outTmp[i] = v[start:stop]
- }
-
- addTo := inputOffset != 0 // Except for the first input column, add to previous results
- _galMulAVX512Parallel82(inTmp[:inputEnd-inputOffset], outTmp[:outputEnd-outputOffset], matrix82, addTo)
-
- done = start + ((done >> 6) << 6)
- if done < stop {
- galMulAVX512LastInput(inputOffset, inputEnd, outputOffset, outputEnd, matrixRows, done, stop, out, in)
- }
-}
-
-// Invoke AVX512 routine for 4 output rows in parallel
-func galMulAVX512Parallel84(in, out [][]byte, matrixRows [][]byte, inputOffset, outputOffset, start, stop int, matrix84 *[matrixSize84]byte) {
- done := stop - start
- if done <= 0 || len(in) == 0 || len(out) == 0 {
- return
- }
-
- inputEnd := inputOffset + dimIn
- if inputEnd > len(in) {
- inputEnd = len(in)
- }
- outputEnd := outputOffset + dimOut84
- if outputEnd > len(out) {
- outputEnd = len(out)
- }
-
- // We know the max size, alloc temp array.
- var inTmp [dimIn][]byte
- for i, v := range in[inputOffset:inputEnd] {
- inTmp[i] = v[start:stop]
- }
- var outTmp [dimOut84][]byte
- for i, v := range out[outputOffset:outputEnd] {
- outTmp[i] = v[start:stop]
- }
-
- addTo := inputOffset != 0 // Except for the first input column, add to previous results
- _galMulAVX512Parallel84(inTmp[:inputEnd-inputOffset], outTmp[:outputEnd-outputOffset], matrix84, addTo)
-
- done = start + ((done >> 6) << 6)
- if done < stop {
- galMulAVX512LastInput(inputOffset, inputEnd, outputOffset, outputEnd, matrixRows, done, stop, out, in)
- }
-}
-
-func galMulAVX512LastInput(inputOffset int, inputEnd int, outputOffset int, outputEnd int, matrixRows [][]byte, done int, stop int, out [][]byte, in [][]byte) {
- for c := inputOffset; c < inputEnd; c++ {
- for iRow := outputOffset; iRow < outputEnd; iRow++ {
- if c < len(matrixRows[iRow]) {
- mt := mulTable[matrixRows[iRow][c]][:256]
- for i := done; i < stop; i++ {
- if c == 0 { // only set value for first input column
- out[iRow][i] = mt[in[c][i]]
- } else { // and add for all others
- out[iRow][i] ^= mt[in[c][i]]
- }
- }
- }
- }
- }
-}
-
-// Perform the same as codeSomeShards, but taking advantage of
-// AVX512 parallelism for up to 4x faster execution as compared to AVX2
-func (r *reedSolomon) codeSomeShardsAvx512(matrixRows, inputs, outputs [][]byte, byteCount int) {
- // Process using no goroutines
- outputCount := len(outputs)
- start, end := 0, r.o.perRound
- if end > byteCount {
- end = byteCount
- }
- for start < byteCount {
- matrix84 := [matrixSize84]byte{}
- matrix82 := [matrixSize82]byte{}
- matrix81 := [matrixSize81]byte{}
-
- outputRow := 0
- // First process (multiple) batches of 4 output rows in parallel
- if outputRow+dimOut84 <= outputCount {
- for ; outputRow+dimOut84 <= outputCount; outputRow += dimOut84 {
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix84(matrixRows, inputRow, outputRow, &matrix84)
- galMulAVX512Parallel84(inputs, outputs, matrixRows, inputRow, outputRow, start, end, &matrix84)
- }
- }
- }
- // Then process a (single) batch of 2 output rows in parallel
- if outputRow+dimOut82 <= outputCount {
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix82(matrixRows, inputRow, outputRow, &matrix82)
- galMulAVX512Parallel82(inputs, outputs, matrixRows, inputRow, outputRow, start, end, &matrix82)
- }
- outputRow += dimOut82
- }
- // Lastly, we may have a single output row left (for uneven parity)
- if outputRow < outputCount {
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix81(matrixRows, inputRow, outputRow, &matrix81)
- galMulAVX512Parallel81(inputs, outputs, matrixRows, inputRow, outputRow, start, end, &matrix81)
- }
- }
-
- start = end
- end += r.o.perRound
- if end > byteCount {
- end = byteCount
- }
- }
-}
-
-// Perform the same as codeSomeShards, but taking advantage of
-// AVX512 parallelism for up to 4x faster execution as compared to AVX2
-func (r *reedSolomon) codeSomeShardsAvx512P(matrixRows, inputs, outputs [][]byte, byteCount int) {
- outputCount := len(outputs)
- var wg sync.WaitGroup
- do := byteCount / r.o.maxGoroutines
- if do < r.o.minSplitSize {
- do = r.o.minSplitSize
- }
- // Make sizes divisible by 64
- do = (do + 63) & (^63)
- start := 0
- for start < byteCount {
- if start+do > byteCount {
- do = byteCount - start
- }
- wg.Add(1)
- go func(grStart, grStop int) {
- start, stop := grStart, grStart+r.o.perRound
- if stop > grStop {
- stop = grStop
- }
- // Loop for each round.
- matrix84 := [matrixSize84]byte{}
- matrix82 := [matrixSize82]byte{}
- matrix81 := [matrixSize81]byte{}
- for start < grStop {
- outputRow := 0
- // First process (multiple) batches of 4 output rows in parallel
- if outputRow+dimOut84 <= outputCount {
- // 1K matrix buffer
- for ; outputRow+dimOut84 <= outputCount; outputRow += dimOut84 {
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix84(matrixRows, inputRow, outputRow, &matrix84)
- galMulAVX512Parallel84(inputs, outputs, matrixRows, inputRow, outputRow, start, stop, &matrix84)
- }
- }
- }
- // Then process a (single) batch of 2 output rows in parallel
- if outputRow+dimOut82 <= outputCount {
- // 512B matrix buffer
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix82(matrixRows, inputRow, outputRow, &matrix82)
- galMulAVX512Parallel82(inputs, outputs, matrixRows, inputRow, outputRow, start, stop, &matrix82)
- }
- outputRow += dimOut82
- }
- // Lastly, we may have a single output row left (for uneven parity)
- if outputRow < outputCount {
- for inputRow := 0; inputRow < len(inputs); inputRow += dimIn {
- setupMatrix81(matrixRows, inputRow, outputRow, &matrix81)
- galMulAVX512Parallel81(inputs, outputs, matrixRows, inputRow, outputRow, start, stop, &matrix81)
- }
- }
- start = stop
- stop += r.o.perRound
- if stop > grStop {
- stop = grStop
- }
- }
- wg.Done()
- }(start, start+do)
- start += do
- }
- wg.Wait()
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.s b/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.s
deleted file mode 100644
index 09f1d0d..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galoisAvx512_amd64.s
+++ /dev/null
@@ -1,402 +0,0 @@
-//+build !noasm
-//+build !appengine
-//+build !gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2019, Minio, Inc.
-
-#define LOAD(OFFSET) \
- MOVQ OFFSET(SI), BX \
- VMOVDQU64 (BX)(R11*1), Z0 \
- VPSRLQ $4, Z0, Z1 \ // high input
- VPANDQ Z2, Z0, Z0 \ // low input
- VPANDQ Z2, Z1, Z1 // high input
-
-#define GALOIS_MUL(MUL_LO, MUL_HI, LO, HI, OUT) \
- VPSHUFB Z0, MUL_LO, LO \ // mul low part
- VPSHUFB Z1, MUL_HI, HI \ // mul high part
- VPTERNLOGD $0x96, LO, HI, OUT
-
-#define GALOIS(C1, C2, IN, LO, HI, OUT) \
- VSHUFI64X2 $C1, IN, IN, LO \
- VSHUFI64X2 $C2, IN, IN, HI \
- GALOIS_MUL(LO, HI, LO, HI, OUT)
-
-//
-// Process single output row from a total of 8 input rows
-//
-// func _galMulAVX512Parallel81(in, out [][]byte, matrix *[matrixSize81]byte, addTo bool)
-TEXT ·_galMulAVX512Parallel81(SB), 7, $0
- MOVQ in+0(FP), SI
- MOVQ 8(SI), R9 // R9: len(in)
- SHRQ $6, R9 // len(in) / 64
- TESTQ R9, R9
- JZ done_avx512_parallel81
-
- MOVQ matrix+48(FP), SI
- VMOVDQU64 0x000(SI), Z16
- VMOVDQU64 0x040(SI), Z17
- VMOVDQU64 0x080(SI), Z18
- VMOVDQU64 0x0c0(SI), Z19
-
- // Initialize multiplication constants
- VSHUFI64X2 $0x55, Z16, Z16, Z20
- VSHUFI64X2 $0xaa, Z16, Z16, Z24
- VSHUFI64X2 $0xff, Z16, Z16, Z28
- VSHUFI64X2 $0x00, Z16, Z16, Z16
-
- VSHUFI64X2 $0x55, Z17, Z17, Z21
- VSHUFI64X2 $0xaa, Z17, Z17, Z25
- VSHUFI64X2 $0xff, Z17, Z17, Z29
- VSHUFI64X2 $0x00, Z17, Z17, Z17
-
- VSHUFI64X2 $0x55, Z18, Z18, Z22
- VSHUFI64X2 $0xaa, Z18, Z18, Z26
- VSHUFI64X2 $0xff, Z18, Z18, Z30
- VSHUFI64X2 $0x00, Z18, Z18, Z18
-
- VSHUFI64X2 $0x55, Z19, Z19, Z23
- VSHUFI64X2 $0xaa, Z19, Z19, Z27
- VSHUFI64X2 $0xff, Z19, Z19, Z31
- VSHUFI64X2 $0x00, Z19, Z19, Z19
-
- MOVQ $15, BX
- VPBROADCASTB BX, Z2
-
- MOVB addTo+56(FP), AX
- IMULQ $-0x1, AX
- KMOVQ AX, K1
- MOVQ in+0(FP), SI // SI: &in
- MOVQ in_len+8(FP), AX // number of inputs
- XORQ R11, R11
- MOVQ out+24(FP), DX
- MOVQ (DX), DX // DX: &out[0][0]
-
-loopback_avx512_parallel81:
- VMOVDQU64.Z (DX), K1, Z4
-
- LOAD(0x00) // &in[0][0]
- GALOIS_MUL(Z16, Z20, Z14, Z15, Z4)
-
- CMPQ AX, $1
- JE skip_avx512_parallel81
-
- LOAD(0x18) // &in[1][0]
- GALOIS_MUL(Z24, Z28, Z14, Z15, Z4)
-
- CMPQ AX, $2
- JE skip_avx512_parallel81
-
- LOAD(0x30) // &in[2][0]
- GALOIS_MUL(Z17, Z21, Z14, Z15, Z4)
-
- CMPQ AX, $3
- JE skip_avx512_parallel81
-
- LOAD(0x48) // &in[3][0]
- GALOIS_MUL(Z25, Z29, Z14, Z15, Z4)
-
- CMPQ AX, $4
- JE skip_avx512_parallel81
-
- LOAD(0x60) // &in[4][0]
- GALOIS_MUL(Z18, Z22, Z14, Z15, Z4)
-
- CMPQ AX, $5
- JE skip_avx512_parallel81
-
- LOAD(0x78) // &in[5][0]
- GALOIS_MUL(Z26, Z30, Z14, Z15, Z4)
-
- CMPQ AX, $6
- JE skip_avx512_parallel81
-
- LOAD(0x90) // &in[6][0]
- GALOIS_MUL(Z19, Z23, Z14, Z15, Z4)
-
- CMPQ AX, $7
- JE skip_avx512_parallel81
-
- LOAD(0xa8) // &in[7][0]
- GALOIS_MUL(Z27, Z31, Z14, Z15, Z4)
-
-skip_avx512_parallel81:
- VMOVDQU64 Z4, (DX)
-
- ADDQ $64, R11 // in4+=64
-
- ADDQ $64, DX // out+=64
-
- SUBQ $1, R9
- JNZ loopback_avx512_parallel81
-
-done_avx512_parallel81:
- VZEROUPPER
- RET
-
-//
-// Process 2 output rows in parallel from a total of 8 input rows
-//
-// func _galMulAVX512Parallel82(in, out [][]byte, matrix *[matrixSize82]byte, addTo bool)
-TEXT ·_galMulAVX512Parallel82(SB), 7, $0
- MOVQ in+0(FP), SI
- MOVQ 8(SI), R9 // R9: len(in)
- SHRQ $6, R9 // len(in) / 64
- TESTQ R9, R9
- JZ done_avx512_parallel82
-
- MOVQ matrix+48(FP), SI
- VMOVDQU64 0x000(SI), Z16
- VMOVDQU64 0x040(SI), Z17
- VMOVDQU64 0x080(SI), Z18
- VMOVDQU64 0x0c0(SI), Z19
- VMOVDQU64 0x100(SI), Z20
- VMOVDQU64 0x140(SI), Z21
- VMOVDQU64 0x180(SI), Z22
- VMOVDQU64 0x1c0(SI), Z23
-
- // Initialize multiplication constants
- VSHUFI64X2 $0x55, Z16, Z16, Z24
- VSHUFI64X2 $0xaa, Z16, Z16, Z25
- VSHUFI64X2 $0xff, Z16, Z16, Z26
- VSHUFI64X2 $0x00, Z16, Z16, Z16
-
- VSHUFI64X2 $0x55, Z20, Z20, Z27
- VSHUFI64X2 $0xaa, Z20, Z20, Z28
- VSHUFI64X2 $0xff, Z20, Z20, Z29
- VSHUFI64X2 $0x00, Z20, Z20, Z20
-
- VSHUFI64X2 $0x55, Z17, Z17, Z30
- VSHUFI64X2 $0xaa, Z17, Z17, Z31
- VSHUFI64X2 $0xff, Z17, Z17, Z11
- VSHUFI64X2 $0x00, Z17, Z17, Z17
-
- VSHUFI64X2 $0x55, Z21, Z21, Z8
- VSHUFI64X2 $0xaa, Z21, Z21, Z9
- VSHUFI64X2 $0xff, Z21, Z21, Z10
- VSHUFI64X2 $0x00, Z21, Z21, Z21
-
- MOVQ $15, BX
- VPBROADCASTB BX, Z2
-
- MOVB addTo+56(FP), AX
- IMULQ $-0x1, AX
- KMOVQ AX, K1
- MOVQ in+0(FP), SI // SI: &in
- MOVQ in_len+8(FP), AX // number of inputs
- XORQ R11, R11
- MOVQ out+24(FP), DX
- MOVQ 24(DX), CX // CX: &out[1][0]
- MOVQ (DX), DX // DX: &out[0][0]
-
-loopback_avx512_parallel82:
- VMOVDQU64.Z (DX), K1, Z4
- VMOVDQU64.Z (CX), K1, Z5
-
- LOAD(0x00) // &in[0][0]
- GALOIS_MUL(Z16, Z24, Z14, Z15, Z4)
- GALOIS_MUL(Z20, Z27, Z12, Z13, Z5)
-
- CMPQ AX, $1
- JE skip_avx512_parallel82
-
- LOAD(0x18) // &in[1][0]
- GALOIS_MUL(Z25, Z26, Z14, Z15, Z4)
- GALOIS_MUL(Z28, Z29, Z12, Z13, Z5)
-
- CMPQ AX, $2
- JE skip_avx512_parallel82
-
- LOAD(0x30) // &in[2][0]
- GALOIS_MUL(Z17, Z30, Z14, Z15, Z4)
- GALOIS_MUL(Z21, Z8, Z12, Z13, Z5)
-
- CMPQ AX, $3
- JE skip_avx512_parallel82
-
- LOAD(0x48) // &in[3][0]
- GALOIS_MUL(Z31, Z11, Z14, Z15, Z4)
- GALOIS_MUL(Z9, Z10, Z12, Z13, Z5)
-
- CMPQ AX, $4
- JE skip_avx512_parallel82
-
- LOAD(0x60) // &in[4][0]
- GALOIS(0x00, 0x55, Z18, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z22, Z12, Z13, Z5)
-
- CMPQ AX, $5
- JE skip_avx512_parallel82
-
- LOAD(0x78) // &in[5][0]
- GALOIS(0xaa, 0xff, Z18, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z22, Z12, Z13, Z5)
-
- CMPQ AX, $6
- JE skip_avx512_parallel82
-
- LOAD(0x90) // &in[6][0]
- GALOIS(0x00, 0x55, Z19, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z23, Z12, Z13, Z5)
-
- CMPQ AX, $7
- JE skip_avx512_parallel82
-
- LOAD(0xa8) // &in[7][0]
- GALOIS(0xaa, 0xff, Z19, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z23, Z12, Z13, Z5)
-
-skip_avx512_parallel82:
- VMOVDQU64 Z4, (DX)
- VMOVDQU64 Z5, (CX)
-
- ADDQ $64, R11 // in4+=64
-
- ADDQ $64, DX // out+=64
- ADDQ $64, CX // out2+=64
-
- SUBQ $1, R9
- JNZ loopback_avx512_parallel82
-
-done_avx512_parallel82:
- VZEROUPPER
- RET
-
-//
-// Process 4 output rows in parallel from a total of 8 input rows
-//
-// func _galMulAVX512Parallel84(in, out [][]byte, matrix *[matrixSize84]byte, addTo bool)
-TEXT ·_galMulAVX512Parallel84(SB), 7, $0
- MOVQ in+0(FP), SI
- MOVQ 8(SI), R9 // R9: len(in)
- SHRQ $6, R9 // len(in) / 64
- TESTQ R9, R9
- JZ done_avx512_parallel84
-
- MOVQ matrix+48(FP), SI
- VMOVDQU64 0x000(SI), Z16
- VMOVDQU64 0x040(SI), Z17
- VMOVDQU64 0x080(SI), Z18
- VMOVDQU64 0x0c0(SI), Z19
- VMOVDQU64 0x100(SI), Z20
- VMOVDQU64 0x140(SI), Z21
- VMOVDQU64 0x180(SI), Z22
- VMOVDQU64 0x1c0(SI), Z23
- VMOVDQU64 0x200(SI), Z24
- VMOVDQU64 0x240(SI), Z25
- VMOVDQU64 0x280(SI), Z26
- VMOVDQU64 0x2c0(SI), Z27
- VMOVDQU64 0x300(SI), Z28
- VMOVDQU64 0x340(SI), Z29
- VMOVDQU64 0x380(SI), Z30
- VMOVDQU64 0x3c0(SI), Z31
-
- MOVQ $15, BX
- VPBROADCASTB BX, Z2
-
- MOVB addTo+56(FP), AX
- IMULQ $-0x1, AX
- KMOVQ AX, K1
- MOVQ in+0(FP), SI // SI: &in
- MOVQ in_len+8(FP), AX // number of inputs
- XORQ R11, R11
- MOVQ out+24(FP), DX
- MOVQ 24(DX), CX // CX: &out[1][0]
- MOVQ 48(DX), R10 // R10: &out[2][0]
- MOVQ 72(DX), R12 // R12: &out[3][0]
- MOVQ (DX), DX // DX: &out[0][0]
-
-loopback_avx512_parallel84:
- VMOVDQU64.Z (DX), K1, Z4
- VMOVDQU64.Z (CX), K1, Z5
- VMOVDQU64.Z (R10), K1, Z6
- VMOVDQU64.Z (R12), K1, Z7
-
- LOAD(0x00) // &in[0][0]
- GALOIS(0x00, 0x55, Z16, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z20, Z12, Z13, Z5)
- GALOIS(0x00, 0x55, Z24, Z10, Z11, Z6)
- GALOIS(0x00, 0x55, Z28, Z8, Z9, Z7)
-
- CMPQ AX, $1
- JE skip_avx512_parallel84
-
- LOAD(0x18) // &in[1][0]
- GALOIS(0xaa, 0xff, Z16, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z20, Z12, Z13, Z5)
- GALOIS(0xaa, 0xff, Z24, Z10, Z11, Z6)
- GALOIS(0xaa, 0xff, Z28, Z8, Z9, Z7)
-
- CMPQ AX, $2
- JE skip_avx512_parallel84
-
- LOAD(0x30) // &in[2][0]
- GALOIS(0x00, 0x55, Z17, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z21, Z12, Z13, Z5)
- GALOIS(0x00, 0x55, Z25, Z10, Z11, Z6)
- GALOIS(0x00, 0x55, Z29, Z8, Z9, Z7)
-
- CMPQ AX, $3
- JE skip_avx512_parallel84
-
- LOAD(0x48) // &in[3][0]
- GALOIS(0xaa, 0xff, Z17, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z21, Z12, Z13, Z5)
- GALOIS(0xaa, 0xff, Z25, Z10, Z11, Z6)
- GALOIS(0xaa, 0xff, Z29, Z8, Z9, Z7)
-
- CMPQ AX, $4
- JE skip_avx512_parallel84
-
- LOAD(0x60) // &in[4][0]
- GALOIS(0x00, 0x55, Z18, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z22, Z12, Z13, Z5)
- GALOIS(0x00, 0x55, Z26, Z10, Z11, Z6)
- GALOIS(0x00, 0x55, Z30, Z8, Z9, Z7)
-
- CMPQ AX, $5
- JE skip_avx512_parallel84
-
- LOAD(0x78) // &in[5][0]
- GALOIS(0xaa, 0xff, Z18, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z22, Z12, Z13, Z5)
- GALOIS(0xaa, 0xff, Z26, Z10, Z11, Z6)
- GALOIS(0xaa, 0xff, Z30, Z8, Z9, Z7)
-
- CMPQ AX, $6
- JE skip_avx512_parallel84
-
- LOAD(0x90) // &in[6][0]
- GALOIS(0x00, 0x55, Z19, Z14, Z15, Z4)
- GALOIS(0x00, 0x55, Z23, Z12, Z13, Z5)
- GALOIS(0x00, 0x55, Z27, Z10, Z11, Z6)
- GALOIS(0x00, 0x55, Z31, Z8, Z9, Z7)
-
- CMPQ AX, $7
- JE skip_avx512_parallel84
-
- LOAD(0xa8) // &in[7][0]
- GALOIS(0xaa, 0xff, Z19, Z14, Z15, Z4)
- GALOIS(0xaa, 0xff, Z23, Z12, Z13, Z5)
- GALOIS(0xaa, 0xff, Z27, Z10, Z11, Z6)
- GALOIS(0xaa, 0xff, Z31, Z8, Z9, Z7)
-
-skip_avx512_parallel84:
- VMOVDQU64 Z4, (DX)
- VMOVDQU64 Z5, (CX)
- VMOVDQU64 Z6, (R10)
- VMOVDQU64 Z7, (R12)
-
- ADDQ $64, R11 // in4+=64
-
- ADDQ $64, DX // out+=64
- ADDQ $64, CX // out2+=64
- ADDQ $64, R10 // out3+=64
- ADDQ $64, R12 // out4+=64
-
- SUBQ $1, R9
- JNZ loopback_avx512_parallel84
-
-done_avx512_parallel84:
- VZEROUPPER
- RET
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_amd64.go b/vendor/github.com/klauspost/reedsolomon/galois_amd64.go
deleted file mode 100644
index ae6f289..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_amd64.go
+++ /dev/null
@@ -1,346 +0,0 @@
-//go:build !noasm && !appengine && !gccgo
-// +build !noasm,!appengine,!gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-
-package reedsolomon
-
-//go:noescape
-func galMulSSSE3(low, high, in, out []byte)
-
-//go:noescape
-func galMulSSSE3Xor(low, high, in, out []byte)
-
-//go:noescape
-func galMulAVX2Xor(low, high, in, out []byte)
-
-//go:noescape
-func galMulAVX2(low, high, in, out []byte)
-
-//go:noescape
-func sSE2XorSlice(in, out []byte)
-
-//go:noescape
-func galMulAVX2Xor_64(low, high, in, out []byte)
-
-//go:noescape
-func galMulAVX2_64(low, high, in, out []byte)
-
-//go:noescape
-func sSE2XorSlice_64(in, out []byte)
-
-// This is what the assembler routines do in blocks of 16 bytes:
-/*
-func galMulSSSE3(low, high, in, out []byte) {
- for n, input := range in {
- l := input & 0xf
- h := input >> 4
- out[n] = low[l] ^ high[h]
- }
-}
-
-func galMulSSSE3Xor(low, high, in, out []byte) {
- for n, input := range in {
- l := input & 0xf
- h := input >> 4
- out[n] ^= low[l] ^ high[h]
- }
-}
-*/
-
-// bigSwitchover is the size where 64 bytes are processed per loop.
-const bigSwitchover = 128
-
-func galMulSlice(c byte, in, out []byte, o *options) {
- if c == 1 {
- copy(out, in)
- return
- }
- if o.useAVX2 {
- if len(in) >= bigSwitchover {
- galMulAVX2_64(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 6) << 6
- in = in[done:]
- out = out[done:]
- }
- if len(in) > 32 {
- galMulAVX2(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 5) << 5
- in = in[done:]
- out = out[done:]
- }
- } else if o.useSSSE3 {
- galMulSSSE3(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 4) << 4
- in = in[done:]
- out = out[done:]
- }
- out = out[:len(in)]
- mt := mulTable[c][:256]
- for i := range in {
- out[i] = mt[in[i]]
- }
-}
-
-func galMulSliceXor(c byte, in, out []byte, o *options) {
- if c == 1 {
- sliceXor(in, out, o)
- return
- }
-
- if o.useAVX2 {
- if len(in) >= bigSwitchover {
- galMulAVX2Xor_64(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 6) << 6
- in = in[done:]
- out = out[done:]
- }
- if len(in) >= 32 {
- galMulAVX2Xor(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 5) << 5
- in = in[done:]
- out = out[done:]
- }
- } else if o.useSSSE3 {
- galMulSSSE3Xor(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done := (len(in) >> 4) << 4
- in = in[done:]
- out = out[done:]
- }
- if len(in) == 0 {
- return
- }
- out = out[:len(in)]
- mt := mulTable[c][:256]
- for i := range in {
- out[i] ^= mt[in[i]]
- }
-}
-
-// simple slice xor
-func sliceXor(in, out []byte, o *options) {
- if o.useSSE2 {
- if len(in) >= bigSwitchover {
- sSE2XorSlice_64(in, out)
- done := (len(in) >> 6) << 6
- in = in[done:]
- out = out[done:]
- }
- if len(in) >= 16 {
- sSE2XorSlice(in, out)
- done := (len(in) >> 4) << 4
- in = in[done:]
- out = out[done:]
- }
- } else {
- sliceXorGo(in, out, o)
- return
- }
- out = out[:len(in)]
- for i := range in {
- out[i] ^= in[i]
- }
-}
-
-// 4-way butterfly
-func ifftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- if len(work[0]) == 0 {
- return
- }
-
- t01 := &multiply256LUT[log_m01]
- t23 := &multiply256LUT[log_m23]
- t02 := &multiply256LUT[log_m02]
- if o.useAVX512 {
- if log_m01 == modulus {
- if log_m23 == modulus {
- if log_m02 == modulus {
- ifftDIT4_avx512_7(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx512_3(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m02 == modulus {
- ifftDIT4_avx512_5(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx512_1(work, dist*24, t01, t23, t02)
- }
- }
- } else {
- if log_m23 == modulus {
- if log_m02 == modulus {
- ifftDIT4_avx512_6(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx512_2(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m02 == modulus {
- ifftDIT4_avx512_4(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx512_0(work, dist*24, t01, t23, t02)
- }
- }
- }
- return
- } else if o.useAVX2 {
- if log_m01 == modulus {
- if log_m23 == modulus {
- if log_m02 == modulus {
- ifftDIT4_avx2_7(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx2_3(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m02 == modulus {
- ifftDIT4_avx2_5(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx2_1(work, dist*24, t01, t23, t02)
- }
- }
- } else {
- if log_m23 == modulus {
- if log_m02 == modulus {
- ifftDIT4_avx2_6(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx2_2(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m02 == modulus {
- ifftDIT4_avx2_4(work, dist*24, t01, t23, t02)
- } else {
- ifftDIT4_avx2_0(work, dist*24, t01, t23, t02)
- }
- }
- }
- return
- }
- ifftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-func fftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- if len(work[0]) == 0 {
- return
- }
-
- t01 := &multiply256LUT[log_m01]
- t23 := &multiply256LUT[log_m23]
- t02 := &multiply256LUT[log_m02]
- if o.useAVX512 {
- if log_m02 == modulus {
- if log_m01 == modulus {
- if log_m23 == modulus {
- fftDIT4_avx512_7(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx512_3(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m23 == modulus {
- fftDIT4_avx512_5(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx512_1(work, dist*24, t01, t23, t02)
- }
- }
- } else {
- if log_m01 == modulus {
- if log_m23 == modulus {
- fftDIT4_avx512_6(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx512_2(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m23 == modulus {
- fftDIT4_avx512_4(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx512_0(work, dist*24, t01, t23, t02)
- }
- }
- }
- return
- } else if o.useAVX2 {
- if log_m02 == modulus {
- if log_m01 == modulus {
- if log_m23 == modulus {
- fftDIT4_avx2_7(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx2_3(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m23 == modulus {
- fftDIT4_avx2_5(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx2_1(work, dist*24, t01, t23, t02)
- }
- }
- } else {
- if log_m01 == modulus {
- if log_m23 == modulus {
- fftDIT4_avx2_6(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx2_2(work, dist*24, t01, t23, t02)
- }
- } else {
- if log_m23 == modulus {
- fftDIT4_avx2_4(work, dist*24, t01, t23, t02)
- } else {
- fftDIT4_avx2_0(work, dist*24, t01, t23, t02)
- }
- }
- }
- return
- }
- fftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 2-way butterfly forward
-func fftDIT2(x, y []byte, log_m ffe, o *options) {
- if o.useAVX2 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- fftDIT2_avx2(x, y, tmp)
- }
- } else if o.useSSSE3 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- fftDIT2_ssse3(x, y, tmp)
- }
- } else {
- // Reference version:
- refMulAdd(x, y, log_m)
- sliceXor(x, y, o)
- }
-}
-
-// 2-way butterfly
-func ifftDIT2(x, y []byte, log_m ffe, o *options) {
- if o.useAVX2 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- ifftDIT2_avx2(x, y, tmp)
- }
- } else if o.useSSSE3 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- ifftDIT2_ssse3(x, y, tmp)
- }
- } else {
- // Reference version:
- sliceXor(x, y, o)
- refMulAdd(x, y, log_m)
- }
-}
-
-func mulgf16(x, y []byte, log_m ffe, o *options) {
- if o.useAVX2 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- mulgf16_avx2(x, y, tmp)
- }
- } else if o.useSSSE3 {
- if len(x) > 0 {
- tmp := &multiply256LUT[log_m]
- mulgf16_ssse3(x, y, tmp)
- }
- } else {
- refMul(x, y, log_m)
- }
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_amd64.s b/vendor/github.com/klauspost/reedsolomon/galois_amd64.s
deleted file mode 100644
index f1dc8d5..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_amd64.s
+++ /dev/null
@@ -1,370 +0,0 @@
-//+build !noasm
-//+build !appengine
-//+build !gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-
-// Based on http://www.snia.org/sites/default/files2/SDC2013/presentations/NewThinking/EthanMiller_Screaming_Fast_Galois_Field%20Arithmetic_SIMD%20Instructions.pdf
-// and http://jerasure.org/jerasure/gf-complete/tree/master
-
-// func galMulSSSE3Xor(low, high, in, out []byte)
-TEXT ·galMulSSSE3Xor(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVOU (SI), X6 // X6 low
- MOVOU (DX), X7 // X7: high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X8
- PXOR X5, X5
- MOVQ in+48(FP), SI // R11: &in
- MOVQ in_len+56(FP), R9 // R9: len(in)
- MOVQ out+72(FP), DX // DX: &out
- PSHUFB X5, X8 // X8: lomask (unpacked)
- SHRQ $4, R9 // len(in) / 16
- MOVQ SI, AX
- MOVQ DX, BX
- ANDQ $15, AX
- ANDQ $15, BX
- CMPQ R9, $0
- JEQ done_xor
- ORQ AX, BX
- CMPQ BX, $0
- JNZ loopback_xor
-
-loopback_xor_aligned:
- MOVOA (SI), X0 // in[x]
- MOVOA (DX), X4 // out[x]
- MOVOA X0, X1 // in[x]
- MOVOA X6, X2 // low copy
- MOVOA X7, X3 // high copy
- PSRLQ $4, X1 // X1: high input
- PAND X8, X0 // X0: low input
- PAND X8, X1 // X0: high input
- PSHUFB X0, X2 // X2: mul low part
- PSHUFB X1, X3 // X3: mul high part
- PXOR X2, X3 // X3: Result
- PXOR X4, X3 // X3: Result xor existing out
- MOVOA X3, (DX) // Store
- ADDQ $16, SI // in+=16
- ADDQ $16, DX // out+=16
- SUBQ $1, R9
- JNZ loopback_xor_aligned
- JMP done_xor
-
-loopback_xor:
- MOVOU (SI), X0 // in[x]
- MOVOU (DX), X4 // out[x]
- MOVOU X0, X1 // in[x]
- MOVOU X6, X2 // low copy
- MOVOU X7, X3 // high copy
- PSRLQ $4, X1 // X1: high input
- PAND X8, X0 // X0: low input
- PAND X8, X1 // X0: high input
- PSHUFB X0, X2 // X2: mul low part
- PSHUFB X1, X3 // X3: mul high part
- PXOR X2, X3 // X3: Result
- PXOR X4, X3 // X3: Result xor existing out
- MOVOU X3, (DX) // Store
- ADDQ $16, SI // in+=16
- ADDQ $16, DX // out+=16
- SUBQ $1, R9
- JNZ loopback_xor
-
-done_xor:
- RET
-
-// func galMulSSSE3(low, high, in, out []byte)
-TEXT ·galMulSSSE3(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVOU (SI), X6 // X6 low
- MOVOU (DX), X7 // X7: high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X8
- PXOR X5, X5
- MOVQ in+48(FP), SI // R11: &in
- MOVQ in_len+56(FP), R9 // R9: len(in)
- MOVQ out+72(FP), DX // DX: &out
- PSHUFB X5, X8 // X8: lomask (unpacked)
- MOVQ SI, AX
- MOVQ DX, BX
- SHRQ $4, R9 // len(in) / 16
- ANDQ $15, AX
- ANDQ $15, BX
- CMPQ R9, $0
- JEQ done
- ORQ AX, BX
- CMPQ BX, $0
- JNZ loopback
-
-loopback_aligned:
- MOVOA (SI), X0 // in[x]
- MOVOA X0, X1 // in[x]
- MOVOA X6, X2 // low copy
- MOVOA X7, X3 // high copy
- PSRLQ $4, X1 // X1: high input
- PAND X8, X0 // X0: low input
- PAND X8, X1 // X0: high input
- PSHUFB X0, X2 // X2: mul low part
- PSHUFB X1, X3 // X3: mul high part
- PXOR X2, X3 // X3: Result
- MOVOA X3, (DX) // Store
- ADDQ $16, SI // in+=16
- ADDQ $16, DX // out+=16
- SUBQ $1, R9
- JNZ loopback_aligned
- JMP done
-
-loopback:
- MOVOU (SI), X0 // in[x]
- MOVOU X0, X1 // in[x]
- MOVOA X6, X2 // low copy
- MOVOA X7, X3 // high copy
- PSRLQ $4, X1 // X1: high input
- PAND X8, X0 // X0: low input
- PAND X8, X1 // X0: high input
- PSHUFB X0, X2 // X2: mul low part
- PSHUFB X1, X3 // X3: mul high part
- PXOR X2, X3 // X3: Result
- MOVOU X3, (DX) // Store
- ADDQ $16, SI // in+=16
- ADDQ $16, DX // out+=16
- SUBQ $1, R9
- JNZ loopback
-
-done:
- RET
-
-// func galMulAVX2Xor(low, high, in, out []byte)
-TEXT ·galMulAVX2Xor(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X5
- MOVOU (SI), X6 // X6: low
- MOVOU (DX), X7 // X7: high
- MOVQ in_len+56(FP), R9 // R9: len(in)
-
- VINSERTI128 $1, X6, Y6, Y6 // low
- VINSERTI128 $1, X7, Y7, Y7 // high
- VPBROADCASTB X5, Y8 // Y8: lomask (unpacked)
-
- SHRQ $5, R9 // len(in) / 32
- MOVQ out+72(FP), DX // DX: &out
- MOVQ in+48(FP), SI // SI: &in
- TESTQ R9, R9
- JZ done_xor_avx2
-
-loopback_xor_avx2:
- VMOVDQU (SI), Y0
- VMOVDQU (DX), Y4
- VPSRLQ $4, Y0, Y1 // Y1: high input
- VPAND Y8, Y0, Y0 // Y0: low input
- VPAND Y8, Y1, Y1 // Y1: high input
- VPSHUFB Y0, Y6, Y2 // Y2: mul low part
- VPSHUFB Y1, Y7, Y3 // Y3: mul high part
- VPXOR Y3, Y2, Y3 // Y3: Result
- VPXOR Y4, Y3, Y4 // Y4: Result
- VMOVDQU Y4, (DX)
-
- ADDQ $32, SI // in+=32
- ADDQ $32, DX // out+=32
- SUBQ $1, R9
- JNZ loopback_xor_avx2
-
-done_xor_avx2:
- VZEROUPPER
- RET
-
-// func galMulAVX2(low, high, in, out []byte)
-TEXT ·galMulAVX2(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X5
- MOVOU (SI), X6 // X6: low
- MOVOU (DX), X7 // X7: high
- MOVQ in_len+56(FP), R9 // R9: len(in)
-
- VINSERTI128 $1, X6, Y6, Y6 // low
- VINSERTI128 $1, X7, Y7, Y7 // high
- VPBROADCASTB X5, Y8 // Y8: lomask (unpacked)
-
- SHRQ $5, R9 // len(in) / 32
- MOVQ out+72(FP), DX // DX: &out
- MOVQ in+48(FP), SI // SI: &in
- TESTQ R9, R9
- JZ done_avx2
-
-loopback_avx2:
- VMOVDQU (SI), Y0
- VPSRLQ $4, Y0, Y1 // Y1: high input
- VPAND Y8, Y0, Y0 // Y0: low input
- VPAND Y8, Y1, Y1 // Y1: high input
- VPSHUFB Y0, Y6, Y2 // Y2: mul low part
- VPSHUFB Y1, Y7, Y3 // Y3: mul high part
- VPXOR Y3, Y2, Y4 // Y4: Result
- VMOVDQU Y4, (DX)
-
- ADDQ $32, SI // in+=32
- ADDQ $32, DX // out+=32
- SUBQ $1, R9
- JNZ loopback_avx2
-
-done_avx2:
- VZEROUPPER
- RET
-
-// func sSE2XorSlice(in, out []byte)
-TEXT ·sSE2XorSlice(SB), 7, $0
- MOVQ in+0(FP), SI // SI: &in
- MOVQ in_len+8(FP), R9 // R9: len(in)
- MOVQ out+24(FP), DX // DX: &out
- SHRQ $4, R9 // len(in) / 16
- CMPQ R9, $0
- JEQ done_xor_sse2
-
-loopback_xor_sse2:
- MOVOU (SI), X0 // in[x]
- MOVOU (DX), X1 // out[x]
- PXOR X0, X1
- MOVOU X1, (DX)
- ADDQ $16, SI // in+=16
- ADDQ $16, DX // out+=16
- SUBQ $1, R9
- JNZ loopback_xor_sse2
-
-done_xor_sse2:
- RET
-
-// func galMulAVX2Xor_64(low, high, in, out []byte)
-TEXT ·galMulAVX2Xor_64(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X5
- MOVOU (SI), X6 // X6: low
- MOVOU (DX), X7 // X7: high
- MOVQ in_len+56(FP), R9 // R9: len(in)
-
- VINSERTI128 $1, X6, Y6, Y6 // low
- VINSERTI128 $1, X7, Y7, Y7 // high
- VPBROADCASTB X5, Y8 // Y8: lomask (unpacked)
-
- SHRQ $6, R9 // len(in) / 64
- MOVQ out+72(FP), DX // DX: &out
- MOVQ in+48(FP), SI // SI: &in
- TESTQ R9, R9
- JZ done_xor_avx2_64
-
-loopback_xor_avx2_64:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y10
- VMOVDQU (DX), Y4
- VMOVDQU 32(DX), Y14
- VPSRLQ $4, Y0, Y1 // Y1: high input
- VPSRLQ $4, Y10, Y11 // Y11: high input 2
- VPAND Y8, Y0, Y0 // Y0: low input
- VPAND Y8, Y10, Y10 // Y10: low input 2
- VPAND Y8, Y1, Y1 // Y11: high input
- VPAND Y8, Y11, Y11 // Y11: high input 2
- VPSHUFB Y0, Y6, Y2 // Y2: mul low part
- VPSHUFB Y10, Y6, Y12 // Y12: mul low part 2
- VPSHUFB Y1, Y7, Y3 // Y3: mul high part
- VPSHUFB Y11, Y7, Y13 // Y13: mul high part 2
- VPXOR Y3, Y2, Y3 // Y3: Result
- VPXOR Y13, Y12, Y13 // Y13: Result 2
- VPXOR Y4, Y3, Y4 // Y4: Result
- VPXOR Y14, Y13, Y14 // Y4: Result 2
- VMOVDQU Y4, (DX)
- VMOVDQU Y14, 32(DX)
-
- ADDQ $64, SI // in+=64
- ADDQ $64, DX // out+=64
- SUBQ $1, R9
- JNZ loopback_xor_avx2_64
-
-done_xor_avx2_64:
- VZEROUPPER
- RET
-
-// func galMulAVX2_64(low, high, in, out []byte)
-TEXT ·galMulAVX2_64(SB), 7, $0
- MOVQ low+0(FP), SI // SI: &low
- MOVQ high+24(FP), DX // DX: &high
- MOVQ $15, BX // BX: low mask
- MOVQ BX, X5
- MOVOU (SI), X6 // X6: low
- MOVOU (DX), X7 // X7: high
- MOVQ in_len+56(FP), R9 // R9: len(in)
-
- VINSERTI128 $1, X6, Y6, Y6 // low
- VINSERTI128 $1, X7, Y7, Y7 // high
- VPBROADCASTB X5, Y8 // Y8: lomask (unpacked)
-
- SHRQ $6, R9 // len(in) / 64
- MOVQ out+72(FP), DX // DX: &out
- MOVQ in+48(FP), SI // SI: &in
- TESTQ R9, R9
- JZ done_avx2_64
-
-loopback_avx2_64:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y10
- VPSRLQ $4, Y0, Y1 // Y1: high input
- VPSRLQ $4, Y10, Y11 // Y11: high input 2
- VPAND Y8, Y0, Y0 // Y0: low input
- VPAND Y8, Y10, Y10 // Y10: low input
- VPAND Y8, Y1, Y1 // Y1: high input
- VPAND Y8, Y11, Y11 // Y11: high input 2
- VPSHUFB Y0, Y6, Y2 // Y2: mul low part
- VPSHUFB Y10, Y6, Y12 // Y12: mul low part 2
- VPSHUFB Y1, Y7, Y3 // Y3: mul high part
- VPSHUFB Y11, Y7, Y13 // Y13: mul high part 2
- VPXOR Y3, Y2, Y4 // Y4: Result
- VPXOR Y13, Y12, Y14 // Y14: Result 2
- VMOVDQU Y4, (DX)
- VMOVDQU Y14, 32(DX)
-
- ADDQ $64, SI // in+=64
- ADDQ $64, DX // out+=64
- SUBQ $1, R9
- JNZ loopback_avx2_64
-
-done_avx2_64:
- VZEROUPPER
- RET
-
-// func sSE2XorSlice_64(in, out []byte)
-TEXT ·sSE2XorSlice_64(SB), 7, $0
- MOVQ in+0(FP), SI // SI: &in
- MOVQ in_len+8(FP), R9 // R9: len(in)
- MOVQ out+24(FP), DX // DX: &out
- SHRQ $6, R9 // len(in) / 64
- CMPQ R9, $0
- JEQ done_xor_sse2_64
-
-loopback_xor_sse2_64:
- MOVOU (SI), X0 // in[x]
- MOVOU 16(SI), X2 // in[x]
- MOVOU 32(SI), X4 // in[x]
- MOVOU 48(SI), X6 // in[x]
- MOVOU (DX), X1 // out[x]
- MOVOU 16(DX), X3 // out[x]
- MOVOU 32(DX), X5 // out[x]
- MOVOU 48(DX), X7 // out[x]
- PXOR X0, X1
- PXOR X2, X3
- PXOR X4, X5
- PXOR X6, X7
- MOVOU X1, (DX)
- MOVOU X3, 16(DX)
- MOVOU X5, 32(DX)
- MOVOU X7, 48(DX)
- ADDQ $64, SI // in+=64
- ADDQ $64, DX // out+=64
- SUBQ $1, R9
- JNZ loopback_xor_sse2_64
-
-done_xor_sse2_64:
- RET
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_arm64.go b/vendor/github.com/klauspost/reedsolomon/galois_arm64.go
deleted file mode 100644
index 92b67b8..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_arm64.go
+++ /dev/null
@@ -1,96 +0,0 @@
-//go:build !noasm && !appengine && !gccgo
-// +build !noasm,!appengine,!gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2017, Minio, Inc.
-
-package reedsolomon
-
-//go:noescape
-func galMulNEON(low, high, in, out []byte)
-
-//go:noescape
-func galMulXorNEON(low, high, in, out []byte)
-
-//go:noescape
-func galXorNEON(in, out []byte)
-
-func galMulSlice(c byte, in, out []byte, o *options) {
- if c == 1 {
- copy(out, in)
- return
- }
- var done int
- galMulNEON(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done = (len(in) >> 5) << 5
-
- remain := len(in) - done
- if remain > 0 {
- mt := mulTable[c][:256]
- for i := done; i < len(in); i++ {
- out[i] = mt[in[i]]
- }
- }
-}
-
-func galMulSliceXor(c byte, in, out []byte, o *options) {
- if c == 1 {
- sliceXor(in, out, o)
- return
- }
- var done int
- galMulXorNEON(mulTableLow[c][:], mulTableHigh[c][:], in, out)
- done = (len(in) >> 5) << 5
-
- remain := len(in) - done
- if remain > 0 {
- mt := mulTable[c][:256]
- for i := done; i < len(in); i++ {
- out[i] ^= mt[in[i]]
- }
- }
-}
-
-// simple slice xor
-func sliceXor(in, out []byte, o *options) {
-
- galXorNEON(in, out)
- done := (len(in) >> 5) << 5
-
- remain := len(in) - done
- if remain > 0 {
- for i := done; i < len(in); i++ {
- out[i] ^= in[i]
- }
- }
-}
-
-// 4-way butterfly
-func ifftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- ifftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 4-way butterfly
-func fftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- fftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 2-way butterfly forward
-func fftDIT2(x, y []byte, log_m ffe, o *options) {
- // Reference version:
- refMulAdd(x, y, log_m)
- // 64 byte aligned, always full.
- galXorNEON(x, y)
-}
-
-// 2-way butterfly
-func ifftDIT2(x, y []byte, log_m ffe, o *options) {
- // 64 byte aligned, always full.
- galXorNEON(x, y)
- // Reference version:
- refMulAdd(x, y, log_m)
-}
-
-func mulgf16(x, y []byte, log_m ffe, o *options) {
- refMul(x, y, log_m)
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_arm64.s b/vendor/github.com/klauspost/reedsolomon/galois_arm64.s
deleted file mode 100644
index 3ae3237..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_arm64.s
+++ /dev/null
@@ -1,127 +0,0 @@
-//+build !noasm
-//+build !appengine
-//+build !gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2017, Minio, Inc.
-
-#define LOAD(LO1, LO2, HI1, HI2) \
- VLD1.P 32(R1), [LO1.B16, LO2.B16] \
- \
- \ // Get low input and high input
- VUSHR $4, LO1.B16, HI1.B16 \
- VUSHR $4, LO2.B16, HI2.B16 \
- VAND V8.B16, LO1.B16, LO1.B16 \
- VAND V8.B16, LO2.B16, LO2.B16
-
-#define GALOIS_MUL(MUL_LO, MUL_HI, OUT1, OUT2, TMP1, TMP2) \
- \ // Mul low part and mul high part
- VTBL V0.B16, [MUL_LO.B16], OUT1.B16 \
- VTBL V10.B16, [MUL_HI.B16], OUT2.B16 \
- VTBL V1.B16, [MUL_LO.B16], TMP1.B16 \
- VTBL V11.B16, [MUL_HI.B16], TMP2.B16 \
- \
- \ // Combine results
- VEOR OUT2.B16, OUT1.B16, OUT1.B16 \
- VEOR TMP2.B16, TMP1.B16, OUT2.B16
-
-// func galMulNEON(low, high, in, out []byte)
-TEXT ·galMulNEON(SB), 7, $0
- MOVD in_base+48(FP), R1
- MOVD in_len+56(FP), R2 // length of message
- MOVD out_base+72(FP), R5
- SUBS $32, R2
- BMI complete
-
- MOVD low+0(FP), R10 // R10: &low
- MOVD high+24(FP), R11 // R11: &high
- VLD1 (R10), [V6.B16]
- VLD1 (R11), [V7.B16]
-
- //
- // Use an extra instruction below since `VDUP R3, V8.B16` generates assembler error
- // WORD $0x4e010c68 // dup v8.16b, w3
- //
- MOVD $0x0f, R3
- VMOV R3, V8.B[0]
- VDUP V8.B[0], V8.B16
-
-loop:
- // Main loop
- LOAD(V0, V1, V10, V11)
- GALOIS_MUL(V6, V7, V4, V5, V14, V15)
-
- // Store result
- VST1.P [V4.D2, V5.D2], 32(R5)
-
- SUBS $32, R2
- BPL loop
-
-complete:
- RET
-
-// func galMulXorNEON(low, high, in, out []byte)
-TEXT ·galMulXorNEON(SB), 7, $0
- MOVD in_base+48(FP), R1
- MOVD in_len+56(FP), R2 // length of message
- MOVD out_base+72(FP), R5
- SUBS $32, R2
- BMI completeXor
-
- MOVD low+0(FP), R10 // R10: &low
- MOVD high+24(FP), R11 // R11: &high
- VLD1 (R10), [V6.B16]
- VLD1 (R11), [V7.B16]
-
- //
- // Use an extra instruction below since `VDUP R3, V8.B16` generates assembler error
- // WORD $0x4e010c68 // dup v8.16b, w3
- //
- MOVD $0x0f, R3
- VMOV R3, V8.B[0]
- VDUP V8.B[0], V8.B16
-
-loopXor:
- // Main loop
- VLD1 (R5), [V20.B16, V21.B16]
-
- LOAD(V0, V1, V10, V11)
- GALOIS_MUL(V6, V7, V4, V5, V14, V15)
-
- VEOR V20.B16, V4.B16, V4.B16
- VEOR V21.B16, V5.B16, V5.B16
-
- // Store result
- VST1.P [V4.D2, V5.D2], 32(R5)
-
- SUBS $32, R2
- BPL loopXor
-
-completeXor:
- RET
-
-// func galXorNEON(in, out []byte)
-TEXT ·galXorNEON(SB), 7, $0
- MOVD in_base+0(FP), R1
- MOVD in_len+8(FP), R2 // length of message
- MOVD out_base+24(FP), R5
- SUBS $32, R2
- BMI completeXor
-
-loopXor:
- // Main loop
- VLD1.P 32(R1), [V0.B16, V1.B16]
- VLD1 (R5), [V20.B16, V21.B16]
-
- VEOR V20.B16, V0.B16, V4.B16
- VEOR V21.B16, V1.B16, V5.B16
-
- // Store result
- VST1.P [V4.D2, V5.D2], 32(R5)
-
- SUBS $32, R2
- BPL loopXor
-
-completeXor:
- RET
-
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.go b/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.go
deleted file mode 100644
index 24d6a02..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.go
+++ /dev/null
@@ -1,1551 +0,0 @@
-// Code generated by command: go run gen.go -out ../galois_gen_amd64.s -stubs ../galois_gen_amd64.go -pkg=reedsolomon. DO NOT EDIT.
-
-//go:build !appengine && !noasm && !nogen && gc
-
-package reedsolomon
-
-func _dummy_()
-
-// mulAvxTwo_1x1 takes 1 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x1_64 takes 1 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x1Xor takes 1 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x1_64Xor takes 1 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x2 takes 1 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x2_64 takes 1 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x2Xor takes 1 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x2_64Xor takes 1 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x3 takes 1 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x3_64 takes 1 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x3Xor takes 1 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x3_64Xor takes 1 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x4 takes 1 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x4Xor takes 1 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x5 takes 1 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x5Xor takes 1 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x6 takes 1 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x6Xor takes 1 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x7 takes 1 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x7Xor takes 1 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x8 takes 1 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x8Xor takes 1 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x9 takes 1 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x9Xor takes 1 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x10 takes 1 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_1x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_1x10Xor takes 1 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_1x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x1 takes 2 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x1_64 takes 2 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x1Xor takes 2 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x1_64Xor takes 2 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x2 takes 2 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x2_64 takes 2 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x2Xor takes 2 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x2_64Xor takes 2 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x3 takes 2 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x3_64 takes 2 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x3Xor takes 2 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x3_64Xor takes 2 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x4 takes 2 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x4Xor takes 2 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x5 takes 2 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x5Xor takes 2 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x6 takes 2 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x6Xor takes 2 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x7 takes 2 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x7Xor takes 2 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x8 takes 2 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x8Xor takes 2 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x9 takes 2 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x9Xor takes 2 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x10 takes 2 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_2x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_2x10Xor takes 2 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_2x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x1 takes 3 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x1_64 takes 3 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x1Xor takes 3 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x1_64Xor takes 3 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x2 takes 3 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x2_64 takes 3 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x2Xor takes 3 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x2_64Xor takes 3 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x3 takes 3 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x3_64 takes 3 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x3Xor takes 3 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x3_64Xor takes 3 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x4 takes 3 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x4Xor takes 3 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x5 takes 3 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x5Xor takes 3 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x6 takes 3 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x6Xor takes 3 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x7 takes 3 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x7Xor takes 3 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x8 takes 3 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x8Xor takes 3 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x9 takes 3 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x9Xor takes 3 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x10 takes 3 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_3x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_3x10Xor takes 3 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_3x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x1 takes 4 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x1_64 takes 4 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x1Xor takes 4 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x1_64Xor takes 4 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x2 takes 4 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x2_64 takes 4 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x2Xor takes 4 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x2_64Xor takes 4 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x3 takes 4 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x3_64 takes 4 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x3Xor takes 4 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x3_64Xor takes 4 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x4 takes 4 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x4Xor takes 4 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x5 takes 4 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x5Xor takes 4 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x6 takes 4 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x6Xor takes 4 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x7 takes 4 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x7Xor takes 4 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x8 takes 4 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x8Xor takes 4 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x9 takes 4 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x9Xor takes 4 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x10 takes 4 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_4x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_4x10Xor takes 4 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_4x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x1 takes 5 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x1_64 takes 5 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x1Xor takes 5 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x1_64Xor takes 5 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x2 takes 5 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x2_64 takes 5 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x2Xor takes 5 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x2_64Xor takes 5 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x3 takes 5 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x3_64 takes 5 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x3Xor takes 5 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x3_64Xor takes 5 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x4 takes 5 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x4Xor takes 5 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x5 takes 5 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x5Xor takes 5 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x6 takes 5 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x6Xor takes 5 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x7 takes 5 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x7Xor takes 5 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x8 takes 5 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x8Xor takes 5 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x9 takes 5 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x9Xor takes 5 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x10 takes 5 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_5x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_5x10Xor takes 5 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_5x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x1 takes 6 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x1_64 takes 6 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x1Xor takes 6 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x1_64Xor takes 6 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x2 takes 6 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x2_64 takes 6 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x2Xor takes 6 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x2_64Xor takes 6 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x3 takes 6 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x3_64 takes 6 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x3Xor takes 6 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x3_64Xor takes 6 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x4 takes 6 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x4Xor takes 6 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x5 takes 6 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x5Xor takes 6 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x6 takes 6 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x6Xor takes 6 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x7 takes 6 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x7Xor takes 6 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x8 takes 6 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x8Xor takes 6 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x9 takes 6 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x9Xor takes 6 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x10 takes 6 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_6x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_6x10Xor takes 6 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_6x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x1 takes 7 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x1_64 takes 7 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x1Xor takes 7 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x1_64Xor takes 7 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x2 takes 7 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x2_64 takes 7 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x2Xor takes 7 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x2_64Xor takes 7 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x3 takes 7 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x3_64 takes 7 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x3Xor takes 7 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x3_64Xor takes 7 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x4 takes 7 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x4Xor takes 7 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x5 takes 7 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x5Xor takes 7 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x6 takes 7 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x6Xor takes 7 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x7 takes 7 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x7Xor takes 7 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x8 takes 7 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x8Xor takes 7 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x9 takes 7 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x9Xor takes 7 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x10 takes 7 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_7x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_7x10Xor takes 7 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_7x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x1 takes 8 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x1_64 takes 8 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x1Xor takes 8 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x1_64Xor takes 8 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x2 takes 8 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x2_64 takes 8 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x2Xor takes 8 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x2_64Xor takes 8 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x3 takes 8 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x3_64 takes 8 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x3Xor takes 8 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x3_64Xor takes 8 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x4 takes 8 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x4Xor takes 8 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x5 takes 8 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x5Xor takes 8 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x6 takes 8 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x6Xor takes 8 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x7 takes 8 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x7Xor takes 8 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x8 takes 8 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x8Xor takes 8 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x9 takes 8 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x9Xor takes 8 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x10 takes 8 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_8x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_8x10Xor takes 8 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_8x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x1 takes 9 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x1_64 takes 9 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x1Xor takes 9 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x1_64Xor takes 9 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x2 takes 9 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x2_64 takes 9 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x2Xor takes 9 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x2_64Xor takes 9 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x3 takes 9 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x3_64 takes 9 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x3Xor takes 9 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x3_64Xor takes 9 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x4 takes 9 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x4Xor takes 9 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x5 takes 9 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x5Xor takes 9 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x6 takes 9 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x6Xor takes 9 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x7 takes 9 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x7Xor takes 9 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x8 takes 9 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x8Xor takes 9 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x9 takes 9 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x9Xor takes 9 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x10 takes 9 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_9x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_9x10Xor takes 9 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_9x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x1 takes 10 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x1_64 takes 10 inputs and produces 1 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x1Xor takes 10 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x1_64Xor takes 10 inputs and produces 1 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x2 takes 10 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x2_64 takes 10 inputs and produces 2 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x2Xor takes 10 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x2_64Xor takes 10 inputs and produces 2 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x3 takes 10 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x3_64 takes 10 inputs and produces 3 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x3Xor takes 10 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x3_64Xor takes 10 inputs and produces 3 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x4 takes 10 inputs and produces 4 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x4Xor takes 10 inputs and produces 4 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x5 takes 10 inputs and produces 5 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x5Xor takes 10 inputs and produces 5 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x6 takes 10 inputs and produces 6 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x6Xor takes 10 inputs and produces 6 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x7 takes 10 inputs and produces 7 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x7Xor takes 10 inputs and produces 7 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x8 takes 10 inputs and produces 8 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x8Xor takes 10 inputs and produces 8 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x9 takes 10 inputs and produces 9 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x9Xor takes 10 inputs and produces 9 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x10 takes 10 inputs and produces 10 outputs.
-// The output is initialized to 0.
-//
-//go:noescape
-func mulAvxTwo_10x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-// mulAvxTwo_10x10Xor takes 10 inputs and produces 10 outputs.
-//
-//go:noescape
-func mulAvxTwo_10x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-
-//go:noescape
-func ifftDIT2_avx2(x []byte, y []byte, table *[128]uint8)
-
-//go:noescape
-func fftDIT2_avx2(x []byte, y []byte, table *[128]uint8)
-
-//go:noescape
-func mulgf16_avx2(x []byte, y []byte, table *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx512_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx512_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT4_avx2_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func fftDIT4_avx2_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-
-//go:noescape
-func ifftDIT2_ssse3(x []byte, y []byte, table *[128]uint8)
-
-//go:noescape
-func fftDIT2_ssse3(x []byte, y []byte, table *[128]uint8)
-
-//go:noescape
-func mulgf16_ssse3(x []byte, y []byte, table *[128]uint8)
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.s b/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.s
deleted file mode 100644
index 890461e..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_gen_amd64.s
+++ /dev/null
@@ -1,67609 +0,0 @@
-// Code generated by command: go run gen.go -out ../galois_gen_amd64.s -stubs ../galois_gen_amd64.go -pkg=reedsolomon. DO NOT EDIT.
-
-//go:build !appengine && !noasm && !nogen && gc
-
-#include "textflag.h"
-
-// func _dummy_()
-TEXT ·_dummy_(SB), $0
-#ifdef GOAMD64_v4
-#define XOR3WAY(ignore, a, b, dst) \
- VPTERNLOGD $0x96, a, b, dst
-
-#else
-#define XOR3WAY(ignore, a, b, dst) \
- VPXOR a, dst, dst \
- VPXOR b, dst, dst
-
-#endif
- RET
-
-// func mulAvxTwo_1x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 6 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), DX
- MOVQ start+72(FP), BX
-
- // Add start offset to output
- ADDQ BX, DX
-
- // Add start offset to input
- ADDQ BX, CX
- MOVQ $0x0000000f, BX
- MOVQ BX, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_1x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (CX), Y2
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y2, Y4
- VPAND Y3, Y2, Y2
- VPAND Y3, Y4, Y4
- VPSHUFB Y2, Y0, Y2
- VPSHUFB Y4, Y1, Y4
- VPXOR Y2, Y4, Y2
-
- // Store 1 outputs
- VMOVDQU Y2, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x1_loop
- VZEROUPPER
-
-mulAvxTwo_1x1_end:
- RET
-
-// func mulAvxTwo_1x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x1_64(SB), $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 10 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x1_64_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), DX
- MOVQ start+72(FP), BX
-
- // Add start offset to output
- ADDQ BX, DX
-
- // Add start offset to input
- ADDQ BX, CX
- MOVQ $0x0000000f, BX
- MOVQ BX, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- ADDQ $0x40, CX
- VPSRLQ $0x04, Y2, Y6
- VPSRLQ $0x04, Y3, Y5
- VPAND Y4, Y2, Y2
- VPAND Y4, Y3, Y3
- VPAND Y4, Y6, Y6
- VPAND Y4, Y5, Y5
- VPSHUFB Y2, Y0, Y2
- VPSHUFB Y3, Y0, Y3
- VPSHUFB Y6, Y1, Y6
- VPSHUFB Y5, Y1, Y5
- VPXOR Y2, Y6, Y2
- VPXOR Y3, Y5, Y3
-
- // Store 1 outputs
- VMOVDQU Y2, (DX)
- VMOVDQU Y3, 32(DX)
- ADDQ $0x40, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_1x1_64_end:
- RET
-
-// func mulAvxTwo_1x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 6 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), DX
- MOVQ start+72(FP), BX
-
- // Add start offset to output
- ADDQ BX, DX
-
- // Add start offset to input
- ADDQ BX, CX
- MOVQ $0x0000000f, BX
- MOVQ BX, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_1x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (CX), Y4
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y3, Y4, Y4
- VPAND Y3, Y5, Y5
- VMOVDQU (DX), Y2
- VPSHUFB Y4, Y0, Y4
- VPSHUFB Y5, Y1, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 1 outputs
- VMOVDQU Y2, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x1Xor_end:
- RET
-
-// func mulAvxTwo_1x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x1_64Xor(SB), $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 10 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x1_64Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), DX
- MOVQ start+72(FP), BX
-
- // Add start offset to output
- ADDQ BX, DX
-
- // Add start offset to input
- ADDQ BX, CX
- MOVQ $0x0000000f, BX
- MOVQ BX, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (DX), Y2
- VMOVDQU 32(DX), Y3
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y7
- ADDQ $0x40, CX
- VPSRLQ $0x04, Y5, Y6
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y5, Y5
- VPAND Y4, Y7, Y7
- VPAND Y4, Y6, Y6
- VPAND Y4, Y8, Y8
- VPSHUFB Y5, Y0, Y5
- VPSHUFB Y7, Y0, Y7
- VPSHUFB Y6, Y1, Y6
- VPSHUFB Y8, Y1, Y8
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 1 outputs
- VMOVDQU Y2, (DX)
- VMOVDQU Y3, 32(DX)
- ADDQ $0x40, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x1_64Xor_end:
- RET
-
-// func mulAvxTwo_1x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x2(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 11 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x2_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
- ADDQ SI, DX
-
- // Add start offset to input
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (CX), Y8
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y9, Y9
- VPSHUFB Y8, Y0, Y5
- VPSHUFB Y9, Y1, Y7
- VPXOR Y5, Y7, Y4
- VPSHUFB Y8, Y2, Y5
- VPSHUFB Y9, Y3, Y7
- VPXOR Y5, Y7, Y5
-
- // Store 2 outputs
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
- VMOVDQU Y5, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x2_loop
- VZEROUPPER
-
-mulAvxTwo_1x2_end:
- RET
-
-// func mulAvxTwo_1x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 17 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), BX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
- ADDQ DI, BX
-
- // Add start offset to input
- ADDQ DI, DX
- MOVQ $0x0000000f, DI
- MOVQ DI, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y9
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y7, Y8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y4, Y7, Y7
- VPAND Y4, Y9, Y9
- VPAND Y4, Y8, Y8
- VPAND Y4, Y10, Y10
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y6
- VPSHUFB Y9, Y2, Y3
- VPSHUFB Y7, Y2, Y2
- VPSHUFB Y10, Y6, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y2, Y6, Y0
- VPXOR Y3, Y5, Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y6
- VPSHUFB Y9, Y2, Y3
- VPSHUFB Y7, Y2, Y2
- VPSHUFB Y10, Y6, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y2, Y6, Y2
- VPXOR Y3, Y5, Y3
-
- // Store 2 outputs
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (BX)
- VMOVDQU Y3, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_1x2_64_end:
- RET
-
-// func mulAvxTwo_1x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x2Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 11 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x2Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
- ADDQ SI, DX
-
- // Add start offset to input
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (CX), Y9
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (BX), Y4
- VPSHUFB Y9, Y0, Y7
- VPSHUFB Y10, Y1, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (DX), Y5
- VPSHUFB Y9, Y2, Y7
- VPSHUFB Y10, Y3, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 2 outputs
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
- VMOVDQU Y5, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x2Xor_end:
- RET
-
-// func mulAvxTwo_1x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 17 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), BX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
- ADDQ DI, BX
-
- // Add start offset to input
- ADDQ DI, DX
- MOVQ $0x0000000f, DI
- MOVQ DI, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (BX), Y2
- VMOVDQU 32(BX), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (BX)
- VMOVDQU Y3, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x2_64Xor_end:
- RET
-
-// func mulAvxTwo_1x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x3(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x3_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, BX
- ADDQ DI, SI
- ADDQ DI, DX
-
- // Add start offset to input
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_1x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (CX), Y11
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y9, Y11, Y11
- VPAND Y9, Y12, Y12
- VPSHUFB Y11, Y0, Y8
- VPSHUFB Y12, Y1, Y10
- VPXOR Y8, Y10, Y6
- VPSHUFB Y11, Y2, Y8
- VPSHUFB Y12, Y3, Y10
- VPXOR Y8, Y10, Y7
- VPSHUFB Y11, Y4, Y8
- VPSHUFB Y12, Y5, Y10
- VPXOR Y8, Y10, Y8
-
- // Store 3 outputs
- VMOVDQU Y6, (BX)
- ADDQ $0x20, BX
- VMOVDQU Y7, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y8, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x3_loop
- VZEROUPPER
-
-mulAvxTwo_1x3_end:
- RET
-
-// func mulAvxTwo_1x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), BX
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, SI
- ADDQ R8, DI
- ADDQ R8, BX
-
- // Add start offset to input
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y6, Y9, Y9
- VPAND Y6, Y11, Y11
- VPAND Y6, Y10, Y10
- VPAND Y6, Y12, Y12
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y8
- VPSHUFB Y11, Y4, Y5
- VPSHUFB Y9, Y4, Y4
- VPSHUFB Y12, Y8, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y4, Y8, Y0
- VPXOR Y5, Y7, Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y8
- VPSHUFB Y11, Y4, Y5
- VPSHUFB Y9, Y4, Y4
- VPSHUFB Y12, Y8, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y4, Y8, Y2
- VPXOR Y5, Y7, Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y8
- VPSHUFB Y11, Y4, Y5
- VPSHUFB Y9, Y4, Y4
- VPSHUFB Y12, Y8, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y4, Y8, Y4
- VPXOR Y5, Y7, Y5
-
- // Store 3 outputs
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y4, (BX)
- VMOVDQU Y5, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_1x3_64_end:
- RET
-
-// func mulAvxTwo_1x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x3Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x3Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- MOVQ in_base+24(FP), CX
- MOVQ (CX), CX
- MOVQ out_base+48(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, BX
- ADDQ DI, SI
- ADDQ DI, DX
-
- // Add start offset to input
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_1x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (CX), Y12
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (BX), Y6
- VPSHUFB Y12, Y0, Y10
- VPSHUFB Y13, Y1, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU (SI), Y7
- VPSHUFB Y12, Y2, Y10
- VPSHUFB Y13, Y3, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU (DX), Y8
- VPSHUFB Y12, Y4, Y10
- VPSHUFB Y13, Y5, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 3 outputs
- VMOVDQU Y6, (BX)
- ADDQ $0x20, BX
- VMOVDQU Y7, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y8, (DX)
- ADDQ $0x20, DX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x3Xor_end:
- RET
-
-// func mulAvxTwo_1x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), BX
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, SI
- ADDQ R8, DI
- ADDQ R8, BX
-
- // Add start offset to input
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
- VMOVDQU (BX), Y4
- VMOVDQU 32(BX), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y4, (BX)
- VMOVDQU Y5, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x3_64Xor_end:
- RET
-
-// func mulAvxTwo_1x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 17 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), BX
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, BX
-
- // Add start offset to input
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y4, Y6, Y6
- VPAND Y4, Y7, Y7
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y7, Y5, Y5
- VPXOR Y3, Y5, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y7, Y5, Y5
- VPXOR Y3, Y5, Y1
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y7, Y5, Y5
- VPXOR Y3, Y5, Y2
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y7, Y5, Y5
- VPXOR Y3, Y5, Y3
-
- // Store 4 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x4_loop
- VZEROUPPER
-
-mulAvxTwo_1x4_end:
- RET
-
-// func mulAvxTwo_1x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 17 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), BX
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, BX
-
- // Add start offset to input
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_1x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (BX), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x4Xor_end:
- RET
-
-// func mulAvxTwo_1x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), BX
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, BX
-
- // Add start offset to input
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_1x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y5, Y7, Y7
- VPAND Y5, Y8, Y8
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y4, Y4
- VPSHUFB Y8, Y6, Y6
- VPXOR Y4, Y6, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y4, Y4
- VPSHUFB Y8, Y6, Y6
- VPXOR Y4, Y6, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y4, Y4
- VPSHUFB Y8, Y6, Y6
- VPXOR Y4, Y6, Y2
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y4, Y4
- VPSHUFB Y8, Y6, Y6
- VPXOR Y4, Y6, Y3
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y4, Y4
- VPSHUFB Y8, Y6, Y6
- VPXOR Y4, Y6, Y4
-
- // Store 5 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x5_loop
- VZEROUPPER
-
-mulAvxTwo_1x5_end:
- RET
-
-// func mulAvxTwo_1x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), BX
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, BX
-
- // Add start offset to input
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_1x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (BX), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x5Xor_end:
- RET
-
-// func mulAvxTwo_1x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 23 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), BX
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, BX
-
- // Add start offset to input
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y9, Y9
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y3
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y4
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y5, Y5
- VPSHUFB Y9, Y7, Y7
- VPXOR Y5, Y7, Y5
-
- // Store 6 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x6_loop
- VZEROUPPER
-
-mulAvxTwo_1x6_end:
- RET
-
-// func mulAvxTwo_1x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 23 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), BX
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, BX
-
- // Add start offset to input
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_1x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (BX), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x6Xor_end:
- RET
-
-// func mulAvxTwo_1x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), BX
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, BX
-
- // Add start offset to input
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_1x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y7, Y9, Y9
- VPAND Y7, Y10, Y10
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y4
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y5
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y6, Y6
- VPSHUFB Y10, Y8, Y8
- VPXOR Y6, Y8, Y6
-
- // Store 7 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x7_loop
- VZEROUPPER
-
-mulAvxTwo_1x7_end:
- RET
-
-// func mulAvxTwo_1x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), BX
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, BX
-
- // Add start offset to input
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_1x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R11), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (BX), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x7Xor_end:
- RET
-
-// func mulAvxTwo_1x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 29 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), BX
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, BX
-
- // Add start offset to input
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_1x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y8, Y10, Y10
- VPAND Y8, Y11, Y11
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y5
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y6
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y7, Y7
- VPSHUFB Y11, Y9, Y9
- VPXOR Y7, Y9, Y7
-
- // Store 8 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x8_loop
- VZEROUPPER
-
-mulAvxTwo_1x8_end:
- RET
-
-// func mulAvxTwo_1x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 29 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), BX
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, BX
-
- // Add start offset to input
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_1x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU (R11), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU (R12), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU (BX), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x8Xor_end:
- RET
-
-// func mulAvxTwo_1x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 32 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), R13
- MOVQ 192(BX), BX
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, BX
-
- // Add start offset to input
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_1x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y9, Y11, Y11
- VPAND Y9, Y12, Y12
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y6
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y7
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y8, Y8
- VPSHUFB Y12, Y10, Y10
- VPXOR Y8, Y10, Y8
-
- // Store 9 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y8, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x9_loop
- VZEROUPPER
-
-mulAvxTwo_1x9_end:
- RET
-
-// func mulAvxTwo_1x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 32 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), R13
- MOVQ 192(BX), BX
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, BX
-
- // Add start offset to input
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_1x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU (R11), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU (R12), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU (R13), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU (BX), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y8, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x9Xor_end:
- RET
-
-// func mulAvxTwo_1x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulAvxTwo_1x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), R13
- MOVQ 192(BX), R14
- MOVQ 216(BX), BX
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, BX
-
- // Add start offset to input
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_1x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y10, Y12, Y12
- VPAND Y10, Y13, Y13
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y7
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y8
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y9, Y9
- VPSHUFB Y13, Y11, Y11
- VPXOR Y9, Y11, Y9
-
- // Store 10 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y8, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y9, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x10_loop
- VZEROUPPER
-
-mulAvxTwo_1x10_end:
- RET
-
-// func mulAvxTwo_1x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_1x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_1x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), DX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), DI
- MOVQ 48(BX), R8
- MOVQ 72(BX), R9
- MOVQ 96(BX), R10
- MOVQ 120(BX), R11
- MOVQ 144(BX), R12
- MOVQ 168(BX), R13
- MOVQ 192(BX), R14
- MOVQ 216(BX), BX
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, BX
-
- // Add start offset to input
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_1x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (SI), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU (R11), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU (R12), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU (R13), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU (R14), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU (BX), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- VMOVDQU Y0, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y5, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y6, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y7, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y8, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y9, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_1x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_1x10Xor_end:
- RET
-
-// func mulAvxTwo_2x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 8 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), BX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
-
- // Add start offset to input
- ADDQ SI, DX
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_2x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y5, Y6, Y6
- VPAND Y5, Y7, Y7
- VPSHUFB Y6, Y0, Y6
- VPSHUFB Y7, Y1, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (CX), Y6
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y5, Y6, Y6
- VPAND Y5, Y7, Y7
- VPSHUFB Y6, Y2, Y6
- VPSHUFB Y7, Y3, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 1 outputs
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x1_loop
- VZEROUPPER
-
-mulAvxTwo_2x1_end:
- RET
-
-// func mulAvxTwo_2x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x1_64(SB), $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x1_64_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), BX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
-
- // Add start offset to input
- ADDQ SI, DX
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y9
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y7, Y8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y7, Y7
- VPAND Y6, Y9, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y10, Y10
- VPSHUFB Y7, Y0, Y7
- VPSHUFB Y9, Y0, Y9
- VPSHUFB Y8, Y1, Y8
- VPSHUFB Y10, Y1, Y10
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y9
- ADDQ $0x40, CX
- VPSRLQ $0x04, Y7, Y8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y7, Y7
- VPAND Y6, Y9, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y10, Y10
- VPSHUFB Y7, Y2, Y7
- VPSHUFB Y9, Y2, Y9
- VPSHUFB Y8, Y3, Y8
- VPSHUFB Y10, Y3, Y10
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 1 outputs
- VMOVDQU Y4, (BX)
- VMOVDQU Y5, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_2x1_64_end:
- RET
-
-// func mulAvxTwo_2x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 8 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), BX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
-
- // Add start offset to input
- ADDQ SI, DX
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_2x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y5, Y6, Y6
- VPAND Y5, Y7, Y7
- VMOVDQU (BX), Y4
- VPSHUFB Y6, Y0, Y6
- VPSHUFB Y7, Y1, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (CX), Y6
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y5, Y6, Y6
- VPAND Y5, Y7, Y7
- VPSHUFB Y6, Y2, Y6
- VPSHUFB Y7, Y3, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 1 outputs
- VMOVDQU Y4, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x1Xor_end:
- RET
-
-// func mulAvxTwo_2x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x1_64Xor(SB), $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x1_64Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), BX
- MOVQ start+72(FP), SI
-
- // Add start offset to output
- ADDQ SI, BX
-
- // Add start offset to input
- ADDQ SI, DX
- ADDQ SI, CX
- MOVQ $0x0000000f, SI
- MOVQ SI, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (BX), Y4
- VMOVDQU 32(BX), Y5
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y9
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y7, Y8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y7, Y7
- VPAND Y6, Y9, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y10, Y10
- VPSHUFB Y7, Y0, Y7
- VPSHUFB Y9, Y0, Y9
- VPSHUFB Y8, Y1, Y8
- VPSHUFB Y10, Y1, Y10
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y9
- ADDQ $0x40, CX
- VPSRLQ $0x04, Y7, Y8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y7, Y7
- VPAND Y6, Y9, Y9
- VPAND Y6, Y8, Y8
- VPAND Y6, Y10, Y10
- VPSHUFB Y7, Y2, Y7
- VPSHUFB Y9, Y2, Y9
- VPSHUFB Y8, Y3, Y8
- VPSHUFB Y10, Y3, Y10
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 1 outputs
- VMOVDQU Y4, (BX)
- VMOVDQU Y5, 32(BX)
- ADDQ $0x40, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x1_64Xor_end:
- RET
-
-// func mulAvxTwo_2x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x2(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 15 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x2_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), BX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
- ADDQ DI, BX
-
- // Add start offset to input
- ADDQ DI, DX
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_2x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VPSHUFB Y13, Y0, Y11
- VPSHUFB Y14, Y1, Y12
- VPXOR Y11, Y12, Y8
- VPSHUFB Y13, Y2, Y11
- VPSHUFB Y14, Y3, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (CX), Y13
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VPSHUFB Y13, Y4, Y11
- VPSHUFB Y14, Y5, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VPSHUFB Y13, Y6, Y11
- VPSHUFB Y14, Y7, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 2 outputs
- VMOVDQU Y8, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y9, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x2_loop
- VZEROUPPER
-
-mulAvxTwo_2x2_end:
- RET
-
-// func mulAvxTwo_2x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 25 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), SI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
- ADDQ R8, SI
-
- // Add start offset to input
- ADDQ R8, BX
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_2x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y2, (SI)
- VMOVDQU Y3, 32(SI)
- ADDQ $0x40, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_2x2_64_end:
- RET
-
-// func mulAvxTwo_2x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x2Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 15 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x2Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), CX
- MOVQ out_base+48(FP), BX
- MOVQ (BX), SI
- MOVQ 24(BX), BX
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
- ADDQ DI, BX
-
- // Add start offset to input
- ADDQ DI, DX
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_2x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (SI), Y8
- VPSHUFB Y13, Y0, Y11
- VPSHUFB Y14, Y1, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU (BX), Y9
- VPSHUFB Y13, Y2, Y11
- VPSHUFB Y14, Y3, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (CX), Y13
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VPSHUFB Y13, Y4, Y11
- VPSHUFB Y14, Y5, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VPSHUFB Y13, Y6, Y11
- VPSHUFB Y14, Y7, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 2 outputs
- VMOVDQU Y8, (SI)
- ADDQ $0x20, SI
- VMOVDQU Y9, (BX)
- ADDQ $0x20, BX
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x2Xor_end:
- RET
-
-// func mulAvxTwo_2x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 25 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), SI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
- ADDQ R8, SI
-
- // Add start offset to input
- ADDQ R8, BX
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_2x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (DI), Y0
- VMOVDQU 32(DI), Y1
- VMOVDQU (SI), Y2
- VMOVDQU 32(SI), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y2, (SI)
- VMOVDQU Y3, 32(SI)
- ADDQ $0x40, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x2_64Xor_end:
- RET
-
-// func mulAvxTwo_2x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), SI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, SI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_2x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x3_loop
- VZEROUPPER
-
-mulAvxTwo_2x3_end:
- RET
-
-// func mulAvxTwo_2x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 34 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), SI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, SI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y2, (R8)
- VMOVDQU Y3, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y4, (SI)
- VMOVDQU Y5, 32(SI)
- ADDQ $0x40, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_2x3_64_end:
- RET
-
-// func mulAvxTwo_2x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), SI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, SI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_2x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (SI), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x3Xor_end:
- RET
-
-// func mulAvxTwo_2x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 34 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), SI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, DI
- ADDQ R9, R8
- ADDQ R9, SI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (DI), Y0
- VMOVDQU 32(DI), Y1
- VMOVDQU (R8), Y2
- VMOVDQU 32(R8), Y3
- VMOVDQU (SI), Y4
- VMOVDQU 32(SI), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y2, (R8)
- VMOVDQU Y3, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y4, (SI)
- VMOVDQU Y5, 32(SI)
- ADDQ $0x40, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x3_64Xor_end:
- RET
-
-// func mulAvxTwo_2x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 25 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), SI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, SI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_2x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x4_loop
- VZEROUPPER
-
-mulAvxTwo_2x4_end:
- RET
-
-// func mulAvxTwo_2x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 25 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), SI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, SI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_2x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (SI), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x4Xor_end:
- RET
-
-// func mulAvxTwo_2x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 30 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), SI
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, SI
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_2x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x5_loop
- VZEROUPPER
-
-mulAvxTwo_2x5_end:
- RET
-
-// func mulAvxTwo_2x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 30 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), SI
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, SI
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_2x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (SI), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x5Xor_end:
- RET
-
-// func mulAvxTwo_2x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), SI
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, SI
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x6_loop
- VZEROUPPER
-
-mulAvxTwo_2x6_end:
- RET
-
-// func mulAvxTwo_2x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), SI
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, SI
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_2x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (SI), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x6Xor_end:
- RET
-
-// func mulAvxTwo_2x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 40 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), SI
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, SI
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_2x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x7_loop
- VZEROUPPER
-
-mulAvxTwo_2x7_end:
- RET
-
-// func mulAvxTwo_2x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 40 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), SI
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, SI
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_2x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R12), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (SI), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x7Xor_end:
- RET
-
-// func mulAvxTwo_2x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 45 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), SI
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, SI
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_2x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x8_loop
- VZEROUPPER
-
-mulAvxTwo_2x8_end:
- RET
-
-// func mulAvxTwo_2x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 45 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), SI
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, SI
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_2x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU (R12), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU (R13), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU (SI), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x8Xor_end:
- RET
-
-// func mulAvxTwo_2x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), SI
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, SI
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_2x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x9_loop
- VZEROUPPER
-
-mulAvxTwo_2x9_end:
- RET
-
-// func mulAvxTwo_2x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), SI
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, SI
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_2x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU (R12), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU (R13), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU (R14), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU (SI), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x9Xor_end:
- RET
-
-// func mulAvxTwo_2x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x10(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 55 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), R15
- MOVQ 216(SI), SI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, SI
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_2x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y9, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x10_loop
- VZEROUPPER
-
-mulAvxTwo_2x10_end:
- RET
-
-// func mulAvxTwo_2x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_2x10Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 55 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_2x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), DX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), R15
- MOVQ 216(SI), SI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, SI
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_2x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU (R12), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU (R13), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU (R14), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU (R15), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU (SI), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y9, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_2x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_2x10Xor_end:
- RET
-
-// func mulAvxTwo_3x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 10 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), CX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), SI
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
-
- // Add start offset to input
- ADDQ DI, DX
- ADDQ DI, BX
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_3x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VPSHUFB Y8, Y0, Y8
- VPSHUFB Y9, Y1, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VPSHUFB Y8, Y2, Y8
- VPSHUFB Y9, Y3, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (CX), Y8
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VPSHUFB Y8, Y4, Y8
- VPSHUFB Y9, Y5, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 1 outputs
- VMOVDQU Y6, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x1_loop
- VZEROUPPER
-
-mulAvxTwo_3x1_end:
- RET
-
-// func mulAvxTwo_3x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 18 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), DI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
-
- // Add start offset to input
- ADDQ R8, BX
- ADDQ R8, SI
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_3x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_3x1_64_end:
- RET
-
-// func mulAvxTwo_3x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 10 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), CX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), SI
- MOVQ start+72(FP), DI
-
- // Add start offset to output
- ADDQ DI, SI
-
- // Add start offset to input
- ADDQ DI, DX
- ADDQ DI, BX
- ADDQ DI, CX
- MOVQ $0x0000000f, DI
- MOVQ DI, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_3x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VMOVDQU (SI), Y6
- VPSHUFB Y8, Y0, Y8
- VPSHUFB Y9, Y1, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VPSHUFB Y8, Y2, Y8
- VPSHUFB Y9, Y3, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (CX), Y8
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y7, Y8, Y8
- VPAND Y7, Y9, Y9
- VPSHUFB Y8, Y4, Y8
- VPSHUFB Y9, Y5, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 1 outputs
- VMOVDQU Y6, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x1Xor_end:
- RET
-
-// func mulAvxTwo_3x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 18 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), DI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
-
- // Add start offset to input
- ADDQ R8, BX
- ADDQ R8, SI
- ADDQ R8, DX
- MOVQ $0x0000000f, R8
- MOVQ R8, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_3x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (DI), Y0
- VMOVDQU 32(DI), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (DI)
- VMOVDQU Y1, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x1_64Xor_end:
- RET
-
-// func mulAvxTwo_3x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 19 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), DI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
- ADDQ R9, DI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_3x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x2_loop
- VZEROUPPER
-
-mulAvxTwo_3x2_end:
- RET
-
-// func mulAvxTwo_3x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 33 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), DI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
- ADDQ R9, DI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_3x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_3x2_64_end:
- RET
-
-// func mulAvxTwo_3x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 19 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), DI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
- ADDQ R9, DI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_3x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (DI), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x2Xor_end:
- RET
-
-// func mulAvxTwo_3x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 33 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), DI
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
- ADDQ R9, DI
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_3x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R8), Y0
- VMOVDQU 32(R8), Y1
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x2_64Xor_end:
- RET
-
-// func mulAvxTwo_3x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), DI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, DI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_3x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x3_loop
- VZEROUPPER
-
-mulAvxTwo_3x3_end:
- RET
-
-// func mulAvxTwo_3x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 46 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), DI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, DI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_3x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y2, (R9)
- VMOVDQU Y3, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y4, (DI)
- VMOVDQU Y5, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_3x3_64_end:
- RET
-
-// func mulAvxTwo_3x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), DI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, DI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_3x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (DI), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x3Xor_end:
- RET
-
-// func mulAvxTwo_3x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 46 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), DI
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R8
- ADDQ R10, R9
- ADDQ R10, DI
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_3x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R8), Y0
- VMOVDQU 32(R8), Y1
- VMOVDQU (R9), Y2
- VMOVDQU 32(R9), Y3
- VMOVDQU (DI), Y4
- VMOVDQU 32(DI), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y2, (R9)
- VMOVDQU Y3, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y4, (DI)
- VMOVDQU Y5, 32(DI)
- ADDQ $0x40, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x3_64Xor_end:
- RET
-
-// func mulAvxTwo_3x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 33 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), DI
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, DI
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_3x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x4_loop
- VZEROUPPER
-
-mulAvxTwo_3x4_end:
- RET
-
-// func mulAvxTwo_3x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 33 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), DI
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, DI
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_3x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (DI), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x4Xor_end:
- RET
-
-// func mulAvxTwo_3x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 40 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), DI
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, DI
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_3x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x5_loop
- VZEROUPPER
-
-mulAvxTwo_3x5_end:
- RET
-
-// func mulAvxTwo_3x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 40 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), DI
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, DI
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_3x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (DI), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x5Xor_end:
- RET
-
-// func mulAvxTwo_3x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 47 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), DI
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, DI
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_3x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x6_loop
- VZEROUPPER
-
-mulAvxTwo_3x6_end:
- RET
-
-// func mulAvxTwo_3x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 47 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), DI
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, DI
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_3x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R12), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (DI), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x6Xor_end:
- RET
-
-// func mulAvxTwo_3x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 54 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), DI
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, DI
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_3x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x7_loop
- VZEROUPPER
-
-mulAvxTwo_3x7_end:
- RET
-
-// func mulAvxTwo_3x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 54 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), DI
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, DI
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_3x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R12), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R13), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (DI), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x7Xor_end:
- RET
-
-// func mulAvxTwo_3x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 61 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), DI
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, DI
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_3x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x8_loop
- VZEROUPPER
-
-mulAvxTwo_3x8_end:
- RET
-
-// func mulAvxTwo_3x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 61 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), DI
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, DI
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_3x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU (R12), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU (R13), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU (R14), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU (DI), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x8Xor_end:
- RET
-
-// func mulAvxTwo_3x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x9(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), R15
- MOVQ 192(DI), DI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, DI
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_3x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x9_loop
- VZEROUPPER
-
-mulAvxTwo_3x9_end:
- RET
-
-// func mulAvxTwo_3x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x9Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), R15
- MOVQ 192(DI), DI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, DI
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_3x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU (R12), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU (R13), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU (R14), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU (R15), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU (DI), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_3x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x9Xor_end:
- RET
-
-// func mulAvxTwo_3x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x10(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 75 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x10_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), AX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), R15
- MOVQ 216(SI), SI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, SI
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_3x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (AX), Y13
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y9, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_3x10_loop
- VZEROUPPER
-
-mulAvxTwo_3x10_end:
- RET
-
-// func mulAvxTwo_3x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_3x10Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 75 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_3x10Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), AX
- MOVQ out_base+48(FP), SI
- MOVQ (SI), DI
- MOVQ 24(SI), R8
- MOVQ 48(SI), R9
- MOVQ 72(SI), R10
- MOVQ 96(SI), R11
- MOVQ 120(SI), R12
- MOVQ 144(SI), R13
- MOVQ 168(SI), R14
- MOVQ 192(SI), R15
- MOVQ 216(SI), SI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, SI
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_3x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (DI), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU (R12), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU (R13), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU (R14), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU (R15), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU (SI), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (AX), Y13
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- VMOVDQU Y0, (DI)
- ADDQ $0x20, DI
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y5, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y6, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y7, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y8, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y9, (SI)
- ADDQ $0x20, SI
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_3x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_3x10Xor_end:
- RET
-
-// func mulAvxTwo_4x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 12 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), CX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), DI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
-
- // Add start offset to input
- ADDQ R8, DX
- ADDQ R8, BX
- ADDQ R8, SI
- ADDQ R8, CX
- MOVQ $0x0000000f, R8
- MOVQ R8, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_4x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y0, Y10
- VPSHUFB Y11, Y1, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y2, Y10
- VPSHUFB Y11, Y3, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y4, Y10
- VPSHUFB Y11, Y5, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (CX), Y10
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y6, Y10
- VPSHUFB Y11, Y7, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 1 outputs
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x1_loop
- VZEROUPPER
-
-mulAvxTwo_4x1_end:
- RET
-
-// func mulAvxTwo_4x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_4x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_4x1_64_end:
- RET
-
-// func mulAvxTwo_4x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 12 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), CX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), DI
- MOVQ start+72(FP), R8
-
- // Add start offset to output
- ADDQ R8, DI
-
- // Add start offset to input
- ADDQ R8, DX
- ADDQ R8, BX
- ADDQ R8, SI
- ADDQ R8, CX
- MOVQ $0x0000000f, R8
- MOVQ R8, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_4x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VMOVDQU (DI), Y8
- VPSHUFB Y10, Y0, Y10
- VPSHUFB Y11, Y1, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y2, Y10
- VPSHUFB Y11, Y3, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y4, Y10
- VPSHUFB Y11, Y5, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (CX), Y10
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y9, Y10, Y10
- VPAND Y9, Y11, Y11
- VPSHUFB Y10, Y6, Y10
- VPSHUFB Y11, Y7, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 1 outputs
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x1Xor_end:
- RET
-
-// func mulAvxTwo_4x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, DX
- MOVQ $0x0000000f, R9
- MOVQ R9, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_4x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R8), Y0
- VMOVDQU 32(R8), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R8)
- VMOVDQU Y1, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x1_64Xor_end:
- RET
-
-// func mulAvxTwo_4x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 23 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R8
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
- ADDQ R10, R8
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_4x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x2_loop
- VZEROUPPER
-
-mulAvxTwo_4x2_end:
- RET
-
-// func mulAvxTwo_4x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 41 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R8
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
- ADDQ R10, R8
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_4x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y2, (R8)
- VMOVDQU Y3, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_4x2_64_end:
- RET
-
-// func mulAvxTwo_4x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 23 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R8
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
- ADDQ R10, R8
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_4x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R8), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x2Xor_end:
- RET
-
-// func mulAvxTwo_4x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 41 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R8
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
- ADDQ R10, R8
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_4x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R9), Y0
- VMOVDQU 32(R9), Y1
- VMOVDQU (R8), Y2
- VMOVDQU 32(R8), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y2, (R8)
- VMOVDQU Y3, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x2_64Xor_end:
- RET
-
-// func mulAvxTwo_4x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 32 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R8
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, R8
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_4x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x3_loop
- VZEROUPPER
-
-mulAvxTwo_4x3_end:
- RET
-
-// func mulAvxTwo_4x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 58 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R8
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, R8
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_4x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y2, (R10)
- VMOVDQU Y3, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_4x3_64_end:
- RET
-
-// func mulAvxTwo_4x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 32 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R8
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, R8
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_4x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R8), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x3Xor_end:
- RET
-
-// func mulAvxTwo_4x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 58 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R8
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R9
- ADDQ R11, R10
- ADDQ R11, R8
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_4x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R9), Y0
- VMOVDQU 32(R9), Y1
- VMOVDQU (R10), Y2
- VMOVDQU 32(R10), Y3
- VMOVDQU (R8), Y4
- VMOVDQU 32(R8), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y2, (R10)
- VMOVDQU Y3, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x3_64Xor_end:
- RET
-
-// func mulAvxTwo_4x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 41 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R8
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R8
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_4x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x4_loop
- VZEROUPPER
-
-mulAvxTwo_4x4_end:
- RET
-
-// func mulAvxTwo_4x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 41 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R8
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R8
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_4x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R8), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x4Xor_end:
- RET
-
-// func mulAvxTwo_4x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R8
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R8
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_4x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x5_loop
- VZEROUPPER
-
-mulAvxTwo_4x5_end:
- RET
-
-// func mulAvxTwo_4x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R8
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R8
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_4x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (R8), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x5Xor_end:
- RET
-
-// func mulAvxTwo_4x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 59 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R8
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R8
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_4x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x6_loop
- VZEROUPPER
-
-mulAvxTwo_4x6_end:
- RET
-
-// func mulAvxTwo_4x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 59 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R8
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R8
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_4x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R13), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (R8), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x6Xor_end:
- RET
-
-// func mulAvxTwo_4x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R8
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R8
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_4x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x7_loop
- VZEROUPPER
-
-mulAvxTwo_4x7_end:
- RET
-
-// func mulAvxTwo_4x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R8
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R8
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_4x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R13), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R14), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (R8), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x7Xor_end:
- RET
-
-// func mulAvxTwo_4x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x8(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 77 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R15
- MOVQ 168(R8), R8
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R8
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_4x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y7, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x8_loop
- VZEROUPPER
-
-mulAvxTwo_4x8_end:
- RET
-
-// func mulAvxTwo_4x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x8Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 77 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R15
- MOVQ 168(R8), R8
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R8
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_4x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU (R13), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU (R14), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU (R15), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU (R8), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y7, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_4x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x8Xor_end:
- RET
-
-// func mulAvxTwo_4x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x9(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 86 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x9_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), AX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), R15
- MOVQ 192(DI), DI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, DI
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_4x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (AX), Y12
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_4x9_loop
- VZEROUPPER
-
-mulAvxTwo_4x9_end:
- RET
-
-// func mulAvxTwo_4x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x9Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 86 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x9Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), AX
- MOVQ out_base+48(FP), DI
- MOVQ (DI), R8
- MOVQ 24(DI), R9
- MOVQ 48(DI), R10
- MOVQ 72(DI), R11
- MOVQ 96(DI), R12
- MOVQ 120(DI), R13
- MOVQ 144(DI), R14
- MOVQ 168(DI), R15
- MOVQ 192(DI), DI
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, DI
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_4x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (R8), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU (R12), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU (R13), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU (R14), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU (R15), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU (DI), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (AX), Y12
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- VMOVDQU Y0, (R8)
- ADDQ $0x20, R8
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y4, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y5, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y6, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y7, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y8, (DI)
- ADDQ $0x20, DI
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_4x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x9Xor_end:
- RET
-
-// func mulAvxTwo_4x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 95 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_4x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R8), R10
- VMOVDQU Y0, (R10)(R9*1)
- MOVQ 24(R8), R10
- VMOVDQU Y1, (R10)(R9*1)
- MOVQ 48(R8), R10
- VMOVDQU Y2, (R10)(R9*1)
- MOVQ 72(R8), R10
- VMOVDQU Y3, (R10)(R9*1)
- MOVQ 96(R8), R10
- VMOVDQU Y4, (R10)(R9*1)
- MOVQ 120(R8), R10
- VMOVDQU Y5, (R10)(R9*1)
- MOVQ 144(R8), R10
- VMOVDQU Y6, (R10)(R9*1)
- MOVQ 168(R8), R10
- VMOVDQU Y7, (R10)(R9*1)
- MOVQ 192(R8), R10
- VMOVDQU Y8, (R10)(R9*1)
- MOVQ 216(R8), R10
- VMOVDQU Y9, (R10)(R9*1)
-
- // Prepare for next loop
- ADDQ $0x20, R9
- DECQ AX
- JNZ mulAvxTwo_4x10_loop
- VZEROUPPER
-
-mulAvxTwo_4x10_end:
- RET
-
-// func mulAvxTwo_4x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_4x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 95 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_4x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), DX
- MOVQ out_base+48(FP), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to input
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_4x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R8), R10
- VMOVDQU (R10)(R9*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R8), R10
- VMOVDQU (R10)(R9*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R8), R10
- VMOVDQU (R10)(R9*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R8), R10
- VMOVDQU (R10)(R9*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R8), R10
- VMOVDQU (R10)(R9*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R8), R10
- VMOVDQU (R10)(R9*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R8), R10
- VMOVDQU (R10)(R9*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R8), R10
- VMOVDQU (R10)(R9*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R8), R10
- VMOVDQU (R10)(R9*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R8), R10
- VMOVDQU (R10)(R9*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R8), R10
- VMOVDQU Y0, (R10)(R9*1)
- MOVQ 24(R8), R10
- VMOVDQU Y1, (R10)(R9*1)
- MOVQ 48(R8), R10
- VMOVDQU Y2, (R10)(R9*1)
- MOVQ 72(R8), R10
- VMOVDQU Y3, (R10)(R9*1)
- MOVQ 96(R8), R10
- VMOVDQU Y4, (R10)(R9*1)
- MOVQ 120(R8), R10
- VMOVDQU Y5, (R10)(R9*1)
- MOVQ 144(R8), R10
- VMOVDQU Y6, (R10)(R9*1)
- MOVQ 168(R8), R10
- VMOVDQU Y7, (R10)(R9*1)
- MOVQ 192(R8), R10
- VMOVDQU Y8, (R10)(R9*1)
- MOVQ 216(R8), R10
- VMOVDQU Y9, (R10)(R9*1)
-
- // Prepare for next loop
- ADDQ $0x20, R9
- DECQ AX
- JNZ mulAvxTwo_4x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_4x10Xor_end:
- RET
-
-// func mulAvxTwo_5x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), DI
- MOVQ 96(CX), CX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
-
- // Add start offset to input
- ADDQ R9, DX
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, CX
- MOVQ $0x0000000f, R9
- MOVQ R9, X11
- VPBROADCASTB X11, Y11
-
-mulAvxTwo_5x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y0, Y12
- VPSHUFB Y13, Y1, Y13
- VPXOR Y12, Y13, Y10
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y2, Y12
- VPSHUFB Y13, Y3, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y4, Y12
- VPSHUFB Y13, Y5, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y6, Y12
- VPSHUFB Y13, Y7, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (CX), Y12
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y8, Y12
- VPSHUFB Y13, Y9, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Store 1 outputs
- VMOVDQU Y10, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x1_loop
- VZEROUPPER
-
-mulAvxTwo_5x1_end:
- RET
-
-// func mulAvxTwo_5x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_5x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_5x1_64_end:
- RET
-
-// func mulAvxTwo_5x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 14 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), DI
- MOVQ 96(CX), CX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R8
- MOVQ start+72(FP), R9
-
- // Add start offset to output
- ADDQ R9, R8
-
- // Add start offset to input
- ADDQ R9, DX
- ADDQ R9, BX
- ADDQ R9, SI
- ADDQ R9, DI
- ADDQ R9, CX
- MOVQ $0x0000000f, R9
- MOVQ R9, X11
- VPBROADCASTB X11, Y11
-
-mulAvxTwo_5x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VMOVDQU (R8), Y10
- VPSHUFB Y12, Y0, Y12
- VPSHUFB Y13, Y1, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y2, Y12
- VPSHUFB Y13, Y3, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y4, Y12
- VPSHUFB Y13, Y5, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y6, Y12
- VPSHUFB Y13, Y7, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (CX), Y12
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y11, Y12, Y12
- VPAND Y11, Y13, Y13
- VPSHUFB Y12, Y8, Y12
- VPSHUFB Y13, Y9, Y13
- XOR3WAY( $0x00, Y12, Y13, Y10)
-
- // Store 1 outputs
- VMOVDQU Y10, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x1Xor_end:
- RET
-
-// func mulAvxTwo_5x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 26 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R10
- MOVQ R10, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_5x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R9), Y0
- VMOVDQU 32(R9), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R9)
- VMOVDQU Y1, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x1_64Xor_end:
- RET
-
-// func mulAvxTwo_5x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 27 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R9
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
- ADDQ R11, R9
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_5x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x2_loop
- VZEROUPPER
-
-mulAvxTwo_5x2_end:
- RET
-
-// func mulAvxTwo_5x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 49 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R9
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
- ADDQ R11, R9
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_5x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y2, (R9)
- VMOVDQU Y3, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_5x2_64_end:
- RET
-
-// func mulAvxTwo_5x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 27 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R9
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
- ADDQ R11, R9
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_5x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R9), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x2Xor_end:
- RET
-
-// func mulAvxTwo_5x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 49 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R9
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
- ADDQ R11, R9
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_5x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R10), Y0
- VMOVDQU 32(R10), Y1
- VMOVDQU (R9), Y2
- VMOVDQU 32(R9), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y2, (R9)
- VMOVDQU Y3, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x2_64Xor_end:
- RET
-
-// func mulAvxTwo_5x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 38 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R9
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R9
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_5x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x3_loop
- VZEROUPPER
-
-mulAvxTwo_5x3_end:
- RET
-
-// func mulAvxTwo_5x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 70 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R9
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R9
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_5x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y2, (R11)
- VMOVDQU Y3, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y4, (R9)
- VMOVDQU Y5, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_5x3_64_end:
- RET
-
-// func mulAvxTwo_5x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 38 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R9
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R9
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_5x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R9), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x3Xor_end:
- RET
-
-// func mulAvxTwo_5x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 70 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R9
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R10
- ADDQ R12, R11
- ADDQ R12, R9
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_5x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R10), Y0
- VMOVDQU 32(R10), Y1
- VMOVDQU (R11), Y2
- VMOVDQU 32(R11), Y3
- VMOVDQU (R9), Y4
- VMOVDQU 32(R9), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y2, (R11)
- VMOVDQU Y3, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y4, (R9)
- VMOVDQU Y5, 32(R9)
- ADDQ $0x40, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x3_64Xor_end:
- RET
-
-// func mulAvxTwo_5x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 49 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R9
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R9
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_5x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x4_loop
- VZEROUPPER
-
-mulAvxTwo_5x4_end:
- RET
-
-// func mulAvxTwo_5x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 49 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R9
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R9
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_5x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R9), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x4Xor_end:
- RET
-
-// func mulAvxTwo_5x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 60 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R9
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R9
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_5x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x5_loop
- VZEROUPPER
-
-mulAvxTwo_5x5_end:
- RET
-
-// func mulAvxTwo_5x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 60 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R9
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R9
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_5x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R13), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (R9), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x5Xor_end:
- RET
-
-// func mulAvxTwo_5x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 71 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R9
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R9
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_5x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x6_loop
- VZEROUPPER
-
-mulAvxTwo_5x6_end:
- RET
-
-// func mulAvxTwo_5x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 71 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R9
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R9
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_5x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R13), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R14), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (R9), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x6Xor_end:
- RET
-
-// func mulAvxTwo_5x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x7(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 82 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R15
- MOVQ 144(R9), R9
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R9
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_5x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y6, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x7_loop
- VZEROUPPER
-
-mulAvxTwo_5x7_end:
- RET
-
-// func mulAvxTwo_5x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x7Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 82 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R15
- MOVQ 144(R9), R9
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R9
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_5x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R13), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R14), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R15), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (R9), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y6, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_5x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x7Xor_end:
- RET
-
-// func mulAvxTwo_5x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x8(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 93 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x8_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), AX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R15
- MOVQ 168(R8), R8
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R8
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_5x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (AX), Y11
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y7, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_5x8_loop
- VZEROUPPER
-
-mulAvxTwo_5x8_end:
- RET
-
-// func mulAvxTwo_5x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x8Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 93 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x8Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), AX
- MOVQ out_base+48(FP), R8
- MOVQ (R8), R9
- MOVQ 24(R8), R10
- MOVQ 48(R8), R11
- MOVQ 72(R8), R12
- MOVQ 96(R8), R13
- MOVQ 120(R8), R14
- MOVQ 144(R8), R15
- MOVQ 168(R8), R8
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R8
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_5x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (R9), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU (R13), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU (R14), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU (R15), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU (R8), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (AX), Y11
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- VMOVDQU Y0, (R9)
- ADDQ $0x20, R9
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y4, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y5, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y6, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y7, (R8)
- ADDQ $0x20, R8
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_5x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x8Xor_end:
- RET
-
-// func mulAvxTwo_5x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 104 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_5x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R9), R11
- VMOVDQU Y0, (R11)(R10*1)
- MOVQ 24(R9), R11
- VMOVDQU Y1, (R11)(R10*1)
- MOVQ 48(R9), R11
- VMOVDQU Y2, (R11)(R10*1)
- MOVQ 72(R9), R11
- VMOVDQU Y3, (R11)(R10*1)
- MOVQ 96(R9), R11
- VMOVDQU Y4, (R11)(R10*1)
- MOVQ 120(R9), R11
- VMOVDQU Y5, (R11)(R10*1)
- MOVQ 144(R9), R11
- VMOVDQU Y6, (R11)(R10*1)
- MOVQ 168(R9), R11
- VMOVDQU Y7, (R11)(R10*1)
- MOVQ 192(R9), R11
- VMOVDQU Y8, (R11)(R10*1)
-
- // Prepare for next loop
- ADDQ $0x20, R10
- DECQ AX
- JNZ mulAvxTwo_5x9_loop
- VZEROUPPER
-
-mulAvxTwo_5x9_end:
- RET
-
-// func mulAvxTwo_5x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 104 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_5x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R9), R11
- VMOVDQU (R11)(R10*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R9), R11
- VMOVDQU (R11)(R10*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R9), R11
- VMOVDQU (R11)(R10*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R9), R11
- VMOVDQU (R11)(R10*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R9), R11
- VMOVDQU (R11)(R10*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R9), R11
- VMOVDQU (R11)(R10*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R9), R11
- VMOVDQU (R11)(R10*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R9), R11
- VMOVDQU (R11)(R10*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R9), R11
- VMOVDQU (R11)(R10*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R9), R11
- VMOVDQU Y0, (R11)(R10*1)
- MOVQ 24(R9), R11
- VMOVDQU Y1, (R11)(R10*1)
- MOVQ 48(R9), R11
- VMOVDQU Y2, (R11)(R10*1)
- MOVQ 72(R9), R11
- VMOVDQU Y3, (R11)(R10*1)
- MOVQ 96(R9), R11
- VMOVDQU Y4, (R11)(R10*1)
- MOVQ 120(R9), R11
- VMOVDQU Y5, (R11)(R10*1)
- MOVQ 144(R9), R11
- VMOVDQU Y6, (R11)(R10*1)
- MOVQ 168(R9), R11
- VMOVDQU Y7, (R11)(R10*1)
- MOVQ 192(R9), R11
- VMOVDQU Y8, (R11)(R10*1)
-
- // Prepare for next loop
- ADDQ $0x20, R10
- DECQ AX
- JNZ mulAvxTwo_5x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x9Xor_end:
- RET
-
-// func mulAvxTwo_5x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 115 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_5x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R9), R11
- VMOVDQU Y0, (R11)(R10*1)
- MOVQ 24(R9), R11
- VMOVDQU Y1, (R11)(R10*1)
- MOVQ 48(R9), R11
- VMOVDQU Y2, (R11)(R10*1)
- MOVQ 72(R9), R11
- VMOVDQU Y3, (R11)(R10*1)
- MOVQ 96(R9), R11
- VMOVDQU Y4, (R11)(R10*1)
- MOVQ 120(R9), R11
- VMOVDQU Y5, (R11)(R10*1)
- MOVQ 144(R9), R11
- VMOVDQU Y6, (R11)(R10*1)
- MOVQ 168(R9), R11
- VMOVDQU Y7, (R11)(R10*1)
- MOVQ 192(R9), R11
- VMOVDQU Y8, (R11)(R10*1)
- MOVQ 216(R9), R11
- VMOVDQU Y9, (R11)(R10*1)
-
- // Prepare for next loop
- ADDQ $0x20, R10
- DECQ AX
- JNZ mulAvxTwo_5x10_loop
- VZEROUPPER
-
-mulAvxTwo_5x10_end:
- RET
-
-// func mulAvxTwo_5x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_5x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 115 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_5x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), DX
- MOVQ out_base+48(FP), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to input
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_5x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R9), R11
- VMOVDQU (R11)(R10*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R9), R11
- VMOVDQU (R11)(R10*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R9), R11
- VMOVDQU (R11)(R10*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R9), R11
- VMOVDQU (R11)(R10*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R9), R11
- VMOVDQU (R11)(R10*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R9), R11
- VMOVDQU (R11)(R10*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R9), R11
- VMOVDQU (R11)(R10*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R9), R11
- VMOVDQU (R11)(R10*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R9), R11
- VMOVDQU (R11)(R10*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R9), R11
- VMOVDQU (R11)(R10*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R9), R11
- VMOVDQU Y0, (R11)(R10*1)
- MOVQ 24(R9), R11
- VMOVDQU Y1, (R11)(R10*1)
- MOVQ 48(R9), R11
- VMOVDQU Y2, (R11)(R10*1)
- MOVQ 72(R9), R11
- VMOVDQU Y3, (R11)(R10*1)
- MOVQ 96(R9), R11
- VMOVDQU Y4, (R11)(R10*1)
- MOVQ 120(R9), R11
- VMOVDQU Y5, (R11)(R10*1)
- MOVQ 144(R9), R11
- VMOVDQU Y6, (R11)(R10*1)
- MOVQ 168(R9), R11
- VMOVDQU Y7, (R11)(R10*1)
- MOVQ 192(R9), R11
- VMOVDQU Y8, (R11)(R10*1)
- MOVQ 216(R9), R11
- VMOVDQU Y9, (R11)(R10*1)
-
- // Prepare for next loop
- ADDQ $0x20, R10
- DECQ AX
- JNZ mulAvxTwo_5x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_5x10Xor_end:
- RET
-
-// func mulAvxTwo_6x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x1(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 16 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x1_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), DI
- MOVQ 96(CX), R8
- MOVQ 120(CX), CX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
-
- // Add start offset to input
- ADDQ R10, DX
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, CX
- MOVQ $0x0000000f, R10
- MOVQ R10, X13
- VPBROADCASTB X13, Y13
-
-mulAvxTwo_6x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y14
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y0, Y14
- VPSHUFB Y15, Y1, Y15
- VPXOR Y14, Y15, Y12
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y14
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y2, Y14
- VPSHUFB Y15, Y3, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y14
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y4, Y14
- VPSHUFB Y15, Y5, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (DI), Y14
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y6, Y14
- VPSHUFB Y15, Y7, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R8), Y14
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y8, Y14
- VPSHUFB Y15, Y9, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (CX), Y14
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y10, Y14
- VPSHUFB Y15, Y11, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Store 1 outputs
- VMOVDQU Y12, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x1_loop
- VZEROUPPER
-
-mulAvxTwo_6x1_end:
- RET
-
-// func mulAvxTwo_6x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 30 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_6x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_6x1_64_end:
- RET
-
-// func mulAvxTwo_6x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x1Xor(SB), NOSPLIT, $0-88
- // Loading all tables to registers
- // Destination kept in GP registers
- // Full registers estimated 16 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x1Xor_end
- VMOVDQU (CX), Y0
- VMOVDQU 32(CX), Y1
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- MOVQ in_base+24(FP), CX
- MOVQ (CX), DX
- MOVQ 24(CX), BX
- MOVQ 48(CX), SI
- MOVQ 72(CX), DI
- MOVQ 96(CX), R8
- MOVQ 120(CX), CX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R9
- MOVQ start+72(FP), R10
-
- // Add start offset to output
- ADDQ R10, R9
-
- // Add start offset to input
- ADDQ R10, DX
- ADDQ R10, BX
- ADDQ R10, SI
- ADDQ R10, DI
- ADDQ R10, R8
- ADDQ R10, CX
- MOVQ $0x0000000f, R10
- MOVQ R10, X13
- VPBROADCASTB X13, Y13
-
-mulAvxTwo_6x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (DX), Y14
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VMOVDQU (R9), Y12
- VPSHUFB Y14, Y0, Y14
- VPSHUFB Y15, Y1, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (BX), Y14
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y2, Y14
- VPSHUFB Y15, Y3, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (SI), Y14
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y4, Y14
- VPSHUFB Y15, Y5, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (DI), Y14
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y6, Y14
- VPSHUFB Y15, Y7, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R8), Y14
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y8, Y14
- VPSHUFB Y15, Y9, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (CX), Y14
- ADDQ $0x20, CX
- VPSRLQ $0x04, Y14, Y15
- VPAND Y13, Y14, Y14
- VPAND Y13, Y15, Y15
- VPSHUFB Y14, Y10, Y14
- VPSHUFB Y15, Y11, Y15
- XOR3WAY( $0x00, Y14, Y15, Y12)
-
- // Store 1 outputs
- VMOVDQU Y12, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x1Xor_end:
- RET
-
-// func mulAvxTwo_6x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 30 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to output
- ADDQ R11, R10
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R11
- MOVQ R11, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_6x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R10), Y0
- VMOVDQU 32(R10), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R10)
- VMOVDQU Y1, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x1_64Xor_end:
- RET
-
-// func mulAvxTwo_6x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 31 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R10
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
- ADDQ R12, R10
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_6x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x2_loop
- VZEROUPPER
-
-mulAvxTwo_6x2_end:
- RET
-
-// func mulAvxTwo_6x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 57 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R10
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
- ADDQ R12, R10
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_6x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y2, (R10)
- VMOVDQU Y3, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_6x2_64_end:
- RET
-
-// func mulAvxTwo_6x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 31 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R10
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
- ADDQ R12, R10
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_6x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R10), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x2Xor_end:
- RET
-
-// func mulAvxTwo_6x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 57 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R10
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
- ADDQ R12, R10
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_6x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R11), Y0
- VMOVDQU 32(R11), Y1
- VMOVDQU (R10), Y2
- VMOVDQU 32(R10), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y2, (R10)
- VMOVDQU Y3, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x2_64Xor_end:
- RET
-
-// func mulAvxTwo_6x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 44 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R10
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R10
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_6x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x3_loop
- VZEROUPPER
-
-mulAvxTwo_6x3_end:
- RET
-
-// func mulAvxTwo_6x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 82 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R10
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R10
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_6x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y2, (R12)
- VMOVDQU Y3, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y4, (R10)
- VMOVDQU Y5, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_6x3_64_end:
- RET
-
-// func mulAvxTwo_6x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 44 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R10
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R10
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_6x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R10), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x3Xor_end:
- RET
-
-// func mulAvxTwo_6x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 82 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R10
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R11
- ADDQ R13, R12
- ADDQ R13, R10
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_6x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R11), Y0
- VMOVDQU 32(R11), Y1
- VMOVDQU (R12), Y2
- VMOVDQU 32(R12), Y3
- VMOVDQU (R10), Y4
- VMOVDQU 32(R10), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y2, (R12)
- VMOVDQU Y3, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y4, (R10)
- VMOVDQU Y5, 32(R10)
- ADDQ $0x40, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x3_64Xor_end:
- RET
-
-// func mulAvxTwo_6x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 57 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R10
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R10
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_6x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x4_loop
- VZEROUPPER
-
-mulAvxTwo_6x4_end:
- RET
-
-// func mulAvxTwo_6x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 57 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R10
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R10
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_6x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R10), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x4Xor_end:
- RET
-
-// func mulAvxTwo_6x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 70 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R10
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R10
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_6x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x5_loop
- VZEROUPPER
-
-mulAvxTwo_6x5_end:
- RET
-
-// func mulAvxTwo_6x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 70 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R10
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R10
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_6x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R14), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (R10), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x5Xor_end:
- RET
-
-// func mulAvxTwo_6x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x6(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 83 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R15
- MOVQ 120(R10), R10
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R10
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_6x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y5, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x6_loop
- VZEROUPPER
-
-mulAvxTwo_6x6_end:
- RET
-
-// func mulAvxTwo_6x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x6Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 83 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R15
- MOVQ 120(R10), R10
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R10
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_6x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R14), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R15), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (R10), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y5, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_6x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x6Xor_end:
- RET
-
-// func mulAvxTwo_6x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x7(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 96 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x7_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), AX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R15
- MOVQ 144(R9), R9
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R9
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_6x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (AX), Y10
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y6, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_6x7_loop
- VZEROUPPER
-
-mulAvxTwo_6x7_end:
- RET
-
-// func mulAvxTwo_6x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x7Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 96 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x7Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), AX
- MOVQ out_base+48(FP), R9
- MOVQ (R9), R10
- MOVQ 24(R9), R11
- MOVQ 48(R9), R12
- MOVQ 72(R9), R13
- MOVQ 96(R9), R14
- MOVQ 120(R9), R15
- MOVQ 144(R9), R9
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R9
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_6x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (R10), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU (R13), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU (R14), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU (R15), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU (R9), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (AX), Y10
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- VMOVDQU Y0, (R10)
- ADDQ $0x20, R10
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y3, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y4, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y5, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y6, (R9)
- ADDQ $0x20, R9
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_6x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x7Xor_end:
- RET
-
-// func mulAvxTwo_6x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 109 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_6x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x8_loop
- VZEROUPPER
-
-mulAvxTwo_6x8_end:
- RET
-
-// func mulAvxTwo_6x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 109 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_6x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- MOVQ (R10), R12
- VMOVDQU (R12)(R11*1), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- MOVQ 24(R10), R12
- VMOVDQU (R12)(R11*1), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- MOVQ 48(R10), R12
- VMOVDQU (R12)(R11*1), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- MOVQ 72(R10), R12
- VMOVDQU (R12)(R11*1), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- MOVQ 96(R10), R12
- VMOVDQU (R12)(R11*1), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- MOVQ 120(R10), R12
- VMOVDQU (R12)(R11*1), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- MOVQ 144(R10), R12
- VMOVDQU (R12)(R11*1), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- MOVQ 168(R10), R12
- VMOVDQU (R12)(R11*1), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x8Xor_end:
- RET
-
-// func mulAvxTwo_6x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 122 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_6x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
- MOVQ 192(R10), R12
- VMOVDQU Y8, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x9_loop
- VZEROUPPER
-
-mulAvxTwo_6x9_end:
- RET
-
-// func mulAvxTwo_6x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 122 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_6x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R10), R12
- VMOVDQU (R12)(R11*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R10), R12
- VMOVDQU (R12)(R11*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R10), R12
- VMOVDQU (R12)(R11*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R10), R12
- VMOVDQU (R12)(R11*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R10), R12
- VMOVDQU (R12)(R11*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R10), R12
- VMOVDQU (R12)(R11*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R10), R12
- VMOVDQU (R12)(R11*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R10), R12
- VMOVDQU (R12)(R11*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R10), R12
- VMOVDQU (R12)(R11*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
- MOVQ 192(R10), R12
- VMOVDQU Y8, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x9Xor_end:
- RET
-
-// func mulAvxTwo_6x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 135 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_6x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
- MOVQ 192(R10), R12
- VMOVDQU Y8, (R12)(R11*1)
- MOVQ 216(R10), R12
- VMOVDQU Y9, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x10_loop
- VZEROUPPER
-
-mulAvxTwo_6x10_end:
- RET
-
-// func mulAvxTwo_6x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_6x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 135 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_6x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), DX
- MOVQ out_base+48(FP), R10
- MOVQ start+72(FP), R11
-
- // Add start offset to input
- ADDQ R11, BX
- ADDQ R11, SI
- ADDQ R11, DI
- ADDQ R11, R8
- ADDQ R11, R9
- ADDQ R11, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_6x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R10), R12
- VMOVDQU (R12)(R11*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R10), R12
- VMOVDQU (R12)(R11*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R10), R12
- VMOVDQU (R12)(R11*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R10), R12
- VMOVDQU (R12)(R11*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R10), R12
- VMOVDQU (R12)(R11*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R10), R12
- VMOVDQU (R12)(R11*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R10), R12
- VMOVDQU (R12)(R11*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R10), R12
- VMOVDQU (R12)(R11*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R10), R12
- VMOVDQU (R12)(R11*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R10), R12
- VMOVDQU (R12)(R11*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R10), R12
- VMOVDQU Y0, (R12)(R11*1)
- MOVQ 24(R10), R12
- VMOVDQU Y1, (R12)(R11*1)
- MOVQ 48(R10), R12
- VMOVDQU Y2, (R12)(R11*1)
- MOVQ 72(R10), R12
- VMOVDQU Y3, (R12)(R11*1)
- MOVQ 96(R10), R12
- VMOVDQU Y4, (R12)(R11*1)
- MOVQ 120(R10), R12
- VMOVDQU Y5, (R12)(R11*1)
- MOVQ 144(R10), R12
- VMOVDQU Y6, (R12)(R11*1)
- MOVQ 168(R10), R12
- VMOVDQU Y7, (R12)(R11*1)
- MOVQ 192(R10), R12
- VMOVDQU Y8, (R12)(R11*1)
- MOVQ 216(R10), R12
- VMOVDQU Y9, (R12)(R11*1)
-
- // Prepare for next loop
- ADDQ $0x20, R11
- DECQ AX
- JNZ mulAvxTwo_6x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_6x10Xor_end:
- RET
-
-// func mulAvxTwo_7x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x1(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 18 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x1_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_7x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- VPXOR Y2, Y3, Y0
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x1_loop
- VZEROUPPER
-
-mulAvxTwo_7x1_end:
- RET
-
-// func mulAvxTwo_7x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 34 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_7x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_7x1_64_end:
- RET
-
-// func mulAvxTwo_7x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x1Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 18 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x1Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_7x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x1Xor_end:
- RET
-
-// func mulAvxTwo_7x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 34 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to output
- ADDQ R12, R11
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R12
- MOVQ R12, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_7x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R11), Y0
- VMOVDQU 32(R11), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R11)
- VMOVDQU Y1, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x1_64Xor_end:
- RET
-
-// func mulAvxTwo_7x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R11
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
- ADDQ R13, R11
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_7x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x2_loop
- VZEROUPPER
-
-mulAvxTwo_7x2_end:
- RET
-
-// func mulAvxTwo_7x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 65 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R11
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
- ADDQ R13, R11
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_7x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y2, (R11)
- VMOVDQU Y3, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_7x2_64_end:
- RET
-
-// func mulAvxTwo_7x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 35 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R11
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
- ADDQ R13, R11
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_7x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R11), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x2Xor_end:
- RET
-
-// func mulAvxTwo_7x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 65 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R11
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
- ADDQ R13, R11
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_7x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R12), Y0
- VMOVDQU 32(R12), Y1
- VMOVDQU (R11), Y2
- VMOVDQU 32(R11), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y2, (R11)
- VMOVDQU Y3, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x2_64Xor_end:
- RET
-
-// func mulAvxTwo_7x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R11
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R11
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_7x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x3_loop
- VZEROUPPER
-
-mulAvxTwo_7x3_end:
- RET
-
-// func mulAvxTwo_7x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 94 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R11
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R11
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_7x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y2, (R13)
- VMOVDQU Y3, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y4, (R11)
- VMOVDQU Y5, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_7x3_64_end:
- RET
-
-// func mulAvxTwo_7x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 50 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R11
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R11
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_7x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R13), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R11), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x3Xor_end:
- RET
-
-// func mulAvxTwo_7x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 94 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R11
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R12
- ADDQ R14, R13
- ADDQ R14, R11
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_7x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R12), Y0
- VMOVDQU 32(R12), Y1
- VMOVDQU (R13), Y2
- VMOVDQU 32(R13), Y3
- VMOVDQU (R11), Y4
- VMOVDQU 32(R11), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
- VMOVDQU Y2, (R13)
- VMOVDQU Y3, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y4, (R11)
- VMOVDQU Y5, 32(R11)
- ADDQ $0x40, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x3_64Xor_end:
- RET
-
-// func mulAvxTwo_7x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x4(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 65 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R11
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R11
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_7x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x4_loop
- VZEROUPPER
-
-mulAvxTwo_7x4_end:
- RET
-
-// func mulAvxTwo_7x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x4Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 65 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R11
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R11
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_7x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R13), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R14), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R11), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x4Xor_end:
- RET
-
-// func mulAvxTwo_7x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x5(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 80 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R15
- MOVQ 96(R11), R11
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R11
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_7x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x5_loop
- VZEROUPPER
-
-mulAvxTwo_7x5_end:
- RET
-
-// func mulAvxTwo_7x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x5Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 80 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R15
- MOVQ 96(R11), R11
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R11
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_7x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R13), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R14), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R15), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_7x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x5Xor_end:
- RET
-
-// func mulAvxTwo_7x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x6(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 95 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x6_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), AX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R15
- MOVQ 120(R10), R10
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R10
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_7x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (AX), Y9
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y5, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_7x6_loop
- VZEROUPPER
-
-mulAvxTwo_7x6_end:
- RET
-
-// func mulAvxTwo_7x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x6Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 95 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x6Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), AX
- MOVQ out_base+48(FP), R10
- MOVQ (R10), R11
- MOVQ 24(R10), R12
- MOVQ 48(R10), R13
- MOVQ 72(R10), R14
- MOVQ 96(R10), R15
- MOVQ 120(R10), R10
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R10
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_7x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (R11), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU (R14), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU (R15), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU (R10), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (AX), Y9
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- VMOVDQU Y0, (R11)
- ADDQ $0x20, R11
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y3, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y4, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y5, (R10)
- ADDQ $0x20, R10
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_7x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x6Xor_end:
- RET
-
-// func mulAvxTwo_7x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 110 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_7x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x7_loop
- VZEROUPPER
-
-mulAvxTwo_7x7_end:
- RET
-
-// func mulAvxTwo_7x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 110 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_7x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- MOVQ (R11), R13
- VMOVDQU (R13)(R12*1), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- MOVQ 24(R11), R13
- VMOVDQU (R13)(R12*1), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- MOVQ 48(R11), R13
- VMOVDQU (R13)(R12*1), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- MOVQ 72(R11), R13
- VMOVDQU (R13)(R12*1), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- MOVQ 96(R11), R13
- VMOVDQU (R13)(R12*1), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- MOVQ 120(R11), R13
- VMOVDQU (R13)(R12*1), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- MOVQ 144(R11), R13
- VMOVDQU (R13)(R12*1), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x7Xor_end:
- RET
-
-// func mulAvxTwo_7x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 125 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_7x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x8_loop
- VZEROUPPER
-
-mulAvxTwo_7x8_end:
- RET
-
-// func mulAvxTwo_7x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 125 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_7x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- MOVQ (R11), R13
- VMOVDQU (R13)(R12*1), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- MOVQ 24(R11), R13
- VMOVDQU (R13)(R12*1), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- MOVQ 48(R11), R13
- VMOVDQU (R13)(R12*1), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- MOVQ 72(R11), R13
- VMOVDQU (R13)(R12*1), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- MOVQ 96(R11), R13
- VMOVDQU (R13)(R12*1), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- MOVQ 120(R11), R13
- VMOVDQU (R13)(R12*1), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- MOVQ 144(R11), R13
- VMOVDQU (R13)(R12*1), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- MOVQ 168(R11), R13
- VMOVDQU (R13)(R12*1), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x8Xor_end:
- RET
-
-// func mulAvxTwo_7x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 140 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_7x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
- MOVQ 192(R11), R13
- VMOVDQU Y8, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x9_loop
- VZEROUPPER
-
-mulAvxTwo_7x9_end:
- RET
-
-// func mulAvxTwo_7x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 140 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_7x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R11), R13
- VMOVDQU (R13)(R12*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R11), R13
- VMOVDQU (R13)(R12*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R11), R13
- VMOVDQU (R13)(R12*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R11), R13
- VMOVDQU (R13)(R12*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R11), R13
- VMOVDQU (R13)(R12*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R11), R13
- VMOVDQU (R13)(R12*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R11), R13
- VMOVDQU (R13)(R12*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R11), R13
- VMOVDQU (R13)(R12*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R11), R13
- VMOVDQU (R13)(R12*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
- MOVQ 192(R11), R13
- VMOVDQU Y8, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x9Xor_end:
- RET
-
-// func mulAvxTwo_7x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 155 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_7x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
- MOVQ 192(R11), R13
- VMOVDQU Y8, (R13)(R12*1)
- MOVQ 216(R11), R13
- VMOVDQU Y9, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x10_loop
- VZEROUPPER
-
-mulAvxTwo_7x10_end:
- RET
-
-// func mulAvxTwo_7x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_7x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 155 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_7x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), DX
- MOVQ out_base+48(FP), R11
- MOVQ start+72(FP), R12
-
- // Add start offset to input
- ADDQ R12, BX
- ADDQ R12, SI
- ADDQ R12, DI
- ADDQ R12, R8
- ADDQ R12, R9
- ADDQ R12, R10
- ADDQ R12, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_7x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R11), R13
- VMOVDQU (R13)(R12*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R11), R13
- VMOVDQU (R13)(R12*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R11), R13
- VMOVDQU (R13)(R12*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R11), R13
- VMOVDQU (R13)(R12*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R11), R13
- VMOVDQU (R13)(R12*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R11), R13
- VMOVDQU (R13)(R12*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R11), R13
- VMOVDQU (R13)(R12*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R11), R13
- VMOVDQU (R13)(R12*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R11), R13
- VMOVDQU (R13)(R12*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R11), R13
- VMOVDQU (R13)(R12*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R11), R13
- VMOVDQU Y0, (R13)(R12*1)
- MOVQ 24(R11), R13
- VMOVDQU Y1, (R13)(R12*1)
- MOVQ 48(R11), R13
- VMOVDQU Y2, (R13)(R12*1)
- MOVQ 72(R11), R13
- VMOVDQU Y3, (R13)(R12*1)
- MOVQ 96(R11), R13
- VMOVDQU Y4, (R13)(R12*1)
- MOVQ 120(R11), R13
- VMOVDQU Y5, (R13)(R12*1)
- MOVQ 144(R11), R13
- VMOVDQU Y6, (R13)(R12*1)
- MOVQ 168(R11), R13
- VMOVDQU Y7, (R13)(R12*1)
- MOVQ 192(R11), R13
- VMOVDQU Y8, (R13)(R12*1)
- MOVQ 216(R11), R13
- VMOVDQU Y9, (R13)(R12*1)
-
- // Prepare for next loop
- ADDQ $0x20, R12
- DECQ AX
- JNZ mulAvxTwo_7x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_7x10Xor_end:
- RET
-
-// func mulAvxTwo_8x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x1(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x1_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_8x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- VPXOR Y2, Y3, Y0
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x1_loop
- VZEROUPPER
-
-mulAvxTwo_8x1_end:
- RET
-
-// func mulAvxTwo_8x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 38 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_8x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_8x1_64_end:
- RET
-
-// func mulAvxTwo_8x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x1Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 20 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x1Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_8x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x1Xor_end:
- RET
-
-// func mulAvxTwo_8x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 38 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to output
- ADDQ R13, R12
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R13
- MOVQ R13, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_8x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R12), Y0
- VMOVDQU 32(R12), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R12)
- VMOVDQU Y1, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x1_64Xor_end:
- RET
-
-// func mulAvxTwo_8x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 39 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R12
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
- ADDQ R14, R12
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_8x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x2_loop
- VZEROUPPER
-
-mulAvxTwo_8x2_end:
- RET
-
-// func mulAvxTwo_8x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 73 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R12
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
- ADDQ R14, R12
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_8x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y2, (R12)
- VMOVDQU Y3, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_8x2_64_end:
- RET
-
-// func mulAvxTwo_8x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 39 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R12
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
- ADDQ R14, R12
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_8x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R13), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R12), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x2Xor_end:
- RET
-
-// func mulAvxTwo_8x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 73 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R12
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
- ADDQ R14, R12
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_8x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R13), Y0
- VMOVDQU 32(R13), Y1
- VMOVDQU (R12), Y2
- VMOVDQU 32(R12), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y2, (R12)
- VMOVDQU Y3, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x2_64Xor_end:
- RET
-
-// func mulAvxTwo_8x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x3(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 56 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R12
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R12
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_8x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x3_loop
- VZEROUPPER
-
-mulAvxTwo_8x3_end:
- RET
-
-// func mulAvxTwo_8x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x3_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 106 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R12
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R12
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_8x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y2, (R14)
- VMOVDQU Y3, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y4, (R12)
- VMOVDQU Y5, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_8x3_64_end:
- RET
-
-// func mulAvxTwo_8x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x3Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 56 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R12
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R12
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_8x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R13), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R14), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R12), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x3Xor_end:
- RET
-
-// func mulAvxTwo_8x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x3_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 106 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R12
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R13
- ADDQ R15, R14
- ADDQ R15, R12
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_8x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R13), Y0
- VMOVDQU 32(R13), Y1
- VMOVDQU (R14), Y2
- VMOVDQU 32(R14), Y3
- VMOVDQU (R12), Y4
- VMOVDQU 32(R12), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
- VMOVDQU Y2, (R14)
- VMOVDQU Y3, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y4, (R12)
- VMOVDQU Y5, 32(R12)
- ADDQ $0x40, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x3_64Xor_end:
- RET
-
-// func mulAvxTwo_8x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x4(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 73 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R15
- MOVQ 72(R12), R12
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R12
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_8x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x4_loop
- VZEROUPPER
-
-mulAvxTwo_8x4_end:
- RET
-
-// func mulAvxTwo_8x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x4Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 73 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R15
- MOVQ 72(R12), R12
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R12
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_8x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R13), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R14), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R15), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_8x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x4Xor_end:
- RET
-
-// func mulAvxTwo_8x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x5(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 90 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x5_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), AX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R15
- MOVQ 96(R11), R11
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R11
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_8x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (AX), Y8
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_8x5_loop
- VZEROUPPER
-
-mulAvxTwo_8x5_end:
- RET
-
-// func mulAvxTwo_8x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x5Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 90 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x5Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), AX
- MOVQ out_base+48(FP), R11
- MOVQ (R11), R12
- MOVQ 24(R11), R13
- MOVQ 48(R11), R14
- MOVQ 72(R11), R15
- MOVQ 96(R11), R11
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R11
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_8x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (R12), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU (R13), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU (R14), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU (R15), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU (R11), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (AX), Y8
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- VMOVDQU Y0, (R12)
- ADDQ $0x20, R12
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y2, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y3, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y4, (R11)
- ADDQ $0x20, R11
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_8x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x5Xor_end:
- RET
-
-// func mulAvxTwo_8x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 107 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_8x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x6_loop
- VZEROUPPER
-
-mulAvxTwo_8x6_end:
- RET
-
-// func mulAvxTwo_8x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 107 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_8x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- MOVQ (R12), R14
- VMOVDQU (R14)(R13*1), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- MOVQ 24(R12), R14
- VMOVDQU (R14)(R13*1), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- MOVQ 48(R12), R14
- VMOVDQU (R14)(R13*1), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- MOVQ 72(R12), R14
- VMOVDQU (R14)(R13*1), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- MOVQ 96(R12), R14
- VMOVDQU (R14)(R13*1), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- MOVQ 120(R12), R14
- VMOVDQU (R14)(R13*1), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x6Xor_end:
- RET
-
-// func mulAvxTwo_8x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 124 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_8x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x7_loop
- VZEROUPPER
-
-mulAvxTwo_8x7_end:
- RET
-
-// func mulAvxTwo_8x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 124 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_8x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- MOVQ (R12), R14
- VMOVDQU (R14)(R13*1), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- MOVQ 24(R12), R14
- VMOVDQU (R14)(R13*1), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- MOVQ 48(R12), R14
- VMOVDQU (R14)(R13*1), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- MOVQ 72(R12), R14
- VMOVDQU (R14)(R13*1), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- MOVQ 96(R12), R14
- VMOVDQU (R14)(R13*1), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- MOVQ 120(R12), R14
- VMOVDQU (R14)(R13*1), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- MOVQ 144(R12), R14
- VMOVDQU (R14)(R13*1), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x7Xor_end:
- RET
-
-// func mulAvxTwo_8x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 141 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_8x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x8_loop
- VZEROUPPER
-
-mulAvxTwo_8x8_end:
- RET
-
-// func mulAvxTwo_8x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 141 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_8x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- MOVQ (R12), R14
- VMOVDQU (R14)(R13*1), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- MOVQ 24(R12), R14
- VMOVDQU (R14)(R13*1), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- MOVQ 48(R12), R14
- VMOVDQU (R14)(R13*1), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- MOVQ 72(R12), R14
- VMOVDQU (R14)(R13*1), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- MOVQ 96(R12), R14
- VMOVDQU (R14)(R13*1), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- MOVQ 120(R12), R14
- VMOVDQU (R14)(R13*1), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- MOVQ 144(R12), R14
- VMOVDQU (R14)(R13*1), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- MOVQ 168(R12), R14
- VMOVDQU (R14)(R13*1), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x8Xor_end:
- RET
-
-// func mulAvxTwo_8x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 158 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_8x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
- MOVQ 192(R12), R14
- VMOVDQU Y8, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x9_loop
- VZEROUPPER
-
-mulAvxTwo_8x9_end:
- RET
-
-// func mulAvxTwo_8x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 158 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_8x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R12), R14
- VMOVDQU (R14)(R13*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R12), R14
- VMOVDQU (R14)(R13*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R12), R14
- VMOVDQU (R14)(R13*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R12), R14
- VMOVDQU (R14)(R13*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R12), R14
- VMOVDQU (R14)(R13*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R12), R14
- VMOVDQU (R14)(R13*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R12), R14
- VMOVDQU (R14)(R13*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R12), R14
- VMOVDQU (R14)(R13*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R12), R14
- VMOVDQU (R14)(R13*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
- MOVQ 192(R12), R14
- VMOVDQU Y8, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x9Xor_end:
- RET
-
-// func mulAvxTwo_8x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 175 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_8x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
- MOVQ 192(R12), R14
- VMOVDQU Y8, (R14)(R13*1)
- MOVQ 216(R12), R14
- VMOVDQU Y9, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x10_loop
- VZEROUPPER
-
-mulAvxTwo_8x10_end:
- RET
-
-// func mulAvxTwo_8x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_8x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 175 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_8x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), DX
- MOVQ out_base+48(FP), R12
- MOVQ start+72(FP), R13
-
- // Add start offset to input
- ADDQ R13, BX
- ADDQ R13, SI
- ADDQ R13, DI
- ADDQ R13, R8
- ADDQ R13, R9
- ADDQ R13, R10
- ADDQ R13, R11
- ADDQ R13, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_8x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R12), R14
- VMOVDQU (R14)(R13*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R12), R14
- VMOVDQU (R14)(R13*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R12), R14
- VMOVDQU (R14)(R13*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R12), R14
- VMOVDQU (R14)(R13*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R12), R14
- VMOVDQU (R14)(R13*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R12), R14
- VMOVDQU (R14)(R13*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R12), R14
- VMOVDQU (R14)(R13*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R12), R14
- VMOVDQU (R14)(R13*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R12), R14
- VMOVDQU (R14)(R13*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R12), R14
- VMOVDQU (R14)(R13*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R12), R14
- VMOVDQU Y0, (R14)(R13*1)
- MOVQ 24(R12), R14
- VMOVDQU Y1, (R14)(R13*1)
- MOVQ 48(R12), R14
- VMOVDQU Y2, (R14)(R13*1)
- MOVQ 72(R12), R14
- VMOVDQU Y3, (R14)(R13*1)
- MOVQ 96(R12), R14
- VMOVDQU Y4, (R14)(R13*1)
- MOVQ 120(R12), R14
- VMOVDQU Y5, (R14)(R13*1)
- MOVQ 144(R12), R14
- VMOVDQU Y6, (R14)(R13*1)
- MOVQ 168(R12), R14
- VMOVDQU Y7, (R14)(R13*1)
- MOVQ 192(R12), R14
- VMOVDQU Y8, (R14)(R13*1)
- MOVQ 216(R12), R14
- VMOVDQU Y9, (R14)(R13*1)
-
- // Prepare for next loop
- ADDQ $0x20, R13
- DECQ AX
- JNZ mulAvxTwo_8x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_8x10Xor_end:
- RET
-
-// func mulAvxTwo_9x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x1(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x1_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_9x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- VPXOR Y2, Y3, Y0
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y4
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 8 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 512(CX), Y2
- VMOVDQU 544(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x1_loop
- VZEROUPPER
-
-mulAvxTwo_9x1_end:
- RET
-
-// func mulAvxTwo_9x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 42 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_9x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y6
- VMOVDQU 32(R12), Y5
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 8 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_9x1_64_end:
- RET
-
-// func mulAvxTwo_9x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x1Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 22 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x1Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_9x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (R13), Y0
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y4
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 8 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 512(CX), Y2
- VMOVDQU 544(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x1Xor_end:
- RET
-
-// func mulAvxTwo_9x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 42 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to output
- ADDQ R14, R13
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R14
- MOVQ R14, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_9x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R13), Y0
- VMOVDQU 32(R13), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y6
- VMOVDQU 32(R12), Y5
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 8 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R13)
- VMOVDQU Y1, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x1_64Xor_end:
- RET
-
-// func mulAvxTwo_9x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x2(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 43 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R13
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
- ADDQ R15, R13
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_9x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y5
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 8 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1024(CX), Y3
- VMOVDQU 1056(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1088(CX), Y3
- VMOVDQU 1120(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x2_loop
- VZEROUPPER
-
-mulAvxTwo_9x2_end:
- RET
-
-// func mulAvxTwo_9x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x2_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 81 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R13
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
- ADDQ R15, R13
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_9x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y9
- VMOVDQU 32(R12), Y11
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 8 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R13)
- VMOVDQU Y3, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_9x2_64_end:
- RET
-
-// func mulAvxTwo_9x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x2Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 43 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R13
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
- ADDQ R15, R13
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_9x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R14), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R13), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y5
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 8 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1024(CX), Y3
- VMOVDQU 1056(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1088(CX), Y3
- VMOVDQU 1120(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x2Xor_end:
- RET
-
-// func mulAvxTwo_9x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x2_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 81 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R13
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
- ADDQ R15, R13
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_9x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R14), Y0
- VMOVDQU 32(R14), Y1
- VMOVDQU (R13), Y2
- VMOVDQU 32(R13), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y9
- VMOVDQU 32(R12), Y11
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 8 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R13)
- VMOVDQU Y3, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x2_64Xor_end:
- RET
-
-// func mulAvxTwo_9x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x3(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 62 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x3_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_9x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (R12), Y6
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 8 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1536(CX), Y4
- VMOVDQU 1568(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1600(CX), Y4
- VMOVDQU 1632(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1664(CX), Y4
- VMOVDQU 1696(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x3_loop
- VZEROUPPER
-
-mulAvxTwo_9x3_end:
- RET
-
-// func mulAvxTwo_9x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x3_64(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 118 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x3_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_9x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (R12), Y11
- VMOVDQU 32(R12), Y13
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 8 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R15)
- VMOVDQU Y3, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y4, (R13)
- VMOVDQU Y5, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_9x3_64_end:
- RET
-
-// func mulAvxTwo_9x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x3Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 62 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x3Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X3
- VPBROADCASTB X3, Y3
-
-mulAvxTwo_9x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R14), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R15), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (R12), Y6
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 8 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1536(CX), Y4
- VMOVDQU 1568(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1600(CX), Y4
- VMOVDQU 1632(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1664(CX), Y4
- VMOVDQU 1696(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x3Xor_end:
- RET
-
-// func mulAvxTwo_9x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x3_64Xor(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 118 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x3_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_9x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R14), Y0
- VMOVDQU 32(R14), Y1
- VMOVDQU (R15), Y2
- VMOVDQU 32(R15), Y3
- VMOVDQU (R13), Y4
- VMOVDQU 32(R13), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (R12), Y11
- VMOVDQU 32(R12), Y13
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 8 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R15)
- VMOVDQU Y3, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y4, (R13)
- VMOVDQU Y5, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_9x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x3_64Xor_end:
- RET
-
-// func mulAvxTwo_9x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x4(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 81 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x4_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), AX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R15
- MOVQ 72(R12), R12
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R12
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_9x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 8 to 4 outputs
- VMOVDQU (AX), Y7
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2048(CX), Y5
- VMOVDQU 2080(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2112(CX), Y5
- VMOVDQU 2144(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2176(CX), Y5
- VMOVDQU 2208(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2240(CX), Y5
- VMOVDQU 2272(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_9x4_loop
- VZEROUPPER
-
-mulAvxTwo_9x4_end:
- RET
-
-// func mulAvxTwo_9x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x4Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 81 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x4Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), AX
- MOVQ out_base+48(FP), R12
- MOVQ (R12), R13
- MOVQ 24(R12), R14
- MOVQ 48(R12), R15
- MOVQ 72(R12), R12
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R13
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R12
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_9x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (R13), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU (R14), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU (R15), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R12), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 8 to 4 outputs
- VMOVDQU (AX), Y7
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2048(CX), Y5
- VMOVDQU 2080(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2112(CX), Y5
- VMOVDQU 2144(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2176(CX), Y5
- VMOVDQU 2208(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2240(CX), Y5
- VMOVDQU 2272(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- VMOVDQU Y0, (R13)
- ADDQ $0x20, R13
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y2, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y3, (R12)
- ADDQ $0x20, R12
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_9x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x4Xor_end:
- RET
-
-// func mulAvxTwo_9x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x5(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 100 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_9x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R11), Y8
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (R12), Y8
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 8 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2560(CX), Y6
- VMOVDQU 2592(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2624(CX), Y6
- VMOVDQU 2656(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2688(CX), Y6
- VMOVDQU 2720(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2752(CX), Y6
- VMOVDQU 2784(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2816(CX), Y6
- VMOVDQU 2848(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x5_loop
- VZEROUPPER
-
-mulAvxTwo_9x5_end:
- RET
-
-// func mulAvxTwo_9x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x5Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 100 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_9x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R11), Y8
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (R12), Y8
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 8 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2560(CX), Y6
- VMOVDQU 2592(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2624(CX), Y6
- VMOVDQU 2656(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2688(CX), Y6
- VMOVDQU 2720(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2752(CX), Y6
- VMOVDQU 2784(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2816(CX), Y6
- VMOVDQU 2848(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x5Xor_end:
- RET
-
-// func mulAvxTwo_9x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x6(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 119 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_9x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (R12), Y9
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 8 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3072(CX), Y7
- VMOVDQU 3104(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3136(CX), Y7
- VMOVDQU 3168(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3200(CX), Y7
- VMOVDQU 3232(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3264(CX), Y7
- VMOVDQU 3296(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3328(CX), Y7
- VMOVDQU 3360(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3392(CX), Y7
- VMOVDQU 3424(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x6_loop
- VZEROUPPER
-
-mulAvxTwo_9x6_end:
- RET
-
-// func mulAvxTwo_9x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x6Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 119 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_9x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- MOVQ 120(R13), R15
- VMOVDQU (R15)(R14*1), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (R12), Y9
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 8 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3072(CX), Y7
- VMOVDQU 3104(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3136(CX), Y7
- VMOVDQU 3168(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3200(CX), Y7
- VMOVDQU 3232(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3264(CX), Y7
- VMOVDQU 3296(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3328(CX), Y7
- VMOVDQU 3360(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3392(CX), Y7
- VMOVDQU 3424(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x6Xor_end:
- RET
-
-// func mulAvxTwo_9x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x7(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 138 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_9x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (R12), Y10
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 8 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3584(CX), Y8
- VMOVDQU 3616(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3648(CX), Y8
- VMOVDQU 3680(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3712(CX), Y8
- VMOVDQU 3744(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3776(CX), Y8
- VMOVDQU 3808(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3840(CX), Y8
- VMOVDQU 3872(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3904(CX), Y8
- VMOVDQU 3936(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3968(CX), Y8
- VMOVDQU 4000(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x7_loop
- VZEROUPPER
-
-mulAvxTwo_9x7_end:
- RET
-
-// func mulAvxTwo_9x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x7Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 138 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_9x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- MOVQ 120(R13), R15
- VMOVDQU (R15)(R14*1), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- MOVQ 144(R13), R15
- VMOVDQU (R15)(R14*1), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (R12), Y10
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 8 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3584(CX), Y8
- VMOVDQU 3616(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3648(CX), Y8
- VMOVDQU 3680(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3712(CX), Y8
- VMOVDQU 3744(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3776(CX), Y8
- VMOVDQU 3808(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3840(CX), Y8
- VMOVDQU 3872(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3904(CX), Y8
- VMOVDQU 3936(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3968(CX), Y8
- VMOVDQU 4000(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x7Xor_end:
- RET
-
-// func mulAvxTwo_9x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x8(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 157 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_9x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (R12), Y11
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 8 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4096(CX), Y9
- VMOVDQU 4128(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4160(CX), Y9
- VMOVDQU 4192(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4224(CX), Y9
- VMOVDQU 4256(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4288(CX), Y9
- VMOVDQU 4320(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4352(CX), Y9
- VMOVDQU 4384(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4416(CX), Y9
- VMOVDQU 4448(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4480(CX), Y9
- VMOVDQU 4512(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4544(CX), Y9
- VMOVDQU 4576(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x8_loop
- VZEROUPPER
-
-mulAvxTwo_9x8_end:
- RET
-
-// func mulAvxTwo_9x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x8Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 157 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_9x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- MOVQ 120(R13), R15
- VMOVDQU (R15)(R14*1), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- MOVQ 144(R13), R15
- VMOVDQU (R15)(R14*1), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- MOVQ 168(R13), R15
- VMOVDQU (R15)(R14*1), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (R12), Y11
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 8 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4096(CX), Y9
- VMOVDQU 4128(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4160(CX), Y9
- VMOVDQU 4192(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4224(CX), Y9
- VMOVDQU 4256(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4288(CX), Y9
- VMOVDQU 4320(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4352(CX), Y9
- VMOVDQU 4384(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4416(CX), Y9
- VMOVDQU 4448(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4480(CX), Y9
- VMOVDQU 4512(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4544(CX), Y9
- VMOVDQU 4576(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x8Xor_end:
- RET
-
-// func mulAvxTwo_9x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x9(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 176 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_9x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (R12), Y12
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 8 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4608(CX), Y10
- VMOVDQU 4640(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4672(CX), Y10
- VMOVDQU 4704(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4736(CX), Y10
- VMOVDQU 4768(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4800(CX), Y10
- VMOVDQU 4832(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4864(CX), Y10
- VMOVDQU 4896(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4928(CX), Y10
- VMOVDQU 4960(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4992(CX), Y10
- VMOVDQU 5024(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5056(CX), Y10
- VMOVDQU 5088(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5120(CX), Y10
- VMOVDQU 5152(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
- MOVQ 192(R13), R15
- VMOVDQU Y8, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x9_loop
- VZEROUPPER
-
-mulAvxTwo_9x9_end:
- RET
-
-// func mulAvxTwo_9x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x9Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 176 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_9x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R13), R15
- VMOVDQU (R15)(R14*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R13), R15
- VMOVDQU (R15)(R14*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R13), R15
- VMOVDQU (R15)(R14*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R13), R15
- VMOVDQU (R15)(R14*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (R12), Y12
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 8 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4608(CX), Y10
- VMOVDQU 4640(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4672(CX), Y10
- VMOVDQU 4704(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4736(CX), Y10
- VMOVDQU 4768(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4800(CX), Y10
- VMOVDQU 4832(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4864(CX), Y10
- VMOVDQU 4896(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4928(CX), Y10
- VMOVDQU 4960(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4992(CX), Y10
- VMOVDQU 5024(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5056(CX), Y10
- VMOVDQU 5088(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5120(CX), Y10
- VMOVDQU 5152(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
- MOVQ 192(R13), R15
- VMOVDQU Y8, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x9Xor_end:
- RET
-
-// func mulAvxTwo_9x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x10(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 195 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_9x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (R12), Y13
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 8 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5120(CX), Y11
- VMOVDQU 5152(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5184(CX), Y11
- VMOVDQU 5216(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5248(CX), Y11
- VMOVDQU 5280(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5312(CX), Y11
- VMOVDQU 5344(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 5376(CX), Y11
- VMOVDQU 5408(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 5440(CX), Y11
- VMOVDQU 5472(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 5504(CX), Y11
- VMOVDQU 5536(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 5568(CX), Y11
- VMOVDQU 5600(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 5632(CX), Y11
- VMOVDQU 5664(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5696(CX), Y11
- VMOVDQU 5728(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
- MOVQ 192(R13), R15
- VMOVDQU Y8, (R15)(R14*1)
- MOVQ 216(R13), R15
- VMOVDQU Y9, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x10_loop
- VZEROUPPER
-
-mulAvxTwo_9x10_end:
- RET
-
-// func mulAvxTwo_9x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_9x10Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 195 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_9x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), DX
- MOVQ out_base+48(FP), R13
- MOVQ start+72(FP), R14
-
- // Add start offset to input
- ADDQ R14, BX
- ADDQ R14, SI
- ADDQ R14, DI
- ADDQ R14, R8
- ADDQ R14, R9
- ADDQ R14, R10
- ADDQ R14, R11
- ADDQ R14, R12
- ADDQ R14, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_9x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R13), R15
- VMOVDQU (R15)(R14*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R13), R15
- VMOVDQU (R15)(R14*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R13), R15
- VMOVDQU (R15)(R14*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R13), R15
- VMOVDQU (R15)(R14*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R13), R15
- VMOVDQU (R15)(R14*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R13), R15
- VMOVDQU (R15)(R14*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R13), R15
- VMOVDQU (R15)(R14*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R13), R15
- VMOVDQU (R15)(R14*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R13), R15
- VMOVDQU (R15)(R14*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R13), R15
- VMOVDQU (R15)(R14*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (R12), Y13
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 8 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5120(CX), Y11
- VMOVDQU 5152(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5184(CX), Y11
- VMOVDQU 5216(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5248(CX), Y11
- VMOVDQU 5280(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5312(CX), Y11
- VMOVDQU 5344(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 5376(CX), Y11
- VMOVDQU 5408(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 5440(CX), Y11
- VMOVDQU 5472(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 5504(CX), Y11
- VMOVDQU 5536(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 5568(CX), Y11
- VMOVDQU 5600(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 5632(CX), Y11
- VMOVDQU 5664(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5696(CX), Y11
- VMOVDQU 5728(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R13), R15
- VMOVDQU Y0, (R15)(R14*1)
- MOVQ 24(R13), R15
- VMOVDQU Y1, (R15)(R14*1)
- MOVQ 48(R13), R15
- VMOVDQU Y2, (R15)(R14*1)
- MOVQ 72(R13), R15
- VMOVDQU Y3, (R15)(R14*1)
- MOVQ 96(R13), R15
- VMOVDQU Y4, (R15)(R14*1)
- MOVQ 120(R13), R15
- VMOVDQU Y5, (R15)(R14*1)
- MOVQ 144(R13), R15
- VMOVDQU Y6, (R15)(R14*1)
- MOVQ 168(R13), R15
- VMOVDQU Y7, (R15)(R14*1)
- MOVQ 192(R13), R15
- VMOVDQU Y8, (R15)(R14*1)
- MOVQ 216(R13), R15
- VMOVDQU Y9, (R15)(R14*1)
-
- // Prepare for next loop
- ADDQ $0x20, R14
- DECQ AX
- JNZ mulAvxTwo_9x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_9x10Xor_end:
- RET
-
-// func mulAvxTwo_10x1(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x1(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 24 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x1_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_10x1_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- VPXOR Y2, Y3, Y0
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y4
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 8 to 1 outputs
- VMOVDQU (R13), Y4
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 512(CX), Y2
- VMOVDQU 544(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 9 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 576(CX), Y2
- VMOVDQU 608(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x1_loop
- VZEROUPPER
-
-mulAvxTwo_10x1_end:
- RET
-
-// func mulAvxTwo_10x1_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x1_64(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 46 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x1_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_10x1_64_loop:
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- VPXOR Y3, Y4, Y0
- VPXOR Y5, Y6, Y1
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y6
- VMOVDQU 32(R12), Y5
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 8 to 1 outputs
- VMOVDQU (R13), Y6
- VMOVDQU 32(R13), Y5
- ADDQ $0x40, R13
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 9 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x1_64_loop
- VZEROUPPER
-
-mulAvxTwo_10x1_64_end:
- RET
-
-// func mulAvxTwo_10x1Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x1Xor(SB), NOSPLIT, $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 24 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x1Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X1
- VPBROADCASTB X1, Y1
-
-mulAvxTwo_10x1Xor_loop:
- // Load and process 32 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y4
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU (R14), Y0
- VMOVDQU (CX), Y2
- VMOVDQU 32(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y4
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 64(CX), Y2
- VMOVDQU 96(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y4
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 128(CX), Y2
- VMOVDQU 160(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y4
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 192(CX), Y2
- VMOVDQU 224(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y4
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 256(CX), Y2
- VMOVDQU 288(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y4
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 320(CX), Y2
- VMOVDQU 352(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y4
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 384(CX), Y2
- VMOVDQU 416(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y4
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 448(CX), Y2
- VMOVDQU 480(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 8 to 1 outputs
- VMOVDQU (R13), Y4
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 512(CX), Y2
- VMOVDQU 544(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Load and process 32 bytes from input 9 to 1 outputs
- VMOVDQU (DX), Y4
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y4, Y5
- VPAND Y1, Y4, Y4
- VPAND Y1, Y5, Y5
- VMOVDQU 576(CX), Y2
- VMOVDQU 608(CX), Y3
- VPSHUFB Y4, Y2, Y2
- VPSHUFB Y5, Y3, Y3
- XOR3WAY( $0x00, Y2, Y3, Y0)
-
- // Store 1 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x1Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x1Xor_end:
- RET
-
-// func mulAvxTwo_10x1_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x1_64Xor(SB), $0-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 46 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x1_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to output
- ADDQ R15, R14
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, R15
- MOVQ R15, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_10x1_64Xor_loop:
- // Load 1 outputs
- VMOVDQU (R14), Y0
- VMOVDQU 32(R14), Y1
-
- // Load and process 64 bytes from input 0 to 1 outputs
- VMOVDQU (BX), Y6
- VMOVDQU 32(BX), Y5
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 1 to 1 outputs
- VMOVDQU (SI), Y6
- VMOVDQU 32(SI), Y5
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 2 to 1 outputs
- VMOVDQU (DI), Y6
- VMOVDQU 32(DI), Y5
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 3 to 1 outputs
- VMOVDQU (R8), Y6
- VMOVDQU 32(R8), Y5
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 4 to 1 outputs
- VMOVDQU (R9), Y6
- VMOVDQU 32(R9), Y5
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 5 to 1 outputs
- VMOVDQU (R10), Y6
- VMOVDQU 32(R10), Y5
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 6 to 1 outputs
- VMOVDQU (R11), Y6
- VMOVDQU 32(R11), Y5
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 7 to 1 outputs
- VMOVDQU (R12), Y6
- VMOVDQU 32(R12), Y5
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 8 to 1 outputs
- VMOVDQU (R13), Y6
- VMOVDQU 32(R13), Y5
- ADDQ $0x40, R13
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Load and process 64 bytes from input 9 to 1 outputs
- VMOVDQU (DX), Y6
- VMOVDQU 32(DX), Y5
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y6, Y7
- VPSRLQ $0x04, Y5, Y8
- VPAND Y2, Y6, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y7, Y7
- VPAND Y2, Y8, Y8
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y5
- VPSHUFB Y6, Y3, Y3
- VPSHUFB Y8, Y4, Y6
- VPSHUFB Y7, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- XOR3WAY( $0x00, Y5, Y6, Y1)
-
- // Store 1 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x1_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x1_64Xor_end:
- RET
-
-// func mulAvxTwo_10x2(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x2(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 47 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x2_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R15
- MOVQ 24(R14), R14
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R15
- ADDQ BP, R14
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_10x2_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y0
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- VPXOR Y3, Y4, Y1
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y5
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 8 to 2 outputs
- VMOVDQU (R13), Y5
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1024(CX), Y3
- VMOVDQU 1056(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1088(CX), Y3
- VMOVDQU 1120(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 9 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1152(CX), Y3
- VMOVDQU 1184(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1216(CX), Y3
- VMOVDQU 1248(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x2_loop
- VZEROUPPER
-
-mulAvxTwo_10x2_end:
- RET
-
-// func mulAvxTwo_10x2_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x2_64(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 89 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x2_64_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R15
- MOVQ 24(R14), R14
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R15
- ADDQ BP, R14
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_10x2_64_loop:
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y0
- VPXOR Y7, Y8, Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- VPXOR Y5, Y6, Y2
- VPXOR Y7, Y8, Y3
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y9
- VMOVDQU 32(R12), Y11
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 8 to 2 outputs
- VMOVDQU (R13), Y9
- VMOVDQU 32(R13), Y11
- ADDQ $0x40, R13
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 9 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R15)
- VMOVDQU Y1, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y2, (R14)
- VMOVDQU Y3, 32(R14)
- ADDQ $0x40, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x2_64_loop
- VZEROUPPER
-
-mulAvxTwo_10x2_64_end:
- RET
-
-// func mulAvxTwo_10x2Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x2Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 47 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x2Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R15
- MOVQ 24(R14), R14
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R15
- ADDQ BP, R14
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X2
- VPBROADCASTB X2, Y2
-
-mulAvxTwo_10x2Xor_loop:
- // Load and process 32 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y5
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU (R15), Y0
- VMOVDQU (CX), Y3
- VMOVDQU 32(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU (R14), Y1
- VMOVDQU 64(CX), Y3
- VMOVDQU 96(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y5
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 128(CX), Y3
- VMOVDQU 160(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 192(CX), Y3
- VMOVDQU 224(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y5
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 256(CX), Y3
- VMOVDQU 288(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 320(CX), Y3
- VMOVDQU 352(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y5
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 384(CX), Y3
- VMOVDQU 416(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 448(CX), Y3
- VMOVDQU 480(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y5
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 512(CX), Y3
- VMOVDQU 544(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 576(CX), Y3
- VMOVDQU 608(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y5
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 640(CX), Y3
- VMOVDQU 672(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 704(CX), Y3
- VMOVDQU 736(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y5
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 768(CX), Y3
- VMOVDQU 800(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 832(CX), Y3
- VMOVDQU 864(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y5
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 896(CX), Y3
- VMOVDQU 928(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 960(CX), Y3
- VMOVDQU 992(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 8 to 2 outputs
- VMOVDQU (R13), Y5
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1024(CX), Y3
- VMOVDQU 1056(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1088(CX), Y3
- VMOVDQU 1120(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Load and process 32 bytes from input 9 to 2 outputs
- VMOVDQU (DX), Y5
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y5, Y6
- VPAND Y2, Y5, Y5
- VPAND Y2, Y6, Y6
- VMOVDQU 1152(CX), Y3
- VMOVDQU 1184(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y0)
- VMOVDQU 1216(CX), Y3
- VMOVDQU 1248(CX), Y4
- VPSHUFB Y5, Y3, Y3
- VPSHUFB Y6, Y4, Y4
- XOR3WAY( $0x00, Y3, Y4, Y1)
-
- // Store 2 outputs
- VMOVDQU Y0, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y1, (R14)
- ADDQ $0x20, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x2Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x2Xor_end:
- RET
-
-// func mulAvxTwo_10x2_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x2_64Xor(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 89 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x2_64Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ out_base+48(FP), R14
- MOVQ (R14), R15
- MOVQ 24(R14), R14
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R15
- ADDQ BP, R14
-
- // Add start offset to input
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, R13
- ADDQ BP, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_10x2_64Xor_loop:
- // Load 2 outputs
- VMOVDQU (R15), Y0
- VMOVDQU 32(R15), Y1
- VMOVDQU (R14), Y2
- VMOVDQU 32(R14), Y3
-
- // Load and process 64 bytes from input 0 to 2 outputs
- VMOVDQU (BX), Y9
- VMOVDQU 32(BX), Y11
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 1 to 2 outputs
- VMOVDQU (SI), Y9
- VMOVDQU 32(SI), Y11
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 2 to 2 outputs
- VMOVDQU (DI), Y9
- VMOVDQU 32(DI), Y11
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 3 to 2 outputs
- VMOVDQU (R8), Y9
- VMOVDQU 32(R8), Y11
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 4 to 2 outputs
- VMOVDQU (R9), Y9
- VMOVDQU 32(R9), Y11
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 5 to 2 outputs
- VMOVDQU (R10), Y9
- VMOVDQU 32(R10), Y11
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 6 to 2 outputs
- VMOVDQU (R11), Y9
- VMOVDQU 32(R11), Y11
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 7 to 2 outputs
- VMOVDQU (R12), Y9
- VMOVDQU 32(R12), Y11
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 8 to 2 outputs
- VMOVDQU (R13), Y9
- VMOVDQU 32(R13), Y11
- ADDQ $0x40, R13
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Load and process 64 bytes from input 9 to 2 outputs
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y11
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y9, Y10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y4, Y9, Y9
- VPAND Y4, Y11, Y11
- VPAND Y4, Y10, Y10
- VPAND Y4, Y12, Y12
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y11, Y5, Y7
- VPSHUFB Y9, Y5, Y5
- VPSHUFB Y12, Y6, Y8
- VPSHUFB Y10, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- XOR3WAY( $0x00, Y7, Y8, Y3)
-
- // Store 2 outputs
- VMOVDQU Y0, (R15)
- VMOVDQU Y1, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y2, (R14)
- VMOVDQU Y3, 32(R14)
- ADDQ $0x40, R14
-
- // Prepare for next loop
- DECQ AX
- JNZ mulAvxTwo_10x2_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x2_64Xor_end:
- RET
-
-// func mulAvxTwo_10x3(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x3(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x3_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), R12
- MOVQ 216(AX), AX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X3
- VPBROADCASTB X3, Y3
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_10x3_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y0
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y1
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- VPXOR Y4, Y5, Y2
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 8 to 3 outputs
- VMOVDQU (R12), Y6
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1536(CX), Y4
- VMOVDQU 1568(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1600(CX), Y4
- VMOVDQU 1632(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1664(CX), Y4
- VMOVDQU 1696(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 9 to 3 outputs
- VMOVDQU (AX), Y6
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1728(CX), Y4
- VMOVDQU 1760(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1792(CX), Y4
- VMOVDQU 1824(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1856(CX), Y4
- VMOVDQU 1888(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_10x3_loop
- VZEROUPPER
-
-mulAvxTwo_10x3_end:
- RET
-
-// func mulAvxTwo_10x3_64(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x3_64(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 130 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x3_64_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), R12
- MOVQ 216(AX), AX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
- // Reload length to save a register
- MOVQ n+80(FP), BP
- SHRQ $0x06, BP
-
-mulAvxTwo_10x3_64_loop:
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y0
- VPXOR Y9, Y10, Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y2
- VPXOR Y9, Y10, Y3
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- VPXOR Y7, Y8, Y4
- VPXOR Y9, Y10, Y5
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 8 to 3 outputs
- VMOVDQU (R12), Y11
- VMOVDQU 32(R12), Y13
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 9 to 3 outputs
- VMOVDQU (AX), Y11
- VMOVDQU 32(AX), Y13
- ADDQ $0x40, AX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R15)
- VMOVDQU Y3, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y4, (R13)
- VMOVDQU Y5, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_10x3_64_loop
- VZEROUPPER
-
-mulAvxTwo_10x3_64_end:
- RET
-
-// func mulAvxTwo_10x3Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x3Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 68 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x3Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), R12
- MOVQ 216(AX), AX
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X3
- VPBROADCASTB X3, Y3
- MOVQ n+80(FP), BP
- SHRQ $0x05, BP
-
-mulAvxTwo_10x3Xor_loop:
- // Load and process 32 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y6
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU (R14), Y0
- VMOVDQU (CX), Y4
- VMOVDQU 32(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU (R15), Y1
- VMOVDQU 64(CX), Y4
- VMOVDQU 96(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU (R13), Y2
- VMOVDQU 128(CX), Y4
- VMOVDQU 160(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 1 to 3 outputs
- VMOVDQU (BX), Y6
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 192(CX), Y4
- VMOVDQU 224(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 256(CX), Y4
- VMOVDQU 288(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 320(CX), Y4
- VMOVDQU 352(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 2 to 3 outputs
- VMOVDQU (SI), Y6
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 384(CX), Y4
- VMOVDQU 416(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 448(CX), Y4
- VMOVDQU 480(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 512(CX), Y4
- VMOVDQU 544(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 3 to 3 outputs
- VMOVDQU (DI), Y6
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 576(CX), Y4
- VMOVDQU 608(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 640(CX), Y4
- VMOVDQU 672(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 704(CX), Y4
- VMOVDQU 736(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 4 to 3 outputs
- VMOVDQU (R8), Y6
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 768(CX), Y4
- VMOVDQU 800(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 832(CX), Y4
- VMOVDQU 864(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 896(CX), Y4
- VMOVDQU 928(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 5 to 3 outputs
- VMOVDQU (R9), Y6
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 960(CX), Y4
- VMOVDQU 992(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1024(CX), Y4
- VMOVDQU 1056(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1088(CX), Y4
- VMOVDQU 1120(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 6 to 3 outputs
- VMOVDQU (R10), Y6
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1152(CX), Y4
- VMOVDQU 1184(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1216(CX), Y4
- VMOVDQU 1248(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1280(CX), Y4
- VMOVDQU 1312(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 7 to 3 outputs
- VMOVDQU (R11), Y6
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1344(CX), Y4
- VMOVDQU 1376(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1408(CX), Y4
- VMOVDQU 1440(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1472(CX), Y4
- VMOVDQU 1504(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 8 to 3 outputs
- VMOVDQU (R12), Y6
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1536(CX), Y4
- VMOVDQU 1568(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1600(CX), Y4
- VMOVDQU 1632(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1664(CX), Y4
- VMOVDQU 1696(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Load and process 32 bytes from input 9 to 3 outputs
- VMOVDQU (AX), Y6
- ADDQ $0x20, AX
- VPSRLQ $0x04, Y6, Y7
- VPAND Y3, Y6, Y6
- VPAND Y3, Y7, Y7
- VMOVDQU 1728(CX), Y4
- VMOVDQU 1760(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y0)
- VMOVDQU 1792(CX), Y4
- VMOVDQU 1824(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y1)
- VMOVDQU 1856(CX), Y4
- VMOVDQU 1888(CX), Y5
- VPSHUFB Y6, Y4, Y4
- VPSHUFB Y7, Y5, Y5
- XOR3WAY( $0x00, Y4, Y5, Y2)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- ADDQ $0x20, R14
- VMOVDQU Y1, (R15)
- ADDQ $0x20, R15
- VMOVDQU Y2, (R13)
- ADDQ $0x20, R13
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_10x3Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x3Xor_end:
- RET
-
-// func mulAvxTwo_10x3_64Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x3_64Xor(SB), $8-88
- // Loading no tables to registers
- // Destination kept in GP registers
- // Full registers estimated 130 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x06, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x3_64Xor_end
- MOVQ in_base+24(FP), AX
- MOVQ (AX), DX
- MOVQ 24(AX), BX
- MOVQ 48(AX), SI
- MOVQ 72(AX), DI
- MOVQ 96(AX), R8
- MOVQ 120(AX), R9
- MOVQ 144(AX), R10
- MOVQ 168(AX), R11
- MOVQ 192(AX), R12
- MOVQ 216(AX), AX
- MOVQ out_base+48(FP), R13
- MOVQ out_base+48(FP), R13
- MOVQ (R13), R14
- MOVQ 24(R13), R15
- MOVQ 48(R13), R13
- MOVQ start+72(FP), BP
-
- // Add start offset to output
- ADDQ BP, R14
- ADDQ BP, R15
- ADDQ BP, R13
-
- // Add start offset to input
- ADDQ BP, DX
- ADDQ BP, BX
- ADDQ BP, SI
- ADDQ BP, DI
- ADDQ BP, R8
- ADDQ BP, R9
- ADDQ BP, R10
- ADDQ BP, R11
- ADDQ BP, R12
- ADDQ BP, AX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
- // Reload length to save a register
- MOVQ n+80(FP), BP
- SHRQ $0x06, BP
-
-mulAvxTwo_10x3_64Xor_loop:
- // Load 3 outputs
- VMOVDQU (R14), Y0
- VMOVDQU 32(R14), Y1
- VMOVDQU (R15), Y2
- VMOVDQU 32(R15), Y3
- VMOVDQU (R13), Y4
- VMOVDQU 32(R13), Y5
-
- // Load and process 64 bytes from input 0 to 3 outputs
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y13
- ADDQ $0x40, DX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 1 to 3 outputs
- VMOVDQU (BX), Y11
- VMOVDQU 32(BX), Y13
- ADDQ $0x40, BX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 2 to 3 outputs
- VMOVDQU (SI), Y11
- VMOVDQU 32(SI), Y13
- ADDQ $0x40, SI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 3 to 3 outputs
- VMOVDQU (DI), Y11
- VMOVDQU 32(DI), Y13
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 4 to 3 outputs
- VMOVDQU (R8), Y11
- VMOVDQU 32(R8), Y13
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 5 to 3 outputs
- VMOVDQU (R9), Y11
- VMOVDQU 32(R9), Y13
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 6 to 3 outputs
- VMOVDQU (R10), Y11
- VMOVDQU 32(R10), Y13
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 7 to 3 outputs
- VMOVDQU (R11), Y11
- VMOVDQU 32(R11), Y13
- ADDQ $0x40, R11
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 8 to 3 outputs
- VMOVDQU (R12), Y11
- VMOVDQU 32(R12), Y13
- ADDQ $0x40, R12
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Load and process 64 bytes from input 9 to 3 outputs
- VMOVDQU (AX), Y11
- VMOVDQU 32(AX), Y13
- ADDQ $0x40, AX
- VPSRLQ $0x04, Y11, Y12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y6, Y11, Y11
- VPAND Y6, Y13, Y13
- VPAND Y6, Y12, Y12
- VPAND Y6, Y14, Y14
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y13, Y7, Y9
- VPSHUFB Y11, Y7, Y7
- VPSHUFB Y14, Y8, Y10
- VPSHUFB Y12, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- XOR3WAY( $0x00, Y9, Y10, Y5)
-
- // Store 3 outputs
- VMOVDQU Y0, (R14)
- VMOVDQU Y1, 32(R14)
- ADDQ $0x40, R14
- VMOVDQU Y2, (R15)
- VMOVDQU Y3, 32(R15)
- ADDQ $0x40, R15
- VMOVDQU Y4, (R13)
- VMOVDQU Y5, 32(R13)
- ADDQ $0x40, R13
-
- // Prepare for next loop
- DECQ BP
- JNZ mulAvxTwo_10x3_64Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x3_64Xor_end:
- RET
-
-// func mulAvxTwo_10x4(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x4(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 89 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x4_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_10x4_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y0
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y1
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y2
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- VPXOR Y5, Y6, Y3
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (R12), Y7
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 8 to 4 outputs
- VMOVDQU (R13), Y7
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2048(CX), Y5
- VMOVDQU 2080(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2112(CX), Y5
- VMOVDQU 2144(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2176(CX), Y5
- VMOVDQU 2208(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2240(CX), Y5
- VMOVDQU 2272(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 9 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2304(CX), Y5
- VMOVDQU 2336(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2368(CX), Y5
- VMOVDQU 2400(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2432(CX), Y5
- VMOVDQU 2464(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2496(CX), Y5
- VMOVDQU 2528(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x4_loop
- VZEROUPPER
-
-mulAvxTwo_10x4_end:
- RET
-
-// func mulAvxTwo_10x4Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x4Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 89 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x4Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X4
- VPBROADCASTB X4, Y4
-
-mulAvxTwo_10x4Xor_loop:
- // Load and process 32 bytes from input 0 to 4 outputs
- VMOVDQU (BX), Y7
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y5
- VMOVDQU 32(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y5
- VMOVDQU 96(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y5
- VMOVDQU 160(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y5
- VMOVDQU 224(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 1 to 4 outputs
- VMOVDQU (SI), Y7
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 256(CX), Y5
- VMOVDQU 288(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 320(CX), Y5
- VMOVDQU 352(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 384(CX), Y5
- VMOVDQU 416(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 448(CX), Y5
- VMOVDQU 480(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 2 to 4 outputs
- VMOVDQU (DI), Y7
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 512(CX), Y5
- VMOVDQU 544(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 576(CX), Y5
- VMOVDQU 608(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 640(CX), Y5
- VMOVDQU 672(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 704(CX), Y5
- VMOVDQU 736(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 3 to 4 outputs
- VMOVDQU (R8), Y7
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 768(CX), Y5
- VMOVDQU 800(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 832(CX), Y5
- VMOVDQU 864(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 896(CX), Y5
- VMOVDQU 928(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 960(CX), Y5
- VMOVDQU 992(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 4 to 4 outputs
- VMOVDQU (R9), Y7
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1024(CX), Y5
- VMOVDQU 1056(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1088(CX), Y5
- VMOVDQU 1120(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1152(CX), Y5
- VMOVDQU 1184(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1216(CX), Y5
- VMOVDQU 1248(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 5 to 4 outputs
- VMOVDQU (R10), Y7
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1280(CX), Y5
- VMOVDQU 1312(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1344(CX), Y5
- VMOVDQU 1376(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1408(CX), Y5
- VMOVDQU 1440(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1472(CX), Y5
- VMOVDQU 1504(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 6 to 4 outputs
- VMOVDQU (R11), Y7
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1536(CX), Y5
- VMOVDQU 1568(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1600(CX), Y5
- VMOVDQU 1632(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1664(CX), Y5
- VMOVDQU 1696(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1728(CX), Y5
- VMOVDQU 1760(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 7 to 4 outputs
- VMOVDQU (R12), Y7
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 1792(CX), Y5
- VMOVDQU 1824(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 1856(CX), Y5
- VMOVDQU 1888(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 1920(CX), Y5
- VMOVDQU 1952(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 1984(CX), Y5
- VMOVDQU 2016(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 8 to 4 outputs
- VMOVDQU (R13), Y7
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2048(CX), Y5
- VMOVDQU 2080(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2112(CX), Y5
- VMOVDQU 2144(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2176(CX), Y5
- VMOVDQU 2208(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2240(CX), Y5
- VMOVDQU 2272(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Load and process 32 bytes from input 9 to 4 outputs
- VMOVDQU (DX), Y7
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y7, Y8
- VPAND Y4, Y7, Y7
- VPAND Y4, Y8, Y8
- VMOVDQU 2304(CX), Y5
- VMOVDQU 2336(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y0)
- VMOVDQU 2368(CX), Y5
- VMOVDQU 2400(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y1)
- VMOVDQU 2432(CX), Y5
- VMOVDQU 2464(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU 2496(CX), Y5
- VMOVDQU 2528(CX), Y6
- VPSHUFB Y7, Y5, Y5
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y5, Y6, Y3)
-
- // Store 4 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x4Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x4Xor_end:
- RET
-
-// func mulAvxTwo_10x5(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x5(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 110 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x5_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_10x5_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y0
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y1
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y2
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y3
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- VPXOR Y6, Y7, Y4
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R11), Y8
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (R12), Y8
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 8 to 5 outputs
- VMOVDQU (R13), Y8
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2560(CX), Y6
- VMOVDQU 2592(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2624(CX), Y6
- VMOVDQU 2656(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2688(CX), Y6
- VMOVDQU 2720(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2752(CX), Y6
- VMOVDQU 2784(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2816(CX), Y6
- VMOVDQU 2848(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 9 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2880(CX), Y6
- VMOVDQU 2912(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2944(CX), Y6
- VMOVDQU 2976(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 3008(CX), Y6
- VMOVDQU 3040(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 3072(CX), Y6
- VMOVDQU 3104(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 3136(CX), Y6
- VMOVDQU 3168(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x5_loop
- VZEROUPPER
-
-mulAvxTwo_10x5_end:
- RET
-
-// func mulAvxTwo_10x5Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x5Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 110 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x5Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X5
- VPBROADCASTB X5, Y5
-
-mulAvxTwo_10x5Xor_loop:
- // Load and process 32 bytes from input 0 to 5 outputs
- VMOVDQU (BX), Y8
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y6
- VMOVDQU 32(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y6
- VMOVDQU 96(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y6
- VMOVDQU 160(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y6
- VMOVDQU 224(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y6
- VMOVDQU 288(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 1 to 5 outputs
- VMOVDQU (SI), Y8
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 320(CX), Y6
- VMOVDQU 352(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 384(CX), Y6
- VMOVDQU 416(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 448(CX), Y6
- VMOVDQU 480(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 512(CX), Y6
- VMOVDQU 544(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 576(CX), Y6
- VMOVDQU 608(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 2 to 5 outputs
- VMOVDQU (DI), Y8
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 640(CX), Y6
- VMOVDQU 672(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 704(CX), Y6
- VMOVDQU 736(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 768(CX), Y6
- VMOVDQU 800(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 832(CX), Y6
- VMOVDQU 864(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 896(CX), Y6
- VMOVDQU 928(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 3 to 5 outputs
- VMOVDQU (R8), Y8
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 960(CX), Y6
- VMOVDQU 992(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1024(CX), Y6
- VMOVDQU 1056(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1088(CX), Y6
- VMOVDQU 1120(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1152(CX), Y6
- VMOVDQU 1184(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1216(CX), Y6
- VMOVDQU 1248(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 4 to 5 outputs
- VMOVDQU (R9), Y8
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1280(CX), Y6
- VMOVDQU 1312(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1344(CX), Y6
- VMOVDQU 1376(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1408(CX), Y6
- VMOVDQU 1440(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1472(CX), Y6
- VMOVDQU 1504(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1536(CX), Y6
- VMOVDQU 1568(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 5 to 5 outputs
- VMOVDQU (R10), Y8
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1600(CX), Y6
- VMOVDQU 1632(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1664(CX), Y6
- VMOVDQU 1696(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 1728(CX), Y6
- VMOVDQU 1760(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 1792(CX), Y6
- VMOVDQU 1824(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 1856(CX), Y6
- VMOVDQU 1888(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 6 to 5 outputs
- VMOVDQU (R11), Y8
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 1920(CX), Y6
- VMOVDQU 1952(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 1984(CX), Y6
- VMOVDQU 2016(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2048(CX), Y6
- VMOVDQU 2080(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2112(CX), Y6
- VMOVDQU 2144(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2176(CX), Y6
- VMOVDQU 2208(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 7 to 5 outputs
- VMOVDQU (R12), Y8
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2240(CX), Y6
- VMOVDQU 2272(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2304(CX), Y6
- VMOVDQU 2336(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2368(CX), Y6
- VMOVDQU 2400(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2432(CX), Y6
- VMOVDQU 2464(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2496(CX), Y6
- VMOVDQU 2528(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 8 to 5 outputs
- VMOVDQU (R13), Y8
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2560(CX), Y6
- VMOVDQU 2592(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2624(CX), Y6
- VMOVDQU 2656(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 2688(CX), Y6
- VMOVDQU 2720(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 2752(CX), Y6
- VMOVDQU 2784(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 2816(CX), Y6
- VMOVDQU 2848(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Load and process 32 bytes from input 9 to 5 outputs
- VMOVDQU (DX), Y8
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y8, Y9
- VPAND Y5, Y8, Y8
- VPAND Y5, Y9, Y9
- VMOVDQU 2880(CX), Y6
- VMOVDQU 2912(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y0)
- VMOVDQU 2944(CX), Y6
- VMOVDQU 2976(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y1)
- VMOVDQU 3008(CX), Y6
- VMOVDQU 3040(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y2)
- VMOVDQU 3072(CX), Y6
- VMOVDQU 3104(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y3)
- VMOVDQU 3136(CX), Y6
- VMOVDQU 3168(CX), Y7
- VPSHUFB Y8, Y6, Y6
- VPSHUFB Y9, Y7, Y7
- XOR3WAY( $0x00, Y6, Y7, Y4)
-
- // Store 5 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x5Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x5Xor_end:
- RET
-
-// func mulAvxTwo_10x6(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x6(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 131 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x6_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_10x6_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y0
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y1
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y2
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y3
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y4
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- VPXOR Y7, Y8, Y5
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (R12), Y9
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 8 to 6 outputs
- VMOVDQU (R13), Y9
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3072(CX), Y7
- VMOVDQU 3104(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3136(CX), Y7
- VMOVDQU 3168(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3200(CX), Y7
- VMOVDQU 3232(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3264(CX), Y7
- VMOVDQU 3296(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3328(CX), Y7
- VMOVDQU 3360(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3392(CX), Y7
- VMOVDQU 3424(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 9 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3456(CX), Y7
- VMOVDQU 3488(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3520(CX), Y7
- VMOVDQU 3552(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3584(CX), Y7
- VMOVDQU 3616(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3648(CX), Y7
- VMOVDQU 3680(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3712(CX), Y7
- VMOVDQU 3744(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3776(CX), Y7
- VMOVDQU 3808(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x6_loop
- VZEROUPPER
-
-mulAvxTwo_10x6_end:
- RET
-
-// func mulAvxTwo_10x6Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x6Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 131 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x6Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X6
- VPBROADCASTB X6, Y6
-
-mulAvxTwo_10x6Xor_loop:
- // Load and process 32 bytes from input 0 to 6 outputs
- VMOVDQU (BX), Y9
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y7
- VMOVDQU 96(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y7
- VMOVDQU 160(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y7
- VMOVDQU 224(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y7
- VMOVDQU 288(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- MOVQ 120(R14), BP
- VMOVDQU (BP)(R15*1), Y5
- VMOVDQU 320(CX), Y7
- VMOVDQU 352(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 1 to 6 outputs
- VMOVDQU (SI), Y9
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 384(CX), Y7
- VMOVDQU 416(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 448(CX), Y7
- VMOVDQU 480(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 512(CX), Y7
- VMOVDQU 544(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 576(CX), Y7
- VMOVDQU 608(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 640(CX), Y7
- VMOVDQU 672(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 704(CX), Y7
- VMOVDQU 736(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 2 to 6 outputs
- VMOVDQU (DI), Y9
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 768(CX), Y7
- VMOVDQU 800(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 832(CX), Y7
- VMOVDQU 864(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 896(CX), Y7
- VMOVDQU 928(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 960(CX), Y7
- VMOVDQU 992(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1024(CX), Y7
- VMOVDQU 1056(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1088(CX), Y7
- VMOVDQU 1120(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 3 to 6 outputs
- VMOVDQU (R8), Y9
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1152(CX), Y7
- VMOVDQU 1184(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1216(CX), Y7
- VMOVDQU 1248(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1280(CX), Y7
- VMOVDQU 1312(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1344(CX), Y7
- VMOVDQU 1376(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1408(CX), Y7
- VMOVDQU 1440(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1472(CX), Y7
- VMOVDQU 1504(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 4 to 6 outputs
- VMOVDQU (R9), Y9
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1536(CX), Y7
- VMOVDQU 1568(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1600(CX), Y7
- VMOVDQU 1632(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 1664(CX), Y7
- VMOVDQU 1696(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 1728(CX), Y7
- VMOVDQU 1760(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 1792(CX), Y7
- VMOVDQU 1824(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 1856(CX), Y7
- VMOVDQU 1888(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 5 to 6 outputs
- VMOVDQU (R10), Y9
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 1920(CX), Y7
- VMOVDQU 1952(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 1984(CX), Y7
- VMOVDQU 2016(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2048(CX), Y7
- VMOVDQU 2080(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2112(CX), Y7
- VMOVDQU 2144(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2176(CX), Y7
- VMOVDQU 2208(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2240(CX), Y7
- VMOVDQU 2272(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 6 to 6 outputs
- VMOVDQU (R11), Y9
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2304(CX), Y7
- VMOVDQU 2336(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2368(CX), Y7
- VMOVDQU 2400(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2432(CX), Y7
- VMOVDQU 2464(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2496(CX), Y7
- VMOVDQU 2528(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2560(CX), Y7
- VMOVDQU 2592(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 2624(CX), Y7
- VMOVDQU 2656(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 7 to 6 outputs
- VMOVDQU (R12), Y9
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 2688(CX), Y7
- VMOVDQU 2720(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 2752(CX), Y7
- VMOVDQU 2784(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 2816(CX), Y7
- VMOVDQU 2848(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 2880(CX), Y7
- VMOVDQU 2912(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 2944(CX), Y7
- VMOVDQU 2976(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3008(CX), Y7
- VMOVDQU 3040(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 8 to 6 outputs
- VMOVDQU (R13), Y9
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3072(CX), Y7
- VMOVDQU 3104(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3136(CX), Y7
- VMOVDQU 3168(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3200(CX), Y7
- VMOVDQU 3232(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3264(CX), Y7
- VMOVDQU 3296(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3328(CX), Y7
- VMOVDQU 3360(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3392(CX), Y7
- VMOVDQU 3424(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Load and process 32 bytes from input 9 to 6 outputs
- VMOVDQU (DX), Y9
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y9, Y10
- VPAND Y6, Y9, Y9
- VPAND Y6, Y10, Y10
- VMOVDQU 3456(CX), Y7
- VMOVDQU 3488(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y0)
- VMOVDQU 3520(CX), Y7
- VMOVDQU 3552(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y1)
- VMOVDQU 3584(CX), Y7
- VMOVDQU 3616(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y2)
- VMOVDQU 3648(CX), Y7
- VMOVDQU 3680(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y3)
- VMOVDQU 3712(CX), Y7
- VMOVDQU 3744(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y4)
- VMOVDQU 3776(CX), Y7
- VMOVDQU 3808(CX), Y8
- VPSHUFB Y9, Y7, Y7
- VPSHUFB Y10, Y8, Y8
- XOR3WAY( $0x00, Y7, Y8, Y5)
-
- // Store 6 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x6Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x6Xor_end:
- RET
-
-// func mulAvxTwo_10x7(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x7(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 152 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x7_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_10x7_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y0
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y1
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y2
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y3
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y4
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y5
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- VPXOR Y8, Y9, Y6
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (R12), Y10
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 8 to 7 outputs
- VMOVDQU (R13), Y10
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3584(CX), Y8
- VMOVDQU 3616(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3648(CX), Y8
- VMOVDQU 3680(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3712(CX), Y8
- VMOVDQU 3744(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3776(CX), Y8
- VMOVDQU 3808(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3840(CX), Y8
- VMOVDQU 3872(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3904(CX), Y8
- VMOVDQU 3936(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3968(CX), Y8
- VMOVDQU 4000(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 9 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 4032(CX), Y8
- VMOVDQU 4064(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 4096(CX), Y8
- VMOVDQU 4128(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 4160(CX), Y8
- VMOVDQU 4192(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 4224(CX), Y8
- VMOVDQU 4256(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 4288(CX), Y8
- VMOVDQU 4320(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 4352(CX), Y8
- VMOVDQU 4384(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 4416(CX), Y8
- VMOVDQU 4448(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x7_loop
- VZEROUPPER
-
-mulAvxTwo_10x7_end:
- RET
-
-// func mulAvxTwo_10x7Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x7Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 152 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x7Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X7
- VPBROADCASTB X7, Y7
-
-mulAvxTwo_10x7Xor_loop:
- // Load and process 32 bytes from input 0 to 7 outputs
- VMOVDQU (BX), Y10
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y8
- VMOVDQU 32(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y8
- VMOVDQU 96(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y8
- VMOVDQU 160(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y8
- VMOVDQU 224(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y8
- VMOVDQU 288(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- MOVQ 120(R14), BP
- VMOVDQU (BP)(R15*1), Y5
- VMOVDQU 320(CX), Y8
- VMOVDQU 352(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- MOVQ 144(R14), BP
- VMOVDQU (BP)(R15*1), Y6
- VMOVDQU 384(CX), Y8
- VMOVDQU 416(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 1 to 7 outputs
- VMOVDQU (SI), Y10
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 448(CX), Y8
- VMOVDQU 480(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 512(CX), Y8
- VMOVDQU 544(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 576(CX), Y8
- VMOVDQU 608(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 640(CX), Y8
- VMOVDQU 672(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 704(CX), Y8
- VMOVDQU 736(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 768(CX), Y8
- VMOVDQU 800(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 832(CX), Y8
- VMOVDQU 864(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 2 to 7 outputs
- VMOVDQU (DI), Y10
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 896(CX), Y8
- VMOVDQU 928(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 960(CX), Y8
- VMOVDQU 992(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1024(CX), Y8
- VMOVDQU 1056(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1088(CX), Y8
- VMOVDQU 1120(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1152(CX), Y8
- VMOVDQU 1184(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1216(CX), Y8
- VMOVDQU 1248(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1280(CX), Y8
- VMOVDQU 1312(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 3 to 7 outputs
- VMOVDQU (R8), Y10
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1344(CX), Y8
- VMOVDQU 1376(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1408(CX), Y8
- VMOVDQU 1440(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1472(CX), Y8
- VMOVDQU 1504(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1536(CX), Y8
- VMOVDQU 1568(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 1600(CX), Y8
- VMOVDQU 1632(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 1664(CX), Y8
- VMOVDQU 1696(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 1728(CX), Y8
- VMOVDQU 1760(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 4 to 7 outputs
- VMOVDQU (R9), Y10
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 1792(CX), Y8
- VMOVDQU 1824(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 1856(CX), Y8
- VMOVDQU 1888(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 1920(CX), Y8
- VMOVDQU 1952(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 1984(CX), Y8
- VMOVDQU 2016(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2048(CX), Y8
- VMOVDQU 2080(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2112(CX), Y8
- VMOVDQU 2144(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2176(CX), Y8
- VMOVDQU 2208(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 5 to 7 outputs
- VMOVDQU (R10), Y10
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2240(CX), Y8
- VMOVDQU 2272(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2304(CX), Y8
- VMOVDQU 2336(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2368(CX), Y8
- VMOVDQU 2400(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2432(CX), Y8
- VMOVDQU 2464(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2496(CX), Y8
- VMOVDQU 2528(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 2560(CX), Y8
- VMOVDQU 2592(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 2624(CX), Y8
- VMOVDQU 2656(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 6 to 7 outputs
- VMOVDQU (R11), Y10
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 2688(CX), Y8
- VMOVDQU 2720(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 2752(CX), Y8
- VMOVDQU 2784(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 2816(CX), Y8
- VMOVDQU 2848(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 2880(CX), Y8
- VMOVDQU 2912(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 2944(CX), Y8
- VMOVDQU 2976(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3008(CX), Y8
- VMOVDQU 3040(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3072(CX), Y8
- VMOVDQU 3104(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 7 to 7 outputs
- VMOVDQU (R12), Y10
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3136(CX), Y8
- VMOVDQU 3168(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3200(CX), Y8
- VMOVDQU 3232(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3264(CX), Y8
- VMOVDQU 3296(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3328(CX), Y8
- VMOVDQU 3360(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3392(CX), Y8
- VMOVDQU 3424(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3456(CX), Y8
- VMOVDQU 3488(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3520(CX), Y8
- VMOVDQU 3552(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 8 to 7 outputs
- VMOVDQU (R13), Y10
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 3584(CX), Y8
- VMOVDQU 3616(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 3648(CX), Y8
- VMOVDQU 3680(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 3712(CX), Y8
- VMOVDQU 3744(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 3776(CX), Y8
- VMOVDQU 3808(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 3840(CX), Y8
- VMOVDQU 3872(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 3904(CX), Y8
- VMOVDQU 3936(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 3968(CX), Y8
- VMOVDQU 4000(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Load and process 32 bytes from input 9 to 7 outputs
- VMOVDQU (DX), Y10
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y10, Y11
- VPAND Y7, Y10, Y10
- VPAND Y7, Y11, Y11
- VMOVDQU 4032(CX), Y8
- VMOVDQU 4064(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y0)
- VMOVDQU 4096(CX), Y8
- VMOVDQU 4128(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y1)
- VMOVDQU 4160(CX), Y8
- VMOVDQU 4192(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y2)
- VMOVDQU 4224(CX), Y8
- VMOVDQU 4256(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y3)
- VMOVDQU 4288(CX), Y8
- VMOVDQU 4320(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y4)
- VMOVDQU 4352(CX), Y8
- VMOVDQU 4384(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y5)
- VMOVDQU 4416(CX), Y8
- VMOVDQU 4448(CX), Y9
- VPSHUFB Y10, Y8, Y8
- VPSHUFB Y11, Y9, Y9
- XOR3WAY( $0x00, Y8, Y9, Y6)
-
- // Store 7 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x7Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x7Xor_end:
- RET
-
-// func mulAvxTwo_10x8(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x8(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 173 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x8_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_10x8_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y0
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y1
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y2
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y3
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y4
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y5
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y6
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- VPXOR Y9, Y10, Y7
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (R12), Y11
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 8 to 8 outputs
- VMOVDQU (R13), Y11
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4096(CX), Y9
- VMOVDQU 4128(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4160(CX), Y9
- VMOVDQU 4192(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4224(CX), Y9
- VMOVDQU 4256(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4288(CX), Y9
- VMOVDQU 4320(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4352(CX), Y9
- VMOVDQU 4384(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4416(CX), Y9
- VMOVDQU 4448(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4480(CX), Y9
- VMOVDQU 4512(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4544(CX), Y9
- VMOVDQU 4576(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 9 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4608(CX), Y9
- VMOVDQU 4640(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4672(CX), Y9
- VMOVDQU 4704(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4736(CX), Y9
- VMOVDQU 4768(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4800(CX), Y9
- VMOVDQU 4832(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4864(CX), Y9
- VMOVDQU 4896(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4928(CX), Y9
- VMOVDQU 4960(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4992(CX), Y9
- VMOVDQU 5024(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 5056(CX), Y9
- VMOVDQU 5088(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x8_loop
- VZEROUPPER
-
-mulAvxTwo_10x8_end:
- RET
-
-// func mulAvxTwo_10x8Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x8Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 173 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x8Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X8
- VPBROADCASTB X8, Y8
-
-mulAvxTwo_10x8Xor_loop:
- // Load and process 32 bytes from input 0 to 8 outputs
- VMOVDQU (BX), Y11
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y9
- VMOVDQU 96(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y9
- VMOVDQU 160(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y9
- VMOVDQU 224(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y9
- VMOVDQU 288(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- MOVQ 120(R14), BP
- VMOVDQU (BP)(R15*1), Y5
- VMOVDQU 320(CX), Y9
- VMOVDQU 352(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- MOVQ 144(R14), BP
- VMOVDQU (BP)(R15*1), Y6
- VMOVDQU 384(CX), Y9
- VMOVDQU 416(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- MOVQ 168(R14), BP
- VMOVDQU (BP)(R15*1), Y7
- VMOVDQU 448(CX), Y9
- VMOVDQU 480(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 1 to 8 outputs
- VMOVDQU (SI), Y11
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 512(CX), Y9
- VMOVDQU 544(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 576(CX), Y9
- VMOVDQU 608(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 640(CX), Y9
- VMOVDQU 672(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 704(CX), Y9
- VMOVDQU 736(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 768(CX), Y9
- VMOVDQU 800(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 832(CX), Y9
- VMOVDQU 864(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 896(CX), Y9
- VMOVDQU 928(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 960(CX), Y9
- VMOVDQU 992(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 2 to 8 outputs
- VMOVDQU (DI), Y11
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1024(CX), Y9
- VMOVDQU 1056(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1088(CX), Y9
- VMOVDQU 1120(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1152(CX), Y9
- VMOVDQU 1184(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1216(CX), Y9
- VMOVDQU 1248(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1280(CX), Y9
- VMOVDQU 1312(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1344(CX), Y9
- VMOVDQU 1376(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1408(CX), Y9
- VMOVDQU 1440(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1472(CX), Y9
- VMOVDQU 1504(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 3 to 8 outputs
- VMOVDQU (R8), Y11
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 1536(CX), Y9
- VMOVDQU 1568(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 1600(CX), Y9
- VMOVDQU 1632(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 1664(CX), Y9
- VMOVDQU 1696(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 1728(CX), Y9
- VMOVDQU 1760(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 1792(CX), Y9
- VMOVDQU 1824(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 1856(CX), Y9
- VMOVDQU 1888(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 1920(CX), Y9
- VMOVDQU 1952(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 1984(CX), Y9
- VMOVDQU 2016(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 4 to 8 outputs
- VMOVDQU (R9), Y11
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2048(CX), Y9
- VMOVDQU 2080(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2112(CX), Y9
- VMOVDQU 2144(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2176(CX), Y9
- VMOVDQU 2208(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2240(CX), Y9
- VMOVDQU 2272(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2304(CX), Y9
- VMOVDQU 2336(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2368(CX), Y9
- VMOVDQU 2400(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2432(CX), Y9
- VMOVDQU 2464(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 2496(CX), Y9
- VMOVDQU 2528(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 5 to 8 outputs
- VMOVDQU (R10), Y11
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 2560(CX), Y9
- VMOVDQU 2592(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 2624(CX), Y9
- VMOVDQU 2656(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 2688(CX), Y9
- VMOVDQU 2720(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 2752(CX), Y9
- VMOVDQU 2784(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 2816(CX), Y9
- VMOVDQU 2848(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 2880(CX), Y9
- VMOVDQU 2912(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 2944(CX), Y9
- VMOVDQU 2976(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3008(CX), Y9
- VMOVDQU 3040(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 6 to 8 outputs
- VMOVDQU (R11), Y11
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3072(CX), Y9
- VMOVDQU 3104(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3136(CX), Y9
- VMOVDQU 3168(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3200(CX), Y9
- VMOVDQU 3232(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3264(CX), Y9
- VMOVDQU 3296(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3328(CX), Y9
- VMOVDQU 3360(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3392(CX), Y9
- VMOVDQU 3424(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3456(CX), Y9
- VMOVDQU 3488(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 3520(CX), Y9
- VMOVDQU 3552(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 7 to 8 outputs
- VMOVDQU (R12), Y11
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 3584(CX), Y9
- VMOVDQU 3616(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 3648(CX), Y9
- VMOVDQU 3680(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 3712(CX), Y9
- VMOVDQU 3744(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 3776(CX), Y9
- VMOVDQU 3808(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 3840(CX), Y9
- VMOVDQU 3872(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 3904(CX), Y9
- VMOVDQU 3936(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 3968(CX), Y9
- VMOVDQU 4000(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4032(CX), Y9
- VMOVDQU 4064(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 8 to 8 outputs
- VMOVDQU (R13), Y11
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4096(CX), Y9
- VMOVDQU 4128(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4160(CX), Y9
- VMOVDQU 4192(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4224(CX), Y9
- VMOVDQU 4256(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4288(CX), Y9
- VMOVDQU 4320(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4352(CX), Y9
- VMOVDQU 4384(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4416(CX), Y9
- VMOVDQU 4448(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4480(CX), Y9
- VMOVDQU 4512(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 4544(CX), Y9
- VMOVDQU 4576(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Load and process 32 bytes from input 9 to 8 outputs
- VMOVDQU (DX), Y11
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y11, Y12
- VPAND Y8, Y11, Y11
- VPAND Y8, Y12, Y12
- VMOVDQU 4608(CX), Y9
- VMOVDQU 4640(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y0)
- VMOVDQU 4672(CX), Y9
- VMOVDQU 4704(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y1)
- VMOVDQU 4736(CX), Y9
- VMOVDQU 4768(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VMOVDQU 4800(CX), Y9
- VMOVDQU 4832(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y3)
- VMOVDQU 4864(CX), Y9
- VMOVDQU 4896(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU 4928(CX), Y9
- VMOVDQU 4960(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y5)
- VMOVDQU 4992(CX), Y9
- VMOVDQU 5024(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VMOVDQU 5056(CX), Y9
- VMOVDQU 5088(CX), Y10
- VPSHUFB Y11, Y9, Y9
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y9, Y10, Y7)
-
- // Store 8 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x8Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x8Xor_end:
- RET
-
-// func mulAvxTwo_10x9(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x9(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 194 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x9_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_10x9_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y0
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y1
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y2
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y3
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y4
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y5
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y6
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y7
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- VPXOR Y10, Y11, Y8
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (R12), Y12
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 8 to 9 outputs
- VMOVDQU (R13), Y12
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4608(CX), Y10
- VMOVDQU 4640(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4672(CX), Y10
- VMOVDQU 4704(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4736(CX), Y10
- VMOVDQU 4768(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4800(CX), Y10
- VMOVDQU 4832(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4864(CX), Y10
- VMOVDQU 4896(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4928(CX), Y10
- VMOVDQU 4960(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4992(CX), Y10
- VMOVDQU 5024(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5056(CX), Y10
- VMOVDQU 5088(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5120(CX), Y10
- VMOVDQU 5152(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 9 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 5184(CX), Y10
- VMOVDQU 5216(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 5248(CX), Y10
- VMOVDQU 5280(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 5312(CX), Y10
- VMOVDQU 5344(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 5376(CX), Y10
- VMOVDQU 5408(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 5440(CX), Y10
- VMOVDQU 5472(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 5504(CX), Y10
- VMOVDQU 5536(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 5568(CX), Y10
- VMOVDQU 5600(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5632(CX), Y10
- VMOVDQU 5664(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5696(CX), Y10
- VMOVDQU 5728(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
- MOVQ 192(R14), BP
- VMOVDQU Y8, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x9_loop
- VZEROUPPER
-
-mulAvxTwo_10x9_end:
- RET
-
-// func mulAvxTwo_10x9Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x9Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 194 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x9Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X9
- VPBROADCASTB X9, Y9
-
-mulAvxTwo_10x9Xor_loop:
- // Load and process 32 bytes from input 0 to 9 outputs
- VMOVDQU (BX), Y12
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y10
- VMOVDQU 32(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y10
- VMOVDQU 96(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y10
- VMOVDQU 160(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y10
- VMOVDQU 224(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y10
- VMOVDQU 288(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- MOVQ 120(R14), BP
- VMOVDQU (BP)(R15*1), Y5
- VMOVDQU 320(CX), Y10
- VMOVDQU 352(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- MOVQ 144(R14), BP
- VMOVDQU (BP)(R15*1), Y6
- VMOVDQU 384(CX), Y10
- VMOVDQU 416(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- MOVQ 168(R14), BP
- VMOVDQU (BP)(R15*1), Y7
- VMOVDQU 448(CX), Y10
- VMOVDQU 480(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- MOVQ 192(R14), BP
- VMOVDQU (BP)(R15*1), Y8
- VMOVDQU 512(CX), Y10
- VMOVDQU 544(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 1 to 9 outputs
- VMOVDQU (SI), Y12
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 576(CX), Y10
- VMOVDQU 608(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 640(CX), Y10
- VMOVDQU 672(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 704(CX), Y10
- VMOVDQU 736(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 768(CX), Y10
- VMOVDQU 800(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 832(CX), Y10
- VMOVDQU 864(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 896(CX), Y10
- VMOVDQU 928(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 960(CX), Y10
- VMOVDQU 992(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1024(CX), Y10
- VMOVDQU 1056(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1088(CX), Y10
- VMOVDQU 1120(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 2 to 9 outputs
- VMOVDQU (DI), Y12
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1152(CX), Y10
- VMOVDQU 1184(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1216(CX), Y10
- VMOVDQU 1248(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1280(CX), Y10
- VMOVDQU 1312(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1344(CX), Y10
- VMOVDQU 1376(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1408(CX), Y10
- VMOVDQU 1440(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 1472(CX), Y10
- VMOVDQU 1504(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 1536(CX), Y10
- VMOVDQU 1568(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 1600(CX), Y10
- VMOVDQU 1632(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 1664(CX), Y10
- VMOVDQU 1696(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 3 to 9 outputs
- VMOVDQU (R8), Y12
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 1728(CX), Y10
- VMOVDQU 1760(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 1792(CX), Y10
- VMOVDQU 1824(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 1856(CX), Y10
- VMOVDQU 1888(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 1920(CX), Y10
- VMOVDQU 1952(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 1984(CX), Y10
- VMOVDQU 2016(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2048(CX), Y10
- VMOVDQU 2080(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2112(CX), Y10
- VMOVDQU 2144(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2176(CX), Y10
- VMOVDQU 2208(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2240(CX), Y10
- VMOVDQU 2272(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 4 to 9 outputs
- VMOVDQU (R9), Y12
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2304(CX), Y10
- VMOVDQU 2336(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2368(CX), Y10
- VMOVDQU 2400(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 2432(CX), Y10
- VMOVDQU 2464(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 2496(CX), Y10
- VMOVDQU 2528(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 2560(CX), Y10
- VMOVDQU 2592(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 2624(CX), Y10
- VMOVDQU 2656(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 2688(CX), Y10
- VMOVDQU 2720(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 2752(CX), Y10
- VMOVDQU 2784(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 2816(CX), Y10
- VMOVDQU 2848(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 5 to 9 outputs
- VMOVDQU (R10), Y12
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 2880(CX), Y10
- VMOVDQU 2912(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 2944(CX), Y10
- VMOVDQU 2976(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3008(CX), Y10
- VMOVDQU 3040(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3072(CX), Y10
- VMOVDQU 3104(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3136(CX), Y10
- VMOVDQU 3168(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3200(CX), Y10
- VMOVDQU 3232(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3264(CX), Y10
- VMOVDQU 3296(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3328(CX), Y10
- VMOVDQU 3360(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3392(CX), Y10
- VMOVDQU 3424(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 6 to 9 outputs
- VMOVDQU (R11), Y12
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 3456(CX), Y10
- VMOVDQU 3488(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 3520(CX), Y10
- VMOVDQU 3552(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 3584(CX), Y10
- VMOVDQU 3616(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 3648(CX), Y10
- VMOVDQU 3680(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 3712(CX), Y10
- VMOVDQU 3744(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 3776(CX), Y10
- VMOVDQU 3808(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 3840(CX), Y10
- VMOVDQU 3872(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 3904(CX), Y10
- VMOVDQU 3936(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 3968(CX), Y10
- VMOVDQU 4000(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 7 to 9 outputs
- VMOVDQU (R12), Y12
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4032(CX), Y10
- VMOVDQU 4064(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4096(CX), Y10
- VMOVDQU 4128(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4160(CX), Y10
- VMOVDQU 4192(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4224(CX), Y10
- VMOVDQU 4256(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4288(CX), Y10
- VMOVDQU 4320(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4352(CX), Y10
- VMOVDQU 4384(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4416(CX), Y10
- VMOVDQU 4448(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 4480(CX), Y10
- VMOVDQU 4512(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 4544(CX), Y10
- VMOVDQU 4576(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 8 to 9 outputs
- VMOVDQU (R13), Y12
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 4608(CX), Y10
- VMOVDQU 4640(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 4672(CX), Y10
- VMOVDQU 4704(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 4736(CX), Y10
- VMOVDQU 4768(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 4800(CX), Y10
- VMOVDQU 4832(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 4864(CX), Y10
- VMOVDQU 4896(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 4928(CX), Y10
- VMOVDQU 4960(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 4992(CX), Y10
- VMOVDQU 5024(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5056(CX), Y10
- VMOVDQU 5088(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5120(CX), Y10
- VMOVDQU 5152(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Load and process 32 bytes from input 9 to 9 outputs
- VMOVDQU (DX), Y12
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y12, Y13
- VPAND Y9, Y12, Y12
- VPAND Y9, Y13, Y13
- VMOVDQU 5184(CX), Y10
- VMOVDQU 5216(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y0)
- VMOVDQU 5248(CX), Y10
- VMOVDQU 5280(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y1)
- VMOVDQU 5312(CX), Y10
- VMOVDQU 5344(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y2)
- VMOVDQU 5376(CX), Y10
- VMOVDQU 5408(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y3)
- VMOVDQU 5440(CX), Y10
- VMOVDQU 5472(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y4)
- VMOVDQU 5504(CX), Y10
- VMOVDQU 5536(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y5)
- VMOVDQU 5568(CX), Y10
- VMOVDQU 5600(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y6)
- VMOVDQU 5632(CX), Y10
- VMOVDQU 5664(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y7)
- VMOVDQU 5696(CX), Y10
- VMOVDQU 5728(CX), Y11
- VPSHUFB Y12, Y10, Y10
- VPSHUFB Y13, Y11, Y11
- XOR3WAY( $0x00, Y10, Y11, Y8)
-
- // Store 9 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
- MOVQ 192(R14), BP
- VMOVDQU Y8, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x9Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x9Xor_end:
- RET
-
-// func mulAvxTwo_10x10(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x10(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 215 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x10_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_10x10_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y0
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y1
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y2
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y3
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y4
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y5
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y6
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y7
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y8
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- VPXOR Y11, Y12, Y9
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (R12), Y13
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 8 to 10 outputs
- VMOVDQU (R13), Y13
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5120(CX), Y11
- VMOVDQU 5152(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5184(CX), Y11
- VMOVDQU 5216(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5248(CX), Y11
- VMOVDQU 5280(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5312(CX), Y11
- VMOVDQU 5344(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 5376(CX), Y11
- VMOVDQU 5408(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 5440(CX), Y11
- VMOVDQU 5472(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 5504(CX), Y11
- VMOVDQU 5536(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 5568(CX), Y11
- VMOVDQU 5600(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 5632(CX), Y11
- VMOVDQU 5664(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5696(CX), Y11
- VMOVDQU 5728(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 9 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5760(CX), Y11
- VMOVDQU 5792(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5824(CX), Y11
- VMOVDQU 5856(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5888(CX), Y11
- VMOVDQU 5920(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5952(CX), Y11
- VMOVDQU 5984(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 6016(CX), Y11
- VMOVDQU 6048(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 6080(CX), Y11
- VMOVDQU 6112(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 6144(CX), Y11
- VMOVDQU 6176(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 6208(CX), Y11
- VMOVDQU 6240(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 6272(CX), Y11
- VMOVDQU 6304(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 6336(CX), Y11
- VMOVDQU 6368(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
- MOVQ 192(R14), BP
- VMOVDQU Y8, (BP)(R15*1)
- MOVQ 216(R14), BP
- VMOVDQU Y9, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x10_loop
- VZEROUPPER
-
-mulAvxTwo_10x10_end:
- RET
-
-// func mulAvxTwo_10x10Xor(matrix []byte, in [][]byte, out [][]byte, start int, n int)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·mulAvxTwo_10x10Xor(SB), NOSPLIT, $8-88
- // Loading no tables to registers
- // Destination kept on stack
- // Full registers estimated 215 YMM used
- MOVQ n+80(FP), AX
- MOVQ matrix_base+0(FP), CX
- SHRQ $0x05, AX
- TESTQ AX, AX
- JZ mulAvxTwo_10x10Xor_end
- MOVQ in_base+24(FP), DX
- MOVQ (DX), BX
- MOVQ 24(DX), SI
- MOVQ 48(DX), DI
- MOVQ 72(DX), R8
- MOVQ 96(DX), R9
- MOVQ 120(DX), R10
- MOVQ 144(DX), R11
- MOVQ 168(DX), R12
- MOVQ 192(DX), R13
- MOVQ 216(DX), DX
- MOVQ out_base+48(FP), R14
- MOVQ start+72(FP), R15
-
- // Add start offset to input
- ADDQ R15, BX
- ADDQ R15, SI
- ADDQ R15, DI
- ADDQ R15, R8
- ADDQ R15, R9
- ADDQ R15, R10
- ADDQ R15, R11
- ADDQ R15, R12
- ADDQ R15, R13
- ADDQ R15, DX
- MOVQ $0x0000000f, BP
- MOVQ BP, X10
- VPBROADCASTB X10, Y10
-
-mulAvxTwo_10x10Xor_loop:
- // Load and process 32 bytes from input 0 to 10 outputs
- VMOVDQU (BX), Y13
- ADDQ $0x20, BX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- MOVQ (R14), BP
- VMOVDQU (BP)(R15*1), Y0
- VMOVDQU (CX), Y11
- VMOVDQU 32(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- MOVQ 24(R14), BP
- VMOVDQU (BP)(R15*1), Y1
- VMOVDQU 64(CX), Y11
- VMOVDQU 96(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- MOVQ 48(R14), BP
- VMOVDQU (BP)(R15*1), Y2
- VMOVDQU 128(CX), Y11
- VMOVDQU 160(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- MOVQ 72(R14), BP
- VMOVDQU (BP)(R15*1), Y3
- VMOVDQU 192(CX), Y11
- VMOVDQU 224(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- MOVQ 96(R14), BP
- VMOVDQU (BP)(R15*1), Y4
- VMOVDQU 256(CX), Y11
- VMOVDQU 288(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- MOVQ 120(R14), BP
- VMOVDQU (BP)(R15*1), Y5
- VMOVDQU 320(CX), Y11
- VMOVDQU 352(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- MOVQ 144(R14), BP
- VMOVDQU (BP)(R15*1), Y6
- VMOVDQU 384(CX), Y11
- VMOVDQU 416(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- MOVQ 168(R14), BP
- VMOVDQU (BP)(R15*1), Y7
- VMOVDQU 448(CX), Y11
- VMOVDQU 480(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- MOVQ 192(R14), BP
- VMOVDQU (BP)(R15*1), Y8
- VMOVDQU 512(CX), Y11
- VMOVDQU 544(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- MOVQ 216(R14), BP
- VMOVDQU (BP)(R15*1), Y9
- VMOVDQU 576(CX), Y11
- VMOVDQU 608(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 1 to 10 outputs
- VMOVDQU (SI), Y13
- ADDQ $0x20, SI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 640(CX), Y11
- VMOVDQU 672(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 704(CX), Y11
- VMOVDQU 736(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 768(CX), Y11
- VMOVDQU 800(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 832(CX), Y11
- VMOVDQU 864(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 896(CX), Y11
- VMOVDQU 928(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 960(CX), Y11
- VMOVDQU 992(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1024(CX), Y11
- VMOVDQU 1056(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1088(CX), Y11
- VMOVDQU 1120(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1152(CX), Y11
- VMOVDQU 1184(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1216(CX), Y11
- VMOVDQU 1248(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 2 to 10 outputs
- VMOVDQU (DI), Y13
- ADDQ $0x20, DI
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1280(CX), Y11
- VMOVDQU 1312(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1344(CX), Y11
- VMOVDQU 1376(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 1408(CX), Y11
- VMOVDQU 1440(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 1472(CX), Y11
- VMOVDQU 1504(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 1536(CX), Y11
- VMOVDQU 1568(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 1600(CX), Y11
- VMOVDQU 1632(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 1664(CX), Y11
- VMOVDQU 1696(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 1728(CX), Y11
- VMOVDQU 1760(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 1792(CX), Y11
- VMOVDQU 1824(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 1856(CX), Y11
- VMOVDQU 1888(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 3 to 10 outputs
- VMOVDQU (R8), Y13
- ADDQ $0x20, R8
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 1920(CX), Y11
- VMOVDQU 1952(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 1984(CX), Y11
- VMOVDQU 2016(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2048(CX), Y11
- VMOVDQU 2080(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2112(CX), Y11
- VMOVDQU 2144(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2176(CX), Y11
- VMOVDQU 2208(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2240(CX), Y11
- VMOVDQU 2272(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2304(CX), Y11
- VMOVDQU 2336(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 2368(CX), Y11
- VMOVDQU 2400(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 2432(CX), Y11
- VMOVDQU 2464(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 2496(CX), Y11
- VMOVDQU 2528(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 4 to 10 outputs
- VMOVDQU (R9), Y13
- ADDQ $0x20, R9
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 2560(CX), Y11
- VMOVDQU 2592(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 2624(CX), Y11
- VMOVDQU 2656(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 2688(CX), Y11
- VMOVDQU 2720(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 2752(CX), Y11
- VMOVDQU 2784(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 2816(CX), Y11
- VMOVDQU 2848(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 2880(CX), Y11
- VMOVDQU 2912(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 2944(CX), Y11
- VMOVDQU 2976(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3008(CX), Y11
- VMOVDQU 3040(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3072(CX), Y11
- VMOVDQU 3104(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3136(CX), Y11
- VMOVDQU 3168(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 5 to 10 outputs
- VMOVDQU (R10), Y13
- ADDQ $0x20, R10
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3200(CX), Y11
- VMOVDQU 3232(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3264(CX), Y11
- VMOVDQU 3296(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3328(CX), Y11
- VMOVDQU 3360(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 3392(CX), Y11
- VMOVDQU 3424(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 3456(CX), Y11
- VMOVDQU 3488(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 3520(CX), Y11
- VMOVDQU 3552(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 3584(CX), Y11
- VMOVDQU 3616(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 3648(CX), Y11
- VMOVDQU 3680(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 3712(CX), Y11
- VMOVDQU 3744(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 3776(CX), Y11
- VMOVDQU 3808(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 6 to 10 outputs
- VMOVDQU (R11), Y13
- ADDQ $0x20, R11
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 3840(CX), Y11
- VMOVDQU 3872(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 3904(CX), Y11
- VMOVDQU 3936(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 3968(CX), Y11
- VMOVDQU 4000(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4032(CX), Y11
- VMOVDQU 4064(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4096(CX), Y11
- VMOVDQU 4128(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4160(CX), Y11
- VMOVDQU 4192(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4224(CX), Y11
- VMOVDQU 4256(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4288(CX), Y11
- VMOVDQU 4320(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4352(CX), Y11
- VMOVDQU 4384(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 4416(CX), Y11
- VMOVDQU 4448(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 7 to 10 outputs
- VMOVDQU (R12), Y13
- ADDQ $0x20, R12
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 4480(CX), Y11
- VMOVDQU 4512(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 4544(CX), Y11
- VMOVDQU 4576(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 4608(CX), Y11
- VMOVDQU 4640(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 4672(CX), Y11
- VMOVDQU 4704(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 4736(CX), Y11
- VMOVDQU 4768(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 4800(CX), Y11
- VMOVDQU 4832(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 4864(CX), Y11
- VMOVDQU 4896(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 4928(CX), Y11
- VMOVDQU 4960(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 4992(CX), Y11
- VMOVDQU 5024(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5056(CX), Y11
- VMOVDQU 5088(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 8 to 10 outputs
- VMOVDQU (R13), Y13
- ADDQ $0x20, R13
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5120(CX), Y11
- VMOVDQU 5152(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5184(CX), Y11
- VMOVDQU 5216(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5248(CX), Y11
- VMOVDQU 5280(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5312(CX), Y11
- VMOVDQU 5344(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 5376(CX), Y11
- VMOVDQU 5408(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 5440(CX), Y11
- VMOVDQU 5472(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 5504(CX), Y11
- VMOVDQU 5536(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 5568(CX), Y11
- VMOVDQU 5600(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 5632(CX), Y11
- VMOVDQU 5664(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 5696(CX), Y11
- VMOVDQU 5728(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Load and process 32 bytes from input 9 to 10 outputs
- VMOVDQU (DX), Y13
- ADDQ $0x20, DX
- VPSRLQ $0x04, Y13, Y14
- VPAND Y10, Y13, Y13
- VPAND Y10, Y14, Y14
- VMOVDQU 5760(CX), Y11
- VMOVDQU 5792(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y0)
- VMOVDQU 5824(CX), Y11
- VMOVDQU 5856(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y1)
- VMOVDQU 5888(CX), Y11
- VMOVDQU 5920(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y2)
- VMOVDQU 5952(CX), Y11
- VMOVDQU 5984(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y3)
- VMOVDQU 6016(CX), Y11
- VMOVDQU 6048(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y4)
- VMOVDQU 6080(CX), Y11
- VMOVDQU 6112(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y5)
- VMOVDQU 6144(CX), Y11
- VMOVDQU 6176(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y6)
- VMOVDQU 6208(CX), Y11
- VMOVDQU 6240(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y7)
- VMOVDQU 6272(CX), Y11
- VMOVDQU 6304(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y8)
- VMOVDQU 6336(CX), Y11
- VMOVDQU 6368(CX), Y12
- VPSHUFB Y13, Y11, Y11
- VPSHUFB Y14, Y12, Y12
- XOR3WAY( $0x00, Y11, Y12, Y9)
-
- // Store 10 outputs
- MOVQ (R14), BP
- VMOVDQU Y0, (BP)(R15*1)
- MOVQ 24(R14), BP
- VMOVDQU Y1, (BP)(R15*1)
- MOVQ 48(R14), BP
- VMOVDQU Y2, (BP)(R15*1)
- MOVQ 72(R14), BP
- VMOVDQU Y3, (BP)(R15*1)
- MOVQ 96(R14), BP
- VMOVDQU Y4, (BP)(R15*1)
- MOVQ 120(R14), BP
- VMOVDQU Y5, (BP)(R15*1)
- MOVQ 144(R14), BP
- VMOVDQU Y6, (BP)(R15*1)
- MOVQ 168(R14), BP
- VMOVDQU Y7, (BP)(R15*1)
- MOVQ 192(R14), BP
- VMOVDQU Y8, (BP)(R15*1)
- MOVQ 216(R14), BP
- VMOVDQU Y9, (BP)(R15*1)
-
- // Prepare for next loop
- ADDQ $0x20, R15
- DECQ AX
- JNZ mulAvxTwo_10x10Xor_loop
- VZEROUPPER
-
-mulAvxTwo_10x10Xor_end:
- RET
-
-// func ifftDIT2_avx2(x []byte, y []byte, table *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT2_avx2(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- VBROADCASTI128 (AX), Y0
- VBROADCASTI128 64(AX), Y1
- VBROADCASTI128 16(AX), Y2
- VBROADCASTI128 80(AX), Y3
- VBROADCASTI128 32(AX), Y4
- VBROADCASTI128 96(AX), Y5
- VBROADCASTI128 48(AX), Y6
- VBROADCASTI128 112(AX), Y7
- MOVQ x_len+8(FP), AX
- MOVQ x_base+0(FP), CX
- MOVQ y_base+24(FP), DX
- MOVQ $0x0000000f, BX
- MOVQ BX, X8
- VPBROADCASTB X8, Y8
-
-loop:
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y12
- VPXOR Y11, Y9, Y11
- VPXOR Y12, Y10, Y12
- VMOVDQU Y11, (DX)
- VMOVDQU Y12, 32(DX)
- VPSRLQ $0x04, Y11, Y13
- VPAND Y8, Y11, Y11
- VPAND Y8, Y13, Y13
- VPSHUFB Y11, Y0, Y14
- VPSHUFB Y11, Y1, Y11
- VPSHUFB Y13, Y2, Y15
- VPSHUFB Y13, Y3, Y13
- VPXOR Y14, Y15, Y14
- VPXOR Y11, Y13, Y11
- VPAND Y12, Y8, Y13
- VPSRLQ $0x04, Y12, Y12
- VPAND Y8, Y12, Y12
- VPSHUFB Y13, Y4, Y15
- VPSHUFB Y13, Y5, Y13
- VPXOR Y14, Y15, Y14
- VPXOR Y11, Y13, Y11
- VPSHUFB Y12, Y6, Y15
- VPSHUFB Y12, Y7, Y13
- XOR3WAY( $0x00, Y14, Y15, Y9)
- XOR3WAY( $0x00, Y11, Y13, Y10)
- VMOVDQU Y9, (CX)
- VMOVDQU Y10, 32(CX)
- ADDQ $0x40, CX
- ADDQ $0x40, DX
- SUBQ $0x40, AX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT2_avx2(x []byte, y []byte, table *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT2_avx2(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- VBROADCASTI128 (AX), Y0
- VBROADCASTI128 64(AX), Y1
- VBROADCASTI128 16(AX), Y2
- VBROADCASTI128 80(AX), Y3
- VBROADCASTI128 32(AX), Y4
- VBROADCASTI128 96(AX), Y5
- VBROADCASTI128 48(AX), Y6
- VBROADCASTI128 112(AX), Y7
- MOVQ x_len+8(FP), AX
- MOVQ x_base+0(FP), CX
- MOVQ y_base+24(FP), DX
- MOVQ $0x0000000f, BX
- MOVQ BX, X8
- VPBROADCASTB X8, Y8
-
-loop:
- VMOVDQU (CX), Y9
- VMOVDQU 32(CX), Y10
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y12
- VPSRLQ $0x04, Y11, Y13
- VPAND Y8, Y11, Y11
- VPAND Y8, Y13, Y13
- VPSHUFB Y11, Y0, Y14
- VPSHUFB Y11, Y1, Y11
- VPSHUFB Y13, Y2, Y15
- VPSHUFB Y13, Y3, Y13
- VPXOR Y14, Y15, Y14
- VPXOR Y11, Y13, Y11
- VPAND Y12, Y8, Y13
- VPSRLQ $0x04, Y12, Y12
- VPAND Y8, Y12, Y12
- VPSHUFB Y13, Y4, Y15
- VPSHUFB Y13, Y5, Y13
- VPXOR Y14, Y15, Y14
- VPXOR Y11, Y13, Y11
- VPSHUFB Y12, Y6, Y15
- VPSHUFB Y12, Y7, Y13
- XOR3WAY( $0x00, Y14, Y15, Y9)
- XOR3WAY( $0x00, Y11, Y13, Y10)
- VMOVDQU Y9, (CX)
- VMOVDQU Y10, 32(CX)
- VMOVDQU (DX), Y11
- VMOVDQU 32(DX), Y12
- VPXOR Y11, Y9, Y11
- VPXOR Y12, Y10, Y12
- VMOVDQU Y11, (DX)
- VMOVDQU Y12, 32(DX)
- ADDQ $0x40, CX
- ADDQ $0x40, DX
- SUBQ $0x40, AX
- JNZ loop
- VZEROUPPER
- RET
-
-// func mulgf16_avx2(x []byte, y []byte, table *[128]uint8)
-// Requires: AVX, AVX2, SSE2
-TEXT ·mulgf16_avx2(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- VBROADCASTI128 (AX), Y0
- VBROADCASTI128 64(AX), Y1
- VBROADCASTI128 16(AX), Y2
- VBROADCASTI128 80(AX), Y3
- VBROADCASTI128 32(AX), Y4
- VBROADCASTI128 96(AX), Y5
- VBROADCASTI128 48(AX), Y6
- VBROADCASTI128 112(AX), Y7
- MOVQ x_len+8(FP), AX
- MOVQ x_base+0(FP), CX
- MOVQ y_base+24(FP), DX
- MOVQ $0x0000000f, BX
- MOVQ BX, X8
- VPBROADCASTB X8, Y8
-
-loop:
- VMOVDQU (DX), Y9
- VMOVDQU 32(DX), Y10
- VPSRLQ $0x04, Y9, Y11
- VPAND Y8, Y9, Y9
- VPAND Y8, Y11, Y11
- VPSHUFB Y9, Y0, Y12
- VPSHUFB Y9, Y1, Y9
- VPSHUFB Y11, Y2, Y13
- VPSHUFB Y11, Y3, Y11
- VPXOR Y12, Y13, Y12
- VPXOR Y9, Y11, Y9
- VPAND Y10, Y8, Y11
- VPSRLQ $0x04, Y10, Y10
- VPAND Y8, Y10, Y10
- VPSHUFB Y11, Y4, Y13
- VPSHUFB Y11, Y5, Y11
- VPXOR Y12, Y13, Y12
- VPXOR Y9, Y11, Y9
- VPSHUFB Y10, Y6, Y13
- VPSHUFB Y10, Y7, Y11
- VPXOR Y12, Y13, Y12
- VPXOR Y9, Y11, Y9
- VMOVDQU Y12, (CX)
- VMOVDQU Y9, 32(CX)
- ADDQ $0x40, CX
- ADDQ $0x40, DX
- SUBQ $0x40, AX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_0(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- VBROADCASTI128 (DX), Y1
- VBROADCASTI128 64(DX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(DX), Y1
- VBROADCASTI128 80(DX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(DX), Y1
- VBROADCASTI128 96(DX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(DX), Y1
- VBROADCASTI128 112(DX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ AX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ AX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ AX, SI
- MOVQ (DX)(SI*1), AX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VPSHUFB Y5, Y24, Y7
- VPSHUFB Y5, Y25, Y5
- VPSHUFB Y6, Y26, Y8
- VPSHUFB Y6, Y27, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VPSHUFB Y6, Y28, Y9
- VPSHUFB Y6, Y29, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VPSHUFB Y8, Y30, Y9
- VPSHUFB Y8, Y31, Y6
- VPTERNLOGD $0x96, Y7, Y9, Y1
- VPTERNLOGD $0x96, Y5, Y6, Y2
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y5
- VPTERNLOGD $0x96, Y9, Y10, Y6
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_0(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- VBROADCASTI128 (DX), Y1
- VBROADCASTI128 64(DX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(DX), Y1
- VBROADCASTI128 80(DX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(DX), Y1
- VBROADCASTI128 96(DX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(DX), Y1
- VBROADCASTI128 112(DX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ AX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ AX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ AX, SI
- MOVQ (DX)(SI*1), AX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y24, Y11
- VPSHUFB Y9, Y25, Y9
- VPSHUFB Y10, Y26, Y12
- VPSHUFB Y10, Y27, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y28, Y13
- VPSHUFB Y10, Y29, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y30, Y13
- VPSHUFB Y12, Y31, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VBROADCASTI128 (CX), Y3
- VBROADCASTI128 64(CX), Y4
- VPSHUFB Y1, Y3, Y3
- VPSHUFB Y1, Y4, Y1
- VBROADCASTI128 16(CX), Y4
- VBROADCASTI128 80(CX), Y9
- VPSHUFB Y2, Y4, Y4
- VPSHUFB Y2, Y9, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VBROADCASTI128 32(CX), Y9
- VBROADCASTI128 96(CX), Y10
- VPSHUFB Y2, Y9, Y9
- VPSHUFB Y2, Y10, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VBROADCASTI128 48(CX), Y9
- VBROADCASTI128 112(CX), Y2
- VPSHUFB Y4, Y9, Y9
- VPSHUFB Y4, Y2, Y2
- VPTERNLOGD $0x96, Y3, Y9, Y5
- VPTERNLOGD $0x96, Y1, Y2, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_1(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y24, Y11
- VPSHUFB Y9, Y25, Y9
- VPSHUFB Y10, Y26, Y12
- VPSHUFB Y10, Y27, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y28, Y13
- VPSHUFB Y10, Y29, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y30, Y13
- VPSHUFB Y12, Y31, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y5
- VPTERNLOGD $0x96, Y9, Y10, Y6
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_1(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VPSHUFB Y1, Y24, Y3
- VPSHUFB Y1, Y25, Y1
- VPSHUFB Y2, Y26, Y4
- VPSHUFB Y2, Y27, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VPSHUFB Y2, Y28, Y9
- VPSHUFB Y2, Y29, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VPSHUFB Y4, Y30, Y9
- VPSHUFB Y4, Y31, Y2
- VPTERNLOGD $0x96, Y3, Y9, Y5
- VPTERNLOGD $0x96, Y1, Y2, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_2(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VPSHUFB Y5, Y24, Y7
- VPSHUFB Y5, Y25, Y5
- VPSHUFB Y6, Y26, Y8
- VPSHUFB Y6, Y27, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VPSHUFB Y6, Y28, Y9
- VPSHUFB Y6, Y29, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VPSHUFB Y8, Y30, Y9
- VPSHUFB Y8, Y31, Y6
- VPTERNLOGD $0x96, Y7, Y9, Y1
- VPTERNLOGD $0x96, Y5, Y6, Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_2(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VPSHUFB Y1, Y24, Y3
- VPSHUFB Y1, Y25, Y1
- VPSHUFB Y2, Y26, Y4
- VPSHUFB Y2, Y27, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VPSHUFB Y2, Y28, Y9
- VPSHUFB Y2, Y29, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VPSHUFB Y4, Y30, Y9
- VPSHUFB Y4, Y31, Y2
- VPTERNLOGD $0x96, Y3, Y9, Y5
- VPTERNLOGD $0x96, Y1, Y2, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_3(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_3(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VPSHUFB Y1, Y16, Y3
- VPSHUFB Y1, Y17, Y1
- VPSHUFB Y2, Y18, Y4
- VPSHUFB Y2, Y19, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VPSHUFB Y2, Y20, Y9
- VPSHUFB Y2, Y21, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VPSHUFB Y4, Y22, Y9
- VPSHUFB Y4, Y23, Y2
- VPTERNLOGD $0x96, Y3, Y9, Y5
- VPTERNLOGD $0x96, Y1, Y2, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_4(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VPSHUFB Y5, Y16, Y7
- VPSHUFB Y5, Y17, Y5
- VPSHUFB Y6, Y18, Y8
- VPSHUFB Y6, Y19, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VPSHUFB Y6, Y20, Y9
- VPSHUFB Y6, Y21, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VPSHUFB Y8, Y22, Y9
- VPSHUFB Y8, Y23, Y6
- VPTERNLOGD $0x96, Y7, Y9, Y1
- VPTERNLOGD $0x96, Y5, Y6, Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y24, Y11
- VPSHUFB Y9, Y25, Y9
- VPSHUFB Y10, Y26, Y12
- VPSHUFB Y10, Y27, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y28, Y13
- VPSHUFB Y10, Y29, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y30, Y13
- VPSHUFB Y12, Y31, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y5
- VPTERNLOGD $0x96, Y9, Y10, Y6
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_4(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (CX), Y1
- VBROADCASTI128 64(CX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(CX), Y1
- VBROADCASTI128 80(CX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(CX), Y1
- VBROADCASTI128 96(CX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(CX), Y1
- VBROADCASTI128 112(CX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z24
- VMOVAPS Z0, Z25
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z26
- VMOVAPS Z0, Z27
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z28
- VMOVAPS Z0, Z29
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z30
- VMOVAPS Z0, Z31
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y24, Y11
- VPSHUFB Y9, Y25, Y9
- VPSHUFB Y10, Y26, Y12
- VPSHUFB Y10, Y27, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y28, Y13
- VPSHUFB Y10, Y29, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y30, Y13
- VPSHUFB Y12, Y31, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_5(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y5
- VPTERNLOGD $0x96, Y9, Y10, Y6
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_5(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx512_6(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VPSHUFB Y5, Y16, Y7
- VPSHUFB Y5, Y17, Y5
- VPSHUFB Y6, Y18, Y8
- VPSHUFB Y6, Y19, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VPSHUFB Y6, Y20, Y9
- VPSHUFB Y6, Y21, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VPSHUFB Y8, Y22, Y9
- VPSHUFB Y8, Y23, Y6
- VPTERNLOGD $0x96, Y7, Y9, Y1
- VPTERNLOGD $0x96, Y5, Y6, Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx512_6(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- VBROADCASTI128 (AX), Y1
- VBROADCASTI128 64(AX), Y0
- VMOVAPS Z1, Z16
- VMOVAPS Z0, Z17
- VBROADCASTI128 16(AX), Y1
- VBROADCASTI128 80(AX), Y0
- VMOVAPS Z1, Z18
- VMOVAPS Z0, Z19
- VBROADCASTI128 32(AX), Y1
- VBROADCASTI128 96(AX), Y0
- VMOVAPS Z1, Z20
- VMOVAPS Z0, Z21
- VBROADCASTI128 48(AX), Y1
- VBROADCASTI128 112(AX), Y0
- VMOVAPS Z1, Z22
- VMOVAPS Z0, Z23
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y1
- VMOVDQU 32(SI), Y2
- VMOVDQU (R8), Y5
- VMOVDQU 32(R8), Y6
- VMOVDQU (DI), Y3
- VMOVDQU 32(DI), Y4
- VMOVDQU (AX), Y7
- VMOVDQU 32(AX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y1
- VPTERNLOGD $0x96, Y9, Y10, Y2
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VPSHUFB Y9, Y16, Y11
- VPSHUFB Y9, Y17, Y9
- VPSHUFB Y10, Y18, Y12
- VPSHUFB Y10, Y19, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VPSHUFB Y10, Y20, Y13
- VPSHUFB Y10, Y21, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VPSHUFB Y12, Y22, Y13
- VPSHUFB Y12, Y23, Y10
- VPTERNLOGD $0x96, Y11, Y13, Y3
- VPTERNLOGD $0x96, Y9, Y10, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (SI)
- VMOVDQU Y2, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y3, (DI)
- VMOVDQU Y4, 32(DI)
- ADDQ $0x40, DI
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R8)
- VMOVDQU Y6, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y7, (AX)
- VMOVDQU Y8, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx512_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, SSE2
-TEXT ·ifftDIT4_avx512_7(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
- VPXOR Y0, Y2, Y2
- VPXOR Y1, Y3, Y3
- VMOVDQU (R8), Y4
- VMOVDQU 32(R8), Y5
- VMOVDQU (AX), Y6
- VMOVDQU 32(AX), Y7
- VPXOR Y4, Y6, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y0, Y4, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y6, (AX)
- VMOVDQU Y7, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx512_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, SSE2
-TEXT ·fftDIT4_avx512_7(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (R8), Y4
- VMOVDQU 32(R8), Y5
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
- VMOVDQU (AX), Y6
- VMOVDQU 32(AX), Y7
- VPXOR Y0, Y4, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y0, Y2, Y2
- VPXOR Y1, Y3, Y3
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VPXOR Y4, Y6, Y6
- VPXOR Y5, Y7, Y7
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y6, (AX)
- VMOVDQU Y7, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_0(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- MOVQ $0x0000000f, BX
- MOVQ BX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), BX
- MOVQ work_base+0(FP), SI
- MOVQ 8(SI), DI
- XORQ R8, R8
- MOVQ (SI)(R8*1), R9
- ADDQ BX, R8
- MOVQ (SI)(R8*1), R10
- ADDQ BX, R8
- MOVQ (SI)(R8*1), R11
- ADDQ BX, R8
- MOVQ (SI)(R8*1), BX
-
-loop:
- VMOVDQU (R9), Y1
- VMOVDQU 32(R9), Y2
- VMOVDQU (R10), Y3
- VMOVDQU 32(R10), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VBROADCASTI128 (AX), Y7
- VBROADCASTI128 64(AX), Y8
- VPSHUFB Y5, Y7, Y7
- VPSHUFB Y5, Y8, Y5
- VBROADCASTI128 16(AX), Y8
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y6, Y8, Y8
- VPSHUFB Y6, Y9, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y6, Y9, Y9
- VPSHUFB Y6, Y10, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y6
- VPSHUFB Y8, Y9, Y9
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y7, Y9, Y1)
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R11), Y5
- VMOVDQU 32(R11), Y6
- VMOVDQU (BX), Y7
- VMOVDQU 32(BX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y5)
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (DX), Y11
- VBROADCASTI128 64(DX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(DX), Y12
- VBROADCASTI128 80(DX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(DX), Y13
- VBROADCASTI128 96(DX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(DX), Y13
- VBROADCASTI128 112(DX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (DX), Y11
- VBROADCASTI128 64(DX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(DX), Y12
- VBROADCASTI128 80(DX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(DX), Y13
- VBROADCASTI128 96(DX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(DX), Y13
- VBROADCASTI128 112(DX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU Y1, (R9)
- VMOVDQU Y2, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y3, (R10)
- VMOVDQU Y4, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y5, (R11)
- VMOVDQU Y6, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y7, (BX)
- VMOVDQU Y8, 32(BX)
- ADDQ $0x40, BX
- SUBQ $0x40, DI
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_0(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_0(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- MOVQ $0x0000000f, BX
- MOVQ BX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), BX
- MOVQ work_base+0(FP), SI
- MOVQ 8(SI), DI
- XORQ R8, R8
- MOVQ (SI)(R8*1), R9
- ADDQ BX, R8
- MOVQ (SI)(R8*1), R10
- ADDQ BX, R8
- MOVQ (SI)(R8*1), R11
- ADDQ BX, R8
- MOVQ (SI)(R8*1), BX
-
-loop:
- VMOVDQU (R9), Y1
- VMOVDQU 32(R9), Y2
- VMOVDQU (R11), Y5
- VMOVDQU 32(R11), Y6
- VMOVDQU (R10), Y3
- VMOVDQU 32(R10), Y4
- VMOVDQU (BX), Y7
- VMOVDQU 32(BX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (DX), Y11
- VBROADCASTI128 64(DX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(DX), Y12
- VBROADCASTI128 80(DX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(DX), Y13
- VBROADCASTI128 96(DX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(DX), Y13
- VBROADCASTI128 112(DX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (DX), Y11
- VBROADCASTI128 64(DX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(DX), Y12
- VBROADCASTI128 80(DX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(DX), Y13
- VBROADCASTI128 96(DX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(DX), Y13
- VBROADCASTI128 112(DX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (R9)
- VMOVDQU Y2, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y3, (R10)
- VMOVDQU Y4, 32(R10)
- ADDQ $0x40, R10
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VBROADCASTI128 (CX), Y3
- VBROADCASTI128 64(CX), Y4
- VPSHUFB Y1, Y3, Y3
- VPSHUFB Y1, Y4, Y1
- VBROADCASTI128 16(CX), Y4
- VBROADCASTI128 80(CX), Y9
- VPSHUFB Y2, Y4, Y4
- VPSHUFB Y2, Y9, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VBROADCASTI128 32(CX), Y9
- VBROADCASTI128 96(CX), Y10
- VPSHUFB Y2, Y9, Y9
- VPSHUFB Y2, Y10, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VBROADCASTI128 48(CX), Y9
- VBROADCASTI128 112(CX), Y2
- VPSHUFB Y4, Y9, Y9
- VPSHUFB Y4, Y2, Y2
- XOR3WAY( $0x00, Y3, Y9, Y5)
- XOR3WAY( $0x00, Y1, Y2, Y6)
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R11)
- VMOVDQU Y6, 32(R11)
- ADDQ $0x40, R11
- VMOVDQU Y7, (BX)
- VMOVDQU Y8, 32(BX)
- ADDQ $0x40, BX
- SUBQ $0x40, DI
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_1(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y5)
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_1(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_1(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VBROADCASTI128 (CX), Y3
- VBROADCASTI128 64(CX), Y4
- VPSHUFB Y1, Y3, Y3
- VPSHUFB Y1, Y4, Y1
- VBROADCASTI128 16(CX), Y4
- VBROADCASTI128 80(CX), Y9
- VPSHUFB Y2, Y4, Y4
- VPSHUFB Y2, Y9, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VBROADCASTI128 32(CX), Y9
- VBROADCASTI128 96(CX), Y10
- VPSHUFB Y2, Y9, Y9
- VPSHUFB Y2, Y10, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VBROADCASTI128 48(CX), Y9
- VBROADCASTI128 112(CX), Y2
- VPSHUFB Y4, Y9, Y9
- VPSHUFB Y4, Y2, Y2
- XOR3WAY( $0x00, Y3, Y9, Y5)
- XOR3WAY( $0x00, Y1, Y2, Y6)
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_2(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VBROADCASTI128 (AX), Y7
- VBROADCASTI128 64(AX), Y8
- VPSHUFB Y5, Y7, Y7
- VPSHUFB Y5, Y8, Y5
- VBROADCASTI128 16(AX), Y8
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y6, Y8, Y8
- VPSHUFB Y6, Y9, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y6, Y9, Y9
- VPSHUFB Y6, Y10, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y6
- VPSHUFB Y8, Y9, Y9
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y7, Y9, Y1)
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_2(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_2(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VBROADCASTI128 (AX), Y3
- VBROADCASTI128 64(AX), Y4
- VPSHUFB Y1, Y3, Y3
- VPSHUFB Y1, Y4, Y1
- VBROADCASTI128 16(AX), Y4
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y2, Y4, Y4
- VPSHUFB Y2, Y9, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y2, Y9, Y9
- VPSHUFB Y2, Y10, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y2
- VPSHUFB Y4, Y9, Y9
- VPSHUFB Y4, Y2, Y2
- XOR3WAY( $0x00, Y3, Y9, Y5)
- XOR3WAY( $0x00, Y1, Y2, Y6)
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_3(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_3(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_3(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VPSRLQ $0x04, Y7, Y2
- VPAND Y0, Y7, Y1
- VPAND Y0, Y2, Y2
- VBROADCASTI128 (AX), Y3
- VBROADCASTI128 64(AX), Y4
- VPSHUFB Y1, Y3, Y3
- VPSHUFB Y1, Y4, Y1
- VBROADCASTI128 16(AX), Y4
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y2, Y4, Y4
- VPSHUFB Y2, Y9, Y2
- VPXOR Y3, Y4, Y3
- VPXOR Y1, Y2, Y1
- VPAND Y8, Y0, Y2
- VPSRLQ $0x04, Y8, Y4
- VPAND Y0, Y4, Y4
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y2, Y9, Y9
- VPSHUFB Y2, Y10, Y2
- VPXOR Y3, Y9, Y3
- VPXOR Y1, Y2, Y1
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y2
- VPSHUFB Y4, Y9, Y9
- VPSHUFB Y4, Y2, Y2
- XOR3WAY( $0x00, Y3, Y9, Y5)
- XOR3WAY( $0x00, Y1, Y2, Y6)
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_4(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), DX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VBROADCASTI128 (AX), Y7
- VBROADCASTI128 64(AX), Y8
- VPSHUFB Y5, Y7, Y7
- VPSHUFB Y5, Y8, Y5
- VBROADCASTI128 16(AX), Y8
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y6, Y8, Y8
- VPSHUFB Y6, Y9, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y6, Y9, Y9
- VPSHUFB Y6, Y10, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y6
- VPSHUFB Y8, Y9, Y9
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y7, Y9, Y1)
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y5)
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_4(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_4(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, DX
- MOVQ DX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), DX
- MOVQ work_base+0(FP), BX
- MOVQ 8(BX), SI
- XORQ DI, DI
- MOVQ (BX)(DI*1), R8
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R9
- ADDQ DX, DI
- MOVQ (BX)(DI*1), R10
- ADDQ DX, DI
- MOVQ (BX)(DI*1), DX
-
-loop:
- VMOVDQU (R8), Y1
- VMOVDQU 32(R8), Y2
- VMOVDQU (R10), Y5
- VMOVDQU 32(R10), Y6
- VMOVDQU (R9), Y3
- VMOVDQU 32(R9), Y4
- VMOVDQU (DX), Y7
- VMOVDQU 32(DX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (CX), Y11
- VBROADCASTI128 64(CX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(CX), Y12
- VBROADCASTI128 80(CX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(CX), Y13
- VBROADCASTI128 96(CX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(CX), Y13
- VBROADCASTI128 112(CX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (R8)
- VMOVDQU Y2, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y3, (R9)
- VMOVDQU Y4, 32(R9)
- ADDQ $0x40, R9
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R10)
- VMOVDQU Y6, 32(R10)
- ADDQ $0x40, R10
- VMOVDQU Y7, (DX)
- VMOVDQU Y8, 32(DX)
- ADDQ $0x40, DX
- SUBQ $0x40, SI
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_5(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y5)
- XOR3WAY( $0x00, Y9, Y10, Y6)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_5(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_5(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPSRLQ $0x04, Y3, Y10
- VPAND Y0, Y3, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y4, Y0, Y10
- VPSRLQ $0x04, Y4, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·ifftDIT4_avx2_6(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), CX
- MOVQ table02+48(FP), CX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VPSRLQ $0x04, Y3, Y6
- VPAND Y0, Y3, Y5
- VPAND Y0, Y6, Y6
- VBROADCASTI128 (AX), Y7
- VBROADCASTI128 64(AX), Y8
- VPSHUFB Y5, Y7, Y7
- VPSHUFB Y5, Y8, Y5
- VBROADCASTI128 16(AX), Y8
- VBROADCASTI128 80(AX), Y9
- VPSHUFB Y6, Y8, Y8
- VPSHUFB Y6, Y9, Y6
- VPXOR Y7, Y8, Y7
- VPXOR Y5, Y6, Y5
- VPAND Y4, Y0, Y6
- VPSRLQ $0x04, Y4, Y8
- VPAND Y0, Y8, Y8
- VBROADCASTI128 32(AX), Y9
- VBROADCASTI128 96(AX), Y10
- VPSHUFB Y6, Y9, Y9
- VPSHUFB Y6, Y10, Y6
- VPXOR Y7, Y9, Y7
- VPXOR Y5, Y6, Y5
- VBROADCASTI128 48(AX), Y9
- VBROADCASTI128 112(AX), Y6
- VPSHUFB Y8, Y9, Y9
- VPSHUFB Y8, Y6, Y6
- XOR3WAY( $0x00, Y7, Y9, Y1)
- XOR3WAY( $0x00, Y5, Y6, Y2)
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_6(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, AVX512F, AVX512VL, SSE2
-TEXT ·fftDIT4_avx2_6(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, CX
- MOVQ CX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), CX
- MOVQ work_base+0(FP), DX
- MOVQ 8(DX), BX
- XORQ SI, SI
- MOVQ (DX)(SI*1), DI
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R8
- ADDQ CX, SI
- MOVQ (DX)(SI*1), R9
- ADDQ CX, SI
- MOVQ (DX)(SI*1), CX
-
-loop:
- VMOVDQU (DI), Y1
- VMOVDQU 32(DI), Y2
- VMOVDQU (R9), Y5
- VMOVDQU 32(R9), Y6
- VMOVDQU (R8), Y3
- VMOVDQU 32(R8), Y4
- VMOVDQU (CX), Y7
- VMOVDQU 32(CX), Y8
- VPSRLQ $0x04, Y5, Y10
- VPAND Y0, Y5, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y6, Y0, Y10
- VPSRLQ $0x04, Y6, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y1)
- XOR3WAY( $0x00, Y9, Y10, Y2)
- VPSRLQ $0x04, Y7, Y10
- VPAND Y0, Y7, Y9
- VPAND Y0, Y10, Y10
- VBROADCASTI128 (AX), Y11
- VBROADCASTI128 64(AX), Y12
- VPSHUFB Y9, Y11, Y11
- VPSHUFB Y9, Y12, Y9
- VBROADCASTI128 16(AX), Y12
- VBROADCASTI128 80(AX), Y13
- VPSHUFB Y10, Y12, Y12
- VPSHUFB Y10, Y13, Y10
- VPXOR Y11, Y12, Y11
- VPXOR Y9, Y10, Y9
- VPAND Y8, Y0, Y10
- VPSRLQ $0x04, Y8, Y12
- VPAND Y0, Y12, Y12
- VBROADCASTI128 32(AX), Y13
- VBROADCASTI128 96(AX), Y14
- VPSHUFB Y10, Y13, Y13
- VPSHUFB Y10, Y14, Y10
- VPXOR Y11, Y13, Y11
- VPXOR Y9, Y10, Y9
- VBROADCASTI128 48(AX), Y13
- VBROADCASTI128 112(AX), Y10
- VPSHUFB Y12, Y13, Y13
- VPSHUFB Y12, Y10, Y10
- XOR3WAY( $0x00, Y11, Y13, Y3)
- XOR3WAY( $0x00, Y9, Y10, Y4)
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y4, Y8, Y8
- VPXOR Y1, Y3, Y3
- VPXOR Y2, Y4, Y4
- VMOVDQU Y1, (DI)
- VMOVDQU Y2, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y3, (R8)
- VMOVDQU Y4, 32(R8)
- ADDQ $0x40, R8
- VPXOR Y5, Y7, Y7
- VPXOR Y6, Y8, Y8
- VMOVDQU Y5, (R9)
- VMOVDQU Y6, 32(R9)
- ADDQ $0x40, R9
- VMOVDQU Y7, (CX)
- VMOVDQU Y8, 32(CX)
- ADDQ $0x40, CX
- SUBQ $0x40, BX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT4_avx2_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, SSE2
-TEXT ·ifftDIT4_avx2_7(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
- VPXOR Y0, Y2, Y2
- VPXOR Y1, Y3, Y3
- VMOVDQU (R8), Y4
- VMOVDQU 32(R8), Y5
- VMOVDQU (AX), Y6
- VMOVDQU 32(AX), Y7
- VPXOR Y4, Y6, Y6
- VPXOR Y5, Y7, Y7
- VPXOR Y0, Y4, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y6, (AX)
- VMOVDQU Y7, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func fftDIT4_avx2_7(work [][]byte, dist int, table01 *[128]uint8, table23 *[128]uint8, table02 *[128]uint8)
-// Requires: AVX, AVX2, SSE2
-TEXT ·fftDIT4_avx2_7(SB), NOSPLIT, $0-56
- // dist must be multiplied by 24 (size of slice header)
- MOVQ table01+32(FP), AX
- MOVQ table23+40(FP), AX
- MOVQ table02+48(FP), AX
- MOVQ $0x0000000f, AX
- MOVQ AX, X0
- VPBROADCASTB X0, Y0
- MOVQ dist+24(FP), AX
- MOVQ work_base+0(FP), CX
- MOVQ 8(CX), DX
- XORQ BX, BX
- MOVQ (CX)(BX*1), SI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), DI
- ADDQ AX, BX
- MOVQ (CX)(BX*1), R8
- ADDQ AX, BX
- MOVQ (CX)(BX*1), AX
-
-loop:
- VMOVDQU (SI), Y0
- VMOVDQU 32(SI), Y1
- VMOVDQU (R8), Y4
- VMOVDQU 32(R8), Y5
- VMOVDQU (DI), Y2
- VMOVDQU 32(DI), Y3
- VMOVDQU (AX), Y6
- VMOVDQU 32(AX), Y7
- VPXOR Y0, Y4, Y4
- VPXOR Y1, Y5, Y5
- VPXOR Y2, Y6, Y6
- VPXOR Y3, Y7, Y7
- VPXOR Y0, Y2, Y2
- VPXOR Y1, Y3, Y3
- VMOVDQU Y0, (SI)
- VMOVDQU Y1, 32(SI)
- ADDQ $0x40, SI
- VMOVDQU Y2, (DI)
- VMOVDQU Y3, 32(DI)
- ADDQ $0x40, DI
- VPXOR Y4, Y6, Y6
- VPXOR Y5, Y7, Y7
- VMOVDQU Y4, (R8)
- VMOVDQU Y5, 32(R8)
- ADDQ $0x40, R8
- VMOVDQU Y6, (AX)
- VMOVDQU Y7, 32(AX)
- ADDQ $0x40, AX
- SUBQ $0x40, DX
- JNZ loop
- VZEROUPPER
- RET
-
-// func ifftDIT2_ssse3(x []byte, y []byte, table *[128]uint8)
-// Requires: SSE, SSE2, SSSE3
-TEXT ·ifftDIT2_ssse3(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- MOVUPS (AX), X0
- MOVUPS 64(AX), X1
- MOVUPS 16(AX), X2
- MOVUPS 80(AX), X3
- MOVUPS 32(AX), X4
- MOVUPS 96(AX), X5
- XORPS X6, X6
- MOVQ $0x0000000f, CX
- MOVQ CX, X7
- PSHUFB X6, X7
- MOVQ x_len+8(FP), CX
- MOVQ x_base+0(FP), DX
- MOVQ y_base+24(FP), BX
-
-loop:
- MOVUPS (DX), X6
- MOVUPS 32(DX), X8
- MOVUPS (BX), X9
- MOVUPS 32(BX), X10
- PXOR X6, X9
- PXOR X8, X10
- MOVUPS X9, (BX)
- MOVUPS X10, 32(BX)
- MOVAPS X9, X11
- PSRLQ $0x04, X11
- MOVAPS X9, X9
- PAND X7, X9
- PAND X7, X11
- MOVUPS X0, X12
- MOVUPS X1, X13
- PSHUFB X9, X12
- PSHUFB X9, X13
- MOVUPS X2, X9
- MOVUPS X3, X14
- PSHUFB X11, X9
- PSHUFB X11, X14
- PXOR X9, X12
- PXOR X14, X13
- MOVAPS X10, X9
- MOVAPS X10, X10
- PAND X7, X9
- PSRLQ $0x04, X10
- PAND X7, X10
- MOVUPS X4, X11
- MOVUPS X5, X14
- PSHUFB X9, X11
- PSHUFB X9, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS 48(AX), X11
- MOVUPS 112(AX), X14
- PSHUFB X10, X11
- PSHUFB X10, X14
- PXOR X11, X12
- PXOR X14, X13
- PXOR X12, X6
- PXOR X13, X8
- MOVUPS X6, (DX)
- MOVUPS X8, 32(DX)
- MOVUPS 16(DX), X6
- MOVUPS 48(DX), X8
- MOVUPS 16(BX), X9
- MOVUPS 48(BX), X10
- PXOR X6, X9
- PXOR X8, X10
- MOVUPS X9, 16(BX)
- MOVUPS X10, 48(BX)
- MOVAPS X9, X11
- PSRLQ $0x04, X11
- MOVAPS X9, X9
- PAND X7, X9
- PAND X7, X11
- MOVUPS X0, X12
- MOVUPS X1, X13
- PSHUFB X9, X12
- PSHUFB X9, X13
- MOVUPS X2, X9
- MOVUPS X3, X14
- PSHUFB X11, X9
- PSHUFB X11, X14
- PXOR X9, X12
- PXOR X14, X13
- MOVAPS X10, X9
- MOVAPS X10, X10
- PAND X7, X9
- PSRLQ $0x04, X10
- PAND X7, X10
- MOVUPS X4, X11
- MOVUPS X5, X14
- PSHUFB X9, X11
- PSHUFB X9, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS 48(AX), X11
- MOVUPS 112(AX), X14
- PSHUFB X10, X11
- PSHUFB X10, X14
- PXOR X11, X12
- PXOR X14, X13
- PXOR X12, X6
- PXOR X13, X8
- MOVUPS X6, 16(DX)
- MOVUPS X8, 48(DX)
- ADDQ $0x40, DX
- ADDQ $0x40, BX
- SUBQ $0x40, CX
- JNZ loop
- RET
-
-// func fftDIT2_ssse3(x []byte, y []byte, table *[128]uint8)
-// Requires: SSE, SSE2, SSSE3
-TEXT ·fftDIT2_ssse3(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- MOVUPS (AX), X0
- MOVUPS 64(AX), X1
- MOVUPS 16(AX), X2
- MOVUPS 80(AX), X3
- MOVUPS 32(AX), X4
- MOVUPS 96(AX), X5
- XORPS X6, X6
- MOVQ $0x0000000f, CX
- MOVQ CX, X7
- PSHUFB X6, X7
- MOVQ x_len+8(FP), CX
- MOVQ x_base+0(FP), DX
- MOVQ y_base+24(FP), BX
-
-loop:
- MOVUPS (BX), X9
- MOVUPS 32(BX), X10
- MOVAPS X9, X8
- PSRLQ $0x04, X8
- MOVAPS X9, X6
- PAND X7, X6
- PAND X7, X8
- MOVUPS X0, X11
- MOVUPS X1, X12
- PSHUFB X6, X11
- PSHUFB X6, X12
- MOVUPS X2, X6
- MOVUPS X3, X13
- PSHUFB X8, X6
- PSHUFB X8, X13
- PXOR X6, X11
- PXOR X13, X12
- MOVAPS X10, X6
- MOVAPS X10, X8
- PAND X7, X6
- PSRLQ $0x04, X8
- PAND X7, X8
- MOVUPS X4, X13
- MOVUPS X5, X14
- PSHUFB X6, X13
- PSHUFB X6, X14
- PXOR X13, X11
- PXOR X14, X12
- MOVUPS 48(AX), X13
- MOVUPS 112(AX), X14
- PSHUFB X8, X13
- PSHUFB X8, X14
- PXOR X13, X11
- PXOR X14, X12
- MOVUPS (DX), X6
- MOVUPS 32(DX), X8
- PXOR X11, X6
- PXOR X12, X8
- MOVUPS X6, (DX)
- MOVUPS X8, 32(DX)
- PXOR X6, X9
- PXOR X8, X10
- MOVUPS X9, (BX)
- MOVUPS X10, 32(BX)
- MOVUPS 16(BX), X9
- MOVUPS 48(BX), X10
- MOVAPS X9, X8
- PSRLQ $0x04, X8
- MOVAPS X9, X6
- PAND X7, X6
- PAND X7, X8
- MOVUPS X0, X11
- MOVUPS X1, X12
- PSHUFB X6, X11
- PSHUFB X6, X12
- MOVUPS X2, X6
- MOVUPS X3, X13
- PSHUFB X8, X6
- PSHUFB X8, X13
- PXOR X6, X11
- PXOR X13, X12
- MOVAPS X10, X6
- MOVAPS X10, X8
- PAND X7, X6
- PSRLQ $0x04, X8
- PAND X7, X8
- MOVUPS X4, X13
- MOVUPS X5, X14
- PSHUFB X6, X13
- PSHUFB X6, X14
- PXOR X13, X11
- PXOR X14, X12
- MOVUPS 48(AX), X13
- MOVUPS 112(AX), X14
- PSHUFB X8, X13
- PSHUFB X8, X14
- PXOR X13, X11
- PXOR X14, X12
- MOVUPS 16(DX), X6
- MOVUPS 48(DX), X8
- PXOR X11, X6
- PXOR X12, X8
- MOVUPS X6, 16(DX)
- MOVUPS X8, 48(DX)
- PXOR X6, X9
- PXOR X8, X10
- MOVUPS X9, 16(BX)
- MOVUPS X10, 48(BX)
- ADDQ $0x40, DX
- ADDQ $0x40, BX
- SUBQ $0x40, CX
- JNZ loop
- RET
-
-// func mulgf16_ssse3(x []byte, y []byte, table *[128]uint8)
-// Requires: SSE, SSE2, SSSE3
-TEXT ·mulgf16_ssse3(SB), NOSPLIT, $0-56
- MOVQ table+48(FP), AX
- MOVUPS (AX), X0
- MOVUPS 64(AX), X1
- MOVUPS 16(AX), X2
- MOVUPS 80(AX), X3
- MOVUPS 32(AX), X4
- MOVUPS 96(AX), X5
- MOVUPS 48(AX), X6
- MOVUPS 112(AX), X7
- MOVQ x_len+8(FP), AX
- MOVQ x_base+0(FP), CX
- MOVQ y_base+24(FP), DX
- XORPS X8, X8
- MOVQ $0x0000000f, BX
- MOVQ BX, X9
- PSHUFB X8, X9
-
-loop:
- MOVUPS (DX), X8
- MOVUPS 32(DX), X10
- MOVAPS X8, X11
- PSRLQ $0x04, X11
- MOVAPS X8, X8
- PAND X9, X8
- PAND X9, X11
- MOVUPS X0, X12
- MOVUPS X1, X13
- PSHUFB X8, X12
- PSHUFB X8, X13
- MOVUPS X2, X8
- MOVUPS X3, X14
- PSHUFB X11, X8
- PSHUFB X11, X14
- PXOR X8, X12
- PXOR X14, X13
- MOVAPS X10, X8
- MOVAPS X10, X10
- PAND X9, X8
- PSRLQ $0x04, X10
- PAND X9, X10
- MOVUPS X4, X11
- MOVUPS X5, X14
- PSHUFB X8, X11
- PSHUFB X8, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS X6, X11
- MOVUPS X7, X14
- PSHUFB X10, X11
- PSHUFB X10, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS X12, (CX)
- MOVUPS X13, 32(CX)
- MOVUPS 16(DX), X8
- MOVUPS 48(DX), X10
- MOVAPS X8, X11
- PSRLQ $0x04, X11
- MOVAPS X8, X8
- PAND X9, X8
- PAND X9, X11
- MOVUPS X0, X12
- MOVUPS X1, X13
- PSHUFB X8, X12
- PSHUFB X8, X13
- MOVUPS X2, X8
- MOVUPS X3, X14
- PSHUFB X11, X8
- PSHUFB X11, X14
- PXOR X8, X12
- PXOR X14, X13
- MOVAPS X10, X8
- MOVAPS X10, X10
- PAND X9, X8
- PSRLQ $0x04, X10
- PAND X9, X10
- MOVUPS X4, X11
- MOVUPS X5, X14
- PSHUFB X8, X11
- PSHUFB X8, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS X6, X11
- MOVUPS X7, X14
- PSHUFB X10, X11
- PSHUFB X10, X14
- PXOR X11, X12
- PXOR X14, X13
- MOVUPS X12, 16(CX)
- MOVUPS X13, 48(CX)
- ADDQ $0x40, CX
- ADDQ $0x40, DX
- SUBQ $0x40, AX
- JNZ loop
- RET
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_gen_none.go b/vendor/github.com/klauspost/reedsolomon/galois_gen_none.go
deleted file mode 100644
index 303d6a9..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_gen_none.go
+++ /dev/null
@@ -1,18 +0,0 @@
-//go:build !amd64 || noasm || appengine || gccgo || nogen
-// +build !amd64 noasm appengine gccgo nogen
-
-package reedsolomon
-
-const maxAvx2Inputs = 1
-const maxAvx2Outputs = 1
-const minAvx2Size = 1
-const avxSizeMask = 0
-const avx2CodeGen = false
-
-func galMulSlicesAvx2(matrix []byte, in, out [][]byte, start, stop int) int {
- panic("avx2 codegen not available")
-}
-
-func galMulSlicesAvx2Xor(matrix []byte, in, out [][]byte, start, stop int) int {
- panic("avx2 codegen not available")
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_gen_switch_amd64.go b/vendor/github.com/klauspost/reedsolomon/galois_gen_switch_amd64.go
deleted file mode 100644
index 3078114..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_gen_switch_amd64.go
+++ /dev/null
@@ -1,694 +0,0 @@
-// Code generated by command: go generate gen.go. DO NOT EDIT.
-
-//go:build !appengine && !noasm && gc && !nogen
-// +build !appengine,!noasm,gc,!nogen
-
-package reedsolomon
-
-import (
- "fmt"
-)
-
-const (
- avx2CodeGen = true
- maxAvx2Inputs = 10
- maxAvx2Outputs = 10
- minAvx2Size = 64
- avxSizeMask = maxInt - (minAvx2Size - 1)
-)
-
-func galMulSlicesAvx2(matrix []byte, in, out [][]byte, start, stop int) int {
- n := (stop - start) & avxSizeMask
-
- switch len(in) {
- case 1:
- switch len(out) {
- case 1:
- mulAvxTwo_1x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_1x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_1x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_1x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_1x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_1x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_1x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_1x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_1x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_1x10(matrix, in, out, start, n)
- return n
- }
- case 2:
- switch len(out) {
- case 1:
- mulAvxTwo_2x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_2x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_2x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_2x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_2x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_2x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_2x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_2x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_2x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_2x10(matrix, in, out, start, n)
- return n
- }
- case 3:
- switch len(out) {
- case 1:
- mulAvxTwo_3x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_3x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_3x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_3x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_3x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_3x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_3x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_3x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_3x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_3x10(matrix, in, out, start, n)
- return n
- }
- case 4:
- switch len(out) {
- case 1:
- mulAvxTwo_4x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_4x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_4x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_4x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_4x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_4x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_4x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_4x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_4x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_4x10(matrix, in, out, start, n)
- return n
- }
- case 5:
- switch len(out) {
- case 1:
- mulAvxTwo_5x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_5x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_5x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_5x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_5x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_5x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_5x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_5x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_5x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_5x10(matrix, in, out, start, n)
- return n
- }
- case 6:
- switch len(out) {
- case 1:
- mulAvxTwo_6x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_6x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_6x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_6x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_6x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_6x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_6x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_6x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_6x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_6x10(matrix, in, out, start, n)
- return n
- }
- case 7:
- switch len(out) {
- case 1:
- mulAvxTwo_7x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_7x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_7x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_7x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_7x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_7x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_7x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_7x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_7x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_7x10(matrix, in, out, start, n)
- return n
- }
- case 8:
- switch len(out) {
- case 1:
- mulAvxTwo_8x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_8x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_8x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_8x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_8x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_8x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_8x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_8x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_8x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_8x10(matrix, in, out, start, n)
- return n
- }
- case 9:
- switch len(out) {
- case 1:
- mulAvxTwo_9x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_9x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_9x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_9x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_9x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_9x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_9x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_9x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_9x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_9x10(matrix, in, out, start, n)
- return n
- }
- case 10:
- switch len(out) {
- case 1:
- mulAvxTwo_10x1_64(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_10x2_64(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_10x3_64(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_10x4(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_10x5(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_10x6(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_10x7(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_10x8(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_10x9(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_10x10(matrix, in, out, start, n)
- return n
- }
- }
- panic(fmt.Sprintf("unhandled size: %dx%d", len(in), len(out)))
-}
-
-func galMulSlicesAvx2Xor(matrix []byte, in, out [][]byte, start, stop int) int {
- n := (stop - start) & avxSizeMask
-
- switch len(in) {
- case 1:
- switch len(out) {
- case 1:
- mulAvxTwo_1x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_1x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_1x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_1x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_1x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_1x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_1x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_1x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_1x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_1x10Xor(matrix, in, out, start, n)
- return n
- }
- case 2:
- switch len(out) {
- case 1:
- mulAvxTwo_2x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_2x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_2x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_2x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_2x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_2x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_2x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_2x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_2x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_2x10Xor(matrix, in, out, start, n)
- return n
- }
- case 3:
- switch len(out) {
- case 1:
- mulAvxTwo_3x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_3x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_3x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_3x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_3x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_3x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_3x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_3x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_3x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_3x10Xor(matrix, in, out, start, n)
- return n
- }
- case 4:
- switch len(out) {
- case 1:
- mulAvxTwo_4x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_4x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_4x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_4x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_4x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_4x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_4x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_4x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_4x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_4x10Xor(matrix, in, out, start, n)
- return n
- }
- case 5:
- switch len(out) {
- case 1:
- mulAvxTwo_5x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_5x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_5x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_5x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_5x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_5x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_5x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_5x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_5x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_5x10Xor(matrix, in, out, start, n)
- return n
- }
- case 6:
- switch len(out) {
- case 1:
- mulAvxTwo_6x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_6x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_6x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_6x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_6x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_6x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_6x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_6x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_6x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_6x10Xor(matrix, in, out, start, n)
- return n
- }
- case 7:
- switch len(out) {
- case 1:
- mulAvxTwo_7x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_7x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_7x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_7x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_7x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_7x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_7x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_7x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_7x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_7x10Xor(matrix, in, out, start, n)
- return n
- }
- case 8:
- switch len(out) {
- case 1:
- mulAvxTwo_8x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_8x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_8x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_8x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_8x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_8x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_8x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_8x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_8x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_8x10Xor(matrix, in, out, start, n)
- return n
- }
- case 9:
- switch len(out) {
- case 1:
- mulAvxTwo_9x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_9x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_9x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_9x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_9x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_9x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_9x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_9x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_9x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_9x10Xor(matrix, in, out, start, n)
- return n
- }
- case 10:
- switch len(out) {
- case 1:
- mulAvxTwo_10x1_64Xor(matrix, in, out, start, n)
- return n
- case 2:
- mulAvxTwo_10x2_64Xor(matrix, in, out, start, n)
- return n
- case 3:
- mulAvxTwo_10x3_64Xor(matrix, in, out, start, n)
- return n
- case 4:
- mulAvxTwo_10x4Xor(matrix, in, out, start, n)
- return n
- case 5:
- mulAvxTwo_10x5Xor(matrix, in, out, start, n)
- return n
- case 6:
- mulAvxTwo_10x6Xor(matrix, in, out, start, n)
- return n
- case 7:
- mulAvxTwo_10x7Xor(matrix, in, out, start, n)
- return n
- case 8:
- mulAvxTwo_10x8Xor(matrix, in, out, start, n)
- return n
- case 9:
- mulAvxTwo_10x9Xor(matrix, in, out, start, n)
- return n
- case 10:
- mulAvxTwo_10x10Xor(matrix, in, out, start, n)
- return n
- }
- }
- panic(fmt.Sprintf("unhandled size: %dx%d", len(in), len(out)))
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_noasm.go b/vendor/github.com/klauspost/reedsolomon/galois_noasm.go
deleted file mode 100644
index 47e24d7..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_noasm.go
+++ /dev/null
@@ -1,69 +0,0 @@
-//go:build (!amd64 || noasm || appengine || gccgo) && (!arm64 || noasm || appengine || gccgo) && (!ppc64le || noasm || appengine || gccgo)
-// +build !amd64 noasm appengine gccgo
-// +build !arm64 noasm appengine gccgo
-// +build !ppc64le noasm appengine gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-
-package reedsolomon
-
-func galMulSlice(c byte, in, out []byte, o *options) {
- out = out[:len(in)]
- if c == 1 {
- copy(out, in)
- return
- }
- mt := mulTable[c][:256]
- for n, input := range in {
- out[n] = mt[input]
- }
-}
-
-func galMulSliceXor(c byte, in, out []byte, o *options) {
- out = out[:len(in)]
- if c == 1 {
- sliceXor(in, out, o)
- return
- }
- mt := mulTable[c][:256]
- for n, input := range in {
- out[n] ^= mt[input]
- }
-}
-
-// simple slice xor
-func sliceXor(in, out []byte, o *options) {
- sliceXorGo(in, out, o)
-}
-
-func init() {
- defaultOptions.useAVX512 = false
-}
-
-// 4-way butterfly
-func ifftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- ifftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 4-way butterfly
-func fftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- fftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 2-way butterfly forward
-func fftDIT2(x, y []byte, log_m ffe, o *options) {
- // Reference version:
- refMulAdd(x, y, log_m)
- sliceXorGo(x, y, o)
-}
-
-// 2-way butterfly inverse
-func ifftDIT2(x, y []byte, log_m ffe, o *options) {
- // Reference version:
- sliceXorGo(x, y, o)
- refMulAdd(x, y, log_m)
-}
-
-func mulgf16(x, y []byte, log_m ffe, o *options) {
- refMul(x, y, log_m)
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_notamd64.go b/vendor/github.com/klauspost/reedsolomon/galois_notamd64.go
deleted file mode 100644
index e67905b..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_notamd64.go
+++ /dev/null
@@ -1,14 +0,0 @@
-//go:build !amd64 || noasm || appengine || gccgo
-// +build !amd64 noasm appengine gccgo
-
-// Copyright 2020, Klaus Post, see LICENSE for details.
-
-package reedsolomon
-
-func (r *reedSolomon) codeSomeShardsAvx512(matrixRows, inputs, outputs [][]byte, byteCount int) {
- panic("codeSomeShardsAvx512 should not be called if built without asm")
-}
-
-func (r *reedSolomon) codeSomeShardsAvx512P(matrixRows, inputs, outputs [][]byte, byteCount int) {
- panic("codeSomeShardsAvx512P should not be called if built without asm")
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.go b/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.go
deleted file mode 100644
index 415828a..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.go
+++ /dev/null
@@ -1,102 +0,0 @@
-//go:build !noasm && !appengine && !gccgo
-// +build !noasm,!appengine,!gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2018, Minio, Inc.
-
-package reedsolomon
-
-//go:noescape
-func galMulPpc(low, high, in, out []byte)
-
-//go:noescape
-func galMulPpcXor(low, high, in, out []byte)
-
-// This is what the assembler routines do in blocks of 16 bytes:
-/*
-func galMulPpc(low, high, in, out []byte) {
- for n, input := range in {
- l := input & 0xf
- h := input >> 4
- out[n] = low[l] ^ high[h]
- }
-}
-func galMulPpcXor(low, high, in, out []byte) {
- for n, input := range in {
- l := input & 0xf
- h := input >> 4
- out[n] ^= low[l] ^ high[h]
- }
-}
-*/
-
-func galMulSlice(c byte, in, out []byte, o *options) {
- if c == 1 {
- copy(out, in)
- return
- }
- done := (len(in) >> 4) << 4
- if done > 0 {
- galMulPpc(mulTableLow[c][:], mulTableHigh[c][:], in[:done], out)
- }
- remain := len(in) - done
- if remain > 0 {
- mt := mulTable[c][:256]
- for i := done; i < len(in); i++ {
- out[i] = mt[in[i]]
- }
- }
-}
-
-func galMulSliceXor(c byte, in, out []byte, o *options) {
- if c == 1 {
- sliceXor(in, out, o)
- return
- }
- done := (len(in) >> 4) << 4
- if done > 0 {
- galMulPpcXor(mulTableLow[c][:], mulTableHigh[c][:], in[:done], out)
- }
- remain := len(in) - done
- if remain > 0 {
- mt := mulTable[c][:256]
- for i := done; i < len(in); i++ {
- out[i] ^= mt[in[i]]
- }
- }
-}
-
-// slice galois add
-func sliceXor(in, out []byte, o *options) {
- for n, input := range in {
- out[n] ^= input
- }
-}
-
-// 4-way butterfly
-func ifftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- ifftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 4-way butterfly
-func fftDIT4(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- fftDIT4Ref(work, dist, log_m01, log_m23, log_m02, o)
-}
-
-// 2-way butterfly forward
-func fftDIT2(x, y []byte, log_m ffe, o *options) {
- // Reference version:
- refMulAdd(x, y, log_m)
- sliceXor(x, y, o)
-}
-
-// 2-way butterfly inverse
-func ifftDIT2(x, y []byte, log_m ffe, o *options) {
- // Reference version:
- sliceXor(x, y, o)
- refMulAdd(x, y, log_m)
-}
-
-func mulgf16(x, y []byte, log_m ffe, o *options) {
- refMul(x, y, log_m)
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.s b/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.s
deleted file mode 100644
index 7213c61..0000000
--- a/vendor/github.com/klauspost/reedsolomon/galois_ppc64le.s
+++ /dev/null
@@ -1,126 +0,0 @@
-//+build !noasm
-//+build !appengine
-//+build !gccgo
-
-// Copyright 2015, Klaus Post, see LICENSE for details.
-// Copyright 2018, Minio, Inc.
-
-#include "textflag.h"
-
-#define LOW R3
-#define HIGH R4
-#define IN R5
-#define LEN R6
-#define OUT R7
-#define CONSTANTS R8
-#define OFFSET R9
-#define OFFSET1 R10
-#define OFFSET2 R11
-
-#define X6 VS34
-#define X6_ V2
-#define X7 VS35
-#define X7_ V3
-#define MSG VS36
-#define MSG_ V4
-#define MSG_HI VS37
-#define MSG_HI_ V5
-#define RESULT VS38
-#define RESULT_ V6
-#define ROTATE VS39
-#define ROTATE_ V7
-#define MASK VS40
-#define MASK_ V8
-#define FLIP VS41
-#define FLIP_ V9
-
-// func galMulPpc(low, high, in, out []byte)
-TEXT ·galMulPpc(SB), NOFRAME|NOSPLIT, $0-96
- MOVD low+0(FP), LOW
- MOVD high+24(FP), HIGH
- MOVD in+48(FP), IN
- MOVD in_len+56(FP), LEN
- MOVD out+72(FP), OUT
-
- MOVD $16, OFFSET1
- MOVD $32, OFFSET2
-
- MOVD $·constants(SB), CONSTANTS
- LXVD2X (CONSTANTS)(R0), ROTATE
- LXVD2X (CONSTANTS)(OFFSET1), MASK
- LXVD2X (CONSTANTS)(OFFSET2), FLIP
-
- LXVD2X (LOW)(R0), X6
- LXVD2X (HIGH)(R0), X7
- VPERM X6_, V31, FLIP_, X6_
- VPERM X7_, V31, FLIP_, X7_
-
- MOVD $0, OFFSET
-
-loop:
- LXVD2X (IN)(OFFSET), MSG
-
- VSRB MSG_, ROTATE_, MSG_HI_
- VAND MSG_, MASK_, MSG_
- VPERM X6_, V31, MSG_, MSG_
- VPERM X7_, V31, MSG_HI_, MSG_HI_
-
- VXOR MSG_, MSG_HI_, MSG_
-
- STXVD2X MSG, (OUT)(OFFSET)
-
- ADD $16, OFFSET, OFFSET
- CMP LEN, OFFSET
- BGT loop
- RET
-
-// func galMulPpcXorlow, high, in, out []byte)
-TEXT ·galMulPpcXor(SB), NOFRAME|NOSPLIT, $0-96
- MOVD low+0(FP), LOW
- MOVD high+24(FP), HIGH
- MOVD in+48(FP), IN
- MOVD in_len+56(FP), LEN
- MOVD out+72(FP), OUT
-
- MOVD $16, OFFSET1
- MOVD $32, OFFSET2
-
- MOVD $·constants(SB), CONSTANTS
- LXVD2X (CONSTANTS)(R0), ROTATE
- LXVD2X (CONSTANTS)(OFFSET1), MASK
- LXVD2X (CONSTANTS)(OFFSET2), FLIP
-
- LXVD2X (LOW)(R0), X6
- LXVD2X (HIGH)(R0), X7
- VPERM X6_, V31, FLIP_, X6_
- VPERM X7_, V31, FLIP_, X7_
-
- MOVD $0, OFFSET
-
-loopXor:
- LXVD2X (IN)(OFFSET), MSG
- LXVD2X (OUT)(OFFSET), RESULT
-
- VSRB MSG_, ROTATE_, MSG_HI_
- VAND MSG_, MASK_, MSG_
- VPERM X6_, V31, MSG_, MSG_
- VPERM X7_, V31, MSG_HI_, MSG_HI_
-
- VXOR MSG_, MSG_HI_, MSG_
- VXOR MSG_, RESULT_, RESULT_
-
- STXVD2X RESULT, (OUT)(OFFSET)
-
- ADD $16, OFFSET, OFFSET
- CMP LEN, OFFSET
- BGT loopXor
- RET
-
-DATA ·constants+0x0(SB)/8, $0x0404040404040404
-DATA ·constants+0x8(SB)/8, $0x0404040404040404
-DATA ·constants+0x10(SB)/8, $0x0f0f0f0f0f0f0f0f
-DATA ·constants+0x18(SB)/8, $0x0f0f0f0f0f0f0f0f
-DATA ·constants+0x20(SB)/8, $0x0706050403020100
-DATA ·constants+0x28(SB)/8, $0x0f0e0d0c0b0a0908
-
-GLOBL ·constants(SB), 8, $48
diff --git a/vendor/github.com/klauspost/reedsolomon/inversion_tree.go b/vendor/github.com/klauspost/reedsolomon/inversion_tree.go
deleted file mode 100644
index 3f97f81..0000000
--- a/vendor/github.com/klauspost/reedsolomon/inversion_tree.go
+++ /dev/null
@@ -1,164 +0,0 @@
-/**
- * A thread-safe tree which caches inverted matrices.
- *
- * Copyright 2016, Peter Collins
- */
-
-package reedsolomon
-
-import (
- "errors"
- "sync"
-)
-
-// The tree uses a Reader-Writer mutex to make it thread-safe
-// when accessing cached matrices and inserting new ones.
-type inversionTree struct {
- mutex sync.RWMutex
- root inversionNode
-}
-
-type inversionNode struct {
- matrix matrix
- children []*inversionNode
-}
-
-// newInversionTree initializes a tree for storing inverted matrices.
-// Note that the root node is the identity matrix as it implies
-// there were no errors with the original data.
-func newInversionTree(dataShards, parityShards int) *inversionTree {
- identity, _ := identityMatrix(dataShards)
- return &inversionTree{
- root: inversionNode{
- matrix: identity,
- children: make([]*inversionNode, dataShards+parityShards),
- },
- }
-}
-
-// GetInvertedMatrix returns the cached inverted matrix or nil if it
-// is not found in the tree keyed on the indices of invalid rows.
-func (t *inversionTree) GetInvertedMatrix(invalidIndices []int) matrix {
- if t == nil {
- return nil
- }
- // Lock the tree for reading before accessing the tree.
- t.mutex.RLock()
- defer t.mutex.RUnlock()
-
- // If no invalid indices were give we should return the root
- // identity matrix.
- if len(invalidIndices) == 0 {
- return t.root.matrix
- }
-
- // Recursively search for the inverted matrix in the tree, passing in
- // 0 as the parent index as we start at the root of the tree.
- return t.root.getInvertedMatrix(invalidIndices, 0)
-}
-
-// errAlreadySet is returned if the root node matrix is overwritten
-var errAlreadySet = errors.New("the root node identity matrix is already set")
-
-// InsertInvertedMatrix inserts a new inverted matrix into the tree
-// keyed by the indices of invalid rows. The total number of shards
-// is required for creating the proper length lists of child nodes for
-// each node.
-func (t *inversionTree) InsertInvertedMatrix(invalidIndices []int, matrix matrix, shards int) error {
- if t == nil {
- return nil
- }
- // If no invalid indices were given then we are done because the
- // root node is already set with the identity matrix.
- if len(invalidIndices) == 0 {
- return errAlreadySet
- }
-
- if !matrix.IsSquare() {
- return errNotSquare
- }
-
- // Lock the tree for writing and reading before accessing the tree.
- t.mutex.Lock()
- defer t.mutex.Unlock()
-
- // Recursively create nodes for the inverted matrix in the tree until
- // we reach the node to insert the matrix to. We start by passing in
- // 0 as the parent index as we start at the root of the tree.
- t.root.insertInvertedMatrix(invalidIndices, matrix, shards, 0)
-
- return nil
-}
-
-func (n *inversionNode) getInvertedMatrix(invalidIndices []int, parent int) matrix {
- // Get the child node to search next from the list of children. The
- // list of children starts relative to the parent index passed in
- // because the indices of invalid rows is sorted (by default). As we
- // search recursively, the first invalid index gets popped off the list,
- // so when searching through the list of children, use that first invalid
- // index to find the child node.
- firstIndex := invalidIndices[0]
- node := n.children[firstIndex-parent]
-
- // If the child node doesn't exist in the list yet, fail fast by
- // returning, so we can construct and insert the proper inverted matrix.
- if node == nil {
- return nil
- }
-
- // If there's more than one invalid index left in the list we should
- // keep searching recursively.
- if len(invalidIndices) > 1 {
- // Search recursively on the child node by passing in the invalid indices
- // with the first index popped off the front. Also the parent index to
- // pass down is the first index plus one.
- return node.getInvertedMatrix(invalidIndices[1:], firstIndex+1)
- }
- // If there aren't any more invalid indices to search, we've found our
- // node. Return it, however keep in mind that the matrix could still be
- // nil because intermediary nodes in the tree are created sometimes with
- // their inversion matrices uninitialized.
- return node.matrix
-}
-
-func (n *inversionNode) insertInvertedMatrix(invalidIndices []int, matrix matrix, shards, parent int) {
- // As above, get the child node to search next from the list of children.
- // The list of children starts relative to the parent index passed in
- // because the indices of invalid rows is sorted (by default). As we
- // search recursively, the first invalid index gets popped off the list,
- // so when searching through the list of children, use that first invalid
- // index to find the child node.
- firstIndex := invalidIndices[0]
- node := n.children[firstIndex-parent]
-
- // If the child node doesn't exist in the list yet, create a new
- // node because we have the writer lock and add it to the list
- // of children.
- if node == nil {
- // Make the length of the list of children equal to the number
- // of shards minus the first invalid index because the list of
- // invalid indices is sorted, so only this length of errors
- // are possible in the tree.
- node = &inversionNode{
- children: make([]*inversionNode, shards-firstIndex),
- }
- // Insert the new node into the tree at the first index relative
- // to the parent index that was given in this recursive call.
- n.children[firstIndex-parent] = node
- }
-
- // If there's more than one invalid index left in the list we should
- // keep searching recursively in order to find the node to add our
- // matrix.
- if len(invalidIndices) > 1 {
- // As above, search recursively on the child node by passing in
- // the invalid indices with the first index popped off the front.
- // Also the total number of shards and parent index are passed down
- // which is equal to the first index plus one.
- node.insertInvertedMatrix(invalidIndices[1:], matrix, shards, firstIndex+1)
- } else {
- // If there aren't any more invalid indices to search, we've found our
- // node. Cache the inverted matrix in this node.
- node.matrix = matrix
- }
-}
diff --git a/vendor/github.com/klauspost/reedsolomon/leopard.go b/vendor/github.com/klauspost/reedsolomon/leopard.go
deleted file mode 100644
index e9d122a..0000000
--- a/vendor/github.com/klauspost/reedsolomon/leopard.go
+++ /dev/null
@@ -1,1234 +0,0 @@
-package reedsolomon
-
-// This is a O(n*log n) implementation of Reed-Solomon
-// codes, ported from the C++ library https://github.com/catid/leopard.
-//
-// The implementation is based on the paper
-//
-// S.-J. Lin, T. Y. Al-Naffouri, Y. S. Han, and W.-H. Chung,
-// "Novel Polynomial Basis with Fast Fourier Transform
-// and Its Application to Reed-Solomon Erasure Codes"
-// IEEE Trans. on Information Theory, pp. 6284-6299, November, 2016.
-
-import (
- "bytes"
- "io"
- "math/bits"
- "sync"
- "unsafe"
-
- "github.com/klauspost/cpuid/v2"
-)
-
-// leopardFF16 is like reedSolomon but for more than 256 total shards.
-type leopardFF16 struct {
- dataShards int // Number of data shards, should not be modified.
- parityShards int // Number of parity shards, should not be modified.
- totalShards int // Total number of shards. Calculated, and should not be modified.
-
- workPool sync.Pool
-
- o options
-}
-
-// newFF16 is like New, but for more than 256 total shards.
-func newFF16(dataShards, parityShards int, opt options) (*leopardFF16, error) {
- initConstants()
-
- if dataShards <= 0 || parityShards <= 0 {
- return nil, ErrInvShardNum
- }
-
- if dataShards+parityShards > 65536 {
- return nil, ErrMaxShardNum
- }
-
- r := &leopardFF16{
- dataShards: dataShards,
- parityShards: parityShards,
- totalShards: dataShards + parityShards,
- o: opt,
- }
- return r, nil
-}
-
-var _ = Extensions(&leopardFF16{})
-
-func (r *leopardFF16) ShardSizeMultiple() int {
- return 64
-}
-
-func (r *leopardFF16) DataShards() int {
- return r.dataShards
-}
-
-func (r *leopardFF16) ParityShards() int {
- return r.parityShards
-}
-
-func (r *leopardFF16) TotalShards() int {
- return r.parityShards
-}
-
-type ffe uint16
-
-const (
- bitwidth = 16
- order = 1 << bitwidth
- modulus = order - 1
- polynomial = 0x1002D
-)
-
-var (
- fftSkew *[modulus]ffe
- logWalsh *[order]ffe
-)
-
-// Logarithm Tables
-var (
- logLUT *[order]ffe
- expLUT *[order]ffe
-)
-
-// Stores the partial products of x * y at offset x + y * 65536
-// Repeated accesses from the same y value are faster
-var mul16LUTs *[order]mul16LUT
-
-type mul16LUT struct {
- // Contains Lo product as a single lookup.
- // Should be XORed with Hi lookup for result.
- Lo [256]ffe
- Hi [256]ffe
-}
-
-// Stores lookup for avx2
-var multiply256LUT *[order][8 * 16]byte
-
-func (r *leopardFF16) Encode(shards [][]byte) error {
- if len(shards) != r.totalShards {
- return ErrTooFewShards
- }
-
- if err := checkShards(shards, false); err != nil {
- return err
- }
- return r.encode(shards)
-}
-
-func (r *leopardFF16) encode(shards [][]byte) error {
- shardSize := shardSize(shards)
- if shardSize%64 != 0 {
- return ErrShardSize
- }
-
- m := ceilPow2(r.parityShards)
- var work [][]byte
- if w, ok := r.workPool.Get().([][]byte); ok {
- work = w
- }
- if cap(work) >= m*2 {
- work = work[:m*2]
- } else {
- work = make([][]byte, m*2)
- }
- for i := range work {
- if cap(work[i]) < shardSize {
- work[i] = make([]byte, shardSize)
- } else {
- work[i] = work[i][:shardSize]
- }
- }
- defer r.workPool.Put(work)
-
- mtrunc := m
- if r.dataShards < mtrunc {
- mtrunc = r.dataShards
- }
-
- skewLUT := fftSkew[m-1:]
-
- sh := shards
- ifftDITEncoder(
- sh[:r.dataShards],
- mtrunc,
- work,
- nil, // No xor output
- m,
- skewLUT,
- &r.o,
- )
-
- lastCount := r.dataShards % m
- if m >= r.dataShards {
- goto skip_body
- }
-
- // For sets of m data pieces:
- for i := m; i+m <= r.dataShards; i += m {
- sh = sh[m:]
- skewLUT = skewLUT[m:]
-
- // work <- work xor IFFT(data + i, m, m + i)
-
- ifftDITEncoder(
- sh, // data source
- m,
- work[m:], // temporary workspace
- work, // xor destination
- m,
- skewLUT,
- &r.o,
- )
- }
-
- // Handle final partial set of m pieces:
- if lastCount != 0 {
- sh = sh[m:]
- skewLUT = skewLUT[m:]
-
- // work <- work xor IFFT(data + i, m, m + i)
-
- ifftDITEncoder(
- sh, // data source
- lastCount,
- work[m:], // temporary workspace
- work, // xor destination
- m,
- skewLUT,
- &r.o,
- )
- }
-
-skip_body:
- // work <- FFT(work, m, 0)
- fftDIT(work, r.parityShards, m, fftSkew[:], &r.o)
-
- for i, w := range work[:r.parityShards] {
- sh := shards[i+r.dataShards]
- if cap(sh) >= shardSize {
- sh = append(sh[:0], w...)
- } else {
- sh = w
- }
- shards[i+r.dataShards] = sh
- }
-
- return nil
-}
-
-func (r *leopardFF16) EncodeIdx(dataShard []byte, idx int, parity [][]byte) error {
- return ErrNotSupported
-}
-
-func (r *leopardFF16) Join(dst io.Writer, shards [][]byte, outSize int) error {
- // Do we have enough shards?
- if len(shards) < r.dataShards {
- return ErrTooFewShards
- }
- shards = shards[:r.dataShards]
-
- // Do we have enough data?
- size := 0
- for _, shard := range shards {
- if shard == nil {
- return ErrReconstructRequired
- }
- size += len(shard)
-
- // Do we have enough data already?
- if size >= outSize {
- break
- }
- }
- if size < outSize {
- return ErrShortData
- }
-
- // Copy data to dst
- write := outSize
- for _, shard := range shards {
- if write < len(shard) {
- _, err := dst.Write(shard[:write])
- return err
- }
- n, err := dst.Write(shard)
- if err != nil {
- return err
- }
- write -= n
- }
- return nil
-}
-
-func (r *leopardFF16) Update(shards [][]byte, newDatashards [][]byte) error {
- return ErrNotSupported
-}
-
-func (r *leopardFF16) Split(data []byte) ([][]byte, error) {
- if len(data) == 0 {
- return nil, ErrShortData
- }
- dataLen := len(data)
- // Calculate number of bytes per data shard.
- perShard := (len(data) + r.dataShards - 1) / r.dataShards
- perShard = ((perShard + 63) / 64) * 64
-
- if cap(data) > len(data) {
- data = data[:cap(data)]
- }
-
- // Only allocate memory if necessary
- var padding []byte
- if len(data) < (r.totalShards * perShard) {
- // calculate maximum number of full shards in `data` slice
- fullShards := len(data) / perShard
- padding = make([]byte, r.totalShards*perShard-perShard*fullShards)
- copy(padding, data[perShard*fullShards:])
- data = data[0 : perShard*fullShards]
- } else {
- for i := dataLen; i < dataLen+r.dataShards; i++ {
- data[i] = 0
- }
- }
-
- // Split into equal-length shards.
- dst := make([][]byte, r.totalShards)
- i := 0
- for ; i < len(dst) && len(data) >= perShard; i++ {
- dst[i] = data[:perShard:perShard]
- data = data[perShard:]
- }
-
- for j := 0; i+j < len(dst); j++ {
- dst[i+j] = padding[:perShard:perShard]
- padding = padding[perShard:]
- }
-
- return dst, nil
-}
-
-func (r *leopardFF16) ReconstructSome(shards [][]byte, required []bool) error {
- return r.ReconstructData(shards)
-}
-
-func (r *leopardFF16) Reconstruct(shards [][]byte) error {
- return r.reconstruct(shards, true)
-}
-
-func (r *leopardFF16) ReconstructData(shards [][]byte) error {
- return r.reconstruct(shards, false)
-}
-
-func (r *leopardFF16) Verify(shards [][]byte) (bool, error) {
- if len(shards) != r.totalShards {
- return false, ErrTooFewShards
- }
- if err := checkShards(shards, false); err != nil {
- return false, err
- }
-
- // Re-encode parity shards to temporary storage.
- shardSize := len(shards[0])
- outputs := make([][]byte, r.totalShards)
- copy(outputs, shards[:r.dataShards])
- for i := r.dataShards; i < r.totalShards; i++ {
- outputs[i] = make([]byte, shardSize)
- }
- if err := r.Encode(outputs); err != nil {
- return false, err
- }
-
- // Compare.
- for i := r.dataShards; i < r.totalShards; i++ {
- if !bytes.Equal(outputs[i], shards[i]) {
- return false, nil
- }
- }
- return true, nil
-}
-
-func (r *leopardFF16) reconstruct(shards [][]byte, recoverAll bool) error {
- if len(shards) != r.totalShards {
- return ErrTooFewShards
- }
-
- if err := checkShards(shards, true); err != nil {
- return err
- }
-
- // Quick check: are all of the shards present? If so, there's
- // nothing to do.
- numberPresent := 0
- dataPresent := 0
- for i := 0; i < r.totalShards; i++ {
- if len(shards[i]) != 0 {
- numberPresent++
- if i < r.dataShards {
- dataPresent++
- }
- }
- }
- if numberPresent == r.totalShards || !recoverAll && dataPresent == r.dataShards {
- // Cool. All of the shards have data. We don't
- // need to do anything.
- return nil
- }
-
- // Use only if we are missing less than 1/4 parity.
- useBits := r.totalShards-numberPresent <= r.parityShards/4
-
- // Check if we have enough to reconstruct.
- if numberPresent < r.dataShards {
- return ErrTooFewShards
- }
-
- shardSize := shardSize(shards)
- if shardSize%64 != 0 {
- return ErrShardSize
- }
-
- m := ceilPow2(r.parityShards)
- n := ceilPow2(m + r.dataShards)
-
- const LEO_ERROR_BITFIELD_OPT = true
-
- // Fill in error locations.
- var errorBits errorBitfield
- var errLocs [order]ffe
- for i := 0; i < r.parityShards; i++ {
- if len(shards[i+r.dataShards]) == 0 {
- errLocs[i] = 1
- if LEO_ERROR_BITFIELD_OPT && recoverAll {
- errorBits.set(i)
- }
- }
- }
- for i := r.parityShards; i < m; i++ {
- errLocs[i] = 1
- if LEO_ERROR_BITFIELD_OPT && recoverAll {
- errorBits.set(i)
- }
- }
- for i := 0; i < r.dataShards; i++ {
- if len(shards[i]) == 0 {
- errLocs[i+m] = 1
- if LEO_ERROR_BITFIELD_OPT {
- errorBits.set(i + m)
- }
- }
- }
-
- if LEO_ERROR_BITFIELD_OPT && useBits {
- errorBits.prepare()
- }
-
- // Evaluate error locator polynomial
- fwht(&errLocs, order, m+r.dataShards)
-
- for i := 0; i < order; i++ {
- errLocs[i] = ffe((uint(errLocs[i]) * uint(logWalsh[i])) % modulus)
- }
-
- fwht(&errLocs, order, order)
-
- var work [][]byte
- if w, ok := r.workPool.Get().([][]byte); ok {
- work = w
- }
- if cap(work) >= n {
- work = work[:n]
- } else {
- work = make([][]byte, n)
- }
- for i := range work {
- if cap(work[i]) < shardSize {
- work[i] = make([]byte, shardSize)
- } else {
- work[i] = work[i][:shardSize]
- }
- }
- defer r.workPool.Put(work)
-
- // work <- recovery data
-
- for i := 0; i < r.parityShards; i++ {
- if len(shards[i+r.dataShards]) != 0 {
- mulgf16(work[i], shards[i+r.dataShards], errLocs[i], &r.o)
- } else {
- memclr(work[i])
- }
- }
- for i := r.parityShards; i < m; i++ {
- memclr(work[i])
- }
-
- // work <- original data
-
- for i := 0; i < r.dataShards; i++ {
- if len(shards[i]) != 0 {
- mulgf16(work[m+i], shards[i], errLocs[m+i], &r.o)
- } else {
- memclr(work[m+i])
- }
- }
- for i := m + r.dataShards; i < n; i++ {
- memclr(work[i])
- }
-
- // work <- IFFT(work, n, 0)
-
- ifftDITDecoder(
- m+r.dataShards,
- work,
- n,
- fftSkew[:],
- &r.o,
- )
-
- // work <- FormalDerivative(work, n)
-
- for i := 1; i < n; i++ {
- width := ((i ^ (i - 1)) + 1) >> 1
- slicesXor(work[i-width:i], work[i:i+width], &r.o)
- }
-
- // work <- FFT(work, n, 0) truncated to m + dataShards
-
- outputCount := m + r.dataShards
-
- if LEO_ERROR_BITFIELD_OPT && useBits {
- errorBits.fftDIT(work, outputCount, n, fftSkew[:], &r.o)
- } else {
- fftDIT(work, outputCount, n, fftSkew[:], &r.o)
- }
-
- // Reveal erasures
- //
- // Original = -ErrLocator * FFT( Derivative( IFFT( ErrLocator * ReceivedData ) ) )
- // mul_mem(x, y, log_m, ) equals x[] = y[] * log_m
- //
- // mem layout: [Recovery Data (Power of Two = M)] [Original Data (K)] [Zero Padding out to N]
- end := r.dataShards
- if recoverAll {
- end = r.totalShards
- }
- for i := 0; i < end; i++ {
- if len(shards[i]) != 0 {
- continue
- }
- if cap(shards[i]) >= shardSize {
- shards[i] = shards[i][:shardSize]
- } else {
- shards[i] = make([]byte, shardSize)
- }
- if i >= r.dataShards {
- // Parity shard.
- mulgf16(shards[i], work[i-r.dataShards], modulus-errLocs[i-r.dataShards], &r.o)
- } else {
- // Data shard.
- mulgf16(shards[i], work[i+m], modulus-errLocs[i+m], &r.o)
- }
- }
- return nil
-}
-
-// Basic no-frills version for decoder
-func ifftDITDecoder(mtrunc int, work [][]byte, m int, skewLUT []ffe, o *options) {
- // Decimation in time: Unroll 2 layers at a time
- dist := 1
- dist4 := 4
- for dist4 <= m {
- // For each set of dist*4 elements:
- for r := 0; r < mtrunc; r += dist4 {
- iend := r + dist
- log_m01 := skewLUT[iend-1]
- log_m02 := skewLUT[iend+dist-1]
- log_m23 := skewLUT[iend+dist*2-1]
-
- // For each set of dist elements:
- for i := r; i < iend; i++ {
- ifftDIT4(work[i:], dist, log_m01, log_m23, log_m02, o)
- }
- }
- dist = dist4
- dist4 <<= 2
- }
-
- // If there is one layer left:
- if dist < m {
- // Assuming that dist = m / 2
- if dist*2 != m {
- panic("internal error")
- }
-
- log_m := skewLUT[dist-1]
-
- if log_m == modulus {
- slicesXor(work[dist:2*dist], work[:dist], o)
- } else {
- for i := 0; i < dist; i++ {
- ifftDIT2(
- work[i],
- work[i+dist],
- log_m,
- o,
- )
- }
- }
- }
-}
-
-// In-place FFT for encoder and decoder
-func fftDIT(work [][]byte, mtrunc, m int, skewLUT []ffe, o *options) {
- // Decimation in time: Unroll 2 layers at a time
- dist4 := m
- dist := m >> 2
- for dist != 0 {
- // For each set of dist*4 elements:
- for r := 0; r < mtrunc; r += dist4 {
- iend := r + dist
- log_m01 := skewLUT[iend-1]
- log_m02 := skewLUT[iend+dist-1]
- log_m23 := skewLUT[iend+dist*2-1]
-
- // For each set of dist elements:
- for i := r; i < iend; i++ {
- fftDIT4(
- work[i:],
- dist,
- log_m01,
- log_m23,
- log_m02,
- o,
- )
- }
- }
- dist4 = dist
- dist >>= 2
- }
-
- // If there is one layer left:
- if dist4 == 2 {
- for r := 0; r < mtrunc; r += 2 {
- log_m := skewLUT[r+1-1]
-
- if log_m == modulus {
- sliceXor(work[r], work[r+1], o)
- } else {
- fftDIT2(work[r], work[r+1], log_m, o)
- }
- }
- }
-}
-
-// 4-way butterfly
-func fftDIT4Ref(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- // First layer:
- if log_m02 == modulus {
- sliceXor(work[0], work[dist*2], o)
- sliceXor(work[dist], work[dist*3], o)
- } else {
- fftDIT2(work[0], work[dist*2], log_m02, o)
- fftDIT2(work[dist], work[dist*3], log_m02, o)
- }
-
- // Second layer:
- if log_m01 == modulus {
- sliceXor(work[0], work[dist], o)
- } else {
- fftDIT2(work[0], work[dist], log_m01, o)
- }
-
- if log_m23 == modulus {
- sliceXor(work[dist*2], work[dist*3], o)
- } else {
- fftDIT2(work[dist*2], work[dist*3], log_m23, o)
- }
-}
-
-// Unrolled IFFT for encoder
-func ifftDITEncoder(data [][]byte, mtrunc int, work [][]byte, xorRes [][]byte, m int, skewLUT []ffe, o *options) {
- // I tried rolling the memcpy/memset into the first layer of the FFT and
- // found that it only yields a 4% performance improvement, which is not
- // worth the extra complexity.
- for i := 0; i < mtrunc; i++ {
- copy(work[i], data[i])
- }
- for i := mtrunc; i < m; i++ {
- memclr(work[i])
- }
-
- // I tried splitting up the first few layers into L3-cache sized blocks but
- // found that it only provides about 5% performance boost, which is not
- // worth the extra complexity.
-
- // Decimation in time: Unroll 2 layers at a time
- dist := 1
- dist4 := 4
- for dist4 <= m {
- // For each set of dist*4 elements:
- for r := 0; r < mtrunc; r += dist4 {
- iend := r + dist
- log_m01 := skewLUT[iend]
- log_m02 := skewLUT[iend+dist]
- log_m23 := skewLUT[iend+dist*2]
-
- // For each set of dist elements:
- for i := r; i < iend; i++ {
- ifftDIT4(
- work[i:],
- dist,
- log_m01,
- log_m23,
- log_m02,
- o,
- )
- }
- }
-
- dist = dist4
- dist4 <<= 2
- // I tried alternating sweeps left->right and right->left to reduce cache misses.
- // It provides about 1% performance boost when done for both FFT and IFFT, so it
- // does not seem to be worth the extra complexity.
- }
-
- // If there is one layer left:
- if dist < m {
- // Assuming that dist = m / 2
- if dist*2 != m {
- panic("internal error")
- }
-
- logm := skewLUT[dist]
-
- if logm == modulus {
- slicesXor(work[dist:dist*2], work[:dist], o)
- } else {
- for i := 0; i < dist; i++ {
- ifftDIT2(work[i], work[i+dist], logm, o)
- }
- }
- }
-
- // I tried unrolling this but it does not provide more than 5% performance
- // improvement for 16-bit finite fields, so it's not worth the complexity.
- if xorRes != nil {
- slicesXor(xorRes[:m], work[:m], o)
- }
-}
-
-func ifftDIT4Ref(work [][]byte, dist int, log_m01, log_m23, log_m02 ffe, o *options) {
- // First layer:
- if log_m01 == modulus {
- sliceXor(work[0], work[dist], o)
- } else {
- ifftDIT2(work[0], work[dist], log_m01, o)
- }
-
- if log_m23 == modulus {
- sliceXor(work[dist*2], work[dist*3], o)
- } else {
- ifftDIT2(work[dist*2], work[dist*3], log_m23, o)
- }
-
- // Second layer:
- if log_m02 == modulus {
- sliceXor(work[0], work[dist*2], o)
- sliceXor(work[dist], work[dist*3], o)
- } else {
- ifftDIT2(work[0], work[dist*2], log_m02, o)
- ifftDIT2(work[dist], work[dist*3], log_m02, o)
- }
-}
-
-// Reference version of muladd: x[] ^= y[] * log_m
-func refMulAdd(x, y []byte, log_m ffe) {
- lut := &mul16LUTs[log_m]
-
- for len(x) >= 64 {
- // Assert sizes for no bounds checks in loop
- hiA := y[32:64]
- loA := y[:32]
- dst := x[:64] // Needed, but not checked...
- for i, lo := range loA {
- hi := hiA[i]
- prod := lut.Lo[lo] ^ lut.Hi[hi]
-
- dst[i] ^= byte(prod)
- dst[i+32] ^= byte(prod >> 8)
- }
- x = x[64:]
- y = y[64:]
- }
-}
-
-func memclr(s []byte) {
- for i := range s {
- s[i] = 0
- }
-}
-
-// slicesXor calls xor for every slice pair in v1, v2.
-func slicesXor(v1, v2 [][]byte, o *options) {
- for i, v := range v1 {
- sliceXor(v2[i], v, o)
- }
-}
-
-// Reference version of mul: x[] = y[] * log_m
-func refMul(x, y []byte, log_m ffe) {
- lut := &mul16LUTs[log_m]
-
- for off := 0; off < len(x); off += 64 {
- loA := y[off : off+32]
- hiA := y[off+32:]
- hiA = hiA[:len(loA)]
- for i, lo := range loA {
- hi := hiA[i]
- prod := lut.Lo[lo] ^ lut.Hi[hi]
-
- x[off+i] = byte(prod)
- x[off+i+32] = byte(prod >> 8)
- }
- }
-}
-
-// Returns a * Log(b)
-func mulLog(a, log_b ffe) ffe {
- /*
- Note that this operation is not a normal multiplication in a finite
- field because the right operand is already a logarithm. This is done
- because it moves K table lookups from the Decode() method into the
- initialization step that is less performance critical. The LogWalsh[]
- table below contains precalculated logarithms so it is easier to do
- all the other multiplies in that form as well.
- */
- if a == 0 {
- return 0
- }
- return expLUT[addMod(logLUT[a], log_b)]
-}
-
-// z = x + y (mod kModulus)
-func addMod(a, b ffe) ffe {
- sum := uint(a) + uint(b)
-
- // Partial reduction step, allowing for kModulus to be returned
- return ffe(sum + sum>>bitwidth)
-}
-
-// z = x - y (mod kModulus)
-func subMod(a, b ffe) ffe {
- dif := uint(a) - uint(b)
-
- // Partial reduction step, allowing for kModulus to be returned
- return ffe(dif + dif>>bitwidth)
-}
-
-// ceilPow2 returns power of two at or above n.
-func ceilPow2(n int) int {
- const w = int(unsafe.Sizeof(n) * 8)
- return 1 << (w - bits.LeadingZeros(uint(n-1)))
-}
-
-// Decimation in time (DIT) Fast Walsh-Hadamard Transform
-// Unrolls pairs of layers to perform cross-layer operations in registers
-// mtrunc: Number of elements that are non-zero at the front of data
-func fwht(data *[order]ffe, m, mtrunc int) {
- // Decimation in time: Unroll 2 layers at a time
- dist := 1
- dist4 := 4
- for dist4 <= m {
- // For each set of dist*4 elements:
- for r := 0; r < mtrunc; r += dist4 {
- // For each set of dist elements:
- // Use 16 bit indices to avoid bounds check on [65536]ffe.
- dist := uint16(dist)
- off := uint16(r)
- for i := uint16(0); i < dist; i++ {
- // fwht4(data[i:], dist) inlined...
- // Reading values appear faster than updating pointers.
- // Casting to uint is not faster.
- t0 := data[off]
- t1 := data[off+dist]
- t2 := data[off+dist*2]
- t3 := data[off+dist*3]
-
- t0, t1 = fwht2alt(t0, t1)
- t2, t3 = fwht2alt(t2, t3)
- t0, t2 = fwht2alt(t0, t2)
- t1, t3 = fwht2alt(t1, t3)
-
- data[off] = t0
- data[off+dist] = t1
- data[off+dist*2] = t2
- data[off+dist*3] = t3
- off++
- }
- }
- dist = dist4
- dist4 <<= 2
- }
-
- // If there is one layer left:
- if dist < m {
- dist := uint16(dist)
- for i := uint16(0); i < dist; i++ {
- fwht2(&data[i], &data[i+dist])
- }
- }
-}
-
-func fwht4(data []ffe, s int) {
- s2 := s << 1
-
- t0 := &data[0]
- t1 := &data[s]
- t2 := &data[s2]
- t3 := &data[s2+s]
-
- fwht2(t0, t1)
- fwht2(t2, t3)
- fwht2(t0, t2)
- fwht2(t1, t3)
-}
-
-// {a, b} = {a + b, a - b} (Mod Q)
-func fwht2(a, b *ffe) {
- sum := addMod(*a, *b)
- dif := subMod(*a, *b)
- *a = sum
- *b = dif
-}
-
-// fwht2alt is as fwht2, but returns result.
-func fwht2alt(a, b ffe) (ffe, ffe) {
- return addMod(a, b), subMod(a, b)
-}
-
-var initOnce sync.Once
-
-func initConstants() {
- initOnce.Do(func() {
- initLUTs()
- initFFTSkew()
- initMul16LUT()
- })
-}
-
-// Initialize logLUT, expLUT.
-func initLUTs() {
- cantorBasis := [bitwidth]ffe{
- 0x0001, 0xACCA, 0x3C0E, 0x163E,
- 0xC582, 0xED2E, 0x914C, 0x4012,
- 0x6C98, 0x10D8, 0x6A72, 0xB900,
- 0xFDB8, 0xFB34, 0xFF38, 0x991E,
- }
-
- expLUT = &[order]ffe{}
- logLUT = &[order]ffe{}
-
- // LFSR table generation:
- state := 1
- for i := ffe(0); i < modulus; i++ {
- expLUT[state] = i
- state <<= 1
- if state >= order {
- state ^= polynomial
- }
- }
- expLUT[0] = modulus
-
- // Conversion to Cantor basis:
-
- logLUT[0] = 0
- for i := 0; i < bitwidth; i++ {
- basis := cantorBasis[i]
- width := 1 << i
-
- for j := 0; j < width; j++ {
- logLUT[j+width] = logLUT[j] ^ basis
- }
- }
-
- for i := 0; i < order; i++ {
- logLUT[i] = expLUT[logLUT[i]]
- }
-
- for i := 0; i < order; i++ {
- expLUT[logLUT[i]] = ffe(i)
- }
-
- expLUT[modulus] = expLUT[0]
-}
-
-// Initialize fftSkew.
-func initFFTSkew() {
- var temp [bitwidth - 1]ffe
-
- // Generate FFT skew vector {1}:
-
- for i := 1; i < bitwidth; i++ {
- temp[i-1] = ffe(1 << i)
- }
-
- fftSkew = &[modulus]ffe{}
- logWalsh = &[order]ffe{}
-
- for m := 0; m < bitwidth-1; m++ {
- step := 1 << (m + 1)
-
- fftSkew[1<See older changes
- -## May 2020 - -* ARM64 optimizations, up to 2.5x faster. -* Added [WithFastOneParityMatrix](https://pkg.go.dev/github.com/klauspost/reedsolomon?tab=doc#WithFastOneParityMatrix) for faster operation with 1 parity shard. -* Much better performance when using a limited number of goroutines. -* AVX512 is now using multiple cores. -* Stream processing overhaul, big speedups in most cases. -* AVX512 optimizations - -## March 6, 2019 - -The pure Go implementation is about 30% faster. Minor tweaks to assembler implementations. - -## February 8, 2019 - -AVX512 accelerated version added for Intel Skylake CPUs. This can give up to a 4x speed improvement as compared to AVX2. -See [here](https://github.com/klauspost/reedsolomon#performance-on-avx512) for more details. - -## December 18, 2018 - -Assembly code for ppc64le has been contributed, this boosts performance by about 10x on this platform. - -## November 18, 2017 - -Added [WithAutoGoroutines](https://godoc.org/github.com/klauspost/reedsolomon#WithAutoGoroutines) which will attempt -to calculate the optimal number of goroutines to use based on your expected shard size and detected CPU. - -## October 1, 2017 - -* [Cauchy Matrix](https://godoc.org/github.com/klauspost/reedsolomon#WithCauchyMatrix) is now an option. -Thanks to [templexxx](https://github.com/templexxx) for the basis of this. - -* Default maximum number of [goroutines](https://godoc.org/github.com/klauspost/reedsolomon#WithMaxGoroutines) -has been increased for better multi-core scaling. - -* After several requests the Reconstruct and ReconstructData now slices of zero length but sufficient capacity to -be used instead of allocating new memory. - -## August 26, 2017 - -* The [`Encoder()`](https://godoc.org/github.com/klauspost/reedsolomon#Encoder) now contains an `Update` -function contributed by [chenzhongtao](https://github.com/chenzhongtao). - -* [Frank Wessels](https://github.com/fwessels) kindly contributed ARM 64 bit assembly, -which gives a huge performance boost on this platform. - -## July 20, 2017 - -`ReconstructData` added to [`Encoder`](https://godoc.org/github.com/klauspost/reedsolomon#Encoder) interface. -This can cause compatibility issues if you implement your own Encoder. A simple workaround can be added: - -```Go -func (e *YourEnc) ReconstructData(shards [][]byte) error { - return ReconstructData(shards) -} -``` - -You can of course also do your own implementation. -The [`StreamEncoder`](https://godoc.org/github.com/klauspost/reedsolomon#StreamEncoder) -handles this without modifying the interface. -This is a good lesson on why returning interfaces is not a good design. - -
-
-```
-NONCE:
- 16bytes cryptographically secure random number, nonce changes for every packet.
-
-CRC32:
- CRC-32 checksum of data using the IEEE polynomial
-
-FEC TYPE:
- typeData = 0xF1
- typeParity = 0xF2
-
-FEC SEQID:
- monotonically increasing in range: [0, (0xffffffff/shardSize) * shardSize - 1]
-
-SIZE:
- The size of KCP frame plus 2
-```
-
-```
-+-----------------+
-| SESSION |
-+-----------------+
-| KCP(ARQ) |
-+-----------------+
-| FEC(OPTIONAL) |
-+-----------------+
-| CRYPTO(OPTIONAL)|
-+-----------------+
-| UDP(PACKET) |
-+-----------------+
-| IP |
-+-----------------+
-| LINK |
-+-----------------+
-| PHY |
-+-----------------+
-(LAYER MODEL OF KCP-GO)
-```
-
-
-## Examples
-
-1. [simple examples](https://github.com/xtaci/kcp-go/tree/master/examples)
-2. [kcptun client](https://github.com/xtaci/kcptun/blob/master/client/main.go)
-3. [kcptun server](https://github.com/xtaci/kcptun/blob/master/server/main.go)
-
-## Benchmark
-```
- Model Name: MacBook Pro
- Model Identifier: MacBookPro14,1
- Processor Name: Intel Core i5
- Processor Speed: 3.1 GHz
- Number of Processors: 1
- Total Number of Cores: 2
- L2 Cache (per Core): 256 KB
- L3 Cache: 4 MB
- Memory: 8 GB
-```
-```
-$ go test -v -run=^$ -bench .
-beginning tests, encryption:salsa20, fec:10/3
-goos: darwin
-goarch: amd64
-pkg: github.com/xtaci/kcp-go
-BenchmarkSM4-4 50000 32180 ns/op 93.23 MB/s 0 B/op 0 allocs/op
-BenchmarkAES128-4 500000 3285 ns/op 913.21 MB/s 0 B/op 0 allocs/op
-BenchmarkAES192-4 300000 3623 ns/op 827.85 MB/s 0 B/op 0 allocs/op
-BenchmarkAES256-4 300000 3874 ns/op 774.20 MB/s 0 B/op 0 allocs/op
-BenchmarkTEA-4 100000 15384 ns/op 195.00 MB/s 0 B/op 0 allocs/op
-BenchmarkXOR-4 20000000 89.9 ns/op 33372.00 MB/s 0 B/op 0 allocs/op
-BenchmarkBlowfish-4 50000 26927 ns/op 111.41 MB/s 0 B/op 0 allocs/op
-BenchmarkNone-4 30000000 45.7 ns/op 65597.94 MB/s 0 B/op 0 allocs/op
-BenchmarkCast5-4 50000 34258 ns/op 87.57 MB/s 0 B/op 0 allocs/op
-Benchmark3DES-4 10000 117149 ns/op 25.61 MB/s 0 B/op 0 allocs/op
-BenchmarkTwofish-4 50000 33538 ns/op 89.45 MB/s 0 B/op 0 allocs/op
-BenchmarkXTEA-4 30000 45666 ns/op 65.69 MB/s 0 B/op 0 allocs/op
-BenchmarkSalsa20-4 500000 3308 ns/op 906.76 MB/s 0 B/op 0 allocs/op
-BenchmarkCRC32-4 20000000 65.2 ns/op 15712.43 MB/s
-BenchmarkCsprngSystem-4 1000000 1150 ns/op 13.91 MB/s
-BenchmarkCsprngMD5-4 10000000 145 ns/op 110.26 MB/s
-BenchmarkCsprngSHA1-4 10000000 158 ns/op 126.54 MB/s
-BenchmarkCsprngNonceMD5-4 10000000 153 ns/op 104.22 MB/s
-BenchmarkCsprngNonceAES128-4 100000000 19.1 ns/op 837.81 MB/s
-BenchmarkFECDecode-4 1000000 1119 ns/op 1339.61 MB/s 1606 B/op 2 allocs/op
-BenchmarkFECEncode-4 2000000 832 ns/op 1801.83 MB/s 17 B/op 0 allocs/op
-BenchmarkFlush-4 5000000 272 ns/op 0 B/op 0 allocs/op
-BenchmarkEchoSpeed4K-4 5000 259617 ns/op 15.78 MB/s 5451 B/op 149 allocs/op
-BenchmarkEchoSpeed64K-4 1000 1706084 ns/op 38.41 MB/s 56002 B/op 1604 allocs/op
-BenchmarkEchoSpeed512K-4 100 14345505 ns/op 36.55 MB/s 482597 B/op 13045 allocs/op
-BenchmarkEchoSpeed1M-4 30 34859104 ns/op 30.08 MB/s 1143773 B/op 27186 allocs/op
-BenchmarkSinkSpeed4K-4 50000 31369 ns/op 130.57 MB/s 1566 B/op 30 allocs/op
-BenchmarkSinkSpeed64K-4 5000 329065 ns/op 199.16 MB/s 21529 B/op 453 allocs/op
-BenchmarkSinkSpeed256K-4 500 2373354 ns/op 220.91 MB/s 166332 B/op 3554 allocs/op
-BenchmarkSinkSpeed1M-4 300 5117927 ns/op 204.88 MB/s 310378 B/op 6988 allocs/op
-PASS
-ok github.com/xtaci/kcp-go 50.349s
-```
-
-
-## Typical Flame Graph
-
-
-## Key Design Considerations
-
-1. slice vs. container/list
-
-`kcp.flush()` loops through the send queue for retransmission checking for every 20ms(interval).
-
-I've wrote a benchmark for comparing sequential loop through *slice* and *container/list* here:
-
-https://github.com/xtaci/notes/blob/master/golang/benchmark2/cachemiss_test.go
-
-```
-BenchmarkLoopSlice-4 2000000000 0.39 ns/op
-BenchmarkLoopList-4 100000000 54.6 ns/op
-```
-
-List structure introduces **heavy cache misses** compared to slice which owns better **locality**, 5000 connections with 32 window size and 20ms interval will cost 6us/0.03%(cpu) using slice, and 8.7ms/43.5%(cpu) for list for each `kcp.flush()`.
-
-2. Timing accuracy vs. syscall clock_gettime
-
-Timing is **critical** to **RTT estimator**, inaccurate timing leads to false retransmissions in KCP, but calling `time.Now()` costs 42 cycles(10.5ns on 4GHz CPU, 15.6ns on my MacBook Pro 2.7GHz).
-
-The benchmark for time.Now() lies here:
-
-https://github.com/xtaci/notes/blob/master/golang/benchmark2/syscall_test.go
-
-```
-BenchmarkNow-4 100000000 15.6 ns/op
-```
-
-In kcp-go, after each `kcp.output()` function call, current clock time will be updated upon return, and for a single `kcp.flush()` operation, current time will be queried from system once. For most of the time, 5000 connections costs 5000 * 15.6ns = 78us(a fixed cost while no packet needs to be sent), as for 10MB/s data transfering with 1400 MTU, `kcp.output()` will be called around 7500 times and costs 117us for `time.Now()` in **every second**.
-
-3. Memory management
-
-Primary memory allocation are done from a global buffer pool xmit.Buf, in kcp-go, when we need to allocate some bytes, we can get from that pool, and a fixed-capacity 1500 bytes(mtuLimit) will be returned, the rx queue, tx queue and fec queue all receive bytes from there, and they will return the bytes to the pool after using to prevent unnecessary zer0ing of bytes. The pool mechanism maintained a high watermark for slice objects, these in-flight objects from the pool will survive from the perodical garbage collection, meanwhile the pool kept the ability to return the memory to runtime if in idle.
-
-4. Information security
-
-kcp-go is shipped with builtin packet encryption powered by various block encryption algorithms and works in [Cipher Feedback Mode](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Feedback_(CFB)), for each packet to be sent, the encryption process will start from encrypting a [nonce](https://en.wikipedia.org/wiki/Cryptographic_nonce) from the [system entropy](https://en.wikipedia.org/wiki//dev/random), so encryption to same plaintexts never leads to a same ciphertexts thereafter.
-
-The contents of the packets are completely anonymous with encryption, including the headers(FEC,KCP), checksums and contents. Note that, no matter which encryption method you choose on you upper layer, if you disable encryption, the transmit will be insecure somehow, since the header is ***PLAINTEXT*** to everyone it would be susceptible to header tampering, such as jamming the *sliding window size*, *round-trip time*, *FEC property* and *checksums*. ```AES-128``` is suggested for minimal encryption since modern CPUs are shipped with [AES-NI](https://en.wikipedia.org/wiki/AES_instruction_set) instructions and performs even better than `salsa20`(check the table above).
-
-Other possible attacks to kcp-go includes: a) [traffic analysis](https://en.wikipedia.org/wiki/Traffic_analysis), dataflow on specific websites may have pattern while interchanging data, but this type of eavesdropping has been mitigated by adapting [smux](https://github.com/xtaci/smux) to mix data streams so as to introduce noises, perfect solution to this has not appeared yet, theroretically by shuffling/mixing messages on larger scale network may mitigate this problem. b) [replay attack](https://en.wikipedia.org/wiki/Replay_attack), since the asymmetrical encryption has not been introduced into kcp-go for some reason, capturing the packets and replay them on a different machine is possible, (notice: hijacking the session and decrypting the contents is still *impossible*), so upper layers should contain a asymmetrical encryption system to guarantee the authenticity of each message(to process message exactly once), such as HTTPS/OpenSSL/LibreSSL, only by signing the requests with private keys can eliminate this type of attack.
-
-## Connection Termination
-
-Control messages like **SYN/FIN/RST** in TCP **are not defined** in KCP, you need some **keepalive/heartbeat mechanism** in the application-level. A real world example is to use some **multiplexing** protocol over session, such as [smux](https://github.com/xtaci/smux)(with embedded keepalive mechanism), see [kcptun](https://github.com/xtaci/kcptun) for example.
-
-## FAQ
-
-Q: I'm handling >5K connections on my server, the CPU utilization is so high.
-
-A: A standalone `agent` or `gate` server for running kcp-go is suggested, not only for CPU utilization, but also important to the **precision** of RTT measurements(timing) which indirectly affects retransmission. By increasing update `interval` with `SetNoDelay` like `conn.SetNoDelay(1, 40, 1, 1)` will dramatically reduce system load, but lower the performance.
-
-Q: When should I enable FEC?
-
-A: Forward error correction is critical to long-distance transmission, because a packet loss will lead to a huge penalty in time. And for the complicated packet routing network in modern world, round-trip time based loss check will not always be efficient, the big deviation of RTT samples in the long way usually leads to a larger RTO value in typical rtt estimator, which in other words, slows down the transmission.
-
-Q: Should I enable encryption?
-
-A: Yes, for the safety of protocol, even if the upper layer has encrypted.
-
-## Who is using this?
-
-1. https://github.com/xtaci/kcptun -- A Secure Tunnel Based On KCP over UDP.
-2. https://github.com/getlantern/lantern -- Lantern delivers fast access to the open Internet.
-3. https://github.com/smallnest/rpcx -- A RPC service framework based on net/rpc like alibaba Dubbo and weibo Motan.
-4. https://github.com/gonet2/agent -- A gateway for games with stream multiplexing.
-5. https://github.com/syncthing/syncthing -- Open Source Continuous File Synchronization.
-
-## Links
-
-1. https://github.com/xtaci/smux/ -- A Stream Multiplexing Library for golang with least memory
-1. https://github.com/xtaci/libkcp -- FEC enhanced KCP session library for iOS/Android in C++
-1. https://github.com/skywind3000/kcp -- A Fast and Reliable ARQ Protocol
-1. https://github.com/klauspost/reedsolomon -- Reed-Solomon Erasure Coding in Go
-
-## Consulting
-
-WeChat(付费技术咨询)
-
-
diff --git a/vendor/github.com/xtaci/kcp-go/batchconn.go b/vendor/github.com/xtaci/kcp-go/batchconn.go
deleted file mode 100644
index 6c30701..0000000
--- a/vendor/github.com/xtaci/kcp-go/batchconn.go
+++ /dev/null
@@ -1,12 +0,0 @@
-package kcp
-
-import "golang.org/x/net/ipv4"
-
-const (
- batchSize = 16
-)
-
-type batchConn interface {
- WriteBatch(ms []ipv4.Message, flags int) (int, error)
- ReadBatch(ms []ipv4.Message, flags int) (int, error)
-}
diff --git a/vendor/github.com/xtaci/kcp-go/crypt.go b/vendor/github.com/xtaci/kcp-go/crypt.go
deleted file mode 100644
index 958fdea..0000000
--- a/vendor/github.com/xtaci/kcp-go/crypt.go
+++ /dev/null
@@ -1,785 +0,0 @@
-package kcp
-
-import (
- "crypto/aes"
- "crypto/cipher"
- "crypto/des"
- "crypto/sha1"
-
- "github.com/templexxx/xor"
- "github.com/tjfoc/gmsm/sm4"
-
- "golang.org/x/crypto/blowfish"
- "golang.org/x/crypto/cast5"
- "golang.org/x/crypto/pbkdf2"
- "golang.org/x/crypto/salsa20"
- "golang.org/x/crypto/tea"
- "golang.org/x/crypto/twofish"
- "golang.org/x/crypto/xtea"
-)
-
-var (
- initialVector = []byte{167, 115, 79, 156, 18, 172, 27, 1, 164, 21, 242, 193, 252, 120, 230, 107}
- saltxor = `sH3CIVoF#rWLtJo6`
-)
-
-// BlockCrypt defines encryption/decryption methods for a given byte slice.
-// Notes on implementing: the data to be encrypted contains a builtin
-// nonce at the first 16 bytes
-type BlockCrypt interface {
- // Encrypt encrypts the whole block in src into dst.
- // Dst and src may point at the same memory.
- Encrypt(dst, src []byte)
-
- // Decrypt decrypts the whole block in src into dst.
- // Dst and src may point at the same memory.
- Decrypt(dst, src []byte)
-}
-
-type salsa20BlockCrypt struct {
- key [32]byte
-}
-
-// NewSalsa20BlockCrypt https://en.wikipedia.org/wiki/Salsa20
-func NewSalsa20BlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(salsa20BlockCrypt)
- copy(c.key[:], key)
- return c, nil
-}
-
-func (c *salsa20BlockCrypt) Encrypt(dst, src []byte) {
- salsa20.XORKeyStream(dst[8:], src[8:], src[:8], &c.key)
- copy(dst[:8], src[:8])
-}
-func (c *salsa20BlockCrypt) Decrypt(dst, src []byte) {
- salsa20.XORKeyStream(dst[8:], src[8:], src[:8], &c.key)
- copy(dst[:8], src[:8])
-}
-
-type sm4BlockCrypt struct {
- encbuf [sm4.BlockSize]byte
- decbuf [2 * sm4.BlockSize]byte
- block cipher.Block
-}
-
-// NewSM4BlockCrypt https://github.com/tjfoc/gmsm/tree/master/sm4
-func NewSM4BlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(sm4BlockCrypt)
- block, err := sm4.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *sm4BlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *sm4BlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type twofishBlockCrypt struct {
- encbuf [twofish.BlockSize]byte
- decbuf [2 * twofish.BlockSize]byte
- block cipher.Block
-}
-
-// NewTwofishBlockCrypt https://en.wikipedia.org/wiki/Twofish
-func NewTwofishBlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(twofishBlockCrypt)
- block, err := twofish.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *twofishBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *twofishBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type tripleDESBlockCrypt struct {
- encbuf [des.BlockSize]byte
- decbuf [2 * des.BlockSize]byte
- block cipher.Block
-}
-
-// NewTripleDESBlockCrypt https://en.wikipedia.org/wiki/Triple_DES
-func NewTripleDESBlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(tripleDESBlockCrypt)
- block, err := des.NewTripleDESCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *tripleDESBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *tripleDESBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type cast5BlockCrypt struct {
- encbuf [cast5.BlockSize]byte
- decbuf [2 * cast5.BlockSize]byte
- block cipher.Block
-}
-
-// NewCast5BlockCrypt https://en.wikipedia.org/wiki/CAST-128
-func NewCast5BlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(cast5BlockCrypt)
- block, err := cast5.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *cast5BlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *cast5BlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type blowfishBlockCrypt struct {
- encbuf [blowfish.BlockSize]byte
- decbuf [2 * blowfish.BlockSize]byte
- block cipher.Block
-}
-
-// NewBlowfishBlockCrypt https://en.wikipedia.org/wiki/Blowfish_(cipher)
-func NewBlowfishBlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(blowfishBlockCrypt)
- block, err := blowfish.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *blowfishBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *blowfishBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type aesBlockCrypt struct {
- encbuf [aes.BlockSize]byte
- decbuf [2 * aes.BlockSize]byte
- block cipher.Block
-}
-
-// NewAESBlockCrypt https://en.wikipedia.org/wiki/Advanced_Encryption_Standard
-func NewAESBlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(aesBlockCrypt)
- block, err := aes.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *aesBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *aesBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type teaBlockCrypt struct {
- encbuf [tea.BlockSize]byte
- decbuf [2 * tea.BlockSize]byte
- block cipher.Block
-}
-
-// NewTEABlockCrypt https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm
-func NewTEABlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(teaBlockCrypt)
- block, err := tea.NewCipherWithRounds(key, 16)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *teaBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *teaBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type xteaBlockCrypt struct {
- encbuf [xtea.BlockSize]byte
- decbuf [2 * xtea.BlockSize]byte
- block cipher.Block
-}
-
-// NewXTEABlockCrypt https://en.wikipedia.org/wiki/XTEA
-func NewXTEABlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(xteaBlockCrypt)
- block, err := xtea.NewCipher(key)
- if err != nil {
- return nil, err
- }
- c.block = block
- return c, nil
-}
-
-func (c *xteaBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) }
-func (c *xteaBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) }
-
-type simpleXORBlockCrypt struct {
- xortbl []byte
-}
-
-// NewSimpleXORBlockCrypt simple xor with key expanding
-func NewSimpleXORBlockCrypt(key []byte) (BlockCrypt, error) {
- c := new(simpleXORBlockCrypt)
- c.xortbl = pbkdf2.Key(key, []byte(saltxor), 32, mtuLimit, sha1.New)
- return c, nil
-}
-
-func (c *simpleXORBlockCrypt) Encrypt(dst, src []byte) { xor.Bytes(dst, src, c.xortbl) }
-func (c *simpleXORBlockCrypt) Decrypt(dst, src []byte) { xor.Bytes(dst, src, c.xortbl) }
-
-type noneBlockCrypt struct{}
-
-// NewNoneBlockCrypt does nothing but copying
-func NewNoneBlockCrypt(key []byte) (BlockCrypt, error) {
- return new(noneBlockCrypt), nil
-}
-
-func (c *noneBlockCrypt) Encrypt(dst, src []byte) { copy(dst, src) }
-func (c *noneBlockCrypt) Decrypt(dst, src []byte) { copy(dst, src) }
-
-// packet encryption with local CFB mode
-func encrypt(block cipher.Block, dst, src, buf []byte) {
- switch block.BlockSize() {
- case 8:
- encrypt8(block, dst, src, buf)
- case 16:
- encrypt16(block, dst, src, buf)
- default:
- encryptVariant(block, dst, src, buf)
- }
-}
-
-// optimized encryption for the ciphers which works in 8-bytes
-func encrypt8(block cipher.Block, dst, src, buf []byte) {
- tbl := buf[:8]
- block.Encrypt(tbl, initialVector)
- n := len(src) / 8
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- s := src[base:][0:64]
- d := dst[base:][0:64]
- // 1
- xor.BytesSrc1(d[0:8], s[0:8], tbl)
- block.Encrypt(tbl, d[0:8])
- // 2
- xor.BytesSrc1(d[8:16], s[8:16], tbl)
- block.Encrypt(tbl, d[8:16])
- // 3
- xor.BytesSrc1(d[16:24], s[16:24], tbl)
- block.Encrypt(tbl, d[16:24])
- // 4
- xor.BytesSrc1(d[24:32], s[24:32], tbl)
- block.Encrypt(tbl, d[24:32])
- // 5
- xor.BytesSrc1(d[32:40], s[32:40], tbl)
- block.Encrypt(tbl, d[32:40])
- // 6
- xor.BytesSrc1(d[40:48], s[40:48], tbl)
- block.Encrypt(tbl, d[40:48])
- // 7
- xor.BytesSrc1(d[48:56], s[48:56], tbl)
- block.Encrypt(tbl, d[48:56])
- // 8
- xor.BytesSrc1(d[56:64], s[56:64], tbl)
- block.Encrypt(tbl, d[56:64])
- base += 64
- }
-
- switch left {
- case 7:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 6:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 5:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 4:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 3:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 2:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 1:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 8
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
-
-// optimized encryption for the ciphers which works in 16-bytes
-func encrypt16(block cipher.Block, dst, src, buf []byte) {
- tbl := buf[:16]
- block.Encrypt(tbl, initialVector)
- n := len(src) / 16
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- s := src[base:][0:128]
- d := dst[base:][0:128]
- // 1
- xor.BytesSrc1(d[0:16], s[0:16], tbl)
- block.Encrypt(tbl, d[0:16])
- // 2
- xor.BytesSrc1(d[16:32], s[16:32], tbl)
- block.Encrypt(tbl, d[16:32])
- // 3
- xor.BytesSrc1(d[32:48], s[32:48], tbl)
- block.Encrypt(tbl, d[32:48])
- // 4
- xor.BytesSrc1(d[48:64], s[48:64], tbl)
- block.Encrypt(tbl, d[48:64])
- // 5
- xor.BytesSrc1(d[64:80], s[64:80], tbl)
- block.Encrypt(tbl, d[64:80])
- // 6
- xor.BytesSrc1(d[80:96], s[80:96], tbl)
- block.Encrypt(tbl, d[80:96])
- // 7
- xor.BytesSrc1(d[96:112], s[96:112], tbl)
- block.Encrypt(tbl, d[96:112])
- // 8
- xor.BytesSrc1(d[112:128], s[112:128], tbl)
- block.Encrypt(tbl, d[112:128])
- base += 128
- }
-
- switch left {
- case 7:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 6:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 5:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 4:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 3:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 2:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 1:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += 16
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
-
-func encryptVariant(block cipher.Block, dst, src, buf []byte) {
- blocksize := block.BlockSize()
- tbl := buf[:blocksize]
- block.Encrypt(tbl, initialVector)
- n := len(src) / blocksize
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- // 1
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 2
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 3
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 4
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 5
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 6
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 7
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
-
- // 8
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- }
-
- switch left {
- case 7:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 6:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 5:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 4:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 3:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 2:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 1:
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- block.Encrypt(tbl, dst[base:])
- base += blocksize
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
-
-// decryption
-func decrypt(block cipher.Block, dst, src, buf []byte) {
- switch block.BlockSize() {
- case 8:
- decrypt8(block, dst, src, buf)
- case 16:
- decrypt16(block, dst, src, buf)
- default:
- decryptVariant(block, dst, src, buf)
- }
-}
-
-func decrypt8(block cipher.Block, dst, src, buf []byte) {
- tbl := buf[0:8]
- next := buf[8:16]
- block.Encrypt(tbl, initialVector)
- n := len(src) / 8
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- s := src[base:][0:64]
- d := dst[base:][0:64]
- // 1
- block.Encrypt(next, s[0:8])
- xor.BytesSrc1(d[0:8], s[0:8], tbl)
- // 2
- block.Encrypt(tbl, s[8:16])
- xor.BytesSrc1(d[8:16], s[8:16], next)
- // 3
- block.Encrypt(next, s[16:24])
- xor.BytesSrc1(d[16:24], s[16:24], tbl)
- // 4
- block.Encrypt(tbl, s[24:32])
- xor.BytesSrc1(d[24:32], s[24:32], next)
- // 5
- block.Encrypt(next, s[32:40])
- xor.BytesSrc1(d[32:40], s[32:40], tbl)
- // 6
- block.Encrypt(tbl, s[40:48])
- xor.BytesSrc1(d[40:48], s[40:48], next)
- // 7
- block.Encrypt(next, s[48:56])
- xor.BytesSrc1(d[48:56], s[48:56], tbl)
- // 8
- block.Encrypt(tbl, s[56:64])
- xor.BytesSrc1(d[56:64], s[56:64], next)
- base += 64
- }
-
- switch left {
- case 7:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 6:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 5:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 4:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 3:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 2:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 1:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 8
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
-
-func decrypt16(block cipher.Block, dst, src, buf []byte) {
- tbl := buf[0:16]
- next := buf[16:32]
- block.Encrypt(tbl, initialVector)
- n := len(src) / 16
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- s := src[base:][0:128]
- d := dst[base:][0:128]
- // 1
- block.Encrypt(next, s[0:16])
- xor.BytesSrc1(d[0:16], s[0:16], tbl)
- // 2
- block.Encrypt(tbl, s[16:32])
- xor.BytesSrc1(d[16:32], s[16:32], next)
- // 3
- block.Encrypt(next, s[32:48])
- xor.BytesSrc1(d[32:48], s[32:48], tbl)
- // 4
- block.Encrypt(tbl, s[48:64])
- xor.BytesSrc1(d[48:64], s[48:64], next)
- // 5
- block.Encrypt(next, s[64:80])
- xor.BytesSrc1(d[64:80], s[64:80], tbl)
- // 6
- block.Encrypt(tbl, s[80:96])
- xor.BytesSrc1(d[80:96], s[80:96], next)
- // 7
- block.Encrypt(next, s[96:112])
- xor.BytesSrc1(d[96:112], s[96:112], tbl)
- // 8
- block.Encrypt(tbl, s[112:128])
- xor.BytesSrc1(d[112:128], s[112:128], next)
- base += 128
- }
-
- switch left {
- case 7:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 6:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 5:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 4:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 3:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 2:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 1:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += 16
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
-
-func decryptVariant(block cipher.Block, dst, src, buf []byte) {
- blocksize := block.BlockSize()
- tbl := buf[:blocksize]
- next := buf[blocksize:]
- block.Encrypt(tbl, initialVector)
- n := len(src) / blocksize
- base := 0
- repeat := n / 8
- left := n % 8
- for i := 0; i < repeat; i++ {
- // 1
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- base += blocksize
-
- // 2
- block.Encrypt(tbl, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], next)
- base += blocksize
-
- // 3
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- base += blocksize
-
- // 4
- block.Encrypt(tbl, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], next)
- base += blocksize
-
- // 5
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- base += blocksize
-
- // 6
- block.Encrypt(tbl, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], next)
- base += blocksize
-
- // 7
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- base += blocksize
-
- // 8
- block.Encrypt(tbl, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], next)
- base += blocksize
- }
-
- switch left {
- case 7:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 6:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 5:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 4:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 3:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 2:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 1:
- block.Encrypt(next, src[base:])
- xor.BytesSrc1(dst[base:], src[base:], tbl)
- tbl, next = next, tbl
- base += blocksize
- fallthrough
- case 0:
- xor.BytesSrc0(dst[base:], src[base:], tbl)
- }
-}
diff --git a/vendor/github.com/xtaci/kcp-go/donate.png b/vendor/github.com/xtaci/kcp-go/donate.png
deleted file mode 100644
index 0f353d9..0000000
Binary files a/vendor/github.com/xtaci/kcp-go/donate.png and /dev/null differ
diff --git a/vendor/github.com/xtaci/kcp-go/entropy.go b/vendor/github.com/xtaci/kcp-go/entropy.go
deleted file mode 100644
index 156c1cd..0000000
--- a/vendor/github.com/xtaci/kcp-go/entropy.go
+++ /dev/null
@@ -1,52 +0,0 @@
-package kcp
-
-import (
- "crypto/aes"
- "crypto/cipher"
- "crypto/md5"
- "crypto/rand"
- "io"
-)
-
-// Entropy defines a entropy source
-type Entropy interface {
- Init()
- Fill(nonce []byte)
-}
-
-// nonceMD5 nonce generator for packet header
-type nonceMD5 struct {
- seed [md5.Size]byte
-}
-
-func (n *nonceMD5) Init() { /*nothing required*/ }
-
-func (n *nonceMD5) Fill(nonce []byte) {
- if n.seed[0] == 0 { // entropy update
- io.ReadFull(rand.Reader, n.seed[:])
- }
- n.seed = md5.Sum(n.seed[:])
- copy(nonce, n.seed[:])
-}
-
-// nonceAES128 nonce generator for packet headers
-type nonceAES128 struct {
- seed [aes.BlockSize]byte
- block cipher.Block
-}
-
-func (n *nonceAES128) Init() {
- var key [16]byte //aes-128
- io.ReadFull(rand.Reader, key[:])
- io.ReadFull(rand.Reader, n.seed[:])
- block, _ := aes.NewCipher(key[:])
- n.block = block
-}
-
-func (n *nonceAES128) Fill(nonce []byte) {
- if n.seed[0] == 0 { // entropy update
- io.ReadFull(rand.Reader, n.seed[:])
- }
- n.block.Encrypt(n.seed[:], n.seed[:])
- copy(nonce, n.seed[:])
-}
diff --git a/vendor/github.com/xtaci/kcp-go/fec.go b/vendor/github.com/xtaci/kcp-go/fec.go
deleted file mode 100644
index 97cd40b..0000000
--- a/vendor/github.com/xtaci/kcp-go/fec.go
+++ /dev/null
@@ -1,337 +0,0 @@
-package kcp
-
-import (
- "encoding/binary"
- "sync/atomic"
-
- "github.com/klauspost/reedsolomon"
-)
-
-const (
- fecHeaderSize = 6
- fecHeaderSizePlus2 = fecHeaderSize + 2 // plus 2B data size
- typeData = 0xf1
- typeParity = 0xf2
- fecExpire = 60000
-)
-
-// fecPacket is a decoded FEC packet
-type fecPacket []byte
-
-func (bts fecPacket) seqid() uint32 { return binary.LittleEndian.Uint32(bts) }
-func (bts fecPacket) flag() uint16 { return binary.LittleEndian.Uint16(bts[4:]) }
-func (bts fecPacket) data() []byte { return bts[6:] }
-
-// fecElement has auxcilliary time field
-type fecElement struct {
- fecPacket
- ts uint32
-}
-
-// fecDecoder for decoding incoming packets
-type fecDecoder struct {
- rxlimit int // queue size limit
- dataShards int
- parityShards int
- shardSize int
- rx []fecElement // ordered receive queue
-
- // caches
- decodeCache [][]byte
- flagCache []bool
-
- // zeros
- zeros []byte
-
- // RS decoder
- codec reedsolomon.Encoder
-}
-
-func newFECDecoder(rxlimit, dataShards, parityShards int) *fecDecoder {
- if dataShards <= 0 || parityShards <= 0 {
- return nil
- }
- if rxlimit < dataShards+parityShards {
- return nil
- }
-
- dec := new(fecDecoder)
- dec.rxlimit = rxlimit
- dec.dataShards = dataShards
- dec.parityShards = parityShards
- dec.shardSize = dataShards + parityShards
- codec, err := reedsolomon.New(dataShards, parityShards)
- if err != nil {
- return nil
- }
- dec.codec = codec
- dec.decodeCache = make([][]byte, dec.shardSize)
- dec.flagCache = make([]bool, dec.shardSize)
- dec.zeros = make([]byte, mtuLimit)
- return dec
-}
-
-// decode a fec packet
-func (dec *fecDecoder) decode(in fecPacket) (recovered [][]byte) {
- // insertion
- n := len(dec.rx) - 1
- insertIdx := 0
- for i := n; i >= 0; i-- {
- if in.seqid() == dec.rx[i].seqid() { // de-duplicate
- return nil
- } else if _itimediff(in.seqid(), dec.rx[i].seqid()) > 0 { // insertion
- insertIdx = i + 1
- break
- }
- }
-
- // make a copy
- pkt := fecPacket(xmitBuf.Get().([]byte)[:len(in)])
- copy(pkt, in)
- elem := fecElement{pkt, currentMs()}
-
- // insert into ordered rx queue
- if insertIdx == n+1 {
- dec.rx = append(dec.rx, elem)
- } else {
- dec.rx = append(dec.rx, fecElement{})
- copy(dec.rx[insertIdx+1:], dec.rx[insertIdx:]) // shift right
- dec.rx[insertIdx] = elem
- }
-
- // shard range for current packet
- shardBegin := pkt.seqid() - pkt.seqid()%uint32(dec.shardSize)
- shardEnd := shardBegin + uint32(dec.shardSize) - 1
-
- // max search range in ordered queue for current shard
- searchBegin := insertIdx - int(pkt.seqid()%uint32(dec.shardSize))
- if searchBegin < 0 {
- searchBegin = 0
- }
- searchEnd := searchBegin + dec.shardSize - 1
- if searchEnd >= len(dec.rx) {
- searchEnd = len(dec.rx) - 1
- }
-
- // re-construct datashards
- if searchEnd-searchBegin+1 >= dec.dataShards {
- var numshard, numDataShard, first, maxlen int
-
- // zero caches
- shards := dec.decodeCache
- shardsflag := dec.flagCache
- for k := range dec.decodeCache {
- shards[k] = nil
- shardsflag[k] = false
- }
-
- // shard assembly
- for i := searchBegin; i <= searchEnd; i++ {
- seqid := dec.rx[i].seqid()
- if _itimediff(seqid, shardEnd) > 0 {
- break
- } else if _itimediff(seqid, shardBegin) >= 0 {
- shards[seqid%uint32(dec.shardSize)] = dec.rx[i].data()
- shardsflag[seqid%uint32(dec.shardSize)] = true
- numshard++
- if dec.rx[i].flag() == typeData {
- numDataShard++
- }
- if numshard == 1 {
- first = i
- }
- if len(dec.rx[i].data()) > maxlen {
- maxlen = len(dec.rx[i].data())
- }
- }
- }
-
- if numDataShard == dec.dataShards {
- // case 1: no loss on data shards
- dec.rx = dec.freeRange(first, numshard, dec.rx)
- } else if numshard >= dec.dataShards {
- // case 2: loss on data shards, but it's recoverable from parity shards
- for k := range shards {
- if shards[k] != nil {
- dlen := len(shards[k])
- shards[k] = shards[k][:maxlen]
- copy(shards[k][dlen:], dec.zeros)
- } else if k < dec.dataShards {
- shards[k] = xmitBuf.Get().([]byte)[:0]
- }
- }
- if err := dec.codec.ReconstructData(shards); err == nil {
- for k := range shards[:dec.dataShards] {
- if !shardsflag[k] {
- // recovered data should be recycled
- recovered = append(recovered, shards[k])
- }
- }
- }
- dec.rx = dec.freeRange(first, numshard, dec.rx)
- }
- }
-
- // keep rxlimit
- if len(dec.rx) > dec.rxlimit {
- if dec.rx[0].flag() == typeData { // track the unrecoverable data
- atomic.AddUint64(&DefaultSnmp.FECShortShards, 1)
- }
- dec.rx = dec.freeRange(0, 1, dec.rx)
- }
-
- // timeout policy
- current := currentMs()
- numExpired := 0
- for k := range dec.rx {
- if _itimediff(current, dec.rx[k].ts) > fecExpire {
- numExpired++
- continue
- }
- break
- }
- if numExpired > 0 {
- dec.rx = dec.freeRange(0, numExpired, dec.rx)
- }
- return
-}
-
-// free a range of fecPacket
-func (dec *fecDecoder) freeRange(first, n int, q []fecElement) []fecElement {
- for i := first; i < first+n; i++ { // recycle buffer
- xmitBuf.Put([]byte(q[i].fecPacket))
- }
-
- if first == 0 && n < cap(q)/2 {
- return q[n:]
- }
- copy(q[first:], q[first+n:])
- return q[:len(q)-n]
-}
-
-// release all segments back to xmitBuf
-func (dec *fecDecoder) release() {
- if n := len(dec.rx); n > 0 {
- dec.rx = dec.freeRange(0, n, dec.rx)
- }
-}
-
-type (
- // fecEncoder for encoding outgoing packets
- fecEncoder struct {
- dataShards int
- parityShards int
- shardSize int
- paws uint32 // Protect Against Wrapped Sequence numbers
- next uint32 // next seqid
-
- shardCount int // count the number of datashards collected
- maxSize int // track maximum data length in datashard
-
- headerOffset int // FEC header offset
- payloadOffset int // FEC payload offset
-
- // caches
- shardCache [][]byte
- encodeCache [][]byte
-
- // zeros
- zeros []byte
-
- // RS encoder
- codec reedsolomon.Encoder
- }
-)
-
-func newFECEncoder(dataShards, parityShards, offset int) *fecEncoder {
- if dataShards <= 0 || parityShards <= 0 {
- return nil
- }
- enc := new(fecEncoder)
- enc.dataShards = dataShards
- enc.parityShards = parityShards
- enc.shardSize = dataShards + parityShards
- enc.paws = 0xffffffff / uint32(enc.shardSize) * uint32(enc.shardSize)
- enc.headerOffset = offset
- enc.payloadOffset = enc.headerOffset + fecHeaderSize
-
- codec, err := reedsolomon.New(dataShards, parityShards)
- if err != nil {
- return nil
- }
- enc.codec = codec
-
- // caches
- enc.encodeCache = make([][]byte, enc.shardSize)
- enc.shardCache = make([][]byte, enc.shardSize)
- for k := range enc.shardCache {
- enc.shardCache[k] = make([]byte, mtuLimit)
- }
- enc.zeros = make([]byte, mtuLimit)
- return enc
-}
-
-// encodes the packet, outputs parity shards if we have collected quorum datashards
-// notice: the contents of 'ps' will be re-written in successive calling
-func (enc *fecEncoder) encode(b []byte) (ps [][]byte) {
- // The header format:
- // | FEC SEQID(4B) | FEC TYPE(2B) | SIZE (2B) | PAYLOAD(SIZE-2) |
- // |<-headerOffset |<-payloadOffset
- enc.markData(b[enc.headerOffset:])
- binary.LittleEndian.PutUint16(b[enc.payloadOffset:], uint16(len(b[enc.payloadOffset:])))
-
- // copy data from payloadOffset to fec shard cache
- sz := len(b)
- enc.shardCache[enc.shardCount] = enc.shardCache[enc.shardCount][:sz]
- copy(enc.shardCache[enc.shardCount][enc.payloadOffset:], b[enc.payloadOffset:])
- enc.shardCount++
-
- // track max datashard length
- if sz > enc.maxSize {
- enc.maxSize = sz
- }
-
- // Generation of Reed-Solomon Erasure Code
- if enc.shardCount == enc.dataShards {
- // fill '0' into the tail of each datashard
- for i := 0; i < enc.dataShards; i++ {
- shard := enc.shardCache[i]
- slen := len(shard)
- copy(shard[slen:enc.maxSize], enc.zeros)
- }
-
- // construct equal-sized slice with stripped header
- cache := enc.encodeCache
- for k := range cache {
- cache[k] = enc.shardCache[k][enc.payloadOffset:enc.maxSize]
- }
-
- // encoding
- if err := enc.codec.Encode(cache); err == nil {
- ps = enc.shardCache[enc.dataShards:]
- for k := range ps {
- enc.markParity(ps[k][enc.headerOffset:])
- ps[k] = ps[k][:enc.maxSize]
- }
- }
-
- // counters resetting
- enc.shardCount = 0
- enc.maxSize = 0
- }
-
- return
-}
-
-func (enc *fecEncoder) markData(data []byte) {
- binary.LittleEndian.PutUint32(data, enc.next)
- binary.LittleEndian.PutUint16(data[4:], typeData)
- enc.next++
-}
-
-func (enc *fecEncoder) markParity(data []byte) {
- binary.LittleEndian.PutUint32(data, enc.next)
- binary.LittleEndian.PutUint16(data[4:], typeParity)
- // sequence wrap will only happen at parity shard
- enc.next = (enc.next + 1) % enc.paws
-}
diff --git a/vendor/github.com/xtaci/kcp-go/flame.png b/vendor/github.com/xtaci/kcp-go/flame.png
deleted file mode 100644
index 672f649..0000000
Binary files a/vendor/github.com/xtaci/kcp-go/flame.png and /dev/null differ
diff --git a/vendor/github.com/xtaci/kcp-go/frame.png b/vendor/github.com/xtaci/kcp-go/frame.png
deleted file mode 100644
index 0b0aefd..0000000
Binary files a/vendor/github.com/xtaci/kcp-go/frame.png and /dev/null differ
diff --git a/vendor/github.com/xtaci/kcp-go/kcp-go.png b/vendor/github.com/xtaci/kcp-go/kcp-go.png
deleted file mode 100644
index 151b7c4..0000000
Binary files a/vendor/github.com/xtaci/kcp-go/kcp-go.png and /dev/null differ
diff --git a/vendor/github.com/xtaci/kcp-go/kcp.go b/vendor/github.com/xtaci/kcp-go/kcp.go
deleted file mode 100644
index 342e69a..0000000
--- a/vendor/github.com/xtaci/kcp-go/kcp.go
+++ /dev/null
@@ -1,1070 +0,0 @@
-package kcp
-
-import (
- "encoding/binary"
- "sync/atomic"
- "time"
-)
-
-const (
- IKCP_RTO_NDL = 30 // no delay min rto
- IKCP_RTO_MIN = 100 // normal min rto
- IKCP_RTO_DEF = 200
- IKCP_RTO_MAX = 60000
- IKCP_CMD_PUSH = 81 // cmd: push data
- IKCP_CMD_ACK = 82 // cmd: ack
- IKCP_CMD_WASK = 83 // cmd: window probe (ask)
- IKCP_CMD_WINS = 84 // cmd: window size (tell)
- IKCP_ASK_SEND = 1 // need to send IKCP_CMD_WASK
- IKCP_ASK_TELL = 2 // need to send IKCP_CMD_WINS
- IKCP_WND_SND = 32
- IKCP_WND_RCV = 32
- IKCP_MTU_DEF = 1400
- IKCP_ACK_FAST = 3
- IKCP_INTERVAL = 100
- IKCP_OVERHEAD = 24
- IKCP_DEADLINK = 20
- IKCP_THRESH_INIT = 2
- IKCP_THRESH_MIN = 2
- IKCP_PROBE_INIT = 7000 // 7 secs to probe window size
- IKCP_PROBE_LIMIT = 120000 // up to 120 secs to probe window
- IKCP_SN_OFFSET = 12
-)
-
-// monotonic reference time point
-var refTime time.Time = time.Now()
-
-// currentMs returns current elasped monotonic milliseconds since program startup
-func currentMs() uint32 { return uint32(time.Now().Sub(refTime) / time.Millisecond) }
-
-// output_callback is a prototype which ought capture conn and call conn.Write
-type output_callback func(buf []byte, size int)
-
-/* encode 8 bits unsigned int */
-func ikcp_encode8u(p []byte, c byte) []byte {
- p[0] = c
- return p[1:]
-}
-
-/* decode 8 bits unsigned int */
-func ikcp_decode8u(p []byte, c *byte) []byte {
- *c = p[0]
- return p[1:]
-}
-
-/* encode 16 bits unsigned int (lsb) */
-func ikcp_encode16u(p []byte, w uint16) []byte {
- binary.LittleEndian.PutUint16(p, w)
- return p[2:]
-}
-
-/* decode 16 bits unsigned int (lsb) */
-func ikcp_decode16u(p []byte, w *uint16) []byte {
- *w = binary.LittleEndian.Uint16(p)
- return p[2:]
-}
-
-/* encode 32 bits unsigned int (lsb) */
-func ikcp_encode32u(p []byte, l uint32) []byte {
- binary.LittleEndian.PutUint32(p, l)
- return p[4:]
-}
-
-/* decode 32 bits unsigned int (lsb) */
-func ikcp_decode32u(p []byte, l *uint32) []byte {
- *l = binary.LittleEndian.Uint32(p)
- return p[4:]
-}
-
-func _imin_(a, b uint32) uint32 {
- if a <= b {
- return a
- }
- return b
-}
-
-func _imax_(a, b uint32) uint32 {
- if a >= b {
- return a
- }
- return b
-}
-
-func _ibound_(lower, middle, upper uint32) uint32 {
- return _imin_(_imax_(lower, middle), upper)
-}
-
-func _itimediff(later, earlier uint32) int32 {
- return (int32)(later - earlier)
-}
-
-// segment defines a KCP segment
-type segment struct {
- conv uint32
- cmd uint8
- frg uint8
- wnd uint16
- ts uint32
- sn uint32
- una uint32
- rto uint32
- xmit uint32
- resendts uint32
- fastack uint32
- acked uint32 // mark if the seg has acked
- data []byte
-}
-
-// encode a segment into buffer
-func (seg *segment) encode(ptr []byte) []byte {
- ptr = ikcp_encode32u(ptr, seg.conv)
- ptr = ikcp_encode8u(ptr, seg.cmd)
- ptr = ikcp_encode8u(ptr, seg.frg)
- ptr = ikcp_encode16u(ptr, seg.wnd)
- ptr = ikcp_encode32u(ptr, seg.ts)
- ptr = ikcp_encode32u(ptr, seg.sn)
- ptr = ikcp_encode32u(ptr, seg.una)
- ptr = ikcp_encode32u(ptr, uint32(len(seg.data)))
- atomic.AddUint64(&DefaultSnmp.OutSegs, 1)
- return ptr
-}
-
-// KCP defines a single KCP connection
-type KCP struct {
- conv, mtu, mss, state uint32
- snd_una, snd_nxt, rcv_nxt uint32
- ssthresh uint32
- rx_rttvar, rx_srtt int32
- rx_rto, rx_minrto uint32
- snd_wnd, rcv_wnd, rmt_wnd, cwnd, probe uint32
- interval, ts_flush uint32
- nodelay, updated uint32
- ts_probe, probe_wait uint32
- dead_link, incr uint32
-
- fastresend int32
- nocwnd, stream int32
-
- snd_queue []segment
- rcv_queue []segment
- snd_buf []segment
- rcv_buf []segment
-
- acklist []ackItem
-
- buffer []byte
- reserved int
- output output_callback
-}
-
-type ackItem struct {
- sn uint32
- ts uint32
-}
-
-// NewKCP create a new kcp state machine
-//
-// 'conv' must be equal in the connection peers, or else data will be silently rejected.
-//
-// 'output' function will be called whenever these is data to be sent on wire.
-func NewKCP(conv uint32, output output_callback) *KCP {
- kcp := new(KCP)
- kcp.conv = conv
- kcp.snd_wnd = IKCP_WND_SND
- kcp.rcv_wnd = IKCP_WND_RCV
- kcp.rmt_wnd = IKCP_WND_RCV
- kcp.mtu = IKCP_MTU_DEF
- kcp.mss = kcp.mtu - IKCP_OVERHEAD
- kcp.buffer = make([]byte, kcp.mtu)
- kcp.rx_rto = IKCP_RTO_DEF
- kcp.rx_minrto = IKCP_RTO_MIN
- kcp.interval = IKCP_INTERVAL
- kcp.ts_flush = IKCP_INTERVAL
- kcp.ssthresh = IKCP_THRESH_INIT
- kcp.dead_link = IKCP_DEADLINK
- kcp.output = output
- return kcp
-}
-
-// newSegment creates a KCP segment
-func (kcp *KCP) newSegment(size int) (seg segment) {
- seg.data = xmitBuf.Get().([]byte)[:size]
- return
-}
-
-// delSegment recycles a KCP segment
-func (kcp *KCP) delSegment(seg *segment) {
- if seg.data != nil {
- xmitBuf.Put(seg.data)
- seg.data = nil
- }
-}
-
-// ReserveBytes keeps n bytes untouched from the beginning of the buffer,
-// the output_callback function should be aware of this.
-//
-// Return false if n >= mss
-func (kcp *KCP) ReserveBytes(n int) bool {
- if n >= int(kcp.mtu-IKCP_OVERHEAD) || n < 0 {
- return false
- }
- kcp.reserved = n
- kcp.mss = kcp.mtu - IKCP_OVERHEAD - uint32(n)
- return true
-}
-
-// PeekSize checks the size of next message in the recv queue
-func (kcp *KCP) PeekSize() (length int) {
- if len(kcp.rcv_queue) == 0 {
- return -1
- }
-
- seg := &kcp.rcv_queue[0]
- if seg.frg == 0 {
- return len(seg.data)
- }
-
- if len(kcp.rcv_queue) < int(seg.frg+1) {
- return -1
- }
-
- for k := range kcp.rcv_queue {
- seg := &kcp.rcv_queue[k]
- length += len(seg.data)
- if seg.frg == 0 {
- break
- }
- }
- return
-}
-
-// Receive data from kcp state machine
-//
-// Return number of bytes read.
-//
-// Return -1 when there is no readable data.
-//
-// Return -2 if len(buffer) is smaller than kcp.PeekSize().
-func (kcp *KCP) Recv(buffer []byte) (n int) {
- peeksize := kcp.PeekSize()
- if peeksize < 0 {
- return -1
- }
-
- if peeksize > len(buffer) {
- return -2
- }
-
- var fast_recover bool
- if len(kcp.rcv_queue) >= int(kcp.rcv_wnd) {
- fast_recover = true
- }
-
- // merge fragment
- count := 0
- for k := range kcp.rcv_queue {
- seg := &kcp.rcv_queue[k]
- copy(buffer, seg.data)
- buffer = buffer[len(seg.data):]
- n += len(seg.data)
- count++
- kcp.delSegment(seg)
- if seg.frg == 0 {
- break
- }
- }
- if count > 0 {
- kcp.rcv_queue = kcp.remove_front(kcp.rcv_queue, count)
- }
-
- // move available data from rcv_buf -> rcv_queue
- count = 0
- for k := range kcp.rcv_buf {
- seg := &kcp.rcv_buf[k]
- if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue)+count < int(kcp.rcv_wnd) {
- kcp.rcv_nxt++
- count++
- } else {
- break
- }
- }
-
- if count > 0 {
- kcp.rcv_queue = append(kcp.rcv_queue, kcp.rcv_buf[:count]...)
- kcp.rcv_buf = kcp.remove_front(kcp.rcv_buf, count)
- }
-
- // fast recover
- if len(kcp.rcv_queue) < int(kcp.rcv_wnd) && fast_recover {
- // ready to send back IKCP_CMD_WINS in ikcp_flush
- // tell remote my window size
- kcp.probe |= IKCP_ASK_TELL
- }
- return
-}
-
-// Send is user/upper level send, returns below zero for error
-func (kcp *KCP) Send(buffer []byte) int {
- var count int
- if len(buffer) == 0 {
- return -1
- }
-
- // append to previous segment in streaming mode (if possible)
- if kcp.stream != 0 {
- n := len(kcp.snd_queue)
- if n > 0 {
- seg := &kcp.snd_queue[n-1]
- if len(seg.data) < int(kcp.mss) {
- capacity := int(kcp.mss) - len(seg.data)
- extend := capacity
- if len(buffer) < capacity {
- extend = len(buffer)
- }
-
- // grow slice, the underlying cap is guaranteed to
- // be larger than kcp.mss
- oldlen := len(seg.data)
- seg.data = seg.data[:oldlen+extend]
- copy(seg.data[oldlen:], buffer)
- buffer = buffer[extend:]
- }
- }
-
- if len(buffer) == 0 {
- return 0
- }
- }
-
- if len(buffer) <= int(kcp.mss) {
- count = 1
- } else {
- count = (len(buffer) + int(kcp.mss) - 1) / int(kcp.mss)
- }
-
- if count > 255 {
- return -2
- }
-
- if count == 0 {
- count = 1
- }
-
- for i := 0; i < count; i++ {
- var size int
- if len(buffer) > int(kcp.mss) {
- size = int(kcp.mss)
- } else {
- size = len(buffer)
- }
- seg := kcp.newSegment(size)
- copy(seg.data, buffer[:size])
- if kcp.stream == 0 { // message mode
- seg.frg = uint8(count - i - 1)
- } else { // stream mode
- seg.frg = 0
- }
- kcp.snd_queue = append(kcp.snd_queue, seg)
- buffer = buffer[size:]
- }
- return 0
-}
-
-func (kcp *KCP) update_ack(rtt int32) {
- // https://tools.ietf.org/html/rfc6298
- var rto uint32
- if kcp.rx_srtt == 0 {
- kcp.rx_srtt = rtt
- kcp.rx_rttvar = rtt >> 1
- } else {
- delta := rtt - kcp.rx_srtt
- kcp.rx_srtt += delta >> 3
- if delta < 0 {
- delta = -delta
- }
- if rtt < kcp.rx_srtt-kcp.rx_rttvar {
- // if the new RTT sample is below the bottom of the range of
- // what an RTT measurement is expected to be.
- // give an 8x reduced weight versus its normal weighting
- kcp.rx_rttvar += (delta - kcp.rx_rttvar) >> 5
- } else {
- kcp.rx_rttvar += (delta - kcp.rx_rttvar) >> 2
- }
- }
- rto = uint32(kcp.rx_srtt) + _imax_(kcp.interval, uint32(kcp.rx_rttvar)<<2)
- kcp.rx_rto = _ibound_(kcp.rx_minrto, rto, IKCP_RTO_MAX)
-}
-
-func (kcp *KCP) shrink_buf() {
- if len(kcp.snd_buf) > 0 {
- seg := &kcp.snd_buf[0]
- kcp.snd_una = seg.sn
- } else {
- kcp.snd_una = kcp.snd_nxt
- }
-}
-
-func (kcp *KCP) parse_ack(sn uint32) {
- if _itimediff(sn, kcp.snd_una) < 0 || _itimediff(sn, kcp.snd_nxt) >= 0 {
- return
- }
-
- for k := range kcp.snd_buf {
- seg := &kcp.snd_buf[k]
- if sn == seg.sn {
- // mark and free space, but leave the segment here,
- // and wait until `una` to delete this, then we don't
- // have to shift the segments behind forward,
- // which is an expensive operation for large window
- seg.acked = 1
- kcp.delSegment(seg)
- break
- }
- if _itimediff(sn, seg.sn) < 0 {
- break
- }
- }
-}
-
-func (kcp *KCP) parse_fastack(sn, ts uint32) {
- if _itimediff(sn, kcp.snd_una) < 0 || _itimediff(sn, kcp.snd_nxt) >= 0 {
- return
- }
-
- for k := range kcp.snd_buf {
- seg := &kcp.snd_buf[k]
- if _itimediff(sn, seg.sn) < 0 {
- break
- } else if sn != seg.sn && _itimediff(seg.ts, ts) <= 0 {
- seg.fastack++
- }
- }
-}
-
-func (kcp *KCP) parse_una(una uint32) {
- count := 0
- for k := range kcp.snd_buf {
- seg := &kcp.snd_buf[k]
- if _itimediff(una, seg.sn) > 0 {
- kcp.delSegment(seg)
- count++
- } else {
- break
- }
- }
- if count > 0 {
- kcp.snd_buf = kcp.remove_front(kcp.snd_buf, count)
- }
-}
-
-// ack append
-func (kcp *KCP) ack_push(sn, ts uint32) {
- kcp.acklist = append(kcp.acklist, ackItem{sn, ts})
-}
-
-// returns true if data has repeated
-func (kcp *KCP) parse_data(newseg segment) bool {
- sn := newseg.sn
- if _itimediff(sn, kcp.rcv_nxt+kcp.rcv_wnd) >= 0 ||
- _itimediff(sn, kcp.rcv_nxt) < 0 {
- return true
- }
-
- n := len(kcp.rcv_buf) - 1
- insert_idx := 0
- repeat := false
- for i := n; i >= 0; i-- {
- seg := &kcp.rcv_buf[i]
- if seg.sn == sn {
- repeat = true
- break
- }
- if _itimediff(sn, seg.sn) > 0 {
- insert_idx = i + 1
- break
- }
- }
-
- if !repeat {
- // replicate the content if it's new
- dataCopy := xmitBuf.Get().([]byte)[:len(newseg.data)]
- copy(dataCopy, newseg.data)
- newseg.data = dataCopy
-
- if insert_idx == n+1 {
- kcp.rcv_buf = append(kcp.rcv_buf, newseg)
- } else {
- kcp.rcv_buf = append(kcp.rcv_buf, segment{})
- copy(kcp.rcv_buf[insert_idx+1:], kcp.rcv_buf[insert_idx:])
- kcp.rcv_buf[insert_idx] = newseg
- }
- }
-
- // move available data from rcv_buf -> rcv_queue
- count := 0
- for k := range kcp.rcv_buf {
- seg := &kcp.rcv_buf[k]
- if seg.sn == kcp.rcv_nxt && len(kcp.rcv_queue)+count < int(kcp.rcv_wnd) {
- kcp.rcv_nxt++
- count++
- } else {
- break
- }
- }
- if count > 0 {
- kcp.rcv_queue = append(kcp.rcv_queue, kcp.rcv_buf[:count]...)
- kcp.rcv_buf = kcp.remove_front(kcp.rcv_buf, count)
- }
-
- return repeat
-}
-
-// Input a packet into kcp state machine.
-//
-// 'regular' indicates it's a real data packet from remote, and it means it's not generated from ReedSolomon
-// codecs.
-//
-// 'ackNoDelay' will trigger immediate ACK, but surely it will not be efficient in bandwidth
-func (kcp *KCP) Input(data []byte, regular, ackNoDelay bool) int {
- snd_una := kcp.snd_una
- if len(data) < IKCP_OVERHEAD {
- return -1
- }
-
- var latest uint32 // the latest ack packet
- var flag int
- var inSegs uint64
-
- for {
- var ts, sn, length, una, conv uint32
- var wnd uint16
- var cmd, frg uint8
-
- if len(data) < int(IKCP_OVERHEAD) {
- break
- }
-
- data = ikcp_decode32u(data, &conv)
- if conv != kcp.conv {
- return -1
- }
-
- data = ikcp_decode8u(data, &cmd)
- data = ikcp_decode8u(data, &frg)
- data = ikcp_decode16u(data, &wnd)
- data = ikcp_decode32u(data, &ts)
- data = ikcp_decode32u(data, &sn)
- data = ikcp_decode32u(data, &una)
- data = ikcp_decode32u(data, &length)
- if len(data) < int(length) {
- return -2
- }
-
- if cmd != IKCP_CMD_PUSH && cmd != IKCP_CMD_ACK &&
- cmd != IKCP_CMD_WASK && cmd != IKCP_CMD_WINS {
- return -3
- }
-
- // only trust window updates from regular packets. i.e: latest update
- if regular {
- kcp.rmt_wnd = uint32(wnd)
- }
- kcp.parse_una(una)
- kcp.shrink_buf()
-
- if cmd == IKCP_CMD_ACK {
- kcp.parse_ack(sn)
- kcp.parse_fastack(sn, ts)
- flag |= 1
- latest = ts
- } else if cmd == IKCP_CMD_PUSH {
- repeat := true
- if _itimediff(sn, kcp.rcv_nxt+kcp.rcv_wnd) < 0 {
- kcp.ack_push(sn, ts)
- if _itimediff(sn, kcp.rcv_nxt) >= 0 {
- var seg segment
- seg.conv = conv
- seg.cmd = cmd
- seg.frg = frg
- seg.wnd = wnd
- seg.ts = ts
- seg.sn = sn
- seg.una = una
- seg.data = data[:length] // delayed data copying
- repeat = kcp.parse_data(seg)
- }
- }
- if regular && repeat {
- atomic.AddUint64(&DefaultSnmp.RepeatSegs, 1)
- }
- } else if cmd == IKCP_CMD_WASK {
- // ready to send back IKCP_CMD_WINS in Ikcp_flush
- // tell remote my window size
- kcp.probe |= IKCP_ASK_TELL
- } else if cmd == IKCP_CMD_WINS {
- // do nothing
- } else {
- return -3
- }
-
- inSegs++
- data = data[length:]
- }
- atomic.AddUint64(&DefaultSnmp.InSegs, inSegs)
-
- // update rtt with the latest ts
- // ignore the FEC packet
- if flag != 0 && regular {
- current := currentMs()
- if _itimediff(current, latest) >= 0 {
- kcp.update_ack(_itimediff(current, latest))
- }
- }
-
- // cwnd update when packet arrived
- if kcp.nocwnd == 0 {
- if _itimediff(kcp.snd_una, snd_una) > 0 {
- if kcp.cwnd < kcp.rmt_wnd {
- mss := kcp.mss
- if kcp.cwnd < kcp.ssthresh {
- kcp.cwnd++
- kcp.incr += mss
- } else {
- if kcp.incr < mss {
- kcp.incr = mss
- }
- kcp.incr += (mss*mss)/kcp.incr + (mss / 16)
- if (kcp.cwnd+1)*mss <= kcp.incr {
- kcp.cwnd++
- }
- }
- if kcp.cwnd > kcp.rmt_wnd {
- kcp.cwnd = kcp.rmt_wnd
- kcp.incr = kcp.rmt_wnd * mss
- }
- }
- }
- }
-
- if ackNoDelay && len(kcp.acklist) > 0 { // ack immediately
- kcp.flush(true)
- }
- return 0
-}
-
-func (kcp *KCP) wnd_unused() uint16 {
- if len(kcp.rcv_queue) < int(kcp.rcv_wnd) {
- return uint16(int(kcp.rcv_wnd) - len(kcp.rcv_queue))
- }
- return 0
-}
-
-// flush pending data
-func (kcp *KCP) flush(ackOnly bool) uint32 {
- var seg segment
- seg.conv = kcp.conv
- seg.cmd = IKCP_CMD_ACK
- seg.wnd = kcp.wnd_unused()
- seg.una = kcp.rcv_nxt
-
- buffer := kcp.buffer
- ptr := buffer[kcp.reserved:] // keep n bytes untouched
-
- // makeSpace makes room for writing
- makeSpace := func(space int) {
- size := len(buffer) - len(ptr)
- if size+space > int(kcp.mtu) {
- kcp.output(buffer, size)
- ptr = buffer[kcp.reserved:]
- }
- }
-
- // flush bytes in buffer if there is any
- flushBuffer := func() {
- size := len(buffer) - len(ptr)
- if size > kcp.reserved {
- kcp.output(buffer, size)
- }
- }
-
- // flush acknowledges
- for i, ack := range kcp.acklist {
- makeSpace(IKCP_OVERHEAD)
- // filter jitters caused by bufferbloat
- if ack.sn >= kcp.rcv_nxt || len(kcp.acklist)-1 == i {
- seg.sn, seg.ts = ack.sn, ack.ts
- ptr = seg.encode(ptr)
- }
- }
- kcp.acklist = kcp.acklist[0:0]
-
- if ackOnly { // flash remain ack segments
- flushBuffer()
- return kcp.interval
- }
-
- // probe window size (if remote window size equals zero)
- if kcp.rmt_wnd == 0 {
- current := currentMs()
- if kcp.probe_wait == 0 {
- kcp.probe_wait = IKCP_PROBE_INIT
- kcp.ts_probe = current + kcp.probe_wait
- } else {
- if _itimediff(current, kcp.ts_probe) >= 0 {
- if kcp.probe_wait < IKCP_PROBE_INIT {
- kcp.probe_wait = IKCP_PROBE_INIT
- }
- kcp.probe_wait += kcp.probe_wait / 2
- if kcp.probe_wait > IKCP_PROBE_LIMIT {
- kcp.probe_wait = IKCP_PROBE_LIMIT
- }
- kcp.ts_probe = current + kcp.probe_wait
- kcp.probe |= IKCP_ASK_SEND
- }
- }
- } else {
- kcp.ts_probe = 0
- kcp.probe_wait = 0
- }
-
- // flush window probing commands
- if (kcp.probe & IKCP_ASK_SEND) != 0 {
- seg.cmd = IKCP_CMD_WASK
- makeSpace(IKCP_OVERHEAD)
- ptr = seg.encode(ptr)
- }
-
- // flush window probing commands
- if (kcp.probe & IKCP_ASK_TELL) != 0 {
- seg.cmd = IKCP_CMD_WINS
- makeSpace(IKCP_OVERHEAD)
- ptr = seg.encode(ptr)
- }
-
- kcp.probe = 0
-
- // calculate window size
- cwnd := _imin_(kcp.snd_wnd, kcp.rmt_wnd)
- if kcp.nocwnd == 0 {
- cwnd = _imin_(kcp.cwnd, cwnd)
- }
-
- // sliding window, controlled by snd_nxt && sna_una+cwnd
- newSegsCount := 0
- for k := range kcp.snd_queue {
- if _itimediff(kcp.snd_nxt, kcp.snd_una+cwnd) >= 0 {
- break
- }
- newseg := kcp.snd_queue[k]
- newseg.conv = kcp.conv
- newseg.cmd = IKCP_CMD_PUSH
- newseg.sn = kcp.snd_nxt
- kcp.snd_buf = append(kcp.snd_buf, newseg)
- kcp.snd_nxt++
- newSegsCount++
- }
- if newSegsCount > 0 {
- kcp.snd_queue = kcp.remove_front(kcp.snd_queue, newSegsCount)
- }
-
- // calculate resent
- resent := uint32(kcp.fastresend)
- if kcp.fastresend <= 0 {
- resent = 0xffffffff
- }
-
- // check for retransmissions
- current := currentMs()
- var change, lostSegs, fastRetransSegs, earlyRetransSegs uint64
- minrto := int32(kcp.interval)
-
- ref := kcp.snd_buf[:len(kcp.snd_buf)] // for bounds check elimination
- for k := range ref {
- segment := &ref[k]
- needsend := false
- if segment.acked == 1 {
- continue
- }
- if segment.xmit == 0 { // initial transmit
- needsend = true
- segment.rto = kcp.rx_rto
- segment.resendts = current + segment.rto
- } else if segment.fastack >= resent { // fast retransmit
- needsend = true
- segment.fastack = 0
- segment.rto = kcp.rx_rto
- segment.resendts = current + segment.rto
- change++
- fastRetransSegs++
- } else if segment.fastack > 0 && newSegsCount == 0 { // early retransmit
- needsend = true
- segment.fastack = 0
- segment.rto = kcp.rx_rto
- segment.resendts = current + segment.rto
- change++
- earlyRetransSegs++
- } else if _itimediff(current, segment.resendts) >= 0 { // RTO
- needsend = true
- if kcp.nodelay == 0 {
- segment.rto += kcp.rx_rto
- } else {
- segment.rto += kcp.rx_rto / 2
- }
- segment.fastack = 0
- segment.resendts = current + segment.rto
- lostSegs++
- }
-
- if needsend {
- current = currentMs()
- segment.xmit++
- segment.ts = current
- segment.wnd = seg.wnd
- segment.una = seg.una
-
- need := IKCP_OVERHEAD + len(segment.data)
- makeSpace(need)
- ptr = segment.encode(ptr)
- copy(ptr, segment.data)
- ptr = ptr[len(segment.data):]
-
- if segment.xmit >= kcp.dead_link {
- kcp.state = 0xFFFFFFFF
- }
- }
-
- // get the nearest rto
- if rto := _itimediff(segment.resendts, current); rto > 0 && rto < minrto {
- minrto = rto
- }
- }
-
- // flash remain segments
- flushBuffer()
-
- // counter updates
- sum := lostSegs
- if lostSegs > 0 {
- atomic.AddUint64(&DefaultSnmp.LostSegs, lostSegs)
- }
- if fastRetransSegs > 0 {
- atomic.AddUint64(&DefaultSnmp.FastRetransSegs, fastRetransSegs)
- sum += fastRetransSegs
- }
- if earlyRetransSegs > 0 {
- atomic.AddUint64(&DefaultSnmp.EarlyRetransSegs, earlyRetransSegs)
- sum += earlyRetransSegs
- }
- if sum > 0 {
- atomic.AddUint64(&DefaultSnmp.RetransSegs, sum)
- }
-
- // cwnd update
- if kcp.nocwnd == 0 {
- // update ssthresh
- // rate halving, https://tools.ietf.org/html/rfc6937
- if change > 0 {
- inflight := kcp.snd_nxt - kcp.snd_una
- kcp.ssthresh = inflight / 2
- if kcp.ssthresh < IKCP_THRESH_MIN {
- kcp.ssthresh = IKCP_THRESH_MIN
- }
- kcp.cwnd = kcp.ssthresh + resent
- kcp.incr = kcp.cwnd * kcp.mss
- }
-
- // congestion control, https://tools.ietf.org/html/rfc5681
- if lostSegs > 0 {
- kcp.ssthresh = cwnd / 2
- if kcp.ssthresh < IKCP_THRESH_MIN {
- kcp.ssthresh = IKCP_THRESH_MIN
- }
- kcp.cwnd = 1
- kcp.incr = kcp.mss
- }
-
- if kcp.cwnd < 1 {
- kcp.cwnd = 1
- kcp.incr = kcp.mss
- }
- }
-
- return uint32(minrto)
-}
-
-// (deprecated)
-//
-// Update updates state (call it repeatedly, every 10ms-100ms), or you can ask
-// ikcp_check when to call it again (without ikcp_input/_send calling).
-// 'current' - current timestamp in millisec.
-func (kcp *KCP) Update() {
- var slap int32
-
- current := currentMs()
- if kcp.updated == 0 {
- kcp.updated = 1
- kcp.ts_flush = current
- }
-
- slap = _itimediff(current, kcp.ts_flush)
-
- if slap >= 10000 || slap < -10000 {
- kcp.ts_flush = current
- slap = 0
- }
-
- if slap >= 0 {
- kcp.ts_flush += kcp.interval
- if _itimediff(current, kcp.ts_flush) >= 0 {
- kcp.ts_flush = current + kcp.interval
- }
- kcp.flush(false)
- }
-}
-
-// (deprecated)
-//
-// Check determines when should you invoke ikcp_update:
-// returns when you should invoke ikcp_update in millisec, if there
-// is no ikcp_input/_send calling. you can call ikcp_update in that
-// time, instead of call update repeatly.
-// Important to reduce unnacessary ikcp_update invoking. use it to
-// schedule ikcp_update (eg. implementing an epoll-like mechanism,
-// or optimize ikcp_update when handling massive kcp connections)
-func (kcp *KCP) Check() uint32 {
- current := currentMs()
- ts_flush := kcp.ts_flush
- tm_flush := int32(0x7fffffff)
- tm_packet := int32(0x7fffffff)
- minimal := uint32(0)
- if kcp.updated == 0 {
- return current
- }
-
- if _itimediff(current, ts_flush) >= 10000 ||
- _itimediff(current, ts_flush) < -10000 {
- ts_flush = current
- }
-
- if _itimediff(current, ts_flush) >= 0 {
- return current
- }
-
- tm_flush = _itimediff(ts_flush, current)
-
- for k := range kcp.snd_buf {
- seg := &kcp.snd_buf[k]
- diff := _itimediff(seg.resendts, current)
- if diff <= 0 {
- return current
- }
- if diff < tm_packet {
- tm_packet = diff
- }
- }
-
- minimal = uint32(tm_packet)
- if tm_packet >= tm_flush {
- minimal = uint32(tm_flush)
- }
- if minimal >= kcp.interval {
- minimal = kcp.interval
- }
-
- return current + minimal
-}
-
-// SetMtu changes MTU size, default is 1400
-func (kcp *KCP) SetMtu(mtu int) int {
- if mtu < 50 || mtu < IKCP_OVERHEAD {
- return -1
- }
- if kcp.reserved >= int(kcp.mtu-IKCP_OVERHEAD) || kcp.reserved < 0 {
- return -1
- }
-
- buffer := make([]byte, mtu)
- if buffer == nil {
- return -2
- }
- kcp.mtu = uint32(mtu)
- kcp.mss = kcp.mtu - IKCP_OVERHEAD - uint32(kcp.reserved)
- kcp.buffer = buffer
- return 0
-}
-
-// NoDelay options
-// fastest: ikcp_nodelay(kcp, 1, 20, 2, 1)
-// nodelay: 0:disable(default), 1:enable
-// interval: internal update timer interval in millisec, default is 100ms
-// resend: 0:disable fast resend(default), 1:enable fast resend
-// nc: 0:normal congestion control(default), 1:disable congestion control
-func (kcp *KCP) NoDelay(nodelay, interval, resend, nc int) int {
- if nodelay >= 0 {
- kcp.nodelay = uint32(nodelay)
- if nodelay != 0 {
- kcp.rx_minrto = IKCP_RTO_NDL
- } else {
- kcp.rx_minrto = IKCP_RTO_MIN
- }
- }
- if interval >= 0 {
- if interval > 5000 {
- interval = 5000
- } else if interval < 10 {
- interval = 10
- }
- kcp.interval = uint32(interval)
- }
- if resend >= 0 {
- kcp.fastresend = int32(resend)
- }
- if nc >= 0 {
- kcp.nocwnd = int32(nc)
- }
- return 0
-}
-
-// WndSize sets maximum window size: sndwnd=32, rcvwnd=32 by default
-func (kcp *KCP) WndSize(sndwnd, rcvwnd int) int {
- if sndwnd > 0 {
- kcp.snd_wnd = uint32(sndwnd)
- }
- if rcvwnd > 0 {
- kcp.rcv_wnd = uint32(rcvwnd)
- }
- return 0
-}
-
-// WaitSnd gets how many packet is waiting to be sent
-func (kcp *KCP) WaitSnd() int {
- return len(kcp.snd_buf) + len(kcp.snd_queue)
-}
-
-// remove front n elements from queue
-// if the number of elements to remove is more than half of the size.
-// just shift the rear elements to front, otherwise just reslice q to q[n:]
-// then the cost of runtime.growslice can always be less than n/2
-func (kcp *KCP) remove_front(q []segment, n int) []segment {
- if n > cap(q)/2 {
- newn := copy(q, q[n:])
- return q[:newn]
- }
- return q[n:]
-}
-
-// Release all cached outgoing segments
-func (kcp *KCP) ReleaseTX() {
- for k := range kcp.snd_queue {
- if kcp.snd_queue[k].data != nil {
- xmitBuf.Put(kcp.snd_queue[k].data)
- }
- }
- for k := range kcp.snd_buf {
- if kcp.snd_buf[k].data != nil {
- xmitBuf.Put(kcp.snd_buf[k].data)
- }
- }
- kcp.snd_queue = nil
- kcp.snd_buf = nil
-}
diff --git a/vendor/github.com/xtaci/kcp-go/readloop.go b/vendor/github.com/xtaci/kcp-go/readloop.go
deleted file mode 100644
index bc48e84..0000000
--- a/vendor/github.com/xtaci/kcp-go/readloop.go
+++ /dev/null
@@ -1,48 +0,0 @@
-package kcp
-
-import (
- "sync/atomic"
-
- "github.com/pkg/errors"
-)
-
-func (s *UDPSession) defaultReadLoop() {
- buf := make([]byte, mtuLimit)
- var src string
- for {
- if n, addr, err := s.conn.ReadFrom(buf); err == nil {
- // make sure the packet is from the same source
- if src == "" { // set source address
- src = addr.String()
- } else if addr.String() != src {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- continue
- }
-
- if n >= s.headerSize+IKCP_OVERHEAD {
- s.packetInput(buf[:n])
- } else {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- }
- } else {
- s.notifyReadError(errors.WithStack(err))
- return
- }
- }
-}
-
-func (l *Listener) defaultMonitor() {
- buf := make([]byte, mtuLimit)
- for {
- if n, from, err := l.conn.ReadFrom(buf); err == nil {
- if n >= l.headerSize+IKCP_OVERHEAD {
- l.packetInput(buf[:n], from)
- } else {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- }
- } else {
- l.notifyReadError(errors.WithStack(err))
- return
- }
- }
-}
diff --git a/vendor/github.com/xtaci/kcp-go/readloop_generic.go b/vendor/github.com/xtaci/kcp-go/readloop_generic.go
deleted file mode 100644
index 5dbe4f4..0000000
--- a/vendor/github.com/xtaci/kcp-go/readloop_generic.go
+++ /dev/null
@@ -1,11 +0,0 @@
-// +build !linux
-
-package kcp
-
-func (s *UDPSession) readLoop() {
- s.defaultReadLoop()
-}
-
-func (l *Listener) monitor() {
- l.defaultMonitor()
-}
diff --git a/vendor/github.com/xtaci/kcp-go/readloop_linux.go b/vendor/github.com/xtaci/kcp-go/readloop_linux.go
deleted file mode 100644
index 6226478..0000000
--- a/vendor/github.com/xtaci/kcp-go/readloop_linux.go
+++ /dev/null
@@ -1,120 +0,0 @@
-// +build linux
-
-package kcp
-
-import (
- "net"
- "os"
- "sync/atomic"
-
- "github.com/pkg/errors"
- "golang.org/x/net/ipv4"
- "golang.org/x/net/ipv6"
-)
-
-// the read loop for a client session
-func (s *UDPSession) readLoop() {
- // default version
- if s.xconn == nil {
- s.defaultReadLoop()
- return
- }
-
- // x/net version
- var src string
- msgs := make([]ipv4.Message, batchSize)
- for k := range msgs {
- msgs[k].Buffers = [][]byte{make([]byte, mtuLimit)}
- }
-
- for {
- if count, err := s.xconn.ReadBatch(msgs, 0); err == nil {
- for i := 0; i < count; i++ {
- msg := &msgs[i]
- // make sure the packet is from the same source
- if src == "" { // set source address if nil
- src = msg.Addr.String()
- } else if msg.Addr.String() != src {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- continue
- }
-
- if msg.N < s.headerSize+IKCP_OVERHEAD {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- continue
- }
-
- // source and size has validated
- s.packetInput(msg.Buffers[0][:msg.N])
- }
- } else {
- // compatibility issue:
- // for linux kernel<=2.6.32, support for sendmmsg is not available
- // an error of type os.SyscallError will be returned
- if operr, ok := err.(*net.OpError); ok {
- if se, ok := operr.Err.(*os.SyscallError); ok {
- if se.Syscall == "recvmmsg" {
- s.defaultReadLoop()
- return
- }
- }
- }
- s.notifyReadError(errors.WithStack(err))
- return
- }
- }
-}
-
-// monitor incoming data for all connections of server
-func (l *Listener) monitor() {
- var xconn batchConn
- if _, ok := l.conn.(*net.UDPConn); ok {
- addr, err := net.ResolveUDPAddr("udp", l.conn.LocalAddr().String())
- if err == nil {
- if addr.IP.To4() != nil {
- xconn = ipv4.NewPacketConn(l.conn)
- } else {
- xconn = ipv6.NewPacketConn(l.conn)
- }
- }
- }
-
- // default version
- if xconn == nil {
- l.defaultMonitor()
- return
- }
-
- // x/net version
- msgs := make([]ipv4.Message, batchSize)
- for k := range msgs {
- msgs[k].Buffers = [][]byte{make([]byte, mtuLimit)}
- }
-
- for {
- if count, err := xconn.ReadBatch(msgs, 0); err == nil {
- for i := 0; i < count; i++ {
- msg := &msgs[i]
- if msg.N >= l.headerSize+IKCP_OVERHEAD {
- l.packetInput(msg.Buffers[0][:msg.N], msg.Addr)
- } else {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- }
- }
- } else {
- // compatibility issue:
- // for linux kernel<=2.6.32, support for sendmmsg is not available
- // an error of type os.SyscallError will be returned
- if operr, ok := err.(*net.OpError); ok {
- if se, ok := operr.Err.(*os.SyscallError); ok {
- if se.Syscall == "recvmmsg" {
- l.defaultMonitor()
- return
- }
- }
- }
- l.notifyReadError(errors.WithStack(err))
- return
- }
- }
-}
diff --git a/vendor/github.com/xtaci/kcp-go/sess.go b/vendor/github.com/xtaci/kcp-go/sess.go
deleted file mode 100644
index 2384816..0000000
--- a/vendor/github.com/xtaci/kcp-go/sess.go
+++ /dev/null
@@ -1,1054 +0,0 @@
-// Package kcp-go is a Reliable-UDP library for golang.
-//
-// This library intents to provide a smooth, resilient, ordered,
-// error-checked and anonymous delivery of streams over UDP packets.
-//
-// The interfaces of this package aims to be compatible with
-// net.Conn in standard library, but offers powerful features for advanced users.
-package kcp
-
-import (
- "crypto/rand"
- "encoding/binary"
- "hash/crc32"
- "io"
- "net"
- "sync"
- "sync/atomic"
- "time"
-
- "github.com/pkg/errors"
- "golang.org/x/net/ipv4"
- "golang.org/x/net/ipv6"
-)
-
-const (
- // 16-bytes nonce for each packet
- nonceSize = 16
-
- // 4-bytes packet checksum
- crcSize = 4
-
- // overall crypto header size
- cryptHeaderSize = nonceSize + crcSize
-
- // maximum packet size
- mtuLimit = 1500
-
- // FEC keeps rxFECMulti* (dataShard+parityShard) ordered packets in memory
- rxFECMulti = 3
-
- // accept backlog
- acceptBacklog = 128
-)
-
-var (
- errInvalidOperation = errors.New("invalid operation")
- errTimeout = errors.New("timeout")
-)
-
-var (
- // a system-wide packet buffer shared among sending, receiving and FEC
- // to mitigate high-frequency memory allocation for packets
- xmitBuf sync.Pool
-)
-
-func init() {
- xmitBuf.New = func() interface{} {
- return make([]byte, mtuLimit)
- }
-}
-
-type (
- // UDPSession defines a KCP session implemented by UDP
- UDPSession struct {
- conn net.PacketConn // the underlying packet connection
- kcp *KCP // KCP ARQ protocol
- l *Listener // pointing to the Listener object if it's been accepted by a Listener
- block BlockCrypt // block encryption object
-
- // kcp receiving is based on packets
- // recvbuf turns packets into stream
- recvbuf []byte
- bufptr []byte
-
- // FEC codec
- fecDecoder *fecDecoder
- fecEncoder *fecEncoder
-
- // settings
- remote net.Addr // remote peer address
- rd time.Time // read deadline
- wd time.Time // write deadline
- headerSize int // the header size additional to a KCP frame
- ackNoDelay bool // send ack immediately for each incoming packet(testing purpose)
- writeDelay bool // delay kcp.flush() for Write() for bulk transfer
- dup int // duplicate udp packets(testing purpose)
-
- // notifications
- die chan struct{} // notify current session has Closed
- dieOnce sync.Once
- chReadEvent chan struct{} // notify Read() can be called without blocking
- chWriteEvent chan struct{} // notify Write() can be called without blocking
-
- // socket error handling
- socketReadError atomic.Value
- socketWriteError atomic.Value
- chSocketReadError chan struct{}
- chSocketWriteError chan struct{}
- socketReadErrorOnce sync.Once
- socketWriteErrorOnce sync.Once
-
- // nonce generator
- nonce Entropy
-
- // packets waiting to be sent on wire
- txqueue []ipv4.Message
- xconn batchConn // for x/net
- xconnWriteError error
-
- mu sync.Mutex
- }
-
- setReadBuffer interface {
- SetReadBuffer(bytes int) error
- }
-
- setWriteBuffer interface {
- SetWriteBuffer(bytes int) error
- }
-
- setDSCP interface {
- SetDSCP(int) error
- }
-)
-
-// newUDPSession create a new udp session for client or server
-func newUDPSession(conv uint32, dataShards, parityShards int, l *Listener, conn net.PacketConn, remote net.Addr, block BlockCrypt) *UDPSession {
- sess := new(UDPSession)
- sess.die = make(chan struct{})
- sess.nonce = new(nonceAES128)
- sess.nonce.Init()
- sess.chReadEvent = make(chan struct{}, 1)
- sess.chWriteEvent = make(chan struct{}, 1)
- sess.chSocketReadError = make(chan struct{})
- sess.chSocketWriteError = make(chan struct{})
- sess.remote = remote
- sess.conn = conn
- sess.l = l
- sess.block = block
- sess.recvbuf = make([]byte, mtuLimit)
-
- // cast to writebatch conn
- if _, ok := conn.(*net.UDPConn); ok {
- addr, err := net.ResolveUDPAddr("udp", conn.LocalAddr().String())
- if err == nil {
- if addr.IP.To4() != nil {
- sess.xconn = ipv4.NewPacketConn(conn)
- } else {
- sess.xconn = ipv6.NewPacketConn(conn)
- }
- }
- }
-
- // FEC codec initialization
- sess.fecDecoder = newFECDecoder(rxFECMulti*(dataShards+parityShards), dataShards, parityShards)
- if sess.block != nil {
- sess.fecEncoder = newFECEncoder(dataShards, parityShards, cryptHeaderSize)
- } else {
- sess.fecEncoder = newFECEncoder(dataShards, parityShards, 0)
- }
-
- // calculate additional header size introduced by FEC and encryption
- if sess.block != nil {
- sess.headerSize += cryptHeaderSize
- }
- if sess.fecEncoder != nil {
- sess.headerSize += fecHeaderSizePlus2
- }
-
- sess.kcp = NewKCP(conv, func(buf []byte, size int) {
- if size >= IKCP_OVERHEAD+sess.headerSize {
- sess.output(buf[:size])
- }
- })
- sess.kcp.ReserveBytes(sess.headerSize)
-
- if sess.l == nil { // it's a client connection
- go sess.readLoop()
- atomic.AddUint64(&DefaultSnmp.ActiveOpens, 1)
- } else {
- atomic.AddUint64(&DefaultSnmp.PassiveOpens, 1)
- }
-
- // start per-session updater
- go sess.updater()
-
- currestab := atomic.AddUint64(&DefaultSnmp.CurrEstab, 1)
- maxconn := atomic.LoadUint64(&DefaultSnmp.MaxConn)
- if currestab > maxconn {
- atomic.CompareAndSwapUint64(&DefaultSnmp.MaxConn, maxconn, currestab)
- }
-
- return sess
-}
-
-// Read implements net.Conn
-func (s *UDPSession) Read(b []byte) (n int, err error) {
- for {
- s.mu.Lock()
- if len(s.bufptr) > 0 { // copy from buffer into b
- n = copy(b, s.bufptr)
- s.bufptr = s.bufptr[n:]
- s.mu.Unlock()
- atomic.AddUint64(&DefaultSnmp.BytesReceived, uint64(n))
- return n, nil
- }
-
- if size := s.kcp.PeekSize(); size > 0 { // peek data size from kcp
- if len(b) >= size { // receive data into 'b' directly
- s.kcp.Recv(b)
- s.mu.Unlock()
- atomic.AddUint64(&DefaultSnmp.BytesReceived, uint64(size))
- return size, nil
- }
-
- // if necessary resize the stream buffer to guarantee a sufficent buffer space
- if cap(s.recvbuf) < size {
- s.recvbuf = make([]byte, size)
- }
-
- // resize the length of recvbuf to correspond to data size
- s.recvbuf = s.recvbuf[:size]
- s.kcp.Recv(s.recvbuf)
- n = copy(b, s.recvbuf) // copy to 'b'
- s.bufptr = s.recvbuf[n:] // pointer update
- s.mu.Unlock()
- atomic.AddUint64(&DefaultSnmp.BytesReceived, uint64(n))
- return n, nil
- }
-
- // deadline for current reading operation
- var timeout *time.Timer
- var c <-chan time.Time
- if !s.rd.IsZero() {
- if time.Now().After(s.rd) {
- s.mu.Unlock()
- return 0, errors.WithStack(errTimeout)
- }
-
- delay := s.rd.Sub(time.Now())
- timeout = time.NewTimer(delay)
- c = timeout.C
- }
- s.mu.Unlock()
-
- // wait for read event or timeout or error
- select {
- case <-s.chReadEvent:
- if timeout != nil {
- timeout.Stop()
- }
- case <-c:
- return 0, errors.WithStack(errTimeout)
- case <-s.chSocketReadError:
- return 0, s.socketReadError.Load().(error)
- case <-s.die:
- return 0, errors.WithStack(io.ErrClosedPipe)
- }
- }
-}
-
-// Write implements net.Conn
-func (s *UDPSession) Write(b []byte) (n int, err error) { return s.WriteBuffers([][]byte{b}) }
-
-// WriteBuffers write a vector of byte slices to the underlying connection
-func (s *UDPSession) WriteBuffers(v [][]byte) (n int, err error) {
- for {
- select {
- case <-s.chSocketWriteError:
- return 0, s.socketWriteError.Load().(error)
- case <-s.die:
- return 0, errors.WithStack(io.ErrClosedPipe)
- default:
- }
-
- s.mu.Lock()
-
- // make sure write do not overflow the max sliding window on both side
- waitsnd := s.kcp.WaitSnd()
- if waitsnd < int(s.kcp.snd_wnd) && waitsnd < int(s.kcp.rmt_wnd) {
- for _, b := range v {
- n += len(b)
- for {
- if len(b) <= int(s.kcp.mss) {
- s.kcp.Send(b)
- break
- } else {
- s.kcp.Send(b[:s.kcp.mss])
- b = b[s.kcp.mss:]
- }
- }
- }
-
- waitsnd = s.kcp.WaitSnd()
- if waitsnd >= int(s.kcp.snd_wnd) || waitsnd >= int(s.kcp.rmt_wnd) || !s.writeDelay {
- s.kcp.flush(false)
- s.uncork()
- }
- s.mu.Unlock()
- atomic.AddUint64(&DefaultSnmp.BytesSent, uint64(n))
- return n, nil
- }
-
- var timeout *time.Timer
- var c <-chan time.Time
- if !s.wd.IsZero() {
- if time.Now().After(s.wd) {
- s.mu.Unlock()
- return 0, errors.WithStack(errTimeout)
- }
- delay := s.wd.Sub(time.Now())
- timeout = time.NewTimer(delay)
- c = timeout.C
- }
- s.mu.Unlock()
-
- select {
- case <-s.chWriteEvent:
- if timeout != nil {
- timeout.Stop()
- }
- case <-c:
- return 0, errors.WithStack(errTimeout)
- case <-s.chSocketWriteError:
- return 0, s.socketWriteError.Load().(error)
- case <-s.die:
- return 0, errors.WithStack(io.ErrClosedPipe)
- }
- }
-}
-
-// uncork sends data in txqueue if there is any
-func (s *UDPSession) uncork() {
- if len(s.txqueue) > 0 {
- s.tx(s.txqueue)
- // recycle
- for k := range s.txqueue {
- xmitBuf.Put(s.txqueue[k].Buffers[0])
- s.txqueue[k].Buffers = nil
- }
- s.txqueue = s.txqueue[:0]
- }
- return
-}
-
-// Close closes the connection.
-func (s *UDPSession) Close() error {
- var once bool
- s.dieOnce.Do(func() {
- close(s.die)
- once = true
- })
-
- if once {
- atomic.AddUint64(&DefaultSnmp.CurrEstab, ^uint64(0))
-
- // try best to send all queued messages
- s.mu.Lock()
- s.kcp.flush(false)
- s.uncork()
- // release pending segments
- s.kcp.ReleaseTX()
- if s.fecDecoder != nil {
- s.fecDecoder.release()
- }
- s.mu.Unlock()
-
- if s.l != nil { // belongs to listener
- s.l.closeSession(s.remote)
- return nil
- } else { // client socket close
- return s.conn.Close()
- }
- } else {
- return errors.WithStack(io.ErrClosedPipe)
- }
-}
-
-// LocalAddr returns the local network address. The Addr returned is shared by all invocations of LocalAddr, so do not modify it.
-func (s *UDPSession) LocalAddr() net.Addr { return s.conn.LocalAddr() }
-
-// RemoteAddr returns the remote network address. The Addr returned is shared by all invocations of RemoteAddr, so do not modify it.
-func (s *UDPSession) RemoteAddr() net.Addr { return s.remote }
-
-// SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.
-func (s *UDPSession) SetDeadline(t time.Time) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.rd = t
- s.wd = t
- s.notifyReadEvent()
- s.notifyWriteEvent()
- return nil
-}
-
-// SetReadDeadline implements the Conn SetReadDeadline method.
-func (s *UDPSession) SetReadDeadline(t time.Time) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.rd = t
- s.notifyReadEvent()
- return nil
-}
-
-// SetWriteDeadline implements the Conn SetWriteDeadline method.
-func (s *UDPSession) SetWriteDeadline(t time.Time) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.wd = t
- s.notifyWriteEvent()
- return nil
-}
-
-// SetWriteDelay delays write for bulk transfer until the next update interval
-func (s *UDPSession) SetWriteDelay(delay bool) {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.writeDelay = delay
-}
-
-// SetWindowSize set maximum window size
-func (s *UDPSession) SetWindowSize(sndwnd, rcvwnd int) {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.kcp.WndSize(sndwnd, rcvwnd)
-}
-
-// SetMtu sets the maximum transmission unit(not including UDP header)
-func (s *UDPSession) SetMtu(mtu int) bool {
- if mtu > mtuLimit {
- return false
- }
-
- s.mu.Lock()
- defer s.mu.Unlock()
- s.kcp.SetMtu(mtu)
- return true
-}
-
-// SetStreamMode toggles the stream mode on/off
-func (s *UDPSession) SetStreamMode(enable bool) {
- s.mu.Lock()
- defer s.mu.Unlock()
- if enable {
- s.kcp.stream = 1
- } else {
- s.kcp.stream = 0
- }
-}
-
-// SetACKNoDelay changes ack flush option, set true to flush ack immediately,
-func (s *UDPSession) SetACKNoDelay(nodelay bool) {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.ackNoDelay = nodelay
-}
-
-// (deprecated)
-//
-// SetDUP duplicates udp packets for kcp output.
-func (s *UDPSession) SetDUP(dup int) {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.dup = dup
-}
-
-// SetNoDelay calls nodelay() of kcp
-// https://github.com/skywind3000/kcp/blob/master/README.en.md#protocol-configuration
-func (s *UDPSession) SetNoDelay(nodelay, interval, resend, nc int) {
- s.mu.Lock()
- defer s.mu.Unlock()
- s.kcp.NoDelay(nodelay, interval, resend, nc)
-}
-
-// SetDSCP sets the 6bit DSCP field in IPv4 header, or 8bit Traffic Class in IPv6 header.
-//
-// if the underlying connection has implemented `func SetDSCP(int) error`, SetDSCP() will invoke
-// this function instead.
-//
-// It has no effect if it's accepted from Listener.
-func (s *UDPSession) SetDSCP(dscp int) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- if s.l != nil {
- return errInvalidOperation
- }
-
- // interface enabled
- if ts, ok := s.conn.(setDSCP); ok {
- return ts.SetDSCP(dscp)
- }
-
- if nc, ok := s.conn.(net.Conn); ok {
- var succeed bool
- if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err == nil {
- succeed = true
- }
- if err := ipv6.NewConn(nc).SetTrafficClass(dscp); err == nil {
- succeed = true
- }
-
- if succeed {
- return nil
- }
- }
- return errInvalidOperation
-}
-
-// SetReadBuffer sets the socket read buffer, no effect if it's accepted from Listener
-func (s *UDPSession) SetReadBuffer(bytes int) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- if s.l == nil {
- if nc, ok := s.conn.(setReadBuffer); ok {
- return nc.SetReadBuffer(bytes)
- }
- }
- return errInvalidOperation
-}
-
-// SetWriteBuffer sets the socket write buffer, no effect if it's accepted from Listener
-func (s *UDPSession) SetWriteBuffer(bytes int) error {
- s.mu.Lock()
- defer s.mu.Unlock()
- if s.l == nil {
- if nc, ok := s.conn.(setWriteBuffer); ok {
- return nc.SetWriteBuffer(bytes)
- }
- }
- return errInvalidOperation
-}
-
-// post-processing for sending a packet from kcp core
-// steps:
-// 1. FEC packet generation
-// 2. CRC32 integrity
-// 3. Encryption
-// 4. TxQueue
-func (s *UDPSession) output(buf []byte) {
- var ecc [][]byte
-
- // 1. FEC encoding
- if s.fecEncoder != nil {
- ecc = s.fecEncoder.encode(buf)
- }
-
- // 2&3. crc32 & encryption
- if s.block != nil {
- s.nonce.Fill(buf[:nonceSize])
- checksum := crc32.ChecksumIEEE(buf[cryptHeaderSize:])
- binary.LittleEndian.PutUint32(buf[nonceSize:], checksum)
- s.block.Encrypt(buf, buf)
-
- for k := range ecc {
- s.nonce.Fill(ecc[k][:nonceSize])
- checksum := crc32.ChecksumIEEE(ecc[k][cryptHeaderSize:])
- binary.LittleEndian.PutUint32(ecc[k][nonceSize:], checksum)
- s.block.Encrypt(ecc[k], ecc[k])
- }
- }
-
- // 4. TxQueue
- var msg ipv4.Message
- for i := 0; i < s.dup+1; i++ {
- bts := xmitBuf.Get().([]byte)[:len(buf)]
- copy(bts, buf)
- msg.Buffers = [][]byte{bts}
- msg.Addr = s.remote
- s.txqueue = append(s.txqueue, msg)
- }
-
- for k := range ecc {
- bts := xmitBuf.Get().([]byte)[:len(ecc[k])]
- copy(bts, ecc[k])
- msg.Buffers = [][]byte{bts}
- msg.Addr = s.remote
- s.txqueue = append(s.txqueue, msg)
- }
-}
-
-// sess updater to trigger protocol
-func (s *UDPSession) updater() {
- timer := time.NewTimer(0)
- for {
- select {
- case <-timer.C:
- s.mu.Lock()
- interval := time.Duration(s.kcp.flush(false)) * time.Millisecond
- waitsnd := s.kcp.WaitSnd()
- if waitsnd < int(s.kcp.snd_wnd) && waitsnd < int(s.kcp.rmt_wnd) {
- s.notifyWriteEvent()
- }
- s.uncork()
- s.mu.Unlock()
- timer.Reset(interval)
- case <-s.die:
- timer.Stop()
- return
- }
- }
-}
-
-// GetConv gets conversation id of a session
-func (s *UDPSession) GetConv() uint32 { return s.kcp.conv }
-
-func (s *UDPSession) notifyReadEvent() {
- select {
- case s.chReadEvent <- struct{}{}:
- default:
- }
-}
-
-func (s *UDPSession) notifyWriteEvent() {
- select {
- case s.chWriteEvent <- struct{}{}:
- default:
- }
-}
-
-func (s *UDPSession) notifyReadError(err error) {
- s.socketReadErrorOnce.Do(func() {
- s.socketReadError.Store(err)
- close(s.chSocketReadError)
- })
-}
-
-func (s *UDPSession) notifyWriteError(err error) {
- s.socketWriteErrorOnce.Do(func() {
- s.socketWriteError.Store(err)
- close(s.chSocketWriteError)
- })
-}
-
-// packet input stage
-func (s *UDPSession) packetInput(data []byte) {
- dataValid := false
- if s.block != nil {
- s.block.Decrypt(data, data)
- data = data[nonceSize:]
- checksum := crc32.ChecksumIEEE(data[crcSize:])
- if checksum == binary.LittleEndian.Uint32(data) {
- data = data[crcSize:]
- dataValid = true
- } else {
- atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
- }
- } else if s.block == nil {
- dataValid = true
- }
-
- if dataValid {
- s.kcpInput(data)
- }
-}
-
-func (s *UDPSession) kcpInput(data []byte) {
- var kcpInErrors, fecErrs, fecRecovered, fecParityShards uint64
-
- if s.fecDecoder != nil {
- if len(data) > fecHeaderSize { // must be larger than fec header size
- f := fecPacket(data)
- if f.flag() == typeData || f.flag() == typeParity { // header check
- if f.flag() == typeParity {
- fecParityShards++
- }
-
- // lock
- s.mu.Lock()
- recovers := s.fecDecoder.decode(f)
- if f.flag() == typeData {
- if ret := s.kcp.Input(data[fecHeaderSizePlus2:], true, s.ackNoDelay); ret != 0 {
- kcpInErrors++
- }
- }
-
- for _, r := range recovers {
- if len(r) >= 2 { // must be larger than 2bytes
- sz := binary.LittleEndian.Uint16(r)
- if int(sz) <= len(r) && sz >= 2 {
- if ret := s.kcp.Input(r[2:sz], false, s.ackNoDelay); ret == 0 {
- fecRecovered++
- } else {
- kcpInErrors++
- }
- } else {
- fecErrs++
- }
- } else {
- fecErrs++
- }
- // recycle the recovers
- xmitBuf.Put(r)
- }
-
- // to notify the readers to receive the data
- if n := s.kcp.PeekSize(); n > 0 {
- s.notifyReadEvent()
- }
- // to notify the writers
- waitsnd := s.kcp.WaitSnd()
- if waitsnd < int(s.kcp.snd_wnd) && waitsnd < int(s.kcp.rmt_wnd) {
- s.notifyWriteEvent()
- }
-
- s.uncork()
- s.mu.Unlock()
- } else {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- }
- } else {
- atomic.AddUint64(&DefaultSnmp.InErrs, 1)
- }
- } else {
- s.mu.Lock()
- if ret := s.kcp.Input(data, true, s.ackNoDelay); ret != 0 {
- kcpInErrors++
- }
- if n := s.kcp.PeekSize(); n > 0 {
- s.notifyReadEvent()
- }
- waitsnd := s.kcp.WaitSnd()
- if waitsnd < int(s.kcp.snd_wnd) && waitsnd < int(s.kcp.rmt_wnd) {
- s.notifyWriteEvent()
- }
- s.uncork()
- s.mu.Unlock()
- }
-
- atomic.AddUint64(&DefaultSnmp.InPkts, 1)
- atomic.AddUint64(&DefaultSnmp.InBytes, uint64(len(data)))
- if fecParityShards > 0 {
- atomic.AddUint64(&DefaultSnmp.FECParityShards, fecParityShards)
- }
- if kcpInErrors > 0 {
- atomic.AddUint64(&DefaultSnmp.KCPInErrors, kcpInErrors)
- }
- if fecErrs > 0 {
- atomic.AddUint64(&DefaultSnmp.FECErrs, fecErrs)
- }
- if fecRecovered > 0 {
- atomic.AddUint64(&DefaultSnmp.FECRecovered, fecRecovered)
- }
-
-}
-
-type (
- // Listener defines a server which will be waiting to accept incoming connections
- Listener struct {
- block BlockCrypt // block encryption
- dataShards int // FEC data shard
- parityShards int // FEC parity shard
- fecDecoder *fecDecoder // FEC mock initialization
- conn net.PacketConn // the underlying packet connection
-
- sessions map[string]*UDPSession // all sessions accepted by this Listener
- sessionLock sync.Mutex
- chAccepts chan *UDPSession // Listen() backlog
- chSessionClosed chan net.Addr // session close queue
- headerSize int // the additional header to a KCP frame
-
- die chan struct{} // notify the listener has closed
- dieOnce sync.Once
-
- // socket error handling
- socketReadError atomic.Value
- chSocketReadError chan struct{}
- socketReadErrorOnce sync.Once
-
- rd atomic.Value // read deadline for Accept()
- }
-)
-
-// packet input stage
-func (l *Listener) packetInput(data []byte, addr net.Addr) {
- dataValid := false
- if l.block != nil {
- l.block.Decrypt(data, data)
- data = data[nonceSize:]
- checksum := crc32.ChecksumIEEE(data[crcSize:])
- if checksum == binary.LittleEndian.Uint32(data) {
- data = data[crcSize:]
- dataValid = true
- } else {
- atomic.AddUint64(&DefaultSnmp.InCsumErrors, 1)
- }
- } else if l.block == nil {
- dataValid = true
- }
-
- if dataValid {
- l.sessionLock.Lock()
- s, ok := l.sessions[addr.String()]
- l.sessionLock.Unlock()
-
- var conv, sn uint32
- convValid := false
- if l.fecDecoder != nil {
- isfec := binary.LittleEndian.Uint16(data[4:])
- if isfec == typeData {
- conv = binary.LittleEndian.Uint32(data[fecHeaderSizePlus2:])
- sn = binary.LittleEndian.Uint32(data[fecHeaderSizePlus2+IKCP_SN_OFFSET:])
- convValid = true
- }
- } else {
- conv = binary.LittleEndian.Uint32(data)
- sn = binary.LittleEndian.Uint32(data[IKCP_SN_OFFSET:])
- convValid = true
- }
-
- if ok { // existing connection
- if !convValid || conv == s.kcp.conv { // parity or valid data shard
- s.kcpInput(data)
- } else if sn == 0 { // should replace current connection
- s.Close()
- s = nil
- }
- }
-
- if s == nil && convValid { // new session
- if len(l.chAccepts) < cap(l.chAccepts) { // do not let the new sessions overwhelm accept queue
- s := newUDPSession(conv, l.dataShards, l.parityShards, l, l.conn, addr, l.block)
- s.kcpInput(data)
- l.sessionLock.Lock()
- l.sessions[addr.String()] = s
- l.sessionLock.Unlock()
- l.chAccepts <- s
- }
- }
- }
-}
-
-func (l *Listener) notifyReadError(err error) {
- l.socketReadErrorOnce.Do(func() {
- l.socketReadError.Store(err)
- close(l.chSocketReadError)
-
- // propagate read error to all sessions
- l.sessionLock.Lock()
- for _, s := range l.sessions {
- s.notifyReadError(err)
- }
- l.sessionLock.Unlock()
- })
-}
-
-// SetReadBuffer sets the socket read buffer for the Listener
-func (l *Listener) SetReadBuffer(bytes int) error {
- if nc, ok := l.conn.(setReadBuffer); ok {
- return nc.SetReadBuffer(bytes)
- }
- return errInvalidOperation
-}
-
-// SetWriteBuffer sets the socket write buffer for the Listener
-func (l *Listener) SetWriteBuffer(bytes int) error {
- if nc, ok := l.conn.(setWriteBuffer); ok {
- return nc.SetWriteBuffer(bytes)
- }
- return errInvalidOperation
-}
-
-// SetDSCP sets the 6bit DSCP field in IPv4 header, or 8bit Traffic Class in IPv6 header.
-//
-// if the underlying connection has implemented `func SetDSCP(int) error`, SetDSCP() will invoke
-// this function instead.
-func (l *Listener) SetDSCP(dscp int) error {
- // interface enabled
- if ts, ok := l.conn.(setDSCP); ok {
- return ts.SetDSCP(dscp)
- }
-
- if nc, ok := l.conn.(net.Conn); ok {
- var succeed bool
- if err := ipv4.NewConn(nc).SetTOS(dscp << 2); err == nil {
- succeed = true
- }
- if err := ipv6.NewConn(nc).SetTrafficClass(dscp); err == nil {
- succeed = true
- }
-
- if succeed {
- return nil
- }
- }
- return errInvalidOperation
-}
-
-// Accept implements the Accept method in the Listener interface; it waits for the next call and returns a generic Conn.
-func (l *Listener) Accept() (net.Conn, error) {
- return l.AcceptKCP()
-}
-
-// AcceptKCP accepts a KCP connection
-func (l *Listener) AcceptKCP() (*UDPSession, error) {
- var timeout <-chan time.Time
- if tdeadline, ok := l.rd.Load().(time.Time); ok && !tdeadline.IsZero() {
- timeout = time.After(tdeadline.Sub(time.Now()))
- }
-
- select {
- case <-timeout:
- return nil, errors.WithStack(errTimeout)
- case c := <-l.chAccepts:
- return c, nil
- case <-l.chSocketReadError:
- return nil, l.socketReadError.Load().(error)
- case <-l.die:
- return nil, errors.WithStack(io.ErrClosedPipe)
- }
-}
-
-// SetDeadline sets the deadline associated with the listener. A zero time value disables the deadline.
-func (l *Listener) SetDeadline(t time.Time) error {
- l.SetReadDeadline(t)
- l.SetWriteDeadline(t)
- return nil
-}
-
-// SetReadDeadline implements the Conn SetReadDeadline method.
-func (l *Listener) SetReadDeadline(t time.Time) error {
- l.rd.Store(t)
- return nil
-}
-
-// SetWriteDeadline implements the Conn SetWriteDeadline method.
-func (l *Listener) SetWriteDeadline(t time.Time) error { return errInvalidOperation }
-
-// Close stops listening on the UDP address, and closes the socket
-func (l *Listener) Close() error {
- var once bool
- l.dieOnce.Do(func() {
- close(l.die)
- once = true
- })
-
- if once {
- return l.conn.Close()
- } else {
- return errors.WithStack(io.ErrClosedPipe)
- }
-}
-
-// closeSession notify the listener that a session has closed
-func (l *Listener) closeSession(remote net.Addr) (ret bool) {
- l.sessionLock.Lock()
- defer l.sessionLock.Unlock()
- if _, ok := l.sessions[remote.String()]; ok {
- delete(l.sessions, remote.String())
- return true
- }
- return false
-}
-
-// Addr returns the listener's network address, The Addr returned is shared by all invocations of Addr, so do not modify it.
-func (l *Listener) Addr() net.Addr { return l.conn.LocalAddr() }
-
-// Listen listens for incoming KCP packets addressed to the local address laddr on the network "udp",
-func Listen(laddr string) (net.Listener, error) { return ListenWithOptions(laddr, nil, 0, 0) }
-
-// ListenWithOptions listens for incoming KCP packets addressed to the local address laddr on the network "udp" with packet encryption.
-//
-// 'block' is the block encryption algorithm to encrypt packets.
-//
-// 'dataShards', 'parityShards' specifiy how many parity packets will be generated following the data packets.
-//
-// Check https://github.com/klauspost/reedsolomon for details
-func ListenWithOptions(laddr string, block BlockCrypt, dataShards, parityShards int) (*Listener, error) {
- udpaddr, err := net.ResolveUDPAddr("udp", laddr)
- if err != nil {
- return nil, errors.WithStack(err)
- }
- conn, err := net.ListenUDP("udp", udpaddr)
- if err != nil {
- return nil, errors.WithStack(err)
- }
-
- return ServeConn(block, dataShards, parityShards, conn)
-}
-
-// ServeConn serves KCP protocol for a single packet connection.
-func ServeConn(block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*Listener, error) {
- l := new(Listener)
- l.conn = conn
- l.sessions = make(map[string]*UDPSession)
- l.chAccepts = make(chan *UDPSession, acceptBacklog)
- l.chSessionClosed = make(chan net.Addr)
- l.die = make(chan struct{})
- l.dataShards = dataShards
- l.parityShards = parityShards
- l.block = block
- l.fecDecoder = newFECDecoder(rxFECMulti*(dataShards+parityShards), dataShards, parityShards)
- l.chSocketReadError = make(chan struct{})
-
- // calculate header size
- if l.block != nil {
- l.headerSize += cryptHeaderSize
- }
- if l.fecDecoder != nil {
- l.headerSize += fecHeaderSizePlus2
- }
-
- go l.monitor()
- return l, nil
-}
-
-// Dial connects to the remote address "raddr" on the network "udp" without encryption and FEC
-func Dial(raddr string) (net.Conn, error) { return DialWithOptions(raddr, nil, 0, 0) }
-
-// DialWithOptions connects to the remote address "raddr" on the network "udp" with packet encryption
-//
-// 'block' is the block encryption algorithm to encrypt packets.
-//
-// 'dataShards', 'parityShards' specifiy how many parity packets will be generated following the data packets.
-//
-// Check https://github.com/klauspost/reedsolomon for details
-func DialWithOptions(raddr string, block BlockCrypt, dataShards, parityShards int) (*UDPSession, error) {
- // network type detection
- udpaddr, err := net.ResolveUDPAddr("udp", raddr)
- if err != nil {
- return nil, errors.WithStack(err)
- }
- network := "udp4"
- if udpaddr.IP.To4() == nil {
- network = "udp"
- }
-
- conn, err := net.ListenUDP(network, nil)
- if err != nil {
- return nil, errors.WithStack(err)
- }
-
- return NewConn(raddr, block, dataShards, parityShards, conn)
-}
-
-// NewConn3 establishes a session and talks KCP protocol over a packet connection.
-func NewConn3(convid uint32, raddr net.Addr, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
- return newUDPSession(convid, dataShards, parityShards, nil, conn, raddr, block), nil
-}
-
-// NewConn2 establishes a session and talks KCP protocol over a packet connection.
-func NewConn2(raddr net.Addr, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
- var convid uint32
- binary.Read(rand.Reader, binary.LittleEndian, &convid)
- return NewConn3(convid, raddr, block, dataShards, parityShards, conn)
-}
-
-// NewConn establishes a session and talks KCP protocol over a packet connection.
-func NewConn(raddr string, block BlockCrypt, dataShards, parityShards int, conn net.PacketConn) (*UDPSession, error) {
- udpaddr, err := net.ResolveUDPAddr("udp", raddr)
- if err != nil {
- return nil, errors.WithStack(err)
- }
- return NewConn2(udpaddr, block, dataShards, parityShards, conn)
-}
diff --git a/vendor/github.com/xtaci/kcp-go/snmp.go b/vendor/github.com/xtaci/kcp-go/snmp.go
deleted file mode 100644
index 607118e..0000000
--- a/vendor/github.com/xtaci/kcp-go/snmp.go
+++ /dev/null
@@ -1,164 +0,0 @@
-package kcp
-
-import (
- "fmt"
- "sync/atomic"
-)
-
-// Snmp defines network statistics indicator
-type Snmp struct {
- BytesSent uint64 // bytes sent from upper level
- BytesReceived uint64 // bytes received to upper level
- MaxConn uint64 // max number of connections ever reached
- ActiveOpens uint64 // accumulated active open connections
- PassiveOpens uint64 // accumulated passive open connections
- CurrEstab uint64 // current number of established connections
- InErrs uint64 // UDP read errors reported from net.PacketConn
- InCsumErrors uint64 // checksum errors from CRC32
- KCPInErrors uint64 // packet iput errors reported from KCP
- InPkts uint64 // incoming packets count
- OutPkts uint64 // outgoing packets count
- InSegs uint64 // incoming KCP segments
- OutSegs uint64 // outgoing KCP segments
- InBytes uint64 // UDP bytes received
- OutBytes uint64 // UDP bytes sent
- RetransSegs uint64 // accmulated retransmited segments
- FastRetransSegs uint64 // accmulated fast retransmitted segments
- EarlyRetransSegs uint64 // accmulated early retransmitted segments
- LostSegs uint64 // number of segs infered as lost
- RepeatSegs uint64 // number of segs duplicated
- FECRecovered uint64 // correct packets recovered from FEC
- FECErrs uint64 // incorrect packets recovered from FEC
- FECParityShards uint64 // FEC segments received
- FECShortShards uint64 // number of data shards that's not enough for recovery
-}
-
-func newSnmp() *Snmp {
- return new(Snmp)
-}
-
-// Header returns all field names
-func (s *Snmp) Header() []string {
- return []string{
- "BytesSent",
- "BytesReceived",
- "MaxConn",
- "ActiveOpens",
- "PassiveOpens",
- "CurrEstab",
- "InErrs",
- "InCsumErrors",
- "KCPInErrors",
- "InPkts",
- "OutPkts",
- "InSegs",
- "OutSegs",
- "InBytes",
- "OutBytes",
- "RetransSegs",
- "FastRetransSegs",
- "EarlyRetransSegs",
- "LostSegs",
- "RepeatSegs",
- "FECParityShards",
- "FECErrs",
- "FECRecovered",
- "FECShortShards",
- }
-}
-
-// ToSlice returns current snmp info as slice
-func (s *Snmp) ToSlice() []string {
- snmp := s.Copy()
- return []string{
- fmt.Sprint(snmp.BytesSent),
- fmt.Sprint(snmp.BytesReceived),
- fmt.Sprint(snmp.MaxConn),
- fmt.Sprint(snmp.ActiveOpens),
- fmt.Sprint(snmp.PassiveOpens),
- fmt.Sprint(snmp.CurrEstab),
- fmt.Sprint(snmp.InErrs),
- fmt.Sprint(snmp.InCsumErrors),
- fmt.Sprint(snmp.KCPInErrors),
- fmt.Sprint(snmp.InPkts),
- fmt.Sprint(snmp.OutPkts),
- fmt.Sprint(snmp.InSegs),
- fmt.Sprint(snmp.OutSegs),
- fmt.Sprint(snmp.InBytes),
- fmt.Sprint(snmp.OutBytes),
- fmt.Sprint(snmp.RetransSegs),
- fmt.Sprint(snmp.FastRetransSegs),
- fmt.Sprint(snmp.EarlyRetransSegs),
- fmt.Sprint(snmp.LostSegs),
- fmt.Sprint(snmp.RepeatSegs),
- fmt.Sprint(snmp.FECParityShards),
- fmt.Sprint(snmp.FECErrs),
- fmt.Sprint(snmp.FECRecovered),
- fmt.Sprint(snmp.FECShortShards),
- }
-}
-
-// Copy make a copy of current snmp snapshot
-func (s *Snmp) Copy() *Snmp {
- d := newSnmp()
- d.BytesSent = atomic.LoadUint64(&s.BytesSent)
- d.BytesReceived = atomic.LoadUint64(&s.BytesReceived)
- d.MaxConn = atomic.LoadUint64(&s.MaxConn)
- d.ActiveOpens = atomic.LoadUint64(&s.ActiveOpens)
- d.PassiveOpens = atomic.LoadUint64(&s.PassiveOpens)
- d.CurrEstab = atomic.LoadUint64(&s.CurrEstab)
- d.InErrs = atomic.LoadUint64(&s.InErrs)
- d.InCsumErrors = atomic.LoadUint64(&s.InCsumErrors)
- d.KCPInErrors = atomic.LoadUint64(&s.KCPInErrors)
- d.InPkts = atomic.LoadUint64(&s.InPkts)
- d.OutPkts = atomic.LoadUint64(&s.OutPkts)
- d.InSegs = atomic.LoadUint64(&s.InSegs)
- d.OutSegs = atomic.LoadUint64(&s.OutSegs)
- d.InBytes = atomic.LoadUint64(&s.InBytes)
- d.OutBytes = atomic.LoadUint64(&s.OutBytes)
- d.RetransSegs = atomic.LoadUint64(&s.RetransSegs)
- d.FastRetransSegs = atomic.LoadUint64(&s.FastRetransSegs)
- d.EarlyRetransSegs = atomic.LoadUint64(&s.EarlyRetransSegs)
- d.LostSegs = atomic.LoadUint64(&s.LostSegs)
- d.RepeatSegs = atomic.LoadUint64(&s.RepeatSegs)
- d.FECParityShards = atomic.LoadUint64(&s.FECParityShards)
- d.FECErrs = atomic.LoadUint64(&s.FECErrs)
- d.FECRecovered = atomic.LoadUint64(&s.FECRecovered)
- d.FECShortShards = atomic.LoadUint64(&s.FECShortShards)
- return d
-}
-
-// Reset values to zero
-func (s *Snmp) Reset() {
- atomic.StoreUint64(&s.BytesSent, 0)
- atomic.StoreUint64(&s.BytesReceived, 0)
- atomic.StoreUint64(&s.MaxConn, 0)
- atomic.StoreUint64(&s.ActiveOpens, 0)
- atomic.StoreUint64(&s.PassiveOpens, 0)
- atomic.StoreUint64(&s.CurrEstab, 0)
- atomic.StoreUint64(&s.InErrs, 0)
- atomic.StoreUint64(&s.InCsumErrors, 0)
- atomic.StoreUint64(&s.KCPInErrors, 0)
- atomic.StoreUint64(&s.InPkts, 0)
- atomic.StoreUint64(&s.OutPkts, 0)
- atomic.StoreUint64(&s.InSegs, 0)
- atomic.StoreUint64(&s.OutSegs, 0)
- atomic.StoreUint64(&s.InBytes, 0)
- atomic.StoreUint64(&s.OutBytes, 0)
- atomic.StoreUint64(&s.RetransSegs, 0)
- atomic.StoreUint64(&s.FastRetransSegs, 0)
- atomic.StoreUint64(&s.EarlyRetransSegs, 0)
- atomic.StoreUint64(&s.LostSegs, 0)
- atomic.StoreUint64(&s.RepeatSegs, 0)
- atomic.StoreUint64(&s.FECParityShards, 0)
- atomic.StoreUint64(&s.FECErrs, 0)
- atomic.StoreUint64(&s.FECRecovered, 0)
- atomic.StoreUint64(&s.FECShortShards, 0)
-}
-
-// DefaultSnmp is the global KCP connection statistics collector
-var DefaultSnmp *Snmp
-
-func init() {
- DefaultSnmp = newSnmp()
-}
diff --git a/vendor/github.com/xtaci/kcp-go/tx.go b/vendor/github.com/xtaci/kcp-go/tx.go
deleted file mode 100644
index 3397b82..0000000
--- a/vendor/github.com/xtaci/kcp-go/tx.go
+++ /dev/null
@@ -1,24 +0,0 @@
-package kcp
-
-import (
- "sync/atomic"
-
- "github.com/pkg/errors"
- "golang.org/x/net/ipv4"
-)
-
-func (s *UDPSession) defaultTx(txqueue []ipv4.Message) {
- nbytes := 0
- npkts := 0
- for k := range txqueue {
- if n, err := s.conn.WriteTo(txqueue[k].Buffers[0], txqueue[k].Addr); err == nil {
- nbytes += n
- npkts++
- } else {
- s.notifyWriteError(errors.WithStack(err))
- break
- }
- }
- atomic.AddUint64(&DefaultSnmp.OutPkts, uint64(npkts))
- atomic.AddUint64(&DefaultSnmp.OutBytes, uint64(nbytes))
-}
diff --git a/vendor/github.com/xtaci/kcp-go/tx_generic.go b/vendor/github.com/xtaci/kcp-go/tx_generic.go
deleted file mode 100644
index 0b4f349..0000000
--- a/vendor/github.com/xtaci/kcp-go/tx_generic.go
+++ /dev/null
@@ -1,11 +0,0 @@
-// +build !linux
-
-package kcp
-
-import (
- "golang.org/x/net/ipv4"
-)
-
-func (s *UDPSession) tx(txqueue []ipv4.Message) {
- s.defaultTx(txqueue)
-}
diff --git a/vendor/github.com/xtaci/kcp-go/tx_linux.go b/vendor/github.com/xtaci/kcp-go/tx_linux.go
deleted file mode 100644
index 4f19df5..0000000
--- a/vendor/github.com/xtaci/kcp-go/tx_linux.go
+++ /dev/null
@@ -1,51 +0,0 @@
-// +build linux
-
-package kcp
-
-import (
- "net"
- "os"
- "sync/atomic"
-
- "github.com/pkg/errors"
- "golang.org/x/net/ipv4"
-)
-
-func (s *UDPSession) tx(txqueue []ipv4.Message) {
- // default version
- if s.xconn == nil || s.xconnWriteError != nil {
- s.defaultTx(txqueue)
- return
- }
-
- // x/net version
- nbytes := 0
- npkts := 0
- for len(txqueue) > 0 {
- if n, err := s.xconn.WriteBatch(txqueue, 0); err == nil {
- for k := range txqueue[:n] {
- nbytes += len(txqueue[k].Buffers[0])
- }
- npkts += n
- txqueue = txqueue[n:]
- } else {
- // compatibility issue:
- // for linux kernel<=2.6.32, support for sendmmsg is not available
- // an error of type os.SyscallError will be returned
- if operr, ok := err.(*net.OpError); ok {
- if se, ok := operr.Err.(*os.SyscallError); ok {
- if se.Syscall == "sendmmsg" {
- s.xconnWriteError = se
- s.defaultTx(txqueue)
- return
- }
- }
- }
- s.notifyWriteError(errors.WithStack(err))
- break
- }
- }
-
- atomic.AddUint64(&DefaultSnmp.OutPkts, uint64(npkts))
- atomic.AddUint64(&DefaultSnmp.OutBytes, uint64(nbytes))
-}
diff --git a/vendor/github.com/xtaci/kcp-go/wechat_donate.jpg b/vendor/github.com/xtaci/kcp-go/wechat_donate.jpg
deleted file mode 100644
index ad72505..0000000
Binary files a/vendor/github.com/xtaci/kcp-go/wechat_donate.jpg and /dev/null differ
diff --git a/vendor/golang.org/x/crypto/LICENSE b/vendor/golang.org/x/crypto/LICENSE
deleted file mode 100644
index 6a66aea..0000000
--- a/vendor/golang.org/x/crypto/LICENSE
+++ /dev/null
@@ -1,27 +0,0 @@
-Copyright (c) 2009 The Go Authors. All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- * Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
- * Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/vendor/golang.org/x/crypto/PATENTS b/vendor/golang.org/x/crypto/PATENTS
deleted file mode 100644
index 7330990..0000000
--- a/vendor/golang.org/x/crypto/PATENTS
+++ /dev/null
@@ -1,22 +0,0 @@
-Additional IP Rights Grant (Patents)
-
-"This implementation" means the copyrightable works distributed by
-Google as part of the Go project.
-
-Google hereby grants to You a perpetual, worldwide, non-exclusive,
-no-charge, royalty-free, irrevocable (except as stated in this section)
-patent license to make, have made, use, offer to sell, sell, import,
-transfer and otherwise run, modify and propagate the contents of this
-implementation of Go, where such license applies only to those patent
-claims, both currently owned or controlled by Google and acquired in
-the future, licensable by Google that are necessarily infringed by this
-implementation of Go. This grant does not include claims that would be
-infringed only as a consequence of further modification of this
-implementation. If you or your agent or exclusive licensee institute or
-order or agree to the institution of patent litigation against any
-entity (including a cross-claim or counterclaim in a lawsuit) alleging
-that this implementation of Go or any code incorporated within this
-implementation of Go constitutes direct or contributory patent
-infringement, or inducement of patent infringement, then any patent
-rights granted to you under this License for this implementation of Go
-shall terminate as of the date such litigation is filed.
diff --git a/vendor/golang.org/x/crypto/argon2/argon2.go b/vendor/golang.org/x/crypto/argon2/argon2.go
deleted file mode 100644
index 29f0a2d..0000000
--- a/vendor/golang.org/x/crypto/argon2/argon2.go
+++ /dev/null
@@ -1,283 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package argon2 implements the key derivation function Argon2.
-// Argon2 was selected as the winner of the Password Hashing Competition and can
-// be used to derive cryptographic keys from passwords.
-//
-// For a detailed specification of Argon2 see [1].
-//
-// If you aren't sure which function you need, use Argon2id (IDKey) and
-// the parameter recommendations for your scenario.
-//
-// # Argon2i
-//
-// Argon2i (implemented by Key) is the side-channel resistant version of Argon2.
-// It uses data-independent memory access, which is preferred for password
-// hashing and password-based key derivation. Argon2i requires more passes over
-// memory than Argon2id to protect from trade-off attacks. The recommended
-// parameters (taken from [2]) for non-interactive operations are time=3 and to
-// use the maximum available memory.
-//
-// # Argon2id
-//
-// Argon2id (implemented by IDKey) is a hybrid version of Argon2 combining
-// Argon2i and Argon2d. It uses data-independent memory access for the first
-// half of the first iteration over the memory and data-dependent memory access
-// for the rest. Argon2id is side-channel resistant and provides better brute-
-// force cost savings due to time-memory tradeoffs than Argon2i. The recommended
-// parameters for non-interactive operations (taken from [2]) are time=1 and to
-// use the maximum available memory.
-//
-// [1] https://github.com/P-H-C/phc-winner-argon2/blob/master/argon2-specs.pdf
-// [2] https://tools.ietf.org/html/draft-irtf-cfrg-argon2-03#section-9.3
-package argon2
-
-import (
- "encoding/binary"
- "sync"
-
- "golang.org/x/crypto/blake2b"
-)
-
-// The Argon2 version implemented by this package.
-const Version = 0x13
-
-const (
- argon2d = iota
- argon2i
- argon2id
-)
-
-// Key derives a key from the password, salt, and cost parameters using Argon2i
-// returning a byte slice of length keyLen that can be used as cryptographic
-// key. The CPU cost and parallelism degree must be greater than zero.
-//
-// For example, you can get a derived key for e.g. AES-256 (which needs a
-// 32-byte key) by doing:
-//
-// key := argon2.Key([]byte("some password"), salt, 3, 32*1024, 4, 32)
-//
-// The draft RFC recommends[2] time=3, and memory=32*1024 is a sensible number.
-// If using that amount of memory (32 MB) is not possible in some contexts then
-// the time parameter can be increased to compensate.
-//
-// The time parameter specifies the number of passes over the memory and the
-// memory parameter specifies the size of the memory in KiB. For example
-// memory=32*1024 sets the memory cost to ~32 MB. The number of threads can be
-// adjusted to the number of available CPUs. The cost parameters should be
-// increased as memory latency and CPU parallelism increases. Remember to get a
-// good random salt.
-func Key(password, salt []byte, time, memory uint32, threads uint8, keyLen uint32) []byte {
- return deriveKey(argon2i, password, salt, nil, nil, time, memory, threads, keyLen)
-}
-
-// IDKey derives a key from the password, salt, and cost parameters using
-// Argon2id returning a byte slice of length keyLen that can be used as
-// cryptographic key. The CPU cost and parallelism degree must be greater than
-// zero.
-//
-// For example, you can get a derived key for e.g. AES-256 (which needs a
-// 32-byte key) by doing:
-//
-// key := argon2.IDKey([]byte("some password"), salt, 1, 64*1024, 4, 32)
-//
-// The draft RFC recommends[2] time=1, and memory=64*1024 is a sensible number.
-// If using that amount of memory (64 MB) is not possible in some contexts then
-// the time parameter can be increased to compensate.
-//
-// The time parameter specifies the number of passes over the memory and the
-// memory parameter specifies the size of the memory in KiB. For example
-// memory=64*1024 sets the memory cost to ~64 MB. The number of threads can be
-// adjusted to the numbers of available CPUs. The cost parameters should be
-// increased as memory latency and CPU parallelism increases. Remember to get a
-// good random salt.
-func IDKey(password, salt []byte, time, memory uint32, threads uint8, keyLen uint32) []byte {
- return deriveKey(argon2id, password, salt, nil, nil, time, memory, threads, keyLen)
-}
-
-func deriveKey(mode int, password, salt, secret, data []byte, time, memory uint32, threads uint8, keyLen uint32) []byte {
- if time < 1 {
- panic("argon2: number of rounds too small")
- }
- if threads < 1 {
- panic("argon2: parallelism degree too low")
- }
- h0 := initHash(password, salt, secret, data, time, memory, uint32(threads), keyLen, mode)
-
- memory = memory / (syncPoints * uint32(threads)) * (syncPoints * uint32(threads))
- if memory < 2*syncPoints*uint32(threads) {
- memory = 2 * syncPoints * uint32(threads)
- }
- B := initBlocks(&h0, memory, uint32(threads))
- processBlocks(B, time, memory, uint32(threads), mode)
- return extractKey(B, memory, uint32(threads), keyLen)
-}
-
-const (
- blockLength = 128
- syncPoints = 4
-)
-
-type block [blockLength]uint64
-
-func initHash(password, salt, key, data []byte, time, memory, threads, keyLen uint32, mode int) [blake2b.Size + 8]byte {
- var (
- h0 [blake2b.Size + 8]byte
- params [24]byte
- tmp [4]byte
- )
-
- b2, _ := blake2b.New512(nil)
- binary.LittleEndian.PutUint32(params[0:4], threads)
- binary.LittleEndian.PutUint32(params[4:8], keyLen)
- binary.LittleEndian.PutUint32(params[8:12], memory)
- binary.LittleEndian.PutUint32(params[12:16], time)
- binary.LittleEndian.PutUint32(params[16:20], uint32(Version))
- binary.LittleEndian.PutUint32(params[20:24], uint32(mode))
- b2.Write(params[:])
- binary.LittleEndian.PutUint32(tmp[:], uint32(len(password)))
- b2.Write(tmp[:])
- b2.Write(password)
- binary.LittleEndian.PutUint32(tmp[:], uint32(len(salt)))
- b2.Write(tmp[:])
- b2.Write(salt)
- binary.LittleEndian.PutUint32(tmp[:], uint32(len(key)))
- b2.Write(tmp[:])
- b2.Write(key)
- binary.LittleEndian.PutUint32(tmp[:], uint32(len(data)))
- b2.Write(tmp[:])
- b2.Write(data)
- b2.Sum(h0[:0])
- return h0
-}
-
-func initBlocks(h0 *[blake2b.Size + 8]byte, memory, threads uint32) []block {
- var block0 [1024]byte
- B := make([]block, memory)
- for lane := uint32(0); lane < threads; lane++ {
- j := lane * (memory / threads)
- binary.LittleEndian.PutUint32(h0[blake2b.Size+4:], lane)
-
- binary.LittleEndian.PutUint32(h0[blake2b.Size:], 0)
- blake2bHash(block0[:], h0[:])
- for i := range B[j+0] {
- B[j+0][i] = binary.LittleEndian.Uint64(block0[i*8:])
- }
-
- binary.LittleEndian.PutUint32(h0[blake2b.Size:], 1)
- blake2bHash(block0[:], h0[:])
- for i := range B[j+1] {
- B[j+1][i] = binary.LittleEndian.Uint64(block0[i*8:])
- }
- }
- return B
-}
-
-func processBlocks(B []block, time, memory, threads uint32, mode int) {
- lanes := memory / threads
- segments := lanes / syncPoints
-
- processSegment := func(n, slice, lane uint32, wg *sync.WaitGroup) {
- var addresses, in, zero block
- if mode == argon2i || (mode == argon2id && n == 0 && slice < syncPoints/2) {
- in[0] = uint64(n)
- in[1] = uint64(lane)
- in[2] = uint64(slice)
- in[3] = uint64(memory)
- in[4] = uint64(time)
- in[5] = uint64(mode)
- }
-
- index := uint32(0)
- if n == 0 && slice == 0 {
- index = 2 // we have already generated the first two blocks
- if mode == argon2i || mode == argon2id {
- in[6]++
- processBlock(&addresses, &in, &zero)
- processBlock(&addresses, &addresses, &zero)
- }
- }
-
- offset := lane*lanes + slice*segments + index
- var random uint64
- for index < segments {
- prev := offset - 1
- if index == 0 && slice == 0 {
- prev += lanes // last block in lane
- }
- if mode == argon2i || (mode == argon2id && n == 0 && slice < syncPoints/2) {
- if index%blockLength == 0 {
- in[6]++
- processBlock(&addresses, &in, &zero)
- processBlock(&addresses, &addresses, &zero)
- }
- random = addresses[index%blockLength]
- } else {
- random = B[prev][0]
- }
- newOffset := indexAlpha(random, lanes, segments, threads, n, slice, lane, index)
- processBlockXOR(&B[offset], &B[prev], &B[newOffset])
- index, offset = index+1, offset+1
- }
- wg.Done()
- }
-
- for n := uint32(0); n < time; n++ {
- for slice := uint32(0); slice < syncPoints; slice++ {
- var wg sync.WaitGroup
- for lane := uint32(0); lane < threads; lane++ {
- wg.Add(1)
- go processSegment(n, slice, lane, &wg)
- }
- wg.Wait()
- }
- }
-
-}
-
-func extractKey(B []block, memory, threads, keyLen uint32) []byte {
- lanes := memory / threads
- for lane := uint32(0); lane < threads-1; lane++ {
- for i, v := range B[(lane*lanes)+lanes-1] {
- B[memory-1][i] ^= v
- }
- }
-
- var block [1024]byte
- for i, v := range B[memory-1] {
- binary.LittleEndian.PutUint64(block[i*8:], v)
- }
- key := make([]byte, keyLen)
- blake2bHash(key, block[:])
- return key
-}
-
-func indexAlpha(rand uint64, lanes, segments, threads, n, slice, lane, index uint32) uint32 {
- refLane := uint32(rand>>32) % threads
- if n == 0 && slice == 0 {
- refLane = lane
- }
- m, s := 3*segments, ((slice+1)%syncPoints)*segments
- if lane == refLane {
- m += index
- }
- if n == 0 {
- m, s = slice*segments, 0
- if slice == 0 || lane == refLane {
- m += index
- }
- }
- if index == 0 || lane == refLane {
- m--
- }
- return phi(rand, uint64(m), uint64(s), refLane, lanes)
-}
-
-func phi(rand, m, s uint64, lane, lanes uint32) uint32 {
- p := rand & 0xFFFFFFFF
- p = (p * p) >> 32
- p = (p * m) >> 32
- return lane*lanes + uint32((s+m-(p+1))%uint64(lanes))
-}
diff --git a/vendor/golang.org/x/crypto/argon2/blake2b.go b/vendor/golang.org/x/crypto/argon2/blake2b.go
deleted file mode 100644
index 10f4694..0000000
--- a/vendor/golang.org/x/crypto/argon2/blake2b.go
+++ /dev/null
@@ -1,53 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package argon2
-
-import (
- "encoding/binary"
- "hash"
-
- "golang.org/x/crypto/blake2b"
-)
-
-// blake2bHash computes an arbitrary long hash value of in
-// and writes the hash to out.
-func blake2bHash(out []byte, in []byte) {
- var b2 hash.Hash
- if n := len(out); n < blake2b.Size {
- b2, _ = blake2b.New(n, nil)
- } else {
- b2, _ = blake2b.New512(nil)
- }
-
- var buffer [blake2b.Size]byte
- binary.LittleEndian.PutUint32(buffer[:4], uint32(len(out)))
- b2.Write(buffer[:4])
- b2.Write(in)
-
- if len(out) <= blake2b.Size {
- b2.Sum(out[:0])
- return
- }
-
- outLen := len(out)
- b2.Sum(buffer[:0])
- b2.Reset()
- copy(out, buffer[:32])
- out = out[32:]
- for len(out) > blake2b.Size {
- b2.Write(buffer[:])
- b2.Sum(buffer[:0])
- copy(out, buffer[:32])
- out = out[32:]
- b2.Reset()
- }
-
- if outLen%blake2b.Size > 0 { // outLen > 64
- r := ((outLen + 31) / 32) - 2 // ⌈τ /32⌉-2
- b2, _ = blake2b.New(outLen-32*r, nil)
- }
- b2.Write(buffer[:])
- b2.Sum(out[:0])
-}
diff --git a/vendor/golang.org/x/crypto/argon2/blamka_amd64.go b/vendor/golang.org/x/crypto/argon2/blamka_amd64.go
deleted file mode 100644
index a014ac9..0000000
--- a/vendor/golang.org/x/crypto/argon2/blamka_amd64.go
+++ /dev/null
@@ -1,61 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && gc && !purego
-// +build amd64,gc,!purego
-
-package argon2
-
-import "golang.org/x/sys/cpu"
-
-func init() {
- useSSE4 = cpu.X86.HasSSE41
-}
-
-//go:noescape
-func mixBlocksSSE2(out, a, b, c *block)
-
-//go:noescape
-func xorBlocksSSE2(out, a, b, c *block)
-
-//go:noescape
-func blamkaSSE4(b *block)
-
-func processBlockSSE(out, in1, in2 *block, xor bool) {
- var t block
- mixBlocksSSE2(&t, in1, in2, &t)
- if useSSE4 {
- blamkaSSE4(&t)
- } else {
- for i := 0; i < blockLength; i += 16 {
- blamkaGeneric(
- &t[i+0], &t[i+1], &t[i+2], &t[i+3],
- &t[i+4], &t[i+5], &t[i+6], &t[i+7],
- &t[i+8], &t[i+9], &t[i+10], &t[i+11],
- &t[i+12], &t[i+13], &t[i+14], &t[i+15],
- )
- }
- for i := 0; i < blockLength/8; i += 2 {
- blamkaGeneric(
- &t[i], &t[i+1], &t[16+i], &t[16+i+1],
- &t[32+i], &t[32+i+1], &t[48+i], &t[48+i+1],
- &t[64+i], &t[64+i+1], &t[80+i], &t[80+i+1],
- &t[96+i], &t[96+i+1], &t[112+i], &t[112+i+1],
- )
- }
- }
- if xor {
- xorBlocksSSE2(out, in1, in2, &t)
- } else {
- mixBlocksSSE2(out, in1, in2, &t)
- }
-}
-
-func processBlock(out, in1, in2 *block) {
- processBlockSSE(out, in1, in2, false)
-}
-
-func processBlockXOR(out, in1, in2 *block) {
- processBlockSSE(out, in1, in2, true)
-}
diff --git a/vendor/golang.org/x/crypto/argon2/blamka_amd64.s b/vendor/golang.org/x/crypto/argon2/blamka_amd64.s
deleted file mode 100644
index b2cc051..0000000
--- a/vendor/golang.org/x/crypto/argon2/blamka_amd64.s
+++ /dev/null
@@ -1,244 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && gc && !purego
-// +build amd64,gc,!purego
-
-#include "textflag.h"
-
-DATA ·c40<>+0x00(SB)/8, $0x0201000706050403
-DATA ·c40<>+0x08(SB)/8, $0x0a09080f0e0d0c0b
-GLOBL ·c40<>(SB), (NOPTR+RODATA), $16
-
-DATA ·c48<>+0x00(SB)/8, $0x0100070605040302
-DATA ·c48<>+0x08(SB)/8, $0x09080f0e0d0c0b0a
-GLOBL ·c48<>(SB), (NOPTR+RODATA), $16
-
-#define SHUFFLE(v2, v3, v4, v5, v6, v7, t1, t2) \
- MOVO v4, t1; \
- MOVO v5, v4; \
- MOVO t1, v5; \
- MOVO v6, t1; \
- PUNPCKLQDQ v6, t2; \
- PUNPCKHQDQ v7, v6; \
- PUNPCKHQDQ t2, v6; \
- PUNPCKLQDQ v7, t2; \
- MOVO t1, v7; \
- MOVO v2, t1; \
- PUNPCKHQDQ t2, v7; \
- PUNPCKLQDQ v3, t2; \
- PUNPCKHQDQ t2, v2; \
- PUNPCKLQDQ t1, t2; \
- PUNPCKHQDQ t2, v3
-
-#define SHUFFLE_INV(v2, v3, v4, v5, v6, v7, t1, t2) \
- MOVO v4, t1; \
- MOVO v5, v4; \
- MOVO t1, v5; \
- MOVO v2, t1; \
- PUNPCKLQDQ v2, t2; \
- PUNPCKHQDQ v3, v2; \
- PUNPCKHQDQ t2, v2; \
- PUNPCKLQDQ v3, t2; \
- MOVO t1, v3; \
- MOVO v6, t1; \
- PUNPCKHQDQ t2, v3; \
- PUNPCKLQDQ v7, t2; \
- PUNPCKHQDQ t2, v6; \
- PUNPCKLQDQ t1, t2; \
- PUNPCKHQDQ t2, v7
-
-#define HALF_ROUND(v0, v1, v2, v3, v4, v5, v6, v7, t0, c40, c48) \
- MOVO v0, t0; \
- PMULULQ v2, t0; \
- PADDQ v2, v0; \
- PADDQ t0, v0; \
- PADDQ t0, v0; \
- PXOR v0, v6; \
- PSHUFD $0xB1, v6, v6; \
- MOVO v4, t0; \
- PMULULQ v6, t0; \
- PADDQ v6, v4; \
- PADDQ t0, v4; \
- PADDQ t0, v4; \
- PXOR v4, v2; \
- PSHUFB c40, v2; \
- MOVO v0, t0; \
- PMULULQ v2, t0; \
- PADDQ v2, v0; \
- PADDQ t0, v0; \
- PADDQ t0, v0; \
- PXOR v0, v6; \
- PSHUFB c48, v6; \
- MOVO v4, t0; \
- PMULULQ v6, t0; \
- PADDQ v6, v4; \
- PADDQ t0, v4; \
- PADDQ t0, v4; \
- PXOR v4, v2; \
- MOVO v2, t0; \
- PADDQ v2, t0; \
- PSRLQ $63, v2; \
- PXOR t0, v2; \
- MOVO v1, t0; \
- PMULULQ v3, t0; \
- PADDQ v3, v1; \
- PADDQ t0, v1; \
- PADDQ t0, v1; \
- PXOR v1, v7; \
- PSHUFD $0xB1, v7, v7; \
- MOVO v5, t0; \
- PMULULQ v7, t0; \
- PADDQ v7, v5; \
- PADDQ t0, v5; \
- PADDQ t0, v5; \
- PXOR v5, v3; \
- PSHUFB c40, v3; \
- MOVO v1, t0; \
- PMULULQ v3, t0; \
- PADDQ v3, v1; \
- PADDQ t0, v1; \
- PADDQ t0, v1; \
- PXOR v1, v7; \
- PSHUFB c48, v7; \
- MOVO v5, t0; \
- PMULULQ v7, t0; \
- PADDQ v7, v5; \
- PADDQ t0, v5; \
- PADDQ t0, v5; \
- PXOR v5, v3; \
- MOVO v3, t0; \
- PADDQ v3, t0; \
- PSRLQ $63, v3; \
- PXOR t0, v3
-
-#define LOAD_MSG_0(block, off) \
- MOVOU 8*(off+0)(block), X0; \
- MOVOU 8*(off+2)(block), X1; \
- MOVOU 8*(off+4)(block), X2; \
- MOVOU 8*(off+6)(block), X3; \
- MOVOU 8*(off+8)(block), X4; \
- MOVOU 8*(off+10)(block), X5; \
- MOVOU 8*(off+12)(block), X6; \
- MOVOU 8*(off+14)(block), X7
-
-#define STORE_MSG_0(block, off) \
- MOVOU X0, 8*(off+0)(block); \
- MOVOU X1, 8*(off+2)(block); \
- MOVOU X2, 8*(off+4)(block); \
- MOVOU X3, 8*(off+6)(block); \
- MOVOU X4, 8*(off+8)(block); \
- MOVOU X5, 8*(off+10)(block); \
- MOVOU X6, 8*(off+12)(block); \
- MOVOU X7, 8*(off+14)(block)
-
-#define LOAD_MSG_1(block, off) \
- MOVOU 8*off+0*8(block), X0; \
- MOVOU 8*off+16*8(block), X1; \
- MOVOU 8*off+32*8(block), X2; \
- MOVOU 8*off+48*8(block), X3; \
- MOVOU 8*off+64*8(block), X4; \
- MOVOU 8*off+80*8(block), X5; \
- MOVOU 8*off+96*8(block), X6; \
- MOVOU 8*off+112*8(block), X7
-
-#define STORE_MSG_1(block, off) \
- MOVOU X0, 8*off+0*8(block); \
- MOVOU X1, 8*off+16*8(block); \
- MOVOU X2, 8*off+32*8(block); \
- MOVOU X3, 8*off+48*8(block); \
- MOVOU X4, 8*off+64*8(block); \
- MOVOU X5, 8*off+80*8(block); \
- MOVOU X6, 8*off+96*8(block); \
- MOVOU X7, 8*off+112*8(block)
-
-#define BLAMKA_ROUND_0(block, off, t0, t1, c40, c48) \
- LOAD_MSG_0(block, off); \
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, t0, c40, c48); \
- SHUFFLE(X2, X3, X4, X5, X6, X7, t0, t1); \
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, t0, c40, c48); \
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, t0, t1); \
- STORE_MSG_0(block, off)
-
-#define BLAMKA_ROUND_1(block, off, t0, t1, c40, c48) \
- LOAD_MSG_1(block, off); \
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, t0, c40, c48); \
- SHUFFLE(X2, X3, X4, X5, X6, X7, t0, t1); \
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, t0, c40, c48); \
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, t0, t1); \
- STORE_MSG_1(block, off)
-
-// func blamkaSSE4(b *block)
-TEXT ·blamkaSSE4(SB), 4, $0-8
- MOVQ b+0(FP), AX
-
- MOVOU ·c40<>(SB), X10
- MOVOU ·c48<>(SB), X11
-
- BLAMKA_ROUND_0(AX, 0, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 16, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 32, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 48, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 64, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 80, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 96, X8, X9, X10, X11)
- BLAMKA_ROUND_0(AX, 112, X8, X9, X10, X11)
-
- BLAMKA_ROUND_1(AX, 0, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 2, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 4, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 6, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 8, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 10, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 12, X8, X9, X10, X11)
- BLAMKA_ROUND_1(AX, 14, X8, X9, X10, X11)
- RET
-
-// func mixBlocksSSE2(out, a, b, c *block)
-TEXT ·mixBlocksSSE2(SB), 4, $0-32
- MOVQ out+0(FP), DX
- MOVQ a+8(FP), AX
- MOVQ b+16(FP), BX
- MOVQ a+24(FP), CX
- MOVQ $128, BP
-
-loop:
- MOVOU 0(AX), X0
- MOVOU 0(BX), X1
- MOVOU 0(CX), X2
- PXOR X1, X0
- PXOR X2, X0
- MOVOU X0, 0(DX)
- ADDQ $16, AX
- ADDQ $16, BX
- ADDQ $16, CX
- ADDQ $16, DX
- SUBQ $2, BP
- JA loop
- RET
-
-// func xorBlocksSSE2(out, a, b, c *block)
-TEXT ·xorBlocksSSE2(SB), 4, $0-32
- MOVQ out+0(FP), DX
- MOVQ a+8(FP), AX
- MOVQ b+16(FP), BX
- MOVQ a+24(FP), CX
- MOVQ $128, BP
-
-loop:
- MOVOU 0(AX), X0
- MOVOU 0(BX), X1
- MOVOU 0(CX), X2
- MOVOU 0(DX), X3
- PXOR X1, X0
- PXOR X2, X0
- PXOR X3, X0
- MOVOU X0, 0(DX)
- ADDQ $16, AX
- ADDQ $16, BX
- ADDQ $16, CX
- ADDQ $16, DX
- SUBQ $2, BP
- JA loop
- RET
diff --git a/vendor/golang.org/x/crypto/argon2/blamka_generic.go b/vendor/golang.org/x/crypto/argon2/blamka_generic.go
deleted file mode 100644
index a481b22..0000000
--- a/vendor/golang.org/x/crypto/argon2/blamka_generic.go
+++ /dev/null
@@ -1,163 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package argon2
-
-var useSSE4 bool
-
-func processBlockGeneric(out, in1, in2 *block, xor bool) {
- var t block
- for i := range t {
- t[i] = in1[i] ^ in2[i]
- }
- for i := 0; i < blockLength; i += 16 {
- blamkaGeneric(
- &t[i+0], &t[i+1], &t[i+2], &t[i+3],
- &t[i+4], &t[i+5], &t[i+6], &t[i+7],
- &t[i+8], &t[i+9], &t[i+10], &t[i+11],
- &t[i+12], &t[i+13], &t[i+14], &t[i+15],
- )
- }
- for i := 0; i < blockLength/8; i += 2 {
- blamkaGeneric(
- &t[i], &t[i+1], &t[16+i], &t[16+i+1],
- &t[32+i], &t[32+i+1], &t[48+i], &t[48+i+1],
- &t[64+i], &t[64+i+1], &t[80+i], &t[80+i+1],
- &t[96+i], &t[96+i+1], &t[112+i], &t[112+i+1],
- )
- }
- if xor {
- for i := range t {
- out[i] ^= in1[i] ^ in2[i] ^ t[i]
- }
- } else {
- for i := range t {
- out[i] = in1[i] ^ in2[i] ^ t[i]
- }
- }
-}
-
-func blamkaGeneric(t00, t01, t02, t03, t04, t05, t06, t07, t08, t09, t10, t11, t12, t13, t14, t15 *uint64) {
- v00, v01, v02, v03 := *t00, *t01, *t02, *t03
- v04, v05, v06, v07 := *t04, *t05, *t06, *t07
- v08, v09, v10, v11 := *t08, *t09, *t10, *t11
- v12, v13, v14, v15 := *t12, *t13, *t14, *t15
-
- v00 += v04 + 2*uint64(uint32(v00))*uint64(uint32(v04))
- v12 ^= v00
- v12 = v12>>32 | v12<<32
- v08 += v12 + 2*uint64(uint32(v08))*uint64(uint32(v12))
- v04 ^= v08
- v04 = v04>>24 | v04<<40
-
- v00 += v04 + 2*uint64(uint32(v00))*uint64(uint32(v04))
- v12 ^= v00
- v12 = v12>>16 | v12<<48
- v08 += v12 + 2*uint64(uint32(v08))*uint64(uint32(v12))
- v04 ^= v08
- v04 = v04>>63 | v04<<1
-
- v01 += v05 + 2*uint64(uint32(v01))*uint64(uint32(v05))
- v13 ^= v01
- v13 = v13>>32 | v13<<32
- v09 += v13 + 2*uint64(uint32(v09))*uint64(uint32(v13))
- v05 ^= v09
- v05 = v05>>24 | v05<<40
-
- v01 += v05 + 2*uint64(uint32(v01))*uint64(uint32(v05))
- v13 ^= v01
- v13 = v13>>16 | v13<<48
- v09 += v13 + 2*uint64(uint32(v09))*uint64(uint32(v13))
- v05 ^= v09
- v05 = v05>>63 | v05<<1
-
- v02 += v06 + 2*uint64(uint32(v02))*uint64(uint32(v06))
- v14 ^= v02
- v14 = v14>>32 | v14<<32
- v10 += v14 + 2*uint64(uint32(v10))*uint64(uint32(v14))
- v06 ^= v10
- v06 = v06>>24 | v06<<40
-
- v02 += v06 + 2*uint64(uint32(v02))*uint64(uint32(v06))
- v14 ^= v02
- v14 = v14>>16 | v14<<48
- v10 += v14 + 2*uint64(uint32(v10))*uint64(uint32(v14))
- v06 ^= v10
- v06 = v06>>63 | v06<<1
-
- v03 += v07 + 2*uint64(uint32(v03))*uint64(uint32(v07))
- v15 ^= v03
- v15 = v15>>32 | v15<<32
- v11 += v15 + 2*uint64(uint32(v11))*uint64(uint32(v15))
- v07 ^= v11
- v07 = v07>>24 | v07<<40
-
- v03 += v07 + 2*uint64(uint32(v03))*uint64(uint32(v07))
- v15 ^= v03
- v15 = v15>>16 | v15<<48
- v11 += v15 + 2*uint64(uint32(v11))*uint64(uint32(v15))
- v07 ^= v11
- v07 = v07>>63 | v07<<1
-
- v00 += v05 + 2*uint64(uint32(v00))*uint64(uint32(v05))
- v15 ^= v00
- v15 = v15>>32 | v15<<32
- v10 += v15 + 2*uint64(uint32(v10))*uint64(uint32(v15))
- v05 ^= v10
- v05 = v05>>24 | v05<<40
-
- v00 += v05 + 2*uint64(uint32(v00))*uint64(uint32(v05))
- v15 ^= v00
- v15 = v15>>16 | v15<<48
- v10 += v15 + 2*uint64(uint32(v10))*uint64(uint32(v15))
- v05 ^= v10
- v05 = v05>>63 | v05<<1
-
- v01 += v06 + 2*uint64(uint32(v01))*uint64(uint32(v06))
- v12 ^= v01
- v12 = v12>>32 | v12<<32
- v11 += v12 + 2*uint64(uint32(v11))*uint64(uint32(v12))
- v06 ^= v11
- v06 = v06>>24 | v06<<40
-
- v01 += v06 + 2*uint64(uint32(v01))*uint64(uint32(v06))
- v12 ^= v01
- v12 = v12>>16 | v12<<48
- v11 += v12 + 2*uint64(uint32(v11))*uint64(uint32(v12))
- v06 ^= v11
- v06 = v06>>63 | v06<<1
-
- v02 += v07 + 2*uint64(uint32(v02))*uint64(uint32(v07))
- v13 ^= v02
- v13 = v13>>32 | v13<<32
- v08 += v13 + 2*uint64(uint32(v08))*uint64(uint32(v13))
- v07 ^= v08
- v07 = v07>>24 | v07<<40
-
- v02 += v07 + 2*uint64(uint32(v02))*uint64(uint32(v07))
- v13 ^= v02
- v13 = v13>>16 | v13<<48
- v08 += v13 + 2*uint64(uint32(v08))*uint64(uint32(v13))
- v07 ^= v08
- v07 = v07>>63 | v07<<1
-
- v03 += v04 + 2*uint64(uint32(v03))*uint64(uint32(v04))
- v14 ^= v03
- v14 = v14>>32 | v14<<32
- v09 += v14 + 2*uint64(uint32(v09))*uint64(uint32(v14))
- v04 ^= v09
- v04 = v04>>24 | v04<<40
-
- v03 += v04 + 2*uint64(uint32(v03))*uint64(uint32(v04))
- v14 ^= v03
- v14 = v14>>16 | v14<<48
- v09 += v14 + 2*uint64(uint32(v09))*uint64(uint32(v14))
- v04 ^= v09
- v04 = v04>>63 | v04<<1
-
- *t00, *t01, *t02, *t03 = v00, v01, v02, v03
- *t04, *t05, *t06, *t07 = v04, v05, v06, v07
- *t08, *t09, *t10, *t11 = v08, v09, v10, v11
- *t12, *t13, *t14, *t15 = v12, v13, v14, v15
-}
diff --git a/vendor/golang.org/x/crypto/argon2/blamka_ref.go b/vendor/golang.org/x/crypto/argon2/blamka_ref.go
deleted file mode 100644
index 167c59d..0000000
--- a/vendor/golang.org/x/crypto/argon2/blamka_ref.go
+++ /dev/null
@@ -1,16 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !amd64 || purego || !gc
-// +build !amd64 purego !gc
-
-package argon2
-
-func processBlock(out, in1, in2 *block) {
- processBlockGeneric(out, in1, in2, false)
-}
-
-func processBlockXOR(out, in1, in2 *block) {
- processBlockGeneric(out, in1, in2, true)
-}
diff --git a/vendor/golang.org/x/crypto/bcrypt/base64.go b/vendor/golang.org/x/crypto/bcrypt/base64.go
deleted file mode 100644
index fc31160..0000000
--- a/vendor/golang.org/x/crypto/bcrypt/base64.go
+++ /dev/null
@@ -1,35 +0,0 @@
-// Copyright 2011 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package bcrypt
-
-import "encoding/base64"
-
-const alphabet = "./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
-
-var bcEncoding = base64.NewEncoding(alphabet)
-
-func base64Encode(src []byte) []byte {
- n := bcEncoding.EncodedLen(len(src))
- dst := make([]byte, n)
- bcEncoding.Encode(dst, src)
- for dst[n-1] == '=' {
- n--
- }
- return dst[:n]
-}
-
-func base64Decode(src []byte) ([]byte, error) {
- numOfEquals := 4 - (len(src) % 4)
- for i := 0; i < numOfEquals; i++ {
- src = append(src, '=')
- }
-
- dst := make([]byte, bcEncoding.DecodedLen(len(src)))
- n, err := bcEncoding.Decode(dst, src)
- if err != nil {
- return nil, err
- }
- return dst[:n], nil
-}
diff --git a/vendor/golang.org/x/crypto/bcrypt/bcrypt.go b/vendor/golang.org/x/crypto/bcrypt/bcrypt.go
deleted file mode 100644
index aeb73f8..0000000
--- a/vendor/golang.org/x/crypto/bcrypt/bcrypt.go
+++ /dev/null
@@ -1,295 +0,0 @@
-// Copyright 2011 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package bcrypt implements Provos and Mazières's bcrypt adaptive hashing
-// algorithm. See http://www.usenix.org/event/usenix99/provos/provos.pdf
-package bcrypt // import "golang.org/x/crypto/bcrypt"
-
-// The code is a port of Provos and Mazières's C implementation.
-import (
- "crypto/rand"
- "crypto/subtle"
- "errors"
- "fmt"
- "io"
- "strconv"
-
- "golang.org/x/crypto/blowfish"
-)
-
-const (
- MinCost int = 4 // the minimum allowable cost as passed in to GenerateFromPassword
- MaxCost int = 31 // the maximum allowable cost as passed in to GenerateFromPassword
- DefaultCost int = 10 // the cost that will actually be set if a cost below MinCost is passed into GenerateFromPassword
-)
-
-// The error returned from CompareHashAndPassword when a password and hash do
-// not match.
-var ErrMismatchedHashAndPassword = errors.New("crypto/bcrypt: hashedPassword is not the hash of the given password")
-
-// The error returned from CompareHashAndPassword when a hash is too short to
-// be a bcrypt hash.
-var ErrHashTooShort = errors.New("crypto/bcrypt: hashedSecret too short to be a bcrypted password")
-
-// The error returned from CompareHashAndPassword when a hash was created with
-// a bcrypt algorithm newer than this implementation.
-type HashVersionTooNewError byte
-
-func (hv HashVersionTooNewError) Error() string {
- return fmt.Sprintf("crypto/bcrypt: bcrypt algorithm version '%c' requested is newer than current version '%c'", byte(hv), majorVersion)
-}
-
-// The error returned from CompareHashAndPassword when a hash starts with something other than '$'
-type InvalidHashPrefixError byte
-
-func (ih InvalidHashPrefixError) Error() string {
- return fmt.Sprintf("crypto/bcrypt: bcrypt hashes must start with '$', but hashedSecret started with '%c'", byte(ih))
-}
-
-type InvalidCostError int
-
-func (ic InvalidCostError) Error() string {
- return fmt.Sprintf("crypto/bcrypt: cost %d is outside allowed range (%d,%d)", int(ic), int(MinCost), int(MaxCost))
-}
-
-const (
- majorVersion = '2'
- minorVersion = 'a'
- maxSaltSize = 16
- maxCryptedHashSize = 23
- encodedSaltSize = 22
- encodedHashSize = 31
- minHashSize = 59
-)
-
-// magicCipherData is an IV for the 64 Blowfish encryption calls in
-// bcrypt(). It's the string "OrpheanBeholderScryDoubt" in big-endian bytes.
-var magicCipherData = []byte{
- 0x4f, 0x72, 0x70, 0x68,
- 0x65, 0x61, 0x6e, 0x42,
- 0x65, 0x68, 0x6f, 0x6c,
- 0x64, 0x65, 0x72, 0x53,
- 0x63, 0x72, 0x79, 0x44,
- 0x6f, 0x75, 0x62, 0x74,
-}
-
-type hashed struct {
- hash []byte
- salt []byte
- cost int // allowed range is MinCost to MaxCost
- major byte
- minor byte
-}
-
-// GenerateFromPassword returns the bcrypt hash of the password at the given
-// cost. If the cost given is less than MinCost, the cost will be set to
-// DefaultCost, instead. Use CompareHashAndPassword, as defined in this package,
-// to compare the returned hashed password with its cleartext version.
-func GenerateFromPassword(password []byte, cost int) ([]byte, error) {
- p, err := newFromPassword(password, cost)
- if err != nil {
- return nil, err
- }
- return p.Hash(), nil
-}
-
-// CompareHashAndPassword compares a bcrypt hashed password with its possible
-// plaintext equivalent. Returns nil on success, or an error on failure.
-func CompareHashAndPassword(hashedPassword, password []byte) error {
- p, err := newFromHash(hashedPassword)
- if err != nil {
- return err
- }
-
- otherHash, err := bcrypt(password, p.cost, p.salt)
- if err != nil {
- return err
- }
-
- otherP := &hashed{otherHash, p.salt, p.cost, p.major, p.minor}
- if subtle.ConstantTimeCompare(p.Hash(), otherP.Hash()) == 1 {
- return nil
- }
-
- return ErrMismatchedHashAndPassword
-}
-
-// Cost returns the hashing cost used to create the given hashed
-// password. When, in the future, the hashing cost of a password system needs
-// to be increased in order to adjust for greater computational power, this
-// function allows one to establish which passwords need to be updated.
-func Cost(hashedPassword []byte) (int, error) {
- p, err := newFromHash(hashedPassword)
- if err != nil {
- return 0, err
- }
- return p.cost, nil
-}
-
-func newFromPassword(password []byte, cost int) (*hashed, error) {
- if cost < MinCost {
- cost = DefaultCost
- }
- p := new(hashed)
- p.major = majorVersion
- p.minor = minorVersion
-
- err := checkCost(cost)
- if err != nil {
- return nil, err
- }
- p.cost = cost
-
- unencodedSalt := make([]byte, maxSaltSize)
- _, err = io.ReadFull(rand.Reader, unencodedSalt)
- if err != nil {
- return nil, err
- }
-
- p.salt = base64Encode(unencodedSalt)
- hash, err := bcrypt(password, p.cost, p.salt)
- if err != nil {
- return nil, err
- }
- p.hash = hash
- return p, err
-}
-
-func newFromHash(hashedSecret []byte) (*hashed, error) {
- if len(hashedSecret) < minHashSize {
- return nil, ErrHashTooShort
- }
- p := new(hashed)
- n, err := p.decodeVersion(hashedSecret)
- if err != nil {
- return nil, err
- }
- hashedSecret = hashedSecret[n:]
- n, err = p.decodeCost(hashedSecret)
- if err != nil {
- return nil, err
- }
- hashedSecret = hashedSecret[n:]
-
- // The "+2" is here because we'll have to append at most 2 '=' to the salt
- // when base64 decoding it in expensiveBlowfishSetup().
- p.salt = make([]byte, encodedSaltSize, encodedSaltSize+2)
- copy(p.salt, hashedSecret[:encodedSaltSize])
-
- hashedSecret = hashedSecret[encodedSaltSize:]
- p.hash = make([]byte, len(hashedSecret))
- copy(p.hash, hashedSecret)
-
- return p, nil
-}
-
-func bcrypt(password []byte, cost int, salt []byte) ([]byte, error) {
- cipherData := make([]byte, len(magicCipherData))
- copy(cipherData, magicCipherData)
-
- c, err := expensiveBlowfishSetup(password, uint32(cost), salt)
- if err != nil {
- return nil, err
- }
-
- for i := 0; i < 24; i += 8 {
- for j := 0; j < 64; j++ {
- c.Encrypt(cipherData[i:i+8], cipherData[i:i+8])
- }
- }
-
- // Bug compatibility with C bcrypt implementations. We only encode 23 of
- // the 24 bytes encrypted.
- hsh := base64Encode(cipherData[:maxCryptedHashSize])
- return hsh, nil
-}
-
-func expensiveBlowfishSetup(key []byte, cost uint32, salt []byte) (*blowfish.Cipher, error) {
- csalt, err := base64Decode(salt)
- if err != nil {
- return nil, err
- }
-
- // Bug compatibility with C bcrypt implementations. They use the trailing
- // NULL in the key string during expansion.
- // We copy the key to prevent changing the underlying array.
- ckey := append(key[:len(key):len(key)], 0)
-
- c, err := blowfish.NewSaltedCipher(ckey, csalt)
- if err != nil {
- return nil, err
- }
-
- var i, rounds uint64
- rounds = 1 << cost
- for i = 0; i < rounds; i++ {
- blowfish.ExpandKey(ckey, c)
- blowfish.ExpandKey(csalt, c)
- }
-
- return c, nil
-}
-
-func (p *hashed) Hash() []byte {
- arr := make([]byte, 60)
- arr[0] = '$'
- arr[1] = p.major
- n := 2
- if p.minor != 0 {
- arr[2] = p.minor
- n = 3
- }
- arr[n] = '$'
- n++
- copy(arr[n:], []byte(fmt.Sprintf("%02d", p.cost)))
- n += 2
- arr[n] = '$'
- n++
- copy(arr[n:], p.salt)
- n += encodedSaltSize
- copy(arr[n:], p.hash)
- n += encodedHashSize
- return arr[:n]
-}
-
-func (p *hashed) decodeVersion(sbytes []byte) (int, error) {
- if sbytes[0] != '$' {
- return -1, InvalidHashPrefixError(sbytes[0])
- }
- if sbytes[1] > majorVersion {
- return -1, HashVersionTooNewError(sbytes[1])
- }
- p.major = sbytes[1]
- n := 3
- if sbytes[2] != '$' {
- p.minor = sbytes[2]
- n++
- }
- return n, nil
-}
-
-// sbytes should begin where decodeVersion left off.
-func (p *hashed) decodeCost(sbytes []byte) (int, error) {
- cost, err := strconv.Atoi(string(sbytes[0:2]))
- if err != nil {
- return -1, err
- }
- err = checkCost(cost)
- if err != nil {
- return -1, err
- }
- p.cost = cost
- return 3, nil
-}
-
-func (p *hashed) String() string {
- return fmt.Sprintf("&{hash: %#v, salt: %#v, cost: %d, major: %c, minor: %c}", string(p.hash), p.salt, p.cost, p.major, p.minor)
-}
-
-func checkCost(cost int) error {
- if cost < MinCost || cost > MaxCost {
- return InvalidCostError(cost)
- }
- return nil
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2b.go b/vendor/golang.org/x/crypto/blake2b/blake2b.go
deleted file mode 100644
index d2e98d4..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2b.go
+++ /dev/null
@@ -1,291 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package blake2b implements the BLAKE2b hash algorithm defined by RFC 7693
-// and the extendable output function (XOF) BLAKE2Xb.
-//
-// BLAKE2b is optimized for 64-bit platforms—including NEON-enabled ARMs—and
-// produces digests of any size between 1 and 64 bytes.
-// For a detailed specification of BLAKE2b see https://blake2.net/blake2.pdf
-// and for BLAKE2Xb see https://blake2.net/blake2x.pdf
-//
-// If you aren't sure which function you need, use BLAKE2b (Sum512 or New512).
-// If you need a secret-key MAC (message authentication code), use the New512
-// function with a non-nil key.
-//
-// BLAKE2X is a construction to compute hash values larger than 64 bytes. It
-// can produce hash values between 0 and 4 GiB.
-package blake2b
-
-import (
- "encoding/binary"
- "errors"
- "hash"
-)
-
-const (
- // The blocksize of BLAKE2b in bytes.
- BlockSize = 128
- // The hash size of BLAKE2b-512 in bytes.
- Size = 64
- // The hash size of BLAKE2b-384 in bytes.
- Size384 = 48
- // The hash size of BLAKE2b-256 in bytes.
- Size256 = 32
-)
-
-var (
- useAVX2 bool
- useAVX bool
- useSSE4 bool
-)
-
-var (
- errKeySize = errors.New("blake2b: invalid key size")
- errHashSize = errors.New("blake2b: invalid hash size")
-)
-
-var iv = [8]uint64{
- 0x6a09e667f3bcc908, 0xbb67ae8584caa73b, 0x3c6ef372fe94f82b, 0xa54ff53a5f1d36f1,
- 0x510e527fade682d1, 0x9b05688c2b3e6c1f, 0x1f83d9abfb41bd6b, 0x5be0cd19137e2179,
-}
-
-// Sum512 returns the BLAKE2b-512 checksum of the data.
-func Sum512(data []byte) [Size]byte {
- var sum [Size]byte
- checkSum(&sum, Size, data)
- return sum
-}
-
-// Sum384 returns the BLAKE2b-384 checksum of the data.
-func Sum384(data []byte) [Size384]byte {
- var sum [Size]byte
- var sum384 [Size384]byte
- checkSum(&sum, Size384, data)
- copy(sum384[:], sum[:Size384])
- return sum384
-}
-
-// Sum256 returns the BLAKE2b-256 checksum of the data.
-func Sum256(data []byte) [Size256]byte {
- var sum [Size]byte
- var sum256 [Size256]byte
- checkSum(&sum, Size256, data)
- copy(sum256[:], sum[:Size256])
- return sum256
-}
-
-// New512 returns a new hash.Hash computing the BLAKE2b-512 checksum. A non-nil
-// key turns the hash into a MAC. The key must be between zero and 64 bytes long.
-func New512(key []byte) (hash.Hash, error) { return newDigest(Size, key) }
-
-// New384 returns a new hash.Hash computing the BLAKE2b-384 checksum. A non-nil
-// key turns the hash into a MAC. The key must be between zero and 64 bytes long.
-func New384(key []byte) (hash.Hash, error) { return newDigest(Size384, key) }
-
-// New256 returns a new hash.Hash computing the BLAKE2b-256 checksum. A non-nil
-// key turns the hash into a MAC. The key must be between zero and 64 bytes long.
-func New256(key []byte) (hash.Hash, error) { return newDigest(Size256, key) }
-
-// New returns a new hash.Hash computing the BLAKE2b checksum with a custom length.
-// A non-nil key turns the hash into a MAC. The key must be between zero and 64 bytes long.
-// The hash size can be a value between 1 and 64 but it is highly recommended to use
-// values equal or greater than:
-// - 32 if BLAKE2b is used as a hash function (The key is zero bytes long).
-// - 16 if BLAKE2b is used as a MAC function (The key is at least 16 bytes long).
-// When the key is nil, the returned hash.Hash implements BinaryMarshaler
-// and BinaryUnmarshaler for state (de)serialization as documented by hash.Hash.
-func New(size int, key []byte) (hash.Hash, error) { return newDigest(size, key) }
-
-func newDigest(hashSize int, key []byte) (*digest, error) {
- if hashSize < 1 || hashSize > Size {
- return nil, errHashSize
- }
- if len(key) > Size {
- return nil, errKeySize
- }
- d := &digest{
- size: hashSize,
- keyLen: len(key),
- }
- copy(d.key[:], key)
- d.Reset()
- return d, nil
-}
-
-func checkSum(sum *[Size]byte, hashSize int, data []byte) {
- h := iv
- h[0] ^= uint64(hashSize) | (1 << 16) | (1 << 24)
- var c [2]uint64
-
- if length := len(data); length > BlockSize {
- n := length &^ (BlockSize - 1)
- if length == n {
- n -= BlockSize
- }
- hashBlocks(&h, &c, 0, data[:n])
- data = data[n:]
- }
-
- var block [BlockSize]byte
- offset := copy(block[:], data)
- remaining := uint64(BlockSize - offset)
- if c[0] < remaining {
- c[1]--
- }
- c[0] -= remaining
-
- hashBlocks(&h, &c, 0xFFFFFFFFFFFFFFFF, block[:])
-
- for i, v := range h[:(hashSize+7)/8] {
- binary.LittleEndian.PutUint64(sum[8*i:], v)
- }
-}
-
-type digest struct {
- h [8]uint64
- c [2]uint64
- size int
- block [BlockSize]byte
- offset int
-
- key [BlockSize]byte
- keyLen int
-}
-
-const (
- magic = "b2b"
- marshaledSize = len(magic) + 8*8 + 2*8 + 1 + BlockSize + 1
-)
-
-func (d *digest) MarshalBinary() ([]byte, error) {
- if d.keyLen != 0 {
- return nil, errors.New("crypto/blake2b: cannot marshal MACs")
- }
- b := make([]byte, 0, marshaledSize)
- b = append(b, magic...)
- for i := 0; i < 8; i++ {
- b = appendUint64(b, d.h[i])
- }
- b = appendUint64(b, d.c[0])
- b = appendUint64(b, d.c[1])
- // Maximum value for size is 64
- b = append(b, byte(d.size))
- b = append(b, d.block[:]...)
- b = append(b, byte(d.offset))
- return b, nil
-}
-
-func (d *digest) UnmarshalBinary(b []byte) error {
- if len(b) < len(magic) || string(b[:len(magic)]) != magic {
- return errors.New("crypto/blake2b: invalid hash state identifier")
- }
- if len(b) != marshaledSize {
- return errors.New("crypto/blake2b: invalid hash state size")
- }
- b = b[len(magic):]
- for i := 0; i < 8; i++ {
- b, d.h[i] = consumeUint64(b)
- }
- b, d.c[0] = consumeUint64(b)
- b, d.c[1] = consumeUint64(b)
- d.size = int(b[0])
- b = b[1:]
- copy(d.block[:], b[:BlockSize])
- b = b[BlockSize:]
- d.offset = int(b[0])
- return nil
-}
-
-func (d *digest) BlockSize() int { return BlockSize }
-
-func (d *digest) Size() int { return d.size }
-
-func (d *digest) Reset() {
- d.h = iv
- d.h[0] ^= uint64(d.size) | (uint64(d.keyLen) << 8) | (1 << 16) | (1 << 24)
- d.offset, d.c[0], d.c[1] = 0, 0, 0
- if d.keyLen > 0 {
- d.block = d.key
- d.offset = BlockSize
- }
-}
-
-func (d *digest) Write(p []byte) (n int, err error) {
- n = len(p)
-
- if d.offset > 0 {
- remaining := BlockSize - d.offset
- if n <= remaining {
- d.offset += copy(d.block[d.offset:], p)
- return
- }
- copy(d.block[d.offset:], p[:remaining])
- hashBlocks(&d.h, &d.c, 0, d.block[:])
- d.offset = 0
- p = p[remaining:]
- }
-
- if length := len(p); length > BlockSize {
- nn := length &^ (BlockSize - 1)
- if length == nn {
- nn -= BlockSize
- }
- hashBlocks(&d.h, &d.c, 0, p[:nn])
- p = p[nn:]
- }
-
- if len(p) > 0 {
- d.offset += copy(d.block[:], p)
- }
-
- return
-}
-
-func (d *digest) Sum(sum []byte) []byte {
- var hash [Size]byte
- d.finalize(&hash)
- return append(sum, hash[:d.size]...)
-}
-
-func (d *digest) finalize(hash *[Size]byte) {
- var block [BlockSize]byte
- copy(block[:], d.block[:d.offset])
- remaining := uint64(BlockSize - d.offset)
-
- c := d.c
- if c[0] < remaining {
- c[1]--
- }
- c[0] -= remaining
-
- h := d.h
- hashBlocks(&h, &c, 0xFFFFFFFFFFFFFFFF, block[:])
-
- for i, v := range h {
- binary.LittleEndian.PutUint64(hash[8*i:], v)
- }
-}
-
-func appendUint64(b []byte, x uint64) []byte {
- var a [8]byte
- binary.BigEndian.PutUint64(a[:], x)
- return append(b, a[:]...)
-}
-
-func appendUint32(b []byte, x uint32) []byte {
- var a [4]byte
- binary.BigEndian.PutUint32(a[:], x)
- return append(b, a[:]...)
-}
-
-func consumeUint64(b []byte) ([]byte, uint64) {
- x := binary.BigEndian.Uint64(b)
- return b[8:], x
-}
-
-func consumeUint32(b []byte) ([]byte, uint32) {
- x := binary.BigEndian.Uint32(b)
- return b[4:], x
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.go b/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.go
deleted file mode 100644
index 56bfaaa..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.go
+++ /dev/null
@@ -1,38 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build go1.7 && amd64 && gc && !purego
-// +build go1.7,amd64,gc,!purego
-
-package blake2b
-
-import "golang.org/x/sys/cpu"
-
-func init() {
- useAVX2 = cpu.X86.HasAVX2
- useAVX = cpu.X86.HasAVX
- useSSE4 = cpu.X86.HasSSE41
-}
-
-//go:noescape
-func hashBlocksAVX2(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-
-//go:noescape
-func hashBlocksAVX(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-
-//go:noescape
-func hashBlocksSSE4(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-
-func hashBlocks(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte) {
- switch {
- case useAVX2:
- hashBlocksAVX2(h, c, flag, blocks)
- case useAVX:
- hashBlocksAVX(h, c, flag, blocks)
- case useSSE4:
- hashBlocksSSE4(h, c, flag, blocks)
- default:
- hashBlocksGeneric(h, c, flag, blocks)
- }
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.s b/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.s
deleted file mode 100644
index 4b9daa1..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2bAVX2_amd64.s
+++ /dev/null
@@ -1,745 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build go1.7 && amd64 && gc && !purego
-// +build go1.7,amd64,gc,!purego
-
-#include "textflag.h"
-
-DATA ·AVX2_iv0<>+0x00(SB)/8, $0x6a09e667f3bcc908
-DATA ·AVX2_iv0<>+0x08(SB)/8, $0xbb67ae8584caa73b
-DATA ·AVX2_iv0<>+0x10(SB)/8, $0x3c6ef372fe94f82b
-DATA ·AVX2_iv0<>+0x18(SB)/8, $0xa54ff53a5f1d36f1
-GLOBL ·AVX2_iv0<>(SB), (NOPTR+RODATA), $32
-
-DATA ·AVX2_iv1<>+0x00(SB)/8, $0x510e527fade682d1
-DATA ·AVX2_iv1<>+0x08(SB)/8, $0x9b05688c2b3e6c1f
-DATA ·AVX2_iv1<>+0x10(SB)/8, $0x1f83d9abfb41bd6b
-DATA ·AVX2_iv1<>+0x18(SB)/8, $0x5be0cd19137e2179
-GLOBL ·AVX2_iv1<>(SB), (NOPTR+RODATA), $32
-
-DATA ·AVX2_c40<>+0x00(SB)/8, $0x0201000706050403
-DATA ·AVX2_c40<>+0x08(SB)/8, $0x0a09080f0e0d0c0b
-DATA ·AVX2_c40<>+0x10(SB)/8, $0x0201000706050403
-DATA ·AVX2_c40<>+0x18(SB)/8, $0x0a09080f0e0d0c0b
-GLOBL ·AVX2_c40<>(SB), (NOPTR+RODATA), $32
-
-DATA ·AVX2_c48<>+0x00(SB)/8, $0x0100070605040302
-DATA ·AVX2_c48<>+0x08(SB)/8, $0x09080f0e0d0c0b0a
-DATA ·AVX2_c48<>+0x10(SB)/8, $0x0100070605040302
-DATA ·AVX2_c48<>+0x18(SB)/8, $0x09080f0e0d0c0b0a
-GLOBL ·AVX2_c48<>(SB), (NOPTR+RODATA), $32
-
-DATA ·AVX_iv0<>+0x00(SB)/8, $0x6a09e667f3bcc908
-DATA ·AVX_iv0<>+0x08(SB)/8, $0xbb67ae8584caa73b
-GLOBL ·AVX_iv0<>(SB), (NOPTR+RODATA), $16
-
-DATA ·AVX_iv1<>+0x00(SB)/8, $0x3c6ef372fe94f82b
-DATA ·AVX_iv1<>+0x08(SB)/8, $0xa54ff53a5f1d36f1
-GLOBL ·AVX_iv1<>(SB), (NOPTR+RODATA), $16
-
-DATA ·AVX_iv2<>+0x00(SB)/8, $0x510e527fade682d1
-DATA ·AVX_iv2<>+0x08(SB)/8, $0x9b05688c2b3e6c1f
-GLOBL ·AVX_iv2<>(SB), (NOPTR+RODATA), $16
-
-DATA ·AVX_iv3<>+0x00(SB)/8, $0x1f83d9abfb41bd6b
-DATA ·AVX_iv3<>+0x08(SB)/8, $0x5be0cd19137e2179
-GLOBL ·AVX_iv3<>(SB), (NOPTR+RODATA), $16
-
-DATA ·AVX_c40<>+0x00(SB)/8, $0x0201000706050403
-DATA ·AVX_c40<>+0x08(SB)/8, $0x0a09080f0e0d0c0b
-GLOBL ·AVX_c40<>(SB), (NOPTR+RODATA), $16
-
-DATA ·AVX_c48<>+0x00(SB)/8, $0x0100070605040302
-DATA ·AVX_c48<>+0x08(SB)/8, $0x09080f0e0d0c0b0a
-GLOBL ·AVX_c48<>(SB), (NOPTR+RODATA), $16
-
-#define VPERMQ_0x39_Y1_Y1 BYTE $0xc4; BYTE $0xe3; BYTE $0xfd; BYTE $0x00; BYTE $0xc9; BYTE $0x39
-#define VPERMQ_0x93_Y1_Y1 BYTE $0xc4; BYTE $0xe3; BYTE $0xfd; BYTE $0x00; BYTE $0xc9; BYTE $0x93
-#define VPERMQ_0x4E_Y2_Y2 BYTE $0xc4; BYTE $0xe3; BYTE $0xfd; BYTE $0x00; BYTE $0xd2; BYTE $0x4e
-#define VPERMQ_0x93_Y3_Y3 BYTE $0xc4; BYTE $0xe3; BYTE $0xfd; BYTE $0x00; BYTE $0xdb; BYTE $0x93
-#define VPERMQ_0x39_Y3_Y3 BYTE $0xc4; BYTE $0xe3; BYTE $0xfd; BYTE $0x00; BYTE $0xdb; BYTE $0x39
-
-#define ROUND_AVX2(m0, m1, m2, m3, t, c40, c48) \
- VPADDQ m0, Y0, Y0; \
- VPADDQ Y1, Y0, Y0; \
- VPXOR Y0, Y3, Y3; \
- VPSHUFD $-79, Y3, Y3; \
- VPADDQ Y3, Y2, Y2; \
- VPXOR Y2, Y1, Y1; \
- VPSHUFB c40, Y1, Y1; \
- VPADDQ m1, Y0, Y0; \
- VPADDQ Y1, Y0, Y0; \
- VPXOR Y0, Y3, Y3; \
- VPSHUFB c48, Y3, Y3; \
- VPADDQ Y3, Y2, Y2; \
- VPXOR Y2, Y1, Y1; \
- VPADDQ Y1, Y1, t; \
- VPSRLQ $63, Y1, Y1; \
- VPXOR t, Y1, Y1; \
- VPERMQ_0x39_Y1_Y1; \
- VPERMQ_0x4E_Y2_Y2; \
- VPERMQ_0x93_Y3_Y3; \
- VPADDQ m2, Y0, Y0; \
- VPADDQ Y1, Y0, Y0; \
- VPXOR Y0, Y3, Y3; \
- VPSHUFD $-79, Y3, Y3; \
- VPADDQ Y3, Y2, Y2; \
- VPXOR Y2, Y1, Y1; \
- VPSHUFB c40, Y1, Y1; \
- VPADDQ m3, Y0, Y0; \
- VPADDQ Y1, Y0, Y0; \
- VPXOR Y0, Y3, Y3; \
- VPSHUFB c48, Y3, Y3; \
- VPADDQ Y3, Y2, Y2; \
- VPXOR Y2, Y1, Y1; \
- VPADDQ Y1, Y1, t; \
- VPSRLQ $63, Y1, Y1; \
- VPXOR t, Y1, Y1; \
- VPERMQ_0x39_Y3_Y3; \
- VPERMQ_0x4E_Y2_Y2; \
- VPERMQ_0x93_Y1_Y1
-
-#define VMOVQ_SI_X11_0 BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x1E
-#define VMOVQ_SI_X12_0 BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x26
-#define VMOVQ_SI_X13_0 BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x2E
-#define VMOVQ_SI_X14_0 BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x36
-#define VMOVQ_SI_X15_0 BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x3E
-
-#define VMOVQ_SI_X11(n) BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x5E; BYTE $n
-#define VMOVQ_SI_X12(n) BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x66; BYTE $n
-#define VMOVQ_SI_X13(n) BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x6E; BYTE $n
-#define VMOVQ_SI_X14(n) BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x76; BYTE $n
-#define VMOVQ_SI_X15(n) BYTE $0xC5; BYTE $0x7A; BYTE $0x7E; BYTE $0x7E; BYTE $n
-
-#define VPINSRQ_1_SI_X11_0 BYTE $0xC4; BYTE $0x63; BYTE $0xA1; BYTE $0x22; BYTE $0x1E; BYTE $0x01
-#define VPINSRQ_1_SI_X12_0 BYTE $0xC4; BYTE $0x63; BYTE $0x99; BYTE $0x22; BYTE $0x26; BYTE $0x01
-#define VPINSRQ_1_SI_X13_0 BYTE $0xC4; BYTE $0x63; BYTE $0x91; BYTE $0x22; BYTE $0x2E; BYTE $0x01
-#define VPINSRQ_1_SI_X14_0 BYTE $0xC4; BYTE $0x63; BYTE $0x89; BYTE $0x22; BYTE $0x36; BYTE $0x01
-#define VPINSRQ_1_SI_X15_0 BYTE $0xC4; BYTE $0x63; BYTE $0x81; BYTE $0x22; BYTE $0x3E; BYTE $0x01
-
-#define VPINSRQ_1_SI_X11(n) BYTE $0xC4; BYTE $0x63; BYTE $0xA1; BYTE $0x22; BYTE $0x5E; BYTE $n; BYTE $0x01
-#define VPINSRQ_1_SI_X12(n) BYTE $0xC4; BYTE $0x63; BYTE $0x99; BYTE $0x22; BYTE $0x66; BYTE $n; BYTE $0x01
-#define VPINSRQ_1_SI_X13(n) BYTE $0xC4; BYTE $0x63; BYTE $0x91; BYTE $0x22; BYTE $0x6E; BYTE $n; BYTE $0x01
-#define VPINSRQ_1_SI_X14(n) BYTE $0xC4; BYTE $0x63; BYTE $0x89; BYTE $0x22; BYTE $0x76; BYTE $n; BYTE $0x01
-#define VPINSRQ_1_SI_X15(n) BYTE $0xC4; BYTE $0x63; BYTE $0x81; BYTE $0x22; BYTE $0x7E; BYTE $n; BYTE $0x01
-
-#define VMOVQ_R8_X15 BYTE $0xC4; BYTE $0x41; BYTE $0xF9; BYTE $0x6E; BYTE $0xF8
-#define VPINSRQ_1_R9_X15 BYTE $0xC4; BYTE $0x43; BYTE $0x81; BYTE $0x22; BYTE $0xF9; BYTE $0x01
-
-// load msg: Y12 = (i0, i1, i2, i3)
-// i0, i1, i2, i3 must not be 0
-#define LOAD_MSG_AVX2_Y12(i0, i1, i2, i3) \
- VMOVQ_SI_X12(i0*8); \
- VMOVQ_SI_X11(i2*8); \
- VPINSRQ_1_SI_X12(i1*8); \
- VPINSRQ_1_SI_X11(i3*8); \
- VINSERTI128 $1, X11, Y12, Y12
-
-// load msg: Y13 = (i0, i1, i2, i3)
-// i0, i1, i2, i3 must not be 0
-#define LOAD_MSG_AVX2_Y13(i0, i1, i2, i3) \
- VMOVQ_SI_X13(i0*8); \
- VMOVQ_SI_X11(i2*8); \
- VPINSRQ_1_SI_X13(i1*8); \
- VPINSRQ_1_SI_X11(i3*8); \
- VINSERTI128 $1, X11, Y13, Y13
-
-// load msg: Y14 = (i0, i1, i2, i3)
-// i0, i1, i2, i3 must not be 0
-#define LOAD_MSG_AVX2_Y14(i0, i1, i2, i3) \
- VMOVQ_SI_X14(i0*8); \
- VMOVQ_SI_X11(i2*8); \
- VPINSRQ_1_SI_X14(i1*8); \
- VPINSRQ_1_SI_X11(i3*8); \
- VINSERTI128 $1, X11, Y14, Y14
-
-// load msg: Y15 = (i0, i1, i2, i3)
-// i0, i1, i2, i3 must not be 0
-#define LOAD_MSG_AVX2_Y15(i0, i1, i2, i3) \
- VMOVQ_SI_X15(i0*8); \
- VMOVQ_SI_X11(i2*8); \
- VPINSRQ_1_SI_X15(i1*8); \
- VPINSRQ_1_SI_X11(i3*8); \
- VINSERTI128 $1, X11, Y15, Y15
-
-#define LOAD_MSG_AVX2_0_2_4_6_1_3_5_7_8_10_12_14_9_11_13_15() \
- VMOVQ_SI_X12_0; \
- VMOVQ_SI_X11(4*8); \
- VPINSRQ_1_SI_X12(2*8); \
- VPINSRQ_1_SI_X11(6*8); \
- VINSERTI128 $1, X11, Y12, Y12; \
- LOAD_MSG_AVX2_Y13(1, 3, 5, 7); \
- LOAD_MSG_AVX2_Y14(8, 10, 12, 14); \
- LOAD_MSG_AVX2_Y15(9, 11, 13, 15)
-
-#define LOAD_MSG_AVX2_14_4_9_13_10_8_15_6_1_0_11_5_12_2_7_3() \
- LOAD_MSG_AVX2_Y12(14, 4, 9, 13); \
- LOAD_MSG_AVX2_Y13(10, 8, 15, 6); \
- VMOVQ_SI_X11(11*8); \
- VPSHUFD $0x4E, 0*8(SI), X14; \
- VPINSRQ_1_SI_X11(5*8); \
- VINSERTI128 $1, X11, Y14, Y14; \
- LOAD_MSG_AVX2_Y15(12, 2, 7, 3)
-
-#define LOAD_MSG_AVX2_11_12_5_15_8_0_2_13_10_3_7_9_14_6_1_4() \
- VMOVQ_SI_X11(5*8); \
- VMOVDQU 11*8(SI), X12; \
- VPINSRQ_1_SI_X11(15*8); \
- VINSERTI128 $1, X11, Y12, Y12; \
- VMOVQ_SI_X13(8*8); \
- VMOVQ_SI_X11(2*8); \
- VPINSRQ_1_SI_X13_0; \
- VPINSRQ_1_SI_X11(13*8); \
- VINSERTI128 $1, X11, Y13, Y13; \
- LOAD_MSG_AVX2_Y14(10, 3, 7, 9); \
- LOAD_MSG_AVX2_Y15(14, 6, 1, 4)
-
-#define LOAD_MSG_AVX2_7_3_13_11_9_1_12_14_2_5_4_15_6_10_0_8() \
- LOAD_MSG_AVX2_Y12(7, 3, 13, 11); \
- LOAD_MSG_AVX2_Y13(9, 1, 12, 14); \
- LOAD_MSG_AVX2_Y14(2, 5, 4, 15); \
- VMOVQ_SI_X15(6*8); \
- VMOVQ_SI_X11_0; \
- VPINSRQ_1_SI_X15(10*8); \
- VPINSRQ_1_SI_X11(8*8); \
- VINSERTI128 $1, X11, Y15, Y15
-
-#define LOAD_MSG_AVX2_9_5_2_10_0_7_4_15_14_11_6_3_1_12_8_13() \
- LOAD_MSG_AVX2_Y12(9, 5, 2, 10); \
- VMOVQ_SI_X13_0; \
- VMOVQ_SI_X11(4*8); \
- VPINSRQ_1_SI_X13(7*8); \
- VPINSRQ_1_SI_X11(15*8); \
- VINSERTI128 $1, X11, Y13, Y13; \
- LOAD_MSG_AVX2_Y14(14, 11, 6, 3); \
- LOAD_MSG_AVX2_Y15(1, 12, 8, 13)
-
-#define LOAD_MSG_AVX2_2_6_0_8_12_10_11_3_4_7_15_1_13_5_14_9() \
- VMOVQ_SI_X12(2*8); \
- VMOVQ_SI_X11_0; \
- VPINSRQ_1_SI_X12(6*8); \
- VPINSRQ_1_SI_X11(8*8); \
- VINSERTI128 $1, X11, Y12, Y12; \
- LOAD_MSG_AVX2_Y13(12, 10, 11, 3); \
- LOAD_MSG_AVX2_Y14(4, 7, 15, 1); \
- LOAD_MSG_AVX2_Y15(13, 5, 14, 9)
-
-#define LOAD_MSG_AVX2_12_1_14_4_5_15_13_10_0_6_9_8_7_3_2_11() \
- LOAD_MSG_AVX2_Y12(12, 1, 14, 4); \
- LOAD_MSG_AVX2_Y13(5, 15, 13, 10); \
- VMOVQ_SI_X14_0; \
- VPSHUFD $0x4E, 8*8(SI), X11; \
- VPINSRQ_1_SI_X14(6*8); \
- VINSERTI128 $1, X11, Y14, Y14; \
- LOAD_MSG_AVX2_Y15(7, 3, 2, 11)
-
-#define LOAD_MSG_AVX2_13_7_12_3_11_14_1_9_5_15_8_2_0_4_6_10() \
- LOAD_MSG_AVX2_Y12(13, 7, 12, 3); \
- LOAD_MSG_AVX2_Y13(11, 14, 1, 9); \
- LOAD_MSG_AVX2_Y14(5, 15, 8, 2); \
- VMOVQ_SI_X15_0; \
- VMOVQ_SI_X11(6*8); \
- VPINSRQ_1_SI_X15(4*8); \
- VPINSRQ_1_SI_X11(10*8); \
- VINSERTI128 $1, X11, Y15, Y15
-
-#define LOAD_MSG_AVX2_6_14_11_0_15_9_3_8_12_13_1_10_2_7_4_5() \
- VMOVQ_SI_X12(6*8); \
- VMOVQ_SI_X11(11*8); \
- VPINSRQ_1_SI_X12(14*8); \
- VPINSRQ_1_SI_X11_0; \
- VINSERTI128 $1, X11, Y12, Y12; \
- LOAD_MSG_AVX2_Y13(15, 9, 3, 8); \
- VMOVQ_SI_X11(1*8); \
- VMOVDQU 12*8(SI), X14; \
- VPINSRQ_1_SI_X11(10*8); \
- VINSERTI128 $1, X11, Y14, Y14; \
- VMOVQ_SI_X15(2*8); \
- VMOVDQU 4*8(SI), X11; \
- VPINSRQ_1_SI_X15(7*8); \
- VINSERTI128 $1, X11, Y15, Y15
-
-#define LOAD_MSG_AVX2_10_8_7_1_2_4_6_5_15_9_3_13_11_14_12_0() \
- LOAD_MSG_AVX2_Y12(10, 8, 7, 1); \
- VMOVQ_SI_X13(2*8); \
- VPSHUFD $0x4E, 5*8(SI), X11; \
- VPINSRQ_1_SI_X13(4*8); \
- VINSERTI128 $1, X11, Y13, Y13; \
- LOAD_MSG_AVX2_Y14(15, 9, 3, 13); \
- VMOVQ_SI_X15(11*8); \
- VMOVQ_SI_X11(12*8); \
- VPINSRQ_1_SI_X15(14*8); \
- VPINSRQ_1_SI_X11_0; \
- VINSERTI128 $1, X11, Y15, Y15
-
-// func hashBlocksAVX2(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-TEXT ·hashBlocksAVX2(SB), 4, $320-48 // frame size = 288 + 32 byte alignment
- MOVQ h+0(FP), AX
- MOVQ c+8(FP), BX
- MOVQ flag+16(FP), CX
- MOVQ blocks_base+24(FP), SI
- MOVQ blocks_len+32(FP), DI
-
- MOVQ SP, DX
- ADDQ $31, DX
- ANDQ $~31, DX
-
- MOVQ CX, 16(DX)
- XORQ CX, CX
- MOVQ CX, 24(DX)
-
- VMOVDQU ·AVX2_c40<>(SB), Y4
- VMOVDQU ·AVX2_c48<>(SB), Y5
-
- VMOVDQU 0(AX), Y8
- VMOVDQU 32(AX), Y9
- VMOVDQU ·AVX2_iv0<>(SB), Y6
- VMOVDQU ·AVX2_iv1<>(SB), Y7
-
- MOVQ 0(BX), R8
- MOVQ 8(BX), R9
- MOVQ R9, 8(DX)
-
-loop:
- ADDQ $128, R8
- MOVQ R8, 0(DX)
- CMPQ R8, $128
- JGE noinc
- INCQ R9
- MOVQ R9, 8(DX)
-
-noinc:
- VMOVDQA Y8, Y0
- VMOVDQA Y9, Y1
- VMOVDQA Y6, Y2
- VPXOR 0(DX), Y7, Y3
-
- LOAD_MSG_AVX2_0_2_4_6_1_3_5_7_8_10_12_14_9_11_13_15()
- VMOVDQA Y12, 32(DX)
- VMOVDQA Y13, 64(DX)
- VMOVDQA Y14, 96(DX)
- VMOVDQA Y15, 128(DX)
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_14_4_9_13_10_8_15_6_1_0_11_5_12_2_7_3()
- VMOVDQA Y12, 160(DX)
- VMOVDQA Y13, 192(DX)
- VMOVDQA Y14, 224(DX)
- VMOVDQA Y15, 256(DX)
-
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_11_12_5_15_8_0_2_13_10_3_7_9_14_6_1_4()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_7_3_13_11_9_1_12_14_2_5_4_15_6_10_0_8()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_9_5_2_10_0_7_4_15_14_11_6_3_1_12_8_13()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_2_6_0_8_12_10_11_3_4_7_15_1_13_5_14_9()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_12_1_14_4_5_15_13_10_0_6_9_8_7_3_2_11()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_13_7_12_3_11_14_1_9_5_15_8_2_0_4_6_10()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_6_14_11_0_15_9_3_8_12_13_1_10_2_7_4_5()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
- LOAD_MSG_AVX2_10_8_7_1_2_4_6_5_15_9_3_13_11_14_12_0()
- ROUND_AVX2(Y12, Y13, Y14, Y15, Y10, Y4, Y5)
-
- ROUND_AVX2(32(DX), 64(DX), 96(DX), 128(DX), Y10, Y4, Y5)
- ROUND_AVX2(160(DX), 192(DX), 224(DX), 256(DX), Y10, Y4, Y5)
-
- VPXOR Y0, Y8, Y8
- VPXOR Y1, Y9, Y9
- VPXOR Y2, Y8, Y8
- VPXOR Y3, Y9, Y9
-
- LEAQ 128(SI), SI
- SUBQ $128, DI
- JNE loop
-
- MOVQ R8, 0(BX)
- MOVQ R9, 8(BX)
-
- VMOVDQU Y8, 0(AX)
- VMOVDQU Y9, 32(AX)
- VZEROUPPER
-
- RET
-
-#define VPUNPCKLQDQ_X2_X2_X15 BYTE $0xC5; BYTE $0x69; BYTE $0x6C; BYTE $0xFA
-#define VPUNPCKLQDQ_X3_X3_X15 BYTE $0xC5; BYTE $0x61; BYTE $0x6C; BYTE $0xFB
-#define VPUNPCKLQDQ_X7_X7_X15 BYTE $0xC5; BYTE $0x41; BYTE $0x6C; BYTE $0xFF
-#define VPUNPCKLQDQ_X13_X13_X15 BYTE $0xC4; BYTE $0x41; BYTE $0x11; BYTE $0x6C; BYTE $0xFD
-#define VPUNPCKLQDQ_X14_X14_X15 BYTE $0xC4; BYTE $0x41; BYTE $0x09; BYTE $0x6C; BYTE $0xFE
-
-#define VPUNPCKHQDQ_X15_X2_X2 BYTE $0xC4; BYTE $0xC1; BYTE $0x69; BYTE $0x6D; BYTE $0xD7
-#define VPUNPCKHQDQ_X15_X3_X3 BYTE $0xC4; BYTE $0xC1; BYTE $0x61; BYTE $0x6D; BYTE $0xDF
-#define VPUNPCKHQDQ_X15_X6_X6 BYTE $0xC4; BYTE $0xC1; BYTE $0x49; BYTE $0x6D; BYTE $0xF7
-#define VPUNPCKHQDQ_X15_X7_X7 BYTE $0xC4; BYTE $0xC1; BYTE $0x41; BYTE $0x6D; BYTE $0xFF
-#define VPUNPCKHQDQ_X15_X3_X2 BYTE $0xC4; BYTE $0xC1; BYTE $0x61; BYTE $0x6D; BYTE $0xD7
-#define VPUNPCKHQDQ_X15_X7_X6 BYTE $0xC4; BYTE $0xC1; BYTE $0x41; BYTE $0x6D; BYTE $0xF7
-#define VPUNPCKHQDQ_X15_X13_X3 BYTE $0xC4; BYTE $0xC1; BYTE $0x11; BYTE $0x6D; BYTE $0xDF
-#define VPUNPCKHQDQ_X15_X13_X7 BYTE $0xC4; BYTE $0xC1; BYTE $0x11; BYTE $0x6D; BYTE $0xFF
-
-#define SHUFFLE_AVX() \
- VMOVDQA X6, X13; \
- VMOVDQA X2, X14; \
- VMOVDQA X4, X6; \
- VPUNPCKLQDQ_X13_X13_X15; \
- VMOVDQA X5, X4; \
- VMOVDQA X6, X5; \
- VPUNPCKHQDQ_X15_X7_X6; \
- VPUNPCKLQDQ_X7_X7_X15; \
- VPUNPCKHQDQ_X15_X13_X7; \
- VPUNPCKLQDQ_X3_X3_X15; \
- VPUNPCKHQDQ_X15_X2_X2; \
- VPUNPCKLQDQ_X14_X14_X15; \
- VPUNPCKHQDQ_X15_X3_X3; \
-
-#define SHUFFLE_AVX_INV() \
- VMOVDQA X2, X13; \
- VMOVDQA X4, X14; \
- VPUNPCKLQDQ_X2_X2_X15; \
- VMOVDQA X5, X4; \
- VPUNPCKHQDQ_X15_X3_X2; \
- VMOVDQA X14, X5; \
- VPUNPCKLQDQ_X3_X3_X15; \
- VMOVDQA X6, X14; \
- VPUNPCKHQDQ_X15_X13_X3; \
- VPUNPCKLQDQ_X7_X7_X15; \
- VPUNPCKHQDQ_X15_X6_X6; \
- VPUNPCKLQDQ_X14_X14_X15; \
- VPUNPCKHQDQ_X15_X7_X7; \
-
-#define HALF_ROUND_AVX(v0, v1, v2, v3, v4, v5, v6, v7, m0, m1, m2, m3, t0, c40, c48) \
- VPADDQ m0, v0, v0; \
- VPADDQ v2, v0, v0; \
- VPADDQ m1, v1, v1; \
- VPADDQ v3, v1, v1; \
- VPXOR v0, v6, v6; \
- VPXOR v1, v7, v7; \
- VPSHUFD $-79, v6, v6; \
- VPSHUFD $-79, v7, v7; \
- VPADDQ v6, v4, v4; \
- VPADDQ v7, v5, v5; \
- VPXOR v4, v2, v2; \
- VPXOR v5, v3, v3; \
- VPSHUFB c40, v2, v2; \
- VPSHUFB c40, v3, v3; \
- VPADDQ m2, v0, v0; \
- VPADDQ v2, v0, v0; \
- VPADDQ m3, v1, v1; \
- VPADDQ v3, v1, v1; \
- VPXOR v0, v6, v6; \
- VPXOR v1, v7, v7; \
- VPSHUFB c48, v6, v6; \
- VPSHUFB c48, v7, v7; \
- VPADDQ v6, v4, v4; \
- VPADDQ v7, v5, v5; \
- VPXOR v4, v2, v2; \
- VPXOR v5, v3, v3; \
- VPADDQ v2, v2, t0; \
- VPSRLQ $63, v2, v2; \
- VPXOR t0, v2, v2; \
- VPADDQ v3, v3, t0; \
- VPSRLQ $63, v3, v3; \
- VPXOR t0, v3, v3
-
-// load msg: X12 = (i0, i1), X13 = (i2, i3), X14 = (i4, i5), X15 = (i6, i7)
-// i0, i1, i2, i3, i4, i5, i6, i7 must not be 0
-#define LOAD_MSG_AVX(i0, i1, i2, i3, i4, i5, i6, i7) \
- VMOVQ_SI_X12(i0*8); \
- VMOVQ_SI_X13(i2*8); \
- VMOVQ_SI_X14(i4*8); \
- VMOVQ_SI_X15(i6*8); \
- VPINSRQ_1_SI_X12(i1*8); \
- VPINSRQ_1_SI_X13(i3*8); \
- VPINSRQ_1_SI_X14(i5*8); \
- VPINSRQ_1_SI_X15(i7*8)
-
-// load msg: X12 = (0, 2), X13 = (4, 6), X14 = (1, 3), X15 = (5, 7)
-#define LOAD_MSG_AVX_0_2_4_6_1_3_5_7() \
- VMOVQ_SI_X12_0; \
- VMOVQ_SI_X13(4*8); \
- VMOVQ_SI_X14(1*8); \
- VMOVQ_SI_X15(5*8); \
- VPINSRQ_1_SI_X12(2*8); \
- VPINSRQ_1_SI_X13(6*8); \
- VPINSRQ_1_SI_X14(3*8); \
- VPINSRQ_1_SI_X15(7*8)
-
-// load msg: X12 = (1, 0), X13 = (11, 5), X14 = (12, 2), X15 = (7, 3)
-#define LOAD_MSG_AVX_1_0_11_5_12_2_7_3() \
- VPSHUFD $0x4E, 0*8(SI), X12; \
- VMOVQ_SI_X13(11*8); \
- VMOVQ_SI_X14(12*8); \
- VMOVQ_SI_X15(7*8); \
- VPINSRQ_1_SI_X13(5*8); \
- VPINSRQ_1_SI_X14(2*8); \
- VPINSRQ_1_SI_X15(3*8)
-
-// load msg: X12 = (11, 12), X13 = (5, 15), X14 = (8, 0), X15 = (2, 13)
-#define LOAD_MSG_AVX_11_12_5_15_8_0_2_13() \
- VMOVDQU 11*8(SI), X12; \
- VMOVQ_SI_X13(5*8); \
- VMOVQ_SI_X14(8*8); \
- VMOVQ_SI_X15(2*8); \
- VPINSRQ_1_SI_X13(15*8); \
- VPINSRQ_1_SI_X14_0; \
- VPINSRQ_1_SI_X15(13*8)
-
-// load msg: X12 = (2, 5), X13 = (4, 15), X14 = (6, 10), X15 = (0, 8)
-#define LOAD_MSG_AVX_2_5_4_15_6_10_0_8() \
- VMOVQ_SI_X12(2*8); \
- VMOVQ_SI_X13(4*8); \
- VMOVQ_SI_X14(6*8); \
- VMOVQ_SI_X15_0; \
- VPINSRQ_1_SI_X12(5*8); \
- VPINSRQ_1_SI_X13(15*8); \
- VPINSRQ_1_SI_X14(10*8); \
- VPINSRQ_1_SI_X15(8*8)
-
-// load msg: X12 = (9, 5), X13 = (2, 10), X14 = (0, 7), X15 = (4, 15)
-#define LOAD_MSG_AVX_9_5_2_10_0_7_4_15() \
- VMOVQ_SI_X12(9*8); \
- VMOVQ_SI_X13(2*8); \
- VMOVQ_SI_X14_0; \
- VMOVQ_SI_X15(4*8); \
- VPINSRQ_1_SI_X12(5*8); \
- VPINSRQ_1_SI_X13(10*8); \
- VPINSRQ_1_SI_X14(7*8); \
- VPINSRQ_1_SI_X15(15*8)
-
-// load msg: X12 = (2, 6), X13 = (0, 8), X14 = (12, 10), X15 = (11, 3)
-#define LOAD_MSG_AVX_2_6_0_8_12_10_11_3() \
- VMOVQ_SI_X12(2*8); \
- VMOVQ_SI_X13_0; \
- VMOVQ_SI_X14(12*8); \
- VMOVQ_SI_X15(11*8); \
- VPINSRQ_1_SI_X12(6*8); \
- VPINSRQ_1_SI_X13(8*8); \
- VPINSRQ_1_SI_X14(10*8); \
- VPINSRQ_1_SI_X15(3*8)
-
-// load msg: X12 = (0, 6), X13 = (9, 8), X14 = (7, 3), X15 = (2, 11)
-#define LOAD_MSG_AVX_0_6_9_8_7_3_2_11() \
- MOVQ 0*8(SI), X12; \
- VPSHUFD $0x4E, 8*8(SI), X13; \
- MOVQ 7*8(SI), X14; \
- MOVQ 2*8(SI), X15; \
- VPINSRQ_1_SI_X12(6*8); \
- VPINSRQ_1_SI_X14(3*8); \
- VPINSRQ_1_SI_X15(11*8)
-
-// load msg: X12 = (6, 14), X13 = (11, 0), X14 = (15, 9), X15 = (3, 8)
-#define LOAD_MSG_AVX_6_14_11_0_15_9_3_8() \
- MOVQ 6*8(SI), X12; \
- MOVQ 11*8(SI), X13; \
- MOVQ 15*8(SI), X14; \
- MOVQ 3*8(SI), X15; \
- VPINSRQ_1_SI_X12(14*8); \
- VPINSRQ_1_SI_X13_0; \
- VPINSRQ_1_SI_X14(9*8); \
- VPINSRQ_1_SI_X15(8*8)
-
-// load msg: X12 = (5, 15), X13 = (8, 2), X14 = (0, 4), X15 = (6, 10)
-#define LOAD_MSG_AVX_5_15_8_2_0_4_6_10() \
- MOVQ 5*8(SI), X12; \
- MOVQ 8*8(SI), X13; \
- MOVQ 0*8(SI), X14; \
- MOVQ 6*8(SI), X15; \
- VPINSRQ_1_SI_X12(15*8); \
- VPINSRQ_1_SI_X13(2*8); \
- VPINSRQ_1_SI_X14(4*8); \
- VPINSRQ_1_SI_X15(10*8)
-
-// load msg: X12 = (12, 13), X13 = (1, 10), X14 = (2, 7), X15 = (4, 5)
-#define LOAD_MSG_AVX_12_13_1_10_2_7_4_5() \
- VMOVDQU 12*8(SI), X12; \
- MOVQ 1*8(SI), X13; \
- MOVQ 2*8(SI), X14; \
- VPINSRQ_1_SI_X13(10*8); \
- VPINSRQ_1_SI_X14(7*8); \
- VMOVDQU 4*8(SI), X15
-
-// load msg: X12 = (15, 9), X13 = (3, 13), X14 = (11, 14), X15 = (12, 0)
-#define LOAD_MSG_AVX_15_9_3_13_11_14_12_0() \
- MOVQ 15*8(SI), X12; \
- MOVQ 3*8(SI), X13; \
- MOVQ 11*8(SI), X14; \
- MOVQ 12*8(SI), X15; \
- VPINSRQ_1_SI_X12(9*8); \
- VPINSRQ_1_SI_X13(13*8); \
- VPINSRQ_1_SI_X14(14*8); \
- VPINSRQ_1_SI_X15_0
-
-// func hashBlocksAVX(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-TEXT ·hashBlocksAVX(SB), 4, $288-48 // frame size = 272 + 16 byte alignment
- MOVQ h+0(FP), AX
- MOVQ c+8(FP), BX
- MOVQ flag+16(FP), CX
- MOVQ blocks_base+24(FP), SI
- MOVQ blocks_len+32(FP), DI
-
- MOVQ SP, R10
- ADDQ $15, R10
- ANDQ $~15, R10
-
- VMOVDQU ·AVX_c40<>(SB), X0
- VMOVDQU ·AVX_c48<>(SB), X1
- VMOVDQA X0, X8
- VMOVDQA X1, X9
-
- VMOVDQU ·AVX_iv3<>(SB), X0
- VMOVDQA X0, 0(R10)
- XORQ CX, 0(R10) // 0(R10) = ·AVX_iv3 ^ (CX || 0)
-
- VMOVDQU 0(AX), X10
- VMOVDQU 16(AX), X11
- VMOVDQU 32(AX), X2
- VMOVDQU 48(AX), X3
-
- MOVQ 0(BX), R8
- MOVQ 8(BX), R9
-
-loop:
- ADDQ $128, R8
- CMPQ R8, $128
- JGE noinc
- INCQ R9
-
-noinc:
- VMOVQ_R8_X15
- VPINSRQ_1_R9_X15
-
- VMOVDQA X10, X0
- VMOVDQA X11, X1
- VMOVDQU ·AVX_iv0<>(SB), X4
- VMOVDQU ·AVX_iv1<>(SB), X5
- VMOVDQU ·AVX_iv2<>(SB), X6
-
- VPXOR X15, X6, X6
- VMOVDQA 0(R10), X7
-
- LOAD_MSG_AVX_0_2_4_6_1_3_5_7()
- VMOVDQA X12, 16(R10)
- VMOVDQA X13, 32(R10)
- VMOVDQA X14, 48(R10)
- VMOVDQA X15, 64(R10)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX(8, 10, 12, 14, 9, 11, 13, 15)
- VMOVDQA X12, 80(R10)
- VMOVDQA X13, 96(R10)
- VMOVDQA X14, 112(R10)
- VMOVDQA X15, 128(R10)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX(14, 4, 9, 13, 10, 8, 15, 6)
- VMOVDQA X12, 144(R10)
- VMOVDQA X13, 160(R10)
- VMOVDQA X14, 176(R10)
- VMOVDQA X15, 192(R10)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_1_0_11_5_12_2_7_3()
- VMOVDQA X12, 208(R10)
- VMOVDQA X13, 224(R10)
- VMOVDQA X14, 240(R10)
- VMOVDQA X15, 256(R10)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX_11_12_5_15_8_0_2_13()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX(10, 3, 7, 9, 14, 6, 1, 4)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX(7, 3, 13, 11, 9, 1, 12, 14)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_2_5_4_15_6_10_0_8()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX_9_5_2_10_0_7_4_15()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX(14, 11, 6, 3, 1, 12, 8, 13)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX_2_6_0_8_12_10_11_3()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX(4, 7, 15, 1, 13, 5, 14, 9)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX(12, 1, 14, 4, 5, 15, 13, 10)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_0_6_9_8_7_3_2_11()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX(13, 7, 12, 3, 11, 14, 1, 9)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_5_15_8_2_0_4_6_10()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX_6_14_11_0_15_9_3_8()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_12_13_1_10_2_7_4_5()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- LOAD_MSG_AVX(10, 8, 7, 1, 2, 4, 6, 5)
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX()
- LOAD_MSG_AVX_15_9_3_13_11_14_12_0()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, X12, X13, X14, X15, X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, 16(R10), 32(R10), 48(R10), 64(R10), X15, X8, X9)
- SHUFFLE_AVX()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, 80(R10), 96(R10), 112(R10), 128(R10), X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, 144(R10), 160(R10), 176(R10), 192(R10), X15, X8, X9)
- SHUFFLE_AVX()
- HALF_ROUND_AVX(X0, X1, X2, X3, X4, X5, X6, X7, 208(R10), 224(R10), 240(R10), 256(R10), X15, X8, X9)
- SHUFFLE_AVX_INV()
-
- VMOVDQU 32(AX), X14
- VMOVDQU 48(AX), X15
- VPXOR X0, X10, X10
- VPXOR X1, X11, X11
- VPXOR X2, X14, X14
- VPXOR X3, X15, X15
- VPXOR X4, X10, X10
- VPXOR X5, X11, X11
- VPXOR X6, X14, X2
- VPXOR X7, X15, X3
- VMOVDQU X2, 32(AX)
- VMOVDQU X3, 48(AX)
-
- LEAQ 128(SI), SI
- SUBQ $128, DI
- JNE loop
-
- VMOVDQU X10, 0(AX)
- VMOVDQU X11, 16(AX)
-
- MOVQ R8, 0(BX)
- MOVQ R9, 8(BX)
- VZEROUPPER
-
- RET
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.go b/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.go
deleted file mode 100644
index 5fa1b32..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.go
+++ /dev/null
@@ -1,25 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !go1.7 && amd64 && gc && !purego
-// +build !go1.7,amd64,gc,!purego
-
-package blake2b
-
-import "golang.org/x/sys/cpu"
-
-func init() {
- useSSE4 = cpu.X86.HasSSE41
-}
-
-//go:noescape
-func hashBlocksSSE4(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-
-func hashBlocks(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte) {
- if useSSE4 {
- hashBlocksSSE4(h, c, flag, blocks)
- } else {
- hashBlocksGeneric(h, c, flag, blocks)
- }
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.s b/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.s
deleted file mode 100644
index ae75eb9..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2b_amd64.s
+++ /dev/null
@@ -1,279 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && gc && !purego
-// +build amd64,gc,!purego
-
-#include "textflag.h"
-
-DATA ·iv0<>+0x00(SB)/8, $0x6a09e667f3bcc908
-DATA ·iv0<>+0x08(SB)/8, $0xbb67ae8584caa73b
-GLOBL ·iv0<>(SB), (NOPTR+RODATA), $16
-
-DATA ·iv1<>+0x00(SB)/8, $0x3c6ef372fe94f82b
-DATA ·iv1<>+0x08(SB)/8, $0xa54ff53a5f1d36f1
-GLOBL ·iv1<>(SB), (NOPTR+RODATA), $16
-
-DATA ·iv2<>+0x00(SB)/8, $0x510e527fade682d1
-DATA ·iv2<>+0x08(SB)/8, $0x9b05688c2b3e6c1f
-GLOBL ·iv2<>(SB), (NOPTR+RODATA), $16
-
-DATA ·iv3<>+0x00(SB)/8, $0x1f83d9abfb41bd6b
-DATA ·iv3<>+0x08(SB)/8, $0x5be0cd19137e2179
-GLOBL ·iv3<>(SB), (NOPTR+RODATA), $16
-
-DATA ·c40<>+0x00(SB)/8, $0x0201000706050403
-DATA ·c40<>+0x08(SB)/8, $0x0a09080f0e0d0c0b
-GLOBL ·c40<>(SB), (NOPTR+RODATA), $16
-
-DATA ·c48<>+0x00(SB)/8, $0x0100070605040302
-DATA ·c48<>+0x08(SB)/8, $0x09080f0e0d0c0b0a
-GLOBL ·c48<>(SB), (NOPTR+RODATA), $16
-
-#define SHUFFLE(v2, v3, v4, v5, v6, v7, t1, t2) \
- MOVO v4, t1; \
- MOVO v5, v4; \
- MOVO t1, v5; \
- MOVO v6, t1; \
- PUNPCKLQDQ v6, t2; \
- PUNPCKHQDQ v7, v6; \
- PUNPCKHQDQ t2, v6; \
- PUNPCKLQDQ v7, t2; \
- MOVO t1, v7; \
- MOVO v2, t1; \
- PUNPCKHQDQ t2, v7; \
- PUNPCKLQDQ v3, t2; \
- PUNPCKHQDQ t2, v2; \
- PUNPCKLQDQ t1, t2; \
- PUNPCKHQDQ t2, v3
-
-#define SHUFFLE_INV(v2, v3, v4, v5, v6, v7, t1, t2) \
- MOVO v4, t1; \
- MOVO v5, v4; \
- MOVO t1, v5; \
- MOVO v2, t1; \
- PUNPCKLQDQ v2, t2; \
- PUNPCKHQDQ v3, v2; \
- PUNPCKHQDQ t2, v2; \
- PUNPCKLQDQ v3, t2; \
- MOVO t1, v3; \
- MOVO v6, t1; \
- PUNPCKHQDQ t2, v3; \
- PUNPCKLQDQ v7, t2; \
- PUNPCKHQDQ t2, v6; \
- PUNPCKLQDQ t1, t2; \
- PUNPCKHQDQ t2, v7
-
-#define HALF_ROUND(v0, v1, v2, v3, v4, v5, v6, v7, m0, m1, m2, m3, t0, c40, c48) \
- PADDQ m0, v0; \
- PADDQ m1, v1; \
- PADDQ v2, v0; \
- PADDQ v3, v1; \
- PXOR v0, v6; \
- PXOR v1, v7; \
- PSHUFD $0xB1, v6, v6; \
- PSHUFD $0xB1, v7, v7; \
- PADDQ v6, v4; \
- PADDQ v7, v5; \
- PXOR v4, v2; \
- PXOR v5, v3; \
- PSHUFB c40, v2; \
- PSHUFB c40, v3; \
- PADDQ m2, v0; \
- PADDQ m3, v1; \
- PADDQ v2, v0; \
- PADDQ v3, v1; \
- PXOR v0, v6; \
- PXOR v1, v7; \
- PSHUFB c48, v6; \
- PSHUFB c48, v7; \
- PADDQ v6, v4; \
- PADDQ v7, v5; \
- PXOR v4, v2; \
- PXOR v5, v3; \
- MOVOU v2, t0; \
- PADDQ v2, t0; \
- PSRLQ $63, v2; \
- PXOR t0, v2; \
- MOVOU v3, t0; \
- PADDQ v3, t0; \
- PSRLQ $63, v3; \
- PXOR t0, v3
-
-#define LOAD_MSG(m0, m1, m2, m3, src, i0, i1, i2, i3, i4, i5, i6, i7) \
- MOVQ i0*8(src), m0; \
- PINSRQ $1, i1*8(src), m0; \
- MOVQ i2*8(src), m1; \
- PINSRQ $1, i3*8(src), m1; \
- MOVQ i4*8(src), m2; \
- PINSRQ $1, i5*8(src), m2; \
- MOVQ i6*8(src), m3; \
- PINSRQ $1, i7*8(src), m3
-
-// func hashBlocksSSE4(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte)
-TEXT ·hashBlocksSSE4(SB), 4, $288-48 // frame size = 272 + 16 byte alignment
- MOVQ h+0(FP), AX
- MOVQ c+8(FP), BX
- MOVQ flag+16(FP), CX
- MOVQ blocks_base+24(FP), SI
- MOVQ blocks_len+32(FP), DI
-
- MOVQ SP, R10
- ADDQ $15, R10
- ANDQ $~15, R10
-
- MOVOU ·iv3<>(SB), X0
- MOVO X0, 0(R10)
- XORQ CX, 0(R10) // 0(R10) = ·iv3 ^ (CX || 0)
-
- MOVOU ·c40<>(SB), X13
- MOVOU ·c48<>(SB), X14
-
- MOVOU 0(AX), X12
- MOVOU 16(AX), X15
-
- MOVQ 0(BX), R8
- MOVQ 8(BX), R9
-
-loop:
- ADDQ $128, R8
- CMPQ R8, $128
- JGE noinc
- INCQ R9
-
-noinc:
- MOVQ R8, X8
- PINSRQ $1, R9, X8
-
- MOVO X12, X0
- MOVO X15, X1
- MOVOU 32(AX), X2
- MOVOU 48(AX), X3
- MOVOU ·iv0<>(SB), X4
- MOVOU ·iv1<>(SB), X5
- MOVOU ·iv2<>(SB), X6
-
- PXOR X8, X6
- MOVO 0(R10), X7
-
- LOAD_MSG(X8, X9, X10, X11, SI, 0, 2, 4, 6, 1, 3, 5, 7)
- MOVO X8, 16(R10)
- MOVO X9, 32(R10)
- MOVO X10, 48(R10)
- MOVO X11, 64(R10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 8, 10, 12, 14, 9, 11, 13, 15)
- MOVO X8, 80(R10)
- MOVO X9, 96(R10)
- MOVO X10, 112(R10)
- MOVO X11, 128(R10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 14, 4, 9, 13, 10, 8, 15, 6)
- MOVO X8, 144(R10)
- MOVO X9, 160(R10)
- MOVO X10, 176(R10)
- MOVO X11, 192(R10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 1, 0, 11, 5, 12, 2, 7, 3)
- MOVO X8, 208(R10)
- MOVO X9, 224(R10)
- MOVO X10, 240(R10)
- MOVO X11, 256(R10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 11, 12, 5, 15, 8, 0, 2, 13)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 10, 3, 7, 9, 14, 6, 1, 4)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 7, 3, 13, 11, 9, 1, 12, 14)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 2, 5, 4, 15, 6, 10, 0, 8)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 9, 5, 2, 10, 0, 7, 4, 15)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 14, 11, 6, 3, 1, 12, 8, 13)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 2, 6, 0, 8, 12, 10, 11, 3)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 4, 7, 15, 1, 13, 5, 14, 9)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 12, 1, 14, 4, 5, 15, 13, 10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 0, 6, 9, 8, 7, 3, 2, 11)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 13, 7, 12, 3, 11, 14, 1, 9)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 5, 15, 8, 2, 0, 4, 6, 10)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 6, 14, 11, 0, 15, 9, 3, 8)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 12, 13, 1, 10, 2, 7, 4, 5)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- LOAD_MSG(X8, X9, X10, X11, SI, 10, 8, 7, 1, 2, 4, 6, 5)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- LOAD_MSG(X8, X9, X10, X11, SI, 15, 9, 3, 13, 11, 14, 12, 0)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, 16(R10), 32(R10), 48(R10), 64(R10), X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, 80(R10), 96(R10), 112(R10), 128(R10), X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, 144(R10), 160(R10), 176(R10), 192(R10), X11, X13, X14)
- SHUFFLE(X2, X3, X4, X5, X6, X7, X8, X9)
- HALF_ROUND(X0, X1, X2, X3, X4, X5, X6, X7, 208(R10), 224(R10), 240(R10), 256(R10), X11, X13, X14)
- SHUFFLE_INV(X2, X3, X4, X5, X6, X7, X8, X9)
-
- MOVOU 32(AX), X10
- MOVOU 48(AX), X11
- PXOR X0, X12
- PXOR X1, X15
- PXOR X2, X10
- PXOR X3, X11
- PXOR X4, X12
- PXOR X5, X15
- PXOR X6, X10
- PXOR X7, X11
- MOVOU X10, 32(AX)
- MOVOU X11, 48(AX)
-
- LEAQ 128(SI), SI
- SUBQ $128, DI
- JNE loop
-
- MOVOU X12, 0(AX)
- MOVOU X15, 16(AX)
-
- MOVQ R8, 0(BX)
- MOVQ R9, 8(BX)
-
- RET
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2b_generic.go b/vendor/golang.org/x/crypto/blake2b/blake2b_generic.go
deleted file mode 100644
index 3168a8a..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2b_generic.go
+++ /dev/null
@@ -1,182 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package blake2b
-
-import (
- "encoding/binary"
- "math/bits"
-)
-
-// the precomputed values for BLAKE2b
-// there are 12 16-byte arrays - one for each round
-// the entries are calculated from the sigma constants.
-var precomputed = [12][16]byte{
- {0, 2, 4, 6, 1, 3, 5, 7, 8, 10, 12, 14, 9, 11, 13, 15},
- {14, 4, 9, 13, 10, 8, 15, 6, 1, 0, 11, 5, 12, 2, 7, 3},
- {11, 12, 5, 15, 8, 0, 2, 13, 10, 3, 7, 9, 14, 6, 1, 4},
- {7, 3, 13, 11, 9, 1, 12, 14, 2, 5, 4, 15, 6, 10, 0, 8},
- {9, 5, 2, 10, 0, 7, 4, 15, 14, 11, 6, 3, 1, 12, 8, 13},
- {2, 6, 0, 8, 12, 10, 11, 3, 4, 7, 15, 1, 13, 5, 14, 9},
- {12, 1, 14, 4, 5, 15, 13, 10, 0, 6, 9, 8, 7, 3, 2, 11},
- {13, 7, 12, 3, 11, 14, 1, 9, 5, 15, 8, 2, 0, 4, 6, 10},
- {6, 14, 11, 0, 15, 9, 3, 8, 12, 13, 1, 10, 2, 7, 4, 5},
- {10, 8, 7, 1, 2, 4, 6, 5, 15, 9, 3, 13, 11, 14, 12, 0},
- {0, 2, 4, 6, 1, 3, 5, 7, 8, 10, 12, 14, 9, 11, 13, 15}, // equal to the first
- {14, 4, 9, 13, 10, 8, 15, 6, 1, 0, 11, 5, 12, 2, 7, 3}, // equal to the second
-}
-
-func hashBlocksGeneric(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte) {
- var m [16]uint64
- c0, c1 := c[0], c[1]
-
- for i := 0; i < len(blocks); {
- c0 += BlockSize
- if c0 < BlockSize {
- c1++
- }
-
- v0, v1, v2, v3, v4, v5, v6, v7 := h[0], h[1], h[2], h[3], h[4], h[5], h[6], h[7]
- v8, v9, v10, v11, v12, v13, v14, v15 := iv[0], iv[1], iv[2], iv[3], iv[4], iv[5], iv[6], iv[7]
- v12 ^= c0
- v13 ^= c1
- v14 ^= flag
-
- for j := range m {
- m[j] = binary.LittleEndian.Uint64(blocks[i:])
- i += 8
- }
-
- for j := range precomputed {
- s := &(precomputed[j])
-
- v0 += m[s[0]]
- v0 += v4
- v12 ^= v0
- v12 = bits.RotateLeft64(v12, -32)
- v8 += v12
- v4 ^= v8
- v4 = bits.RotateLeft64(v4, -24)
- v1 += m[s[1]]
- v1 += v5
- v13 ^= v1
- v13 = bits.RotateLeft64(v13, -32)
- v9 += v13
- v5 ^= v9
- v5 = bits.RotateLeft64(v5, -24)
- v2 += m[s[2]]
- v2 += v6
- v14 ^= v2
- v14 = bits.RotateLeft64(v14, -32)
- v10 += v14
- v6 ^= v10
- v6 = bits.RotateLeft64(v6, -24)
- v3 += m[s[3]]
- v3 += v7
- v15 ^= v3
- v15 = bits.RotateLeft64(v15, -32)
- v11 += v15
- v7 ^= v11
- v7 = bits.RotateLeft64(v7, -24)
-
- v0 += m[s[4]]
- v0 += v4
- v12 ^= v0
- v12 = bits.RotateLeft64(v12, -16)
- v8 += v12
- v4 ^= v8
- v4 = bits.RotateLeft64(v4, -63)
- v1 += m[s[5]]
- v1 += v5
- v13 ^= v1
- v13 = bits.RotateLeft64(v13, -16)
- v9 += v13
- v5 ^= v9
- v5 = bits.RotateLeft64(v5, -63)
- v2 += m[s[6]]
- v2 += v6
- v14 ^= v2
- v14 = bits.RotateLeft64(v14, -16)
- v10 += v14
- v6 ^= v10
- v6 = bits.RotateLeft64(v6, -63)
- v3 += m[s[7]]
- v3 += v7
- v15 ^= v3
- v15 = bits.RotateLeft64(v15, -16)
- v11 += v15
- v7 ^= v11
- v7 = bits.RotateLeft64(v7, -63)
-
- v0 += m[s[8]]
- v0 += v5
- v15 ^= v0
- v15 = bits.RotateLeft64(v15, -32)
- v10 += v15
- v5 ^= v10
- v5 = bits.RotateLeft64(v5, -24)
- v1 += m[s[9]]
- v1 += v6
- v12 ^= v1
- v12 = bits.RotateLeft64(v12, -32)
- v11 += v12
- v6 ^= v11
- v6 = bits.RotateLeft64(v6, -24)
- v2 += m[s[10]]
- v2 += v7
- v13 ^= v2
- v13 = bits.RotateLeft64(v13, -32)
- v8 += v13
- v7 ^= v8
- v7 = bits.RotateLeft64(v7, -24)
- v3 += m[s[11]]
- v3 += v4
- v14 ^= v3
- v14 = bits.RotateLeft64(v14, -32)
- v9 += v14
- v4 ^= v9
- v4 = bits.RotateLeft64(v4, -24)
-
- v0 += m[s[12]]
- v0 += v5
- v15 ^= v0
- v15 = bits.RotateLeft64(v15, -16)
- v10 += v15
- v5 ^= v10
- v5 = bits.RotateLeft64(v5, -63)
- v1 += m[s[13]]
- v1 += v6
- v12 ^= v1
- v12 = bits.RotateLeft64(v12, -16)
- v11 += v12
- v6 ^= v11
- v6 = bits.RotateLeft64(v6, -63)
- v2 += m[s[14]]
- v2 += v7
- v13 ^= v2
- v13 = bits.RotateLeft64(v13, -16)
- v8 += v13
- v7 ^= v8
- v7 = bits.RotateLeft64(v7, -63)
- v3 += m[s[15]]
- v3 += v4
- v14 ^= v3
- v14 = bits.RotateLeft64(v14, -16)
- v9 += v14
- v4 ^= v9
- v4 = bits.RotateLeft64(v4, -63)
-
- }
-
- h[0] ^= v0 ^ v8
- h[1] ^= v1 ^ v9
- h[2] ^= v2 ^ v10
- h[3] ^= v3 ^ v11
- h[4] ^= v4 ^ v12
- h[5] ^= v5 ^ v13
- h[6] ^= v6 ^ v14
- h[7] ^= v7 ^ v15
- }
- c[0], c[1] = c0, c1
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2b_ref.go b/vendor/golang.org/x/crypto/blake2b/blake2b_ref.go
deleted file mode 100644
index b0137cd..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2b_ref.go
+++ /dev/null
@@ -1,12 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !amd64 || purego || !gc
-// +build !amd64 purego !gc
-
-package blake2b
-
-func hashBlocks(h *[8]uint64, c *[2]uint64, flag uint64, blocks []byte) {
- hashBlocksGeneric(h, c, flag, blocks)
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/blake2x.go b/vendor/golang.org/x/crypto/blake2b/blake2x.go
deleted file mode 100644
index 52c414d..0000000
--- a/vendor/golang.org/x/crypto/blake2b/blake2x.go
+++ /dev/null
@@ -1,177 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package blake2b
-
-import (
- "encoding/binary"
- "errors"
- "io"
-)
-
-// XOF defines the interface to hash functions that
-// support arbitrary-length output.
-type XOF interface {
- // Write absorbs more data into the hash's state. It panics if called
- // after Read.
- io.Writer
-
- // Read reads more output from the hash. It returns io.EOF if the limit
- // has been reached.
- io.Reader
-
- // Clone returns a copy of the XOF in its current state.
- Clone() XOF
-
- // Reset resets the XOF to its initial state.
- Reset()
-}
-
-// OutputLengthUnknown can be used as the size argument to NewXOF to indicate
-// the length of the output is not known in advance.
-const OutputLengthUnknown = 0
-
-// magicUnknownOutputLength is a magic value for the output size that indicates
-// an unknown number of output bytes.
-const magicUnknownOutputLength = (1 << 32) - 1
-
-// maxOutputLength is the absolute maximum number of bytes to produce when the
-// number of output bytes is unknown.
-const maxOutputLength = (1 << 32) * 64
-
-// NewXOF creates a new variable-output-length hash. The hash either produce a
-// known number of bytes (1 <= size < 2**32-1), or an unknown number of bytes
-// (size == OutputLengthUnknown). In the latter case, an absolute limit of
-// 256GiB applies.
-//
-// A non-nil key turns the hash into a MAC. The key must between
-// zero and 32 bytes long.
-func NewXOF(size uint32, key []byte) (XOF, error) {
- if len(key) > Size {
- return nil, errKeySize
- }
- if size == magicUnknownOutputLength {
- // 2^32-1 indicates an unknown number of bytes and thus isn't a
- // valid length.
- return nil, errors.New("blake2b: XOF length too large")
- }
- if size == OutputLengthUnknown {
- size = magicUnknownOutputLength
- }
- x := &xof{
- d: digest{
- size: Size,
- keyLen: len(key),
- },
- length: size,
- }
- copy(x.d.key[:], key)
- x.Reset()
- return x, nil
-}
-
-type xof struct {
- d digest
- length uint32
- remaining uint64
- cfg, root, block [Size]byte
- offset int
- nodeOffset uint32
- readMode bool
-}
-
-func (x *xof) Write(p []byte) (n int, err error) {
- if x.readMode {
- panic("blake2b: write to XOF after read")
- }
- return x.d.Write(p)
-}
-
-func (x *xof) Clone() XOF {
- clone := *x
- return &clone
-}
-
-func (x *xof) Reset() {
- x.cfg[0] = byte(Size)
- binary.LittleEndian.PutUint32(x.cfg[4:], uint32(Size)) // leaf length
- binary.LittleEndian.PutUint32(x.cfg[12:], x.length) // XOF length
- x.cfg[17] = byte(Size) // inner hash size
-
- x.d.Reset()
- x.d.h[1] ^= uint64(x.length) << 32
-
- x.remaining = uint64(x.length)
- if x.remaining == magicUnknownOutputLength {
- x.remaining = maxOutputLength
- }
- x.offset, x.nodeOffset = 0, 0
- x.readMode = false
-}
-
-func (x *xof) Read(p []byte) (n int, err error) {
- if !x.readMode {
- x.d.finalize(&x.root)
- x.readMode = true
- }
-
- if x.remaining == 0 {
- return 0, io.EOF
- }
-
- n = len(p)
- if uint64(n) > x.remaining {
- n = int(x.remaining)
- p = p[:n]
- }
-
- if x.offset > 0 {
- blockRemaining := Size - x.offset
- if n < blockRemaining {
- x.offset += copy(p, x.block[x.offset:])
- x.remaining -= uint64(n)
- return
- }
- copy(p, x.block[x.offset:])
- p = p[blockRemaining:]
- x.offset = 0
- x.remaining -= uint64(blockRemaining)
- }
-
- for len(p) >= Size {
- binary.LittleEndian.PutUint32(x.cfg[8:], x.nodeOffset)
- x.nodeOffset++
-
- x.d.initConfig(&x.cfg)
- x.d.Write(x.root[:])
- x.d.finalize(&x.block)
-
- copy(p, x.block[:])
- p = p[Size:]
- x.remaining -= uint64(Size)
- }
-
- if todo := len(p); todo > 0 {
- if x.remaining < uint64(Size) {
- x.cfg[0] = byte(x.remaining)
- }
- binary.LittleEndian.PutUint32(x.cfg[8:], x.nodeOffset)
- x.nodeOffset++
-
- x.d.initConfig(&x.cfg)
- x.d.Write(x.root[:])
- x.d.finalize(&x.block)
-
- x.offset = copy(p, x.block[:todo])
- x.remaining -= uint64(todo)
- }
- return
-}
-
-func (d *digest) initConfig(cfg *[Size]byte) {
- d.offset, d.c[0], d.c[1] = 0, 0, 0
- for i := range d.h {
- d.h[i] = iv[i] ^ binary.LittleEndian.Uint64(cfg[i*8:])
- }
-}
diff --git a/vendor/golang.org/x/crypto/blake2b/register.go b/vendor/golang.org/x/crypto/blake2b/register.go
deleted file mode 100644
index 9d86339..0000000
--- a/vendor/golang.org/x/crypto/blake2b/register.go
+++ /dev/null
@@ -1,33 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build go1.9
-// +build go1.9
-
-package blake2b
-
-import (
- "crypto"
- "hash"
-)
-
-func init() {
- newHash256 := func() hash.Hash {
- h, _ := New256(nil)
- return h
- }
- newHash384 := func() hash.Hash {
- h, _ := New384(nil)
- return h
- }
-
- newHash512 := func() hash.Hash {
- h, _ := New512(nil)
- return h
- }
-
- crypto.RegisterHash(crypto.BLAKE2b_256, newHash256)
- crypto.RegisterHash(crypto.BLAKE2b_384, newHash384)
- crypto.RegisterHash(crypto.BLAKE2b_512, newHash512)
-}
diff --git a/vendor/golang.org/x/crypto/blowfish/block.go b/vendor/golang.org/x/crypto/blowfish/block.go
deleted file mode 100644
index 9d80f19..0000000
--- a/vendor/golang.org/x/crypto/blowfish/block.go
+++ /dev/null
@@ -1,159 +0,0 @@
-// Copyright 2010 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package blowfish
-
-// getNextWord returns the next big-endian uint32 value from the byte slice
-// at the given position in a circular manner, updating the position.
-func getNextWord(b []byte, pos *int) uint32 {
- var w uint32
- j := *pos
- for i := 0; i < 4; i++ {
- w = w<<8 | uint32(b[j])
- j++
- if j >= len(b) {
- j = 0
- }
- }
- *pos = j
- return w
-}
-
-// ExpandKey performs a key expansion on the given *Cipher. Specifically, it
-// performs the Blowfish algorithm's key schedule which sets up the *Cipher's
-// pi and substitution tables for calls to Encrypt. This is used, primarily,
-// by the bcrypt package to reuse the Blowfish key schedule during its
-// set up. It's unlikely that you need to use this directly.
-func ExpandKey(key []byte, c *Cipher) {
- j := 0
- for i := 0; i < 18; i++ {
- // Using inlined getNextWord for performance.
- var d uint32
- for k := 0; k < 4; k++ {
- d = d<<8 | uint32(key[j])
- j++
- if j >= len(key) {
- j = 0
- }
- }
- c.p[i] ^= d
- }
-
- var l, r uint32
- for i := 0; i < 18; i += 2 {
- l, r = encryptBlock(l, r, c)
- c.p[i], c.p[i+1] = l, r
- }
-
- for i := 0; i < 256; i += 2 {
- l, r = encryptBlock(l, r, c)
- c.s0[i], c.s0[i+1] = l, r
- }
- for i := 0; i < 256; i += 2 {
- l, r = encryptBlock(l, r, c)
- c.s1[i], c.s1[i+1] = l, r
- }
- for i := 0; i < 256; i += 2 {
- l, r = encryptBlock(l, r, c)
- c.s2[i], c.s2[i+1] = l, r
- }
- for i := 0; i < 256; i += 2 {
- l, r = encryptBlock(l, r, c)
- c.s3[i], c.s3[i+1] = l, r
- }
-}
-
-// This is similar to ExpandKey, but folds the salt during the key
-// schedule. While ExpandKey is essentially expandKeyWithSalt with an all-zero
-// salt passed in, reusing ExpandKey turns out to be a place of inefficiency
-// and specializing it here is useful.
-func expandKeyWithSalt(key []byte, salt []byte, c *Cipher) {
- j := 0
- for i := 0; i < 18; i++ {
- c.p[i] ^= getNextWord(key, &j)
- }
-
- j = 0
- var l, r uint32
- for i := 0; i < 18; i += 2 {
- l ^= getNextWord(salt, &j)
- r ^= getNextWord(salt, &j)
- l, r = encryptBlock(l, r, c)
- c.p[i], c.p[i+1] = l, r
- }
-
- for i := 0; i < 256; i += 2 {
- l ^= getNextWord(salt, &j)
- r ^= getNextWord(salt, &j)
- l, r = encryptBlock(l, r, c)
- c.s0[i], c.s0[i+1] = l, r
- }
-
- for i := 0; i < 256; i += 2 {
- l ^= getNextWord(salt, &j)
- r ^= getNextWord(salt, &j)
- l, r = encryptBlock(l, r, c)
- c.s1[i], c.s1[i+1] = l, r
- }
-
- for i := 0; i < 256; i += 2 {
- l ^= getNextWord(salt, &j)
- r ^= getNextWord(salt, &j)
- l, r = encryptBlock(l, r, c)
- c.s2[i], c.s2[i+1] = l, r
- }
-
- for i := 0; i < 256; i += 2 {
- l ^= getNextWord(salt, &j)
- r ^= getNextWord(salt, &j)
- l, r = encryptBlock(l, r, c)
- c.s3[i], c.s3[i+1] = l, r
- }
-}
-
-func encryptBlock(l, r uint32, c *Cipher) (uint32, uint32) {
- xl, xr := l, r
- xl ^= c.p[0]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[1]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[2]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[3]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[4]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[5]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[6]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[7]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[8]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[9]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[10]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[11]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[12]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[13]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[14]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[15]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[16]
- xr ^= c.p[17]
- return xr, xl
-}
-
-func decryptBlock(l, r uint32, c *Cipher) (uint32, uint32) {
- xl, xr := l, r
- xl ^= c.p[17]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[16]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[15]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[14]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[13]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[12]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[11]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[10]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[9]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[8]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[7]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[6]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[5]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[4]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[3]
- xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[2]
- xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[1]
- xr ^= c.p[0]
- return xr, xl
-}
diff --git a/vendor/golang.org/x/crypto/blowfish/cipher.go b/vendor/golang.org/x/crypto/blowfish/cipher.go
deleted file mode 100644
index 213bf20..0000000
--- a/vendor/golang.org/x/crypto/blowfish/cipher.go
+++ /dev/null
@@ -1,99 +0,0 @@
-// Copyright 2010 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package blowfish implements Bruce Schneier's Blowfish encryption algorithm.
-//
-// Blowfish is a legacy cipher and its short block size makes it vulnerable to
-// birthday bound attacks (see https://sweet32.info). It should only be used
-// where compatibility with legacy systems, not security, is the goal.
-//
-// Deprecated: any new system should use AES (from crypto/aes, if necessary in
-// an AEAD mode like crypto/cipher.NewGCM) or XChaCha20-Poly1305 (from
-// golang.org/x/crypto/chacha20poly1305).
-package blowfish // import "golang.org/x/crypto/blowfish"
-
-// The code is a port of Bruce Schneier's C implementation.
-// See https://www.schneier.com/blowfish.html.
-
-import "strconv"
-
-// The Blowfish block size in bytes.
-const BlockSize = 8
-
-// A Cipher is an instance of Blowfish encryption using a particular key.
-type Cipher struct {
- p [18]uint32
- s0, s1, s2, s3 [256]uint32
-}
-
-type KeySizeError int
-
-func (k KeySizeError) Error() string {
- return "crypto/blowfish: invalid key size " + strconv.Itoa(int(k))
-}
-
-// NewCipher creates and returns a Cipher.
-// The key argument should be the Blowfish key, from 1 to 56 bytes.
-func NewCipher(key []byte) (*Cipher, error) {
- var result Cipher
- if k := len(key); k < 1 || k > 56 {
- return nil, KeySizeError(k)
- }
- initCipher(&result)
- ExpandKey(key, &result)
- return &result, nil
-}
-
-// NewSaltedCipher creates a returns a Cipher that folds a salt into its key
-// schedule. For most purposes, NewCipher, instead of NewSaltedCipher, is
-// sufficient and desirable. For bcrypt compatibility, the key can be over 56
-// bytes.
-func NewSaltedCipher(key, salt []byte) (*Cipher, error) {
- if len(salt) == 0 {
- return NewCipher(key)
- }
- var result Cipher
- if k := len(key); k < 1 {
- return nil, KeySizeError(k)
- }
- initCipher(&result)
- expandKeyWithSalt(key, salt, &result)
- return &result, nil
-}
-
-// BlockSize returns the Blowfish block size, 8 bytes.
-// It is necessary to satisfy the Block interface in the
-// package "crypto/cipher".
-func (c *Cipher) BlockSize() int { return BlockSize }
-
-// Encrypt encrypts the 8-byte buffer src using the key k
-// and stores the result in dst.
-// Note that for amounts of data larger than a block,
-// it is not safe to just call Encrypt on successive blocks;
-// instead, use an encryption mode like CBC (see crypto/cipher/cbc.go).
-func (c *Cipher) Encrypt(dst, src []byte) {
- l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
- r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
- l, r = encryptBlock(l, r, c)
- dst[0], dst[1], dst[2], dst[3] = byte(l>>24), byte(l>>16), byte(l>>8), byte(l)
- dst[4], dst[5], dst[6], dst[7] = byte(r>>24), byte(r>>16), byte(r>>8), byte(r)
-}
-
-// Decrypt decrypts the 8-byte buffer src using the key k
-// and stores the result in dst.
-func (c *Cipher) Decrypt(dst, src []byte) {
- l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
- r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
- l, r = decryptBlock(l, r, c)
- dst[0], dst[1], dst[2], dst[3] = byte(l>>24), byte(l>>16), byte(l>>8), byte(l)
- dst[4], dst[5], dst[6], dst[7] = byte(r>>24), byte(r>>16), byte(r>>8), byte(r)
-}
-
-func initCipher(c *Cipher) {
- copy(c.p[0:], p[0:])
- copy(c.s0[0:], s0[0:])
- copy(c.s1[0:], s1[0:])
- copy(c.s2[0:], s2[0:])
- copy(c.s3[0:], s3[0:])
-}
diff --git a/vendor/golang.org/x/crypto/blowfish/const.go b/vendor/golang.org/x/crypto/blowfish/const.go
deleted file mode 100644
index d040775..0000000
--- a/vendor/golang.org/x/crypto/blowfish/const.go
+++ /dev/null
@@ -1,199 +0,0 @@
-// Copyright 2010 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// The startup permutation array and substitution boxes.
-// They are the hexadecimal digits of PI; see:
-// https://www.schneier.com/code/constants.txt.
-
-package blowfish
-
-var s0 = [256]uint32{
- 0xd1310ba6, 0x98dfb5ac, 0x2ffd72db, 0xd01adfb7, 0xb8e1afed, 0x6a267e96,
- 0xba7c9045, 0xf12c7f99, 0x24a19947, 0xb3916cf7, 0x0801f2e2, 0x858efc16,
- 0x636920d8, 0x71574e69, 0xa458fea3, 0xf4933d7e, 0x0d95748f, 0x728eb658,
- 0x718bcd58, 0x82154aee, 0x7b54a41d, 0xc25a59b5, 0x9c30d539, 0x2af26013,
- 0xc5d1b023, 0x286085f0, 0xca417918, 0xb8db38ef, 0x8e79dcb0, 0x603a180e,
- 0x6c9e0e8b, 0xb01e8a3e, 0xd71577c1, 0xbd314b27, 0x78af2fda, 0x55605c60,
- 0xe65525f3, 0xaa55ab94, 0x57489862, 0x63e81440, 0x55ca396a, 0x2aab10b6,
- 0xb4cc5c34, 0x1141e8ce, 0xa15486af, 0x7c72e993, 0xb3ee1411, 0x636fbc2a,
- 0x2ba9c55d, 0x741831f6, 0xce5c3e16, 0x9b87931e, 0xafd6ba33, 0x6c24cf5c,
- 0x7a325381, 0x28958677, 0x3b8f4898, 0x6b4bb9af, 0xc4bfe81b, 0x66282193,
- 0x61d809cc, 0xfb21a991, 0x487cac60, 0x5dec8032, 0xef845d5d, 0xe98575b1,
- 0xdc262302, 0xeb651b88, 0x23893e81, 0xd396acc5, 0x0f6d6ff3, 0x83f44239,
- 0x2e0b4482, 0xa4842004, 0x69c8f04a, 0x9e1f9b5e, 0x21c66842, 0xf6e96c9a,
- 0x670c9c61, 0xabd388f0, 0x6a51a0d2, 0xd8542f68, 0x960fa728, 0xab5133a3,
- 0x6eef0b6c, 0x137a3be4, 0xba3bf050, 0x7efb2a98, 0xa1f1651d, 0x39af0176,
- 0x66ca593e, 0x82430e88, 0x8cee8619, 0x456f9fb4, 0x7d84a5c3, 0x3b8b5ebe,
- 0xe06f75d8, 0x85c12073, 0x401a449f, 0x56c16aa6, 0x4ed3aa62, 0x363f7706,
- 0x1bfedf72, 0x429b023d, 0x37d0d724, 0xd00a1248, 0xdb0fead3, 0x49f1c09b,
- 0x075372c9, 0x80991b7b, 0x25d479d8, 0xf6e8def7, 0xe3fe501a, 0xb6794c3b,
- 0x976ce0bd, 0x04c006ba, 0xc1a94fb6, 0x409f60c4, 0x5e5c9ec2, 0x196a2463,
- 0x68fb6faf, 0x3e6c53b5, 0x1339b2eb, 0x3b52ec6f, 0x6dfc511f, 0x9b30952c,
- 0xcc814544, 0xaf5ebd09, 0xbee3d004, 0xde334afd, 0x660f2807, 0x192e4bb3,
- 0xc0cba857, 0x45c8740f, 0xd20b5f39, 0xb9d3fbdb, 0x5579c0bd, 0x1a60320a,
- 0xd6a100c6, 0x402c7279, 0x679f25fe, 0xfb1fa3cc, 0x8ea5e9f8, 0xdb3222f8,
- 0x3c7516df, 0xfd616b15, 0x2f501ec8, 0xad0552ab, 0x323db5fa, 0xfd238760,
- 0x53317b48, 0x3e00df82, 0x9e5c57bb, 0xca6f8ca0, 0x1a87562e, 0xdf1769db,
- 0xd542a8f6, 0x287effc3, 0xac6732c6, 0x8c4f5573, 0x695b27b0, 0xbbca58c8,
- 0xe1ffa35d, 0xb8f011a0, 0x10fa3d98, 0xfd2183b8, 0x4afcb56c, 0x2dd1d35b,
- 0x9a53e479, 0xb6f84565, 0xd28e49bc, 0x4bfb9790, 0xe1ddf2da, 0xa4cb7e33,
- 0x62fb1341, 0xcee4c6e8, 0xef20cada, 0x36774c01, 0xd07e9efe, 0x2bf11fb4,
- 0x95dbda4d, 0xae909198, 0xeaad8e71, 0x6b93d5a0, 0xd08ed1d0, 0xafc725e0,
- 0x8e3c5b2f, 0x8e7594b7, 0x8ff6e2fb, 0xf2122b64, 0x8888b812, 0x900df01c,
- 0x4fad5ea0, 0x688fc31c, 0xd1cff191, 0xb3a8c1ad, 0x2f2f2218, 0xbe0e1777,
- 0xea752dfe, 0x8b021fa1, 0xe5a0cc0f, 0xb56f74e8, 0x18acf3d6, 0xce89e299,
- 0xb4a84fe0, 0xfd13e0b7, 0x7cc43b81, 0xd2ada8d9, 0x165fa266, 0x80957705,
- 0x93cc7314, 0x211a1477, 0xe6ad2065, 0x77b5fa86, 0xc75442f5, 0xfb9d35cf,
- 0xebcdaf0c, 0x7b3e89a0, 0xd6411bd3, 0xae1e7e49, 0x00250e2d, 0x2071b35e,
- 0x226800bb, 0x57b8e0af, 0x2464369b, 0xf009b91e, 0x5563911d, 0x59dfa6aa,
- 0x78c14389, 0xd95a537f, 0x207d5ba2, 0x02e5b9c5, 0x83260376, 0x6295cfa9,
- 0x11c81968, 0x4e734a41, 0xb3472dca, 0x7b14a94a, 0x1b510052, 0x9a532915,
- 0xd60f573f, 0xbc9bc6e4, 0x2b60a476, 0x81e67400, 0x08ba6fb5, 0x571be91f,
- 0xf296ec6b, 0x2a0dd915, 0xb6636521, 0xe7b9f9b6, 0xff34052e, 0xc5855664,
- 0x53b02d5d, 0xa99f8fa1, 0x08ba4799, 0x6e85076a,
-}
-
-var s1 = [256]uint32{
- 0x4b7a70e9, 0xb5b32944, 0xdb75092e, 0xc4192623, 0xad6ea6b0, 0x49a7df7d,
- 0x9cee60b8, 0x8fedb266, 0xecaa8c71, 0x699a17ff, 0x5664526c, 0xc2b19ee1,
- 0x193602a5, 0x75094c29, 0xa0591340, 0xe4183a3e, 0x3f54989a, 0x5b429d65,
- 0x6b8fe4d6, 0x99f73fd6, 0xa1d29c07, 0xefe830f5, 0x4d2d38e6, 0xf0255dc1,
- 0x4cdd2086, 0x8470eb26, 0x6382e9c6, 0x021ecc5e, 0x09686b3f, 0x3ebaefc9,
- 0x3c971814, 0x6b6a70a1, 0x687f3584, 0x52a0e286, 0xb79c5305, 0xaa500737,
- 0x3e07841c, 0x7fdeae5c, 0x8e7d44ec, 0x5716f2b8, 0xb03ada37, 0xf0500c0d,
- 0xf01c1f04, 0x0200b3ff, 0xae0cf51a, 0x3cb574b2, 0x25837a58, 0xdc0921bd,
- 0xd19113f9, 0x7ca92ff6, 0x94324773, 0x22f54701, 0x3ae5e581, 0x37c2dadc,
- 0xc8b57634, 0x9af3dda7, 0xa9446146, 0x0fd0030e, 0xecc8c73e, 0xa4751e41,
- 0xe238cd99, 0x3bea0e2f, 0x3280bba1, 0x183eb331, 0x4e548b38, 0x4f6db908,
- 0x6f420d03, 0xf60a04bf, 0x2cb81290, 0x24977c79, 0x5679b072, 0xbcaf89af,
- 0xde9a771f, 0xd9930810, 0xb38bae12, 0xdccf3f2e, 0x5512721f, 0x2e6b7124,
- 0x501adde6, 0x9f84cd87, 0x7a584718, 0x7408da17, 0xbc9f9abc, 0xe94b7d8c,
- 0xec7aec3a, 0xdb851dfa, 0x63094366, 0xc464c3d2, 0xef1c1847, 0x3215d908,
- 0xdd433b37, 0x24c2ba16, 0x12a14d43, 0x2a65c451, 0x50940002, 0x133ae4dd,
- 0x71dff89e, 0x10314e55, 0x81ac77d6, 0x5f11199b, 0x043556f1, 0xd7a3c76b,
- 0x3c11183b, 0x5924a509, 0xf28fe6ed, 0x97f1fbfa, 0x9ebabf2c, 0x1e153c6e,
- 0x86e34570, 0xeae96fb1, 0x860e5e0a, 0x5a3e2ab3, 0x771fe71c, 0x4e3d06fa,
- 0x2965dcb9, 0x99e71d0f, 0x803e89d6, 0x5266c825, 0x2e4cc978, 0x9c10b36a,
- 0xc6150eba, 0x94e2ea78, 0xa5fc3c53, 0x1e0a2df4, 0xf2f74ea7, 0x361d2b3d,
- 0x1939260f, 0x19c27960, 0x5223a708, 0xf71312b6, 0xebadfe6e, 0xeac31f66,
- 0xe3bc4595, 0xa67bc883, 0xb17f37d1, 0x018cff28, 0xc332ddef, 0xbe6c5aa5,
- 0x65582185, 0x68ab9802, 0xeecea50f, 0xdb2f953b, 0x2aef7dad, 0x5b6e2f84,
- 0x1521b628, 0x29076170, 0xecdd4775, 0x619f1510, 0x13cca830, 0xeb61bd96,
- 0x0334fe1e, 0xaa0363cf, 0xb5735c90, 0x4c70a239, 0xd59e9e0b, 0xcbaade14,
- 0xeecc86bc, 0x60622ca7, 0x9cab5cab, 0xb2f3846e, 0x648b1eaf, 0x19bdf0ca,
- 0xa02369b9, 0x655abb50, 0x40685a32, 0x3c2ab4b3, 0x319ee9d5, 0xc021b8f7,
- 0x9b540b19, 0x875fa099, 0x95f7997e, 0x623d7da8, 0xf837889a, 0x97e32d77,
- 0x11ed935f, 0x16681281, 0x0e358829, 0xc7e61fd6, 0x96dedfa1, 0x7858ba99,
- 0x57f584a5, 0x1b227263, 0x9b83c3ff, 0x1ac24696, 0xcdb30aeb, 0x532e3054,
- 0x8fd948e4, 0x6dbc3128, 0x58ebf2ef, 0x34c6ffea, 0xfe28ed61, 0xee7c3c73,
- 0x5d4a14d9, 0xe864b7e3, 0x42105d14, 0x203e13e0, 0x45eee2b6, 0xa3aaabea,
- 0xdb6c4f15, 0xfacb4fd0, 0xc742f442, 0xef6abbb5, 0x654f3b1d, 0x41cd2105,
- 0xd81e799e, 0x86854dc7, 0xe44b476a, 0x3d816250, 0xcf62a1f2, 0x5b8d2646,
- 0xfc8883a0, 0xc1c7b6a3, 0x7f1524c3, 0x69cb7492, 0x47848a0b, 0x5692b285,
- 0x095bbf00, 0xad19489d, 0x1462b174, 0x23820e00, 0x58428d2a, 0x0c55f5ea,
- 0x1dadf43e, 0x233f7061, 0x3372f092, 0x8d937e41, 0xd65fecf1, 0x6c223bdb,
- 0x7cde3759, 0xcbee7460, 0x4085f2a7, 0xce77326e, 0xa6078084, 0x19f8509e,
- 0xe8efd855, 0x61d99735, 0xa969a7aa, 0xc50c06c2, 0x5a04abfc, 0x800bcadc,
- 0x9e447a2e, 0xc3453484, 0xfdd56705, 0x0e1e9ec9, 0xdb73dbd3, 0x105588cd,
- 0x675fda79, 0xe3674340, 0xc5c43465, 0x713e38d8, 0x3d28f89e, 0xf16dff20,
- 0x153e21e7, 0x8fb03d4a, 0xe6e39f2b, 0xdb83adf7,
-}
-
-var s2 = [256]uint32{
- 0xe93d5a68, 0x948140f7, 0xf64c261c, 0x94692934, 0x411520f7, 0x7602d4f7,
- 0xbcf46b2e, 0xd4a20068, 0xd4082471, 0x3320f46a, 0x43b7d4b7, 0x500061af,
- 0x1e39f62e, 0x97244546, 0x14214f74, 0xbf8b8840, 0x4d95fc1d, 0x96b591af,
- 0x70f4ddd3, 0x66a02f45, 0xbfbc09ec, 0x03bd9785, 0x7fac6dd0, 0x31cb8504,
- 0x96eb27b3, 0x55fd3941, 0xda2547e6, 0xabca0a9a, 0x28507825, 0x530429f4,
- 0x0a2c86da, 0xe9b66dfb, 0x68dc1462, 0xd7486900, 0x680ec0a4, 0x27a18dee,
- 0x4f3ffea2, 0xe887ad8c, 0xb58ce006, 0x7af4d6b6, 0xaace1e7c, 0xd3375fec,
- 0xce78a399, 0x406b2a42, 0x20fe9e35, 0xd9f385b9, 0xee39d7ab, 0x3b124e8b,
- 0x1dc9faf7, 0x4b6d1856, 0x26a36631, 0xeae397b2, 0x3a6efa74, 0xdd5b4332,
- 0x6841e7f7, 0xca7820fb, 0xfb0af54e, 0xd8feb397, 0x454056ac, 0xba489527,
- 0x55533a3a, 0x20838d87, 0xfe6ba9b7, 0xd096954b, 0x55a867bc, 0xa1159a58,
- 0xcca92963, 0x99e1db33, 0xa62a4a56, 0x3f3125f9, 0x5ef47e1c, 0x9029317c,
- 0xfdf8e802, 0x04272f70, 0x80bb155c, 0x05282ce3, 0x95c11548, 0xe4c66d22,
- 0x48c1133f, 0xc70f86dc, 0x07f9c9ee, 0x41041f0f, 0x404779a4, 0x5d886e17,
- 0x325f51eb, 0xd59bc0d1, 0xf2bcc18f, 0x41113564, 0x257b7834, 0x602a9c60,
- 0xdff8e8a3, 0x1f636c1b, 0x0e12b4c2, 0x02e1329e, 0xaf664fd1, 0xcad18115,
- 0x6b2395e0, 0x333e92e1, 0x3b240b62, 0xeebeb922, 0x85b2a20e, 0xe6ba0d99,
- 0xde720c8c, 0x2da2f728, 0xd0127845, 0x95b794fd, 0x647d0862, 0xe7ccf5f0,
- 0x5449a36f, 0x877d48fa, 0xc39dfd27, 0xf33e8d1e, 0x0a476341, 0x992eff74,
- 0x3a6f6eab, 0xf4f8fd37, 0xa812dc60, 0xa1ebddf8, 0x991be14c, 0xdb6e6b0d,
- 0xc67b5510, 0x6d672c37, 0x2765d43b, 0xdcd0e804, 0xf1290dc7, 0xcc00ffa3,
- 0xb5390f92, 0x690fed0b, 0x667b9ffb, 0xcedb7d9c, 0xa091cf0b, 0xd9155ea3,
- 0xbb132f88, 0x515bad24, 0x7b9479bf, 0x763bd6eb, 0x37392eb3, 0xcc115979,
- 0x8026e297, 0xf42e312d, 0x6842ada7, 0xc66a2b3b, 0x12754ccc, 0x782ef11c,
- 0x6a124237, 0xb79251e7, 0x06a1bbe6, 0x4bfb6350, 0x1a6b1018, 0x11caedfa,
- 0x3d25bdd8, 0xe2e1c3c9, 0x44421659, 0x0a121386, 0xd90cec6e, 0xd5abea2a,
- 0x64af674e, 0xda86a85f, 0xbebfe988, 0x64e4c3fe, 0x9dbc8057, 0xf0f7c086,
- 0x60787bf8, 0x6003604d, 0xd1fd8346, 0xf6381fb0, 0x7745ae04, 0xd736fccc,
- 0x83426b33, 0xf01eab71, 0xb0804187, 0x3c005e5f, 0x77a057be, 0xbde8ae24,
- 0x55464299, 0xbf582e61, 0x4e58f48f, 0xf2ddfda2, 0xf474ef38, 0x8789bdc2,
- 0x5366f9c3, 0xc8b38e74, 0xb475f255, 0x46fcd9b9, 0x7aeb2661, 0x8b1ddf84,
- 0x846a0e79, 0x915f95e2, 0x466e598e, 0x20b45770, 0x8cd55591, 0xc902de4c,
- 0xb90bace1, 0xbb8205d0, 0x11a86248, 0x7574a99e, 0xb77f19b6, 0xe0a9dc09,
- 0x662d09a1, 0xc4324633, 0xe85a1f02, 0x09f0be8c, 0x4a99a025, 0x1d6efe10,
- 0x1ab93d1d, 0x0ba5a4df, 0xa186f20f, 0x2868f169, 0xdcb7da83, 0x573906fe,
- 0xa1e2ce9b, 0x4fcd7f52, 0x50115e01, 0xa70683fa, 0xa002b5c4, 0x0de6d027,
- 0x9af88c27, 0x773f8641, 0xc3604c06, 0x61a806b5, 0xf0177a28, 0xc0f586e0,
- 0x006058aa, 0x30dc7d62, 0x11e69ed7, 0x2338ea63, 0x53c2dd94, 0xc2c21634,
- 0xbbcbee56, 0x90bcb6de, 0xebfc7da1, 0xce591d76, 0x6f05e409, 0x4b7c0188,
- 0x39720a3d, 0x7c927c24, 0x86e3725f, 0x724d9db9, 0x1ac15bb4, 0xd39eb8fc,
- 0xed545578, 0x08fca5b5, 0xd83d7cd3, 0x4dad0fc4, 0x1e50ef5e, 0xb161e6f8,
- 0xa28514d9, 0x6c51133c, 0x6fd5c7e7, 0x56e14ec4, 0x362abfce, 0xddc6c837,
- 0xd79a3234, 0x92638212, 0x670efa8e, 0x406000e0,
-}
-
-var s3 = [256]uint32{
- 0x3a39ce37, 0xd3faf5cf, 0xabc27737, 0x5ac52d1b, 0x5cb0679e, 0x4fa33742,
- 0xd3822740, 0x99bc9bbe, 0xd5118e9d, 0xbf0f7315, 0xd62d1c7e, 0xc700c47b,
- 0xb78c1b6b, 0x21a19045, 0xb26eb1be, 0x6a366eb4, 0x5748ab2f, 0xbc946e79,
- 0xc6a376d2, 0x6549c2c8, 0x530ff8ee, 0x468dde7d, 0xd5730a1d, 0x4cd04dc6,
- 0x2939bbdb, 0xa9ba4650, 0xac9526e8, 0xbe5ee304, 0xa1fad5f0, 0x6a2d519a,
- 0x63ef8ce2, 0x9a86ee22, 0xc089c2b8, 0x43242ef6, 0xa51e03aa, 0x9cf2d0a4,
- 0x83c061ba, 0x9be96a4d, 0x8fe51550, 0xba645bd6, 0x2826a2f9, 0xa73a3ae1,
- 0x4ba99586, 0xef5562e9, 0xc72fefd3, 0xf752f7da, 0x3f046f69, 0x77fa0a59,
- 0x80e4a915, 0x87b08601, 0x9b09e6ad, 0x3b3ee593, 0xe990fd5a, 0x9e34d797,
- 0x2cf0b7d9, 0x022b8b51, 0x96d5ac3a, 0x017da67d, 0xd1cf3ed6, 0x7c7d2d28,
- 0x1f9f25cf, 0xadf2b89b, 0x5ad6b472, 0x5a88f54c, 0xe029ac71, 0xe019a5e6,
- 0x47b0acfd, 0xed93fa9b, 0xe8d3c48d, 0x283b57cc, 0xf8d56629, 0x79132e28,
- 0x785f0191, 0xed756055, 0xf7960e44, 0xe3d35e8c, 0x15056dd4, 0x88f46dba,
- 0x03a16125, 0x0564f0bd, 0xc3eb9e15, 0x3c9057a2, 0x97271aec, 0xa93a072a,
- 0x1b3f6d9b, 0x1e6321f5, 0xf59c66fb, 0x26dcf319, 0x7533d928, 0xb155fdf5,
- 0x03563482, 0x8aba3cbb, 0x28517711, 0xc20ad9f8, 0xabcc5167, 0xccad925f,
- 0x4de81751, 0x3830dc8e, 0x379d5862, 0x9320f991, 0xea7a90c2, 0xfb3e7bce,
- 0x5121ce64, 0x774fbe32, 0xa8b6e37e, 0xc3293d46, 0x48de5369, 0x6413e680,
- 0xa2ae0810, 0xdd6db224, 0x69852dfd, 0x09072166, 0xb39a460a, 0x6445c0dd,
- 0x586cdecf, 0x1c20c8ae, 0x5bbef7dd, 0x1b588d40, 0xccd2017f, 0x6bb4e3bb,
- 0xdda26a7e, 0x3a59ff45, 0x3e350a44, 0xbcb4cdd5, 0x72eacea8, 0xfa6484bb,
- 0x8d6612ae, 0xbf3c6f47, 0xd29be463, 0x542f5d9e, 0xaec2771b, 0xf64e6370,
- 0x740e0d8d, 0xe75b1357, 0xf8721671, 0xaf537d5d, 0x4040cb08, 0x4eb4e2cc,
- 0x34d2466a, 0x0115af84, 0xe1b00428, 0x95983a1d, 0x06b89fb4, 0xce6ea048,
- 0x6f3f3b82, 0x3520ab82, 0x011a1d4b, 0x277227f8, 0x611560b1, 0xe7933fdc,
- 0xbb3a792b, 0x344525bd, 0xa08839e1, 0x51ce794b, 0x2f32c9b7, 0xa01fbac9,
- 0xe01cc87e, 0xbcc7d1f6, 0xcf0111c3, 0xa1e8aac7, 0x1a908749, 0xd44fbd9a,
- 0xd0dadecb, 0xd50ada38, 0x0339c32a, 0xc6913667, 0x8df9317c, 0xe0b12b4f,
- 0xf79e59b7, 0x43f5bb3a, 0xf2d519ff, 0x27d9459c, 0xbf97222c, 0x15e6fc2a,
- 0x0f91fc71, 0x9b941525, 0xfae59361, 0xceb69ceb, 0xc2a86459, 0x12baa8d1,
- 0xb6c1075e, 0xe3056a0c, 0x10d25065, 0xcb03a442, 0xe0ec6e0e, 0x1698db3b,
- 0x4c98a0be, 0x3278e964, 0x9f1f9532, 0xe0d392df, 0xd3a0342b, 0x8971f21e,
- 0x1b0a7441, 0x4ba3348c, 0xc5be7120, 0xc37632d8, 0xdf359f8d, 0x9b992f2e,
- 0xe60b6f47, 0x0fe3f11d, 0xe54cda54, 0x1edad891, 0xce6279cf, 0xcd3e7e6f,
- 0x1618b166, 0xfd2c1d05, 0x848fd2c5, 0xf6fb2299, 0xf523f357, 0xa6327623,
- 0x93a83531, 0x56cccd02, 0xacf08162, 0x5a75ebb5, 0x6e163697, 0x88d273cc,
- 0xde966292, 0x81b949d0, 0x4c50901b, 0x71c65614, 0xe6c6c7bd, 0x327a140a,
- 0x45e1d006, 0xc3f27b9a, 0xc9aa53fd, 0x62a80f00, 0xbb25bfe2, 0x35bdd2f6,
- 0x71126905, 0xb2040222, 0xb6cbcf7c, 0xcd769c2b, 0x53113ec0, 0x1640e3d3,
- 0x38abbd60, 0x2547adf0, 0xba38209c, 0xf746ce76, 0x77afa1c5, 0x20756060,
- 0x85cbfe4e, 0x8ae88dd8, 0x7aaaf9b0, 0x4cf9aa7e, 0x1948c25c, 0x02fb8a8c,
- 0x01c36ae4, 0xd6ebe1f9, 0x90d4f869, 0xa65cdea0, 0x3f09252d, 0xc208e69f,
- 0xb74e6132, 0xce77e25b, 0x578fdfe3, 0x3ac372e6,
-}
-
-var p = [18]uint32{
- 0x243f6a88, 0x85a308d3, 0x13198a2e, 0x03707344, 0xa4093822, 0x299f31d0,
- 0x082efa98, 0xec4e6c89, 0x452821e6, 0x38d01377, 0xbe5466cf, 0x34e90c6c,
- 0xc0ac29b7, 0xc97c50dd, 0x3f84d5b5, 0xb5470917, 0x9216d5d9, 0x8979fb1b,
-}
diff --git a/vendor/golang.org/x/crypto/cast5/cast5.go b/vendor/golang.org/x/crypto/cast5/cast5.go
deleted file mode 100644
index ddcbeb6..0000000
--- a/vendor/golang.org/x/crypto/cast5/cast5.go
+++ /dev/null
@@ -1,533 +0,0 @@
-// Copyright 2010 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package cast5 implements CAST5, as defined in RFC 2144.
-//
-// CAST5 is a legacy cipher and its short block size makes it vulnerable to
-// birthday bound attacks (see https://sweet32.info). It should only be used
-// where compatibility with legacy systems, not security, is the goal.
-//
-// Deprecated: any new system should use AES (from crypto/aes, if necessary in
-// an AEAD mode like crypto/cipher.NewGCM) or XChaCha20-Poly1305 (from
-// golang.org/x/crypto/chacha20poly1305).
-package cast5 // import "golang.org/x/crypto/cast5"
-
-import "errors"
-
-const BlockSize = 8
-const KeySize = 16
-
-type Cipher struct {
- masking [16]uint32
- rotate [16]uint8
-}
-
-func NewCipher(key []byte) (c *Cipher, err error) {
- if len(key) != KeySize {
- return nil, errors.New("CAST5: keys must be 16 bytes")
- }
-
- c = new(Cipher)
- c.keySchedule(key)
- return
-}
-
-func (c *Cipher) BlockSize() int {
- return BlockSize
-}
-
-func (c *Cipher) Encrypt(dst, src []byte) {
- l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
- r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
-
- l, r = r, l^f1(r, c.masking[0], c.rotate[0])
- l, r = r, l^f2(r, c.masking[1], c.rotate[1])
- l, r = r, l^f3(r, c.masking[2], c.rotate[2])
- l, r = r, l^f1(r, c.masking[3], c.rotate[3])
-
- l, r = r, l^f2(r, c.masking[4], c.rotate[4])
- l, r = r, l^f3(r, c.masking[5], c.rotate[5])
- l, r = r, l^f1(r, c.masking[6], c.rotate[6])
- l, r = r, l^f2(r, c.masking[7], c.rotate[7])
-
- l, r = r, l^f3(r, c.masking[8], c.rotate[8])
- l, r = r, l^f1(r, c.masking[9], c.rotate[9])
- l, r = r, l^f2(r, c.masking[10], c.rotate[10])
- l, r = r, l^f3(r, c.masking[11], c.rotate[11])
-
- l, r = r, l^f1(r, c.masking[12], c.rotate[12])
- l, r = r, l^f2(r, c.masking[13], c.rotate[13])
- l, r = r, l^f3(r, c.masking[14], c.rotate[14])
- l, r = r, l^f1(r, c.masking[15], c.rotate[15])
-
- dst[0] = uint8(r >> 24)
- dst[1] = uint8(r >> 16)
- dst[2] = uint8(r >> 8)
- dst[3] = uint8(r)
- dst[4] = uint8(l >> 24)
- dst[5] = uint8(l >> 16)
- dst[6] = uint8(l >> 8)
- dst[7] = uint8(l)
-}
-
-func (c *Cipher) Decrypt(dst, src []byte) {
- l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
- r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
-
- l, r = r, l^f1(r, c.masking[15], c.rotate[15])
- l, r = r, l^f3(r, c.masking[14], c.rotate[14])
- l, r = r, l^f2(r, c.masking[13], c.rotate[13])
- l, r = r, l^f1(r, c.masking[12], c.rotate[12])
-
- l, r = r, l^f3(r, c.masking[11], c.rotate[11])
- l, r = r, l^f2(r, c.masking[10], c.rotate[10])
- l, r = r, l^f1(r, c.masking[9], c.rotate[9])
- l, r = r, l^f3(r, c.masking[8], c.rotate[8])
-
- l, r = r, l^f2(r, c.masking[7], c.rotate[7])
- l, r = r, l^f1(r, c.masking[6], c.rotate[6])
- l, r = r, l^f3(r, c.masking[5], c.rotate[5])
- l, r = r, l^f2(r, c.masking[4], c.rotate[4])
-
- l, r = r, l^f1(r, c.masking[3], c.rotate[3])
- l, r = r, l^f3(r, c.masking[2], c.rotate[2])
- l, r = r, l^f2(r, c.masking[1], c.rotate[1])
- l, r = r, l^f1(r, c.masking[0], c.rotate[0])
-
- dst[0] = uint8(r >> 24)
- dst[1] = uint8(r >> 16)
- dst[2] = uint8(r >> 8)
- dst[3] = uint8(r)
- dst[4] = uint8(l >> 24)
- dst[5] = uint8(l >> 16)
- dst[6] = uint8(l >> 8)
- dst[7] = uint8(l)
-}
-
-type keyScheduleA [4][7]uint8
-type keyScheduleB [4][5]uint8
-
-// keyScheduleRound contains the magic values for a round of the key schedule.
-// The keyScheduleA deals with the lines like:
-// z0z1z2z3 = x0x1x2x3 ^ S5[xD] ^ S6[xF] ^ S7[xC] ^ S8[xE] ^ S7[x8]
-// Conceptually, both x and z are in the same array, x first. The first
-// element describes which word of this array gets written to and the
-// second, which word gets read. So, for the line above, it's "4, 0", because
-// it's writing to the first word of z, which, being after x, is word 4, and
-// reading from the first word of x: word 0.
-//
-// Next are the indexes into the S-boxes. Now the array is treated as bytes. So
-// "xD" is 0xd. The first byte of z is written as "16 + 0", just to be clear
-// that it's z that we're indexing.
-//
-// keyScheduleB deals with lines like:
-// K1 = S5[z8] ^ S6[z9] ^ S7[z7] ^ S8[z6] ^ S5[z2]
-// "K1" is ignored because key words are always written in order. So the five
-// elements are the S-box indexes. They use the same form as in keyScheduleA,
-// above.
-
-type keyScheduleRound struct{}
-type keySchedule []keyScheduleRound
-
-var schedule = []struct {
- a keyScheduleA
- b keyScheduleB
-}{
- {
- keyScheduleA{
- {4, 0, 0xd, 0xf, 0xc, 0xe, 0x8},
- {5, 2, 16 + 0, 16 + 2, 16 + 1, 16 + 3, 0xa},
- {6, 3, 16 + 7, 16 + 6, 16 + 5, 16 + 4, 9},
- {7, 1, 16 + 0xa, 16 + 9, 16 + 0xb, 16 + 8, 0xb},
- },
- keyScheduleB{
- {16 + 8, 16 + 9, 16 + 7, 16 + 6, 16 + 2},
- {16 + 0xa, 16 + 0xb, 16 + 5, 16 + 4, 16 + 6},
- {16 + 0xc, 16 + 0xd, 16 + 3, 16 + 2, 16 + 9},
- {16 + 0xe, 16 + 0xf, 16 + 1, 16 + 0, 16 + 0xc},
- },
- },
- {
- keyScheduleA{
- {0, 6, 16 + 5, 16 + 7, 16 + 4, 16 + 6, 16 + 0},
- {1, 4, 0, 2, 1, 3, 16 + 2},
- {2, 5, 7, 6, 5, 4, 16 + 1},
- {3, 7, 0xa, 9, 0xb, 8, 16 + 3},
- },
- keyScheduleB{
- {3, 2, 0xc, 0xd, 8},
- {1, 0, 0xe, 0xf, 0xd},
- {7, 6, 8, 9, 3},
- {5, 4, 0xa, 0xb, 7},
- },
- },
- {
- keyScheduleA{
- {4, 0, 0xd, 0xf, 0xc, 0xe, 8},
- {5, 2, 16 + 0, 16 + 2, 16 + 1, 16 + 3, 0xa},
- {6, 3, 16 + 7, 16 + 6, 16 + 5, 16 + 4, 9},
- {7, 1, 16 + 0xa, 16 + 9, 16 + 0xb, 16 + 8, 0xb},
- },
- keyScheduleB{
- {16 + 3, 16 + 2, 16 + 0xc, 16 + 0xd, 16 + 9},
- {16 + 1, 16 + 0, 16 + 0xe, 16 + 0xf, 16 + 0xc},
- {16 + 7, 16 + 6, 16 + 8, 16 + 9, 16 + 2},
- {16 + 5, 16 + 4, 16 + 0xa, 16 + 0xb, 16 + 6},
- },
- },
- {
- keyScheduleA{
- {0, 6, 16 + 5, 16 + 7, 16 + 4, 16 + 6, 16 + 0},
- {1, 4, 0, 2, 1, 3, 16 + 2},
- {2, 5, 7, 6, 5, 4, 16 + 1},
- {3, 7, 0xa, 9, 0xb, 8, 16 + 3},
- },
- keyScheduleB{
- {8, 9, 7, 6, 3},
- {0xa, 0xb, 5, 4, 7},
- {0xc, 0xd, 3, 2, 8},
- {0xe, 0xf, 1, 0, 0xd},
- },
- },
-}
-
-func (c *Cipher) keySchedule(in []byte) {
- var t [8]uint32
- var k [32]uint32
-
- for i := 0; i < 4; i++ {
- j := i * 4
- t[i] = uint32(in[j])<<24 | uint32(in[j+1])<<16 | uint32(in[j+2])<<8 | uint32(in[j+3])
- }
-
- x := []byte{6, 7, 4, 5}
- ki := 0
-
- for half := 0; half < 2; half++ {
- for _, round := range schedule {
- for j := 0; j < 4; j++ {
- var a [7]uint8
- copy(a[:], round.a[j][:])
- w := t[a[1]]
- w ^= sBox[4][(t[a[2]>>2]>>(24-8*(a[2]&3)))&0xff]
- w ^= sBox[5][(t[a[3]>>2]>>(24-8*(a[3]&3)))&0xff]
- w ^= sBox[6][(t[a[4]>>2]>>(24-8*(a[4]&3)))&0xff]
- w ^= sBox[7][(t[a[5]>>2]>>(24-8*(a[5]&3)))&0xff]
- w ^= sBox[x[j]][(t[a[6]>>2]>>(24-8*(a[6]&3)))&0xff]
- t[a[0]] = w
- }
-
- for j := 0; j < 4; j++ {
- var b [5]uint8
- copy(b[:], round.b[j][:])
- w := sBox[4][(t[b[0]>>2]>>(24-8*(b[0]&3)))&0xff]
- w ^= sBox[5][(t[b[1]>>2]>>(24-8*(b[1]&3)))&0xff]
- w ^= sBox[6][(t[b[2]>>2]>>(24-8*(b[2]&3)))&0xff]
- w ^= sBox[7][(t[b[3]>>2]>>(24-8*(b[3]&3)))&0xff]
- w ^= sBox[4+j][(t[b[4]>>2]>>(24-8*(b[4]&3)))&0xff]
- k[ki] = w
- ki++
- }
- }
- }
-
- for i := 0; i < 16; i++ {
- c.masking[i] = k[i]
- c.rotate[i] = uint8(k[16+i] & 0x1f)
- }
-}
-
-// These are the three 'f' functions. See RFC 2144, section 2.2.
-func f1(d, m uint32, r uint8) uint32 {
- t := m + d
- I := (t << r) | (t >> (32 - r))
- return ((sBox[0][I>>24] ^ sBox[1][(I>>16)&0xff]) - sBox[2][(I>>8)&0xff]) + sBox[3][I&0xff]
-}
-
-func f2(d, m uint32, r uint8) uint32 {
- t := m ^ d
- I := (t << r) | (t >> (32 - r))
- return ((sBox[0][I>>24] - sBox[1][(I>>16)&0xff]) + sBox[2][(I>>8)&0xff]) ^ sBox[3][I&0xff]
-}
-
-func f3(d, m uint32, r uint8) uint32 {
- t := m - d
- I := (t << r) | (t >> (32 - r))
- return ((sBox[0][I>>24] + sBox[1][(I>>16)&0xff]) ^ sBox[2][(I>>8)&0xff]) - sBox[3][I&0xff]
-}
-
-var sBox = [8][256]uint32{
- {
- 0x30fb40d4, 0x9fa0ff0b, 0x6beccd2f, 0x3f258c7a, 0x1e213f2f, 0x9c004dd3, 0x6003e540, 0xcf9fc949,
- 0xbfd4af27, 0x88bbbdb5, 0xe2034090, 0x98d09675, 0x6e63a0e0, 0x15c361d2, 0xc2e7661d, 0x22d4ff8e,
- 0x28683b6f, 0xc07fd059, 0xff2379c8, 0x775f50e2, 0x43c340d3, 0xdf2f8656, 0x887ca41a, 0xa2d2bd2d,
- 0xa1c9e0d6, 0x346c4819, 0x61b76d87, 0x22540f2f, 0x2abe32e1, 0xaa54166b, 0x22568e3a, 0xa2d341d0,
- 0x66db40c8, 0xa784392f, 0x004dff2f, 0x2db9d2de, 0x97943fac, 0x4a97c1d8, 0x527644b7, 0xb5f437a7,
- 0xb82cbaef, 0xd751d159, 0x6ff7f0ed, 0x5a097a1f, 0x827b68d0, 0x90ecf52e, 0x22b0c054, 0xbc8e5935,
- 0x4b6d2f7f, 0x50bb64a2, 0xd2664910, 0xbee5812d, 0xb7332290, 0xe93b159f, 0xb48ee411, 0x4bff345d,
- 0xfd45c240, 0xad31973f, 0xc4f6d02e, 0x55fc8165, 0xd5b1caad, 0xa1ac2dae, 0xa2d4b76d, 0xc19b0c50,
- 0x882240f2, 0x0c6e4f38, 0xa4e4bfd7, 0x4f5ba272, 0x564c1d2f, 0xc59c5319, 0xb949e354, 0xb04669fe,
- 0xb1b6ab8a, 0xc71358dd, 0x6385c545, 0x110f935d, 0x57538ad5, 0x6a390493, 0xe63d37e0, 0x2a54f6b3,
- 0x3a787d5f, 0x6276a0b5, 0x19a6fcdf, 0x7a42206a, 0x29f9d4d5, 0xf61b1891, 0xbb72275e, 0xaa508167,
- 0x38901091, 0xc6b505eb, 0x84c7cb8c, 0x2ad75a0f, 0x874a1427, 0xa2d1936b, 0x2ad286af, 0xaa56d291,
- 0xd7894360, 0x425c750d, 0x93b39e26, 0x187184c9, 0x6c00b32d, 0x73e2bb14, 0xa0bebc3c, 0x54623779,
- 0x64459eab, 0x3f328b82, 0x7718cf82, 0x59a2cea6, 0x04ee002e, 0x89fe78e6, 0x3fab0950, 0x325ff6c2,
- 0x81383f05, 0x6963c5c8, 0x76cb5ad6, 0xd49974c9, 0xca180dcf, 0x380782d5, 0xc7fa5cf6, 0x8ac31511,
- 0x35e79e13, 0x47da91d0, 0xf40f9086, 0xa7e2419e, 0x31366241, 0x051ef495, 0xaa573b04, 0x4a805d8d,
- 0x548300d0, 0x00322a3c, 0xbf64cddf, 0xba57a68e, 0x75c6372b, 0x50afd341, 0xa7c13275, 0x915a0bf5,
- 0x6b54bfab, 0x2b0b1426, 0xab4cc9d7, 0x449ccd82, 0xf7fbf265, 0xab85c5f3, 0x1b55db94, 0xaad4e324,
- 0xcfa4bd3f, 0x2deaa3e2, 0x9e204d02, 0xc8bd25ac, 0xeadf55b3, 0xd5bd9e98, 0xe31231b2, 0x2ad5ad6c,
- 0x954329de, 0xadbe4528, 0xd8710f69, 0xaa51c90f, 0xaa786bf6, 0x22513f1e, 0xaa51a79b, 0x2ad344cc,
- 0x7b5a41f0, 0xd37cfbad, 0x1b069505, 0x41ece491, 0xb4c332e6, 0x032268d4, 0xc9600acc, 0xce387e6d,
- 0xbf6bb16c, 0x6a70fb78, 0x0d03d9c9, 0xd4df39de, 0xe01063da, 0x4736f464, 0x5ad328d8, 0xb347cc96,
- 0x75bb0fc3, 0x98511bfb, 0x4ffbcc35, 0xb58bcf6a, 0xe11f0abc, 0xbfc5fe4a, 0xa70aec10, 0xac39570a,
- 0x3f04442f, 0x6188b153, 0xe0397a2e, 0x5727cb79, 0x9ceb418f, 0x1cacd68d, 0x2ad37c96, 0x0175cb9d,
- 0xc69dff09, 0xc75b65f0, 0xd9db40d8, 0xec0e7779, 0x4744ead4, 0xb11c3274, 0xdd24cb9e, 0x7e1c54bd,
- 0xf01144f9, 0xd2240eb1, 0x9675b3fd, 0xa3ac3755, 0xd47c27af, 0x51c85f4d, 0x56907596, 0xa5bb15e6,
- 0x580304f0, 0xca042cf1, 0x011a37ea, 0x8dbfaadb, 0x35ba3e4a, 0x3526ffa0, 0xc37b4d09, 0xbc306ed9,
- 0x98a52666, 0x5648f725, 0xff5e569d, 0x0ced63d0, 0x7c63b2cf, 0x700b45e1, 0xd5ea50f1, 0x85a92872,
- 0xaf1fbda7, 0xd4234870, 0xa7870bf3, 0x2d3b4d79, 0x42e04198, 0x0cd0ede7, 0x26470db8, 0xf881814c,
- 0x474d6ad7, 0x7c0c5e5c, 0xd1231959, 0x381b7298, 0xf5d2f4db, 0xab838653, 0x6e2f1e23, 0x83719c9e,
- 0xbd91e046, 0x9a56456e, 0xdc39200c, 0x20c8c571, 0x962bda1c, 0xe1e696ff, 0xb141ab08, 0x7cca89b9,
- 0x1a69e783, 0x02cc4843, 0xa2f7c579, 0x429ef47d, 0x427b169c, 0x5ac9f049, 0xdd8f0f00, 0x5c8165bf,
- },
- {
- 0x1f201094, 0xef0ba75b, 0x69e3cf7e, 0x393f4380, 0xfe61cf7a, 0xeec5207a, 0x55889c94, 0x72fc0651,
- 0xada7ef79, 0x4e1d7235, 0xd55a63ce, 0xde0436ba, 0x99c430ef, 0x5f0c0794, 0x18dcdb7d, 0xa1d6eff3,
- 0xa0b52f7b, 0x59e83605, 0xee15b094, 0xe9ffd909, 0xdc440086, 0xef944459, 0xba83ccb3, 0xe0c3cdfb,
- 0xd1da4181, 0x3b092ab1, 0xf997f1c1, 0xa5e6cf7b, 0x01420ddb, 0xe4e7ef5b, 0x25a1ff41, 0xe180f806,
- 0x1fc41080, 0x179bee7a, 0xd37ac6a9, 0xfe5830a4, 0x98de8b7f, 0x77e83f4e, 0x79929269, 0x24fa9f7b,
- 0xe113c85b, 0xacc40083, 0xd7503525, 0xf7ea615f, 0x62143154, 0x0d554b63, 0x5d681121, 0xc866c359,
- 0x3d63cf73, 0xcee234c0, 0xd4d87e87, 0x5c672b21, 0x071f6181, 0x39f7627f, 0x361e3084, 0xe4eb573b,
- 0x602f64a4, 0xd63acd9c, 0x1bbc4635, 0x9e81032d, 0x2701f50c, 0x99847ab4, 0xa0e3df79, 0xba6cf38c,
- 0x10843094, 0x2537a95e, 0xf46f6ffe, 0xa1ff3b1f, 0x208cfb6a, 0x8f458c74, 0xd9e0a227, 0x4ec73a34,
- 0xfc884f69, 0x3e4de8df, 0xef0e0088, 0x3559648d, 0x8a45388c, 0x1d804366, 0x721d9bfd, 0xa58684bb,
- 0xe8256333, 0x844e8212, 0x128d8098, 0xfed33fb4, 0xce280ae1, 0x27e19ba5, 0xd5a6c252, 0xe49754bd,
- 0xc5d655dd, 0xeb667064, 0x77840b4d, 0xa1b6a801, 0x84db26a9, 0xe0b56714, 0x21f043b7, 0xe5d05860,
- 0x54f03084, 0x066ff472, 0xa31aa153, 0xdadc4755, 0xb5625dbf, 0x68561be6, 0x83ca6b94, 0x2d6ed23b,
- 0xeccf01db, 0xa6d3d0ba, 0xb6803d5c, 0xaf77a709, 0x33b4a34c, 0x397bc8d6, 0x5ee22b95, 0x5f0e5304,
- 0x81ed6f61, 0x20e74364, 0xb45e1378, 0xde18639b, 0x881ca122, 0xb96726d1, 0x8049a7e8, 0x22b7da7b,
- 0x5e552d25, 0x5272d237, 0x79d2951c, 0xc60d894c, 0x488cb402, 0x1ba4fe5b, 0xa4b09f6b, 0x1ca815cf,
- 0xa20c3005, 0x8871df63, 0xb9de2fcb, 0x0cc6c9e9, 0x0beeff53, 0xe3214517, 0xb4542835, 0x9f63293c,
- 0xee41e729, 0x6e1d2d7c, 0x50045286, 0x1e6685f3, 0xf33401c6, 0x30a22c95, 0x31a70850, 0x60930f13,
- 0x73f98417, 0xa1269859, 0xec645c44, 0x52c877a9, 0xcdff33a6, 0xa02b1741, 0x7cbad9a2, 0x2180036f,
- 0x50d99c08, 0xcb3f4861, 0xc26bd765, 0x64a3f6ab, 0x80342676, 0x25a75e7b, 0xe4e6d1fc, 0x20c710e6,
- 0xcdf0b680, 0x17844d3b, 0x31eef84d, 0x7e0824e4, 0x2ccb49eb, 0x846a3bae, 0x8ff77888, 0xee5d60f6,
- 0x7af75673, 0x2fdd5cdb, 0xa11631c1, 0x30f66f43, 0xb3faec54, 0x157fd7fa, 0xef8579cc, 0xd152de58,
- 0xdb2ffd5e, 0x8f32ce19, 0x306af97a, 0x02f03ef8, 0x99319ad5, 0xc242fa0f, 0xa7e3ebb0, 0xc68e4906,
- 0xb8da230c, 0x80823028, 0xdcdef3c8, 0xd35fb171, 0x088a1bc8, 0xbec0c560, 0x61a3c9e8, 0xbca8f54d,
- 0xc72feffa, 0x22822e99, 0x82c570b4, 0xd8d94e89, 0x8b1c34bc, 0x301e16e6, 0x273be979, 0xb0ffeaa6,
- 0x61d9b8c6, 0x00b24869, 0xb7ffce3f, 0x08dc283b, 0x43daf65a, 0xf7e19798, 0x7619b72f, 0x8f1c9ba4,
- 0xdc8637a0, 0x16a7d3b1, 0x9fc393b7, 0xa7136eeb, 0xc6bcc63e, 0x1a513742, 0xef6828bc, 0x520365d6,
- 0x2d6a77ab, 0x3527ed4b, 0x821fd216, 0x095c6e2e, 0xdb92f2fb, 0x5eea29cb, 0x145892f5, 0x91584f7f,
- 0x5483697b, 0x2667a8cc, 0x85196048, 0x8c4bacea, 0x833860d4, 0x0d23e0f9, 0x6c387e8a, 0x0ae6d249,
- 0xb284600c, 0xd835731d, 0xdcb1c647, 0xac4c56ea, 0x3ebd81b3, 0x230eabb0, 0x6438bc87, 0xf0b5b1fa,
- 0x8f5ea2b3, 0xfc184642, 0x0a036b7a, 0x4fb089bd, 0x649da589, 0xa345415e, 0x5c038323, 0x3e5d3bb9,
- 0x43d79572, 0x7e6dd07c, 0x06dfdf1e, 0x6c6cc4ef, 0x7160a539, 0x73bfbe70, 0x83877605, 0x4523ecf1,
- },
- {
- 0x8defc240, 0x25fa5d9f, 0xeb903dbf, 0xe810c907, 0x47607fff, 0x369fe44b, 0x8c1fc644, 0xaececa90,
- 0xbeb1f9bf, 0xeefbcaea, 0xe8cf1950, 0x51df07ae, 0x920e8806, 0xf0ad0548, 0xe13c8d83, 0x927010d5,
- 0x11107d9f, 0x07647db9, 0xb2e3e4d4, 0x3d4f285e, 0xb9afa820, 0xfade82e0, 0xa067268b, 0x8272792e,
- 0x553fb2c0, 0x489ae22b, 0xd4ef9794, 0x125e3fbc, 0x21fffcee, 0x825b1bfd, 0x9255c5ed, 0x1257a240,
- 0x4e1a8302, 0xbae07fff, 0x528246e7, 0x8e57140e, 0x3373f7bf, 0x8c9f8188, 0xa6fc4ee8, 0xc982b5a5,
- 0xa8c01db7, 0x579fc264, 0x67094f31, 0xf2bd3f5f, 0x40fff7c1, 0x1fb78dfc, 0x8e6bd2c1, 0x437be59b,
- 0x99b03dbf, 0xb5dbc64b, 0x638dc0e6, 0x55819d99, 0xa197c81c, 0x4a012d6e, 0xc5884a28, 0xccc36f71,
- 0xb843c213, 0x6c0743f1, 0x8309893c, 0x0feddd5f, 0x2f7fe850, 0xd7c07f7e, 0x02507fbf, 0x5afb9a04,
- 0xa747d2d0, 0x1651192e, 0xaf70bf3e, 0x58c31380, 0x5f98302e, 0x727cc3c4, 0x0a0fb402, 0x0f7fef82,
- 0x8c96fdad, 0x5d2c2aae, 0x8ee99a49, 0x50da88b8, 0x8427f4a0, 0x1eac5790, 0x796fb449, 0x8252dc15,
- 0xefbd7d9b, 0xa672597d, 0xada840d8, 0x45f54504, 0xfa5d7403, 0xe83ec305, 0x4f91751a, 0x925669c2,
- 0x23efe941, 0xa903f12e, 0x60270df2, 0x0276e4b6, 0x94fd6574, 0x927985b2, 0x8276dbcb, 0x02778176,
- 0xf8af918d, 0x4e48f79e, 0x8f616ddf, 0xe29d840e, 0x842f7d83, 0x340ce5c8, 0x96bbb682, 0x93b4b148,
- 0xef303cab, 0x984faf28, 0x779faf9b, 0x92dc560d, 0x224d1e20, 0x8437aa88, 0x7d29dc96, 0x2756d3dc,
- 0x8b907cee, 0xb51fd240, 0xe7c07ce3, 0xe566b4a1, 0xc3e9615e, 0x3cf8209d, 0x6094d1e3, 0xcd9ca341,
- 0x5c76460e, 0x00ea983b, 0xd4d67881, 0xfd47572c, 0xf76cedd9, 0xbda8229c, 0x127dadaa, 0x438a074e,
- 0x1f97c090, 0x081bdb8a, 0x93a07ebe, 0xb938ca15, 0x97b03cff, 0x3dc2c0f8, 0x8d1ab2ec, 0x64380e51,
- 0x68cc7bfb, 0xd90f2788, 0x12490181, 0x5de5ffd4, 0xdd7ef86a, 0x76a2e214, 0xb9a40368, 0x925d958f,
- 0x4b39fffa, 0xba39aee9, 0xa4ffd30b, 0xfaf7933b, 0x6d498623, 0x193cbcfa, 0x27627545, 0x825cf47a,
- 0x61bd8ba0, 0xd11e42d1, 0xcead04f4, 0x127ea392, 0x10428db7, 0x8272a972, 0x9270c4a8, 0x127de50b,
- 0x285ba1c8, 0x3c62f44f, 0x35c0eaa5, 0xe805d231, 0x428929fb, 0xb4fcdf82, 0x4fb66a53, 0x0e7dc15b,
- 0x1f081fab, 0x108618ae, 0xfcfd086d, 0xf9ff2889, 0x694bcc11, 0x236a5cae, 0x12deca4d, 0x2c3f8cc5,
- 0xd2d02dfe, 0xf8ef5896, 0xe4cf52da, 0x95155b67, 0x494a488c, 0xb9b6a80c, 0x5c8f82bc, 0x89d36b45,
- 0x3a609437, 0xec00c9a9, 0x44715253, 0x0a874b49, 0xd773bc40, 0x7c34671c, 0x02717ef6, 0x4feb5536,
- 0xa2d02fff, 0xd2bf60c4, 0xd43f03c0, 0x50b4ef6d, 0x07478cd1, 0x006e1888, 0xa2e53f55, 0xb9e6d4bc,
- 0xa2048016, 0x97573833, 0xd7207d67, 0xde0f8f3d, 0x72f87b33, 0xabcc4f33, 0x7688c55d, 0x7b00a6b0,
- 0x947b0001, 0x570075d2, 0xf9bb88f8, 0x8942019e, 0x4264a5ff, 0x856302e0, 0x72dbd92b, 0xee971b69,
- 0x6ea22fde, 0x5f08ae2b, 0xaf7a616d, 0xe5c98767, 0xcf1febd2, 0x61efc8c2, 0xf1ac2571, 0xcc8239c2,
- 0x67214cb8, 0xb1e583d1, 0xb7dc3e62, 0x7f10bdce, 0xf90a5c38, 0x0ff0443d, 0x606e6dc6, 0x60543a49,
- 0x5727c148, 0x2be98a1d, 0x8ab41738, 0x20e1be24, 0xaf96da0f, 0x68458425, 0x99833be5, 0x600d457d,
- 0x282f9350, 0x8334b362, 0xd91d1120, 0x2b6d8da0, 0x642b1e31, 0x9c305a00, 0x52bce688, 0x1b03588a,
- 0xf7baefd5, 0x4142ed9c, 0xa4315c11, 0x83323ec5, 0xdfef4636, 0xa133c501, 0xe9d3531c, 0xee353783,
- },
- {
- 0x9db30420, 0x1fb6e9de, 0xa7be7bef, 0xd273a298, 0x4a4f7bdb, 0x64ad8c57, 0x85510443, 0xfa020ed1,
- 0x7e287aff, 0xe60fb663, 0x095f35a1, 0x79ebf120, 0xfd059d43, 0x6497b7b1, 0xf3641f63, 0x241e4adf,
- 0x28147f5f, 0x4fa2b8cd, 0xc9430040, 0x0cc32220, 0xfdd30b30, 0xc0a5374f, 0x1d2d00d9, 0x24147b15,
- 0xee4d111a, 0x0fca5167, 0x71ff904c, 0x2d195ffe, 0x1a05645f, 0x0c13fefe, 0x081b08ca, 0x05170121,
- 0x80530100, 0xe83e5efe, 0xac9af4f8, 0x7fe72701, 0xd2b8ee5f, 0x06df4261, 0xbb9e9b8a, 0x7293ea25,
- 0xce84ffdf, 0xf5718801, 0x3dd64b04, 0xa26f263b, 0x7ed48400, 0x547eebe6, 0x446d4ca0, 0x6cf3d6f5,
- 0x2649abdf, 0xaea0c7f5, 0x36338cc1, 0x503f7e93, 0xd3772061, 0x11b638e1, 0x72500e03, 0xf80eb2bb,
- 0xabe0502e, 0xec8d77de, 0x57971e81, 0xe14f6746, 0xc9335400, 0x6920318f, 0x081dbb99, 0xffc304a5,
- 0x4d351805, 0x7f3d5ce3, 0xa6c866c6, 0x5d5bcca9, 0xdaec6fea, 0x9f926f91, 0x9f46222f, 0x3991467d,
- 0xa5bf6d8e, 0x1143c44f, 0x43958302, 0xd0214eeb, 0x022083b8, 0x3fb6180c, 0x18f8931e, 0x281658e6,
- 0x26486e3e, 0x8bd78a70, 0x7477e4c1, 0xb506e07c, 0xf32d0a25, 0x79098b02, 0xe4eabb81, 0x28123b23,
- 0x69dead38, 0x1574ca16, 0xdf871b62, 0x211c40b7, 0xa51a9ef9, 0x0014377b, 0x041e8ac8, 0x09114003,
- 0xbd59e4d2, 0xe3d156d5, 0x4fe876d5, 0x2f91a340, 0x557be8de, 0x00eae4a7, 0x0ce5c2ec, 0x4db4bba6,
- 0xe756bdff, 0xdd3369ac, 0xec17b035, 0x06572327, 0x99afc8b0, 0x56c8c391, 0x6b65811c, 0x5e146119,
- 0x6e85cb75, 0xbe07c002, 0xc2325577, 0x893ff4ec, 0x5bbfc92d, 0xd0ec3b25, 0xb7801ab7, 0x8d6d3b24,
- 0x20c763ef, 0xc366a5fc, 0x9c382880, 0x0ace3205, 0xaac9548a, 0xeca1d7c7, 0x041afa32, 0x1d16625a,
- 0x6701902c, 0x9b757a54, 0x31d477f7, 0x9126b031, 0x36cc6fdb, 0xc70b8b46, 0xd9e66a48, 0x56e55a79,
- 0x026a4ceb, 0x52437eff, 0x2f8f76b4, 0x0df980a5, 0x8674cde3, 0xedda04eb, 0x17a9be04, 0x2c18f4df,
- 0xb7747f9d, 0xab2af7b4, 0xefc34d20, 0x2e096b7c, 0x1741a254, 0xe5b6a035, 0x213d42f6, 0x2c1c7c26,
- 0x61c2f50f, 0x6552daf9, 0xd2c231f8, 0x25130f69, 0xd8167fa2, 0x0418f2c8, 0x001a96a6, 0x0d1526ab,
- 0x63315c21, 0x5e0a72ec, 0x49bafefd, 0x187908d9, 0x8d0dbd86, 0x311170a7, 0x3e9b640c, 0xcc3e10d7,
- 0xd5cad3b6, 0x0caec388, 0xf73001e1, 0x6c728aff, 0x71eae2a1, 0x1f9af36e, 0xcfcbd12f, 0xc1de8417,
- 0xac07be6b, 0xcb44a1d8, 0x8b9b0f56, 0x013988c3, 0xb1c52fca, 0xb4be31cd, 0xd8782806, 0x12a3a4e2,
- 0x6f7de532, 0x58fd7eb6, 0xd01ee900, 0x24adffc2, 0xf4990fc5, 0x9711aac5, 0x001d7b95, 0x82e5e7d2,
- 0x109873f6, 0x00613096, 0xc32d9521, 0xada121ff, 0x29908415, 0x7fbb977f, 0xaf9eb3db, 0x29c9ed2a,
- 0x5ce2a465, 0xa730f32c, 0xd0aa3fe8, 0x8a5cc091, 0xd49e2ce7, 0x0ce454a9, 0xd60acd86, 0x015f1919,
- 0x77079103, 0xdea03af6, 0x78a8565e, 0xdee356df, 0x21f05cbe, 0x8b75e387, 0xb3c50651, 0xb8a5c3ef,
- 0xd8eeb6d2, 0xe523be77, 0xc2154529, 0x2f69efdf, 0xafe67afb, 0xf470c4b2, 0xf3e0eb5b, 0xd6cc9876,
- 0x39e4460c, 0x1fda8538, 0x1987832f, 0xca007367, 0xa99144f8, 0x296b299e, 0x492fc295, 0x9266beab,
- 0xb5676e69, 0x9bd3ddda, 0xdf7e052f, 0xdb25701c, 0x1b5e51ee, 0xf65324e6, 0x6afce36c, 0x0316cc04,
- 0x8644213e, 0xb7dc59d0, 0x7965291f, 0xccd6fd43, 0x41823979, 0x932bcdf6, 0xb657c34d, 0x4edfd282,
- 0x7ae5290c, 0x3cb9536b, 0x851e20fe, 0x9833557e, 0x13ecf0b0, 0xd3ffb372, 0x3f85c5c1, 0x0aef7ed2,
- },
- {
- 0x7ec90c04, 0x2c6e74b9, 0x9b0e66df, 0xa6337911, 0xb86a7fff, 0x1dd358f5, 0x44dd9d44, 0x1731167f,
- 0x08fbf1fa, 0xe7f511cc, 0xd2051b00, 0x735aba00, 0x2ab722d8, 0x386381cb, 0xacf6243a, 0x69befd7a,
- 0xe6a2e77f, 0xf0c720cd, 0xc4494816, 0xccf5c180, 0x38851640, 0x15b0a848, 0xe68b18cb, 0x4caadeff,
- 0x5f480a01, 0x0412b2aa, 0x259814fc, 0x41d0efe2, 0x4e40b48d, 0x248eb6fb, 0x8dba1cfe, 0x41a99b02,
- 0x1a550a04, 0xba8f65cb, 0x7251f4e7, 0x95a51725, 0xc106ecd7, 0x97a5980a, 0xc539b9aa, 0x4d79fe6a,
- 0xf2f3f763, 0x68af8040, 0xed0c9e56, 0x11b4958b, 0xe1eb5a88, 0x8709e6b0, 0xd7e07156, 0x4e29fea7,
- 0x6366e52d, 0x02d1c000, 0xc4ac8e05, 0x9377f571, 0x0c05372a, 0x578535f2, 0x2261be02, 0xd642a0c9,
- 0xdf13a280, 0x74b55bd2, 0x682199c0, 0xd421e5ec, 0x53fb3ce8, 0xc8adedb3, 0x28a87fc9, 0x3d959981,
- 0x5c1ff900, 0xfe38d399, 0x0c4eff0b, 0x062407ea, 0xaa2f4fb1, 0x4fb96976, 0x90c79505, 0xb0a8a774,
- 0xef55a1ff, 0xe59ca2c2, 0xa6b62d27, 0xe66a4263, 0xdf65001f, 0x0ec50966, 0xdfdd55bc, 0x29de0655,
- 0x911e739a, 0x17af8975, 0x32c7911c, 0x89f89468, 0x0d01e980, 0x524755f4, 0x03b63cc9, 0x0cc844b2,
- 0xbcf3f0aa, 0x87ac36e9, 0xe53a7426, 0x01b3d82b, 0x1a9e7449, 0x64ee2d7e, 0xcddbb1da, 0x01c94910,
- 0xb868bf80, 0x0d26f3fd, 0x9342ede7, 0x04a5c284, 0x636737b6, 0x50f5b616, 0xf24766e3, 0x8eca36c1,
- 0x136e05db, 0xfef18391, 0xfb887a37, 0xd6e7f7d4, 0xc7fb7dc9, 0x3063fcdf, 0xb6f589de, 0xec2941da,
- 0x26e46695, 0xb7566419, 0xf654efc5, 0xd08d58b7, 0x48925401, 0xc1bacb7f, 0xe5ff550f, 0xb6083049,
- 0x5bb5d0e8, 0x87d72e5a, 0xab6a6ee1, 0x223a66ce, 0xc62bf3cd, 0x9e0885f9, 0x68cb3e47, 0x086c010f,
- 0xa21de820, 0xd18b69de, 0xf3f65777, 0xfa02c3f6, 0x407edac3, 0xcbb3d550, 0x1793084d, 0xb0d70eba,
- 0x0ab378d5, 0xd951fb0c, 0xded7da56, 0x4124bbe4, 0x94ca0b56, 0x0f5755d1, 0xe0e1e56e, 0x6184b5be,
- 0x580a249f, 0x94f74bc0, 0xe327888e, 0x9f7b5561, 0xc3dc0280, 0x05687715, 0x646c6bd7, 0x44904db3,
- 0x66b4f0a3, 0xc0f1648a, 0x697ed5af, 0x49e92ff6, 0x309e374f, 0x2cb6356a, 0x85808573, 0x4991f840,
- 0x76f0ae02, 0x083be84d, 0x28421c9a, 0x44489406, 0x736e4cb8, 0xc1092910, 0x8bc95fc6, 0x7d869cf4,
- 0x134f616f, 0x2e77118d, 0xb31b2be1, 0xaa90b472, 0x3ca5d717, 0x7d161bba, 0x9cad9010, 0xaf462ba2,
- 0x9fe459d2, 0x45d34559, 0xd9f2da13, 0xdbc65487, 0xf3e4f94e, 0x176d486f, 0x097c13ea, 0x631da5c7,
- 0x445f7382, 0x175683f4, 0xcdc66a97, 0x70be0288, 0xb3cdcf72, 0x6e5dd2f3, 0x20936079, 0x459b80a5,
- 0xbe60e2db, 0xa9c23101, 0xeba5315c, 0x224e42f2, 0x1c5c1572, 0xf6721b2c, 0x1ad2fff3, 0x8c25404e,
- 0x324ed72f, 0x4067b7fd, 0x0523138e, 0x5ca3bc78, 0xdc0fd66e, 0x75922283, 0x784d6b17, 0x58ebb16e,
- 0x44094f85, 0x3f481d87, 0xfcfeae7b, 0x77b5ff76, 0x8c2302bf, 0xaaf47556, 0x5f46b02a, 0x2b092801,
- 0x3d38f5f7, 0x0ca81f36, 0x52af4a8a, 0x66d5e7c0, 0xdf3b0874, 0x95055110, 0x1b5ad7a8, 0xf61ed5ad,
- 0x6cf6e479, 0x20758184, 0xd0cefa65, 0x88f7be58, 0x4a046826, 0x0ff6f8f3, 0xa09c7f70, 0x5346aba0,
- 0x5ce96c28, 0xe176eda3, 0x6bac307f, 0x376829d2, 0x85360fa9, 0x17e3fe2a, 0x24b79767, 0xf5a96b20,
- 0xd6cd2595, 0x68ff1ebf, 0x7555442c, 0xf19f06be, 0xf9e0659a, 0xeeb9491d, 0x34010718, 0xbb30cab8,
- 0xe822fe15, 0x88570983, 0x750e6249, 0xda627e55, 0x5e76ffa8, 0xb1534546, 0x6d47de08, 0xefe9e7d4,
- },
- {
- 0xf6fa8f9d, 0x2cac6ce1, 0x4ca34867, 0xe2337f7c, 0x95db08e7, 0x016843b4, 0xeced5cbc, 0x325553ac,
- 0xbf9f0960, 0xdfa1e2ed, 0x83f0579d, 0x63ed86b9, 0x1ab6a6b8, 0xde5ebe39, 0xf38ff732, 0x8989b138,
- 0x33f14961, 0xc01937bd, 0xf506c6da, 0xe4625e7e, 0xa308ea99, 0x4e23e33c, 0x79cbd7cc, 0x48a14367,
- 0xa3149619, 0xfec94bd5, 0xa114174a, 0xeaa01866, 0xa084db2d, 0x09a8486f, 0xa888614a, 0x2900af98,
- 0x01665991, 0xe1992863, 0xc8f30c60, 0x2e78ef3c, 0xd0d51932, 0xcf0fec14, 0xf7ca07d2, 0xd0a82072,
- 0xfd41197e, 0x9305a6b0, 0xe86be3da, 0x74bed3cd, 0x372da53c, 0x4c7f4448, 0xdab5d440, 0x6dba0ec3,
- 0x083919a7, 0x9fbaeed9, 0x49dbcfb0, 0x4e670c53, 0x5c3d9c01, 0x64bdb941, 0x2c0e636a, 0xba7dd9cd,
- 0xea6f7388, 0xe70bc762, 0x35f29adb, 0x5c4cdd8d, 0xf0d48d8c, 0xb88153e2, 0x08a19866, 0x1ae2eac8,
- 0x284caf89, 0xaa928223, 0x9334be53, 0x3b3a21bf, 0x16434be3, 0x9aea3906, 0xefe8c36e, 0xf890cdd9,
- 0x80226dae, 0xc340a4a3, 0xdf7e9c09, 0xa694a807, 0x5b7c5ecc, 0x221db3a6, 0x9a69a02f, 0x68818a54,
- 0xceb2296f, 0x53c0843a, 0xfe893655, 0x25bfe68a, 0xb4628abc, 0xcf222ebf, 0x25ac6f48, 0xa9a99387,
- 0x53bddb65, 0xe76ffbe7, 0xe967fd78, 0x0ba93563, 0x8e342bc1, 0xe8a11be9, 0x4980740d, 0xc8087dfc,
- 0x8de4bf99, 0xa11101a0, 0x7fd37975, 0xda5a26c0, 0xe81f994f, 0x9528cd89, 0xfd339fed, 0xb87834bf,
- 0x5f04456d, 0x22258698, 0xc9c4c83b, 0x2dc156be, 0x4f628daa, 0x57f55ec5, 0xe2220abe, 0xd2916ebf,
- 0x4ec75b95, 0x24f2c3c0, 0x42d15d99, 0xcd0d7fa0, 0x7b6e27ff, 0xa8dc8af0, 0x7345c106, 0xf41e232f,
- 0x35162386, 0xe6ea8926, 0x3333b094, 0x157ec6f2, 0x372b74af, 0x692573e4, 0xe9a9d848, 0xf3160289,
- 0x3a62ef1d, 0xa787e238, 0xf3a5f676, 0x74364853, 0x20951063, 0x4576698d, 0xb6fad407, 0x592af950,
- 0x36f73523, 0x4cfb6e87, 0x7da4cec0, 0x6c152daa, 0xcb0396a8, 0xc50dfe5d, 0xfcd707ab, 0x0921c42f,
- 0x89dff0bb, 0x5fe2be78, 0x448f4f33, 0x754613c9, 0x2b05d08d, 0x48b9d585, 0xdc049441, 0xc8098f9b,
- 0x7dede786, 0xc39a3373, 0x42410005, 0x6a091751, 0x0ef3c8a6, 0x890072d6, 0x28207682, 0xa9a9f7be,
- 0xbf32679d, 0xd45b5b75, 0xb353fd00, 0xcbb0e358, 0x830f220a, 0x1f8fb214, 0xd372cf08, 0xcc3c4a13,
- 0x8cf63166, 0x061c87be, 0x88c98f88, 0x6062e397, 0x47cf8e7a, 0xb6c85283, 0x3cc2acfb, 0x3fc06976,
- 0x4e8f0252, 0x64d8314d, 0xda3870e3, 0x1e665459, 0xc10908f0, 0x513021a5, 0x6c5b68b7, 0x822f8aa0,
- 0x3007cd3e, 0x74719eef, 0xdc872681, 0x073340d4, 0x7e432fd9, 0x0c5ec241, 0x8809286c, 0xf592d891,
- 0x08a930f6, 0x957ef305, 0xb7fbffbd, 0xc266e96f, 0x6fe4ac98, 0xb173ecc0, 0xbc60b42a, 0x953498da,
- 0xfba1ae12, 0x2d4bd736, 0x0f25faab, 0xa4f3fceb, 0xe2969123, 0x257f0c3d, 0x9348af49, 0x361400bc,
- 0xe8816f4a, 0x3814f200, 0xa3f94043, 0x9c7a54c2, 0xbc704f57, 0xda41e7f9, 0xc25ad33a, 0x54f4a084,
- 0xb17f5505, 0x59357cbe, 0xedbd15c8, 0x7f97c5ab, 0xba5ac7b5, 0xb6f6deaf, 0x3a479c3a, 0x5302da25,
- 0x653d7e6a, 0x54268d49, 0x51a477ea, 0x5017d55b, 0xd7d25d88, 0x44136c76, 0x0404a8c8, 0xb8e5a121,
- 0xb81a928a, 0x60ed5869, 0x97c55b96, 0xeaec991b, 0x29935913, 0x01fdb7f1, 0x088e8dfa, 0x9ab6f6f5,
- 0x3b4cbf9f, 0x4a5de3ab, 0xe6051d35, 0xa0e1d855, 0xd36b4cf1, 0xf544edeb, 0xb0e93524, 0xbebb8fbd,
- 0xa2d762cf, 0x49c92f54, 0x38b5f331, 0x7128a454, 0x48392905, 0xa65b1db8, 0x851c97bd, 0xd675cf2f,
- },
- {
- 0x85e04019, 0x332bf567, 0x662dbfff, 0xcfc65693, 0x2a8d7f6f, 0xab9bc912, 0xde6008a1, 0x2028da1f,
- 0x0227bce7, 0x4d642916, 0x18fac300, 0x50f18b82, 0x2cb2cb11, 0xb232e75c, 0x4b3695f2, 0xb28707de,
- 0xa05fbcf6, 0xcd4181e9, 0xe150210c, 0xe24ef1bd, 0xb168c381, 0xfde4e789, 0x5c79b0d8, 0x1e8bfd43,
- 0x4d495001, 0x38be4341, 0x913cee1d, 0x92a79c3f, 0x089766be, 0xbaeeadf4, 0x1286becf, 0xb6eacb19,
- 0x2660c200, 0x7565bde4, 0x64241f7a, 0x8248dca9, 0xc3b3ad66, 0x28136086, 0x0bd8dfa8, 0x356d1cf2,
- 0x107789be, 0xb3b2e9ce, 0x0502aa8f, 0x0bc0351e, 0x166bf52a, 0xeb12ff82, 0xe3486911, 0xd34d7516,
- 0x4e7b3aff, 0x5f43671b, 0x9cf6e037, 0x4981ac83, 0x334266ce, 0x8c9341b7, 0xd0d854c0, 0xcb3a6c88,
- 0x47bc2829, 0x4725ba37, 0xa66ad22b, 0x7ad61f1e, 0x0c5cbafa, 0x4437f107, 0xb6e79962, 0x42d2d816,
- 0x0a961288, 0xe1a5c06e, 0x13749e67, 0x72fc081a, 0xb1d139f7, 0xf9583745, 0xcf19df58, 0xbec3f756,
- 0xc06eba30, 0x07211b24, 0x45c28829, 0xc95e317f, 0xbc8ec511, 0x38bc46e9, 0xc6e6fa14, 0xbae8584a,
- 0xad4ebc46, 0x468f508b, 0x7829435f, 0xf124183b, 0x821dba9f, 0xaff60ff4, 0xea2c4e6d, 0x16e39264,
- 0x92544a8b, 0x009b4fc3, 0xaba68ced, 0x9ac96f78, 0x06a5b79a, 0xb2856e6e, 0x1aec3ca9, 0xbe838688,
- 0x0e0804e9, 0x55f1be56, 0xe7e5363b, 0xb3a1f25d, 0xf7debb85, 0x61fe033c, 0x16746233, 0x3c034c28,
- 0xda6d0c74, 0x79aac56c, 0x3ce4e1ad, 0x51f0c802, 0x98f8f35a, 0x1626a49f, 0xeed82b29, 0x1d382fe3,
- 0x0c4fb99a, 0xbb325778, 0x3ec6d97b, 0x6e77a6a9, 0xcb658b5c, 0xd45230c7, 0x2bd1408b, 0x60c03eb7,
- 0xb9068d78, 0xa33754f4, 0xf430c87d, 0xc8a71302, 0xb96d8c32, 0xebd4e7be, 0xbe8b9d2d, 0x7979fb06,
- 0xe7225308, 0x8b75cf77, 0x11ef8da4, 0xe083c858, 0x8d6b786f, 0x5a6317a6, 0xfa5cf7a0, 0x5dda0033,
- 0xf28ebfb0, 0xf5b9c310, 0xa0eac280, 0x08b9767a, 0xa3d9d2b0, 0x79d34217, 0x021a718d, 0x9ac6336a,
- 0x2711fd60, 0x438050e3, 0x069908a8, 0x3d7fedc4, 0x826d2bef, 0x4eeb8476, 0x488dcf25, 0x36c9d566,
- 0x28e74e41, 0xc2610aca, 0x3d49a9cf, 0xbae3b9df, 0xb65f8de6, 0x92aeaf64, 0x3ac7d5e6, 0x9ea80509,
- 0xf22b017d, 0xa4173f70, 0xdd1e16c3, 0x15e0d7f9, 0x50b1b887, 0x2b9f4fd5, 0x625aba82, 0x6a017962,
- 0x2ec01b9c, 0x15488aa9, 0xd716e740, 0x40055a2c, 0x93d29a22, 0xe32dbf9a, 0x058745b9, 0x3453dc1e,
- 0xd699296e, 0x496cff6f, 0x1c9f4986, 0xdfe2ed07, 0xb87242d1, 0x19de7eae, 0x053e561a, 0x15ad6f8c,
- 0x66626c1c, 0x7154c24c, 0xea082b2a, 0x93eb2939, 0x17dcb0f0, 0x58d4f2ae, 0x9ea294fb, 0x52cf564c,
- 0x9883fe66, 0x2ec40581, 0x763953c3, 0x01d6692e, 0xd3a0c108, 0xa1e7160e, 0xe4f2dfa6, 0x693ed285,
- 0x74904698, 0x4c2b0edd, 0x4f757656, 0x5d393378, 0xa132234f, 0x3d321c5d, 0xc3f5e194, 0x4b269301,
- 0xc79f022f, 0x3c997e7e, 0x5e4f9504, 0x3ffafbbd, 0x76f7ad0e, 0x296693f4, 0x3d1fce6f, 0xc61e45be,
- 0xd3b5ab34, 0xf72bf9b7, 0x1b0434c0, 0x4e72b567, 0x5592a33d, 0xb5229301, 0xcfd2a87f, 0x60aeb767,
- 0x1814386b, 0x30bcc33d, 0x38a0c07d, 0xfd1606f2, 0xc363519b, 0x589dd390, 0x5479f8e6, 0x1cb8d647,
- 0x97fd61a9, 0xea7759f4, 0x2d57539d, 0x569a58cf, 0xe84e63ad, 0x462e1b78, 0x6580f87e, 0xf3817914,
- 0x91da55f4, 0x40a230f3, 0xd1988f35, 0xb6e318d2, 0x3ffa50bc, 0x3d40f021, 0xc3c0bdae, 0x4958c24c,
- 0x518f36b2, 0x84b1d370, 0x0fedce83, 0x878ddada, 0xf2a279c7, 0x94e01be8, 0x90716f4b, 0x954b8aa3,
- },
- {
- 0xe216300d, 0xbbddfffc, 0xa7ebdabd, 0x35648095, 0x7789f8b7, 0xe6c1121b, 0x0e241600, 0x052ce8b5,
- 0x11a9cfb0, 0xe5952f11, 0xece7990a, 0x9386d174, 0x2a42931c, 0x76e38111, 0xb12def3a, 0x37ddddfc,
- 0xde9adeb1, 0x0a0cc32c, 0xbe197029, 0x84a00940, 0xbb243a0f, 0xb4d137cf, 0xb44e79f0, 0x049eedfd,
- 0x0b15a15d, 0x480d3168, 0x8bbbde5a, 0x669ded42, 0xc7ece831, 0x3f8f95e7, 0x72df191b, 0x7580330d,
- 0x94074251, 0x5c7dcdfa, 0xabbe6d63, 0xaa402164, 0xb301d40a, 0x02e7d1ca, 0x53571dae, 0x7a3182a2,
- 0x12a8ddec, 0xfdaa335d, 0x176f43e8, 0x71fb46d4, 0x38129022, 0xce949ad4, 0xb84769ad, 0x965bd862,
- 0x82f3d055, 0x66fb9767, 0x15b80b4e, 0x1d5b47a0, 0x4cfde06f, 0xc28ec4b8, 0x57e8726e, 0x647a78fc,
- 0x99865d44, 0x608bd593, 0x6c200e03, 0x39dc5ff6, 0x5d0b00a3, 0xae63aff2, 0x7e8bd632, 0x70108c0c,
- 0xbbd35049, 0x2998df04, 0x980cf42a, 0x9b6df491, 0x9e7edd53, 0x06918548, 0x58cb7e07, 0x3b74ef2e,
- 0x522fffb1, 0xd24708cc, 0x1c7e27cd, 0xa4eb215b, 0x3cf1d2e2, 0x19b47a38, 0x424f7618, 0x35856039,
- 0x9d17dee7, 0x27eb35e6, 0xc9aff67b, 0x36baf5b8, 0x09c467cd, 0xc18910b1, 0xe11dbf7b, 0x06cd1af8,
- 0x7170c608, 0x2d5e3354, 0xd4de495a, 0x64c6d006, 0xbcc0c62c, 0x3dd00db3, 0x708f8f34, 0x77d51b42,
- 0x264f620f, 0x24b8d2bf, 0x15c1b79e, 0x46a52564, 0xf8d7e54e, 0x3e378160, 0x7895cda5, 0x859c15a5,
- 0xe6459788, 0xc37bc75f, 0xdb07ba0c, 0x0676a3ab, 0x7f229b1e, 0x31842e7b, 0x24259fd7, 0xf8bef472,
- 0x835ffcb8, 0x6df4c1f2, 0x96f5b195, 0xfd0af0fc, 0xb0fe134c, 0xe2506d3d, 0x4f9b12ea, 0xf215f225,
- 0xa223736f, 0x9fb4c428, 0x25d04979, 0x34c713f8, 0xc4618187, 0xea7a6e98, 0x7cd16efc, 0x1436876c,
- 0xf1544107, 0xbedeee14, 0x56e9af27, 0xa04aa441, 0x3cf7c899, 0x92ecbae6, 0xdd67016d, 0x151682eb,
- 0xa842eedf, 0xfdba60b4, 0xf1907b75, 0x20e3030f, 0x24d8c29e, 0xe139673b, 0xefa63fb8, 0x71873054,
- 0xb6f2cf3b, 0x9f326442, 0xcb15a4cc, 0xb01a4504, 0xf1e47d8d, 0x844a1be5, 0xbae7dfdc, 0x42cbda70,
- 0xcd7dae0a, 0x57e85b7a, 0xd53f5af6, 0x20cf4d8c, 0xcea4d428, 0x79d130a4, 0x3486ebfb, 0x33d3cddc,
- 0x77853b53, 0x37effcb5, 0xc5068778, 0xe580b3e6, 0x4e68b8f4, 0xc5c8b37e, 0x0d809ea2, 0x398feb7c,
- 0x132a4f94, 0x43b7950e, 0x2fee7d1c, 0x223613bd, 0xdd06caa2, 0x37df932b, 0xc4248289, 0xacf3ebc3,
- 0x5715f6b7, 0xef3478dd, 0xf267616f, 0xc148cbe4, 0x9052815e, 0x5e410fab, 0xb48a2465, 0x2eda7fa4,
- 0xe87b40e4, 0xe98ea084, 0x5889e9e1, 0xefd390fc, 0xdd07d35b, 0xdb485694, 0x38d7e5b2, 0x57720101,
- 0x730edebc, 0x5b643113, 0x94917e4f, 0x503c2fba, 0x646f1282, 0x7523d24a, 0xe0779695, 0xf9c17a8f,
- 0x7a5b2121, 0xd187b896, 0x29263a4d, 0xba510cdf, 0x81f47c9f, 0xad1163ed, 0xea7b5965, 0x1a00726e,
- 0x11403092, 0x00da6d77, 0x4a0cdd61, 0xad1f4603, 0x605bdfb0, 0x9eedc364, 0x22ebe6a8, 0xcee7d28a,
- 0xa0e736a0, 0x5564a6b9, 0x10853209, 0xc7eb8f37, 0x2de705ca, 0x8951570f, 0xdf09822b, 0xbd691a6c,
- 0xaa12e4f2, 0x87451c0f, 0xe0f6a27a, 0x3ada4819, 0x4cf1764f, 0x0d771c2b, 0x67cdb156, 0x350d8384,
- 0x5938fa0f, 0x42399ef3, 0x36997b07, 0x0e84093d, 0x4aa93e61, 0x8360d87b, 0x1fa98b0c, 0x1149382c,
- 0xe97625a5, 0x0614d1b7, 0x0e25244b, 0x0c768347, 0x589e8d82, 0x0d2059d1, 0xa466bb1e, 0xf8da0a82,
- 0x04f19130, 0xba6e4ec0, 0x99265164, 0x1ee7230d, 0x50b2ad80, 0xeaee6801, 0x8db2a283, 0xea8bf59e,
- },
-}
diff --git a/vendor/golang.org/x/crypto/internal/alias/alias.go b/vendor/golang.org/x/crypto/internal/alias/alias.go
deleted file mode 100644
index 69c17f8..0000000
--- a/vendor/golang.org/x/crypto/internal/alias/alias.go
+++ /dev/null
@@ -1,32 +0,0 @@
-// Copyright 2018 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !purego
-// +build !purego
-
-// Package alias implements memory aliasing tests.
-package alias
-
-import "unsafe"
-
-// AnyOverlap reports whether x and y share memory at any (not necessarily
-// corresponding) index. The memory beyond the slice length is ignored.
-func AnyOverlap(x, y []byte) bool {
- return len(x) > 0 && len(y) > 0 &&
- uintptr(unsafe.Pointer(&x[0])) <= uintptr(unsafe.Pointer(&y[len(y)-1])) &&
- uintptr(unsafe.Pointer(&y[0])) <= uintptr(unsafe.Pointer(&x[len(x)-1]))
-}
-
-// InexactOverlap reports whether x and y share memory at any non-corresponding
-// index. The memory beyond the slice length is ignored. Note that x and y can
-// have different lengths and still not have any inexact overlap.
-//
-// InexactOverlap can be used to implement the requirements of the crypto/cipher
-// AEAD, Block, BlockMode and Stream interfaces.
-func InexactOverlap(x, y []byte) bool {
- if len(x) == 0 || len(y) == 0 || &x[0] == &y[0] {
- return false
- }
- return AnyOverlap(x, y)
-}
diff --git a/vendor/golang.org/x/crypto/internal/alias/alias_purego.go b/vendor/golang.org/x/crypto/internal/alias/alias_purego.go
deleted file mode 100644
index 4775b0a..0000000
--- a/vendor/golang.org/x/crypto/internal/alias/alias_purego.go
+++ /dev/null
@@ -1,35 +0,0 @@
-// Copyright 2018 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build purego
-// +build purego
-
-// Package alias implements memory aliasing tests.
-package alias
-
-// This is the Google App Engine standard variant based on reflect
-// because the unsafe package and cgo are disallowed.
-
-import "reflect"
-
-// AnyOverlap reports whether x and y share memory at any (not necessarily
-// corresponding) index. The memory beyond the slice length is ignored.
-func AnyOverlap(x, y []byte) bool {
- return len(x) > 0 && len(y) > 0 &&
- reflect.ValueOf(&x[0]).Pointer() <= reflect.ValueOf(&y[len(y)-1]).Pointer() &&
- reflect.ValueOf(&y[0]).Pointer() <= reflect.ValueOf(&x[len(x)-1]).Pointer()
-}
-
-// InexactOverlap reports whether x and y share memory at any non-corresponding
-// index. The memory beyond the slice length is ignored. Note that x and y can
-// have different lengths and still not have any inexact overlap.
-//
-// InexactOverlap can be used to implement the requirements of the crypto/cipher
-// AEAD, Block, BlockMode and Stream interfaces.
-func InexactOverlap(x, y []byte) bool {
- if len(x) == 0 || len(y) == 0 || &x[0] == &y[0] {
- return false
- }
- return AnyOverlap(x, y)
-}
diff --git a/vendor/golang.org/x/crypto/pbkdf2/pbkdf2.go b/vendor/golang.org/x/crypto/pbkdf2/pbkdf2.go
deleted file mode 100644
index 904b57e..0000000
--- a/vendor/golang.org/x/crypto/pbkdf2/pbkdf2.go
+++ /dev/null
@@ -1,77 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-/*
-Package pbkdf2 implements the key derivation function PBKDF2 as defined in RFC
-2898 / PKCS #5 v2.0.
-
-A key derivation function is useful when encrypting data based on a password
-or any other not-fully-random data. It uses a pseudorandom function to derive
-a secure encryption key based on the password.
-
-While v2.0 of the standard defines only one pseudorandom function to use,
-HMAC-SHA1, the drafted v2.1 specification allows use of all five FIPS Approved
-Hash Functions SHA-1, SHA-224, SHA-256, SHA-384 and SHA-512 for HMAC. To
-choose, you can pass the `New` functions from the different SHA packages to
-pbkdf2.Key.
-*/
-package pbkdf2 // import "golang.org/x/crypto/pbkdf2"
-
-import (
- "crypto/hmac"
- "hash"
-)
-
-// Key derives a key from the password, salt and iteration count, returning a
-// []byte of length keylen that can be used as cryptographic key. The key is
-// derived based on the method described as PBKDF2 with the HMAC variant using
-// the supplied hash function.
-//
-// For example, to use a HMAC-SHA-1 based PBKDF2 key derivation function, you
-// can get a derived key for e.g. AES-256 (which needs a 32-byte key) by
-// doing:
-//
-// dk := pbkdf2.Key([]byte("some password"), salt, 4096, 32, sha1.New)
-//
-// Remember to get a good random salt. At least 8 bytes is recommended by the
-// RFC.
-//
-// Using a higher iteration count will increase the cost of an exhaustive
-// search but will also make derivation proportionally slower.
-func Key(password, salt []byte, iter, keyLen int, h func() hash.Hash) []byte {
- prf := hmac.New(h, password)
- hashLen := prf.Size()
- numBlocks := (keyLen + hashLen - 1) / hashLen
-
- var buf [4]byte
- dk := make([]byte, 0, numBlocks*hashLen)
- U := make([]byte, hashLen)
- for block := 1; block <= numBlocks; block++ {
- // N.B.: || means concatenation, ^ means XOR
- // for each block T_i = U_1 ^ U_2 ^ ... ^ U_iter
- // U_1 = PRF(password, salt || uint(i))
- prf.Reset()
- prf.Write(salt)
- buf[0] = byte(block >> 24)
- buf[1] = byte(block >> 16)
- buf[2] = byte(block >> 8)
- buf[3] = byte(block)
- prf.Write(buf[:4])
- dk = prf.Sum(dk)
- T := dk[len(dk)-hashLen:]
- copy(U, T)
-
- // U_n = PRF(password, U_(n-1))
- for n := 2; n <= iter; n++ {
- prf.Reset()
- prf.Write(U)
- U = U[:0]
- U = prf.Sum(U)
- for x := range U {
- T[x] ^= U[x]
- }
- }
- }
- return dk[:keyLen]
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/hsalsa20.go b/vendor/golang.org/x/crypto/salsa20/salsa/hsalsa20.go
deleted file mode 100644
index 4c96147..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/hsalsa20.go
+++ /dev/null
@@ -1,144 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package salsa provides low-level access to functions in the Salsa family.
-package salsa // import "golang.org/x/crypto/salsa20/salsa"
-
-// Sigma is the Salsa20 constant for 256-bit keys.
-var Sigma = [16]byte{'e', 'x', 'p', 'a', 'n', 'd', ' ', '3', '2', '-', 'b', 'y', 't', 'e', ' ', 'k'}
-
-// HSalsa20 applies the HSalsa20 core function to a 16-byte input in, 32-byte
-// key k, and 16-byte constant c, and puts the result into the 32-byte array
-// out.
-func HSalsa20(out *[32]byte, in *[16]byte, k *[32]byte, c *[16]byte) {
- x0 := uint32(c[0]) | uint32(c[1])<<8 | uint32(c[2])<<16 | uint32(c[3])<<24
- x1 := uint32(k[0]) | uint32(k[1])<<8 | uint32(k[2])<<16 | uint32(k[3])<<24
- x2 := uint32(k[4]) | uint32(k[5])<<8 | uint32(k[6])<<16 | uint32(k[7])<<24
- x3 := uint32(k[8]) | uint32(k[9])<<8 | uint32(k[10])<<16 | uint32(k[11])<<24
- x4 := uint32(k[12]) | uint32(k[13])<<8 | uint32(k[14])<<16 | uint32(k[15])<<24
- x5 := uint32(c[4]) | uint32(c[5])<<8 | uint32(c[6])<<16 | uint32(c[7])<<24
- x6 := uint32(in[0]) | uint32(in[1])<<8 | uint32(in[2])<<16 | uint32(in[3])<<24
- x7 := uint32(in[4]) | uint32(in[5])<<8 | uint32(in[6])<<16 | uint32(in[7])<<24
- x8 := uint32(in[8]) | uint32(in[9])<<8 | uint32(in[10])<<16 | uint32(in[11])<<24
- x9 := uint32(in[12]) | uint32(in[13])<<8 | uint32(in[14])<<16 | uint32(in[15])<<24
- x10 := uint32(c[8]) | uint32(c[9])<<8 | uint32(c[10])<<16 | uint32(c[11])<<24
- x11 := uint32(k[16]) | uint32(k[17])<<8 | uint32(k[18])<<16 | uint32(k[19])<<24
- x12 := uint32(k[20]) | uint32(k[21])<<8 | uint32(k[22])<<16 | uint32(k[23])<<24
- x13 := uint32(k[24]) | uint32(k[25])<<8 | uint32(k[26])<<16 | uint32(k[27])<<24
- x14 := uint32(k[28]) | uint32(k[29])<<8 | uint32(k[30])<<16 | uint32(k[31])<<24
- x15 := uint32(c[12]) | uint32(c[13])<<8 | uint32(c[14])<<16 | uint32(c[15])<<24
-
- for i := 0; i < 20; i += 2 {
- u := x0 + x12
- x4 ^= u<<7 | u>>(32-7)
- u = x4 + x0
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x4
- x12 ^= u<<13 | u>>(32-13)
- u = x12 + x8
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x1
- x9 ^= u<<7 | u>>(32-7)
- u = x9 + x5
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x9
- x1 ^= u<<13 | u>>(32-13)
- u = x1 + x13
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x6
- x14 ^= u<<7 | u>>(32-7)
- u = x14 + x10
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x14
- x6 ^= u<<13 | u>>(32-13)
- u = x6 + x2
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x11
- x3 ^= u<<7 | u>>(32-7)
- u = x3 + x15
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x3
- x11 ^= u<<13 | u>>(32-13)
- u = x11 + x7
- x15 ^= u<<18 | u>>(32-18)
-
- u = x0 + x3
- x1 ^= u<<7 | u>>(32-7)
- u = x1 + x0
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x1
- x3 ^= u<<13 | u>>(32-13)
- u = x3 + x2
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x4
- x6 ^= u<<7 | u>>(32-7)
- u = x6 + x5
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x6
- x4 ^= u<<13 | u>>(32-13)
- u = x4 + x7
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x9
- x11 ^= u<<7 | u>>(32-7)
- u = x11 + x10
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x11
- x9 ^= u<<13 | u>>(32-13)
- u = x9 + x8
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x14
- x12 ^= u<<7 | u>>(32-7)
- u = x12 + x15
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x12
- x14 ^= u<<13 | u>>(32-13)
- u = x14 + x13
- x15 ^= u<<18 | u>>(32-18)
- }
- out[0] = byte(x0)
- out[1] = byte(x0 >> 8)
- out[2] = byte(x0 >> 16)
- out[3] = byte(x0 >> 24)
-
- out[4] = byte(x5)
- out[5] = byte(x5 >> 8)
- out[6] = byte(x5 >> 16)
- out[7] = byte(x5 >> 24)
-
- out[8] = byte(x10)
- out[9] = byte(x10 >> 8)
- out[10] = byte(x10 >> 16)
- out[11] = byte(x10 >> 24)
-
- out[12] = byte(x15)
- out[13] = byte(x15 >> 8)
- out[14] = byte(x15 >> 16)
- out[15] = byte(x15 >> 24)
-
- out[16] = byte(x6)
- out[17] = byte(x6 >> 8)
- out[18] = byte(x6 >> 16)
- out[19] = byte(x6 >> 24)
-
- out[20] = byte(x7)
- out[21] = byte(x7 >> 8)
- out[22] = byte(x7 >> 16)
- out[23] = byte(x7 >> 24)
-
- out[24] = byte(x8)
- out[25] = byte(x8 >> 8)
- out[26] = byte(x8 >> 16)
- out[27] = byte(x8 >> 24)
-
- out[28] = byte(x9)
- out[29] = byte(x9 >> 8)
- out[30] = byte(x9 >> 16)
- out[31] = byte(x9 >> 24)
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/salsa208.go b/vendor/golang.org/x/crypto/salsa20/salsa/salsa208.go
deleted file mode 100644
index 9bfc092..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/salsa208.go
+++ /dev/null
@@ -1,199 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package salsa
-
-// Core208 applies the Salsa20/8 core function to the 64-byte array in and puts
-// the result into the 64-byte array out. The input and output may be the same array.
-func Core208(out *[64]byte, in *[64]byte) {
- j0 := uint32(in[0]) | uint32(in[1])<<8 | uint32(in[2])<<16 | uint32(in[3])<<24
- j1 := uint32(in[4]) | uint32(in[5])<<8 | uint32(in[6])<<16 | uint32(in[7])<<24
- j2 := uint32(in[8]) | uint32(in[9])<<8 | uint32(in[10])<<16 | uint32(in[11])<<24
- j3 := uint32(in[12]) | uint32(in[13])<<8 | uint32(in[14])<<16 | uint32(in[15])<<24
- j4 := uint32(in[16]) | uint32(in[17])<<8 | uint32(in[18])<<16 | uint32(in[19])<<24
- j5 := uint32(in[20]) | uint32(in[21])<<8 | uint32(in[22])<<16 | uint32(in[23])<<24
- j6 := uint32(in[24]) | uint32(in[25])<<8 | uint32(in[26])<<16 | uint32(in[27])<<24
- j7 := uint32(in[28]) | uint32(in[29])<<8 | uint32(in[30])<<16 | uint32(in[31])<<24
- j8 := uint32(in[32]) | uint32(in[33])<<8 | uint32(in[34])<<16 | uint32(in[35])<<24
- j9 := uint32(in[36]) | uint32(in[37])<<8 | uint32(in[38])<<16 | uint32(in[39])<<24
- j10 := uint32(in[40]) | uint32(in[41])<<8 | uint32(in[42])<<16 | uint32(in[43])<<24
- j11 := uint32(in[44]) | uint32(in[45])<<8 | uint32(in[46])<<16 | uint32(in[47])<<24
- j12 := uint32(in[48]) | uint32(in[49])<<8 | uint32(in[50])<<16 | uint32(in[51])<<24
- j13 := uint32(in[52]) | uint32(in[53])<<8 | uint32(in[54])<<16 | uint32(in[55])<<24
- j14 := uint32(in[56]) | uint32(in[57])<<8 | uint32(in[58])<<16 | uint32(in[59])<<24
- j15 := uint32(in[60]) | uint32(in[61])<<8 | uint32(in[62])<<16 | uint32(in[63])<<24
-
- x0, x1, x2, x3, x4, x5, x6, x7, x8 := j0, j1, j2, j3, j4, j5, j6, j7, j8
- x9, x10, x11, x12, x13, x14, x15 := j9, j10, j11, j12, j13, j14, j15
-
- for i := 0; i < 8; i += 2 {
- u := x0 + x12
- x4 ^= u<<7 | u>>(32-7)
- u = x4 + x0
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x4
- x12 ^= u<<13 | u>>(32-13)
- u = x12 + x8
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x1
- x9 ^= u<<7 | u>>(32-7)
- u = x9 + x5
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x9
- x1 ^= u<<13 | u>>(32-13)
- u = x1 + x13
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x6
- x14 ^= u<<7 | u>>(32-7)
- u = x14 + x10
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x14
- x6 ^= u<<13 | u>>(32-13)
- u = x6 + x2
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x11
- x3 ^= u<<7 | u>>(32-7)
- u = x3 + x15
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x3
- x11 ^= u<<13 | u>>(32-13)
- u = x11 + x7
- x15 ^= u<<18 | u>>(32-18)
-
- u = x0 + x3
- x1 ^= u<<7 | u>>(32-7)
- u = x1 + x0
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x1
- x3 ^= u<<13 | u>>(32-13)
- u = x3 + x2
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x4
- x6 ^= u<<7 | u>>(32-7)
- u = x6 + x5
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x6
- x4 ^= u<<13 | u>>(32-13)
- u = x4 + x7
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x9
- x11 ^= u<<7 | u>>(32-7)
- u = x11 + x10
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x11
- x9 ^= u<<13 | u>>(32-13)
- u = x9 + x8
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x14
- x12 ^= u<<7 | u>>(32-7)
- u = x12 + x15
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x12
- x14 ^= u<<13 | u>>(32-13)
- u = x14 + x13
- x15 ^= u<<18 | u>>(32-18)
- }
- x0 += j0
- x1 += j1
- x2 += j2
- x3 += j3
- x4 += j4
- x5 += j5
- x6 += j6
- x7 += j7
- x8 += j8
- x9 += j9
- x10 += j10
- x11 += j11
- x12 += j12
- x13 += j13
- x14 += j14
- x15 += j15
-
- out[0] = byte(x0)
- out[1] = byte(x0 >> 8)
- out[2] = byte(x0 >> 16)
- out[3] = byte(x0 >> 24)
-
- out[4] = byte(x1)
- out[5] = byte(x1 >> 8)
- out[6] = byte(x1 >> 16)
- out[7] = byte(x1 >> 24)
-
- out[8] = byte(x2)
- out[9] = byte(x2 >> 8)
- out[10] = byte(x2 >> 16)
- out[11] = byte(x2 >> 24)
-
- out[12] = byte(x3)
- out[13] = byte(x3 >> 8)
- out[14] = byte(x3 >> 16)
- out[15] = byte(x3 >> 24)
-
- out[16] = byte(x4)
- out[17] = byte(x4 >> 8)
- out[18] = byte(x4 >> 16)
- out[19] = byte(x4 >> 24)
-
- out[20] = byte(x5)
- out[21] = byte(x5 >> 8)
- out[22] = byte(x5 >> 16)
- out[23] = byte(x5 >> 24)
-
- out[24] = byte(x6)
- out[25] = byte(x6 >> 8)
- out[26] = byte(x6 >> 16)
- out[27] = byte(x6 >> 24)
-
- out[28] = byte(x7)
- out[29] = byte(x7 >> 8)
- out[30] = byte(x7 >> 16)
- out[31] = byte(x7 >> 24)
-
- out[32] = byte(x8)
- out[33] = byte(x8 >> 8)
- out[34] = byte(x8 >> 16)
- out[35] = byte(x8 >> 24)
-
- out[36] = byte(x9)
- out[37] = byte(x9 >> 8)
- out[38] = byte(x9 >> 16)
- out[39] = byte(x9 >> 24)
-
- out[40] = byte(x10)
- out[41] = byte(x10 >> 8)
- out[42] = byte(x10 >> 16)
- out[43] = byte(x10 >> 24)
-
- out[44] = byte(x11)
- out[45] = byte(x11 >> 8)
- out[46] = byte(x11 >> 16)
- out[47] = byte(x11 >> 24)
-
- out[48] = byte(x12)
- out[49] = byte(x12 >> 8)
- out[50] = byte(x12 >> 16)
- out[51] = byte(x12 >> 24)
-
- out[52] = byte(x13)
- out[53] = byte(x13 >> 8)
- out[54] = byte(x13 >> 16)
- out[55] = byte(x13 >> 24)
-
- out[56] = byte(x14)
- out[57] = byte(x14 >> 8)
- out[58] = byte(x14 >> 16)
- out[59] = byte(x14 >> 24)
-
- out[60] = byte(x15)
- out[61] = byte(x15 >> 8)
- out[62] = byte(x15 >> 16)
- out[63] = byte(x15 >> 24)
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.go b/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.go
deleted file mode 100644
index c400dfc..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.go
+++ /dev/null
@@ -1,24 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && !purego && gc
-// +build amd64,!purego,gc
-
-package salsa
-
-//go:noescape
-
-// salsa2020XORKeyStream is implemented in salsa20_amd64.s.
-func salsa2020XORKeyStream(out, in *byte, n uint64, nonce, key *byte)
-
-// XORKeyStream crypts bytes from in to out using the given key and counters.
-// In and out must overlap entirely or not at all. Counter
-// contains the raw salsa20 counter bytes (both nonce and block counter).
-func XORKeyStream(out, in []byte, counter *[16]byte, key *[32]byte) {
- if len(in) == 0 {
- return
- }
- _ = out[len(in)-1]
- salsa2020XORKeyStream(&out[0], &in[0], uint64(len(in)), &counter[0], &key[0])
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.s b/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.s
deleted file mode 100644
index c089277..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_amd64.s
+++ /dev/null
@@ -1,881 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && !purego && gc
-// +build amd64,!purego,gc
-
-// This code was translated into a form compatible with 6a from the public
-// domain sources in SUPERCOP: https://bench.cr.yp.to/supercop.html
-
-// func salsa2020XORKeyStream(out, in *byte, n uint64, nonce, key *byte)
-// This needs up to 64 bytes at 360(R12); hence the non-obvious frame size.
-TEXT ·salsa2020XORKeyStream(SB),0,$456-40 // frame = 424 + 32 byte alignment
- MOVQ out+0(FP),DI
- MOVQ in+8(FP),SI
- MOVQ n+16(FP),DX
- MOVQ nonce+24(FP),CX
- MOVQ key+32(FP),R8
-
- MOVQ SP,R12
- ADDQ $31, R12
- ANDQ $~31, R12
-
- MOVQ DX,R9
- MOVQ CX,DX
- MOVQ R8,R10
- CMPQ R9,$0
- JBE DONE
- START:
- MOVL 20(R10),CX
- MOVL 0(R10),R8
- MOVL 0(DX),AX
- MOVL 16(R10),R11
- MOVL CX,0(R12)
- MOVL R8, 4 (R12)
- MOVL AX, 8 (R12)
- MOVL R11, 12 (R12)
- MOVL 8(DX),CX
- MOVL 24(R10),R8
- MOVL 4(R10),AX
- MOVL 4(DX),R11
- MOVL CX,16(R12)
- MOVL R8, 20 (R12)
- MOVL AX, 24 (R12)
- MOVL R11, 28 (R12)
- MOVL 12(DX),CX
- MOVL 12(R10),DX
- MOVL 28(R10),R8
- MOVL 8(R10),AX
- MOVL DX,32(R12)
- MOVL CX, 36 (R12)
- MOVL R8, 40 (R12)
- MOVL AX, 44 (R12)
- MOVQ $1634760805,DX
- MOVQ $857760878,CX
- MOVQ $2036477234,R8
- MOVQ $1797285236,AX
- MOVL DX,48(R12)
- MOVL CX, 52 (R12)
- MOVL R8, 56 (R12)
- MOVL AX, 60 (R12)
- CMPQ R9,$256
- JB BYTESBETWEEN1AND255
- MOVOA 48(R12),X0
- PSHUFL $0X55,X0,X1
- PSHUFL $0XAA,X0,X2
- PSHUFL $0XFF,X0,X3
- PSHUFL $0X00,X0,X0
- MOVOA X1,64(R12)
- MOVOA X2,80(R12)
- MOVOA X3,96(R12)
- MOVOA X0,112(R12)
- MOVOA 0(R12),X0
- PSHUFL $0XAA,X0,X1
- PSHUFL $0XFF,X0,X2
- PSHUFL $0X00,X0,X3
- PSHUFL $0X55,X0,X0
- MOVOA X1,128(R12)
- MOVOA X2,144(R12)
- MOVOA X3,160(R12)
- MOVOA X0,176(R12)
- MOVOA 16(R12),X0
- PSHUFL $0XFF,X0,X1
- PSHUFL $0X55,X0,X2
- PSHUFL $0XAA,X0,X0
- MOVOA X1,192(R12)
- MOVOA X2,208(R12)
- MOVOA X0,224(R12)
- MOVOA 32(R12),X0
- PSHUFL $0X00,X0,X1
- PSHUFL $0XAA,X0,X2
- PSHUFL $0XFF,X0,X0
- MOVOA X1,240(R12)
- MOVOA X2,256(R12)
- MOVOA X0,272(R12)
- BYTESATLEAST256:
- MOVL 16(R12),DX
- MOVL 36 (R12),CX
- MOVL DX,288(R12)
- MOVL CX,304(R12)
- SHLQ $32,CX
- ADDQ CX,DX
- ADDQ $1,DX
- MOVQ DX,CX
- SHRQ $32,CX
- MOVL DX, 292 (R12)
- MOVL CX, 308 (R12)
- ADDQ $1,DX
- MOVQ DX,CX
- SHRQ $32,CX
- MOVL DX, 296 (R12)
- MOVL CX, 312 (R12)
- ADDQ $1,DX
- MOVQ DX,CX
- SHRQ $32,CX
- MOVL DX, 300 (R12)
- MOVL CX, 316 (R12)
- ADDQ $1,DX
- MOVQ DX,CX
- SHRQ $32,CX
- MOVL DX,16(R12)
- MOVL CX, 36 (R12)
- MOVQ R9,352(R12)
- MOVQ $20,DX
- MOVOA 64(R12),X0
- MOVOA 80(R12),X1
- MOVOA 96(R12),X2
- MOVOA 256(R12),X3
- MOVOA 272(R12),X4
- MOVOA 128(R12),X5
- MOVOA 144(R12),X6
- MOVOA 176(R12),X7
- MOVOA 192(R12),X8
- MOVOA 208(R12),X9
- MOVOA 224(R12),X10
- MOVOA 304(R12),X11
- MOVOA 112(R12),X12
- MOVOA 160(R12),X13
- MOVOA 240(R12),X14
- MOVOA 288(R12),X15
- MAINLOOP1:
- MOVOA X1,320(R12)
- MOVOA X2,336(R12)
- MOVOA X13,X1
- PADDL X12,X1
- MOVOA X1,X2
- PSLLL $7,X1
- PXOR X1,X14
- PSRLL $25,X2
- PXOR X2,X14
- MOVOA X7,X1
- PADDL X0,X1
- MOVOA X1,X2
- PSLLL $7,X1
- PXOR X1,X11
- PSRLL $25,X2
- PXOR X2,X11
- MOVOA X12,X1
- PADDL X14,X1
- MOVOA X1,X2
- PSLLL $9,X1
- PXOR X1,X15
- PSRLL $23,X2
- PXOR X2,X15
- MOVOA X0,X1
- PADDL X11,X1
- MOVOA X1,X2
- PSLLL $9,X1
- PXOR X1,X9
- PSRLL $23,X2
- PXOR X2,X9
- MOVOA X14,X1
- PADDL X15,X1
- MOVOA X1,X2
- PSLLL $13,X1
- PXOR X1,X13
- PSRLL $19,X2
- PXOR X2,X13
- MOVOA X11,X1
- PADDL X9,X1
- MOVOA X1,X2
- PSLLL $13,X1
- PXOR X1,X7
- PSRLL $19,X2
- PXOR X2,X7
- MOVOA X15,X1
- PADDL X13,X1
- MOVOA X1,X2
- PSLLL $18,X1
- PXOR X1,X12
- PSRLL $14,X2
- PXOR X2,X12
- MOVOA 320(R12),X1
- MOVOA X12,320(R12)
- MOVOA X9,X2
- PADDL X7,X2
- MOVOA X2,X12
- PSLLL $18,X2
- PXOR X2,X0
- PSRLL $14,X12
- PXOR X12,X0
- MOVOA X5,X2
- PADDL X1,X2
- MOVOA X2,X12
- PSLLL $7,X2
- PXOR X2,X3
- PSRLL $25,X12
- PXOR X12,X3
- MOVOA 336(R12),X2
- MOVOA X0,336(R12)
- MOVOA X6,X0
- PADDL X2,X0
- MOVOA X0,X12
- PSLLL $7,X0
- PXOR X0,X4
- PSRLL $25,X12
- PXOR X12,X4
- MOVOA X1,X0
- PADDL X3,X0
- MOVOA X0,X12
- PSLLL $9,X0
- PXOR X0,X10
- PSRLL $23,X12
- PXOR X12,X10
- MOVOA X2,X0
- PADDL X4,X0
- MOVOA X0,X12
- PSLLL $9,X0
- PXOR X0,X8
- PSRLL $23,X12
- PXOR X12,X8
- MOVOA X3,X0
- PADDL X10,X0
- MOVOA X0,X12
- PSLLL $13,X0
- PXOR X0,X5
- PSRLL $19,X12
- PXOR X12,X5
- MOVOA X4,X0
- PADDL X8,X0
- MOVOA X0,X12
- PSLLL $13,X0
- PXOR X0,X6
- PSRLL $19,X12
- PXOR X12,X6
- MOVOA X10,X0
- PADDL X5,X0
- MOVOA X0,X12
- PSLLL $18,X0
- PXOR X0,X1
- PSRLL $14,X12
- PXOR X12,X1
- MOVOA 320(R12),X0
- MOVOA X1,320(R12)
- MOVOA X4,X1
- PADDL X0,X1
- MOVOA X1,X12
- PSLLL $7,X1
- PXOR X1,X7
- PSRLL $25,X12
- PXOR X12,X7
- MOVOA X8,X1
- PADDL X6,X1
- MOVOA X1,X12
- PSLLL $18,X1
- PXOR X1,X2
- PSRLL $14,X12
- PXOR X12,X2
- MOVOA 336(R12),X12
- MOVOA X2,336(R12)
- MOVOA X14,X1
- PADDL X12,X1
- MOVOA X1,X2
- PSLLL $7,X1
- PXOR X1,X5
- PSRLL $25,X2
- PXOR X2,X5
- MOVOA X0,X1
- PADDL X7,X1
- MOVOA X1,X2
- PSLLL $9,X1
- PXOR X1,X10
- PSRLL $23,X2
- PXOR X2,X10
- MOVOA X12,X1
- PADDL X5,X1
- MOVOA X1,X2
- PSLLL $9,X1
- PXOR X1,X8
- PSRLL $23,X2
- PXOR X2,X8
- MOVOA X7,X1
- PADDL X10,X1
- MOVOA X1,X2
- PSLLL $13,X1
- PXOR X1,X4
- PSRLL $19,X2
- PXOR X2,X4
- MOVOA X5,X1
- PADDL X8,X1
- MOVOA X1,X2
- PSLLL $13,X1
- PXOR X1,X14
- PSRLL $19,X2
- PXOR X2,X14
- MOVOA X10,X1
- PADDL X4,X1
- MOVOA X1,X2
- PSLLL $18,X1
- PXOR X1,X0
- PSRLL $14,X2
- PXOR X2,X0
- MOVOA 320(R12),X1
- MOVOA X0,320(R12)
- MOVOA X8,X0
- PADDL X14,X0
- MOVOA X0,X2
- PSLLL $18,X0
- PXOR X0,X12
- PSRLL $14,X2
- PXOR X2,X12
- MOVOA X11,X0
- PADDL X1,X0
- MOVOA X0,X2
- PSLLL $7,X0
- PXOR X0,X6
- PSRLL $25,X2
- PXOR X2,X6
- MOVOA 336(R12),X2
- MOVOA X12,336(R12)
- MOVOA X3,X0
- PADDL X2,X0
- MOVOA X0,X12
- PSLLL $7,X0
- PXOR X0,X13
- PSRLL $25,X12
- PXOR X12,X13
- MOVOA X1,X0
- PADDL X6,X0
- MOVOA X0,X12
- PSLLL $9,X0
- PXOR X0,X15
- PSRLL $23,X12
- PXOR X12,X15
- MOVOA X2,X0
- PADDL X13,X0
- MOVOA X0,X12
- PSLLL $9,X0
- PXOR X0,X9
- PSRLL $23,X12
- PXOR X12,X9
- MOVOA X6,X0
- PADDL X15,X0
- MOVOA X0,X12
- PSLLL $13,X0
- PXOR X0,X11
- PSRLL $19,X12
- PXOR X12,X11
- MOVOA X13,X0
- PADDL X9,X0
- MOVOA X0,X12
- PSLLL $13,X0
- PXOR X0,X3
- PSRLL $19,X12
- PXOR X12,X3
- MOVOA X15,X0
- PADDL X11,X0
- MOVOA X0,X12
- PSLLL $18,X0
- PXOR X0,X1
- PSRLL $14,X12
- PXOR X12,X1
- MOVOA X9,X0
- PADDL X3,X0
- MOVOA X0,X12
- PSLLL $18,X0
- PXOR X0,X2
- PSRLL $14,X12
- PXOR X12,X2
- MOVOA 320(R12),X12
- MOVOA 336(R12),X0
- SUBQ $2,DX
- JA MAINLOOP1
- PADDL 112(R12),X12
- PADDL 176(R12),X7
- PADDL 224(R12),X10
- PADDL 272(R12),X4
- MOVD X12,DX
- MOVD X7,CX
- MOVD X10,R8
- MOVD X4,R9
- PSHUFL $0X39,X12,X12
- PSHUFL $0X39,X7,X7
- PSHUFL $0X39,X10,X10
- PSHUFL $0X39,X4,X4
- XORL 0(SI),DX
- XORL 4(SI),CX
- XORL 8(SI),R8
- XORL 12(SI),R9
- MOVL DX,0(DI)
- MOVL CX,4(DI)
- MOVL R8,8(DI)
- MOVL R9,12(DI)
- MOVD X12,DX
- MOVD X7,CX
- MOVD X10,R8
- MOVD X4,R9
- PSHUFL $0X39,X12,X12
- PSHUFL $0X39,X7,X7
- PSHUFL $0X39,X10,X10
- PSHUFL $0X39,X4,X4
- XORL 64(SI),DX
- XORL 68(SI),CX
- XORL 72(SI),R8
- XORL 76(SI),R9
- MOVL DX,64(DI)
- MOVL CX,68(DI)
- MOVL R8,72(DI)
- MOVL R9,76(DI)
- MOVD X12,DX
- MOVD X7,CX
- MOVD X10,R8
- MOVD X4,R9
- PSHUFL $0X39,X12,X12
- PSHUFL $0X39,X7,X7
- PSHUFL $0X39,X10,X10
- PSHUFL $0X39,X4,X4
- XORL 128(SI),DX
- XORL 132(SI),CX
- XORL 136(SI),R8
- XORL 140(SI),R9
- MOVL DX,128(DI)
- MOVL CX,132(DI)
- MOVL R8,136(DI)
- MOVL R9,140(DI)
- MOVD X12,DX
- MOVD X7,CX
- MOVD X10,R8
- MOVD X4,R9
- XORL 192(SI),DX
- XORL 196(SI),CX
- XORL 200(SI),R8
- XORL 204(SI),R9
- MOVL DX,192(DI)
- MOVL CX,196(DI)
- MOVL R8,200(DI)
- MOVL R9,204(DI)
- PADDL 240(R12),X14
- PADDL 64(R12),X0
- PADDL 128(R12),X5
- PADDL 192(R12),X8
- MOVD X14,DX
- MOVD X0,CX
- MOVD X5,R8
- MOVD X8,R9
- PSHUFL $0X39,X14,X14
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X5,X5
- PSHUFL $0X39,X8,X8
- XORL 16(SI),DX
- XORL 20(SI),CX
- XORL 24(SI),R8
- XORL 28(SI),R9
- MOVL DX,16(DI)
- MOVL CX,20(DI)
- MOVL R8,24(DI)
- MOVL R9,28(DI)
- MOVD X14,DX
- MOVD X0,CX
- MOVD X5,R8
- MOVD X8,R9
- PSHUFL $0X39,X14,X14
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X5,X5
- PSHUFL $0X39,X8,X8
- XORL 80(SI),DX
- XORL 84(SI),CX
- XORL 88(SI),R8
- XORL 92(SI),R9
- MOVL DX,80(DI)
- MOVL CX,84(DI)
- MOVL R8,88(DI)
- MOVL R9,92(DI)
- MOVD X14,DX
- MOVD X0,CX
- MOVD X5,R8
- MOVD X8,R9
- PSHUFL $0X39,X14,X14
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X5,X5
- PSHUFL $0X39,X8,X8
- XORL 144(SI),DX
- XORL 148(SI),CX
- XORL 152(SI),R8
- XORL 156(SI),R9
- MOVL DX,144(DI)
- MOVL CX,148(DI)
- MOVL R8,152(DI)
- MOVL R9,156(DI)
- MOVD X14,DX
- MOVD X0,CX
- MOVD X5,R8
- MOVD X8,R9
- XORL 208(SI),DX
- XORL 212(SI),CX
- XORL 216(SI),R8
- XORL 220(SI),R9
- MOVL DX,208(DI)
- MOVL CX,212(DI)
- MOVL R8,216(DI)
- MOVL R9,220(DI)
- PADDL 288(R12),X15
- PADDL 304(R12),X11
- PADDL 80(R12),X1
- PADDL 144(R12),X6
- MOVD X15,DX
- MOVD X11,CX
- MOVD X1,R8
- MOVD X6,R9
- PSHUFL $0X39,X15,X15
- PSHUFL $0X39,X11,X11
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X6,X6
- XORL 32(SI),DX
- XORL 36(SI),CX
- XORL 40(SI),R8
- XORL 44(SI),R9
- MOVL DX,32(DI)
- MOVL CX,36(DI)
- MOVL R8,40(DI)
- MOVL R9,44(DI)
- MOVD X15,DX
- MOVD X11,CX
- MOVD X1,R8
- MOVD X6,R9
- PSHUFL $0X39,X15,X15
- PSHUFL $0X39,X11,X11
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X6,X6
- XORL 96(SI),DX
- XORL 100(SI),CX
- XORL 104(SI),R8
- XORL 108(SI),R9
- MOVL DX,96(DI)
- MOVL CX,100(DI)
- MOVL R8,104(DI)
- MOVL R9,108(DI)
- MOVD X15,DX
- MOVD X11,CX
- MOVD X1,R8
- MOVD X6,R9
- PSHUFL $0X39,X15,X15
- PSHUFL $0X39,X11,X11
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X6,X6
- XORL 160(SI),DX
- XORL 164(SI),CX
- XORL 168(SI),R8
- XORL 172(SI),R9
- MOVL DX,160(DI)
- MOVL CX,164(DI)
- MOVL R8,168(DI)
- MOVL R9,172(DI)
- MOVD X15,DX
- MOVD X11,CX
- MOVD X1,R8
- MOVD X6,R9
- XORL 224(SI),DX
- XORL 228(SI),CX
- XORL 232(SI),R8
- XORL 236(SI),R9
- MOVL DX,224(DI)
- MOVL CX,228(DI)
- MOVL R8,232(DI)
- MOVL R9,236(DI)
- PADDL 160(R12),X13
- PADDL 208(R12),X9
- PADDL 256(R12),X3
- PADDL 96(R12),X2
- MOVD X13,DX
- MOVD X9,CX
- MOVD X3,R8
- MOVD X2,R9
- PSHUFL $0X39,X13,X13
- PSHUFL $0X39,X9,X9
- PSHUFL $0X39,X3,X3
- PSHUFL $0X39,X2,X2
- XORL 48(SI),DX
- XORL 52(SI),CX
- XORL 56(SI),R8
- XORL 60(SI),R9
- MOVL DX,48(DI)
- MOVL CX,52(DI)
- MOVL R8,56(DI)
- MOVL R9,60(DI)
- MOVD X13,DX
- MOVD X9,CX
- MOVD X3,R8
- MOVD X2,R9
- PSHUFL $0X39,X13,X13
- PSHUFL $0X39,X9,X9
- PSHUFL $0X39,X3,X3
- PSHUFL $0X39,X2,X2
- XORL 112(SI),DX
- XORL 116(SI),CX
- XORL 120(SI),R8
- XORL 124(SI),R9
- MOVL DX,112(DI)
- MOVL CX,116(DI)
- MOVL R8,120(DI)
- MOVL R9,124(DI)
- MOVD X13,DX
- MOVD X9,CX
- MOVD X3,R8
- MOVD X2,R9
- PSHUFL $0X39,X13,X13
- PSHUFL $0X39,X9,X9
- PSHUFL $0X39,X3,X3
- PSHUFL $0X39,X2,X2
- XORL 176(SI),DX
- XORL 180(SI),CX
- XORL 184(SI),R8
- XORL 188(SI),R9
- MOVL DX,176(DI)
- MOVL CX,180(DI)
- MOVL R8,184(DI)
- MOVL R9,188(DI)
- MOVD X13,DX
- MOVD X9,CX
- MOVD X3,R8
- MOVD X2,R9
- XORL 240(SI),DX
- XORL 244(SI),CX
- XORL 248(SI),R8
- XORL 252(SI),R9
- MOVL DX,240(DI)
- MOVL CX,244(DI)
- MOVL R8,248(DI)
- MOVL R9,252(DI)
- MOVQ 352(R12),R9
- SUBQ $256,R9
- ADDQ $256,SI
- ADDQ $256,DI
- CMPQ R9,$256
- JAE BYTESATLEAST256
- CMPQ R9,$0
- JBE DONE
- BYTESBETWEEN1AND255:
- CMPQ R9,$64
- JAE NOCOPY
- MOVQ DI,DX
- LEAQ 360(R12),DI
- MOVQ R9,CX
- REP; MOVSB
- LEAQ 360(R12),DI
- LEAQ 360(R12),SI
- NOCOPY:
- MOVQ R9,352(R12)
- MOVOA 48(R12),X0
- MOVOA 0(R12),X1
- MOVOA 16(R12),X2
- MOVOA 32(R12),X3
- MOVOA X1,X4
- MOVQ $20,CX
- MAINLOOP2:
- PADDL X0,X4
- MOVOA X0,X5
- MOVOA X4,X6
- PSLLL $7,X4
- PSRLL $25,X6
- PXOR X4,X3
- PXOR X6,X3
- PADDL X3,X5
- MOVOA X3,X4
- MOVOA X5,X6
- PSLLL $9,X5
- PSRLL $23,X6
- PXOR X5,X2
- PSHUFL $0X93,X3,X3
- PXOR X6,X2
- PADDL X2,X4
- MOVOA X2,X5
- MOVOA X4,X6
- PSLLL $13,X4
- PSRLL $19,X6
- PXOR X4,X1
- PSHUFL $0X4E,X2,X2
- PXOR X6,X1
- PADDL X1,X5
- MOVOA X3,X4
- MOVOA X5,X6
- PSLLL $18,X5
- PSRLL $14,X6
- PXOR X5,X0
- PSHUFL $0X39,X1,X1
- PXOR X6,X0
- PADDL X0,X4
- MOVOA X0,X5
- MOVOA X4,X6
- PSLLL $7,X4
- PSRLL $25,X6
- PXOR X4,X1
- PXOR X6,X1
- PADDL X1,X5
- MOVOA X1,X4
- MOVOA X5,X6
- PSLLL $9,X5
- PSRLL $23,X6
- PXOR X5,X2
- PSHUFL $0X93,X1,X1
- PXOR X6,X2
- PADDL X2,X4
- MOVOA X2,X5
- MOVOA X4,X6
- PSLLL $13,X4
- PSRLL $19,X6
- PXOR X4,X3
- PSHUFL $0X4E,X2,X2
- PXOR X6,X3
- PADDL X3,X5
- MOVOA X1,X4
- MOVOA X5,X6
- PSLLL $18,X5
- PSRLL $14,X6
- PXOR X5,X0
- PSHUFL $0X39,X3,X3
- PXOR X6,X0
- PADDL X0,X4
- MOVOA X0,X5
- MOVOA X4,X6
- PSLLL $7,X4
- PSRLL $25,X6
- PXOR X4,X3
- PXOR X6,X3
- PADDL X3,X5
- MOVOA X3,X4
- MOVOA X5,X6
- PSLLL $9,X5
- PSRLL $23,X6
- PXOR X5,X2
- PSHUFL $0X93,X3,X3
- PXOR X6,X2
- PADDL X2,X4
- MOVOA X2,X5
- MOVOA X4,X6
- PSLLL $13,X4
- PSRLL $19,X6
- PXOR X4,X1
- PSHUFL $0X4E,X2,X2
- PXOR X6,X1
- PADDL X1,X5
- MOVOA X3,X4
- MOVOA X5,X6
- PSLLL $18,X5
- PSRLL $14,X6
- PXOR X5,X0
- PSHUFL $0X39,X1,X1
- PXOR X6,X0
- PADDL X0,X4
- MOVOA X0,X5
- MOVOA X4,X6
- PSLLL $7,X4
- PSRLL $25,X6
- PXOR X4,X1
- PXOR X6,X1
- PADDL X1,X5
- MOVOA X1,X4
- MOVOA X5,X6
- PSLLL $9,X5
- PSRLL $23,X6
- PXOR X5,X2
- PSHUFL $0X93,X1,X1
- PXOR X6,X2
- PADDL X2,X4
- MOVOA X2,X5
- MOVOA X4,X6
- PSLLL $13,X4
- PSRLL $19,X6
- PXOR X4,X3
- PSHUFL $0X4E,X2,X2
- PXOR X6,X3
- SUBQ $4,CX
- PADDL X3,X5
- MOVOA X1,X4
- MOVOA X5,X6
- PSLLL $18,X5
- PXOR X7,X7
- PSRLL $14,X6
- PXOR X5,X0
- PSHUFL $0X39,X3,X3
- PXOR X6,X0
- JA MAINLOOP2
- PADDL 48(R12),X0
- PADDL 0(R12),X1
- PADDL 16(R12),X2
- PADDL 32(R12),X3
- MOVD X0,CX
- MOVD X1,R8
- MOVD X2,R9
- MOVD X3,AX
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X2,X2
- PSHUFL $0X39,X3,X3
- XORL 0(SI),CX
- XORL 48(SI),R8
- XORL 32(SI),R9
- XORL 16(SI),AX
- MOVL CX,0(DI)
- MOVL R8,48(DI)
- MOVL R9,32(DI)
- MOVL AX,16(DI)
- MOVD X0,CX
- MOVD X1,R8
- MOVD X2,R9
- MOVD X3,AX
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X2,X2
- PSHUFL $0X39,X3,X3
- XORL 20(SI),CX
- XORL 4(SI),R8
- XORL 52(SI),R9
- XORL 36(SI),AX
- MOVL CX,20(DI)
- MOVL R8,4(DI)
- MOVL R9,52(DI)
- MOVL AX,36(DI)
- MOVD X0,CX
- MOVD X1,R8
- MOVD X2,R9
- MOVD X3,AX
- PSHUFL $0X39,X0,X0
- PSHUFL $0X39,X1,X1
- PSHUFL $0X39,X2,X2
- PSHUFL $0X39,X3,X3
- XORL 40(SI),CX
- XORL 24(SI),R8
- XORL 8(SI),R9
- XORL 56(SI),AX
- MOVL CX,40(DI)
- MOVL R8,24(DI)
- MOVL R9,8(DI)
- MOVL AX,56(DI)
- MOVD X0,CX
- MOVD X1,R8
- MOVD X2,R9
- MOVD X3,AX
- XORL 60(SI),CX
- XORL 44(SI),R8
- XORL 28(SI),R9
- XORL 12(SI),AX
- MOVL CX,60(DI)
- MOVL R8,44(DI)
- MOVL R9,28(DI)
- MOVL AX,12(DI)
- MOVQ 352(R12),R9
- MOVL 16(R12),CX
- MOVL 36 (R12),R8
- ADDQ $1,CX
- SHLQ $32,R8
- ADDQ R8,CX
- MOVQ CX,R8
- SHRQ $32,R8
- MOVL CX,16(R12)
- MOVL R8, 36 (R12)
- CMPQ R9,$64
- JA BYTESATLEAST65
- JAE BYTESATLEAST64
- MOVQ DI,SI
- MOVQ DX,DI
- MOVQ R9,CX
- REP; MOVSB
- BYTESATLEAST64:
- DONE:
- RET
- BYTESATLEAST65:
- SUBQ $64,R9
- ADDQ $64,DI
- ADDQ $64,SI
- JMP BYTESBETWEEN1AND255
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_noasm.go b/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_noasm.go
deleted file mode 100644
index 4392cc1..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_noasm.go
+++ /dev/null
@@ -1,15 +0,0 @@
-// Copyright 2019 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !amd64 || purego || !gc
-// +build !amd64 purego !gc
-
-package salsa
-
-// XORKeyStream crypts bytes from in to out using the given key and counters.
-// In and out must overlap entirely or not at all. Counter
-// contains the raw salsa20 counter bytes (both nonce and block counter).
-func XORKeyStream(out, in []byte, counter *[16]byte, key *[32]byte) {
- genericXORKeyStream(out, in, counter, key)
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_ref.go b/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_ref.go
deleted file mode 100644
index 68169c6..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa/salsa20_ref.go
+++ /dev/null
@@ -1,231 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package salsa
-
-const rounds = 20
-
-// core applies the Salsa20 core function to 16-byte input in, 32-byte key k,
-// and 16-byte constant c, and puts the result into 64-byte array out.
-func core(out *[64]byte, in *[16]byte, k *[32]byte, c *[16]byte) {
- j0 := uint32(c[0]) | uint32(c[1])<<8 | uint32(c[2])<<16 | uint32(c[3])<<24
- j1 := uint32(k[0]) | uint32(k[1])<<8 | uint32(k[2])<<16 | uint32(k[3])<<24
- j2 := uint32(k[4]) | uint32(k[5])<<8 | uint32(k[6])<<16 | uint32(k[7])<<24
- j3 := uint32(k[8]) | uint32(k[9])<<8 | uint32(k[10])<<16 | uint32(k[11])<<24
- j4 := uint32(k[12]) | uint32(k[13])<<8 | uint32(k[14])<<16 | uint32(k[15])<<24
- j5 := uint32(c[4]) | uint32(c[5])<<8 | uint32(c[6])<<16 | uint32(c[7])<<24
- j6 := uint32(in[0]) | uint32(in[1])<<8 | uint32(in[2])<<16 | uint32(in[3])<<24
- j7 := uint32(in[4]) | uint32(in[5])<<8 | uint32(in[6])<<16 | uint32(in[7])<<24
- j8 := uint32(in[8]) | uint32(in[9])<<8 | uint32(in[10])<<16 | uint32(in[11])<<24
- j9 := uint32(in[12]) | uint32(in[13])<<8 | uint32(in[14])<<16 | uint32(in[15])<<24
- j10 := uint32(c[8]) | uint32(c[9])<<8 | uint32(c[10])<<16 | uint32(c[11])<<24
- j11 := uint32(k[16]) | uint32(k[17])<<8 | uint32(k[18])<<16 | uint32(k[19])<<24
- j12 := uint32(k[20]) | uint32(k[21])<<8 | uint32(k[22])<<16 | uint32(k[23])<<24
- j13 := uint32(k[24]) | uint32(k[25])<<8 | uint32(k[26])<<16 | uint32(k[27])<<24
- j14 := uint32(k[28]) | uint32(k[29])<<8 | uint32(k[30])<<16 | uint32(k[31])<<24
- j15 := uint32(c[12]) | uint32(c[13])<<8 | uint32(c[14])<<16 | uint32(c[15])<<24
-
- x0, x1, x2, x3, x4, x5, x6, x7, x8 := j0, j1, j2, j3, j4, j5, j6, j7, j8
- x9, x10, x11, x12, x13, x14, x15 := j9, j10, j11, j12, j13, j14, j15
-
- for i := 0; i < rounds; i += 2 {
- u := x0 + x12
- x4 ^= u<<7 | u>>(32-7)
- u = x4 + x0
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x4
- x12 ^= u<<13 | u>>(32-13)
- u = x12 + x8
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x1
- x9 ^= u<<7 | u>>(32-7)
- u = x9 + x5
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x9
- x1 ^= u<<13 | u>>(32-13)
- u = x1 + x13
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x6
- x14 ^= u<<7 | u>>(32-7)
- u = x14 + x10
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x14
- x6 ^= u<<13 | u>>(32-13)
- u = x6 + x2
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x11
- x3 ^= u<<7 | u>>(32-7)
- u = x3 + x15
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x3
- x11 ^= u<<13 | u>>(32-13)
- u = x11 + x7
- x15 ^= u<<18 | u>>(32-18)
-
- u = x0 + x3
- x1 ^= u<<7 | u>>(32-7)
- u = x1 + x0
- x2 ^= u<<9 | u>>(32-9)
- u = x2 + x1
- x3 ^= u<<13 | u>>(32-13)
- u = x3 + x2
- x0 ^= u<<18 | u>>(32-18)
-
- u = x5 + x4
- x6 ^= u<<7 | u>>(32-7)
- u = x6 + x5
- x7 ^= u<<9 | u>>(32-9)
- u = x7 + x6
- x4 ^= u<<13 | u>>(32-13)
- u = x4 + x7
- x5 ^= u<<18 | u>>(32-18)
-
- u = x10 + x9
- x11 ^= u<<7 | u>>(32-7)
- u = x11 + x10
- x8 ^= u<<9 | u>>(32-9)
- u = x8 + x11
- x9 ^= u<<13 | u>>(32-13)
- u = x9 + x8
- x10 ^= u<<18 | u>>(32-18)
-
- u = x15 + x14
- x12 ^= u<<7 | u>>(32-7)
- u = x12 + x15
- x13 ^= u<<9 | u>>(32-9)
- u = x13 + x12
- x14 ^= u<<13 | u>>(32-13)
- u = x14 + x13
- x15 ^= u<<18 | u>>(32-18)
- }
- x0 += j0
- x1 += j1
- x2 += j2
- x3 += j3
- x4 += j4
- x5 += j5
- x6 += j6
- x7 += j7
- x8 += j8
- x9 += j9
- x10 += j10
- x11 += j11
- x12 += j12
- x13 += j13
- x14 += j14
- x15 += j15
-
- out[0] = byte(x0)
- out[1] = byte(x0 >> 8)
- out[2] = byte(x0 >> 16)
- out[3] = byte(x0 >> 24)
-
- out[4] = byte(x1)
- out[5] = byte(x1 >> 8)
- out[6] = byte(x1 >> 16)
- out[7] = byte(x1 >> 24)
-
- out[8] = byte(x2)
- out[9] = byte(x2 >> 8)
- out[10] = byte(x2 >> 16)
- out[11] = byte(x2 >> 24)
-
- out[12] = byte(x3)
- out[13] = byte(x3 >> 8)
- out[14] = byte(x3 >> 16)
- out[15] = byte(x3 >> 24)
-
- out[16] = byte(x4)
- out[17] = byte(x4 >> 8)
- out[18] = byte(x4 >> 16)
- out[19] = byte(x4 >> 24)
-
- out[20] = byte(x5)
- out[21] = byte(x5 >> 8)
- out[22] = byte(x5 >> 16)
- out[23] = byte(x5 >> 24)
-
- out[24] = byte(x6)
- out[25] = byte(x6 >> 8)
- out[26] = byte(x6 >> 16)
- out[27] = byte(x6 >> 24)
-
- out[28] = byte(x7)
- out[29] = byte(x7 >> 8)
- out[30] = byte(x7 >> 16)
- out[31] = byte(x7 >> 24)
-
- out[32] = byte(x8)
- out[33] = byte(x8 >> 8)
- out[34] = byte(x8 >> 16)
- out[35] = byte(x8 >> 24)
-
- out[36] = byte(x9)
- out[37] = byte(x9 >> 8)
- out[38] = byte(x9 >> 16)
- out[39] = byte(x9 >> 24)
-
- out[40] = byte(x10)
- out[41] = byte(x10 >> 8)
- out[42] = byte(x10 >> 16)
- out[43] = byte(x10 >> 24)
-
- out[44] = byte(x11)
- out[45] = byte(x11 >> 8)
- out[46] = byte(x11 >> 16)
- out[47] = byte(x11 >> 24)
-
- out[48] = byte(x12)
- out[49] = byte(x12 >> 8)
- out[50] = byte(x12 >> 16)
- out[51] = byte(x12 >> 24)
-
- out[52] = byte(x13)
- out[53] = byte(x13 >> 8)
- out[54] = byte(x13 >> 16)
- out[55] = byte(x13 >> 24)
-
- out[56] = byte(x14)
- out[57] = byte(x14 >> 8)
- out[58] = byte(x14 >> 16)
- out[59] = byte(x14 >> 24)
-
- out[60] = byte(x15)
- out[61] = byte(x15 >> 8)
- out[62] = byte(x15 >> 16)
- out[63] = byte(x15 >> 24)
-}
-
-// genericXORKeyStream is the generic implementation of XORKeyStream to be used
-// when no assembly implementation is available.
-func genericXORKeyStream(out, in []byte, counter *[16]byte, key *[32]byte) {
- var block [64]byte
- var counterCopy [16]byte
- copy(counterCopy[:], counter[:])
-
- for len(in) >= 64 {
- core(&block, &counterCopy, key, &Sigma)
- for i, x := range block {
- out[i] = in[i] ^ x
- }
- u := uint32(1)
- for i := 8; i < 16; i++ {
- u += uint32(counterCopy[i])
- counterCopy[i] = byte(u)
- u >>= 8
- }
- in = in[64:]
- out = out[64:]
- }
-
- if len(in) > 0 {
- core(&block, &counterCopy, key, &Sigma)
- for i, v := range in {
- out[i] = v ^ block[i]
- }
- }
-}
diff --git a/vendor/golang.org/x/crypto/salsa20/salsa20.go b/vendor/golang.org/x/crypto/salsa20/salsa20.go
deleted file mode 100644
index 8f4f896..0000000
--- a/vendor/golang.org/x/crypto/salsa20/salsa20.go
+++ /dev/null
@@ -1,58 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-/*
-Package salsa20 implements the Salsa20 stream cipher as specified in https://cr.yp.to/snuffle/spec.pdf.
-
-Salsa20 differs from many other stream ciphers in that it is message orientated
-rather than byte orientated. Keystream blocks are not preserved between calls,
-therefore each side must encrypt/decrypt data with the same segmentation.
-
-Another aspect of this difference is that part of the counter is exposed as
-a nonce in each call. Encrypting two different messages with the same (key,
-nonce) pair leads to trivial plaintext recovery. This is analogous to
-encrypting two different messages with the same key with a traditional stream
-cipher.
-
-This package also implements XSalsa20: a version of Salsa20 with a 24-byte
-nonce as specified in https://cr.yp.to/snuffle/xsalsa-20081128.pdf. Simply
-passing a 24-byte slice as the nonce triggers XSalsa20.
-*/
-package salsa20 // import "golang.org/x/crypto/salsa20"
-
-// TODO(agl): implement XORKeyStream12 and XORKeyStream8 - the reduced round variants of Salsa20.
-
-import (
- "golang.org/x/crypto/internal/alias"
- "golang.org/x/crypto/salsa20/salsa"
-)
-
-// XORKeyStream crypts bytes from in to out using the given key and nonce.
-// In and out must overlap entirely or not at all. Nonce must
-// be either 8 or 24 bytes long.
-func XORKeyStream(out, in []byte, nonce []byte, key *[32]byte) {
- if len(out) < len(in) {
- panic("salsa20: output smaller than input")
- }
- if alias.InexactOverlap(out[:len(in)], in) {
- panic("salsa20: invalid buffer overlap")
- }
-
- var subNonce [16]byte
-
- if len(nonce) == 24 {
- var subKey [32]byte
- var hNonce [16]byte
- copy(hNonce[:], nonce[:16])
- salsa.HSalsa20(&subKey, &hNonce, key, &salsa.Sigma)
- copy(subNonce[:], nonce[16:])
- key = &subKey
- } else if len(nonce) == 8 {
- copy(subNonce[:], nonce[:])
- } else {
- panic("salsa20: nonce must be 8 or 24 bytes")
- }
-
- salsa.XORKeyStream(out, in, &subNonce, key)
-}
diff --git a/vendor/golang.org/x/crypto/scrypt/scrypt.go b/vendor/golang.org/x/crypto/scrypt/scrypt.go
deleted file mode 100644
index c971a99..0000000
--- a/vendor/golang.org/x/crypto/scrypt/scrypt.go
+++ /dev/null
@@ -1,212 +0,0 @@
-// Copyright 2012 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package scrypt implements the scrypt key derivation function as defined in
-// Colin Percival's paper "Stronger Key Derivation via Sequential Memory-Hard
-// Functions" (https://www.tarsnap.com/scrypt/scrypt.pdf).
-package scrypt // import "golang.org/x/crypto/scrypt"
-
-import (
- "crypto/sha256"
- "encoding/binary"
- "errors"
- "math/bits"
-
- "golang.org/x/crypto/pbkdf2"
-)
-
-const maxInt = int(^uint(0) >> 1)
-
-// blockCopy copies n numbers from src into dst.
-func blockCopy(dst, src []uint32, n int) {
- copy(dst, src[:n])
-}
-
-// blockXOR XORs numbers from dst with n numbers from src.
-func blockXOR(dst, src []uint32, n int) {
- for i, v := range src[:n] {
- dst[i] ^= v
- }
-}
-
-// salsaXOR applies Salsa20/8 to the XOR of 16 numbers from tmp and in,
-// and puts the result into both tmp and out.
-func salsaXOR(tmp *[16]uint32, in, out []uint32) {
- w0 := tmp[0] ^ in[0]
- w1 := tmp[1] ^ in[1]
- w2 := tmp[2] ^ in[2]
- w3 := tmp[3] ^ in[3]
- w4 := tmp[4] ^ in[4]
- w5 := tmp[5] ^ in[5]
- w6 := tmp[6] ^ in[6]
- w7 := tmp[7] ^ in[7]
- w8 := tmp[8] ^ in[8]
- w9 := tmp[9] ^ in[9]
- w10 := tmp[10] ^ in[10]
- w11 := tmp[11] ^ in[11]
- w12 := tmp[12] ^ in[12]
- w13 := tmp[13] ^ in[13]
- w14 := tmp[14] ^ in[14]
- w15 := tmp[15] ^ in[15]
-
- x0, x1, x2, x3, x4, x5, x6, x7, x8 := w0, w1, w2, w3, w4, w5, w6, w7, w8
- x9, x10, x11, x12, x13, x14, x15 := w9, w10, w11, w12, w13, w14, w15
-
- for i := 0; i < 8; i += 2 {
- x4 ^= bits.RotateLeft32(x0+x12, 7)
- x8 ^= bits.RotateLeft32(x4+x0, 9)
- x12 ^= bits.RotateLeft32(x8+x4, 13)
- x0 ^= bits.RotateLeft32(x12+x8, 18)
-
- x9 ^= bits.RotateLeft32(x5+x1, 7)
- x13 ^= bits.RotateLeft32(x9+x5, 9)
- x1 ^= bits.RotateLeft32(x13+x9, 13)
- x5 ^= bits.RotateLeft32(x1+x13, 18)
-
- x14 ^= bits.RotateLeft32(x10+x6, 7)
- x2 ^= bits.RotateLeft32(x14+x10, 9)
- x6 ^= bits.RotateLeft32(x2+x14, 13)
- x10 ^= bits.RotateLeft32(x6+x2, 18)
-
- x3 ^= bits.RotateLeft32(x15+x11, 7)
- x7 ^= bits.RotateLeft32(x3+x15, 9)
- x11 ^= bits.RotateLeft32(x7+x3, 13)
- x15 ^= bits.RotateLeft32(x11+x7, 18)
-
- x1 ^= bits.RotateLeft32(x0+x3, 7)
- x2 ^= bits.RotateLeft32(x1+x0, 9)
- x3 ^= bits.RotateLeft32(x2+x1, 13)
- x0 ^= bits.RotateLeft32(x3+x2, 18)
-
- x6 ^= bits.RotateLeft32(x5+x4, 7)
- x7 ^= bits.RotateLeft32(x6+x5, 9)
- x4 ^= bits.RotateLeft32(x7+x6, 13)
- x5 ^= bits.RotateLeft32(x4+x7, 18)
-
- x11 ^= bits.RotateLeft32(x10+x9, 7)
- x8 ^= bits.RotateLeft32(x11+x10, 9)
- x9 ^= bits.RotateLeft32(x8+x11, 13)
- x10 ^= bits.RotateLeft32(x9+x8, 18)
-
- x12 ^= bits.RotateLeft32(x15+x14, 7)
- x13 ^= bits.RotateLeft32(x12+x15, 9)
- x14 ^= bits.RotateLeft32(x13+x12, 13)
- x15 ^= bits.RotateLeft32(x14+x13, 18)
- }
- x0 += w0
- x1 += w1
- x2 += w2
- x3 += w3
- x4 += w4
- x5 += w5
- x6 += w6
- x7 += w7
- x8 += w8
- x9 += w9
- x10 += w10
- x11 += w11
- x12 += w12
- x13 += w13
- x14 += w14
- x15 += w15
-
- out[0], tmp[0] = x0, x0
- out[1], tmp[1] = x1, x1
- out[2], tmp[2] = x2, x2
- out[3], tmp[3] = x3, x3
- out[4], tmp[4] = x4, x4
- out[5], tmp[5] = x5, x5
- out[6], tmp[6] = x6, x6
- out[7], tmp[7] = x7, x7
- out[8], tmp[8] = x8, x8
- out[9], tmp[9] = x9, x9
- out[10], tmp[10] = x10, x10
- out[11], tmp[11] = x11, x11
- out[12], tmp[12] = x12, x12
- out[13], tmp[13] = x13, x13
- out[14], tmp[14] = x14, x14
- out[15], tmp[15] = x15, x15
-}
-
-func blockMix(tmp *[16]uint32, in, out []uint32, r int) {
- blockCopy(tmp[:], in[(2*r-1)*16:], 16)
- for i := 0; i < 2*r; i += 2 {
- salsaXOR(tmp, in[i*16:], out[i*8:])
- salsaXOR(tmp, in[i*16+16:], out[i*8+r*16:])
- }
-}
-
-func integer(b []uint32, r int) uint64 {
- j := (2*r - 1) * 16
- return uint64(b[j]) | uint64(b[j+1])<<32
-}
-
-func smix(b []byte, r, N int, v, xy []uint32) {
- var tmp [16]uint32
- R := 32 * r
- x := xy
- y := xy[R:]
-
- j := 0
- for i := 0; i < R; i++ {
- x[i] = binary.LittleEndian.Uint32(b[j:])
- j += 4
- }
- for i := 0; i < N; i += 2 {
- blockCopy(v[i*R:], x, R)
- blockMix(&tmp, x, y, r)
-
- blockCopy(v[(i+1)*R:], y, R)
- blockMix(&tmp, y, x, r)
- }
- for i := 0; i < N; i += 2 {
- j := int(integer(x, r) & uint64(N-1))
- blockXOR(x, v[j*R:], R)
- blockMix(&tmp, x, y, r)
-
- j = int(integer(y, r) & uint64(N-1))
- blockXOR(y, v[j*R:], R)
- blockMix(&tmp, y, x, r)
- }
- j = 0
- for _, v := range x[:R] {
- binary.LittleEndian.PutUint32(b[j:], v)
- j += 4
- }
-}
-
-// Key derives a key from the password, salt, and cost parameters, returning
-// a byte slice of length keyLen that can be used as cryptographic key.
-//
-// N is a CPU/memory cost parameter, which must be a power of two greater than 1.
-// r and p must satisfy r * p < 2³⁰. If the parameters do not satisfy the
-// limits, the function returns a nil byte slice and an error.
-//
-// For example, you can get a derived key for e.g. AES-256 (which needs a
-// 32-byte key) by doing:
-//
-// dk, err := scrypt.Key([]byte("some password"), salt, 32768, 8, 1, 32)
-//
-// The recommended parameters for interactive logins as of 2017 are N=32768, r=8
-// and p=1. The parameters N, r, and p should be increased as memory latency and
-// CPU parallelism increases; consider setting N to the highest power of 2 you
-// can derive within 100 milliseconds. Remember to get a good random salt.
-func Key(password, salt []byte, N, r, p, keyLen int) ([]byte, error) {
- if N <= 1 || N&(N-1) != 0 {
- return nil, errors.New("scrypt: N must be > 1 and a power of 2")
- }
- if uint64(r)*uint64(p) >= 1<<30 || r > maxInt/128/p || r > maxInt/256 || N > maxInt/128/r {
- return nil, errors.New("scrypt: parameters are too large")
- }
-
- xy := make([]uint32, 64*r)
- v := make([]uint32, 32*N*r)
- b := pbkdf2.Key(password, salt, 1, p*128*r, sha256.New)
-
- for i := 0; i < p; i++ {
- smix(b[i*128*r:], r, N, v, xy)
- }
-
- return pbkdf2.Key(password, b, 1, keyLen, sha256.New), nil
-}
diff --git a/vendor/golang.org/x/crypto/sha3/doc.go b/vendor/golang.org/x/crypto/sha3/doc.go
deleted file mode 100644
index decd8cf..0000000
--- a/vendor/golang.org/x/crypto/sha3/doc.go
+++ /dev/null
@@ -1,62 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package sha3 implements the SHA-3 fixed-output-length hash functions and
-// the SHAKE variable-output-length hash functions defined by FIPS-202.
-//
-// Both types of hash function use the "sponge" construction and the Keccak
-// permutation. For a detailed specification see http://keccak.noekeon.org/
-//
-// # Guidance
-//
-// If you aren't sure what function you need, use SHAKE256 with at least 64
-// bytes of output. The SHAKE instances are faster than the SHA3 instances;
-// the latter have to allocate memory to conform to the hash.Hash interface.
-//
-// If you need a secret-key MAC (message authentication code), prepend the
-// secret key to the input, hash with SHAKE256 and read at least 32 bytes of
-// output.
-//
-// # Security strengths
-//
-// The SHA3-x (x equals 224, 256, 384, or 512) functions have a security
-// strength against preimage attacks of x bits. Since they only produce "x"
-// bits of output, their collision-resistance is only "x/2" bits.
-//
-// The SHAKE-256 and -128 functions have a generic security strength of 256 and
-// 128 bits against all attacks, provided that at least 2x bits of their output
-// is used. Requesting more than 64 or 32 bytes of output, respectively, does
-// not increase the collision-resistance of the SHAKE functions.
-//
-// # The sponge construction
-//
-// A sponge builds a pseudo-random function from a public pseudo-random
-// permutation, by applying the permutation to a state of "rate + capacity"
-// bytes, but hiding "capacity" of the bytes.
-//
-// A sponge starts out with a zero state. To hash an input using a sponge, up
-// to "rate" bytes of the input are XORed into the sponge's state. The sponge
-// is then "full" and the permutation is applied to "empty" it. This process is
-// repeated until all the input has been "absorbed". The input is then padded.
-// The digest is "squeezed" from the sponge in the same way, except that output
-// is copied out instead of input being XORed in.
-//
-// A sponge is parameterized by its generic security strength, which is equal
-// to half its capacity; capacity + rate is equal to the permutation's width.
-// Since the KeccakF-1600 permutation is 1600 bits (200 bytes) wide, this means
-// that the security strength of a sponge instance is equal to (1600 - bitrate) / 2.
-//
-// # Recommendations
-//
-// The SHAKE functions are recommended for most new uses. They can produce
-// output of arbitrary length. SHAKE256, with an output length of at least
-// 64 bytes, provides 256-bit security against all attacks. The Keccak team
-// recommends it for most applications upgrading from SHA2-512. (NIST chose a
-// much stronger, but much slower, sponge instance for SHA3-512.)
-//
-// The SHA-3 functions are "drop-in" replacements for the SHA-2 functions.
-// They produce output of the same length, with the same security strengths
-// against all attacks. This means, in particular, that SHA3-256 only has
-// 128-bit collision resistance, because its output length is 32 bytes.
-package sha3 // import "golang.org/x/crypto/sha3"
diff --git a/vendor/golang.org/x/crypto/sha3/hashes.go b/vendor/golang.org/x/crypto/sha3/hashes.go
deleted file mode 100644
index 0d8043f..0000000
--- a/vendor/golang.org/x/crypto/sha3/hashes.go
+++ /dev/null
@@ -1,97 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package sha3
-
-// This file provides functions for creating instances of the SHA-3
-// and SHAKE hash functions, as well as utility functions for hashing
-// bytes.
-
-import (
- "hash"
-)
-
-// New224 creates a new SHA3-224 hash.
-// Its generic security strength is 224 bits against preimage attacks,
-// and 112 bits against collision attacks.
-func New224() hash.Hash {
- if h := new224Asm(); h != nil {
- return h
- }
- return &state{rate: 144, outputLen: 28, dsbyte: 0x06}
-}
-
-// New256 creates a new SHA3-256 hash.
-// Its generic security strength is 256 bits against preimage attacks,
-// and 128 bits against collision attacks.
-func New256() hash.Hash {
- if h := new256Asm(); h != nil {
- return h
- }
- return &state{rate: 136, outputLen: 32, dsbyte: 0x06}
-}
-
-// New384 creates a new SHA3-384 hash.
-// Its generic security strength is 384 bits against preimage attacks,
-// and 192 bits against collision attacks.
-func New384() hash.Hash {
- if h := new384Asm(); h != nil {
- return h
- }
- return &state{rate: 104, outputLen: 48, dsbyte: 0x06}
-}
-
-// New512 creates a new SHA3-512 hash.
-// Its generic security strength is 512 bits against preimage attacks,
-// and 256 bits against collision attacks.
-func New512() hash.Hash {
- if h := new512Asm(); h != nil {
- return h
- }
- return &state{rate: 72, outputLen: 64, dsbyte: 0x06}
-}
-
-// NewLegacyKeccak256 creates a new Keccak-256 hash.
-//
-// Only use this function if you require compatibility with an existing cryptosystem
-// that uses non-standard padding. All other users should use New256 instead.
-func NewLegacyKeccak256() hash.Hash { return &state{rate: 136, outputLen: 32, dsbyte: 0x01} }
-
-// NewLegacyKeccak512 creates a new Keccak-512 hash.
-//
-// Only use this function if you require compatibility with an existing cryptosystem
-// that uses non-standard padding. All other users should use New512 instead.
-func NewLegacyKeccak512() hash.Hash { return &state{rate: 72, outputLen: 64, dsbyte: 0x01} }
-
-// Sum224 returns the SHA3-224 digest of the data.
-func Sum224(data []byte) (digest [28]byte) {
- h := New224()
- h.Write(data)
- h.Sum(digest[:0])
- return
-}
-
-// Sum256 returns the SHA3-256 digest of the data.
-func Sum256(data []byte) (digest [32]byte) {
- h := New256()
- h.Write(data)
- h.Sum(digest[:0])
- return
-}
-
-// Sum384 returns the SHA3-384 digest of the data.
-func Sum384(data []byte) (digest [48]byte) {
- h := New384()
- h.Write(data)
- h.Sum(digest[:0])
- return
-}
-
-// Sum512 returns the SHA3-512 digest of the data.
-func Sum512(data []byte) (digest [64]byte) {
- h := New512()
- h.Write(data)
- h.Sum(digest[:0])
- return
-}
diff --git a/vendor/golang.org/x/crypto/sha3/hashes_generic.go b/vendor/golang.org/x/crypto/sha3/hashes_generic.go
deleted file mode 100644
index c74fc20..0000000
--- a/vendor/golang.org/x/crypto/sha3/hashes_generic.go
+++ /dev/null
@@ -1,28 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !gc || purego || !s390x
-// +build !gc purego !s390x
-
-package sha3
-
-import (
- "hash"
-)
-
-// new224Asm returns an assembly implementation of SHA3-224 if available,
-// otherwise it returns nil.
-func new224Asm() hash.Hash { return nil }
-
-// new256Asm returns an assembly implementation of SHA3-256 if available,
-// otherwise it returns nil.
-func new256Asm() hash.Hash { return nil }
-
-// new384Asm returns an assembly implementation of SHA3-384 if available,
-// otherwise it returns nil.
-func new384Asm() hash.Hash { return nil }
-
-// new512Asm returns an assembly implementation of SHA3-512 if available,
-// otherwise it returns nil.
-func new512Asm() hash.Hash { return nil }
diff --git a/vendor/golang.org/x/crypto/sha3/keccakf.go b/vendor/golang.org/x/crypto/sha3/keccakf.go
deleted file mode 100644
index 0f4ae8b..0000000
--- a/vendor/golang.org/x/crypto/sha3/keccakf.go
+++ /dev/null
@@ -1,413 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !amd64 || purego || !gc
-// +build !amd64 purego !gc
-
-package sha3
-
-// rc stores the round constants for use in the ι step.
-var rc = [24]uint64{
- 0x0000000000000001,
- 0x0000000000008082,
- 0x800000000000808A,
- 0x8000000080008000,
- 0x000000000000808B,
- 0x0000000080000001,
- 0x8000000080008081,
- 0x8000000000008009,
- 0x000000000000008A,
- 0x0000000000000088,
- 0x0000000080008009,
- 0x000000008000000A,
- 0x000000008000808B,
- 0x800000000000008B,
- 0x8000000000008089,
- 0x8000000000008003,
- 0x8000000000008002,
- 0x8000000000000080,
- 0x000000000000800A,
- 0x800000008000000A,
- 0x8000000080008081,
- 0x8000000000008080,
- 0x0000000080000001,
- 0x8000000080008008,
-}
-
-// keccakF1600 applies the Keccak permutation to a 1600b-wide
-// state represented as a slice of 25 uint64s.
-func keccakF1600(a *[25]uint64) {
- // Implementation translated from Keccak-inplace.c
- // in the keccak reference code.
- var t, bc0, bc1, bc2, bc3, bc4, d0, d1, d2, d3, d4 uint64
-
- for i := 0; i < 24; i += 4 {
- // Combines the 5 steps in each round into 2 steps.
- // Unrolls 4 rounds per loop and spreads some steps across rounds.
-
- // Round 1
- bc0 = a[0] ^ a[5] ^ a[10] ^ a[15] ^ a[20]
- bc1 = a[1] ^ a[6] ^ a[11] ^ a[16] ^ a[21]
- bc2 = a[2] ^ a[7] ^ a[12] ^ a[17] ^ a[22]
- bc3 = a[3] ^ a[8] ^ a[13] ^ a[18] ^ a[23]
- bc4 = a[4] ^ a[9] ^ a[14] ^ a[19] ^ a[24]
- d0 = bc4 ^ (bc1<<1 | bc1>>63)
- d1 = bc0 ^ (bc2<<1 | bc2>>63)
- d2 = bc1 ^ (bc3<<1 | bc3>>63)
- d3 = bc2 ^ (bc4<<1 | bc4>>63)
- d4 = bc3 ^ (bc0<<1 | bc0>>63)
-
- bc0 = a[0] ^ d0
- t = a[6] ^ d1
- bc1 = t<<44 | t>>(64-44)
- t = a[12] ^ d2
- bc2 = t<<43 | t>>(64-43)
- t = a[18] ^ d3
- bc3 = t<<21 | t>>(64-21)
- t = a[24] ^ d4
- bc4 = t<<14 | t>>(64-14)
- a[0] = bc0 ^ (bc2 &^ bc1) ^ rc[i]
- a[6] = bc1 ^ (bc3 &^ bc2)
- a[12] = bc2 ^ (bc4 &^ bc3)
- a[18] = bc3 ^ (bc0 &^ bc4)
- a[24] = bc4 ^ (bc1 &^ bc0)
-
- t = a[10] ^ d0
- bc2 = t<<3 | t>>(64-3)
- t = a[16] ^ d1
- bc3 = t<<45 | t>>(64-45)
- t = a[22] ^ d2
- bc4 = t<<61 | t>>(64-61)
- t = a[3] ^ d3
- bc0 = t<<28 | t>>(64-28)
- t = a[9] ^ d4
- bc1 = t<<20 | t>>(64-20)
- a[10] = bc0 ^ (bc2 &^ bc1)
- a[16] = bc1 ^ (bc3 &^ bc2)
- a[22] = bc2 ^ (bc4 &^ bc3)
- a[3] = bc3 ^ (bc0 &^ bc4)
- a[9] = bc4 ^ (bc1 &^ bc0)
-
- t = a[20] ^ d0
- bc4 = t<<18 | t>>(64-18)
- t = a[1] ^ d1
- bc0 = t<<1 | t>>(64-1)
- t = a[7] ^ d2
- bc1 = t<<6 | t>>(64-6)
- t = a[13] ^ d3
- bc2 = t<<25 | t>>(64-25)
- t = a[19] ^ d4
- bc3 = t<<8 | t>>(64-8)
- a[20] = bc0 ^ (bc2 &^ bc1)
- a[1] = bc1 ^ (bc3 &^ bc2)
- a[7] = bc2 ^ (bc4 &^ bc3)
- a[13] = bc3 ^ (bc0 &^ bc4)
- a[19] = bc4 ^ (bc1 &^ bc0)
-
- t = a[5] ^ d0
- bc1 = t<<36 | t>>(64-36)
- t = a[11] ^ d1
- bc2 = t<<10 | t>>(64-10)
- t = a[17] ^ d2
- bc3 = t<<15 | t>>(64-15)
- t = a[23] ^ d3
- bc4 = t<<56 | t>>(64-56)
- t = a[4] ^ d4
- bc0 = t<<27 | t>>(64-27)
- a[5] = bc0 ^ (bc2 &^ bc1)
- a[11] = bc1 ^ (bc3 &^ bc2)
- a[17] = bc2 ^ (bc4 &^ bc3)
- a[23] = bc3 ^ (bc0 &^ bc4)
- a[4] = bc4 ^ (bc1 &^ bc0)
-
- t = a[15] ^ d0
- bc3 = t<<41 | t>>(64-41)
- t = a[21] ^ d1
- bc4 = t<<2 | t>>(64-2)
- t = a[2] ^ d2
- bc0 = t<<62 | t>>(64-62)
- t = a[8] ^ d3
- bc1 = t<<55 | t>>(64-55)
- t = a[14] ^ d4
- bc2 = t<<39 | t>>(64-39)
- a[15] = bc0 ^ (bc2 &^ bc1)
- a[21] = bc1 ^ (bc3 &^ bc2)
- a[2] = bc2 ^ (bc4 &^ bc3)
- a[8] = bc3 ^ (bc0 &^ bc4)
- a[14] = bc4 ^ (bc1 &^ bc0)
-
- // Round 2
- bc0 = a[0] ^ a[5] ^ a[10] ^ a[15] ^ a[20]
- bc1 = a[1] ^ a[6] ^ a[11] ^ a[16] ^ a[21]
- bc2 = a[2] ^ a[7] ^ a[12] ^ a[17] ^ a[22]
- bc3 = a[3] ^ a[8] ^ a[13] ^ a[18] ^ a[23]
- bc4 = a[4] ^ a[9] ^ a[14] ^ a[19] ^ a[24]
- d0 = bc4 ^ (bc1<<1 | bc1>>63)
- d1 = bc0 ^ (bc2<<1 | bc2>>63)
- d2 = bc1 ^ (bc3<<1 | bc3>>63)
- d3 = bc2 ^ (bc4<<1 | bc4>>63)
- d4 = bc3 ^ (bc0<<1 | bc0>>63)
-
- bc0 = a[0] ^ d0
- t = a[16] ^ d1
- bc1 = t<<44 | t>>(64-44)
- t = a[7] ^ d2
- bc2 = t<<43 | t>>(64-43)
- t = a[23] ^ d3
- bc3 = t<<21 | t>>(64-21)
- t = a[14] ^ d4
- bc4 = t<<14 | t>>(64-14)
- a[0] = bc0 ^ (bc2 &^ bc1) ^ rc[i+1]
- a[16] = bc1 ^ (bc3 &^ bc2)
- a[7] = bc2 ^ (bc4 &^ bc3)
- a[23] = bc3 ^ (bc0 &^ bc4)
- a[14] = bc4 ^ (bc1 &^ bc0)
-
- t = a[20] ^ d0
- bc2 = t<<3 | t>>(64-3)
- t = a[11] ^ d1
- bc3 = t<<45 | t>>(64-45)
- t = a[2] ^ d2
- bc4 = t<<61 | t>>(64-61)
- t = a[18] ^ d3
- bc0 = t<<28 | t>>(64-28)
- t = a[9] ^ d4
- bc1 = t<<20 | t>>(64-20)
- a[20] = bc0 ^ (bc2 &^ bc1)
- a[11] = bc1 ^ (bc3 &^ bc2)
- a[2] = bc2 ^ (bc4 &^ bc3)
- a[18] = bc3 ^ (bc0 &^ bc4)
- a[9] = bc4 ^ (bc1 &^ bc0)
-
- t = a[15] ^ d0
- bc4 = t<<18 | t>>(64-18)
- t = a[6] ^ d1
- bc0 = t<<1 | t>>(64-1)
- t = a[22] ^ d2
- bc1 = t<<6 | t>>(64-6)
- t = a[13] ^ d3
- bc2 = t<<25 | t>>(64-25)
- t = a[4] ^ d4
- bc3 = t<<8 | t>>(64-8)
- a[15] = bc0 ^ (bc2 &^ bc1)
- a[6] = bc1 ^ (bc3 &^ bc2)
- a[22] = bc2 ^ (bc4 &^ bc3)
- a[13] = bc3 ^ (bc0 &^ bc4)
- a[4] = bc4 ^ (bc1 &^ bc0)
-
- t = a[10] ^ d0
- bc1 = t<<36 | t>>(64-36)
- t = a[1] ^ d1
- bc2 = t<<10 | t>>(64-10)
- t = a[17] ^ d2
- bc3 = t<<15 | t>>(64-15)
- t = a[8] ^ d3
- bc4 = t<<56 | t>>(64-56)
- t = a[24] ^ d4
- bc0 = t<<27 | t>>(64-27)
- a[10] = bc0 ^ (bc2 &^ bc1)
- a[1] = bc1 ^ (bc3 &^ bc2)
- a[17] = bc2 ^ (bc4 &^ bc3)
- a[8] = bc3 ^ (bc0 &^ bc4)
- a[24] = bc4 ^ (bc1 &^ bc0)
-
- t = a[5] ^ d0
- bc3 = t<<41 | t>>(64-41)
- t = a[21] ^ d1
- bc4 = t<<2 | t>>(64-2)
- t = a[12] ^ d2
- bc0 = t<<62 | t>>(64-62)
- t = a[3] ^ d3
- bc1 = t<<55 | t>>(64-55)
- t = a[19] ^ d4
- bc2 = t<<39 | t>>(64-39)
- a[5] = bc0 ^ (bc2 &^ bc1)
- a[21] = bc1 ^ (bc3 &^ bc2)
- a[12] = bc2 ^ (bc4 &^ bc3)
- a[3] = bc3 ^ (bc0 &^ bc4)
- a[19] = bc4 ^ (bc1 &^ bc0)
-
- // Round 3
- bc0 = a[0] ^ a[5] ^ a[10] ^ a[15] ^ a[20]
- bc1 = a[1] ^ a[6] ^ a[11] ^ a[16] ^ a[21]
- bc2 = a[2] ^ a[7] ^ a[12] ^ a[17] ^ a[22]
- bc3 = a[3] ^ a[8] ^ a[13] ^ a[18] ^ a[23]
- bc4 = a[4] ^ a[9] ^ a[14] ^ a[19] ^ a[24]
- d0 = bc4 ^ (bc1<<1 | bc1>>63)
- d1 = bc0 ^ (bc2<<1 | bc2>>63)
- d2 = bc1 ^ (bc3<<1 | bc3>>63)
- d3 = bc2 ^ (bc4<<1 | bc4>>63)
- d4 = bc3 ^ (bc0<<1 | bc0>>63)
-
- bc0 = a[0] ^ d0
- t = a[11] ^ d1
- bc1 = t<<44 | t>>(64-44)
- t = a[22] ^ d2
- bc2 = t<<43 | t>>(64-43)
- t = a[8] ^ d3
- bc3 = t<<21 | t>>(64-21)
- t = a[19] ^ d4
- bc4 = t<<14 | t>>(64-14)
- a[0] = bc0 ^ (bc2 &^ bc1) ^ rc[i+2]
- a[11] = bc1 ^ (bc3 &^ bc2)
- a[22] = bc2 ^ (bc4 &^ bc3)
- a[8] = bc3 ^ (bc0 &^ bc4)
- a[19] = bc4 ^ (bc1 &^ bc0)
-
- t = a[15] ^ d0
- bc2 = t<<3 | t>>(64-3)
- t = a[1] ^ d1
- bc3 = t<<45 | t>>(64-45)
- t = a[12] ^ d2
- bc4 = t<<61 | t>>(64-61)
- t = a[23] ^ d3
- bc0 = t<<28 | t>>(64-28)
- t = a[9] ^ d4
- bc1 = t<<20 | t>>(64-20)
- a[15] = bc0 ^ (bc2 &^ bc1)
- a[1] = bc1 ^ (bc3 &^ bc2)
- a[12] = bc2 ^ (bc4 &^ bc3)
- a[23] = bc3 ^ (bc0 &^ bc4)
- a[9] = bc4 ^ (bc1 &^ bc0)
-
- t = a[5] ^ d0
- bc4 = t<<18 | t>>(64-18)
- t = a[16] ^ d1
- bc0 = t<<1 | t>>(64-1)
- t = a[2] ^ d2
- bc1 = t<<6 | t>>(64-6)
- t = a[13] ^ d3
- bc2 = t<<25 | t>>(64-25)
- t = a[24] ^ d4
- bc3 = t<<8 | t>>(64-8)
- a[5] = bc0 ^ (bc2 &^ bc1)
- a[16] = bc1 ^ (bc3 &^ bc2)
- a[2] = bc2 ^ (bc4 &^ bc3)
- a[13] = bc3 ^ (bc0 &^ bc4)
- a[24] = bc4 ^ (bc1 &^ bc0)
-
- t = a[20] ^ d0
- bc1 = t<<36 | t>>(64-36)
- t = a[6] ^ d1
- bc2 = t<<10 | t>>(64-10)
- t = a[17] ^ d2
- bc3 = t<<15 | t>>(64-15)
- t = a[3] ^ d3
- bc4 = t<<56 | t>>(64-56)
- t = a[14] ^ d4
- bc0 = t<<27 | t>>(64-27)
- a[20] = bc0 ^ (bc2 &^ bc1)
- a[6] = bc1 ^ (bc3 &^ bc2)
- a[17] = bc2 ^ (bc4 &^ bc3)
- a[3] = bc3 ^ (bc0 &^ bc4)
- a[14] = bc4 ^ (bc1 &^ bc0)
-
- t = a[10] ^ d0
- bc3 = t<<41 | t>>(64-41)
- t = a[21] ^ d1
- bc4 = t<<2 | t>>(64-2)
- t = a[7] ^ d2
- bc0 = t<<62 | t>>(64-62)
- t = a[18] ^ d3
- bc1 = t<<55 | t>>(64-55)
- t = a[4] ^ d4
- bc2 = t<<39 | t>>(64-39)
- a[10] = bc0 ^ (bc2 &^ bc1)
- a[21] = bc1 ^ (bc3 &^ bc2)
- a[7] = bc2 ^ (bc4 &^ bc3)
- a[18] = bc3 ^ (bc0 &^ bc4)
- a[4] = bc4 ^ (bc1 &^ bc0)
-
- // Round 4
- bc0 = a[0] ^ a[5] ^ a[10] ^ a[15] ^ a[20]
- bc1 = a[1] ^ a[6] ^ a[11] ^ a[16] ^ a[21]
- bc2 = a[2] ^ a[7] ^ a[12] ^ a[17] ^ a[22]
- bc3 = a[3] ^ a[8] ^ a[13] ^ a[18] ^ a[23]
- bc4 = a[4] ^ a[9] ^ a[14] ^ a[19] ^ a[24]
- d0 = bc4 ^ (bc1<<1 | bc1>>63)
- d1 = bc0 ^ (bc2<<1 | bc2>>63)
- d2 = bc1 ^ (bc3<<1 | bc3>>63)
- d3 = bc2 ^ (bc4<<1 | bc4>>63)
- d4 = bc3 ^ (bc0<<1 | bc0>>63)
-
- bc0 = a[0] ^ d0
- t = a[1] ^ d1
- bc1 = t<<44 | t>>(64-44)
- t = a[2] ^ d2
- bc2 = t<<43 | t>>(64-43)
- t = a[3] ^ d3
- bc3 = t<<21 | t>>(64-21)
- t = a[4] ^ d4
- bc4 = t<<14 | t>>(64-14)
- a[0] = bc0 ^ (bc2 &^ bc1) ^ rc[i+3]
- a[1] = bc1 ^ (bc3 &^ bc2)
- a[2] = bc2 ^ (bc4 &^ bc3)
- a[3] = bc3 ^ (bc0 &^ bc4)
- a[4] = bc4 ^ (bc1 &^ bc0)
-
- t = a[5] ^ d0
- bc2 = t<<3 | t>>(64-3)
- t = a[6] ^ d1
- bc3 = t<<45 | t>>(64-45)
- t = a[7] ^ d2
- bc4 = t<<61 | t>>(64-61)
- t = a[8] ^ d3
- bc0 = t<<28 | t>>(64-28)
- t = a[9] ^ d4
- bc1 = t<<20 | t>>(64-20)
- a[5] = bc0 ^ (bc2 &^ bc1)
- a[6] = bc1 ^ (bc3 &^ bc2)
- a[7] = bc2 ^ (bc4 &^ bc3)
- a[8] = bc3 ^ (bc0 &^ bc4)
- a[9] = bc4 ^ (bc1 &^ bc0)
-
- t = a[10] ^ d0
- bc4 = t<<18 | t>>(64-18)
- t = a[11] ^ d1
- bc0 = t<<1 | t>>(64-1)
- t = a[12] ^ d2
- bc1 = t<<6 | t>>(64-6)
- t = a[13] ^ d3
- bc2 = t<<25 | t>>(64-25)
- t = a[14] ^ d4
- bc3 = t<<8 | t>>(64-8)
- a[10] = bc0 ^ (bc2 &^ bc1)
- a[11] = bc1 ^ (bc3 &^ bc2)
- a[12] = bc2 ^ (bc4 &^ bc3)
- a[13] = bc3 ^ (bc0 &^ bc4)
- a[14] = bc4 ^ (bc1 &^ bc0)
-
- t = a[15] ^ d0
- bc1 = t<<36 | t>>(64-36)
- t = a[16] ^ d1
- bc2 = t<<10 | t>>(64-10)
- t = a[17] ^ d2
- bc3 = t<<15 | t>>(64-15)
- t = a[18] ^ d3
- bc4 = t<<56 | t>>(64-56)
- t = a[19] ^ d4
- bc0 = t<<27 | t>>(64-27)
- a[15] = bc0 ^ (bc2 &^ bc1)
- a[16] = bc1 ^ (bc3 &^ bc2)
- a[17] = bc2 ^ (bc4 &^ bc3)
- a[18] = bc3 ^ (bc0 &^ bc4)
- a[19] = bc4 ^ (bc1 &^ bc0)
-
- t = a[20] ^ d0
- bc3 = t<<41 | t>>(64-41)
- t = a[21] ^ d1
- bc4 = t<<2 | t>>(64-2)
- t = a[22] ^ d2
- bc0 = t<<62 | t>>(64-62)
- t = a[23] ^ d3
- bc1 = t<<55 | t>>(64-55)
- t = a[24] ^ d4
- bc2 = t<<39 | t>>(64-39)
- a[20] = bc0 ^ (bc2 &^ bc1)
- a[21] = bc1 ^ (bc3 &^ bc2)
- a[22] = bc2 ^ (bc4 &^ bc3)
- a[23] = bc3 ^ (bc0 &^ bc4)
- a[24] = bc4 ^ (bc1 &^ bc0)
- }
-}
diff --git a/vendor/golang.org/x/crypto/sha3/keccakf_amd64.go b/vendor/golang.org/x/crypto/sha3/keccakf_amd64.go
deleted file mode 100644
index 248a382..0000000
--- a/vendor/golang.org/x/crypto/sha3/keccakf_amd64.go
+++ /dev/null
@@ -1,14 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && !purego && gc
-// +build amd64,!purego,gc
-
-package sha3
-
-// This function is implemented in keccakf_amd64.s.
-
-//go:noescape
-
-func keccakF1600(a *[25]uint64)
diff --git a/vendor/golang.org/x/crypto/sha3/keccakf_amd64.s b/vendor/golang.org/x/crypto/sha3/keccakf_amd64.s
deleted file mode 100644
index 4cfa543..0000000
--- a/vendor/golang.org/x/crypto/sha3/keccakf_amd64.s
+++ /dev/null
@@ -1,391 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build amd64 && !purego && gc
-// +build amd64,!purego,gc
-
-// This code was translated into a form compatible with 6a from the public
-// domain sources at https://github.com/gvanas/KeccakCodePackage
-
-// Offsets in state
-#define _ba (0*8)
-#define _be (1*8)
-#define _bi (2*8)
-#define _bo (3*8)
-#define _bu (4*8)
-#define _ga (5*8)
-#define _ge (6*8)
-#define _gi (7*8)
-#define _go (8*8)
-#define _gu (9*8)
-#define _ka (10*8)
-#define _ke (11*8)
-#define _ki (12*8)
-#define _ko (13*8)
-#define _ku (14*8)
-#define _ma (15*8)
-#define _me (16*8)
-#define _mi (17*8)
-#define _mo (18*8)
-#define _mu (19*8)
-#define _sa (20*8)
-#define _se (21*8)
-#define _si (22*8)
-#define _so (23*8)
-#define _su (24*8)
-
-// Temporary registers
-#define rT1 AX
-
-// Round vars
-#define rpState DI
-#define rpStack SP
-
-#define rDa BX
-#define rDe CX
-#define rDi DX
-#define rDo R8
-#define rDu R9
-
-#define rBa R10
-#define rBe R11
-#define rBi R12
-#define rBo R13
-#define rBu R14
-
-#define rCa SI
-#define rCe BP
-#define rCi rBi
-#define rCo rBo
-#define rCu R15
-
-#define MOVQ_RBI_RCE MOVQ rBi, rCe
-#define XORQ_RT1_RCA XORQ rT1, rCa
-#define XORQ_RT1_RCE XORQ rT1, rCe
-#define XORQ_RBA_RCU XORQ rBa, rCu
-#define XORQ_RBE_RCU XORQ rBe, rCu
-#define XORQ_RDU_RCU XORQ rDu, rCu
-#define XORQ_RDA_RCA XORQ rDa, rCa
-#define XORQ_RDE_RCE XORQ rDe, rCe
-
-#define mKeccakRound(iState, oState, rc, B_RBI_RCE, G_RT1_RCA, G_RT1_RCE, G_RBA_RCU, K_RT1_RCA, K_RT1_RCE, K_RBA_RCU, M_RT1_RCA, M_RT1_RCE, M_RBE_RCU, S_RDU_RCU, S_RDA_RCA, S_RDE_RCE) \
- /* Prepare round */ \
- MOVQ rCe, rDa; \
- ROLQ $1, rDa; \
- \
- MOVQ _bi(iState), rCi; \
- XORQ _gi(iState), rDi; \
- XORQ rCu, rDa; \
- XORQ _ki(iState), rCi; \
- XORQ _mi(iState), rDi; \
- XORQ rDi, rCi; \
- \
- MOVQ rCi, rDe; \
- ROLQ $1, rDe; \
- \
- MOVQ _bo(iState), rCo; \
- XORQ _go(iState), rDo; \
- XORQ rCa, rDe; \
- XORQ _ko(iState), rCo; \
- XORQ _mo(iState), rDo; \
- XORQ rDo, rCo; \
- \
- MOVQ rCo, rDi; \
- ROLQ $1, rDi; \
- \
- MOVQ rCu, rDo; \
- XORQ rCe, rDi; \
- ROLQ $1, rDo; \
- \
- MOVQ rCa, rDu; \
- XORQ rCi, rDo; \
- ROLQ $1, rDu; \
- \
- /* Result b */ \
- MOVQ _ba(iState), rBa; \
- MOVQ _ge(iState), rBe; \
- XORQ rCo, rDu; \
- MOVQ _ki(iState), rBi; \
- MOVQ _mo(iState), rBo; \
- MOVQ _su(iState), rBu; \
- XORQ rDe, rBe; \
- ROLQ $44, rBe; \
- XORQ rDi, rBi; \
- XORQ rDa, rBa; \
- ROLQ $43, rBi; \
- \
- MOVQ rBe, rCa; \
- MOVQ rc, rT1; \
- ORQ rBi, rCa; \
- XORQ rBa, rT1; \
- XORQ rT1, rCa; \
- MOVQ rCa, _ba(oState); \
- \
- XORQ rDu, rBu; \
- ROLQ $14, rBu; \
- MOVQ rBa, rCu; \
- ANDQ rBe, rCu; \
- XORQ rBu, rCu; \
- MOVQ rCu, _bu(oState); \
- \
- XORQ rDo, rBo; \
- ROLQ $21, rBo; \
- MOVQ rBo, rT1; \
- ANDQ rBu, rT1; \
- XORQ rBi, rT1; \
- MOVQ rT1, _bi(oState); \
- \
- NOTQ rBi; \
- ORQ rBa, rBu; \
- ORQ rBo, rBi; \
- XORQ rBo, rBu; \
- XORQ rBe, rBi; \
- MOVQ rBu, _bo(oState); \
- MOVQ rBi, _be(oState); \
- B_RBI_RCE; \
- \
- /* Result g */ \
- MOVQ _gu(iState), rBe; \
- XORQ rDu, rBe; \
- MOVQ _ka(iState), rBi; \
- ROLQ $20, rBe; \
- XORQ rDa, rBi; \
- ROLQ $3, rBi; \
- MOVQ _bo(iState), rBa; \
- MOVQ rBe, rT1; \
- ORQ rBi, rT1; \
- XORQ rDo, rBa; \
- MOVQ _me(iState), rBo; \
- MOVQ _si(iState), rBu; \
- ROLQ $28, rBa; \
- XORQ rBa, rT1; \
- MOVQ rT1, _ga(oState); \
- G_RT1_RCA; \
- \
- XORQ rDe, rBo; \
- ROLQ $45, rBo; \
- MOVQ rBi, rT1; \
- ANDQ rBo, rT1; \
- XORQ rBe, rT1; \
- MOVQ rT1, _ge(oState); \
- G_RT1_RCE; \
- \
- XORQ rDi, rBu; \
- ROLQ $61, rBu; \
- MOVQ rBu, rT1; \
- ORQ rBa, rT1; \
- XORQ rBo, rT1; \
- MOVQ rT1, _go(oState); \
- \
- ANDQ rBe, rBa; \
- XORQ rBu, rBa; \
- MOVQ rBa, _gu(oState); \
- NOTQ rBu; \
- G_RBA_RCU; \
- \
- ORQ rBu, rBo; \
- XORQ rBi, rBo; \
- MOVQ rBo, _gi(oState); \
- \
- /* Result k */ \
- MOVQ _be(iState), rBa; \
- MOVQ _gi(iState), rBe; \
- MOVQ _ko(iState), rBi; \
- MOVQ _mu(iState), rBo; \
- MOVQ _sa(iState), rBu; \
- XORQ rDi, rBe; \
- ROLQ $6, rBe; \
- XORQ rDo, rBi; \
- ROLQ $25, rBi; \
- MOVQ rBe, rT1; \
- ORQ rBi, rT1; \
- XORQ rDe, rBa; \
- ROLQ $1, rBa; \
- XORQ rBa, rT1; \
- MOVQ rT1, _ka(oState); \
- K_RT1_RCA; \
- \
- XORQ rDu, rBo; \
- ROLQ $8, rBo; \
- MOVQ rBi, rT1; \
- ANDQ rBo, rT1; \
- XORQ rBe, rT1; \
- MOVQ rT1, _ke(oState); \
- K_RT1_RCE; \
- \
- XORQ rDa, rBu; \
- ROLQ $18, rBu; \
- NOTQ rBo; \
- MOVQ rBo, rT1; \
- ANDQ rBu, rT1; \
- XORQ rBi, rT1; \
- MOVQ rT1, _ki(oState); \
- \
- MOVQ rBu, rT1; \
- ORQ rBa, rT1; \
- XORQ rBo, rT1; \
- MOVQ rT1, _ko(oState); \
- \
- ANDQ rBe, rBa; \
- XORQ rBu, rBa; \
- MOVQ rBa, _ku(oState); \
- K_RBA_RCU; \
- \
- /* Result m */ \
- MOVQ _ga(iState), rBe; \
- XORQ rDa, rBe; \
- MOVQ _ke(iState), rBi; \
- ROLQ $36, rBe; \
- XORQ rDe, rBi; \
- MOVQ _bu(iState), rBa; \
- ROLQ $10, rBi; \
- MOVQ rBe, rT1; \
- MOVQ _mi(iState), rBo; \
- ANDQ rBi, rT1; \
- XORQ rDu, rBa; \
- MOVQ _so(iState), rBu; \
- ROLQ $27, rBa; \
- XORQ rBa, rT1; \
- MOVQ rT1, _ma(oState); \
- M_RT1_RCA; \
- \
- XORQ rDi, rBo; \
- ROLQ $15, rBo; \
- MOVQ rBi, rT1; \
- ORQ rBo, rT1; \
- XORQ rBe, rT1; \
- MOVQ rT1, _me(oState); \
- M_RT1_RCE; \
- \
- XORQ rDo, rBu; \
- ROLQ $56, rBu; \
- NOTQ rBo; \
- MOVQ rBo, rT1; \
- ORQ rBu, rT1; \
- XORQ rBi, rT1; \
- MOVQ rT1, _mi(oState); \
- \
- ORQ rBa, rBe; \
- XORQ rBu, rBe; \
- MOVQ rBe, _mu(oState); \
- \
- ANDQ rBa, rBu; \
- XORQ rBo, rBu; \
- MOVQ rBu, _mo(oState); \
- M_RBE_RCU; \
- \
- /* Result s */ \
- MOVQ _bi(iState), rBa; \
- MOVQ _go(iState), rBe; \
- MOVQ _ku(iState), rBi; \
- XORQ rDi, rBa; \
- MOVQ _ma(iState), rBo; \
- ROLQ $62, rBa; \
- XORQ rDo, rBe; \
- MOVQ _se(iState), rBu; \
- ROLQ $55, rBe; \
- \
- XORQ rDu, rBi; \
- MOVQ rBa, rDu; \
- XORQ rDe, rBu; \
- ROLQ $2, rBu; \
- ANDQ rBe, rDu; \
- XORQ rBu, rDu; \
- MOVQ rDu, _su(oState); \
- \
- ROLQ $39, rBi; \
- S_RDU_RCU; \
- NOTQ rBe; \
- XORQ rDa, rBo; \
- MOVQ rBe, rDa; \
- ANDQ rBi, rDa; \
- XORQ rBa, rDa; \
- MOVQ rDa, _sa(oState); \
- S_RDA_RCA; \
- \
- ROLQ $41, rBo; \
- MOVQ rBi, rDe; \
- ORQ rBo, rDe; \
- XORQ rBe, rDe; \
- MOVQ rDe, _se(oState); \
- S_RDE_RCE; \
- \
- MOVQ rBo, rDi; \
- MOVQ rBu, rDo; \
- ANDQ rBu, rDi; \
- ORQ rBa, rDo; \
- XORQ rBi, rDi; \
- XORQ rBo, rDo; \
- MOVQ rDi, _si(oState); \
- MOVQ rDo, _so(oState) \
-
-// func keccakF1600(state *[25]uint64)
-TEXT ·keccakF1600(SB), 0, $200-8
- MOVQ state+0(FP), rpState
-
- // Convert the user state into an internal state
- NOTQ _be(rpState)
- NOTQ _bi(rpState)
- NOTQ _go(rpState)
- NOTQ _ki(rpState)
- NOTQ _mi(rpState)
- NOTQ _sa(rpState)
-
- // Execute the KeccakF permutation
- MOVQ _ba(rpState), rCa
- MOVQ _be(rpState), rCe
- MOVQ _bu(rpState), rCu
-
- XORQ _ga(rpState), rCa
- XORQ _ge(rpState), rCe
- XORQ _gu(rpState), rCu
-
- XORQ _ka(rpState), rCa
- XORQ _ke(rpState), rCe
- XORQ _ku(rpState), rCu
-
- XORQ _ma(rpState), rCa
- XORQ _me(rpState), rCe
- XORQ _mu(rpState), rCu
-
- XORQ _sa(rpState), rCa
- XORQ _se(rpState), rCe
- MOVQ _si(rpState), rDi
- MOVQ _so(rpState), rDo
- XORQ _su(rpState), rCu
-
- mKeccakRound(rpState, rpStack, $0x0000000000000001, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x0000000000008082, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x800000000000808a, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000080008000, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x000000000000808b, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x0000000080000001, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x8000000080008081, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000000008009, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x000000000000008a, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x0000000000000088, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x0000000080008009, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x000000008000000a, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x000000008000808b, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x800000000000008b, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x8000000000008089, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000000008003, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x8000000000008002, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000000000080, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x000000000000800a, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x800000008000000a, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x8000000080008081, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000000008080, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpState, rpStack, $0x0000000080000001, MOVQ_RBI_RCE, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBA_RCU, XORQ_RT1_RCA, XORQ_RT1_RCE, XORQ_RBE_RCU, XORQ_RDU_RCU, XORQ_RDA_RCA, XORQ_RDE_RCE)
- mKeccakRound(rpStack, rpState, $0x8000000080008008, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP, NOP)
-
- // Revert the internal state to the user state
- NOTQ _be(rpState)
- NOTQ _bi(rpState)
- NOTQ _go(rpState)
- NOTQ _ki(rpState)
- NOTQ _mi(rpState)
- NOTQ _sa(rpState)
-
- RET
diff --git a/vendor/golang.org/x/crypto/sha3/register.go b/vendor/golang.org/x/crypto/sha3/register.go
deleted file mode 100644
index 8b4453a..0000000
--- a/vendor/golang.org/x/crypto/sha3/register.go
+++ /dev/null
@@ -1,19 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build go1.4
-// +build go1.4
-
-package sha3
-
-import (
- "crypto"
-)
-
-func init() {
- crypto.RegisterHash(crypto.SHA3_224, New224)
- crypto.RegisterHash(crypto.SHA3_256, New256)
- crypto.RegisterHash(crypto.SHA3_384, New384)
- crypto.RegisterHash(crypto.SHA3_512, New512)
-}
diff --git a/vendor/golang.org/x/crypto/sha3/sha3.go b/vendor/golang.org/x/crypto/sha3/sha3.go
deleted file mode 100644
index fa182be..0000000
--- a/vendor/golang.org/x/crypto/sha3/sha3.go
+++ /dev/null
@@ -1,193 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package sha3
-
-// spongeDirection indicates the direction bytes are flowing through the sponge.
-type spongeDirection int
-
-const (
- // spongeAbsorbing indicates that the sponge is absorbing input.
- spongeAbsorbing spongeDirection = iota
- // spongeSqueezing indicates that the sponge is being squeezed.
- spongeSqueezing
-)
-
-const (
- // maxRate is the maximum size of the internal buffer. SHAKE-256
- // currently needs the largest buffer.
- maxRate = 168
-)
-
-type state struct {
- // Generic sponge components.
- a [25]uint64 // main state of the hash
- buf []byte // points into storage
- rate int // the number of bytes of state to use
-
- // dsbyte contains the "domain separation" bits and the first bit of
- // the padding. Sections 6.1 and 6.2 of [1] separate the outputs of the
- // SHA-3 and SHAKE functions by appending bitstrings to the message.
- // Using a little-endian bit-ordering convention, these are "01" for SHA-3
- // and "1111" for SHAKE, or 00000010b and 00001111b, respectively. Then the
- // padding rule from section 5.1 is applied to pad the message to a multiple
- // of the rate, which involves adding a "1" bit, zero or more "0" bits, and
- // a final "1" bit. We merge the first "1" bit from the padding into dsbyte,
- // giving 00000110b (0x06) and 00011111b (0x1f).
- // [1] http://csrc.nist.gov/publications/drafts/fips-202/fips_202_draft.pdf
- // "Draft FIPS 202: SHA-3 Standard: Permutation-Based Hash and
- // Extendable-Output Functions (May 2014)"
- dsbyte byte
-
- storage storageBuf
-
- // Specific to SHA-3 and SHAKE.
- outputLen int // the default output size in bytes
- state spongeDirection // whether the sponge is absorbing or squeezing
-}
-
-// BlockSize returns the rate of sponge underlying this hash function.
-func (d *state) BlockSize() int { return d.rate }
-
-// Size returns the output size of the hash function in bytes.
-func (d *state) Size() int { return d.outputLen }
-
-// Reset clears the internal state by zeroing the sponge state and
-// the byte buffer, and setting Sponge.state to absorbing.
-func (d *state) Reset() {
- // Zero the permutation's state.
- for i := range d.a {
- d.a[i] = 0
- }
- d.state = spongeAbsorbing
- d.buf = d.storage.asBytes()[:0]
-}
-
-func (d *state) clone() *state {
- ret := *d
- if ret.state == spongeAbsorbing {
- ret.buf = ret.storage.asBytes()[:len(ret.buf)]
- } else {
- ret.buf = ret.storage.asBytes()[d.rate-cap(d.buf) : d.rate]
- }
-
- return &ret
-}
-
-// permute applies the KeccakF-1600 permutation. It handles
-// any input-output buffering.
-func (d *state) permute() {
- switch d.state {
- case spongeAbsorbing:
- // If we're absorbing, we need to xor the input into the state
- // before applying the permutation.
- xorIn(d, d.buf)
- d.buf = d.storage.asBytes()[:0]
- keccakF1600(&d.a)
- case spongeSqueezing:
- // If we're squeezing, we need to apply the permutation before
- // copying more output.
- keccakF1600(&d.a)
- d.buf = d.storage.asBytes()[:d.rate]
- copyOut(d, d.buf)
- }
-}
-
-// pads appends the domain separation bits in dsbyte, applies
-// the multi-bitrate 10..1 padding rule, and permutes the state.
-func (d *state) padAndPermute(dsbyte byte) {
- if d.buf == nil {
- d.buf = d.storage.asBytes()[:0]
- }
- // Pad with this instance's domain-separator bits. We know that there's
- // at least one byte of space in d.buf because, if it were full,
- // permute would have been called to empty it. dsbyte also contains the
- // first one bit for the padding. See the comment in the state struct.
- d.buf = append(d.buf, dsbyte)
- zerosStart := len(d.buf)
- d.buf = d.storage.asBytes()[:d.rate]
- for i := zerosStart; i < d.rate; i++ {
- d.buf[i] = 0
- }
- // This adds the final one bit for the padding. Because of the way that
- // bits are numbered from the LSB upwards, the final bit is the MSB of
- // the last byte.
- d.buf[d.rate-1] ^= 0x80
- // Apply the permutation
- d.permute()
- d.state = spongeSqueezing
- d.buf = d.storage.asBytes()[:d.rate]
- copyOut(d, d.buf)
-}
-
-// Write absorbs more data into the hash's state. It produces an error
-// if more data is written to the ShakeHash after writing
-func (d *state) Write(p []byte) (written int, err error) {
- if d.state != spongeAbsorbing {
- panic("sha3: write to sponge after read")
- }
- if d.buf == nil {
- d.buf = d.storage.asBytes()[:0]
- }
- written = len(p)
-
- for len(p) > 0 {
- if len(d.buf) == 0 && len(p) >= d.rate {
- // The fast path; absorb a full "rate" bytes of input and apply the permutation.
- xorIn(d, p[:d.rate])
- p = p[d.rate:]
- keccakF1600(&d.a)
- } else {
- // The slow path; buffer the input until we can fill the sponge, and then xor it in.
- todo := d.rate - len(d.buf)
- if todo > len(p) {
- todo = len(p)
- }
- d.buf = append(d.buf, p[:todo]...)
- p = p[todo:]
-
- // If the sponge is full, apply the permutation.
- if len(d.buf) == d.rate {
- d.permute()
- }
- }
- }
-
- return
-}
-
-// Read squeezes an arbitrary number of bytes from the sponge.
-func (d *state) Read(out []byte) (n int, err error) {
- // If we're still absorbing, pad and apply the permutation.
- if d.state == spongeAbsorbing {
- d.padAndPermute(d.dsbyte)
- }
-
- n = len(out)
-
- // Now, do the squeezing.
- for len(out) > 0 {
- n := copy(out, d.buf)
- d.buf = d.buf[n:]
- out = out[n:]
-
- // Apply the permutation if we've squeezed the sponge dry.
- if len(d.buf) == 0 {
- d.permute()
- }
- }
-
- return
-}
-
-// Sum applies padding to the hash state and then squeezes out the desired
-// number of output bytes.
-func (d *state) Sum(in []byte) []byte {
- // Make a copy of the original hash so that caller can keep writing
- // and summing.
- dup := d.clone()
- hash := make([]byte, dup.outputLen)
- dup.Read(hash)
- return append(in, hash...)
-}
diff --git a/vendor/golang.org/x/crypto/sha3/sha3_s390x.go b/vendor/golang.org/x/crypto/sha3/sha3_s390x.go
deleted file mode 100644
index 63a3edb..0000000
--- a/vendor/golang.org/x/crypto/sha3/sha3_s390x.go
+++ /dev/null
@@ -1,287 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build gc && !purego
-// +build gc,!purego
-
-package sha3
-
-// This file contains code for using the 'compute intermediate
-// message digest' (KIMD) and 'compute last message digest' (KLMD)
-// instructions to compute SHA-3 and SHAKE hashes on IBM Z.
-
-import (
- "hash"
-
- "golang.org/x/sys/cpu"
-)
-
-// codes represent 7-bit KIMD/KLMD function codes as defined in
-// the Principles of Operation.
-type code uint64
-
-const (
- // function codes for KIMD/KLMD
- sha3_224 code = 32
- sha3_256 = 33
- sha3_384 = 34
- sha3_512 = 35
- shake_128 = 36
- shake_256 = 37
- nopad = 0x100
-)
-
-// kimd is a wrapper for the 'compute intermediate message digest' instruction.
-// src must be a multiple of the rate for the given function code.
-//
-//go:noescape
-func kimd(function code, chain *[200]byte, src []byte)
-
-// klmd is a wrapper for the 'compute last message digest' instruction.
-// src padding is handled by the instruction.
-//
-//go:noescape
-func klmd(function code, chain *[200]byte, dst, src []byte)
-
-type asmState struct {
- a [200]byte // 1600 bit state
- buf []byte // care must be taken to ensure cap(buf) is a multiple of rate
- rate int // equivalent to block size
- storage [3072]byte // underlying storage for buf
- outputLen int // output length if fixed, 0 if not
- function code // KIMD/KLMD function code
- state spongeDirection // whether the sponge is absorbing or squeezing
-}
-
-func newAsmState(function code) *asmState {
- var s asmState
- s.function = function
- switch function {
- case sha3_224:
- s.rate = 144
- s.outputLen = 28
- case sha3_256:
- s.rate = 136
- s.outputLen = 32
- case sha3_384:
- s.rate = 104
- s.outputLen = 48
- case sha3_512:
- s.rate = 72
- s.outputLen = 64
- case shake_128:
- s.rate = 168
- case shake_256:
- s.rate = 136
- default:
- panic("sha3: unrecognized function code")
- }
-
- // limit s.buf size to a multiple of s.rate
- s.resetBuf()
- return &s
-}
-
-func (s *asmState) clone() *asmState {
- c := *s
- c.buf = c.storage[:len(s.buf):cap(s.buf)]
- return &c
-}
-
-// copyIntoBuf copies b into buf. It will panic if there is not enough space to
-// store all of b.
-func (s *asmState) copyIntoBuf(b []byte) {
- bufLen := len(s.buf)
- s.buf = s.buf[:len(s.buf)+len(b)]
- copy(s.buf[bufLen:], b)
-}
-
-// resetBuf points buf at storage, sets the length to 0 and sets cap to be a
-// multiple of the rate.
-func (s *asmState) resetBuf() {
- max := (cap(s.storage) / s.rate) * s.rate
- s.buf = s.storage[:0:max]
-}
-
-// Write (via the embedded io.Writer interface) adds more data to the running hash.
-// It never returns an error.
-func (s *asmState) Write(b []byte) (int, error) {
- if s.state != spongeAbsorbing {
- panic("sha3: write to sponge after read")
- }
- length := len(b)
- for len(b) > 0 {
- if len(s.buf) == 0 && len(b) >= cap(s.buf) {
- // Hash the data directly and push any remaining bytes
- // into the buffer.
- remainder := len(b) % s.rate
- kimd(s.function, &s.a, b[:len(b)-remainder])
- if remainder != 0 {
- s.copyIntoBuf(b[len(b)-remainder:])
- }
- return length, nil
- }
-
- if len(s.buf) == cap(s.buf) {
- // flush the buffer
- kimd(s.function, &s.a, s.buf)
- s.buf = s.buf[:0]
- }
-
- // copy as much as we can into the buffer
- n := len(b)
- if len(b) > cap(s.buf)-len(s.buf) {
- n = cap(s.buf) - len(s.buf)
- }
- s.copyIntoBuf(b[:n])
- b = b[n:]
- }
- return length, nil
-}
-
-// Read squeezes an arbitrary number of bytes from the sponge.
-func (s *asmState) Read(out []byte) (n int, err error) {
- n = len(out)
-
- // need to pad if we were absorbing
- if s.state == spongeAbsorbing {
- s.state = spongeSqueezing
-
- // write hash directly into out if possible
- if len(out)%s.rate == 0 {
- klmd(s.function, &s.a, out, s.buf) // len(out) may be 0
- s.buf = s.buf[:0]
- return
- }
-
- // write hash into buffer
- max := cap(s.buf)
- if max > len(out) {
- max = (len(out)/s.rate)*s.rate + s.rate
- }
- klmd(s.function, &s.a, s.buf[:max], s.buf)
- s.buf = s.buf[:max]
- }
-
- for len(out) > 0 {
- // flush the buffer
- if len(s.buf) != 0 {
- c := copy(out, s.buf)
- out = out[c:]
- s.buf = s.buf[c:]
- continue
- }
-
- // write hash directly into out if possible
- if len(out)%s.rate == 0 {
- klmd(s.function|nopad, &s.a, out, nil)
- return
- }
-
- // write hash into buffer
- s.resetBuf()
- if cap(s.buf) > len(out) {
- s.buf = s.buf[:(len(out)/s.rate)*s.rate+s.rate]
- }
- klmd(s.function|nopad, &s.a, s.buf, nil)
- }
- return
-}
-
-// Sum appends the current hash to b and returns the resulting slice.
-// It does not change the underlying hash state.
-func (s *asmState) Sum(b []byte) []byte {
- if s.outputLen == 0 {
- panic("sha3: cannot call Sum on SHAKE functions")
- }
-
- // Copy the state to preserve the original.
- a := s.a
-
- // Hash the buffer. Note that we don't clear it because we
- // aren't updating the state.
- klmd(s.function, &a, nil, s.buf)
- return append(b, a[:s.outputLen]...)
-}
-
-// Reset resets the Hash to its initial state.
-func (s *asmState) Reset() {
- for i := range s.a {
- s.a[i] = 0
- }
- s.resetBuf()
- s.state = spongeAbsorbing
-}
-
-// Size returns the number of bytes Sum will return.
-func (s *asmState) Size() int {
- return s.outputLen
-}
-
-// BlockSize returns the hash's underlying block size.
-// The Write method must be able to accept any amount
-// of data, but it may operate more efficiently if all writes
-// are a multiple of the block size.
-func (s *asmState) BlockSize() int {
- return s.rate
-}
-
-// Clone returns a copy of the ShakeHash in its current state.
-func (s *asmState) Clone() ShakeHash {
- return s.clone()
-}
-
-// new224Asm returns an assembly implementation of SHA3-224 if available,
-// otherwise it returns nil.
-func new224Asm() hash.Hash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(sha3_224)
- }
- return nil
-}
-
-// new256Asm returns an assembly implementation of SHA3-256 if available,
-// otherwise it returns nil.
-func new256Asm() hash.Hash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(sha3_256)
- }
- return nil
-}
-
-// new384Asm returns an assembly implementation of SHA3-384 if available,
-// otherwise it returns nil.
-func new384Asm() hash.Hash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(sha3_384)
- }
- return nil
-}
-
-// new512Asm returns an assembly implementation of SHA3-512 if available,
-// otherwise it returns nil.
-func new512Asm() hash.Hash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(sha3_512)
- }
- return nil
-}
-
-// newShake128Asm returns an assembly implementation of SHAKE-128 if available,
-// otherwise it returns nil.
-func newShake128Asm() ShakeHash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(shake_128)
- }
- return nil
-}
-
-// newShake256Asm returns an assembly implementation of SHAKE-256 if available,
-// otherwise it returns nil.
-func newShake256Asm() ShakeHash {
- if cpu.S390X.HasSHA3 {
- return newAsmState(shake_256)
- }
- return nil
-}
diff --git a/vendor/golang.org/x/crypto/sha3/sha3_s390x.s b/vendor/golang.org/x/crypto/sha3/sha3_s390x.s
deleted file mode 100644
index a0e051b..0000000
--- a/vendor/golang.org/x/crypto/sha3/sha3_s390x.s
+++ /dev/null
@@ -1,34 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build gc && !purego
-// +build gc,!purego
-
-#include "textflag.h"
-
-// func kimd(function code, chain *[200]byte, src []byte)
-TEXT ·kimd(SB), NOFRAME|NOSPLIT, $0-40
- MOVD function+0(FP), R0
- MOVD chain+8(FP), R1
- LMG src+16(FP), R2, R3 // R2=base, R3=len
-
-continue:
- WORD $0xB93E0002 // KIMD --, R2
- BVS continue // continue if interrupted
- MOVD $0, R0 // reset R0 for pre-go1.8 compilers
- RET
-
-// func klmd(function code, chain *[200]byte, dst, src []byte)
-TEXT ·klmd(SB), NOFRAME|NOSPLIT, $0-64
- // TODO: SHAKE support
- MOVD function+0(FP), R0
- MOVD chain+8(FP), R1
- LMG dst+16(FP), R2, R3 // R2=base, R3=len
- LMG src+40(FP), R4, R5 // R4=base, R5=len
-
-continue:
- WORD $0xB93F0024 // KLMD R2, R4
- BVS continue // continue if interrupted
- MOVD $0, R0 // reset R0 for pre-go1.8 compilers
- RET
diff --git a/vendor/golang.org/x/crypto/sha3/shake.go b/vendor/golang.org/x/crypto/sha3/shake.go
deleted file mode 100644
index d7be295..0000000
--- a/vendor/golang.org/x/crypto/sha3/shake.go
+++ /dev/null
@@ -1,173 +0,0 @@
-// Copyright 2014 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package sha3
-
-// This file defines the ShakeHash interface, and provides
-// functions for creating SHAKE and cSHAKE instances, as well as utility
-// functions for hashing bytes to arbitrary-length output.
-//
-//
-// SHAKE implementation is based on FIPS PUB 202 [1]
-// cSHAKE implementations is based on NIST SP 800-185 [2]
-//
-// [1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
-// [2] https://doi.org/10.6028/NIST.SP.800-185
-
-import (
- "encoding/binary"
- "io"
-)
-
-// ShakeHash defines the interface to hash functions that
-// support arbitrary-length output.
-type ShakeHash interface {
- // Write absorbs more data into the hash's state. It panics if input is
- // written to it after output has been read from it.
- io.Writer
-
- // Read reads more output from the hash; reading affects the hash's
- // state. (ShakeHash.Read is thus very different from Hash.Sum)
- // It never returns an error.
- io.Reader
-
- // Clone returns a copy of the ShakeHash in its current state.
- Clone() ShakeHash
-
- // Reset resets the ShakeHash to its initial state.
- Reset()
-}
-
-// cSHAKE specific context
-type cshakeState struct {
- *state // SHA-3 state context and Read/Write operations
-
- // initBlock is the cSHAKE specific initialization set of bytes. It is initialized
- // by newCShake function and stores concatenation of N followed by S, encoded
- // by the method specified in 3.3 of [1].
- // It is stored here in order for Reset() to be able to put context into
- // initial state.
- initBlock []byte
-}
-
-// Consts for configuring initial SHA-3 state
-const (
- dsbyteShake = 0x1f
- dsbyteCShake = 0x04
- rate128 = 168
- rate256 = 136
-)
-
-func bytepad(input []byte, w int) []byte {
- // leftEncode always returns max 9 bytes
- buf := make([]byte, 0, 9+len(input)+w)
- buf = append(buf, leftEncode(uint64(w))...)
- buf = append(buf, input...)
- padlen := w - (len(buf) % w)
- return append(buf, make([]byte, padlen)...)
-}
-
-func leftEncode(value uint64) []byte {
- var b [9]byte
- binary.BigEndian.PutUint64(b[1:], value)
- // Trim all but last leading zero bytes
- i := byte(1)
- for i < 8 && b[i] == 0 {
- i++
- }
- // Prepend number of encoded bytes
- b[i-1] = 9 - i
- return b[i-1:]
-}
-
-func newCShake(N, S []byte, rate int, dsbyte byte) ShakeHash {
- c := cshakeState{state: &state{rate: rate, dsbyte: dsbyte}}
-
- // leftEncode returns max 9 bytes
- c.initBlock = make([]byte, 0, 9*2+len(N)+len(S))
- c.initBlock = append(c.initBlock, leftEncode(uint64(len(N)*8))...)
- c.initBlock = append(c.initBlock, N...)
- c.initBlock = append(c.initBlock, leftEncode(uint64(len(S)*8))...)
- c.initBlock = append(c.initBlock, S...)
- c.Write(bytepad(c.initBlock, c.rate))
- return &c
-}
-
-// Reset resets the hash to initial state.
-func (c *cshakeState) Reset() {
- c.state.Reset()
- c.Write(bytepad(c.initBlock, c.rate))
-}
-
-// Clone returns copy of a cSHAKE context within its current state.
-func (c *cshakeState) Clone() ShakeHash {
- b := make([]byte, len(c.initBlock))
- copy(b, c.initBlock)
- return &cshakeState{state: c.clone(), initBlock: b}
-}
-
-// Clone returns copy of SHAKE context within its current state.
-func (c *state) Clone() ShakeHash {
- return c.clone()
-}
-
-// NewShake128 creates a new SHAKE128 variable-output-length ShakeHash.
-// Its generic security strength is 128 bits against all attacks if at
-// least 32 bytes of its output are used.
-func NewShake128() ShakeHash {
- if h := newShake128Asm(); h != nil {
- return h
- }
- return &state{rate: rate128, dsbyte: dsbyteShake}
-}
-
-// NewShake256 creates a new SHAKE256 variable-output-length ShakeHash.
-// Its generic security strength is 256 bits against all attacks if
-// at least 64 bytes of its output are used.
-func NewShake256() ShakeHash {
- if h := newShake256Asm(); h != nil {
- return h
- }
- return &state{rate: rate256, dsbyte: dsbyteShake}
-}
-
-// NewCShake128 creates a new instance of cSHAKE128 variable-output-length ShakeHash,
-// a customizable variant of SHAKE128.
-// N is used to define functions based on cSHAKE, it can be empty when plain cSHAKE is
-// desired. S is a customization byte string used for domain separation - two cSHAKE
-// computations on same input with different S yield unrelated outputs.
-// When N and S are both empty, this is equivalent to NewShake128.
-func NewCShake128(N, S []byte) ShakeHash {
- if len(N) == 0 && len(S) == 0 {
- return NewShake128()
- }
- return newCShake(N, S, rate128, dsbyteCShake)
-}
-
-// NewCShake256 creates a new instance of cSHAKE256 variable-output-length ShakeHash,
-// a customizable variant of SHAKE256.
-// N is used to define functions based on cSHAKE, it can be empty when plain cSHAKE is
-// desired. S is a customization byte string used for domain separation - two cSHAKE
-// computations on same input with different S yield unrelated outputs.
-// When N and S are both empty, this is equivalent to NewShake256.
-func NewCShake256(N, S []byte) ShakeHash {
- if len(N) == 0 && len(S) == 0 {
- return NewShake256()
- }
- return newCShake(N, S, rate256, dsbyteCShake)
-}
-
-// ShakeSum128 writes an arbitrary-length digest of data into hash.
-func ShakeSum128(hash, data []byte) {
- h := NewShake128()
- h.Write(data)
- h.Read(hash)
-}
-
-// ShakeSum256 writes an arbitrary-length digest of data into hash.
-func ShakeSum256(hash, data []byte) {
- h := NewShake256()
- h.Write(data)
- h.Read(hash)
-}
diff --git a/vendor/golang.org/x/crypto/sha3/shake_generic.go b/vendor/golang.org/x/crypto/sha3/shake_generic.go
deleted file mode 100644
index 5c0710e..0000000
--- a/vendor/golang.org/x/crypto/sha3/shake_generic.go
+++ /dev/null
@@ -1,20 +0,0 @@
-// Copyright 2017 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build !gc || purego || !s390x
-// +build !gc purego !s390x
-
-package sha3
-
-// newShake128Asm returns an assembly implementation of SHAKE-128 if available,
-// otherwise it returns nil.
-func newShake128Asm() ShakeHash {
- return nil
-}
-
-// newShake256Asm returns an assembly implementation of SHAKE-256 if available,
-// otherwise it returns nil.
-func newShake256Asm() ShakeHash {
- return nil
-}
diff --git a/vendor/golang.org/x/crypto/sha3/xor.go b/vendor/golang.org/x/crypto/sha3/xor.go
deleted file mode 100644
index 59c8eb9..0000000
--- a/vendor/golang.org/x/crypto/sha3/xor.go
+++ /dev/null
@@ -1,24 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build (!amd64 && !386 && !ppc64le) || purego
-// +build !amd64,!386,!ppc64le purego
-
-package sha3
-
-// A storageBuf is an aligned array of maxRate bytes.
-type storageBuf [maxRate]byte
-
-func (b *storageBuf) asBytes() *[maxRate]byte {
- return (*[maxRate]byte)(b)
-}
-
-var (
- xorIn = xorInGeneric
- copyOut = copyOutGeneric
- xorInUnaligned = xorInGeneric
- copyOutUnaligned = copyOutGeneric
-)
-
-const xorImplementationUnaligned = "generic"
diff --git a/vendor/golang.org/x/crypto/sha3/xor_generic.go b/vendor/golang.org/x/crypto/sha3/xor_generic.go
deleted file mode 100644
index 8d94771..0000000
--- a/vendor/golang.org/x/crypto/sha3/xor_generic.go
+++ /dev/null
@@ -1,28 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package sha3
-
-import "encoding/binary"
-
-// xorInGeneric xors the bytes in buf into the state; it
-// makes no non-portable assumptions about memory layout
-// or alignment.
-func xorInGeneric(d *state, buf []byte) {
- n := len(buf) / 8
-
- for i := 0; i < n; i++ {
- a := binary.LittleEndian.Uint64(buf)
- d.a[i] ^= a
- buf = buf[8:]
- }
-}
-
-// copyOutGeneric copies uint64s to a byte buffer.
-func copyOutGeneric(d *state, b []byte) {
- for i := 0; len(b) >= 8; i++ {
- binary.LittleEndian.PutUint64(b, d.a[i])
- b = b[8:]
- }
-}
diff --git a/vendor/golang.org/x/crypto/sha3/xor_unaligned.go b/vendor/golang.org/x/crypto/sha3/xor_unaligned.go
deleted file mode 100644
index 1ce6062..0000000
--- a/vendor/golang.org/x/crypto/sha3/xor_unaligned.go
+++ /dev/null
@@ -1,68 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-//go:build (amd64 || 386 || ppc64le) && !purego
-// +build amd64 386 ppc64le
-// +build !purego
-
-package sha3
-
-import "unsafe"
-
-// A storageBuf is an aligned array of maxRate bytes.
-type storageBuf [maxRate / 8]uint64
-
-func (b *storageBuf) asBytes() *[maxRate]byte {
- return (*[maxRate]byte)(unsafe.Pointer(b))
-}
-
-// xorInUnaligned uses unaligned reads and writes to update d.a to contain d.a
-// XOR buf.
-func xorInUnaligned(d *state, buf []byte) {
- n := len(buf)
- bw := (*[maxRate / 8]uint64)(unsafe.Pointer(&buf[0]))[: n/8 : n/8]
- if n >= 72 {
- d.a[0] ^= bw[0]
- d.a[1] ^= bw[1]
- d.a[2] ^= bw[2]
- d.a[3] ^= bw[3]
- d.a[4] ^= bw[4]
- d.a[5] ^= bw[5]
- d.a[6] ^= bw[6]
- d.a[7] ^= bw[7]
- d.a[8] ^= bw[8]
- }
- if n >= 104 {
- d.a[9] ^= bw[9]
- d.a[10] ^= bw[10]
- d.a[11] ^= bw[11]
- d.a[12] ^= bw[12]
- }
- if n >= 136 {
- d.a[13] ^= bw[13]
- d.a[14] ^= bw[14]
- d.a[15] ^= bw[15]
- d.a[16] ^= bw[16]
- }
- if n >= 144 {
- d.a[17] ^= bw[17]
- }
- if n >= 168 {
- d.a[18] ^= bw[18]
- d.a[19] ^= bw[19]
- d.a[20] ^= bw[20]
- }
-}
-
-func copyOutUnaligned(d *state, buf []byte) {
- ab := (*[maxRate]uint8)(unsafe.Pointer(&d.a[0]))
- copy(buf, ab[:])
-}
-
-var (
- xorIn = xorInUnaligned
- copyOut = copyOutUnaligned
-)
-
-const xorImplementationUnaligned = "unaligned"
diff --git a/vendor/golang.org/x/crypto/tea/cipher.go b/vendor/golang.org/x/crypto/tea/cipher.go
deleted file mode 100644
index c1ff90e..0000000
--- a/vendor/golang.org/x/crypto/tea/cipher.go
+++ /dev/null
@@ -1,116 +0,0 @@
-// Copyright 2015 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package tea implements the TEA algorithm, as defined in Needham and
-// Wheeler's 1994 technical report, “TEA, a Tiny Encryption Algorithm”. See
-// http://www.cix.co.uk/~klockstone/tea.pdf for details.
-//
-// TEA is a legacy cipher and its short block size makes it vulnerable to
-// birthday bound attacks (see https://sweet32.info). It should only be used
-// where compatibility with legacy systems, not security, is the goal.
-//
-// Deprecated: any new system should use AES (from crypto/aes, if necessary in
-// an AEAD mode like crypto/cipher.NewGCM) or XChaCha20-Poly1305 (from
-// golang.org/x/crypto/chacha20poly1305).
-package tea
-
-import (
- "crypto/cipher"
- "encoding/binary"
- "errors"
-)
-
-const (
- // BlockSize is the size of a TEA block, in bytes.
- BlockSize = 8
-
- // KeySize is the size of a TEA key, in bytes.
- KeySize = 16
-
- // delta is the TEA key schedule constant.
- delta = 0x9e3779b9
-
- // numRounds is the standard number of rounds in TEA.
- numRounds = 64
-)
-
-// tea is an instance of the TEA cipher with a particular key.
-type tea struct {
- key [16]byte
- rounds int
-}
-
-// NewCipher returns an instance of the TEA cipher with the standard number of
-// rounds. The key argument must be 16 bytes long.
-func NewCipher(key []byte) (cipher.Block, error) {
- return NewCipherWithRounds(key, numRounds)
-}
-
-// NewCipherWithRounds returns an instance of the TEA cipher with a given
-// number of rounds, which must be even. The key argument must be 16 bytes
-// long.
-func NewCipherWithRounds(key []byte, rounds int) (cipher.Block, error) {
- if len(key) != 16 {
- return nil, errors.New("tea: incorrect key size")
- }
-
- if rounds&1 != 0 {
- return nil, errors.New("tea: odd number of rounds specified")
- }
-
- c := &tea{
- rounds: rounds,
- }
- copy(c.key[:], key)
-
- return c, nil
-}
-
-// BlockSize returns the TEA block size, which is eight bytes. It is necessary
-// to satisfy the Block interface in the package "crypto/cipher".
-func (*tea) BlockSize() int {
- return BlockSize
-}
-
-// Encrypt encrypts the 8 byte buffer src using the key in t and stores the
-// result in dst. Note that for amounts of data larger than a block, it is not
-// safe to just call Encrypt on successive blocks; instead, use an encryption
-// mode like CBC (see crypto/cipher/cbc.go).
-func (t *tea) Encrypt(dst, src []byte) {
- e := binary.BigEndian
- v0, v1 := e.Uint32(src), e.Uint32(src[4:])
- k0, k1, k2, k3 := e.Uint32(t.key[0:]), e.Uint32(t.key[4:]), e.Uint32(t.key[8:]), e.Uint32(t.key[12:])
-
- sum := uint32(0)
- delta := uint32(delta)
-
- for i := 0; i < t.rounds/2; i++ {
- sum += delta
- v0 += ((v1 << 4) + k0) ^ (v1 + sum) ^ ((v1 >> 5) + k1)
- v1 += ((v0 << 4) + k2) ^ (v0 + sum) ^ ((v0 >> 5) + k3)
- }
-
- e.PutUint32(dst, v0)
- e.PutUint32(dst[4:], v1)
-}
-
-// Decrypt decrypts the 8 byte buffer src using the key in t and stores the
-// result in dst.
-func (t *tea) Decrypt(dst, src []byte) {
- e := binary.BigEndian
- v0, v1 := e.Uint32(src), e.Uint32(src[4:])
- k0, k1, k2, k3 := e.Uint32(t.key[0:]), e.Uint32(t.key[4:]), e.Uint32(t.key[8:]), e.Uint32(t.key[12:])
-
- delta := uint32(delta)
- sum := delta * uint32(t.rounds/2) // in general, sum = delta * n
-
- for i := 0; i < t.rounds/2; i++ {
- v1 -= ((v0 << 4) + k2) ^ (v0 + sum) ^ ((v0 >> 5) + k3)
- v0 -= ((v1 << 4) + k0) ^ (v1 + sum) ^ ((v1 >> 5) + k1)
- sum -= delta
- }
-
- e.PutUint32(dst, v0)
- e.PutUint32(dst[4:], v1)
-}
diff --git a/vendor/golang.org/x/crypto/twofish/twofish.go b/vendor/golang.org/x/crypto/twofish/twofish.go
deleted file mode 100644
index 1197d75..0000000
--- a/vendor/golang.org/x/crypto/twofish/twofish.go
+++ /dev/null
@@ -1,348 +0,0 @@
-// Copyright 2011 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package twofish implements Bruce Schneier's Twofish encryption algorithm.
-//
-// Deprecated: Twofish is a legacy cipher and should not be used for new
-// applications. Also, this package does not and will not provide an optimized
-// implementation. Instead, use AES (from crypto/aes, if necessary in an AEAD
-// mode like crypto/cipher.NewGCM) or XChaCha20-Poly1305 (from
-// golang.org/x/crypto/chacha20poly1305).
-package twofish // import "golang.org/x/crypto/twofish"
-
-// Twofish is defined in https://www.schneier.com/paper-twofish-paper.pdf [TWOFISH]
-
-// This code is a port of the LibTom C implementation.
-// See http://libtom.org/?page=features&newsitems=5&whatfile=crypt.
-// LibTomCrypt is free for all purposes under the public domain.
-// It was heavily inspired by the go blowfish package.
-
-import "strconv"
-
-// BlockSize is the constant block size of Twofish.
-const BlockSize = 16
-
-const mdsPolynomial = 0x169 // x^8 + x^6 + x^5 + x^3 + 1, see [TWOFISH] 4.2
-const rsPolynomial = 0x14d // x^8 + x^6 + x^3 + x^2 + 1, see [TWOFISH] 4.3
-
-// A Cipher is an instance of Twofish encryption using a particular key.
-type Cipher struct {
- s [4][256]uint32
- k [40]uint32
-}
-
-type KeySizeError int
-
-func (k KeySizeError) Error() string {
- return "crypto/twofish: invalid key size " + strconv.Itoa(int(k))
-}
-
-// NewCipher creates and returns a Cipher.
-// The key argument should be the Twofish key, 16, 24 or 32 bytes.
-func NewCipher(key []byte) (*Cipher, error) {
- keylen := len(key)
-
- if keylen != 16 && keylen != 24 && keylen != 32 {
- return nil, KeySizeError(keylen)
- }
-
- // k is the number of 64 bit words in key
- k := keylen / 8
-
- // Create the S[..] words
- var S [4 * 4]byte
- for i := 0; i < k; i++ {
- // Computes [y0 y1 y2 y3] = rs . [x0 x1 x2 x3 x4 x5 x6 x7]
- for j, rsRow := range rs {
- for k, rsVal := range rsRow {
- S[4*i+j] ^= gfMult(key[8*i+k], rsVal, rsPolynomial)
- }
- }
- }
-
- // Calculate subkeys
- c := new(Cipher)
- var tmp [4]byte
- for i := byte(0); i < 20; i++ {
- // A = h(p * 2x, Me)
- for j := range tmp {
- tmp[j] = 2 * i
- }
- A := h(tmp[:], key, 0)
-
- // B = rolc(h(p * (2x + 1), Mo), 8)
- for j := range tmp {
- tmp[j] = 2*i + 1
- }
- B := h(tmp[:], key, 1)
- B = rol(B, 8)
-
- c.k[2*i] = A + B
-
- // K[2i+1] = (A + 2B) <<< 9
- c.k[2*i+1] = rol(2*B+A, 9)
- }
-
- // Calculate sboxes
- switch k {
- case 2:
- for i := range c.s[0] {
- c.s[0][i] = mdsColumnMult(sbox[1][sbox[0][sbox[0][byte(i)]^S[0]]^S[4]], 0)
- c.s[1][i] = mdsColumnMult(sbox[0][sbox[0][sbox[1][byte(i)]^S[1]]^S[5]], 1)
- c.s[2][i] = mdsColumnMult(sbox[1][sbox[1][sbox[0][byte(i)]^S[2]]^S[6]], 2)
- c.s[3][i] = mdsColumnMult(sbox[0][sbox[1][sbox[1][byte(i)]^S[3]]^S[7]], 3)
- }
- case 3:
- for i := range c.s[0] {
- c.s[0][i] = mdsColumnMult(sbox[1][sbox[0][sbox[0][sbox[1][byte(i)]^S[0]]^S[4]]^S[8]], 0)
- c.s[1][i] = mdsColumnMult(sbox[0][sbox[0][sbox[1][sbox[1][byte(i)]^S[1]]^S[5]]^S[9]], 1)
- c.s[2][i] = mdsColumnMult(sbox[1][sbox[1][sbox[0][sbox[0][byte(i)]^S[2]]^S[6]]^S[10]], 2)
- c.s[3][i] = mdsColumnMult(sbox[0][sbox[1][sbox[1][sbox[0][byte(i)]^S[3]]^S[7]]^S[11]], 3)
- }
- default:
- for i := range c.s[0] {
- c.s[0][i] = mdsColumnMult(sbox[1][sbox[0][sbox[0][sbox[1][sbox[1][byte(i)]^S[0]]^S[4]]^S[8]]^S[12]], 0)
- c.s[1][i] = mdsColumnMult(sbox[0][sbox[0][sbox[1][sbox[1][sbox[0][byte(i)]^S[1]]^S[5]]^S[9]]^S[13]], 1)
- c.s[2][i] = mdsColumnMult(sbox[1][sbox[1][sbox[0][sbox[0][sbox[0][byte(i)]^S[2]]^S[6]]^S[10]]^S[14]], 2)
- c.s[3][i] = mdsColumnMult(sbox[0][sbox[1][sbox[1][sbox[0][sbox[1][byte(i)]^S[3]]^S[7]]^S[11]]^S[15]], 3)
- }
- }
-
- return c, nil
-}
-
-// BlockSize returns the Twofish block size, 16 bytes.
-func (c *Cipher) BlockSize() int { return BlockSize }
-
-// store32l stores src in dst in little-endian form.
-func store32l(dst []byte, src uint32) {
- dst[0] = byte(src)
- dst[1] = byte(src >> 8)
- dst[2] = byte(src >> 16)
- dst[3] = byte(src >> 24)
- return
-}
-
-// load32l reads a little-endian uint32 from src.
-func load32l(src []byte) uint32 {
- return uint32(src[0]) | uint32(src[1])<<8 | uint32(src[2])<<16 | uint32(src[3])<<24
-}
-
-// rol returns x after a left circular rotation of y bits.
-func rol(x, y uint32) uint32 {
- return (x << (y & 31)) | (x >> (32 - (y & 31)))
-}
-
-// ror returns x after a right circular rotation of y bits.
-func ror(x, y uint32) uint32 {
- return (x >> (y & 31)) | (x << (32 - (y & 31)))
-}
-
-// The RS matrix. See [TWOFISH] 4.3
-var rs = [4][8]byte{
- {0x01, 0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E},
- {0xA4, 0x56, 0x82, 0xF3, 0x1E, 0xC6, 0x68, 0xE5},
- {0x02, 0xA1, 0xFC, 0xC1, 0x47, 0xAE, 0x3D, 0x19},
- {0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E, 0x03},
-}
-
-// sbox tables
-var sbox = [2][256]byte{
- {
- 0xa9, 0x67, 0xb3, 0xe8, 0x04, 0xfd, 0xa3, 0x76, 0x9a, 0x92, 0x80, 0x78, 0xe4, 0xdd, 0xd1, 0x38,
- 0x0d, 0xc6, 0x35, 0x98, 0x18, 0xf7, 0xec, 0x6c, 0x43, 0x75, 0x37, 0x26, 0xfa, 0x13, 0x94, 0x48,
- 0xf2, 0xd0, 0x8b, 0x30, 0x84, 0x54, 0xdf, 0x23, 0x19, 0x5b, 0x3d, 0x59, 0xf3, 0xae, 0xa2, 0x82,
- 0x63, 0x01, 0x83, 0x2e, 0xd9, 0x51, 0x9b, 0x7c, 0xa6, 0xeb, 0xa5, 0xbe, 0x16, 0x0c, 0xe3, 0x61,
- 0xc0, 0x8c, 0x3a, 0xf5, 0x73, 0x2c, 0x25, 0x0b, 0xbb, 0x4e, 0x89, 0x6b, 0x53, 0x6a, 0xb4, 0xf1,
- 0xe1, 0xe6, 0xbd, 0x45, 0xe2, 0xf4, 0xb6, 0x66, 0xcc, 0x95, 0x03, 0x56, 0xd4, 0x1c, 0x1e, 0xd7,
- 0xfb, 0xc3, 0x8e, 0xb5, 0xe9, 0xcf, 0xbf, 0xba, 0xea, 0x77, 0x39, 0xaf, 0x33, 0xc9, 0x62, 0x71,
- 0x81, 0x79, 0x09, 0xad, 0x24, 0xcd, 0xf9, 0xd8, 0xe5, 0xc5, 0xb9, 0x4d, 0x44, 0x08, 0x86, 0xe7,
- 0xa1, 0x1d, 0xaa, 0xed, 0x06, 0x70, 0xb2, 0xd2, 0x41, 0x7b, 0xa0, 0x11, 0x31, 0xc2, 0x27, 0x90,
- 0x20, 0xf6, 0x60, 0xff, 0x96, 0x5c, 0xb1, 0xab, 0x9e, 0x9c, 0x52, 0x1b, 0x5f, 0x93, 0x0a, 0xef,
- 0x91, 0x85, 0x49, 0xee, 0x2d, 0x4f, 0x8f, 0x3b, 0x47, 0x87, 0x6d, 0x46, 0xd6, 0x3e, 0x69, 0x64,
- 0x2a, 0xce, 0xcb, 0x2f, 0xfc, 0x97, 0x05, 0x7a, 0xac, 0x7f, 0xd5, 0x1a, 0x4b, 0x0e, 0xa7, 0x5a,
- 0x28, 0x14, 0x3f, 0x29, 0x88, 0x3c, 0x4c, 0x02, 0xb8, 0xda, 0xb0, 0x17, 0x55, 0x1f, 0x8a, 0x7d,
- 0x57, 0xc7, 0x8d, 0x74, 0xb7, 0xc4, 0x9f, 0x72, 0x7e, 0x15, 0x22, 0x12, 0x58, 0x07, 0x99, 0x34,
- 0x6e, 0x50, 0xde, 0x68, 0x65, 0xbc, 0xdb, 0xf8, 0xc8, 0xa8, 0x2b, 0x40, 0xdc, 0xfe, 0x32, 0xa4,
- 0xca, 0x10, 0x21, 0xf0, 0xd3, 0x5d, 0x0f, 0x00, 0x6f, 0x9d, 0x36, 0x42, 0x4a, 0x5e, 0xc1, 0xe0,
- },
- {
- 0x75, 0xf3, 0xc6, 0xf4, 0xdb, 0x7b, 0xfb, 0xc8, 0x4a, 0xd3, 0xe6, 0x6b, 0x45, 0x7d, 0xe8, 0x4b,
- 0xd6, 0x32, 0xd8, 0xfd, 0x37, 0x71, 0xf1, 0xe1, 0x30, 0x0f, 0xf8, 0x1b, 0x87, 0xfa, 0x06, 0x3f,
- 0x5e, 0xba, 0xae, 0x5b, 0x8a, 0x00, 0xbc, 0x9d, 0x6d, 0xc1, 0xb1, 0x0e, 0x80, 0x5d, 0xd2, 0xd5,
- 0xa0, 0x84, 0x07, 0x14, 0xb5, 0x90, 0x2c, 0xa3, 0xb2, 0x73, 0x4c, 0x54, 0x92, 0x74, 0x36, 0x51,
- 0x38, 0xb0, 0xbd, 0x5a, 0xfc, 0x60, 0x62, 0x96, 0x6c, 0x42, 0xf7, 0x10, 0x7c, 0x28, 0x27, 0x8c,
- 0x13, 0x95, 0x9c, 0xc7, 0x24, 0x46, 0x3b, 0x70, 0xca, 0xe3, 0x85, 0xcb, 0x11, 0xd0, 0x93, 0xb8,
- 0xa6, 0x83, 0x20, 0xff, 0x9f, 0x77, 0xc3, 0xcc, 0x03, 0x6f, 0x08, 0xbf, 0x40, 0xe7, 0x2b, 0xe2,
- 0x79, 0x0c, 0xaa, 0x82, 0x41, 0x3a, 0xea, 0xb9, 0xe4, 0x9a, 0xa4, 0x97, 0x7e, 0xda, 0x7a, 0x17,
- 0x66, 0x94, 0xa1, 0x1d, 0x3d, 0xf0, 0xde, 0xb3, 0x0b, 0x72, 0xa7, 0x1c, 0xef, 0xd1, 0x53, 0x3e,
- 0x8f, 0x33, 0x26, 0x5f, 0xec, 0x76, 0x2a, 0x49, 0x81, 0x88, 0xee, 0x21, 0xc4, 0x1a, 0xeb, 0xd9,
- 0xc5, 0x39, 0x99, 0xcd, 0xad, 0x31, 0x8b, 0x01, 0x18, 0x23, 0xdd, 0x1f, 0x4e, 0x2d, 0xf9, 0x48,
- 0x4f, 0xf2, 0x65, 0x8e, 0x78, 0x5c, 0x58, 0x19, 0x8d, 0xe5, 0x98, 0x57, 0x67, 0x7f, 0x05, 0x64,
- 0xaf, 0x63, 0xb6, 0xfe, 0xf5, 0xb7, 0x3c, 0xa5, 0xce, 0xe9, 0x68, 0x44, 0xe0, 0x4d, 0x43, 0x69,
- 0x29, 0x2e, 0xac, 0x15, 0x59, 0xa8, 0x0a, 0x9e, 0x6e, 0x47, 0xdf, 0x34, 0x35, 0x6a, 0xcf, 0xdc,
- 0x22, 0xc9, 0xc0, 0x9b, 0x89, 0xd4, 0xed, 0xab, 0x12, 0xa2, 0x0d, 0x52, 0xbb, 0x02, 0x2f, 0xa9,
- 0xd7, 0x61, 0x1e, 0xb4, 0x50, 0x04, 0xf6, 0xc2, 0x16, 0x25, 0x86, 0x56, 0x55, 0x09, 0xbe, 0x91,
- },
-}
-
-// gfMult returns a·b in GF(2^8)/p
-func gfMult(a, b byte, p uint32) byte {
- B := [2]uint32{0, uint32(b)}
- P := [2]uint32{0, p}
- var result uint32
-
- // branchless GF multiplier
- for i := 0; i < 7; i++ {
- result ^= B[a&1]
- a >>= 1
- B[1] = P[B[1]>>7] ^ (B[1] << 1)
- }
- result ^= B[a&1]
- return byte(result)
-}
-
-// mdsColumnMult calculates y{col} where [y0 y1 y2 y3] = MDS · [x0]
-func mdsColumnMult(in byte, col int) uint32 {
- mul01 := in
- mul5B := gfMult(in, 0x5B, mdsPolynomial)
- mulEF := gfMult(in, 0xEF, mdsPolynomial)
-
- switch col {
- case 0:
- return uint32(mul01) | uint32(mul5B)<<8 | uint32(mulEF)<<16 | uint32(mulEF)<<24
- case 1:
- return uint32(mulEF) | uint32(mulEF)<<8 | uint32(mul5B)<<16 | uint32(mul01)<<24
- case 2:
- return uint32(mul5B) | uint32(mulEF)<<8 | uint32(mul01)<<16 | uint32(mulEF)<<24
- case 3:
- return uint32(mul5B) | uint32(mul01)<<8 | uint32(mulEF)<<16 | uint32(mul5B)<<24
- }
-
- panic("unreachable")
-}
-
-// h implements the S-box generation function. See [TWOFISH] 4.3.5
-func h(in, key []byte, offset int) uint32 {
- var y [4]byte
- for x := range y {
- y[x] = in[x]
- }
- switch len(key) / 8 {
- case 4:
- y[0] = sbox[1][y[0]] ^ key[4*(6+offset)+0]
- y[1] = sbox[0][y[1]] ^ key[4*(6+offset)+1]
- y[2] = sbox[0][y[2]] ^ key[4*(6+offset)+2]
- y[3] = sbox[1][y[3]] ^ key[4*(6+offset)+3]
- fallthrough
- case 3:
- y[0] = sbox[1][y[0]] ^ key[4*(4+offset)+0]
- y[1] = sbox[1][y[1]] ^ key[4*(4+offset)+1]
- y[2] = sbox[0][y[2]] ^ key[4*(4+offset)+2]
- y[3] = sbox[0][y[3]] ^ key[4*(4+offset)+3]
- fallthrough
- case 2:
- y[0] = sbox[1][sbox[0][sbox[0][y[0]]^key[4*(2+offset)+0]]^key[4*(0+offset)+0]]
- y[1] = sbox[0][sbox[0][sbox[1][y[1]]^key[4*(2+offset)+1]]^key[4*(0+offset)+1]]
- y[2] = sbox[1][sbox[1][sbox[0][y[2]]^key[4*(2+offset)+2]]^key[4*(0+offset)+2]]
- y[3] = sbox[0][sbox[1][sbox[1][y[3]]^key[4*(2+offset)+3]]^key[4*(0+offset)+3]]
- }
- // [y0 y1 y2 y3] = MDS . [x0 x1 x2 x3]
- var mdsMult uint32
- for i := range y {
- mdsMult ^= mdsColumnMult(y[i], i)
- }
- return mdsMult
-}
-
-// Encrypt encrypts a 16-byte block from src to dst, which may overlap.
-// Note that for amounts of data larger than a block,
-// it is not safe to just call Encrypt on successive blocks;
-// instead, use an encryption mode like CBC (see crypto/cipher/cbc.go).
-func (c *Cipher) Encrypt(dst, src []byte) {
- S1 := c.s[0]
- S2 := c.s[1]
- S3 := c.s[2]
- S4 := c.s[3]
-
- // Load input
- ia := load32l(src[0:4])
- ib := load32l(src[4:8])
- ic := load32l(src[8:12])
- id := load32l(src[12:16])
-
- // Pre-whitening
- ia ^= c.k[0]
- ib ^= c.k[1]
- ic ^= c.k[2]
- id ^= c.k[3]
-
- for i := 0; i < 8; i++ {
- k := c.k[8+i*4 : 12+i*4]
- t2 := S2[byte(ib)] ^ S3[byte(ib>>8)] ^ S4[byte(ib>>16)] ^ S1[byte(ib>>24)]
- t1 := S1[byte(ia)] ^ S2[byte(ia>>8)] ^ S3[byte(ia>>16)] ^ S4[byte(ia>>24)] + t2
- ic = ror(ic^(t1+k[0]), 1)
- id = rol(id, 1) ^ (t2 + t1 + k[1])
-
- t2 = S2[byte(id)] ^ S3[byte(id>>8)] ^ S4[byte(id>>16)] ^ S1[byte(id>>24)]
- t1 = S1[byte(ic)] ^ S2[byte(ic>>8)] ^ S3[byte(ic>>16)] ^ S4[byte(ic>>24)] + t2
- ia = ror(ia^(t1+k[2]), 1)
- ib = rol(ib, 1) ^ (t2 + t1 + k[3])
- }
-
- // Output with "undo last swap"
- ta := ic ^ c.k[4]
- tb := id ^ c.k[5]
- tc := ia ^ c.k[6]
- td := ib ^ c.k[7]
-
- store32l(dst[0:4], ta)
- store32l(dst[4:8], tb)
- store32l(dst[8:12], tc)
- store32l(dst[12:16], td)
-}
-
-// Decrypt decrypts a 16-byte block from src to dst, which may overlap.
-func (c *Cipher) Decrypt(dst, src []byte) {
- S1 := c.s[0]
- S2 := c.s[1]
- S3 := c.s[2]
- S4 := c.s[3]
-
- // Load input
- ta := load32l(src[0:4])
- tb := load32l(src[4:8])
- tc := load32l(src[8:12])
- td := load32l(src[12:16])
-
- // Undo undo final swap
- ia := tc ^ c.k[6]
- ib := td ^ c.k[7]
- ic := ta ^ c.k[4]
- id := tb ^ c.k[5]
-
- for i := 8; i > 0; i-- {
- k := c.k[4+i*4 : 8+i*4]
- t2 := S2[byte(id)] ^ S3[byte(id>>8)] ^ S4[byte(id>>16)] ^ S1[byte(id>>24)]
- t1 := S1[byte(ic)] ^ S2[byte(ic>>8)] ^ S3[byte(ic>>16)] ^ S4[byte(ic>>24)] + t2
- ia = rol(ia, 1) ^ (t1 + k[2])
- ib = ror(ib^(t2+t1+k[3]), 1)
-
- t2 = S2[byte(ib)] ^ S3[byte(ib>>8)] ^ S4[byte(ib>>16)] ^ S1[byte(ib>>24)]
- t1 = S1[byte(ia)] ^ S2[byte(ia>>8)] ^ S3[byte(ia>>16)] ^ S4[byte(ia>>24)] + t2
- ic = rol(ic, 1) ^ (t1 + k[0])
- id = ror(id^(t2+t1+k[1]), 1)
- }
-
- // Undo pre-whitening
- ia ^= c.k[0]
- ib ^= c.k[1]
- ic ^= c.k[2]
- id ^= c.k[3]
-
- store32l(dst[0:4], ia)
- store32l(dst[4:8], ib)
- store32l(dst[8:12], ic)
- store32l(dst[12:16], id)
-}
diff --git a/vendor/golang.org/x/crypto/xtea/block.go b/vendor/golang.org/x/crypto/xtea/block.go
deleted file mode 100644
index fcb4e4d..0000000
--- a/vendor/golang.org/x/crypto/xtea/block.go
+++ /dev/null
@@ -1,66 +0,0 @@
-// Copyright 2009 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-/*
- Implementation adapted from Needham and Wheeler's paper:
- http://www.cix.co.uk/~klockstone/xtea.pdf
-
- A precalculated look up table is used during encryption/decryption for values that are based purely on the key.
-*/
-
-package xtea
-
-// XTEA is based on 64 rounds.
-const numRounds = 64
-
-// blockToUint32 reads an 8 byte slice into two uint32s.
-// The block is treated as big endian.
-func blockToUint32(src []byte) (uint32, uint32) {
- r0 := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
- r1 := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
- return r0, r1
-}
-
-// uint32ToBlock writes two uint32s into an 8 byte data block.
-// Values are written as big endian.
-func uint32ToBlock(v0, v1 uint32, dst []byte) {
- dst[0] = byte(v0 >> 24)
- dst[1] = byte(v0 >> 16)
- dst[2] = byte(v0 >> 8)
- dst[3] = byte(v0)
- dst[4] = byte(v1 >> 24)
- dst[5] = byte(v1 >> 16)
- dst[6] = byte(v1 >> 8)
- dst[7] = byte(v1 >> 0)
-}
-
-// encryptBlock encrypts a single 8 byte block using XTEA.
-func encryptBlock(c *Cipher, dst, src []byte) {
- v0, v1 := blockToUint32(src)
-
- // Two rounds of XTEA applied per loop
- for i := 0; i < numRounds; {
- v0 += ((v1<<4 ^ v1>>5) + v1) ^ c.table[i]
- i++
- v1 += ((v0<<4 ^ v0>>5) + v0) ^ c.table[i]
- i++
- }
-
- uint32ToBlock(v0, v1, dst)
-}
-
-// decryptBlock decrypts a single 8 byte block using XTEA.
-func decryptBlock(c *Cipher, dst, src []byte) {
- v0, v1 := blockToUint32(src)
-
- // Two rounds of XTEA applied per loop
- for i := numRounds; i > 0; {
- i--
- v1 -= ((v0<<4 ^ v0>>5) + v0) ^ c.table[i]
- i--
- v0 -= ((v1<<4 ^ v1>>5) + v1) ^ c.table[i]
- }
-
- uint32ToBlock(v0, v1, dst)
-}
diff --git a/vendor/golang.org/x/crypto/xtea/cipher.go b/vendor/golang.org/x/crypto/xtea/cipher.go
deleted file mode 100644
index a4c2fd0..0000000
--- a/vendor/golang.org/x/crypto/xtea/cipher.go
+++ /dev/null
@@ -1,90 +0,0 @@
-// Copyright 2009 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// Package xtea implements XTEA encryption, as defined in Needham and Wheeler's
-// 1997 technical report, "Tea extensions."
-//
-// XTEA is a legacy cipher and its short block size makes it vulnerable to
-// birthday bound attacks (see https://sweet32.info). It should only be used
-// where compatibility with legacy systems, not security, is the goal.
-//
-// Deprecated: any new system should use AES (from crypto/aes, if necessary in
-// an AEAD mode like crypto/cipher.NewGCM) or XChaCha20-Poly1305 (from
-// golang.org/x/crypto/chacha20poly1305).
-package xtea // import "golang.org/x/crypto/xtea"
-
-// For details, see http://www.cix.co.uk/~klockstone/xtea.pdf
-
-import "strconv"
-
-// The XTEA block size in bytes.
-const BlockSize = 8
-
-// A Cipher is an instance of an XTEA cipher using a particular key.
-type Cipher struct {
- // table contains a series of precalculated values that are used each round.
- table [64]uint32
-}
-
-type KeySizeError int
-
-func (k KeySizeError) Error() string {
- return "crypto/xtea: invalid key size " + strconv.Itoa(int(k))
-}
-
-// NewCipher creates and returns a new Cipher.
-// The key argument should be the XTEA key.
-// XTEA only supports 128 bit (16 byte) keys.
-func NewCipher(key []byte) (*Cipher, error) {
- k := len(key)
- switch k {
- default:
- return nil, KeySizeError(k)
- case 16:
- break
- }
-
- c := new(Cipher)
- initCipher(c, key)
-
- return c, nil
-}
-
-// BlockSize returns the XTEA block size, 8 bytes.
-// It is necessary to satisfy the Block interface in the
-// package "crypto/cipher".
-func (c *Cipher) BlockSize() int { return BlockSize }
-
-// Encrypt encrypts the 8 byte buffer src using the key and stores the result in dst.
-// Note that for amounts of data larger than a block,
-// it is not safe to just call Encrypt on successive blocks;
-// instead, use an encryption mode like CBC (see crypto/cipher/cbc.go).
-func (c *Cipher) Encrypt(dst, src []byte) { encryptBlock(c, dst, src) }
-
-// Decrypt decrypts the 8 byte buffer src using the key and stores the result in dst.
-func (c *Cipher) Decrypt(dst, src []byte) { decryptBlock(c, dst, src) }
-
-// initCipher initializes the cipher context by creating a look up table
-// of precalculated values that are based on the key.
-func initCipher(c *Cipher, key []byte) {
- // Load the key into four uint32s
- var k [4]uint32
- for i := 0; i < len(k); i++ {
- j := i << 2 // Multiply by 4
- k[i] = uint32(key[j+0])<<24 | uint32(key[j+1])<<16 | uint32(key[j+2])<<8 | uint32(key[j+3])
- }
-
- // Precalculate the table
- const delta = 0x9E3779B9
- var sum uint32
-
- // Two rounds of XTEA applied per loop
- for i := 0; i < numRounds; {
- c.table[i] = sum + k[sum&3]
- i++
- sum += delta
- c.table[i] = sum + k[(sum>>11)&3]
- i++
- }
-}
diff --git a/vendor/golang.org/x/net/AUTHORS b/vendor/golang.org/x/net/AUTHORS
deleted file mode 100644
index 15167cd..0000000
--- a/vendor/golang.org/x/net/AUTHORS
+++ /dev/null
@@ -1,3 +0,0 @@
-# This source code refers to The Go Authors for copyright purposes.
-# The master list of authors is in the main Go distribution,
-# visible at http://tip.golang.org/AUTHORS.
diff --git a/vendor/golang.org/x/net/CONTRIBUTORS b/vendor/golang.org/x/net/CONTRIBUTORS
deleted file mode 100644
index 1c4577e..0000000
--- a/vendor/golang.org/x/net/CONTRIBUTORS
+++ /dev/null
@@ -1,3 +0,0 @@
-# This source code was written by the Go contributors.
-# The master list of contributors is in the main Go distribution,
-# visible at http://tip.golang.org/CONTRIBUTORS.
diff --git a/vendor/golang.org/x/net/LICENSE b/vendor/golang.org/x/net/LICENSE
deleted file mode 100644
index 6a66aea..0000000
--- a/vendor/golang.org/x/net/LICENSE
+++ /dev/null
@@ -1,27 +0,0 @@
-Copyright (c) 2009 The Go Authors. All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- * Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
- * Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/vendor/golang.org/x/net/PATENTS b/vendor/golang.org/x/net/PATENTS
deleted file mode 100644
index 7330990..0000000
--- a/vendor/golang.org/x/net/PATENTS
+++ /dev/null
@@ -1,22 +0,0 @@
-Additional IP Rights Grant (Patents)
-
-"This implementation" means the copyrightable works distributed by
-Google as part of the Go project.
-
-Google hereby grants to You a perpetual, worldwide, non-exclusive,
-no-charge, royalty-free, irrevocable (except as stated in this section)
-patent license to make, have made, use, offer to sell, sell, import,
-transfer and otherwise run, modify and propagate the contents of this
-implementation of Go, where such license applies only to those patent
-claims, both currently owned or controlled by Google and acquired in
-the future, licensable by Google that are necessarily infringed by this
-implementation of Go. This grant does not include claims that would be
-infringed only as a consequence of further modification of this
-implementation. If you or your agent or exclusive licensee institute or
-order or agree to the institution of patent litigation against any
-entity (including a cross-claim or counterclaim in a lawsuit) alleging
-that this implementation of Go or any code incorporated within this
-implementation of Go constitutes direct or contributory patent
-infringement, or inducement of patent infringement, then any patent
-rights granted to you under this License for this implementation of Go
-shall terminate as of the date such litigation is filed.
diff --git a/vendor/golang.org/x/net/bpf/asm.go b/vendor/golang.org/x/net/bpf/asm.go
deleted file mode 100644
index 15e21b1..0000000
--- a/vendor/golang.org/x/net/bpf/asm.go
+++ /dev/null
@@ -1,41 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package bpf
-
-import "fmt"
-
-// Assemble converts insts into raw instructions suitable for loading
-// into a BPF virtual machine.
-//
-// Currently, no optimization is attempted, the assembled program flow
-// is exactly as provided.
-func Assemble(insts []Instruction) ([]RawInstruction, error) {
- ret := make([]RawInstruction, len(insts))
- var err error
- for i, inst := range insts {
- ret[i], err = inst.Assemble()
- if err != nil {
- return nil, fmt.Errorf("assembling instruction %d: %s", i+1, err)
- }
- }
- return ret, nil
-}
-
-// Disassemble attempts to parse raw back into
-// Instructions. Unrecognized RawInstructions are assumed to be an
-// extension not implemented by this package, and are passed through
-// unchanged to the output. The allDecoded value reports whether insts
-// contains no RawInstructions.
-func Disassemble(raw []RawInstruction) (insts []Instruction, allDecoded bool) {
- insts = make([]Instruction, len(raw))
- allDecoded = true
- for i, r := range raw {
- insts[i] = r.Disassemble()
- if _, ok := insts[i].(RawInstruction); ok {
- allDecoded = false
- }
- }
- return insts, allDecoded
-}
diff --git a/vendor/golang.org/x/net/bpf/constants.go b/vendor/golang.org/x/net/bpf/constants.go
deleted file mode 100644
index 12f3ee8..0000000
--- a/vendor/golang.org/x/net/bpf/constants.go
+++ /dev/null
@@ -1,222 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package bpf
-
-// A Register is a register of the BPF virtual machine.
-type Register uint16
-
-const (
- // RegA is the accumulator register. RegA is always the
- // destination register of ALU operations.
- RegA Register = iota
- // RegX is the indirection register, used by LoadIndirect
- // operations.
- RegX
-)
-
-// An ALUOp is an arithmetic or logic operation.
-type ALUOp uint16
-
-// ALU binary operation types.
-const (
- ALUOpAdd ALUOp = iota << 4
- ALUOpSub
- ALUOpMul
- ALUOpDiv
- ALUOpOr
- ALUOpAnd
- ALUOpShiftLeft
- ALUOpShiftRight
- aluOpNeg // Not exported because it's the only unary ALU operation, and gets its own instruction type.
- ALUOpMod
- ALUOpXor
-)
-
-// A JumpTest is a comparison operator used in conditional jumps.
-type JumpTest uint16
-
-// Supported operators for conditional jumps.
-// K can be RegX for JumpIfX
-const (
- // K == A
- JumpEqual JumpTest = iota
- // K != A
- JumpNotEqual
- // K > A
- JumpGreaterThan
- // K < A
- JumpLessThan
- // K >= A
- JumpGreaterOrEqual
- // K <= A
- JumpLessOrEqual
- // K & A != 0
- JumpBitsSet
- // K & A == 0
- JumpBitsNotSet
-)
-
-// An Extension is a function call provided by the kernel that
-// performs advanced operations that are expensive or impossible
-// within the BPF virtual machine.
-//
-// Extensions are only implemented by the Linux kernel.
-//
-// TODO: should we prune this list? Some of these extensions seem
-// either broken or near-impossible to use correctly, whereas other
-// (len, random, ifindex) are quite useful.
-type Extension int
-
-// Extension functions available in the Linux kernel.
-const (
- // extOffset is the negative maximum number of instructions used
- // to load instructions by overloading the K argument.
- extOffset = -0x1000
- // ExtLen returns the length of the packet.
- ExtLen Extension = 1
- // ExtProto returns the packet's L3 protocol type.
- ExtProto Extension = 0
- // ExtType returns the packet's type (skb->pkt_type in the kernel)
- //
- // TODO: better documentation. How nice an API do we want to
- // provide for these esoteric extensions?
- ExtType Extension = 4
- // ExtPayloadOffset returns the offset of the packet payload, or
- // the first protocol header that the kernel does not know how to
- // parse.
- ExtPayloadOffset Extension = 52
- // ExtInterfaceIndex returns the index of the interface on which
- // the packet was received.
- ExtInterfaceIndex Extension = 8
- // ExtNetlinkAttr returns the netlink attribute of type X at
- // offset A.
- ExtNetlinkAttr Extension = 12
- // ExtNetlinkAttrNested returns the nested netlink attribute of
- // type X at offset A.
- ExtNetlinkAttrNested Extension = 16
- // ExtMark returns the packet's mark value.
- ExtMark Extension = 20
- // ExtQueue returns the packet's assigned hardware queue.
- ExtQueue Extension = 24
- // ExtLinkLayerType returns the packet's hardware address type
- // (e.g. Ethernet, Infiniband).
- ExtLinkLayerType Extension = 28
- // ExtRXHash returns the packets receive hash.
- //
- // TODO: figure out what this rxhash actually is.
- ExtRXHash Extension = 32
- // ExtCPUID returns the ID of the CPU processing the current
- // packet.
- ExtCPUID Extension = 36
- // ExtVLANTag returns the packet's VLAN tag.
- ExtVLANTag Extension = 44
- // ExtVLANTagPresent returns non-zero if the packet has a VLAN
- // tag.
- //
- // TODO: I think this might be a lie: it reads bit 0x1000 of the
- // VLAN header, which changed meaning in recent revisions of the
- // spec - this extension may now return meaningless information.
- ExtVLANTagPresent Extension = 48
- // ExtVLANProto returns 0x8100 if the frame has a VLAN header,
- // 0x88a8 if the frame has a "Q-in-Q" double VLAN header, or some
- // other value if no VLAN information is present.
- ExtVLANProto Extension = 60
- // ExtRand returns a uniformly random uint32.
- ExtRand Extension = 56
-)
-
-// The following gives names to various bit patterns used in opcode construction.
-
-const (
- opMaskCls uint16 = 0x7
- // opClsLoad masks
- opMaskLoadDest = 0x01
- opMaskLoadWidth = 0x18
- opMaskLoadMode = 0xe0
- // opClsALU & opClsJump
- opMaskOperand = 0x08
- opMaskOperator = 0xf0
-)
-
-const (
- // +---------------+-----------------+---+---+---+
- // | AddrMode (3b) | LoadWidth (2b) | 0 | 0 | 0 |
- // +---------------+-----------------+---+---+---+
- opClsLoadA uint16 = iota
- // +---------------+-----------------+---+---+---+
- // | AddrMode (3b) | LoadWidth (2b) | 0 | 0 | 1 |
- // +---------------+-----------------+---+---+---+
- opClsLoadX
- // +---+---+---+---+---+---+---+---+
- // | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
- // +---+---+---+---+---+---+---+---+
- opClsStoreA
- // +---+---+---+---+---+---+---+---+
- // | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
- // +---+---+---+---+---+---+---+---+
- opClsStoreX
- // +---------------+-----------------+---+---+---+
- // | Operator (4b) | OperandSrc (1b) | 1 | 0 | 0 |
- // +---------------+-----------------+---+---+---+
- opClsALU
- // +-----------------------------+---+---+---+---+
- // | TestOperator (4b) | 0 | 1 | 0 | 1 |
- // +-----------------------------+---+---+---+---+
- opClsJump
- // +---+-------------------------+---+---+---+---+
- // | 0 | 0 | 0 | RetSrc (1b) | 0 | 1 | 1 | 0 |
- // +---+-------------------------+---+---+---+---+
- opClsReturn
- // +---+-------------------------+---+---+---+---+
- // | 0 | 0 | 0 | TXAorTAX (1b) | 0 | 1 | 1 | 1 |
- // +---+-------------------------+---+---+---+---+
- opClsMisc
-)
-
-const (
- opAddrModeImmediate uint16 = iota << 5
- opAddrModeAbsolute
- opAddrModeIndirect
- opAddrModeScratch
- opAddrModePacketLen // actually an extension, not an addressing mode.
- opAddrModeMemShift
-)
-
-const (
- opLoadWidth4 uint16 = iota << 3
- opLoadWidth2
- opLoadWidth1
-)
-
-// Operand for ALU and Jump instructions
-type opOperand uint16
-
-// Supported operand sources.
-const (
- opOperandConstant opOperand = iota << 3
- opOperandX
-)
-
-// An jumpOp is a conditional jump condition.
-type jumpOp uint16
-
-// Supported jump conditions.
-const (
- opJumpAlways jumpOp = iota << 4
- opJumpEqual
- opJumpGT
- opJumpGE
- opJumpSet
-)
-
-const (
- opRetSrcConstant uint16 = iota << 4
- opRetSrcA
-)
-
-const (
- opMiscTAX = 0x00
- opMiscTXA = 0x80
-)
diff --git a/vendor/golang.org/x/net/bpf/doc.go b/vendor/golang.org/x/net/bpf/doc.go
deleted file mode 100644
index ae62feb..0000000
--- a/vendor/golang.org/x/net/bpf/doc.go
+++ /dev/null
@@ -1,82 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-/*
-
-Package bpf implements marshaling and unmarshaling of programs for the
-Berkeley Packet Filter virtual machine, and provides a Go implementation
-of the virtual machine.
-
-BPF's main use is to specify a packet filter for network taps, so that
-the kernel doesn't have to expensively copy every packet it sees to
-userspace. However, it's been repurposed to other areas where running
-user code in-kernel is needed. For example, Linux's seccomp uses BPF
-to apply security policies to system calls. For simplicity, this
-documentation refers only to packets, but other uses of BPF have their
-own data payloads.
-
-BPF programs run in a restricted virtual machine. It has almost no
-access to kernel functions, and while conditional branches are
-allowed, they can only jump forwards, to guarantee that there are no
-infinite loops.
-
-The virtual machine
-
-The BPF VM is an accumulator machine. Its main register, called
-register A, is an implicit source and destination in all arithmetic
-and logic operations. The machine also has 16 scratch registers for
-temporary storage, and an indirection register (register X) for
-indirect memory access. All registers are 32 bits wide.
-
-Each run of a BPF program is given one packet, which is placed in the
-VM's read-only "main memory". LoadAbsolute and LoadIndirect
-instructions can fetch up to 32 bits at a time into register A for
-examination.
-
-The goal of a BPF program is to produce and return a verdict (uint32),
-which tells the kernel what to do with the packet. In the context of
-packet filtering, the returned value is the number of bytes of the
-packet to forward to userspace, or 0 to ignore the packet. Other
-contexts like seccomp define their own return values.
-
-In order to simplify programs, attempts to read past the end of the
-packet terminate the program execution with a verdict of 0 (ignore
-packet). This means that the vast majority of BPF programs don't need
-to do any explicit bounds checking.
-
-In addition to the bytes of the packet, some BPF programs have access
-to extensions, which are essentially calls to kernel utility
-functions. Currently, the only extensions supported by this package
-are the Linux packet filter extensions.
-
-Examples
-
-This packet filter selects all ARP packets.
-
- bpf.Assemble([]bpf.Instruction{
- // Load "EtherType" field from the ethernet header.
- bpf.LoadAbsolute{Off: 12, Size: 2},
- // Skip over the next instruction if EtherType is not ARP.
- bpf.JumpIf{Cond: bpf.JumpNotEqual, Val: 0x0806, SkipTrue: 1},
- // Verdict is "send up to 4k of the packet to userspace."
- bpf.RetConstant{Val: 4096},
- // Verdict is "ignore packet."
- bpf.RetConstant{Val: 0},
- })
-
-This packet filter captures a random 1% sample of traffic.
-
- bpf.Assemble([]bpf.Instruction{
- // Get a 32-bit random number from the Linux kernel.
- bpf.LoadExtension{Num: bpf.ExtRand},
- // 1% dice roll?
- bpf.JumpIf{Cond: bpf.JumpLessThan, Val: 2^32/100, SkipFalse: 1},
- // Capture.
- bpf.RetConstant{Val: 4096},
- // Ignore.
- bpf.RetConstant{Val: 0},
- })
-
-*/
-package bpf // import "golang.org/x/net/bpf"
diff --git a/vendor/golang.org/x/net/bpf/instructions.go b/vendor/golang.org/x/net/bpf/instructions.go
deleted file mode 100644
index 3cffcaa..0000000
--- a/vendor/golang.org/x/net/bpf/instructions.go
+++ /dev/null
@@ -1,726 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package bpf
-
-import "fmt"
-
-// An Instruction is one instruction executed by the BPF virtual
-// machine.
-type Instruction interface {
- // Assemble assembles the Instruction into a RawInstruction.
- Assemble() (RawInstruction, error)
-}
-
-// A RawInstruction is a raw BPF virtual machine instruction.
-type RawInstruction struct {
- // Operation to execute.
- Op uint16
- // For conditional jump instructions, the number of instructions
- // to skip if the condition is true/false.
- Jt uint8
- Jf uint8
- // Constant parameter. The meaning depends on the Op.
- K uint32
-}
-
-// Assemble implements the Instruction Assemble method.
-func (ri RawInstruction) Assemble() (RawInstruction, error) { return ri, nil }
-
-// Disassemble parses ri into an Instruction and returns it. If ri is
-// not recognized by this package, ri itself is returned.
-func (ri RawInstruction) Disassemble() Instruction {
- switch ri.Op & opMaskCls {
- case opClsLoadA, opClsLoadX:
- reg := Register(ri.Op & opMaskLoadDest)
- sz := 0
- switch ri.Op & opMaskLoadWidth {
- case opLoadWidth4:
- sz = 4
- case opLoadWidth2:
- sz = 2
- case opLoadWidth1:
- sz = 1
- default:
- return ri
- }
- switch ri.Op & opMaskLoadMode {
- case opAddrModeImmediate:
- if sz != 4 {
- return ri
- }
- return LoadConstant{Dst: reg, Val: ri.K}
- case opAddrModeScratch:
- if sz != 4 || ri.K > 15 {
- return ri
- }
- return LoadScratch{Dst: reg, N: int(ri.K)}
- case opAddrModeAbsolute:
- if ri.K > extOffset+0xffffffff {
- return LoadExtension{Num: Extension(-extOffset + ri.K)}
- }
- return LoadAbsolute{Size: sz, Off: ri.K}
- case opAddrModeIndirect:
- return LoadIndirect{Size: sz, Off: ri.K}
- case opAddrModePacketLen:
- if sz != 4 {
- return ri
- }
- return LoadExtension{Num: ExtLen}
- case opAddrModeMemShift:
- return LoadMemShift{Off: ri.K}
- default:
- return ri
- }
-
- case opClsStoreA:
- if ri.Op != opClsStoreA || ri.K > 15 {
- return ri
- }
- return StoreScratch{Src: RegA, N: int(ri.K)}
-
- case opClsStoreX:
- if ri.Op != opClsStoreX || ri.K > 15 {
- return ri
- }
- return StoreScratch{Src: RegX, N: int(ri.K)}
-
- case opClsALU:
- switch op := ALUOp(ri.Op & opMaskOperator); op {
- case ALUOpAdd, ALUOpSub, ALUOpMul, ALUOpDiv, ALUOpOr, ALUOpAnd, ALUOpShiftLeft, ALUOpShiftRight, ALUOpMod, ALUOpXor:
- switch operand := opOperand(ri.Op & opMaskOperand); operand {
- case opOperandX:
- return ALUOpX{Op: op}
- case opOperandConstant:
- return ALUOpConstant{Op: op, Val: ri.K}
- default:
- return ri
- }
- case aluOpNeg:
- return NegateA{}
- default:
- return ri
- }
-
- case opClsJump:
- switch op := jumpOp(ri.Op & opMaskOperator); op {
- case opJumpAlways:
- return Jump{Skip: ri.K}
- case opJumpEqual, opJumpGT, opJumpGE, opJumpSet:
- cond, skipTrue, skipFalse := jumpOpToTest(op, ri.Jt, ri.Jf)
- switch operand := opOperand(ri.Op & opMaskOperand); operand {
- case opOperandX:
- return JumpIfX{Cond: cond, SkipTrue: skipTrue, SkipFalse: skipFalse}
- case opOperandConstant:
- return JumpIf{Cond: cond, Val: ri.K, SkipTrue: skipTrue, SkipFalse: skipFalse}
- default:
- return ri
- }
- default:
- return ri
- }
-
- case opClsReturn:
- switch ri.Op {
- case opClsReturn | opRetSrcA:
- return RetA{}
- case opClsReturn | opRetSrcConstant:
- return RetConstant{Val: ri.K}
- default:
- return ri
- }
-
- case opClsMisc:
- switch ri.Op {
- case opClsMisc | opMiscTAX:
- return TAX{}
- case opClsMisc | opMiscTXA:
- return TXA{}
- default:
- return ri
- }
-
- default:
- panic("unreachable") // switch is exhaustive on the bit pattern
- }
-}
-
-func jumpOpToTest(op jumpOp, skipTrue uint8, skipFalse uint8) (JumpTest, uint8, uint8) {
- var test JumpTest
-
- // Decode "fake" jump conditions that don't appear in machine code
- // Ensures the Assemble -> Disassemble stage recreates the same instructions
- // See https://github.com/golang/go/issues/18470
- if skipTrue == 0 {
- switch op {
- case opJumpEqual:
- test = JumpNotEqual
- case opJumpGT:
- test = JumpLessOrEqual
- case opJumpGE:
- test = JumpLessThan
- case opJumpSet:
- test = JumpBitsNotSet
- }
-
- return test, skipFalse, 0
- }
-
- switch op {
- case opJumpEqual:
- test = JumpEqual
- case opJumpGT:
- test = JumpGreaterThan
- case opJumpGE:
- test = JumpGreaterOrEqual
- case opJumpSet:
- test = JumpBitsSet
- }
-
- return test, skipTrue, skipFalse
-}
-
-// LoadConstant loads Val into register Dst.
-type LoadConstant struct {
- Dst Register
- Val uint32
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadConstant) Assemble() (RawInstruction, error) {
- return assembleLoad(a.Dst, 4, opAddrModeImmediate, a.Val)
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadConstant) String() string {
- switch a.Dst {
- case RegA:
- return fmt.Sprintf("ld #%d", a.Val)
- case RegX:
- return fmt.Sprintf("ldx #%d", a.Val)
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// LoadScratch loads scratch[N] into register Dst.
-type LoadScratch struct {
- Dst Register
- N int // 0-15
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadScratch) Assemble() (RawInstruction, error) {
- if a.N < 0 || a.N > 15 {
- return RawInstruction{}, fmt.Errorf("invalid scratch slot %d", a.N)
- }
- return assembleLoad(a.Dst, 4, opAddrModeScratch, uint32(a.N))
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadScratch) String() string {
- switch a.Dst {
- case RegA:
- return fmt.Sprintf("ld M[%d]", a.N)
- case RegX:
- return fmt.Sprintf("ldx M[%d]", a.N)
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// LoadAbsolute loads packet[Off:Off+Size] as an integer value into
-// register A.
-type LoadAbsolute struct {
- Off uint32
- Size int // 1, 2 or 4
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadAbsolute) Assemble() (RawInstruction, error) {
- return assembleLoad(RegA, a.Size, opAddrModeAbsolute, a.Off)
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadAbsolute) String() string {
- switch a.Size {
- case 1: // byte
- return fmt.Sprintf("ldb [%d]", a.Off)
- case 2: // half word
- return fmt.Sprintf("ldh [%d]", a.Off)
- case 4: // word
- if a.Off > extOffset+0xffffffff {
- return LoadExtension{Num: Extension(a.Off + 0x1000)}.String()
- }
- return fmt.Sprintf("ld [%d]", a.Off)
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// LoadIndirect loads packet[X+Off:X+Off+Size] as an integer value
-// into register A.
-type LoadIndirect struct {
- Off uint32
- Size int // 1, 2 or 4
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadIndirect) Assemble() (RawInstruction, error) {
- return assembleLoad(RegA, a.Size, opAddrModeIndirect, a.Off)
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadIndirect) String() string {
- switch a.Size {
- case 1: // byte
- return fmt.Sprintf("ldb [x + %d]", a.Off)
- case 2: // half word
- return fmt.Sprintf("ldh [x + %d]", a.Off)
- case 4: // word
- return fmt.Sprintf("ld [x + %d]", a.Off)
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// LoadMemShift multiplies the first 4 bits of the byte at packet[Off]
-// by 4 and stores the result in register X.
-//
-// This instruction is mainly useful to load into X the length of an
-// IPv4 packet header in a single instruction, rather than have to do
-// the arithmetic on the header's first byte by hand.
-type LoadMemShift struct {
- Off uint32
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadMemShift) Assemble() (RawInstruction, error) {
- return assembleLoad(RegX, 1, opAddrModeMemShift, a.Off)
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadMemShift) String() string {
- return fmt.Sprintf("ldx 4*([%d]&0xf)", a.Off)
-}
-
-// LoadExtension invokes a linux-specific extension and stores the
-// result in register A.
-type LoadExtension struct {
- Num Extension
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a LoadExtension) Assemble() (RawInstruction, error) {
- if a.Num == ExtLen {
- return assembleLoad(RegA, 4, opAddrModePacketLen, 0)
- }
- return assembleLoad(RegA, 4, opAddrModeAbsolute, uint32(extOffset+a.Num))
-}
-
-// String returns the instruction in assembler notation.
-func (a LoadExtension) String() string {
- switch a.Num {
- case ExtLen:
- return "ld #len"
- case ExtProto:
- return "ld #proto"
- case ExtType:
- return "ld #type"
- case ExtPayloadOffset:
- return "ld #poff"
- case ExtInterfaceIndex:
- return "ld #ifidx"
- case ExtNetlinkAttr:
- return "ld #nla"
- case ExtNetlinkAttrNested:
- return "ld #nlan"
- case ExtMark:
- return "ld #mark"
- case ExtQueue:
- return "ld #queue"
- case ExtLinkLayerType:
- return "ld #hatype"
- case ExtRXHash:
- return "ld #rxhash"
- case ExtCPUID:
- return "ld #cpu"
- case ExtVLANTag:
- return "ld #vlan_tci"
- case ExtVLANTagPresent:
- return "ld #vlan_avail"
- case ExtVLANProto:
- return "ld #vlan_tpid"
- case ExtRand:
- return "ld #rand"
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// StoreScratch stores register Src into scratch[N].
-type StoreScratch struct {
- Src Register
- N int // 0-15
-}
-
-// Assemble implements the Instruction Assemble method.
-func (a StoreScratch) Assemble() (RawInstruction, error) {
- if a.N < 0 || a.N > 15 {
- return RawInstruction{}, fmt.Errorf("invalid scratch slot %d", a.N)
- }
- var op uint16
- switch a.Src {
- case RegA:
- op = opClsStoreA
- case RegX:
- op = opClsStoreX
- default:
- return RawInstruction{}, fmt.Errorf("invalid source register %v", a.Src)
- }
-
- return RawInstruction{
- Op: op,
- K: uint32(a.N),
- }, nil
-}
-
-// String returns the instruction in assembler notation.
-func (a StoreScratch) String() string {
- switch a.Src {
- case RegA:
- return fmt.Sprintf("st M[%d]", a.N)
- case RegX:
- return fmt.Sprintf("stx M[%d]", a.N)
- default:
- return fmt.Sprintf("unknown instruction: %#v", a)
- }
-}
-
-// ALUOpConstant executes A = A