Verification
In TIG there are two types of verification, solution verification and method verification.
Solution verification checks that, for a given nonce, the benchmarker actually achieves the quality they reported. This is cheap to check since challenges in TIG are asymmetric. Solution verification acts as a sybil defence for the protocol.
Method verification ensures that benchmarkers are running the algorithms and configurations they claim — i.e. that their solutions were actually found using those algorithms and settings.
Method verification is partly decentralised in TIG.
The Mechanism
Here are the key points:
- Benchmarkers will now precommit to an AWS instance type from a whitelist (see How to do it for the whitelist)
- It is the responsibility of Benchmarkers to ensure that benchmarks made on their hardware are reproducible on that particular AWS instance type
- Benchmarkers will be penalised when using algorithms that are non-deterministic (i.e. algorithms that may produce different results on the same seed and hardware)
- During a round, and up until the end of the following round, all the benchmark data for that round will be publicly available
- In the period where the benchmark data is available, any player can pay a fee of 1 TIG to flag a specific nonce of a benchmark where they believe the solution quality is non-reproducible based on the Benchmarker’s precommit
- The Foundation will arbitrate reports by rerunning on the precommitted AWS instance type
- If a report is confirmed to have non-reproducible solution quality, the offending benchmarker will be penalised by having 10 TIG burnt from their round earnings, and the reporting player will be reimbursed 1 TIG by the Foundation
- Benchmarkers can request new instance types to add to the whitelist — get in touch with the team via our Discord.
- Safety restrictions have been put in place to protect against legitimate mistakes, e.g. controls on the number of nonces per benchmark that can be flagged.
See How to do it for details on how to report.