If we can reproduce the binary we downloaded from the public source code, with all bytes accounted for, we call the product reproducible. This does not mean we audited the code but it’s the precondition to make sure the public code has relevance for the provided binary.

If the provider puts your funds at risk on purpose or by accident, security researchers can see this if they care to look. It also means that inside the company, engineers can verify that the release manager is releasing the product based on code known to all engineers on the team. A scammer would have to work under the potential eyes of security researchers. He would have to take more effort in hiding any exploit.

“Reproducible” does not mean “verified”. There is good reason to believe that security researchers as of today would not detect very blatant backdoors in the public source code before it gets exploited, much less if the attacker takes moderate efforts to hide it. This is especially true for less popular projects.

Verdicts apply to very specific releases of products and never to the product as a whole. A new release of a product can change the product completely and thus also the verdict. This product remains listed according to its latest verdict but readers are advised to do their own research as this product might have changed for the better or worse.

This meta verdict is applied manually in cases of reviews that we identify as requiring an update.

This meta verdict applies to all products with verdict “reproducible” as soon as a new version is released until we test that new version, too. It also applies in cases where issues we opened are marked as resolved by the provider.

If we had more resources, we would update reviews more timely instead of assigning this meta verdict ;)